Chad Summerhill wrote a nice post recently about something you should be doing if you manage your own PPC account and something you should be showing your clients if you manage accounts for others – visual statistically significant ad test results by time period. Check out the “how-to” in that post.
I did this exercise and can see how important it is to look at your ad test segmented out by time period (day or week, whatever is most appropriate). The main reason for this is that if you simply report the final results of your test, you won’t tell the whole story. Most of the time, we look at the results of tests like this…
Why is this imperfect? Well, we know cumulatively that the challenger performed better, but does that mean we should delete the champion? Maybe, maybe not. You don’t really know until you look at the data in time. There could be a scenario where the champion really struggled out of the gate in this test, but has actually been beating the challenger in the last week. If so, how would you know? Again, by taking a look at the results in time to view the trend. With this particular ad test, here’s how it turned out…
This way, you can trend out how the winner actually became the winner. In this test, we can see that the challenger was consistently better over time, which now gives us supreme confidence when we hit the delete button for the champion.
One other thing. See those trend lines in the graph? Not only does it show you which ad was better (by the gap between the two lines), but it also shows you the overall trend, meaning you can see if this metric is getting better or worse for you. In this example, the ad group is performing better over time regardless of what ad is running. This shows that whatever optimization efforts have been performed are working.