First, we took a look at types of ads that you can test and then we look at how to analyze the results of your tests and set up new ones. But, one more thing we must know is when to consider a test complete and ready to be analyzed. If you consider a test complete before you have statistically significant results that prove with great confidence that what you observe is actually true, you may find yourself making conclusions that simply are not. Therefore, you need to know when you have enough data for this to be the case.
You’ve got two options –
1. Use the chi-square or analysis of variance methods in Excel. You will have to do some studying to figure this one out, but it’s well worth the time.
2. Follow some basic rules for how much data you need before you can draw conclusions.
You’ll have to dive into #1 on your own. Let’s discuss #2 a little bit. Here are two major guidelines to follow when deciding when to end a test –
- Capture data over the entire span of your buying funnel – What good does it do to get data for the first half of a buying cycle? You don’t want to capture just your “one night stand” users; those that buy on the first click to your site. You want to make sure you collect data long enough to capture a good sample from every phase of the buying funnel from awareness to purchase. Run the test for like 3 sales cycles to get a good sampling.
- Have enough traffic that one click or conversion will not significantly change your results – 500 clicks and 10-20 conversions are good benchmarks here. If your account doesn’t get lots of clicks, you can spread out your ads across ad groups and analyze ad performance across multiple ad groups and keywords. You might write 4 different ads and run them on all of the ad groups in a particular campaign while just changing the headline to contain your ad group’s theme keyword in it. Doing this allows you to still compare different ad messages against each other to find out what’s important to your target audience.
The situation you don’t want to run into here is where the next click or conversion changes the entire scope of your test. For example, if you had 300 impressions and got 10 clicks for a 3% CTR and someone clicked on the next impression that would take you up to 3.7%. That’s too big of a jump. Similarly, if you had 2 conversions in those 10 clicks for a 20% conversion rate and that next click didn’t convert, that takes you down to 18%. Also too big of a jump. So, this is actually a great test to decide if you have enough data. Calculate how getting a click on your next impression and a conversion on your next click affects your numbers. If there isn’t a significant change, your test is over.
To summarize, if you let your ads run through all of the possible conditions that they could (weekdays, weekends, all hours of the day, all stages of the buying funnel, etc.) and you have enough data, then you are ready to analyze it.