The "So What?" Test

A great post by Avinash Kaushik teaches us a simple test to use to identify if an analytics metric is right for your business.  It’s the “So What? Test” and it simply involves asking that question up to three times when you make an observation about a particular metric. The kind of metrics you’re looking for will measure how you’re doing with your business goals and lead to recommendations to move the dial.

Let me try and show this with a couple of my own made-up ones and one from the post.

Example:

Observation: The bounce rate on our homepage fell by 30% last quarter.

So what?

Visitors are believing more that they are in the right place because they are finding what they’re looking for.

So what?

We should implement this, this and this like we did on our homepage to some of the other top landing pages on our site.

Sweet, this works because our observation leads to action.  This metric is right for your business and can be used as a KPI (Key Performance Indicator) of how your business is doing in light of its goals.

Observation: We had 1,000 more visitors this month than last month.

So what?

More people on our site is a good thing.

So what?

More people means more conversions.

So what?

More conversions means more money.

While it may be good that you’re getting more traffic, it may not be.  First of all, this metric is not driving any action.  Second, it gives us no insight at all into why the observation happened.  Did it really lead to more money?  How do you know?  All of the extra visitors you got on your site could have all bounced cause they weren’t looking for what you were providing and now you think that what you are doing is working, but really it’s not.

Here’s my favorite one from Avinash’s post:

Observation: Our task completion rate is down five points this month to 58%.

So what?

Having indexed our performance against that of last quarter, each one percent drop causes a loss of $80,000 in revenue.

So what?

I have drilled down to the Primary Purpose Report and most of the fall is from visitors who were there to purchase on our website.  The most likely cause is the call to action on our landing pages and a reported slowness in response when people add to cart.

Cha-ching! This is where love in your boss’s/client’s heart for you lies.

A good metric will answer business questions and supply you with actionable ideas that recommend you at least experiment with doing more or less of something.