What matters the most to a business: Conversion Volume or Conversion Rate?

There are three camps involved in this debate:

First camp – in the favor of giving more weight to the conversion volumes. I am from this camp.

Second camp – in the favor of giving more weight to the conversion rate.

Third camp – in the favor of giving equal weight to both conversion volume and conversion rate.

Let us first start with the second camp – I think people from this camp are those who have either not read the post on ‘conversion volume optimization’ or they are just averse to any change to traditional systems and beliefs.

If they are from the latter category, I put them in the ‘conservatives’ bucket.

There is nothing much you can do about changing the mindset of these people.

They will continue to talk about CRO and how great it is.

So let us leave them in their state of virtual bliss and move on.

Before we move further, at no point in this article I am saying to completely discard conversion rate metric.

But I am not in the favour of giving equal weight to both conversion volume and conversion rate either.

This is because:

#1 Conversion rate calculations are horribly prone to errors.

#2 Marketing decisions based on erroneous data can’t produce optimal results.

#3 Conversion volumes reflect ‘effect size’ (signal) much more accurately than conversion rate.

#4 Conversions volumes go well with regular people.

#5 Conversions volumes can be optimized.

#6 Conversion volumes are error free.

#1 Conversion rate calculations are horribly prone to errors

Conversion rate calculations are prone to errors (observational error, computation error, statistical errors, interpretation errors, reporting errors etc) and there will always be some sort of inaccuracy no matter how much you segment the data as it is a ratio metric.

This is clearly not the case with conversion volumes as it is a number metrics.

#2 Marketing decisions based on erroneous data can’t produce optimal results

How accurate your marketing decisions could be if they are based on the erroneous conversion rate data?

Consider the following scenario:

Do you think you should be investing more in campaign ‘B’ because its conversion rate is highest?

I would suggest, not.

The sample size in case of campaign ‘B’ (4 transactions out of 20 visits) is too small to be statistically significant.

Had campaign B got 1 transaction out of 1 visit, it conversion rate would be 100%.

Will that make its performance even better?

No.

Do you think you should now be investing in campaign ‘A’ because it has higher conversion rate?

Hold your horses right here.

Are you really sure that the difference between the conversion rates of campaign ‘A’ and Campaign ‘C’ is statistically significant.?

Let us assume that after conducting a statistical test we came to the conclusion that the difference in the conversion rates of the two campaigns can’t be proved to be statistically significant.

Under these circumstances we cannot draw the conclusion that campaign ‘C’ is not performing better.

So what we can do then?

Well we need to collect more data to compute statistical significance of the difference in the conversion rates of the two campaigns.

At this stage investing more money in campaign ‘A’ may not produce optimal results as you may think it will.

If you don’t understand what I meant, check out this article: Is your conversion rate statistically significant?.

Now let me ask you one more question:

How many times do you conduct a statistical test (like Z test) to calculate the ‘confidence’  that difference in the conversion rates of two or more campaigns is statistically significant before you declare one campaign as ‘winner’ and decide to invest more?

I can bet only handful of marketers/analyst go through this hassle.

Can you see yourself conducting such statistical tests every time you look at your conversion rate reports, day in, day out?

#3 Conversion volumes reflect ‘effect size’ (signal) much more accurately than conversion rate

This is one of the biggest reason of using conversion volumes while taking marketing decisions.

It is possible and quite common for a result to be statistically significant and trivial or statistically insignificant but still important.

From the table above we can conclude that the ecommerce conversion rate of Google CPC is higher than that of Google Organic.

Does that mean Google CPC campaigns are performing better than organic campaigns?

Before we jump into any conclusion and invest more into PPC, let us calculate the statistical significance of the difference in conversion rates of Google organic and PPC campaigns.

So according to my statistical test (Z-test), I have only 65% confidence that the difference in the conversion rates of Google organic and Google PPC is not by chance.

As confidence is less than 95% the difference is not statistically significant and we need to collect more data before drawing any conclusions.

To calculate the confidence in my analytics report I used the ‘z-test bookmarklet’ developed by ‘Michael Whitaker’.

Details on installing and using this bookmarket.

Even if the difference in the conversion rates of Google organic and Google PPC turned out to be statistically significant we should still be investing more in Google organic (in this particular case) as the effect size (here revenue) of Google organic is much larger than that of Google PPC.

Just because a result is statistically significant, it doesn’t always mean that it is practically meaningful.

That is why we should interpret both the statistical significance and effect size of our results.

This makes ‘conversion volumes’ such a powerful metric.

#4 Conversions volumes go well with regular people

You don’t need to be a geek to understand conversion volumes.

Conversions volumes reflect marketing efforts much better than conversion rate and are easy to communicate with people from all walks of life.

Nobody gives a crap what your conversion rate is, if the sales are going down.

Your site conversion rate may be increasing but sales may still be go down.

This is possible if traffic to your website is declining.

#5 Conversions volumes can be optimized

You can use conversion volume in any analytics framework you create for your business.

This is because you can set achievable targets for them.

You can’t set achievable numerical targets for conversion rates with any ease.

#6 Conversion volumes are error free

Conversion volumes are not prone to errors (observational error, computation error, statistical errors, interpretation errors, reporting errors etc), are independent of conversion rate and other factors like website traffic.

Remember conversion rate = conversion volume/total visits.

Without conversion volume there will be no conversion rate.

So whenever there is a trade off between ‘conversion volume’ and ‘conversion rate’,  I support ‘conversion volumes’ any time of the day.

So do i still recommend using ‘conversion rate’?

Yes.

But just remember what you are playing with.

This is not just any other metric.

It is a powerful ratio metric.

If you are not segmenting the data, the way it should be segmented, if you are not taking conversion volumes, statistical significance and ‘confidence’ into account then your ‘conversion rate’ focus strategy won’t produce optimal results and you may even incur huge losses.

There is one more thing.

Some people confuse CVO with a metric.

CVO is not a metric, its a process just like CRO.

But ‘conversion volume’ is a metric just like ‘conversion rate’ and conversions volumes not only include macro conversions like ‘orders’ but also micro conversions like avg. time spent on the site, newsletters signups etc.

In the end conversion volume is what matters the most to a business because it speaks the universal language (the language of money) loud and clear.