Attribution Model Comparison Tool in Google Analytics

 

# Is organic search advertising (SEO) undervalued or overvalued?

# If I invest in SEO, then how much incrementality does SEO can bring to my business bottomline?

# How can I make my PPC campaigns more effective?

# If I change my display adverising budget, how it will effect my website sales?

You can get answers to such questions by using the ‘Model Comparison Tool’ of Google Analytics.

This article is related to Attribution modelling in Google Analytics. If you are brand new to Attribution Modelling then I would suggest to read this article first: Beginners Guide to Google Analytics Attribution Modeling.

 

Introduction to Model Comparison Tool

As the name suggest, the model comparison tool (or Attribution Models Comparison Tool) is an attribution tool in Google Analytics, which is used to compare different attribution models to each other.

An attribution model is a set of rules which is used to determine how credit for conversions should be attributed to different marketing channels.

Through Model Comparison tool you can compare different baseline and custom attribution models to each other. This comparison is carried out to determine how a marketing channel can be valued from different perspective.

In order to access this tool, navigate to ‘Model Comparison Tool’ report  (under ‘Conversions’ > ‘Attribution) in your GA view:

model-comparison-tool

In order to use this tool, select at least one attribution model from the ‘select model’ drop down menu:

select-model

Note: You can compare up to 3 attribution models side by side through model comparison tool.

Attribution models in Google Analytics, can be broadly classified into two categories:

  1. Baseline Attribution Models
  2. Custom Attribution Models

To learn more about baseline and custom attribution models, read this article: Baseline and Custom Attribution Models in Google Analytics

 

Requirement for using the model comparison tool

In order to use the model comparison tool, you need to have ecommerce tracking and/or goal conversion tracking setup, in your GA reporting view. Otherwise you won’t see any data in the model comparison tool report.

What you will see instead, when you navigate to the ‘model comparison tool report in GA, is the following message:

requires-tracking

In fact,without goals and/or ecommerce tracking set up, you can not use any multi channel funnel and attribution report in GA.

 

The ‘spend’ column in Model Comparison Tool report

By default, the model comparison tool report does not show the ‘Spend’ column in its report, as shown below:

spend-column

In order to see the ‘Spend’ column in the ‘Model Comparison Tool’ report, you would need to import cost data into your Google Analytics property. There are two ways you can import cost data into GA:

#1 Link your Google Adwords account to your GA account and then run Adwords campaigns.

#2 Manually import cost data into GA via ‘Data import’ feature or via the management API.

Once the cost data is imported into your GA property, you should start seeing the ‘Spend’ column in the ‘Model Comparison tool’ report.

If you see dash sign (‘-’) in the ‘Spend’ column, then it means, no cost data is available for a given marketing channel in the selected time period:

spend-no-data

 

The ‘Conversions & CPA’ drop down menu in Model Comparison tool report

By default, the model comparison tool report does not show the ‘Conversions & CPA’ drop down menu in its report, as shown below:

drop-down-menu

In order to see this drop down menu, you would need to compare at least 2 attribution models to each other, via the model comparison tool:

comparison

Once you see this drop down menu, then you can select metrics combinations like:

  • Conversions & CPA
  • Conversion Value & ROAS
  • ‘Conversions & Value’

from the drop menu, as shown below:drop-down-menu2

 

The ‘CPA’ metric in model comparison tool report

The CPA (cost per acquisition) metric in the model comparison tool report, is calculated for each marketing channel and for each attribution model:

different-cpa

So you can have different ‘CPA’ for organic search under different attribution models. 

Thus depending upon the attribution model you select, you can have following different types of CPAs for each marketing channel:

  1. Last Interaction CPA
  2. Last non-direct click CPA
  3. Last Adwords click CPA
  4. First interaction CPA
  5. Linear CPA
  6. Time decay CPA
  7. Position based CPA

CPA for a particular channel under a particular attribution model is calculated as:

Channel spend / number of conversions attributed to the channel under a particular attribution model

For example, if Paid search ad spend is say $1500 in the last 30 days and its last interaction conversions (conversions attributed to paid search under last interaction attribution model) are say 500 then

Last Interaction CPA (for Paid Search) = $1500 / 500 = $3.00

Similarly, if Paid search ad spend is say $1500 in the last 30 days and its last non-direct click conversions (conversions attributed to paid search under last non-direct click attribution model) are say 300 then

Last Non-Direct click CPA (for Paid Search) = $1500 / 300 = $5.00

If you see dash sign (‘-’) in the ‘CPA’ column, then it means, no data is available for a given marketing channel in the selected time period:

cpa-no-data

 

The ‘ROAS’ metric in model comparison tool report

The ROAS (return on ad spend) metric in the model comparison tool report, is calculated for each marketing channel and for each attribution model:

different-roas

So you can have different ‘ROAS’ for organic search under different attribution models.

Thus, depending upon the attribution model you select, you can have following different types of ROAS for each marketing channel:

  1. Last Interaction ROAS
  2. Last non-direct click ROAS
  3. Last Adwords click ROAS
  4. First interaction ROAS
  5. Linear ROAS
  6. Time decay ROAS
  7. Position based ROAS

ROAS (return on ad spend) for a particular channel under a particular attribution model is calculated as:

(Channel Conversion value under a particular attribution model / Channel ad spend) * 100

For example, if Paid search ad spend is say $1500 in the last 30 days and its last interaction conversion value (conversion value under last interaction attribution model) is say $3000 then

Last Interaction ROAS (for Paid Search) = ($3000 / $1500) * 100 = 200%

Similarly, if Paid search ad spend is say $1500 in the last 30 days and its last non-direct click conversion value (conversion value under last non-direct click attribution model) is say $9000 then

Last Non-Direct click ROAS (for Paid Search) = ($9000 / $1500) * 100 = 600%

If you see dash sign (‘-’) in the ‘ROAS’ column, then it means, no data is available for a given marketing channel in the selected time period:

roas-no-data

 

% change column of the ‘Model Comparison Tool’

The % change column of the ‘Model Comparison Tool’ show the percentage change in ‘conversions’ or percentage change in ‘conversion value’ across attribution models:

percent-change

In order to display the % change column, you need to compare at least two attribution models to each other, via the model comparison tool.

The % change column of the ‘Model Comparison Tool’ does not show the percentage change in ‘CPA’ or percentage change in ‘ROAS’ across attribution models.

Google does not show the percentage change in ‘CPA’ across attribution models because ‘CPA’ is a calculated metric, so its % change between attribution models is going to be identical to that of % change in conversions.

Similarly, Google does not show the percentage change in ‘ROAS’ across attribution models because ‘ROAS’ is also a calculated metric, so its % change between attribution models is going to be identical to that of % change in conversion value.

The % change column of the ‘Model Comparison Tool’ shows % change in ‘conversions’ / ‘conversion value’ between comparison attribution model and reference attribution model:

reference-attribution-model

Under the ‘% change’ column, you can see different symbols next to percentages. Different symbols have got different meaning.

For example, if you see a gray dot next to % change, it means Google Analytics did not detect any identifiable % change between comparison and reference attribution model:gray-dot

If you see an ‘upward arrow’ next to % change, it means Google Analytics detected a % change which is in favour of the comparison attribution model but not in the favour of the reference model:

favor-comparison-model

Here, we can conclude that, if we use last interaction model, to distribute credit for conversion to a marketing channel, than the marketing channel deserves 6.44% more credit for conversion in comparison to linear attribution model.

It also means that, the marketing channel is undervalued by 6.44% under linear attribution model, when this model is compared with last interaction model.

This is the kind of insight you can get from ‘% change in conversions’.

If the positive % change in favour of the comparison attribution model is 10% or more, than the upward arrow next to the % change get the green color:

change-more-than-10

If you see an ‘downward arrow’ next to % change, it means Google Analytics detected a % change which is not in favour of the comparison attribution model but is in favour of the reference model:

negative-change

Here, we can conclude that, if we use Time Decay model, to distribute credit for conversion to a marketing channel, than the marketing channel deserves 1.88% less credit for conversion in comparison to linear attribution model.

It also means that, the marketing channel is overvalued by 1.88% under linear model, when this model is compared with time decay model.

This is the kind of insight you can get from ‘% change in conversions’.

If the negative % change which is not in favour of the comparison attribution model is 10% or more, than the downward arrow next to the % change get the red color:

negative-change-more-10

 

Practical use of ‘Model Comparison Tool’

Let us use ‘Model Comparison Tool’ to determine how organic search can be valued from different perspective.

Follow the steps below:

Step-1: Navigate to ‘Model Comparison Tool’  (under ‘Conversions’ > ‘Attribution) in your GA view:

model-comparison-tool

Step-2: Set date range to the last three months or more.

Step-3: Compare the ‘last interaction’ model with ‘last non-direct click’ and ‘time decay’ models:

model comparison tool2

Now you may want to know, why I selected these 3 particular attribution models for analysis.

I selected last interaction model because this is the default model used in multi-channel funnel reports in Google Analytics.

I selected last non-direct click model because this is the default model used in non-multi-channel funnel reports in GA.

I selected ‘time decay’ model because it is less crappy than other attribution models (First interaction, Linear, Last Adwords click and position based models).

Step-4: Select ‘Conversions and Value’ from the drop down menu, located in the middle of your ‘model comparison tool’ report:

conversion and value

Step-5: Now look at the column named ‘% change in conversion (from last interaction)’ for ‘organic search’:percentage change in conversions

Form this report we can conclude that the % of change in conversion for organic search from last interaction model to ‘last non-direct click’ attribution model is 21.61%.

What that means, if you use ‘last non-direct click’ attribution model (instead of last interaction model) to distribute credit for conversions to organic search, then the ‘organic search’ deserve 21.61% more credit for conversions.

In other words, ‘organic search’ is undervalued by 21.61% under last click attribution model (when this model is compared with ‘last non-direct click’ attribution model).

The upward green arrow next to 21.61% indicates positive change in conversions from last interaction model.

 

Similarly,

The % of change in conversion for organic search from last interaction model to ‘time decay’ attribution model is 8.60%.

What that means, if you use ‘time decay’ attribution model (instead of last interaction model) to distribute credit for conversions to organic search then the ‘organic search’ deserve 8.60% more credit for conversions.

In other words, ‘organic search’ is undervalued by 8.60% under last click attribution model (when this model is compared with ‘time decay’ attribution model).

The upward grey arrow next to 8.60% indicates positive change in conversions from last interaction model.

Now the million dollars question,

So what insight we have got from this analysis?

The insight is that, overall organic search is undervalued by (21.61 + 8.60)/2 = 15.10%

You can now show this report to your client/boss and demand more budget for organic search campaign.

Note: Don’t take the value of 15.10% too seriously.

Conversely, if overall organic search turned out to be overvalued by …%, you know that, your money would be better spend in investing in other marketing channels or finding a new SEO service provider.

Similarly, through model comparison tool you can valuate other marketing channels like Paid Search, Email, Display, Social media etc.

 

Using Data Driven Attribution Model to valuate organic search channel

If you have the option to use ‘data driven attribution model’, then use it, instead of the ‘time decay’ model:

model comparison tool3

I selected ‘data driven’ attribution model as ‘baseline’ model over ‘Time decay’ model for two main reasons:

#1 Data driven attribution model can analyse the data not only from my GA account but also from all those Google Accounts (like Doubleclick Campaign Manager, Google Adwords etc) which are linked to my GA account.

#2 Data driven attribution model assign credit for conversions algorithmically, which I trust much more, than manual conversion credits and/or the credits assigned via time decay model.

Now look at the column named ‘% change in conversion (from last interaction)’ for ‘organic search’:

change in conversion

Form this report we can conclude that the % of change in conversion for organic search from last interaction model to ‘data driven’ attribution model is 22.66%.

What that means, if you use ‘data driven attribution model (instead of last interaction model) to distribute credit for conversions to organic search, then the ‘organic search’ deserve 22.66% more credit for conversions.

In other words, ‘organic search’ is undervalued by 22.66% under last click attribution model (when this model is compared with ‘data driven’ attribution model).

So we can now conclude with confidence, that in this particular case, organic search is undervalued and is undervalued by (21.61 + 22.66)/2 = 22.135%

 

Other article you will find informative: How to use Agile Analytics to quickly solve your Conversion problems 

Quick Announcement about my new books

maths and stats bottom banner email analytics bottom banner attribution modelling bottom banner

Book #1: Maths and Stats for Web Analytics and Conversion Optimization - This expert guide will teach you, how to leverage the knowledge of maths and statistics, in order to accurately interpret data and take actions, which can quickly improve the bottom-line of your online business.

Book #2: Master the Essentials of Email Marketing Analytics - This book focuses solely on the ‘analytics’ that power your email marketing optimization program and help you in dramatically reducing your cost per acquisition and increasing marketing ROI, by tracking the performance of the various KPIs and metrics used for email marketing.

Book #3: Attribution Modelling in Google Analytics and Beyond - Attribution modelling is the process of determining the most effective marketing channels for investment. This book has been written to help you, in implementing attribution modelling. It will teach you, how to leverage the knowledge of attribution modelling, in allocating marketing budget and understanding buying behaviour.


 

Himanshu Sharma

Certified web analyst and founder of OptimizeSmart.com

My name is Himanshu Sharma and I help businesses in finding and fixing their Google Analytics and conversion issues.
  • More than ten years' experience in SEO, PPC and web analytics
  • Certified web analyst (master level) from MarketMotive.com
  • Google Analytics certified
  • Google AdWords certified
  • Nominated for Digital Analytics Association Award for Excellence
  • Bachelors degree in Internet Science
  • Founder of OptimizeSmart.com and EventEducation.com
I am also the author of the book Maths and Stats for Web Analytics and Conversion Optimization If you have any questions or comments please contact me