How to pull custom data from Google Analytics to BigQuery

Follow the steps below to pull custom data from Google Analytics to BigQuery:

Step-1: Navigate to your BigQuery account: https://console.cloud.google.com/bigquery


Step-2: At the top of your screen click on the drop-down menu:

xe 66FuehKkFLHmuVbAhdOV eI


Step-3: Click on the ‘New Project’ button:

create new project bigquery

We are going to use this new project for collecting and querying Google Analytics data in BigQuery.


Step-4: Name your new project and then click on the ‘Create’ button:

Name your new project and then click on the CREATE button


Step-5: Click on the ‘Select project’ link on the top right-hand side of your screen:

Click on the SELECT PROJECT link

You are now in the ‘Google Analytics’ project.

You will know that you are in the right project because the project name will appear at the top of your screen:

google analytics project bigquery


Step-6: Click on the Google Analytics project ID:

Click on the Google Analytics project ID


Step-7: Click on ‘CREATE DATASET’ button:

create dataset bigquery 1


Step-8: Name your data set (say ‘GA_data_set’) and then click on the ‘Create Dataset’ button:

Name your dataset


Before you create a custom schema, you will need to first figure out the overall layout and format of what your data table should look like in the BigQuery.

It’s like creating a wireframe before you actually start designing a website. 

You can create this wireframe either in your head or on a piece of paper. You can also use an existing Google Analytics report as a wireframe.

Let’s extract Google Analytics data in the following format in BigQuery:

Google Analytics data in the following format in BigQuery


Here the screenshot of the GA report acts as a wireframe for us.

From the GA screenshot above we can determine the following things about, how our data table should look like in BigQuery:

  1. Our data table should have one primary dimension called ‘Country’.
  2. Our data table should have the following three metrics: ‘Sessions’, “%New Sessions’ and ‘New users’.
  3. The data table needs to have 10 rows
  4. The data in the data table should be sorted by ‘sessions’ in descending order.
  5. The data in the data table should be from the last month (not shown in the GA screenshot above).

Based on your wireframe, create a custom schema via a third-party solution (connector). We use the same connector to extract Google Analytics data into BigQuery.

You can either search for such third-party solutions online or use the one that I use which is ‘Supermetrics for BigQuery‘. They provide a free 14 days trial of their solution. No credit card required.

Step-9: Navigate to ‘Supermetrics for BigQuery‘.

Step-10: Find and click on the ‘Start a free trial’ button:

move your marketing data into bigquery


Step-11: Fill out the form and then click on the ‘Send’ button:

OOmS2xX3THCmsdXAatVwnOALLbssB ghG0AUpMi4CtWvIdpELtNE6IEqMw5iC7RFkv1h9UNx5yacYz3obUF09Pyfnqcpt1 tv5MQKqp488SOYMoI F3MD1s8So


You should now be redirected to the page like the one below in your Google Cloud platform account:

google cloud platform marketplace 1


Step-12: Navigate to https://team.supermetrics.com/ and then click on ‘Sign in with Google’ button:

hp3jD2ZoisgWC50hK Jyux3ORvL3qVjH 1r3bVPHZUYSaUm8a4jtFW LnWj6


Step-13: Click on the name of your Google Account:

chose an account to continue with supermetrics 2


Step-14: Click on the ‘Integrations’ tab:

INTEGRATIONS


Step-15: Click on the ‘Query Manager’ link:

Click on the Query Manager link


We use the ‘Query Manager’ tool to create a custom schema for our data table.

Step-16: Click on the ‘OK’ button:

oUPwAwlOPxgv8Y2hb8HBe6Gt ZdweNqYPxDp



Step-17: Type ‘Google Analytics’ in the search box:

8HTlwi7HW hrn2FbV6 lGKoVVaaRqAJiQts0lLwrGjFuRUfNg uO4o27IDMoHI1sIAI2Wtf9TDlu7hrZ Bcs4fbtWiHrs26GKk4CAt8X2DV1A VLLX7XOJlEamLUozuXP2fZWqCL


Step-18: Click on the ‘Google Analytics’ data source:

NdGeu6BUUdhn2cJ0kKnlEUSq3sd42ak48VsYGmteLxc hTvsfc6AKswBkPf96kUUdxcgYcVRxSEVjLDKVis4tNSznx 0lSw2MrzqEQHzA7zKZOcQRJEoarvQJ73oX 6LnP0YqKRD

When you click on the ‘Google Analytics’ data source, a new tab would open in your browser window.


Step-19: Click on the ‘Start’ button:

connect supermetrics to google analytics


Step-20: Click on the Google account which is linked to your Google Analytics account:

chose an account to continue with supermetrics 2 1


Step-21: Click on the ‘Allow’ button:

allow

You should now see a message like the one below:

connect to google analytics

Step-22: Navigate back to query manager https://team.supermetrics.com/query-manager

You should now see your new data source connected to the Query Manager:

Ct6nfXb5k6L EiW2NN3fQ6m3NN5WOfV3Fk4Ez9kjW9 JrxMawk8TERzgpUF6OXi s5cjzEjhYzE7JneRcyXGKC0TtiPg boIAFOAZ3O0K9cpFHQhv J2N3fjFsSWcUksRGFeOj5G

Note: The Supermetrics Query Manager works just like Supermetrics for Google Sheets:


Step-23: Click on the ‘SCHEMAS’ tab:

SCHEMAS


Step-24: Make sure that the ‘DWH’ format is selected:

DWH format


Step-25: Click on the ‘+New schema’ link:

New schema


Step-26: Type a name for your custom schema and then click on the ‘OK’ button:

new schema name


You should now see your schema listed:

c SryT5eg36IVU90uWngYIPnBbmTBxrhSB7F5 enT g6Z1kIqTgGID4EcjZQI9yWJSeoIQxwH 4 FMvM1ZyhTnGlpj6ERmzy0g3F9JqXX40Y0ckZiUi5hs4Oov 6 mJ RcYLNTj4


Step-27: Click on the ‘QUERY’ tab:

QUERY


Step-28: Set up the various configuration settings of the query manager like the one below:

Reporting View: < select your reporting view >

Reporting View

Date: <select your date range>

3qfVTICQ6qdeJGaS57ihlmF 4fWB JUSyeehGzijnTzbsDf0R1WRImAsuT1 Nl6cT3IY pMUJrrbVNnp62TARpyJo87iE2a A NjQE6JyEMGUgZ9bq46Ask2oaAQd 19K DUdcCN

Select Metrics: ‘Sessions’, ‘New Sessions’ and ‘New users’

UFKa6rs0EY4l6 maBhYllUVD6RUEfvqQzhrG53hgZ2qFX4H sm4gYyebrDfbkSBkAMqs7xAE6dpieiMKBkJmIMLGhqJDC KJDYZDAkYjZpHKhGjhr1swSyNfkOqI53fi lzMrtG

Split by (aka dimensions): ‘Country’, ‘Date’

Note: Make sure that you always select the “Date” dimension. Otherwise, you will not be able to query data for a particular date in BigQuery.

Number of rows to fetch: 10

Sort rows: Sessions

Sort direction: Descending

Secondary Sort: Automatic

n7kIwqVS6UorBC3X4wTXRsp


Options: Try to avoid Google’s data sampling

MYZk5kpuSQ7sIaa4trsJVHkxHltn5KGVGaC HNm12X4iSAKfqdLQg26ANwQuMTBwoKpjzyuMW HndiCpunLZstwMXDPiAlfoqAUYiljB9SxaIVVBSkXNx7 7gASbDemSlR2Ptpm2


The entire Query Manager set up may look like the one below:

entire Query Manager set up


Step-29: Validate your query by clicking on the ‘Run’ button:

XsOOYtHTd WDqqvftyc8doNldspvjbtzxsVPbLutJizdzJfJ7NJAO9k


You should now see a screen like the one below with status 200 OK:

4vlepvTxYND81xoqmQ2VQu0u7EXUd8BDvS9sY2GrbIZCNqxjACTfXXE dqu7lckW8qLDyjPR5dxbFfqss IdqoN8XDvmUYtG A T8K BdGgFpefdTj4BG jcLyt2B9V CDMRTB8L


Step-30: Click on the ‘Save as’ button to save this query in your new custom schema:

save this query in your new custom schema


Step-31: Give your query a name (say ‘Top Countries by Sessions’):

Give your query a name


Step-32: Select your custom schema from the drop-down menu and then click on the ‘OK’ button:

IfBX7EP884iOVELHvLi2s4XQsM8eLfprGxWK4 jRlh rjiiO 6kVm1R7rvRZGkk3Z2Ec7eeNhuamfqa5qWOw8jjz7 pYdijHZ9qCD2VT BViXY9NCHPH4 grScGtF0euJOQVN24l

Congratulations. 


You have now successfully created your custom schema:

GkUA5 CM3IbigAqYQ7XuWLggi0hzHaguSe1qtXaVOuNiP4snTTJ


Note: Each query in your custom schema corresponds to one data table in your BigQuery project. So if you create multiple queries in your custom schema (which is possible) then multiple data tables would automatically be created in your BigQuery project when you use the custom schema while creating a data transfer. 


Step-33: Navigate back to your BigQuery account: https://console.cloud.google.com/bigquery

Step-34: Make sure that you are in the ‘Google Analytics’ project:

google analytics project bigquery 1


Step-35: Click on link ‘Data transfers’ from the left-hand side navigation:

Data transfers bigquery


Step-36: Click on the ‘Enable’ button to enable the ‘BigQuery Data Transfer API’:

BigQuery Data Transfer API


Step-37: Click on the ‘CREATE A TRANSFER’ link:

CREATE A TRANSFER bigquery


Step-38: Click on the ‘Explore data sources’ button:

EXPLORE DATA SOURCES


Step-39: Type ‘supermetrics google analytics’ in the search box:

kkrOjAxsMgGbjIzE6ck6Gc uDdX72PtplflM4abOneYy2Pbb5zicY4 cSRManBMx2mOHqMm7V 2JAOAlmqKBSFqIbtqLK8 4A1S63pxkNDXo7HNMS


Step-40: Click on the button ‘Google Analytics by Supermetrics’:

Google Analytics by Supermetrics connector


You should now see the ‘Data Source enrolled‘ status:

Google Analytics by Supermetrics connector Data Source enrolled


Step-41: Click on the ‘Configure transfer’ button:

CONFIGURE TRANSFER 2


You should now see a screen like the one below which is about creating a data transfer:

creating a google anlaytics data transfer in bigquery


Step-42: Type a meaningful name for the ‘Transfer config name’ field:

TQ1tACC7OS49qxfZ3eyAKxmkQCiVMJWL CE5MLsi19TZ3r 9DqDu7AafZ8eyF8gQbjRLRWcCHZTzCoR1zTsi9FpPkkkcJZUTls7WwPoNorq5IIOIgW LNV4rsso2HpoEhYNCorJg


Step-43: Keep the scheduling options (which specify when the transfer will run) to ‘Start now’ (unless you want to change it) and then move on to the next step:

scheduling options start now bigquery


Step-44: Keep the ‘Repeats’ setting to ‘Daily’ to have new data added once a day and move on to the next step:

Repeats setting to Daily 1


Note: The ‘Start date and run time‘ setting is locked for editing when ‘Start now‘ setting is selected.

Step-45: Select the data set you created earlier from the drop-down menu:

Dvd80Txo xz1xtkN65 8pmuudEAqxwYLOm Z4eR7VT3baZTFCepGwutF jiFaxs
Tfw6pxZwb9NMRH5 h25lAqGrvgZozvRIJJMLjhWG4YLBwWd64ek54Z


We are going to use this data set for storing Custom Google Analytics data in BigQuery.

Step-46: Click on the ‘Connect source’ button:

CONNECT SOURCE third party connection


Step-47: Click on the ‘Accept Agreement’ button:

enable a third party data connection in bigquery 2


Step-48: Click on ‘Sign in with Google’ button:

RZiDes75R009LMOuRk7f5axHW7GVt fxdPy0Y4wIM9m4kSwEfGsd swioxNeI fmcrVa7nThxS4tCYvSoF8pzGd vdB3XgGVSJ7W PWUwVEB39uCxwu vGhDAVf n6qnCmpwhpB


Step-49: Click on the name of the Google account which is associated with both your Google Analytics account and BigQuery project:

Click on the name of the Google account 1


Step-50: Click on the ‘Authorize with Google Analytics’ button:

Authorize with Google Analytics 1


Step-51: Click on the name of the Google account which is associated with both your Google Analytics account and BigQuery project:

chose an account to continue with supermetrics 3


Step-52: Click on the ‘Allow’ button:

allow 1


Step-53: Click on the ‘Continue’ button:

Click on the Continue button


You should now see a screen like the one below:

ibnb46ZVXt


Step-54: Select ‘My GA Schema’ from the ‘Select Schema’ drop-down menu:

xs3F oNxEIftdxFiTYO M55f7iaZ2AKE2xHnTdT5xpQG mcLS94eM0EDEzbz8a17lLpgrRgbskn5RuZNsqSD uSEOlwo2A1KY55IlmE6HCwGlpmdTcVDKOWvIIGoB g vFC0Rc9v


Step-55: Select the Google Analytics view (from which you want to send data to BigQuery) from the ‘Accounts’ drop-down menu and then click on the ‘Submit’ button:

wPkqg1y DmEhzjY9qY24lueT 8c9VdVC wFkrui G8zZsKELLkmUrTcCQuOkAMTknhlyqwaz1zkZmFCk8vrTTlsKF 8Tqw4qv0OJaGfIy3RJEiUqOuIOnB85BDYKdbWOnxX FqL0


You should now see the ‘Source Connected‘ message just below ‘Third party connection‘:

AClBuSeVc9tThFhAbu9e33NBKMdL3DBi810Rn4sLX9I02jGARdTsQDIKC6p5jI8EBTKkX u29o694C7CGb9IoCuyfwCsECYw4l0e00GeSmgFMiW 8QhKj7T6Wd nH5fyXqM56EDu


Step-56: Click on the ‘Save’ button to save the transfer and also start the initial data transfer:

CMjilVeztB2CQXeITcDisY2TEubQqIz REnIZwL79PYx A9DeLv0 n0vB2WJT4LrLgKqKS mT8jJG9kA


You should now see a screen like the one below:

vVYMORLsj32efY uj55ReqPqbRm vnQvnl2iS9B0SEKEckJa7IKrzIXeoI5X


Step-57: Click on the ‘Schedule backfill’ button to backfill the entire last month of Google Analytics data into your BigQuery table:

SCHEDULE BACKFILL bigquery

Step-58: Select your start and end date and then click on the ‘OK’ button:

rQccQjK97lOzfqpJAo obczYdqHqezv7aSJUCL6Mj0uzMZOYawcdlIciL0cTm28PJsT fsfEo5XQCYVx5 OrUAjs6lQefMv5W01Gqq3R9ADWWSJgQ10c QWePceJC1tN619SooZ


Step-59: Wait for the data transfer to complete:

oCRH24VRr1FaD rk7dSXIAqMDnMfEfSeJG4YWKA f8MufNqC1ul2nE


Step-60: Click on the ‘SQL workspace’ link from the left-hand side navigation:

DrkGC9cKWVAniAEvx fI7n061xtC5rurA79cmevjF5RBuPUVZ0o W8sFLbpnJb RKa58jbM8JvebPJUc1hJ5 v73GfozQiejYcMLgNil4Q G5 ErdXZ3v3U2lHlSGBAhMzjgL0sg


Step-61: Navigate to the data set named ‘GA_data_set’ (which we created earlier).

You should now see the data table(s) automatically created by Supermetrics:

Navigate to the data set named GA data set bigquery


Since our custom schema contains only one query, therefore only one data table is automatically created. 

Had our custom schema got multiple queries (say four queries) then four data tables would have been automatically created.

Step-62: Click on the data table (GA_TOP_COUNTRIES_BY_SESSIONS_) to see what data it contains:

GA TOP COUNTRIES BY SESSIONS


At this point, you can query a particular set of data by clicking on the ‘Query Table’ button:

Query Table bigquery 1
Google Analytics to BigQuery

That’s how you can pull custom data from Google Analytics to BigQuery.

Other articles on Google Analytics BigQuery

  1. Advantages of using Google BigQuery for Google Analytics
  2. Cost of using BigQuery for Google Analytics
  3. Guide to BigQuery Cost optimization
  4. What is Google BigQuery Sandbox and how to use it
  5. Understanding the BigQuery User Interface
  6. Sending data from Google Analytics to BigQuery without 360
  7.  How to connect GA4 (Google Analytics 4) with BigQuery
  8. events_& events_intraday_ tables in BigQuery for GA4 (Google Analytics 4)
  9. Using Google Cloud pricing calculator for BigQuery
  10. How to access BigQuery Public Data Sets
  11. How to use Google Analytics sample dataset for BigQuery
  12. Connect and transfer data from Google Sheets to BigQuery
  13. How to query Google Analytics data in BigQuery
  14. How to send data from Google Ads to BigQuery
  15. What is BigQuery Data Transfer Service & how it works.
  16. How to send data from Facebook ads to BigQuery
  17. How to send data from Google Search Console to BigQuery
  18. Best Supermetrics Alternative – Dataddo
  19. Google Analytics BigQuery Tutorial
  20. How to backfill Google Analytics data in BigQuery
  21. How to connect and export data from GA4 to BigQuery

Register for the FREE TRAINING...

"How to use Digital Analytics to generate floods of new Sales and Customers without spending years figuring everything out on your own."



Here’s what we’re going to cover in this training…

#1 Why digital analytics is the key to online business success.

​#2 The number 1 reason why most marketers are not able to scale their advertising and maximize sales.

#3 Why Google and Facebook ads don’t work for most businesses & how to make them work.

#4 ​Why you won’t get any competitive advantage in the marketplace just by knowing Google Analytics.

#5 The number 1 reason why conversion optimization is not working for your business.

#6 How to advertise on any marketing platform for FREE with an unlimited budget.

​#7 How to learn and master digital analytics and conversion optimization in record time.



   

My best selling books on Digital Analytics and Conversion Optimization

Maths and Stats for Web Analytics and Conversion Optimization
This expert guide will teach you how to leverage the knowledge of maths and statistics in order to accurately interpret data and take actions, which can quickly improve the bottom-line of your online business.

Master the Essentials of Email Marketing Analytics
This book focuses solely on the ‘analytics’ that power your email marketing optimization program and will help you dramatically reduce your cost per acquisition and increase marketing ROI by tracking the performance of the various KPIs and metrics used for email marketing.

Attribution Modelling in Google Analytics and BeyondSECOND EDITION OUT NOW!
Attribution modelling is the process of determining the most effective marketing channels for investment. This book has been written to help you implement attribution modelling. It will teach you how to leverage the knowledge of attribution modelling in order to allocate marketing budget and understand buying behaviour.

Attribution Modelling in Google Ads and Facebook
This book has been written to help you implement attribution modelling in Google Ads (Google AdWords) and Facebook. It will teach you, how to leverage the knowledge of attribution modelling in order to understand the customer purchasing journey and determine the most effective marketing channels for investment.

About the Author

Himanshu Sharma

  • Founder, OptimizeSmart.com
  • Over 15 years of experience in digital analytics and marketing
  • Author of four best-selling books on digital analytics and conversion optimization
  • Nominated for Digital Analytics Association Awards for Excellence
  • Runs one of the most popular blogs in the world on digital analytics
  • Consultant to countless small and big businesses over the decade
error: Alert: Content is protected !!