Last Updated: November 11, 2021

Follow the steps below to pull custom data from Google Analytics to BigQuery:

Step-1: Navigate to your BigQuery account: https://console.cloud.google.com/bigquery

Step-2: At the top of your screen click on the drop-down menu:

Step-3: Click on the ‘New Project’ button:

We are going to use this new project for collecting and querying Google Analytics data in BigQuery.

Step-4: Name your new project and then click on the ‘Create’ button:

Step-5: Click on the ‘Select project’ link on the top right-hand side of your screen:

You are now in the ‘Google Analytics’ project.

You will know that you are in the right project because the project name will appear at the top of your screen:

Step-6: Click on the Google Analytics project ID:

Step-7: Click on ‘CREATE DATASET’ button:

Step-8: Name your data set (say ‘GA_data_set’) and then click on the ‘Create Dataset’ button:

Before you create a custom schema, you will need to first figure out the overall layout and format of what your data table should look like in the BigQuery.

It’s like creating a wireframe before you actually start designing a website. 

You can create this wireframe either in your head or on a piece of paper. You can also use an existing Google Analytics report as a wireframe.

Let’s extract Google Analytics data in the following format in BigQuery:

Here the screenshot of the GA report acts as a wireframe for us.

From the GA screenshot above we can determine the following things about, how our data table should look like in BigQuery:

1. Our data table should have one primary dimension called ‘Country’.
2. Our data table should have the following three metrics: ‘Sessions’, “%New Sessions’ and ‘New users’.
3. The data table needs to have 10 rows
4. The data in the data table should be sorted by ‘sessions’ in descending order.
5. The data in the data table should be from the last month (not shown in the GA screenshot above).

Based on your wireframe, create a custom schema via a third-party solution (connector). We use the same connector to extract Google Analytics data into BigQuery.

You can either search for such third-party solutions online or use the one that I use which is ‘Supermetrics for BigQuery‘. They provide a free 14 days trial of their solution. No credit card required.

Step-9: Navigate to ‘Supermetrics for BigQuery‘.

Step-10: Find and click on the ‘Start a free trial’ button:

Step-11: Fill out the form and then click on the ‘Send’ button:

You should now be redirected to the page like the one below in your Google Cloud platform account:

Step-12: Navigate to https://team.supermetrics.com/ and then click on ‘Sign in with Google’ button:

Step-13: Click on the name of your Google Account:

Step-14: Click on the ‘Integrations’ tab:

Step-15: Click on the ‘Query Manager’ link:

We use the ‘Query Manager’ tool to create a custom schema for our data table.

Step-16: Click on the ‘OK’ button:

Step-17: Type ‘Google Analytics’ in the search box:

Step-18: Click on the ‘Google Analytics’ data source:

When you click on the ‘Google Analytics’ data source, a new tab would open in your browser window.

Step-19: Click on the ‘Start’ button:

Step-20: Click on the Google account which is linked to your Google Analytics account:

Step-21: Click on the ‘Allow’ button:

You should now see a message like the one below:

Step-22: Navigate back to query manager https://team.supermetrics.com/query-manager

You should now see your new data source connected to the Query Manager:

Note: The Supermetrics Query Manager works just like Supermetrics for Google Sheets:

Step-23: Click on the ‘SCHEMAS’ tab:

Step-24: Make sure that the ‘DWH’ format is selected:

Step-25: Click on the ‘+New schema’ link:

Step-26: Type a name for your custom schema and then click on the ‘OK’ button:

You should now see your schema listed:

Step-27: Click on the ‘QUERY’ tab:

Step-28: Set up the various configuration settings of the query manager like the one below:

Reporting View: < select your reporting view >

Date: <select your date range>

Select Metrics: ‘Sessions’, ‘New Sessions’ and ‘New users’

Split by (aka dimensions): ‘Country’, ‘Date’

Note: Make sure that you always select the “Date” dimension. Otherwise, you will not be able to query data for a particular date in BigQuery.

Number of rows to fetch: 10

Sort rows: Sessions

Sort direction: Descending

Secondary Sort: Automatic

Options: Try to avoid Google’s data sampling

The entire Query Manager set up may look like the one below:

Step-29: Validate your query by clicking on the ‘Run’ button:

You should now see a screen like the one below with status 200 OK:

Step-30: Click on the ‘Save as’ button to save this query in your new custom schema:

Step-31: Give your query a name (say ‘Top Countries by Sessions’):

Step-32: Select your custom schema from the drop-down menu and then click on the ‘OK’ button:

Congratulations. 

You have now successfully created your custom schema:

Note: Each query in your custom schema corresponds to one data table in your BigQuery project. So if you create multiple queries in your custom schema (which is possible) then multiple data tables would automatically be created in your BigQuery project when you use the custom schema while creating a data transfer. 

Step-33: Navigate back to your BigQuery account: https://console.cloud.google.com/bigquery

Step-34: Make sure that you are in the ‘Google Analytics’ project:

Step-35: Click on link ‘Data transfers’ from the left-hand side navigation:

Step-36: Click on the ‘Enable’ button to enable the ‘BigQuery Data Transfer API’:

Step-37: Click on the ‘CREATE A TRANSFER’ link:

Step-38: Click on the ‘Explore data sources’ button:

Step-39: Type ‘supermetrics google analytics’ in the search box:

Step-40: Click on the button ‘Google Analytics by Supermetrics’:

You should now see the ‘Data Source enrolled‘ status:

Step-41: Click on the ‘Configure transfer’ button:

You should now see a screen like the one below which is about creating a data transfer:

Step-42: Type a meaningful name for the ‘Transfer config name’ field:

Step-43: Keep the scheduling options (which specify when the transfer will run) to ‘Start now’ (unless you want to change it) and then move on to the next step:

Step-44: Keep the ‘Repeats’ setting to ‘Daily’ to have new data added once a day and move on to the next step:

Note: The ‘Start date and run time‘ setting is locked for editing when ‘Start now‘ setting is selected.

Step-45: Select the data set you created earlier from the drop-down menu:

We are going to use this data set for storing Custom Google Analytics data in BigQuery.

Step-46: Click on the ‘Connect source’ button:

Step-47: Click on the ‘Accept Agreement’ button:

Step-48: Click on ‘Sign in with Google’ button:

Step-49: Click on the name of the Google account which is associated with both your Google Analytics account and BigQuery project:

Step-50: Click on the ‘Authorize with Google Analytics’ button:

Step-51: Click on the name of the Google account which is associated with both your Google Analytics account and BigQuery project:

Step-52: Click on the ‘Allow’ button:

Step-53: Click on the ‘Continue’ button:

You should now see a screen like the one below:

Step-54: Select ‘My GA Schema’ from the ‘Select Schema’ drop-down menu:

Step-55: Select the Google Analytics view (from which you want to send data to BigQuery) from the ‘Accounts’ drop-down menu and then click on the ‘Submit’ button:

You should now see the ‘Source Connected‘ message just below ‘Third party connection‘:

Step-56: Click on the ‘Save’ button to save the transfer and also start the initial data transfer:

You should now see a screen like the one below:

Step-57: Click on the ‘Schedule backfill’ button to backfill the entire last month of Google Analytics data into your BigQuery table:

Step-58: Select your start and end date and then click on the ‘OK’ button:

Step-59: Wait for the data transfer to complete:

Step-60: Click on the ‘SQL workspace’ link from the left-hand side navigation:

Step-61: Navigate to the data set named ‘GA_data_set’ (which we created earlier).

You should now see the data table(s) automatically created by Supermetrics:

Since our custom schema contains only one query, therefore only one data table is automatically created. 

Had our custom schema got multiple queries (say four queries) then four data tables would have been automatically created.

Step-62: Click on the data table (GA_TOP_COUNTRIES_BY_SESSIONS_) to see what data it contains:

At this point, you can query a particular set of data by clicking on the ‘Query Table’ button:

That’s how you can pull custom data from Google Analytics to BigQuery.

Other articles on Google Analytics BigQuery

1. Advantages of using Google BigQuery for Google Analytics
2. Cost of using BigQuery for Google Analytics
3. Guide to BigQuery Cost optimization
4. What is Google BigQuery Sandbox and how to use it
5. Understanding the BigQuery User Interface
6. Sending data from Google Analytics to BigQuery without 360
7.  How to connect GA4 (Google Analytics 4) with BigQuery
8. events_& events_intraday_ tables in BigQuery for GA4 (Google Analytics 4)
9. Using Google Cloud pricing calculator for BigQuery
10. How to access BigQuery Public Data Sets
11. How to use Google Analytics sample dataset for BigQuery
12. Connect and transfer data from Google Sheets to BigQuery
13. How to query Google Analytics data in BigQuery
14. How to send data from Google Ads to BigQuery
15. What is BigQuery Data Transfer Service & how it works.
16. How to send data from Facebook ads to BigQuery
17. How to send data from Google Search Console to BigQuery
18. Best Supermetrics Alternative – Dataddo
19. Google Analytics BigQuery Tutorial
20. How to backfill Google Analytics data in BigQuery
21. How to connect and export data from GA4 to BigQuery