How to create data transfer in BigQuery
The BigQuery data transfer service automates data transfer from popular data sources to BigQuery without the need to write any code.
Just to give you an example of how data transfers work in the Google Cloud console, let’s create a new data transfer that automatically sends Google Ads data to your BigQuery project on a regular basis.
Follow the steps below:
Step-1: Create a new BigQuery project where you are going to store Google ads data in BigQuery. If you already have a BigQuery project, use that one to create a new dataset for storing Google ads data.
Step-2: Create a new dataset in your existing project for storing Google Ads data. You can also use an existing dataset, but I prefer creating a new dataset to make data management easier.
Step-3: Navigate to https://console.cloud.google.com/bigquery
Step-4: Make sure that you are in the correct project:
Step-5: Click on the three dots menu next to your project ID:
Step-6: Click on ‘Create dataset’:
Step-7: Name your data set (e.g. ‘google_ads’) and then click on the ‘Create Dataset’ button:
You should now see the new dataset created under your project ID:
But this dataset does not contain any data table or any data. For that, you would first need to create a data transfer.
Step-8: Click on ‘Data Transfers’ from the left navigation menu:
Step-9: Click on the ‘CREATE A TRANSFER’ button:
Step-10: Enable the BigQuery data transfer API.
Note: You get the option to enable the BigQuery data transfer API only when you are creating your very first data transfer via the Google Cloud console.
Step-11: Click on the ‘Source’ drop-down menu:
Step-12: Click on ‘Google Ads’:
Step-13: Type ‘Google Ads to BigQuery’ under the field ‘Transfer config name’:
Step-14: Set your schedule options:
Step-15: Select the dataset you created earlier (from the drop-down menu) for storing Google Ads data. In our case, it is ‘google_ads’:
Step-16: Enter the customer ID (without any dashes) of the Google Ads account whose data you want to transfer to your BigQuery project.
Step-17: Click on the ‘Save’ button:
You should now see a screen like the one below, which shows the current status of your data transfer:
Step-18: Click on the browser refresh button to check the current status:
Now you can see under the ‘Summary’ section that the transfer status changed from ‘The transfer run is pending’ to ‘The transfer run is in progress’.
You may need to wait for a couple of minutes, depending on the volume of data you are transferring to your BigQuery project.
Step-19: Keep clicking on the browser refresh button after every 5 minutes until you see the transfer status of ‘The transfer run has completed successfully’.
Step-20: Click on the ‘SQL Workspace’ from the left navigation menu:
Step-21: Navigate to the ‘google_ads’ dataset you created earlier. Scroll down until you see the list of new data tables in the dataset:
Step-22: Click on one of the data tables and then on the ‘Preview’ tab to see the imported Google ads data:
Other articles on GA4 BigQuery
#1 BigQuery Introduction
- How to create a new Google Cloud Platform account.
- How to create a new BigQuery project.
- What is Google BigQuery Sandbox and how to use it.
- Understanding the BigQuery User Interface.
- What is BigQuery Data Transfer Service & how it works.
- How to create data transfer in BigQuery.
- Connect and transfer data from Google Sheets to BigQuery.
- How to access BigQuery Public Data Sets.
- Best Supermetrics Alternative – Dataddo.
#2 GA4 BigQuery Introduction
- Google Analytics 4 BigQuery Tutorial for Beginners to Advanced.
- GA4 Bigquery Export Schema Tutorial.
- GA4 BigQuery – Connect Google Analytics 4 with BigQuery.
- events_ & events_intraday_ tables in BigQuery for GA4 (Google Analytics 4).
- pseudonymous_users_ & users_ data tables in BigQuery for GA4 (Google Analytics 4).
- How to access GA4 Sample Data in BigQuery.
- Advantages of using Google BigQuery for Google Analytics 4.
- Impact of Google Advanced Consent Mode on BigQuery & GDPR.
#3 GA4 BigQuery Data Transfer
- How to Connect and Export Data from GA4 to BigQuery
- How to backfill GA4 data in BigQuery.
- How to overcome GA4 BigQuery Export limit.
- How to Send Custom GA4 Data to BigQuery.
- How to backup Universal Analytics data to BigQuery.
- How to send data from Google Ads to BigQuery.
- How to send data from Google Search Console to BigQuery.
- Sending data from Google Analytics to BigQuery without 360.
- How to send data from Facebook ads to BigQuery.
- How to pull custom data from Google Analytics to BigQuery.
#4 BigQuery Cost Optimization
- Guide to BigQuery Cost Optimization.
- Using Google Cloud pricing calculator for BigQuery.
- Cost of using BigQuery for Google Analytics 4.
#5 Query GA4 BigQuery Data
- How to query Google Analytics data in BigQuery.
- Query GA4 data in BigQuery without understanding SQL.
- Using GA4 BigQuery SQL generator to create SQL queries.
- New vs Returning users in GA4 BigQuery data table.
- GA4 BigQuery Composer Tutorial for ChatGPT.
- How to track GA4 BigQuery Schema Change.
- Calculating Sessions and Engaged Sessions in GA4 BigQuery.
- Calculating Total Users in GA4 BigQuery.
#6 GA4 to BigQuery Dimension/Metric Mapping.
- GA4 to BigQuery Mapping Tutorial.
- GA4 Attribution Dimensions to BigQuery Mapping.
- GA4 Google Ads Dimensions to BigQuery Mapping.
- GA4 Demographic Dimensions to BigQuery Mapping.
- GA4 Ecommerce Dimensions to BigQuery Mapping.
- GA4 Event-Scoped Ecommerce Metrics to BigQuery Mapping.
- GA4 Item-Scoped Ecommerce Metrics to BigQuery Mapping.
- GA4 Revenue Metrics to BigQuery Mapping.
- GA4 Event Dimensions to BigQuery Mapping.
- GA4 Event Metrics to BigQuery Mapping.
- GA4 Geography Dimensions to BigQuery Mapping.
- GA4 Link Dimensions to BigQuery Mapping.
- GA4 Page/Screen Dimensions to BigQuery Mapping.
- GA4 Page/Screen Metrics to BigQuery Mapping.
- GA4 Platform/Device Dimensions to BigQuery Mapping.
- GA4 User-Scoped Traffic Dimensions to BigQuery Mapping.
- GA4 Session-Scoped Traffic Dimensions to BigQuery Mapping.
- GA4 Session Metrics to BigQuery Mapping.
- GA4 User Dimensions to BigQuery Mapping.
- GA4 User Metrics to BigQuery Mapping.
- GA4 Advertising Metrics to BigQuery Mapping.
-
The BigQuery data transfer service automates data transfer from popular data sources to BigQuery without the need to write any code.
Just to give you an example of how data transfers work in the Google Cloud console, let’s create a new data transfer that automatically sends Google Ads data to your BigQuery project on a regular basis.
Follow the steps below:
Step-1: Create a new BigQuery project where you are going to store Google ads data in BigQuery. If you already have a BigQuery project, use that one to create a new dataset for storing Google ads data.
Step-2: Create a new dataset in your existing project for storing Google Ads data. You can also use an existing dataset, but I prefer creating a new dataset to make data management easier.
Step-3: Navigate to https://console.cloud.google.com/bigquery
Step-4: Make sure that you are in the correct project:
Step-5: Click on the three dots menu next to your project ID:
Step-6: Click on ‘Create dataset’:
Step-7: Name your data set (e.g. ‘google_ads’) and then click on the ‘Create Dataset’ button:
You should now see the new dataset created under your project ID:
But this dataset does not contain any data table or any data. For that, you would first need to create a data transfer.
Step-8: Click on ‘Data Transfers’ from the left navigation menu:
Step-9: Click on the ‘CREATE A TRANSFER’ button:
Step-10: Enable the BigQuery data transfer API.
Note: You get the option to enable the BigQuery data transfer API only when you are creating your very first data transfer via the Google Cloud console.
Step-11: Click on the ‘Source’ drop-down menu:
Step-12: Click on ‘Google Ads’:
Step-13: Type ‘Google Ads to BigQuery’ under the field ‘Transfer config name’:
Step-14: Set your schedule options:
Step-15: Select the dataset you created earlier (from the drop-down menu) for storing Google Ads data. In our case, it is ‘google_ads’:
Step-16: Enter the customer ID (without any dashes) of the Google Ads account whose data you want to transfer to your BigQuery project.
Step-17: Click on the ‘Save’ button:
You should now see a screen like the one below, which shows the current status of your data transfer:
Step-18: Click on the browser refresh button to check the current status:
Now you can see under the ‘Summary’ section that the transfer status changed from ‘The transfer run is pending’ to ‘The transfer run is in progress’.
You may need to wait for a couple of minutes, depending on the volume of data you are transferring to your BigQuery project.
Step-19: Keep clicking on the browser refresh button after every 5 minutes until you see the transfer status of ‘The transfer run has completed successfully’.
Step-20: Click on the ‘SQL Workspace’ from the left navigation menu:
Step-21: Navigate to the ‘google_ads’ dataset you created earlier. Scroll down until you see the list of new data tables in the dataset:
Step-22: Click on one of the data tables and then on the ‘Preview’ tab to see the imported Google ads data:
Other articles on GA4 BigQuery
#1 BigQuery Introduction
- How to create a new Google Cloud Platform account.
- How to create a new BigQuery project.
- What is Google BigQuery Sandbox and how to use it.
- Understanding the BigQuery User Interface.
- What is BigQuery Data Transfer Service & how it works.
- How to create data transfer in BigQuery.
- Connect and transfer data from Google Sheets to BigQuery.
- How to access BigQuery Public Data Sets.
- Best Supermetrics Alternative – Dataddo.
#2 GA4 BigQuery Introduction
- Google Analytics 4 BigQuery Tutorial for Beginners to Advanced.
- GA4 Bigquery Export Schema Tutorial.
- GA4 BigQuery – Connect Google Analytics 4 with BigQuery.
- events_ & events_intraday_ tables in BigQuery for GA4 (Google Analytics 4).
- pseudonymous_users_ & users_ data tables in BigQuery for GA4 (Google Analytics 4).
- How to access GA4 Sample Data in BigQuery.
- Advantages of using Google BigQuery for Google Analytics 4.
- Impact of Google Advanced Consent Mode on BigQuery & GDPR.
#3 GA4 BigQuery Data Transfer
- How to Connect and Export Data from GA4 to BigQuery
- How to backfill GA4 data in BigQuery.
- How to overcome GA4 BigQuery Export limit.
- How to Send Custom GA4 Data to BigQuery.
- How to backup Universal Analytics data to BigQuery.
- How to send data from Google Ads to BigQuery.
- How to send data from Google Search Console to BigQuery.
- Sending data from Google Analytics to BigQuery without 360.
- How to send data from Facebook ads to BigQuery.
- How to pull custom data from Google Analytics to BigQuery.
#4 BigQuery Cost Optimization
- Guide to BigQuery Cost Optimization.
- Using Google Cloud pricing calculator for BigQuery.
- Cost of using BigQuery for Google Analytics 4.
#5 Query GA4 BigQuery Data
- How to query Google Analytics data in BigQuery.
- Query GA4 data in BigQuery without understanding SQL.
- Using GA4 BigQuery SQL generator to create SQL queries.
- New vs Returning users in GA4 BigQuery data table.
- GA4 BigQuery Composer Tutorial for ChatGPT.
- How to track GA4 BigQuery Schema Change.
- Calculating Sessions and Engaged Sessions in GA4 BigQuery.
- Calculating Total Users in GA4 BigQuery.
#6 GA4 to BigQuery Dimension/Metric Mapping.
- GA4 to BigQuery Mapping Tutorial.
- GA4 Attribution Dimensions to BigQuery Mapping.
- GA4 Google Ads Dimensions to BigQuery Mapping.
- GA4 Demographic Dimensions to BigQuery Mapping.
- GA4 Ecommerce Dimensions to BigQuery Mapping.
- GA4 Event-Scoped Ecommerce Metrics to BigQuery Mapping.
- GA4 Item-Scoped Ecommerce Metrics to BigQuery Mapping.
- GA4 Revenue Metrics to BigQuery Mapping.
- GA4 Event Dimensions to BigQuery Mapping.
- GA4 Event Metrics to BigQuery Mapping.
- GA4 Geography Dimensions to BigQuery Mapping.
- GA4 Link Dimensions to BigQuery Mapping.
- GA4 Page/Screen Dimensions to BigQuery Mapping.
- GA4 Page/Screen Metrics to BigQuery Mapping.
- GA4 Platform/Device Dimensions to BigQuery Mapping.
- GA4 User-Scoped Traffic Dimensions to BigQuery Mapping.
- GA4 Session-Scoped Traffic Dimensions to BigQuery Mapping.
- GA4 Session Metrics to BigQuery Mapping.
- GA4 User Dimensions to BigQuery Mapping.
- GA4 User Metrics to BigQuery Mapping.
- GA4 Advertising Metrics to BigQuery Mapping.
My best selling books on Digital Analytics and Conversion Optimization
Maths and Stats for Web Analytics and Conversion Optimization
This expert guide will teach you how to leverage the knowledge of maths and statistics in order to accurately interpret data and take actions, which can quickly improve the bottom-line of your online business.
Master the Essentials of Email Marketing Analytics
This book focuses solely on the ‘analytics’ that power your email marketing optimization program and will help you dramatically reduce your cost per acquisition and increase marketing ROI by tracking the performance of the various KPIs and metrics used for email marketing.
Attribution Modelling in Google Analytics and BeyondSECOND EDITION OUT NOW!
Attribution modelling is the process of determining the most effective marketing channels for investment. This book has been written to help you implement attribution modelling. It will teach you how to leverage the knowledge of attribution modelling in order to allocate marketing budget and understand buying behaviour.
Attribution Modelling in Google Ads and Facebook
This book has been written to help you implement attribution modelling in Google Ads (Google AdWords) and Facebook. It will teach you, how to leverage the knowledge of attribution modelling in order to understand the customer purchasing journey and determine the most effective marketing channels for investment.