How to create data transfer in BigQuery

The BigQuery data transfer service automates data transfer from popular data sources to BigQuery without the need to write any code.

Just to give you an example of how data transfers work in the Google Cloud console, let’s create a new data transfer that automatically sends Google Ads data to your BigQuery project on a regular basis.

Follow the steps below:

Step-1: Create a new BigQuery project where you are going to store Google ads data in BigQuery. If you already have a BigQuery project, use that one to create a new dataset for storing Google ads data.

Step-2: Create a new dataset in your existing project for storing Google Ads data. You can also use an existing dataset, but I prefer creating a new dataset to make data management easier.

Step-3: Navigate to https://console.cloud.google.com/bigquery 

Step-4: Make sure that you are in the correct project:

you are in the correct project

Step-5: Click on the three dots menu next to your project ID:

three dots menu next to your project ID

Step-6: Click on ‘Create dataset’:

Create dataset bigquery

Step-7: Name your data set (e.g. ‘google_ads’) and then click on the ‘Create Dataset’ button:

google ads 1

You should now see the new dataset created under your project ID:

new dataset created under your project ID 1

But this dataset does not contain any data table or any data. For that, you would first need to create a data transfer. 

Step-8: Click on ‘Data Transfers’ from the left navigation menu:

Data Transfers

Step-9: Click on the ‘CREATE A TRANSFER’ button:

CREATE A TRANSFER

Step-10: Enable the BigQuery data transfer API. 

Enable the BigQuery data transfer API

Note: You get the option to enable the BigQuery data transfer API only when you are creating your very first data transfer via the Google Cloud console.

Step-11: Click on the ‘Source’ drop-down menu:

the ‘Source drop down menu

Step-12: Click on ‘Google Ads’:

Click on ‘Google Ads

Step-13: Type ‘Google Ads to BigQuery’ under the field ‘Transfer config name’:

Transfer config name

Step-14: Set your schedule options:

Set your schedule options

Step-15: Select the dataset you created earlier (from the drop-down menu) for storing Google Ads data. In our case, it is ‘google_ads’:

Select the dataset you created earlier

Step-16: Enter the customer ID (without any dashes) of the Google Ads account whose data you want to transfer to your BigQuery project.

Enter the customer ID

Step-17: Click on the ‘Save’ button:

the ‘Save button

You should now see a screen like the one below, which shows the current status of your data transfer:

the current status of your data transfer

Step-18: Click on the browser refresh button to check the current status:

Click on the browser refresh button to check the current status

Now you can see under the ‘Summary’ section that the transfer status changed from ‘The transfer run is pending’ to ‘The transfer run is in progress’.

You may need to wait for a couple of minutes, depending on the volume of data you are transferring to your BigQuery project.

Step-19: Keep clicking on the browser refresh button after every 5 minutes until you see the transfer status of ‘The transfer run has completed successfully’.

The transfer run has completed successfully

Step-20: Click on the ‘SQL Workspace’ from the left navigation menu:

SQL Workspace

Step-21: Navigate to the ‘google_ads’ dataset you created earlier. Scroll down until you see the list of new data tables in the dataset:

list of new data tables in the dataset

Step-22: Click on one of the data tables and then on the ‘Preview’ tab to see the imported Google ads data:

the ‘Preview tab

My best selling books on Digital Analytics and Conversion Optimization

Maths and Stats for Web Analytics and Conversion Optimization
This expert guide will teach you how to leverage the knowledge of maths and statistics in order to accurately interpret data and take actions, which can quickly improve the bottom-line of your online business.

Master the Essentials of Email Marketing Analytics
This book focuses solely on the ‘analytics’ that power your email marketing optimization program and will help you dramatically reduce your cost per acquisition and increase marketing ROI by tracking the performance of the various KPIs and metrics used for email marketing.

Attribution Modelling in Google Analytics and BeyondSECOND EDITION OUT NOW!
Attribution modelling is the process of determining the most effective marketing channels for investment. This book has been written to help you implement attribution modelling. It will teach you how to leverage the knowledge of attribution modelling in order to allocate marketing budget and understand buying behaviour.

Attribution Modelling in Google Ads and Facebook
This book has been written to help you implement attribution modelling in Google Ads (Google AdWords) and Facebook. It will teach you, how to leverage the knowledge of attribution modelling in order to understand the customer purchasing journey and determine the most effective marketing channels for investment.

About the Author

Himanshu Sharma

  • Founder, OptimizeSmart.com
  • Over 15 years of experience in digital analytics and marketing
  • Author of four best-selling books on digital analytics and conversion optimization
  • Nominated for Digital Analytics Association Awards for Excellence
  • Runs one of the most popular blogs in the world on digital analytics
  • Consultant to countless small and big businesses over the decade