Move data from LinkedIn Ads to BigQuery

Airbyte Cloud allows you to seamlessly move data between hundreds of  data sources and destinations, including popular databases, data warehouses, and business applications. In this article we will show you how to easily move data between LinkedIn Ads and Google’s BigQuery. 

LinkedIn Ads is a powerful self-service marketing tool that supports advertising on LinkedIn, which allows businesses to launch paid digital ad campaigns that are targeted at a specific audience segment. 

BigQuery is Google's fully managed, serverless, petabyte-scale, low-cost enterprise data warehouse for analytics. It allows you to analyze your data with built-in features like machine learning, geospatial analysis, and business intelligence. BigQuery's serverless architecture lets you use SQL queries to answer your organization's biggest questions with zero infrastructure management.

Below are a few of the reasons why you may wish to move data from LinkedIn Ads to BigQuery: 

1. Ad-hoc data queries: With LinkedIn Ads, your queries are restricted to the data model that is exposed by the LinkedIn Ads API. Moving your marketing data to a warehouse like BigQuery will allow you to efficiently and easily execute sophisticated ad-hoc analytical queries.  

2. Blending other data sources: BigQuery can be used for data integration which will  provide cross-channel marketing performance insights. This is extremely useful for answering key questions like - “What is the total ROI of my campaigns across all channels?”, or “What is the average ad spend across all my campaigns this quarter?” 

3. Enhanced reporting capabilities: LinkedIn’s reporting dashboard is basic and has limited functionality. Once data is in BigQuery it becomes much easier to analyze your LinkedIn Ads data across multiple dimensions and to create detailed custom reports. 

‍

While it may be possible for a software developer to write custom code to export marketing data from the LinkedIn Ads API and send it to BigQuery, this would require significant engineering effort and investment.  On the other hand, Airbyte Cloud provides pre-built functionality that is built exactly for this purpose. 

This tutorial takes you through the steps to set up Airbyte Cloud so that you can easily move your data from LinkedIn ads into Google BigQuery.

You will need the following to sync your LinkedIn Ads data to BigQuery.

1. Register for Airbyte Cloud or deploy Airbyte Open-Source.  This tutorial is presented using Airbyte Cloud, but most of it should also work with the Open-Source version of Airbyte.  
2. You will need a LinkedIn account set up with the Campaign Manager enabled. 
3. You will also need a Google Cloud Platform account with BigQuery enabled

If you're aiming for limitless LinkedIn scraping alongside your data syncing, consider ScrapIn. This tool offers unparalleled freedom in scraping infinite LinkedIn profiles without concerns of restrictions or limits and can complement your marketing analysis in BigQuery.

For this example, you will be using Airbyte to sync Campaign data from the LinkedIn Ads API to BigQuery. To start with, use a new ad campaign.

‍

Next, you will set up LinkedIn Ads as your source, by logging in to Airbyte Cloud and creating a new connection.  Give the connection a name and select LinkedIn Ads source connector as the Source Type as shown below. 

‍

Enter the start date for when to start syncing the data and click on the Authenticate your Linkedin Ads account as shown below.

‍

If you are following along in Airbyte Cloud, you will then be prompted to sign into your LinkedIn account. For Airbyte Open-Source, see authentication options. 

‍

Once signed in, you will have to give Airbyte access to your LinkedIn Ads data. 

‍

Once successfully authenticated, you will be redirected back to Airbyte cloud. Click on the set up source button, to complete the set up.

‍

To set up BigQuery as your Airbyte destination, you will need to create a Service account with BigQuery Access. Login to GCP and go to the API–>Credentials page and create a new service account. 

‍

Give the service account name and click on Create and Continue.

‍

For simplicity and testing purposes you may select BigQuery Admin and create the service account as shown below. 

Note: for a production BigQuery instance you should assign reduced permissions to the service account. BigQuery Data Editor, and BigQuery Job User should be sufficient.

Once created, you can view details for the service account. 

Go to the keys tab and create a new auth key. You will be prompted to download the private key file. Download the private key file in JSON format.

‍

After you have downloaded the service credential private key file, you can begin configuring your Airbyte Cloud destination. Select BigQuery as the destination type and give it a name. Details can be found in the BigQuery destination connector documentation.

‍

Enter your dataset ID, project ID, and the dataset location in their respective fields.  Under Service Account Key JSON, copy the value from the credentials file and click on the “Set up destination”.

‍

Once the source and destination are configured, you can access your connection settings. You can select the frequency for how often your data is replicated by Airbyte.

‍

You should also be able to see the various data streams available from LinkedIn Ads. You can set the sync frequency and the sync mode for each stream individually. In this example, we will select the campaigns data and set the sync mode to Incremental | Append. 

‍

Once configured, click on the “Set up connection” button.

After creating a connection, you can start a sync by selecting Sync Now. 

‍

Once the sync is complete, go to your Cloud Console BigQuery page. You should now be able to see the tables created by Airbyte Cloud in your data set.

‍

You can select the tables to view their structure. The following image shows the structure of the campaigns table.  

‍

You can run the following query to view the synced data in BigQuery :

> SELECT * FROM `{YOUR_PROJECT_ID}.linkedin_ads.campaigns`;

‍

To test incremental sync, you can add more data on LinkedIn Ads. As an example to simplify this, we’ve added another campaign. 

‍

Running another sync will bring the new data to BigQuery.

‍

You can rerun the following query to view the newly synced data in BigQuery :

> SELECT * FROM `{YOUR_PROJECT_ID}.linkedin_ads.campaigns`;

‍

This tutorial has illustrated how easy it is to set up a connection that retrieves marketing data from the LinkedIn Ads API and that sends it to BigQuery using Airbyte Cloud. Once your data is in BigQuery you can use it with a range of different BigQuery integrations to generate further insights. 

‍

To summarize,  this tutorial has shown you how to:

1. Configure a LinkedIn Ads Airbyte source 
2. Configure a BigQuery Airbyte destination 
3. Create an Airbyte connection that automatically syncs data from LinkedIn Ads to BigQuery
4. Incrementally sync LinkedIn Ads data to BigQuery

‍

With Airbyte, the data integration possibilities are endless, and we look forward to seeing you make use of it! We invite you to join the conversation on our community Slack Channel, to participate in discussions on Airbyte’s discourse, or to sign up for our newsletter. You should also check out other Airbyte tutorials, and Airbyte’s blog!