Transferring data from Klaviyo to BigQuery

An important part of business marketing strategy is analyzing how your customer acquisition (or onboarding) campaigns perform. Many tools exist in the market that can help you automate some parts of the job. One such tool is Klaviyo which provides powerful tools like campaign management to engage with your new customers successfully. 

With time, the user data tends to increase, needing large-scale analytics over over bigger datasets and requiring you to set up a data warehouse pipeline. Tools like BigQuery, and Snowflake are reliable options. Google BigQuery is a fully managed serverless data warehouse that provides scalable analysis over large datasets. BigQuery also offers real-time analysis of data.

In this tutorial, we will look at how you can move data from Klaviyo to BigQuery without any manual work, fully automated with Airbyte.

Why Klaviyo?

Klaviyo is an E-Commerce marketing automation platform that has several benefits.

• Klaviyo gives online brands direct ownership of their consumer data and interactions, empowering them to turn transactions with customers into long-term relationships—at scale. 
• With Klaviyo, brands can combine unlimited customer data with more than 250 native integrations to automate personalized email and SMS communications that make customers feel seen. 
• Klaviyo makes it easy—no need to start from scratch, piece together multiple platforms, or rely on third-party marketplaces and ad networks. 
• From mom-and-pop shops to established companies, innovative brands like Unilever, Bonobos, Taylor Made, Citizen Watches, and more than 90K other paying users leverage Klaviyo to acquire, engage, and retain customers—and grow on their own terms. 

Prerequisites

1. A Klaviyo account to transfer your customer data from.
2. A Google cloud account to create our BigQuery data warehouse.
3. An active Airbyte cloud account, or you can also choose to use Airbyte locally. You can follow the instructions to set up Airbyte on your system using docker-compose.

Step 1: Setup Klaviyo as an Airbyte source

To set up Klaviyo, login to your dashboard and head over to settings from the left-hand side menu bar.

‍

‍

Once you navigate to the settings page, choose the API Keys option from the top bar.

‍

You will now be able to see two different sections of API keys. One of which is a public key (or site ID) which is usually embedded with the signup forms. To export data from Klaviyo to Airbyte, we need a Private Key instead. Click on the Create Private Key button to generate one.

You should see that Klaviyo has added a new row in the private keys table. You can choose to add a label to this key in case you have multiple third-party integrations set up.

Now head over to your Airbyte Dashboard and, from Sources, select Klaviyo as a new source. Add the API key, which we just created. For the start date, you can add the date you created the Klaviyo account.

Once you are satisfied with your changes, choose the Set-up source button.

Step 2: Setup BigQuery as Airbyte Destination

To set up a big query Airbyte destination, ensure you are logged in to your Google cloud dashboard. Now we need to create a BigQuery dataset to load our customer data to BigQuery.

From the main page, choose the Run a query in BigQuery button.

From the 3-dot menu of your cloud project, choose the only option to create a dataset

For Dataset ID, choose a descriptive name like kalviyo_dataset. Only alphanumeric names are allowed as dataset ID names.

‍

Choose the dataset location from the menu and click Create Dataset. You should now be able to see a newly created dataset. Take note of the Dataset ID that you just entered, since we will need it, later on, to set up BigQuery as an Airbyte Destination.

The only thing left to do is get the appropriate account keys to access our BigQuery project. Choose API & Services from the Quick Access menu in your cloud dashboard.

From the credential menu, choose to create a new Service account.

Choose a valid name for your Service account.

We need to tell google about what resources can be accessed from this service account. From the available roles, choose BigQuery Admin.

Click Done, and you should see a new service account.

‍

Choose this newly created service account email and add a new key by clicking on the Add Key button.

From the Create private key pop-up, choose JSON.

Once you choose to Create, a new file must have been downloaded to your system. The final step involves creating a new Airbyte destination. 

From your Airbyte dashboard, choose Destinations and choose New Destination, and pick BigQuery from the available options.

Under Service Account Key JSON, copy the value from the credentials file.

Step 3: Setup Airbyte connection between Klaviyo and BigQuery

Go to Connections in your dashboard and pick New connection. From the existing source & destinations available, pick the ones which we just created. Once you select the source & destinations, you should see the connection information like this

As you can see, Airbyte is providing us six different available streams to sync data from including campaigns, event flows, global_exclusions, lists, and metrics. You can choose to disable streams to sync data by clicking on the toggle button.

For sync mode, select one from the available modes. A sync mode defines how the data should be stored to the destination. A Full Refresh - Overwrite will overwrite data at the destination, whereas an Incremental mode will sync new records from the source and add them to the destination without overwriting previous data. You can learn more about sync modes in our documentation.

You can choose to set up a connection once you are satisfied with the connection configuration. If you choose Replication frequency as Manual, you will have to start the sync manually. 

Once the sync completes, you will be able to see new tables that were created by Airbyte in your BigQuery project.

Choose a table to preview its contents and schema.

As you can see, Airbyte successfully synchronized two of our sample campaigns from our Klaviyo account.

The real power of BigQuery is being able to run SQL queries over datasets. Here is a sample query to list our campaigns

Wrapping Up

To summarize, this tutorial has shown you how to:

1. Configure a Klaviyo account as an Airbyte source via their API.
2. Configure a data warehouse using Google BigQuery.
3. Create an Airbyte connection that automatically syncs (or migrates) data from a Klaviyo to BigQuery.

With Airbyte, the data integration possibilities are endless, and we look forward to seeing you make use of it! We invite you to join the conversation on our community Slack Channel, to participate in discussions on Airbyte’s discourse, or sign up for our newsletter. You should also check out other Airbyte tutorials, and Airbyte’s blog!