Building your pipeline or Using Airbyte
Airbyte is the only open solution empowering data teams to meet all their growing custom business demands in the new AI era.
- Inconsistent and inaccurate data
- Laborious and expensive
- Brittle and inflexible
- Reliable and accurate
- Extensible and scalable for all your needs
- Deployed and governed your way
Start syncing with Airbyte in 3 easy steps within 10 minutes
Take a virtual tour
Demo video of Airbyte Cloud
Demo video of AI Connector Builder
What sets Airbyte Apart
Modern GenAI Workflows
Move Large Volumes, Fast
An Extensible Open-Source Standard
Full Control & Security
Fully Featured & Integrated
Enterprise Support with SLAs
What our users say
"The intake layer of Datadog’s self-serve analytics platform is largely built on Airbyte.Airbyte’s ease of use and extensibility allowed any team in the company to push their data into the platform - without assistance from the data team!"
“Airbyte helped us accelerate our progress by years, compared to our competitors. We don’t need to worry about connectors and focus on creating value for our users instead of building infrastructure. That’s priceless. The time and energy saved allows us to disrupt and grow faster.”
“We chose Airbyte for its ease of use, its pricing scalability and its absence of vendor lock-in. Having a lean team makes them our top criteria. The value of being able to scale and execute at a high level by maximizing resources is immense”
FAQs
What is ETL?
ETL, an acronym for Extract, Transform, Load, is a vital data integration process. It involves extracting data from diverse sources, transforming it into a usable format, and loading it into a database, data warehouse or data lake. This process enables meaningful data analysis, enhancing business intelligence.
The Google Ads API is the modern programmatic interface to Google Ads and the next generation of the AdWords API and it is a paid online advertising platform offered by Google. Google Ads is a paid search channel. Google Ads is a key digital marketing tool for any business which is looking to get meaningful ad copy in front of its target audience. Google AdWords is a well known marketplace where companies pay to have their website ranked at the top of a search results page, based on keywords.
Google Ads API provides access to a wide range of data related to advertising campaigns, including:
- Campaigns: Information about the campaigns, such as name, status, budget, and targeting settings.
- Ad groups: Details about the ad groups, including name, status, and targeting criteria.
- Ads: Information about the ads, such as type, format, and performance metrics.
- Keywords: Data related to the keywords used in the campaigns, including search volume, competition, and performance metrics.
- Bidding: Details about the bidding strategies used in the campaigns, such as manual bidding or automated bidding.
- Conversions: Information about the conversions generated by the campaigns, including conversion rate, cost per conversion, and conversion tracking settings.
- Audience: Data related to the audience targeting used in the campaigns, such as demographics, interests, and behaviors.
- Location: Information about the geographic targeting used in the campaigns, including location targeting settings and performance metrics.
Overall, the Google Ads API provides a comprehensive set of data that can be used to optimize advertising campaigns and improve their performance.
What is ELT?
ELT, standing for Extract, Load, Transform, is a modern take on the traditional ETL data integration process. In ELT, data is first extracted from various sources, loaded directly into a data warehouse, and then transformed. This approach enhances data processing speed, analytical flexibility and autonomy.
Difference between ETL and ELT?
ETL and ELT are critical data integration strategies with key differences. ETL (Extract, Transform, Load) transforms data before loading, ideal for structured data. In contrast, ELT (Extract, Load, Transform) loads data before transformation, perfect for processing large, diverse data sets in modern data warehouses. ELT is becoming the new standard as it offers a lot more flexibility and autonomy to data analysts.
The Google Ads API is the modern programmatic interface to Google Ads and the next generation of the AdWords API and it is a paid online advertising platform offered by Google. Google Ads is a paid search channel. Google Ads is a key digital marketing tool for any business which is looking to get meaningful ad copy in front of its target audience. Google AdWords is a well known marketplace where companies pay to have their website ranked at the top of a search results page, based on keywords.
A cloud data platform, Snowflake Data Cloud provides a warehouse-as-a-service built specifically for the cloud. The Snowflake platform is designed to empower many types of data workloads, and offers secure, immediate, governed access to a comprehensive network of data. Snowflake’s innovative technology goes above the capabilities of the ordinary database, supplying users all the functionality of database storage, query processing, and cloud services in one package.
1. Go to the Airbyte dashboard and click on "Sources" on the left-hand side of the screen.
2. Click on the "New Source" button and select "Google Ads" from the list of available connectors.
3. Enter a name for your connector and click on "Next".
4. Enter your Google Ads credentials, including your client ID, client secret, refresh token, and developer token.
5. Click on "Test Connection" to ensure that your credentials are correct and that Airbyte can connect to your Google Ads account.
6. Once the connection is successful, select the accounts that you want to sync with Airbyte.
7. Choose the sync mode that you want to use, either "Full Refresh" or "Incremental".
8. Set the frequency of your sync and click on "Create Source" to save your settings.
9. Your Google Ads source connector is now set up and ready to use. You can view your data in the Airbyte dashboard and start syncing it with your destination.
1. First, navigate to the Airbyte website and log in to your account.
2. Once you are logged in, click on the "Destinations" tab on the left-hand side of the screen.
3. Scroll down until you find the Snowflake Data Cloud destination connector and click on it.
4. You will be prompted to enter your Snowflake account information, including your account name, username, and password.
5. After entering your account information, click on the "Test" button to ensure that the connection is successful.
6. If the test is successful, click on the "Save" button to save your Snowflake Data Cloud destination connector settings.
7. You can now use the Snowflake Data Cloud destination connector to transfer data from your Airbyte sources to your Snowflake account.
8. To set up a data transfer, navigate to the "Sources" tab on the left-hand side of the screen and select the source you want to transfer data from.
9. Click on the "Create New Connection" button and select the Snowflake Data Cloud destination connector as your destination.
10. Follow the prompts to set up your data transfer, including selecting the tables or data sources you want to transfer and setting up any necessary transformations or mappings.
11. Once you have set up your data transfer, click on the "Run" button to start the transfer process.
With Airbyte, creating data pipelines take minutes, and the data integration possibilities are endless. Airbyte supports the largest catalog of API tools, databases, and files, among other sources. Airbyte's connectors are open-source, so you can add any custom objects to the connector, or even build a new connector from scratch without any local dev environment or any data engineer within 10 minutes with the no-code connector builder.
We look forward to seeing you make use of it! We invite you to join the conversation on our community Slack Channel, or sign up for our newsletter. You should also check out other Airbyte tutorials, and Airbyte’s content hub!
What should you do next?
Hope you enjoyed the reading. Here are the 3 ways we can help you in your data journey:
Google Ads is an advertising platform that provides a pay-per-click marketing channel. It offers an effective way for advertisers to drive traffic to their businesses when users search for related products or services, and allows advertisers to reach the right target audience.
When you combine data from Google Ads with other marketing data you can gain valuable insights into how you acquire customers, the value of your customer conversions from advertisements, and much more. Airbyte provides a fully managed data pipeline that allows you to seamlessly move your data in Google Ads to Snowflake to get near-real-time insights, which will provide the following benefits.
Build a 360-degree view of your business
Analyzing data scattered across different functions like sales, finance, and marketing can be a challenge. However, if you combine your Google Ads data with all other relevant data in one platform, you will get a more holistic view of your business. A cloud-based data warehouse like Snowflake can become the single source of truth to understand your business. For a deep dive into this topic, see our guide to data integration.
Improved analytics capabilities
The export features provided by Google Ads are limited to simple files such as Excel or CSV files, or other Google products such as BigQuery. On the other hand, if you move your data into a data warehouse such as Snowflake, you will have more flexibility accessing and analyzing your data, including running queries with SQL or using BI tools for data visualization.
Own and retain your data on your own systems
Once you drive your Google Ads data into a data warehouse, you are not tied to Google Ads’ data retention policies – you can keep your data for as long as you want, which gives you full access to your data whenever you need it, no matter how old or new.
With Airbyte's connectors, you can easily create a data pipeline between any supported source and any supported destination. In this tutorial, we will cover the steps to move data from the Google Ads platform to Snowflake. Before we get started, let's review the prerequisites.
{{COMPONENT_CTA}}
Prerequisites
- Signup for an Airbyte Cloud account to do the data replication for you.
- Create a Google Ads account.
- Create a Snowflake account.
Methods to Move Data From Google ads to snowflake
- Method 1: Connecting Google ads to snowflake using Airbyte.
- Method 2: Connecting Google ads to snowflake manually.
Method 1: Connecting Google ads to snowflake using Airbyte
Step 1: Set up your Google Ads account
To start, login to your Google Ads account. Once you have entered into your Google Ads account and have an ad campaign set up, you will need a manager account. If you don’t have one already, you can create a manager account as shown below.
Once you have your Google Ads and your Google Ads manager accounts set up, you will need to note the Customer IDs for both the ad account and the manager account. These values will be required later on to configure Airbyte. You can find these by going to Settings > Sub-account settings.
Step 2: Set up your Snowflake account
When creating a Snowflake account, you’ll need to pick a Snowflake edition and a cloud provider as part of the account creation process. Then you’ll receive an email containing your login URL. Make sure to bookmark this URL for future logins.
Once your account is successfully created, you'll be redirected to the Snowflake dashboard. The worksheet area will be the primary place where you’ll run scripts for creating and modifying resources. You will need to set up the destination database, user, role, and schema on Snowflake which will be used by Airbyte when syncing data.
The good news is that Airbyte provides a script for automating this process, which you can find in the Airbyte Snowflake destination docs. Paste the snippet below into your worksheet area on Snowflake.
-- set variables (these need to be uppercase)
set airbyte_role = 'AIRBYTE_ROLE';
set airbyte_username = 'AIRBYTE_USER';
set airbyte_warehouse = 'AIRBYTE_WAREHOUSE';
set airbyte_database = 'AIRBYTE_DATABASE';
set airbyte_schema = 'AIRBYTE_SCHEMA';
-- set user password
set airbyte_password =’YOUR_AIRBYTE_PASSWORD';
begin;
-- create Airbyte role
use role securityadmin;
create role if not exists identifier($airbyte_role);
grant role identifier($airbyte_role) to role SYSADMIN;
-- create Airbyte user
create user if not exists identifier($airbyte_username)
password = $airbyte_password
default_role = $airbyte_role
default_warehouse = $airbyte_warehouse;
grant role identifier($airbyte_role) to user identifier($airbyte_username);
-- change role to sysadmin for warehouse/database steps
use role sysadmin;
-- create Airbyte warehouse
create warehouse if not exists identifier($airbyte_warehouse)
warehouse_size = xsmall
warehouse_type = standard
auto_suspend = 60
auto_resume = true
initially_suspended = true;
-- create Airbyte database
create database if not exists identifier($airbyte_database);
-- grant Airbyte warehouse access
grant USAGE
on warehouse identifier($airbyte_warehouse)
to role identifier($airbyte_role);
-- grant Airbyte database access
grant OWNERSHIP
on database identifier($airbyte_database)
to role identifier($airbyte_role);
commit;
begin;
USE DATABASE identifier($airbyte_database);
-- create schema for Airbyte data
CREATE SCHEMA IF NOT EXISTS identifier($airbyte_schema);
commit;
begin;
-- grant Airbyte schema access
grant OWNERSHIP
on schema identifier($airbyte_schema)
to role identifier($airbyte_role);
commit;
Note. Be sure to change the airbyte_password before running the script.
Select All queries and run the script by clicking on the run button.
Once successfully executed, you should see the following message:
Step 3: Set up Google Ads as the Airbyte source
Now that you have your Google Ads account set up and the required Customer IDs, you can start configuring the Google Ads source in Airbyte. Login to Airbyte Cloud, select or create a new workspace, and create a new connection. Give the connection a name and choose Google Ads as the Source Type.
You will be prompted to sign in to your Google account.
Once you select the Google account, you will be prompted to give Airbyte additional access that it requires to replicate your Google Ads data to Snowflake.
Once you are successfully signed in, you will be redirected to Airbyte.
Next, enter the Google Ad account ID and the Manager Account ID from Step 1 in the Customer ID and Login Customer ID for Managed Accounts fields. Next, specify the start and end dates, and complete the configuration by clicking on Set up source.
Step 4: Set up Snowflake as the Airbyte destination
The next step is setting up Snowflake as the destination. Select Snowflake as the destination type and give it a name. See our Snowflake docs for more information.
Enter the values for the fields based on the values set in the script in Step 2. For the host, enter the URL you received when signing up for Snowflake. If you updated the password in your script, enter the new password. Once ready, click on the Set up destination button.
Step 5: Set up a Google Ads to Snowflake Airbyte connection
Once the source and destination are configured, you can access your connection settings. Here you can specify the connection settings including the replication frequency, the destination namespace, and also set a destination stream prefix if required.
You can also see the various source data sets (referred as streams by Airbyte) that are available to be copied.
You can set sync mode for each data set individually. In this case, we will set the campaigns data to use the Incremental Sync with the source-defined cursor field. You can also choose between using raw data or basic normalization.
Once configured, save the connection and select Sync now.
Once the sync and normalization are complete, you can go to the Database section in the Snowflake UI to see the tables that have been copied. If you have chosen to normalize your data, each of the datasets available from Google Ads will have been copied into its own table. You should also be able to access the raw data in a separate table with the name _AIRBYTE_RAW_{TABLE_NAME}.
You can view the structure of the table as well as the data types for each of the fields. Airbyte automatically maps the data types from Google Ads to the corresponding data types in Snowflake. For example, the campaign's data has the following structure.
You can test out the incremental sync for the campaigns table by logging on to Google Ads and adding a new campaign. Once you add some more data, run another sync from Airbyte.
Once the sync has been completed, you can go back to Snowflake to see the updated counts for the appended rows.
Method 2: Connecting Google ads to snowflake manually
Moving data from Google Ads to Snowflake without using third-party connectors or integrations will require you to manually extract data from Google Ads, format it properly, and then load it into Snowflake. Below is a step-by-step guide to facilitate this process:
Step 1: Extract Data from Google Ads
- Log in to your Google Ads account.
- Navigate to the Reports section.some text
- You can create custom reports if you need specific data or use predefined reports.
- Select the data you want to extract.some text
- Choose the metrics and dimensions that are relevant to your analysis.
- Export the report.some text
- Google Ads allows you to export data in various formats such as CSV, Excel, or Google Sheets.
Step 2: Prepare the Data
- Clean and format the data.some text
- Open the exported file and make sure the data is clean (e.g., no missing values, correct data types).
- Ensure that the format of the data matches the schema you intend to use in Snowflake (e.g., dates in YYYY-MM-DD format).
- Save the cleaned data as a CSV file.some text
- CSV is a common format for data loading and is supported by Snowflake.
Step 3: Create a Stage in Snowflake
- Log in to your Snowflake account.
- Create a file stage.some text
- Use the CREATE STAGE command to create a stage for your data files. For example:
CREATE STAGE my_google_ads_stage
FILE_FORMAT = (TYPE = 'CSV' FIELD_DELIMITER = ',' SKIP_HEADER = 1);
Step 4: Upload Data to the Stage
- You can manually upload the CSV file through the Snowflake web interface, or use the PUT command in Snowflake to upload the file from your local machine to the stage you created.
- For the PUT command, you’ll need SnowSQL or Snowflake’s command-line tool.
PUT file://path_to_your_csv_file @my_google_ads_stage;
Step 5: Create a Target Table in Snowflake
Create a table that matches the structure of the Google Ads data.
CREATE TABLE google_ads_data (
column1_name column1_datatype,
column2_name column2_datatype,
...
);
Step 6: Copy Data into the Target Table
Load the data from the stage into the Snowflake table.
COPY INTO google_ads_data
FROM @my_google_ads_stage
FILE_FORMAT = (FORMAT_NAME = 'CSV');
Step 7: Verify the Data Load
- Check the loaded data.some text
- Execute a SELECT query to verify the data has been loaded correctly.
SELECT * FROM google_ads_data LIMIT 10;
- Look for any errors or warnings.some text
- If there are any issues with the data load, Snowflake will provide error logs that you can review and correct.
Step 8: Automate the Process (Optional)
- Schedule the data extraction from Google Ads.some text
- Use Google Ads scripts or scheduled reports to automate data extraction.
- Automate the data upload to Snowflake.some text
- Write a script that uses SnowSQL to automate data staging and loading into Snowflake.
- Consider using cron jobs (Linux) or Task Scheduler (Windows) to schedule the script.
Step 9: Clean Up
After successful data loading, clean up the staged files to save storage space.
REMOVE @my_google_ads_stage/pattern='*.csv';
Step 10: Monitor and Maintain
- Monitor the data loading process.some text
- Regularly check for any issues or failures in the data loading process.
- Maintain the data pipeline.some text
- Update the process as necessary, for example, if Google Ads changes their reporting structure or if you need to modify the Snowflake table schema.
By following these steps, you can move data from Google Ads to Snowflake without third-party connectors or integrations. However, this manual approach requires a good understanding of both platforms and might not be as efficient or robust as using dedicated data integration tools.
Conclusion
In this article we have shown you how you can use Airbyte to move from Google Ads to Snowflake. The steps were:
- Set up your Google Ads account
- Set up your Snowflake account
- Set up Google Ads as the Airbyte source
- Set up Snowflake as the Airbyte destination
- Set up a Google Ads to Snowflake Airbyte connection
If you want to easily try out Airbyte to replicate your Google Ads account to Snowflake, you may be interested in our fully managed solution: Airbyte Cloud. We also invite you to join the conversation on our community Slack to share your ideas with thousands of data professionals and help make everyone’s project a success!
What should you do next?
Hope you enjoyed the reading. Here are the 3 ways we can help you in your data journey:
Ready to get started?
Frequently Asked Questions
Google Ads API provides access to a wide range of data related to advertising campaigns, including:
- Campaigns: Information about the campaigns, such as name, status, budget, and targeting settings.
- Ad groups: Details about the ad groups, including name, status, and targeting criteria.
- Ads: Information about the ads, such as type, format, and performance metrics.
- Keywords: Data related to the keywords used in the campaigns, including search volume, competition, and performance metrics.
- Bidding: Details about the bidding strategies used in the campaigns, such as manual bidding or automated bidding.
- Conversions: Information about the conversions generated by the campaigns, including conversion rate, cost per conversion, and conversion tracking settings.
- Audience: Data related to the audience targeting used in the campaigns, such as demographics, interests, and behaviors.
- Location: Information about the geographic targeting used in the campaigns, including location targeting settings and performance metrics.
Overall, the Google Ads API provides a comprehensive set of data that can be used to optimize advertising campaigns and improve their performance.
What should you do next?
Hope you enjoyed the reading. Here are the 3 ways we can help you in your data journey: