Load data into BigQuery from 700+ sources

Home

/

Destinations

/

BigQuery

Airbyte writes into BigQuery already partitioned by extraction time and clustered on your primary keys, so the tables you query cost less to scan from the first sync. No DDL from you.

Try it freeTalk to sales

How it works

How do you load data into BigQuery with Airbyte?

Three steps, and four fields. Most BigQuery destinations are configured and syncing inside fifteen minutes.

01

Create the dataset.

Create a BigQuery dataset in the region your data has to live in. The location cannot be changed after creation, and BigQuery queries can only reference datasets in the same physical location, so create it alongside the datasets you plan to join against.
02

Create a service account.

Grant it BigQuery User and BigQuery Data Editor. Add Storage Object Admin only if you are staging through Cloud Storage. Download the JSON key.
03

Configure the destination.

Add BigQuery as a destination in Airbyte and supply four values. Then select your streams, set each to Incremental Sync - Append + Deduped where the source exposes a primary key, and run the first sync.
Read the BigQuery connector docs
Sync pipeline

What happens to your grants when Airbyte syncs?

Not a philosophy. A set of specific, verifiable guarantees about the infrastructure your data runs on.

Why Airbyte

How tables land in BigQuery

Partitioned by extraction date, clustered on your keys, typed on arrival. No raw tables.

WHAT AIRBYTE CREATES
PARTITION BY DATE(_airbyte_extracted_at) CLUSTER BY _airbyte_extracted_at, order_id;

Partition boundaries are UTC. Require-partition-filter is not enabled, so existing queries keep working.

  • Partitioned by date.

    Queries filtered on extraction date prune partitions instead of scanning the table.

  • Clustered on your keys.

    Queries filtered on a primary key hit clustered blocks.

  • No raw tables.

    Direct Load types data on the way in and writes one final table per stream, so nothing is duplicated in storage.

Airbyte versus BigQuery’s native loading options

Google ships five ways to get data into BigQuery. Several of them are better than Airbyte for the job they were built for, and the honest version of this table is more useful than the flattering one.

MethodBest forLimitsUse Airbyte instead when
Batch load jobsFiles already in Cloud Storage. Avro, CSV, JSON, ORC, ParquetYou own extraction, scheduling, schema drift, and retriesYour data is not already sitting in a bucket in the right shape
Storage Write API and Pub/Sub subscriptionsHigh-throughput real-time event streamsYou write and operate the producer. Billed per GB ingestedYou need operational database or SaaS API data, not an event stream
BigQuery Data Transfer ServiceGoogle-owned sources: Google Ads, Google Analytics, YouTube, Cloud StorageFixed source list. Daily or monthly schedulesYour source is not on Google’s list, or you need sub-hourly syncs
DatastreamNear-real-time CDC from a small set of databasesDatabase sources only. No SaaS APIsYou need CDC and SaaS sources landing in the same dataset
External tables and federationQuerying without loading, including across AWS and AzureQuery-time cost and latency on every readThe data needs to be materialized in BigQuery

Where Airbyte wins

Heterogeneous sources. If your BigQuery project needs Postgres CDC, Salesforce, Stripe, and Google Ads landing in the same dataset on the same schedule with the same schema-drift handling, native tooling means four different mechanisms and four different failure modes. Airbyte is one.

Where it does not

If your data is already Parquet in Cloud Storage, LOAD DATA is free and you do not need a connector platform.

Raman Singh headshot

Raman Singh

Tech Lead at Symend

"With our legacy framework, if one of the pipelines fails for one client, it will stop everything for the rest of our clients. But with Airbyte, things are run in parallel because of the platform’s distributed nature, which means that we can process multiple clients at the same time without impacting performance."

75%
reduction in sync times
$ 900K
in annual savings
Learn more
Sean Carver, smiling man with beard wearing blue jacket in outdoor setting with warm lighting

Sean Carver

Director of Data at PetDesk

"The real ROI is in our ability to iterate quickly, especially at our increasing scale. At the end of the day, you want a tool like that to just work. We can forget about it and know that it's configured and it's connecting and it's working. That hands-free capability is a big appeal for the platform.”

20+
data sources integrated and growing
+1
FTE engineer in productivity efficiency
85%+
reduction in  data source integration time
Learn more
Man with glasses and goatee smiling at camera in professional headshot

Mondor La Grange

Head of BI and Data Engineering

"Unlike Fivetran's credit-based system that created budget uncertainty, Airbyte's pricing model allows Kuda to forecast expenses accurately and avoid surprise bills."

20+
data sources integrated and growing
+1
FTE engineer in productivity efficiency
85%+
reduction in  data source integration time
Learn more
Smiling woman with glasses and dark hair wearing patterned sweater in shopping mall

Amy Zhao

Senior Manager of Data Engineering

"What's different from Stitch Data or Informatica is the way that we can configure Airbyte connections and Airbyte entities through code. That's a huge plus to us as data engineers, because we are used to checking code and being able to manage changes from Github."

3 to 1
reduction in data integrations solutions for reduced TCO
1
week Shopify and Stripe integration with Airbyte
Learn more
Franziska Ibscher, woman with shoulder-length brown hair and white top, smiling at camera

Franziska Ibscher

Product Manager at Drivepoint

"Airbyte allows us to stay flexible while scaling from hundred-million to billion-dollar enterprise clients."

75%
of customers increased profitability
6.7%
EBITDA increase for customers
Learn more

Which sources can you load into BigQuery?

All 700+ of them. These are the ones BigQuery users start with.

Databases

PostgresDatabasesMySQLDatabasesSQL ServerDatabasesMongoDBDatabasesOracleDatabasesDynamoDBDatabases

SaaS applications

SalesforceSaaS applicationsHubSpotSaaS applicationsStripeSaaS applicationsZendeskSaaS applicationsGoogle AdsSaaS applicationsShopifySaaS applications

Files and object storage

Amazon S3Files and object storageGoogle Cloud StorageFiles and object storageGoogle SheetsFiles and object storageSFTPFiles and object storage

Events and streaming

KafkaEvents and streamingAmazon KinesisEvents and streaming
Browse all 700+ sources

How does your data land in BigQuery?

Airbyte writes one final table per stream, typed on arrival. Every table carries four metadata columns: _airbyte_raw_id, _airbyte_generation_id, _airbyte_extracted_at, and _airbyte_meta.
Airbyte typeBigQuery type
STRINGSTRING
INTEGERINT64
NUMBERNUMERIC
BOOLEANBOOL
DATEDATE
TIMESTAMP_WITH_TIMEZONETIMESTAMP
TIMESTAMP_WITHOUT_TIMEZONEDATETIME
OBJECTJSON
ARRAYJSON

Schema evolution

  • Column added.

    Propagates to the BigQuery table automatically on the next sync.

  • Type changed.

    Configurable. Propagate it, ignore it, or pause the connection for manual review.

  • Column removed.

    Retained in BigQuery with NULL written for subsequent rows, so an upstream change does not destroy history. Clearing or refreshing the connection does remove the field and its history.

The numbers

Teams running BigQuery on Airbyte

  • 0M

    pipelines synced daily

  • 18K+

    companies

  • 199%

    ROI, certified by Forrester

Deployment

Where Airbyte runs

Your warehouse credentials and your data do not have to leave your environment. Airbyte does not retain customer data on its servers.

  • Airbyte Cloud

    Fully managed hosting on the Standard and Pro plans. Fastest path to a running pipeline.

  • Airbyte Flex

    Hybrid deployment, with Airbyte’s data plane running in your cloud, VPC, or on-prem environment. You get 700+ connectors and a fully managed experience, while your data never touches our servers.

  • Self-Managed

    Open source, your infrastructure, your rules.

See hybrid deployment

compliance

Enterprise ready

Compliant with standards

SOC 2 Type II certified, GDPR and HIPAA support, with tools to help you meet internal and external regulatory requirements.

Uptime & SLA Guarantees

24/7 support and 99.9% availability backed by contractual SLAs and priority response times for mission-critical workloads.

Flexible deployment options

Move data into your warehouse/lakehouse of choice, VPC, on-prem, or into Airbyte's Context Store.

talk to usTrust Center

Frequently asked questions

Didn’t find your answer?
Please don’t hesitate to reach out.

Talk to us

It depends on where the data starts. If files already sit in Cloud Storage as Avro, CSV, JSON, ORC, or Parquet, batch load jobs are free and you do not need a connector platform. Storage Write API and Pub/Sub are for high-throughput event streams you operate yourself. Data Transfer Service covers Google-owned sources on daily or monthly schedules. Datastream is near-real-time CDC from a small set of databases. If the data starts in a SaaS API or an operational database, or you need those sources landing in the same dataset, Airbyte handles extraction, schema drift, and incremental state, then loads through native BigQuery load jobs.

Data Transfer Service is a fixed list of Google-owned sources on daily or monthly schedules. Airbyte is 700+ sources, including Postgres CDC, Salesforce, Stripe, and sources that are not on Google’s list, with sub-hourly syncs when you need them.

Yes. Sources that expose change data capture or incremental cursors sync into BigQuery. Use Incremental Sync - Append + Deduped where the source exposes a primary key. Datastream covers a small set of databases only and does not include SaaS APIs.

When a column is added at the source, it propagates to the BigQuery table automatically on the next sync. When a type changes, you can propagate it, ignore it, or pause the connection for manual review. When a column is removed, Airbyte keeps the column and writes NULL for subsequent rows. Clearing or refreshing the connection removes the field and its history.

No. Direct Load types data on the way in and writes one final table per stream, so nothing is duplicated in storage.

Airbyte stages files in Google Cloud Storage, then loads them with native load jobs. Add Storage Object Admin on the service account only if you are staging through Cloud Storage.

Yes. With Airbyte Flex, Airbyte’s data plane runs in your Google Cloud project, VPC, or on-prem environment while you keep a fully managed experience. Self-Managed runs entirely on your own infrastructure.

Other destinations

Snowflake

Airbyte clones tables instead of swapping them, so the grants you configured survive every sync. Removed source columns are kept, and oversized values are logged rather than fatal.

Load data into Snowflake

ClickHouse

Airbyte writes to ClickHouse over its native binary protocol with batch inserts. The setup needs a hostname, a port, and a user. No object storage, no staging permissions, no intermediate hop.

Load data into ClickHouse

Databricks

Airbyte stages Avro files inside Unity Catalog Volumes, not an external bucket, so your data stays under your catalog’s governance from the first byte to the final table.

Load data into Databricks