Connect GCS Data Lake data with any data source.

Replicate any source into GCS Data Lake without custom pipelines.

Load data into GCS Data Lake Read the docs

Sources: 590+
Certified support: 1.0.13

Overview

What is the Airbyte GCS Data Lake connector?

The Airbyte GCS Data Lake connector loads data from 590+ Airbyte sources into GCS Data Lake, with full refresh and incremental sync on a schedule you control.

01
Load from any source
Move data into GCS Data Lake from 590+ Airbyte sources on a schedule you control.
02
Incremental syncs
Pull only the records that changed since the last run instead of reloading everything.
03
Cloud or self-hosted
Run the GCS Data Lake connector on Cloud, OSS, Hybrid.

Setup

Live in three steps

Confirm your GCS Data Lake destination credentials and permissions.
Configure the destination in Airbyte with the required connection settings.
Choose a source, map streams, and schedule syncs into GCS Data Lake.

Configuration fields

GCS Data Lake configuration fields
Field	Type	Description
catalog_type*	object	Specifies the type of Iceberg catalog (BigLake or Polaris).
gcp_location*	string	The GCP location (region) for BigLake metastore resources. For example: "us-central1" or "us". See BigLake locations for available regions.
gcs_bucket_name*	string	The name of the GCS bucket that will host the Iceberg data.
main_branch_name*	string	The primary or default branch name in the catalog. Most query engines will use "main" by default. See Iceberg documentation for more information.
namespace*	string	The default namespace to use for tables. This will ONLY be used if the Destination Namespace setting is set to Destination-defined or Source-defined
service_account_json*	string	The contents of the JSON service account key file. See the Google Cloud documentation for more information on how to obtain this.
warehouse_location*	string	The root location of the data warehouse used by the Iceberg catalog. Must include the storage protocol "gs://" for Google Cloud Storage. For example: "gs://your-bucket/path/to/warehouse/
gcp_project_id	string	The GCP project ID where resources are located. If not specified, it will be extracted from the service account credentials.

Hybrid deployment

Sovereign by architecture, not by policy.

The data plane runs in your environment. Records, credentials and keys stay with you; the control plane only receives configuration and job metadata.

Cloud
OSS
Hybrid

Your environment

DataConnectorsCredentialsKeys

↕ metadata only

Airbyte control plane

OrchestrationMonitoringUI

FAQ

GCS Data Lake connector questions

What data can I load into GCS Data Lake with Airbyte?

Airbyte moves data from any supported source into GCS Data Lake on a schedule you control, with full refresh and incremental sync where the destination supports it.

Does the GCS Data Lake connector support incremental sync?

Yes. The GCS Data Lake destination supports full refresh and incremental sync, so Airbyte can load only the records that changed since the last sync.

Where can I run the GCS Data Lake connector?

Run the GCS Data Lake connector on Cloud, OSS, Hybrid. Hybrid deployments keep the data plane in your environment while the control plane orchestrates syncs.

Didn't find your answer? Talk to sales