Connect Databricks Lakehouse data with any data source.

Send data from every Airbyte source to Databricks Lakehouse.

Load data into Databricks Lakehouse Read the docs

Sources: 590+
Certified support: 4.0.5

Overview

What is the Airbyte Databricks Lakehouse connector?

The Airbyte Databricks Lakehouse connector loads data from 590+ Airbyte sources into Databricks Lakehouse, with full refresh and incremental sync on a schedule you control.

01
Load from any source
Move data into Databricks Lakehouse from 590+ Airbyte sources on a schedule you control.
02
Incremental syncs
Pull only the records that changed since the last run instead of reloading everything.
03
Cloud or self-hosted
Run the Databricks Lakehouse connector on Cloud, OSS, Hybrid.

Setup

Live in eleven steps

First, navigate to the Airbyte website and log in to your account.
Once you are logged in, click on the "Destinations" tab on the left-hand side of the screen.
Scroll down until you find the "Databricks Lakehouse" connector and click on it.
You will be prompted to enter your Databricks Lakehouse credentials, including your account name, personal access token, and workspace ID.
Once you have entered your credentials, click on the "Test" button to ensure that the connection is successful.
If the test is successful, click on the "Save" button to save your Databricks Lakehouse destination connector settings.
You can now use the Databricks Lakehouse connector to transfer data from your source connectors to your Databricks Lakehouse destination.
To set up a data transfer, navigate to the "Sources" tab and select the source connector that you want to use.
Follow the prompts to enter your source connector credentials and configure your data transfer settings.
Once you have configured your source connector, select the Databricks Lakehouse connector as your destination and follow the prompts to configure your data transfer settings.
Click on the "Run" button to initiate the data transfer.

Configuration fields

Databricks Lakehouse configuration fields
Field	Type	Description
accept_terms*	boolean	You must agree to the Databricks JDBC Driver Terms & Conditions to use this connector.
authentication*	object	Determines the type of authentication used to connect to Databricks. OAuth2 is recommended for production use.
database*	string	Enter the name of the Unity Catalog that you want to sync data into.
hostname*	string	Enter your Databricks workspace server hostname (e.g., abc-12345678-wxyz.cloud.databricks.com). You can find this in your cluster or SQL warehouse connection details.
http_path*	string	Enter the HTTP path for your Databricks SQL warehouse or cluster. You can find this in the connection details of your compute resource.
cdc_deletion_mode	string	Whether to execute CDC deletions as hard deletes (i.e. propagate source deletions to the destination), or soft deletes (i.e. leave a tombstone record in the destination). Defaults to hard deletes.
port	string	Enter the port number for the Databricks cluster connection. The default port is 443 (HTTPS).
purge_staging_data	boolean	Whether to delete staging files and tables after data has been loaded. Disable this option for debugging purposes.

Hybrid deployment

Sovereign by architecture, not by policy.

The data plane runs in your environment. Records, credentials and keys stay with you; the control plane only receives configuration and job metadata.

Cloud
OSS
Hybrid

Your environment

DataConnectorsCredentialsKeys

↕ metadata only

Airbyte control plane

OrchestrationMonitoringUI

FAQ

Databricks Lakehouse connector questions

What data can I load into Databricks Lakehouse with Airbyte?

Airbyte moves data from any supported source into Databricks Lakehouse on a schedule you control, with full refresh and incremental sync where the destination supports it.

Does the Databricks Lakehouse connector support incremental sync?

Yes. The Databricks Lakehouse destination supports full refresh and incremental sync, so Airbyte can load only the records that changed since the last sync.

Where can I run the Databricks Lakehouse connector?

Run the Databricks Lakehouse connector on Cloud, OSS, Hybrid. Hybrid deployments keep the data plane in your environment while the control plane orchestrates syncs.

Didn't find your answer? Talk to sales