Load data into Databricks from 700+ sources
How it works
How do you load data into Databricks with Airbyte?
Four values from your workspace, one service principal, and an accepted driver license.
Create a service principal.
OAuth2 is the recommended authentication method. Create a service principal, generate a client ID and secret, and grant it access to your target catalog and schema. A personal access token also works.Configure the destination.
Add Databricks as a destination in Airbyte, supply the connection details, and accept the Databricks JDBC driver terms. Then select your streams, set each to Incremental Sync - Append + Deduped where the source exposes a primary key, and run the first sync.Sync pipeline
What happens to your grants when Airbyte syncs?
Not a philosophy. A set of specific, verifiable guarantees about the infrastructure your data runs on.
Why Airbyte
Staging stays in your catalog
Avro lands in Unity Catalog Volumes, then gets purged. Airbyte keeps no copy.
Staged inside the catalog.
The connector uses Unity Catalog Volumes to stage Avro files before loading them into tables.
Cleaned up after.
Purge Staging Files and Tables removes staged Avro from Volumes after loading, and is on by default.
Nothing retained by Airbyte.
Airbyte does not retain customer data on its servers, so no part of the path holds a copy you do not control.
Airbyte versus Databricks Lakeflow Connect
Databricks ships its own managed ingestion, and for a subset of sources it is a reasonable choice. The question worth answering is which subset.
| Method | Best for | Limits | Use Airbyte instead when |
|---|---|---|---|
| Lakeflow Connect | Sources on Databricks’ managed connector list | Bounded catalog, and it ties your ingestion layer to one lakehouse | Your source is not covered, or you load into more than one destination |
| Auto Loader (cloudFiles) | Incrementally ingesting files as they land in object storage | You own extraction and the landing process | Your data starts in an API or an operational database |
| COPY INTO | Idempotent batch loads of files already in storage | Same assumption. Files first | Getting the files is the actual work |
| Delta Live Tables | Declarative transformation pipelines | A transformation framework, not a source connector | You need the extraction layer beneath it |
| Partner Connect | Discovering and provisioning third-party tools | A directory, not an ingestion mechanism | Not applicable, this is how you would find Airbyte |
Where Airbyte wins
Breadth, and not being locked to one lakehouse. 700+ sources, an open-source connector framework you can extend, and the same tooling whether you load into Databricks, Snowflake, BigQuery, or all three.
Where it does not
If Databricks is your only destination and every source you need is on the Lakeflow Connect list, staying inside one vendor is simpler and there is no strong reason to add a second.
Sean Carver
Director of Data at PetDesk
"The real ROI is in our ability to iterate quickly, especially at our increasing scale. At the end of the day, you want a tool like that to just work. We can forget about it and know that it's configured and it's connecting and it's working. That hands-free capability is a big appeal for the platform.”
20+
data sources integrated and growing
+1
FTE engineer in productivity efficiency
85%+
reduction in data source integration time
Mondor La Grange
Head of BI and Data Engineering
"Unlike Fivetran's credit-based system that created budget uncertainty, Airbyte's pricing model allows Kuda to forecast expenses accurately and avoid surprise bills."
20+
data sources integrated and growing
+1
FTE engineer in productivity efficiency
85%+
reduction in data source integration time
Amy Zhao
Senior Manager of Data Engineering
"What's different from Stitch Data or Informatica is the way that we can configure Airbyte connections and Airbyte entities through code. That's a huge plus to us as data engineers, because we are used to checking code and being able to manage changes from Github."
3 to 1
reduction in data integrations solutions for reduced TCO
1
week Shopify and Stripe integration with Airbyte
Franziska Ibscher
Product Manager at Drivepoint
"Airbyte allows us to stay flexible while scaling from hundred-million to billion-dollar enterprise clients."
75%
of customers increased profitability
6.7%
EBITDA increase for customers
Which sources can you load into Databricks?
All 700+ of them. These are the ones Databricks users start with.
Databases
SaaS applications
How does your data land in Databricks?
Each stream is written directly to a final table in your configured schema. Every table carries four metadata columns:
_airbyte_raw_id, _airbyte_extracted_at, _airbyte_meta, and _airbyte_generation_id.| Airbyte type | Databricks type |
|---|---|
| string | STRING |
| integer | LONG |
| number | DECIMAL(38, 10) |
| boolean | BOOLEAN |
| date | DATE |
| timestamp_with_timezone | TIMESTAMP, microsecond precision |
| timestamp_without_timezone | TIMESTAMP_NTZ, microsecond precision |
| time_with_timezone | STRING, no native equivalent |
| object | STRING, serialized as JSON |
| array | STRING, serialized as JSON |
Naming conventions
Schema and table names.
Lowercased automatically. Databricks treats them as case-insensitive identifiers.
Column names.
Casing from your source data is preserved.
Special characters.
Escaped automatically by the connector.
The numbers
Teams running Airbyte in production
0M
pipelines synced daily
18K+
companies
199%
ROI, certified by Forrester
Deployment
Where Airbyte runs
Your warehouse credentials and your data do not have to leave your environment. Airbyte does not retain customer data on its servers.
Airbyte Cloud
Fully managed hosting on the Standard and Pro plans. Fastest path to a running pipeline.
Airbyte Flex
Hybrid deployment, with Airbyte’s data plane running in your cloud, VPC, or on-prem environment. You get 700+ connectors and a fully managed experience, while your data never touches our servers.
Self-Managed
Open source, your infrastructure, your rules.
compliance
Enterprise ready
Compliant with standards
SOC 2 Type II certified, GDPR and HIPAA support, with tools to help you meet internal and external regulatory requirements.
Uptime & SLA Guarantees
24/7 support and 99.9% availability backed by contractual SLAs and priority response times for mission-critical workloads.
Flexible deployment options
Move data into your warehouse/lakehouse of choice, VPC, on-prem, or into Airbyte's Context Store.
'%3e%3ccircle%20cx='422'%20cy='422'%20r='222'%20fill='url(%23paint0_linear_0_4)'/%3e%3c/g%3e%3cdefs%3e%3cfilter%20id='filter0_f_0_4'%20x='0'%20y='0'%20width='844'%20height='844'%20filterUnits='userSpaceOnUse'%20color-interpolation-filters='sRGB'%3e%3cfeFlood%20flood-opacity='0'%20result='BackgroundImageFix'/%3e%3cfeBlend%20mode='normal'%20in='SourceGraphic'%20in2='BackgroundImageFix'%20result='shape'/%3e%3cfeGaussianBlur%20stdDeviation='100'%20result='effect1_foregroundBlur_0_4'/%3e%3c/filter%3e%3clinearGradient%20id='paint0_linear_0_4'%20x1='644'%20y1='477.911'%20x2='200.016'%20y2='475.206'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20offset='0.00480771'%20stop-color='%235D51D5'/%3e%3cstop%20offset='1'%20stop-color='%23D763EC'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e)
'%20stroke='url(%23paint0_linear_0_33)'%20stroke-width='3.99691'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_0_33'%20x1='117.909'%20y1='59.9537'%20x2='1.99846'%20y2='59.9537'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%235D51D5'/%3e%3cstop%20offset='1'%20stop-color='%23CECBF2'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e)
'%20stroke-width='1.35684'%20stroke-miterlimit='1.5'%20stroke-linejoin='round'/%3e%3c/g%3e%3c/svg%3e)
Frequently asked questions
Didn’t find your answer?
Please don’t hesitate to reach out.
What is the best way to load data into Databricks?
Why not just use Lakeflow Connect?
Does Airbyte need an S3 or ADLS bucket for Databricks?
Do I need Unity Catalog?
How does Airbyte authenticate to Databricks?
How do objects and arrays land in Databricks?
Can I run this inside my own cloud?
Other destinations
Snowflake
Airbyte clones tables instead of swapping them, so the grants you configured survive every sync. Removed source columns are kept, and oversized values are logged rather than fatal.
BigQuery
Airbyte stages files in Google Cloud Storage, then loads them into BigQuery with native load jobs. Warehouse compute stays on Google Cloud, and Airbyte never sits in the query path.
ClickHouse
Airbyte writes to ClickHouse over its native binary protocol with batch inserts. The setup needs a hostname, a port, and a user. No object storage, no staging permissions, no intermediate hop.