Oracle Database to Databricks: How to Move Your Data

Moving Oracle Database into Databricks is how organisations get decades of operational data out of a licensed system and into somewhere data scientists can reach it. The usual alternative is a nightly export to flat files, which runs whether anything changed or not and tells you nothing about what was deleted.

This guide covers the managed path with Airbyte. The decision that shapes everything is which of the two Oracle connectors you use, because only one of them can tell you a row was deleted, and it asks considerably more of your DBA.

Oracle Database to Databricks at a glance:

Why move data from Oracle Database to Databricks?

Two situations account for most of these pipelines.

The first is access and cost. Oracle licensing makes it expensive to give every analyst a connection, and plenty of teams are not permitted near it at all. A lakehouse copy gives them the data without adding load or licence cost to the system of record, which is frequently the entire business case.

The second is machine learning. Oracle instances hold the oldest and most authoritative data in the business, and the models people want to build on it need notebooks, Spark and feature engineering rather than a reporting connection. That combination is what a lakehouse offers over a warehouse.

What do you need before you start?

Two of these involve other people, so start them before the work you can do yourself:

A decision about deletes. The standard Oracle source replicates using a cursor and cannot observe a deletion, so a row deleted in Oracle stays in Databricks indefinitely with nothing reporting the discrepancy. Log-based change data capture is on the Enterprise Oracle source.

DBA time, if you want CDC. Enterprise CDC needs supplemental logging plus instance-level settings, and a grant list covering flashback access, select on any table, the catalog roles, select on any transaction, log mining, and rights to create tables and sequences and lock tables. On a multitenant container database each grant needs the container clause.

An identifier length audit. LogMiner ignores tables and columns whose names exceed 30 characters, silently. Run this check before the first sync rather than after a table has quietly stopped updating.

Unity Catalog access. Each stream becomes a table in Unity Catalog and the connector needs permission to create Volumes for Avro staging. Agree the catalog and schema before the first sync, particularly if this data has access restrictions of its own.

If your organisation restricts inbound traffic by IP, add the Airbyte Cloud IP addresses to the allow list on the Oracle side before you begin.

How do you build an Oracle to Databricks pipeline in Airbyte?

Step 1: Prepare Oracle and audit your identifiers

Create a read-only Oracle user, apply the CDC grants if you are using Enterprise, and check identifier lengths. Oracle capped identifiers at 30 characters for most of its history, so genuinely old schemas are safe by construction, while anything built on 12.2 or later can exceed it easily with a descriptive naming convention.

Step 2: Configure the Oracle source

Click Sources in the left navigation, then New Source, and select the Oracle connector you decided on, following adding a source. Supply the host, port, service name or SID, schema and credentials, and pick an encryption method. Native network encryption and TLS with certificate verification are both available, and unencrypted is offered only on self-managed Airbyte.

Step 3: Configure the Databricks destination

Click Destinations, then New Destination, and select Databricks, following adding a destination. Supply your workspace details, the catalog and schema, and authentication. Unlike a relational destination there is no volume ceiling to design around, which matters because Oracle instances are frequently the largest thing in the estate.

Step 4: Create the connection and verify a subset

Click Connections, then New connection, select tables and a sync mode per stream. With CDC, deleted rows arrive flagged rather than vanishing, so append and deduped mirrors current Oracle state while plain append preserves every version. Run against a handful of tables first and compare row counts and numeric values against Oracle before extending.

Oracle instances tend to be the system of record for figures somebody reconciles, so that verification step is worth more here than on a source nobody audits.

Why would CDC silently skip a table?

LogMiner ignores tables whose names exceed 30 characters, and columns whose names exceed 30 characters. It does not warn and it does not error. The table sits in your connection looking configured and never receives an update.

A table called CUSTOMER_SUBSCRIPTION_RENEWALS is exactly 30 characters. One more word and CDC stops seeing it, which is why the audit belongs before the first sync rather than in a debugging session six months later.

Datatypes have the same shape of problem. LogMiner does not support every Oracle type, and user-defined types such as a VARRAY cannot be replicated through CDC at all. If your schema leans on object types, check them before committing to a log-based design.

How should you model the result?

Land the tables as a bronze layer and build your modelled tables on top, which on this source has a specific benefit: Oracle numeric types carry more precision than most destinations preserve by default, so anything financial deserves an explicit cast in the silver layer rather than acceptance of whatever was inferred.

Oracle's DATE type is another one to handle deliberately, because it carries a time component despite the name, which surprises anyone treating it as a plain date. Decide once in your models whether that time matters and apply the decision consistently.

If you took the CDC route with plain append, you have every version of every row, which is a genuine asset for feature engineering and a genuine cost in storage. Decide which tables warrant that history rather than applying one sync mode across the whole schema.

Frequently asked questions

Does the standard Oracle source support CDC?

No. Log-based change data capture is on the Enterprise Oracle source. The standard connector uses a cursor and cannot capture deletes.

Why is one of my Oracle tables never updating?

Check the identifier length. LogMiner ignores tables and columns whose names exceed 30 characters, without raising an error.

Is there a volume limit on this destination?

Not in practice. A lakehouse scales with the source, which is why it suits Oracle instances that are usually the largest system in the estate.

Why do my financial figures look slightly wrong?

Check numeric precision. An Oracle NUMBER declared without precision holds a wider range than many destination types, so cast financial columns explicitly rather than trusting the inferred type.

Can I do this without writing code?

The Airbyte setup is entirely UI-driven. The Oracle side needs a DBA to apply supplemental logging and the CDC grants if you choose Enterprise, which is configuration rather than code.

Get your Oracle data into Databricks

Decide whether you need deletes, because that chooses your connector and brings a DBA into the project. Audit identifier lengths before the first sync, verify numeric precision on anything financial, and decide per table whether you want current state or every version.

Airbyte's connector catalog includes 600+ pre-built connectors, so Oracle data can sit alongside the other sources your models need. For the warehouse version, see Oracle Database to BigQuery, and for another enterprise source in a lakehouse, IBM Db2 to Databricks.

Start syncing now →