Oracle Database to Elasticsearch: How to Index Your Data

Indexing Oracle Database into Elasticsearch starts with two questions that have nothing to do with search, and both can stop the project before it begins. Which Oracle connector do you have access to, and which Airbyte plan are you on.

There are two Oracle sources. The standard one replicates using a cursor, which works and does not capture deletes. The Enterprise Oracle source adds log-based change data capture through LogMiner, with real prerequisites attached. Meanwhile the Elasticsearch destination is available on Airbyte Core and PyAirbyte and not on the Standard, Plus, Pro or Enterprise Flex plans, so this pipeline assumes a self-managed deployment. Establish both facts before you write a design document around it.

The 30 character rule will surprise you

LogMiner ignores tables whose names exceed 30 characters, and it ignores columns whose names exceed 30 characters. Not warns about, not errors on. Ignores.

This catches modern schemas specifically. Oracle capped identifiers at 30 characters for most of its history, so an old schema will be under the limit everywhere by construction. Anything built on 12.2 or later can use longer names, and descriptive naming conventions produce them easily. A table called CUSTOMER_SUBSCRIPTION_RENEWALS is 30 characters exactly. Add one word and CDC stops seeing it. Audit your identifier lengths before the first sync rather than wondering later why one table never updates.

Datatypes have a similar shape of problem. LogMiner does not support every Oracle type, and user-defined types such as a VARRAY are outside what CDC can replicate. If your schema leans on object types, check them against the connector's supported list before committing to a CDC design.

CDC has a setup cost worth pricing in

Enterprise CDC needs supplemental logging enabled and several instance-level settings, plus a grant list longer than most connectors ask for: flashback access, select on any table, the catalog roles, select on any transaction, log mining, and rights to create tables and sequences and to lock tables. On a multitenant container database each of those grants needs the container clause, and an additional privilege applies. The exact set varies by Oracle version.

In most organisations that is a conversation with a DBA rather than a checkbox, and it is worth having before you promise anyone a delivery date. CDC is also bounded by your log retention period in the same way every log-based approach is: if the logs covering Airbyte's position are gone, it cannot resume from them. Encryption is the easier decision. The connector offers native network encryption, TLS with certificate verification requiring a PEM file, and unencrypted, which is available only on self-managed Airbyte. Since this pipeline is self-managed anyway, choose one of the encrypted options rather than the one that is merely available.

What relational rows become in an index

Elasticsearch indexes documents, not rows, and an Oracle table row is a fairly poor document on its own. The thing your users want to search is usually spread across a parent and three child tables, and Elasticsearch does not join.

So decide where the denormalisation happens. Doing it in Oracle with a view that Airbyte reads as a single stream is usually cleanest, because the database is good at joins and you get one document per business entity rather than one per row. The alternative, indexing tables separately and stitching results in the application, works but pushes complexity to every consumer. Whichever you choose, define the index mapping deliberately: decide which fields are analysed for full-text search and which are keyword fields for filtering and aggregation, before you index millions of documents under a mapping you then have to rebuild.

Frequently asked questions

Why is one of my Oracle tables not updating under CDC?

Check the identifier length. LogMiner ignores tables and columns with names longer than 30 characters, without raising an error.

Does the standard Oracle source support CDC?

No. Log-based change data capture is on the Enterprise Oracle source. The standard connector replicates using a cursor and cannot capture deletes.

Should I denormalise in Oracle or in Elasticsearch?

Usually in Oracle, using a view that Airbyte reads as one stream. Elasticsearch does not join, and a database is better at that work than your application layer.

Can I run this on Airbyte Cloud?

Not on the paid tiers. The Elasticsearch destination is offered on Airbyte Core and PyAirbyte, so this pipeline needs a self-managed deployment.

Get your Oracle data into Elasticsearch

Check the plan, check the connector, then check your identifier lengths, in that order. After that the work is modelling: build the document you want to search inside Oracle, define the mapping before the first sync, and treat the index as something you designed rather than something that appeared.

For the same destination fed by a simpler relational source, see PostgreSQL to Elasticsearch. For log-based replication where the setup cost is lower, PostgreSQL to Databricks is a useful comparison.

Start syncing now →