MongoDB to Elasticsearch: How to Move Your Data

Moving MongoDB into Elasticsearch gives an application proper search over data it already stores. MongoDB retrieves documents by key extremely well and is not a search engine, so anything involving relevance, partial matches or typo tolerance ends up as a regular expression somebody regrets writing.

This guide covers the managed path with Airbyte. Two things shape the build, and the first is a hard availability constraint: this destination is offered on Airbyte Core and PyAirbyte only. After that, the index mapping you design before the first sync decides what search can do.

MongoDB to Elasticsearch at a glance:

Why move data from MongoDB to Elasticsearch?

Two situations account for most of these pipelines.

The first is user-facing search. A product catalogue, a help centre, a directory of listings: people type approximate things and expect sensible results, which needs analysis, scoring and tolerance for imperfect input. MongoDB was not built for that and Elasticsearch was, so the sensible architecture keeps the records in one and the search index in the other.

The second is operational exploration, where engineers need to search across documents by content rather than fetch them by identifier. The honest limit is availability before anything technical: this destination exists on Airbyte Core and PyAirbyte only, so on Standard, Plus, Pro or Enterprise Flex it is not an option. If aggregation rather than search is what you actually need, MongoDB to ClickHouse is a better fit anyway.

What do you need before you start?

Four things, and the first will decide whether you can proceed at all:

An Airbyte Core or PyAirbyte deployment. This destination is not offered on Standard, Plus, Pro or Enterprise Flex, which is a planning question rather than a configuration one. The Elasticsearch destination documentation sets out the support position.

A MongoDB replica set and a read user. Change capture works through change streams, which exist only on a replica set. That also gives you deletes, which matters more for a search index than for a warehouse, since a result linking to a document that no longer exists is a visible bug.

A schema mode decision. Schema-enforced produces discrete fields from sampled documents. Schemaless hands the document across whole, which leaves you a single opaque value to index and very little to search against meaningfully.

An index mapping, designed rather than inherited. Which fields are analysed text, which are keywords, and which need to be both. This is the decision that defines what your search can do, and changing it later means reindexing.

If your cluster restricts inbound traffic by IP, add the Airbyte Cloud IP addresses to the allow list before you begin.

How do you build a MongoDB to Elasticsearch pipeline in Airbyte?

Step 1: Design the index mapping from the searches you intend to run

Work through the queries your application will make and classify every field accordingly. Anything people type words into is analysed text. Anything used for filtering, sorting or exact matching is a keyword. Values needing both, and there are usually a few, get a multi-field mapping so they exist as each. Create the mapping in Elasticsearch before the first sync, because letting the index infer its own is how you end up reindexing a month later.

Step 2: Configure the MongoDB source

Click Sources in the left navigation, then New Source, and select MongoDB, following adding a source. Supply the connection string, database and credentials, and choose schema-enforced mode. Review what the sampling inferred, paying attention to fields that appear on only some documents, since those are the ones most likely to be missing from the schema and therefore from your index.

Step 3: Configure the Elasticsearch destination

Click Destinations, then New Destination, and select Elasticsearch, following adding a destination. Supply the endpoint and authentication. Point it at indexes whose mapping you have already created, rather than letting new indexes appear with whatever types Elasticsearch guesses from the first documents it happens to see.

Step 4: Create the connection and use change capture

Click Connections, then New connection, select your collections and a sync mode. Change capture is the right choice because deletes reach the index, and an index holding records the application has removed produces search results that go nowhere. Change streams read from the oplog, so a pipeline paused longer than your oplog window needs a fresh snapshot.

Test relevance with real queries rather than assuming it works. Search that returns results is easy; search that returns the right results in the right order is the thing you are actually building.

Why does schema mode matter so much here?

Because Elasticsearch indexes fields, and schemaless mode does not give it any. Passing documents across whole preserves them perfectly and leaves the destination with one value to work with, which means no per-field analysis, no keyword filtering and no sensible relevance scoring. It is faithful and useless.

Schema-enforced mode samples documents and produces discrete fields, which is what your mapping applies to. The caveat is the same as anywhere sampling is involved: a field present on a small fraction of documents may not make it into the schema, and if that field is one somebody searches on, the search quietly returns nothing for it.

So check the inferred schema against the fields your search depends on, specifically. Ask the application team which fields are optional and which were added recently, and confirm those appear. This is a five-minute conversation that prevents the situation where search works well for most queries and inexplicably fails for one category of content.

What does the analysed and keyword distinction actually change?

Whether a field can be searched by words or matched exactly, and a field can behave as only one of those unless you plan for both. Analysed text is broken into terms, so a product description matches on individual words regardless of order. A keyword is stored whole, so it matches only in its entirety, which is what you want for a category, a status or an identifier.

Getting it the wrong way round produces behaviour that looks like a bug and is actually a mapping. A status field mapped as analysed text will match partially and return records nobody asked for. A description mapped as a keyword will match nothing unless somebody types it perfectly, which they will not. Both are quiet failures: results appear, they are simply wrong.

Multi-field mapping resolves the genuine cases where a value needs both. An email address is the standard example: searchable by the part before the domain, and also matchable exactly when somebody pastes the whole thing. Product names and tags often want the same treatment. Decide these during the mapping design, since adding them afterwards means reindexing everything, and reindexing a large collection is an operation rather than a tweak.

Frequently asked questions

Can I use this destination on any Airbyte plan?

No. Elasticsearch is available on Airbyte Core and PyAirbyte only, and not on Standard, Plus, Pro or Enterprise Flex, so confirm your deployment before planning around it.

Do I need a replica set?

For change capture, yes, since change streams are a replica set feature. It is worth having because deletes then reach the index rather than leaving results pointing at documents that no longer exist.

Why does searching one field return nothing?

Either it was mapped as a keyword when it should be analysed text, or it never made it into the schema because sampling did not see it often enough. Check the mapping first, then the inferred schema.

Can a field be both searchable and exactly matchable?

Yes, with a multi-field mapping, which indexes the value both ways. Decide this up front because adding it later requires reindexing.

Can I do this without writing code?

The pipeline, yes. The index mapping is a piece of configuration you write in Elasticsearch, and it is the part that determines whether the search is any good.

Get your MongoDB data into Elasticsearch

Confirm your deployment supports this destination before anything else, since Core and PyAirbyte are the only options. Choose schema-enforced mode, because a single opaque document gives Elasticsearch nothing to index. Design the mapping from the searches you intend to run, use multi-field where a value must be both searchable and exact, and use change capture so deletions reach the index rather than leaving results pointing nowhere.

Airbyte's connector catalog includes 600+ pre-built connectors, so application data can power search without a bespoke indexer. For the same source into a vector store, see MongoDB to Weaviate, and for a relational source into the same destination, PostgreSQL to Elasticsearch.

Start syncing now →