Salesforce to Elasticsearch: How to Move Your Data
Index Salesforce into Elasticsearch with Airbyte. Plan availability limits, designing the index mapping, and why a partial sync looks like a failing search.
Indexing Salesforce into Elasticsearch is worth doing when the question people keep asking is not how many but where else have I seen this. Salesforce search is adequate for finding a record you already know exists and poor at fuzzy matching across free-text fields, which is exactly what support agents and sales operations do all day.
Before any of that, one constraint decides whether this pipeline is available to you at all. The Elasticsearch destination runs on Airbyte Core and PyAirbyte, and it is not offered on the Standard, Plus, Pro or Enterprise Flex plans. If you are on a paid Cloud tier the connector will not appear in your catalogue, and everything below assumes a self-managed deployment.
Salesforce to Elasticsearch at a glance:
Why move data from Salesforce to Elasticsearch?
Two situations account for most of these pipelines, and neither is analytics.
The first is operational search. An agent has a fragment of a company name, half an address or a phrase from a case description, and needs the record now. Relevance ranking, typo tolerance and matching across several fields at once are what a search engine does and what a CRM does not.
The second is search inside your own product or internal tools. If you have built something that needs to look up accounts or contacts, pointing it at an index rather than at the Salesforce API keeps your interface fast and stops every search consuming API budget. If you want to count and aggregate CRM data instead, a warehouse is the right destination.
What do you need before you start?
The first item can rule the pipeline out, so confirm it before designing anything:
A self-managed Airbyte deployment. The Elasticsearch destination is available on Airbyte Core and PyAirbyte, and not on the paid Cloud tiers. There is no configuration that works around this.
A dedicated Salesforce integration user. The connector sees exactly what the authenticating user sees, so field-level security and sharing rules on that account decide what reaches your index. On a search pipeline that is a security boundary rather than a technicality, because whatever lands in Elasticsearch is searchable by whoever can query it.
A shortlist of objects and fields. Decide which objects people actually search and which fields matter for matching. Accounts, contacts, leads and cases cover most operational search. Syncing everything wastes API budget and inflates an index nobody queries.
An index mapping you have designed. Decide which fields are analysed for full-text search and which are keyword fields for filtering and aggregation, before you index anything. This is the decision that determines whether search works, and correcting it later means reindexing.
Agree a share of the org's daily API budget too, since every connected application draws from the same pool and exhausting it affects other people's integrations rather than only yours.
How do you build a Salesforce to Elasticsearch pipeline in Airbyte?
Step 1: Design the index mapping
Work out what a good search result looks like, then map fields accordingly. A company name and a case description should be analysed text so that partial and fuzzy matches work. A record type, a stage or an owner ID should be a keyword field so it can filter exactly. Get these the wrong way round and your filters return nothing while your searches match everything.
Step 2: Configure the Salesforce source
Click Sources in the left navigation, then New Source, and select Salesforce, following adding a source. Authenticate as your integration user and set a start date. Set the lookback window deliberately as well, because Salesforce's API is eventually consistent and a record modified shortly before a sync may not be visible to it. The default is ten minutes.
Step 3: Configure the Elasticsearch destination
Click Destinations, then New Destination, and select Elasticsearch, following adding a destination. Supply your endpoint and authentication. Create the index with the mapping from step 1 first, so the connector writes into a schema you designed rather than one dynamic mapping invented from the first document it saw.
Step 4: Use incremental append and deduped
Click Connections, then New connection, select your objects, and choose incremental append and deduped as the sync mode. On any org of size this is not really a choice, for the reason in the next section. Deduplication also makes the lookback window free, since re-reading an overlapping period does not leave duplicate documents in the index.
Set the frequency against how quickly users expect a new record to become findable. For support search that is usually minutes; for an account directory it can be hours. Both are cheaper than the alternative of every search hitting the Salesforce API.
Why does a successful sync sometimes index fewer records?
When the connector reaches your org's daily API limit it does not fail. It stops where it is, reports the run as successful, and picks up from that position on the next run. That is sensible, because failing the whole sync would discard work already done.
The self-correction only works on incremental sync. Configure full refresh against a large org and every run truncates somewhere arbitrary, then starts from the beginning again next time, never reaching the end. That is why the sync mode above is stated as a requirement.
On a search index the consequence is specific and worth stating. A partially synced index does not look broken, it looks like a search that returns nothing for a customer who definitely exists. Users blame the search rather than the pipeline, so monitor document counts as well as sync status.
What changes in Salesforce without reaching the index?
Formula fields are the important case. The connector syncs the output of a formula rather than the formula itself, and editing that formula does not change any record's modification timestamp. Incremental sync therefore never re-reads those records, so the index keeps serving values calculated under the old formula indefinitely.
The fix is to reset the stream and run a historical backfill. The prevention is procedural: whoever administers Salesforce needs to know that changing a formula means telling the data team, in the same way a schema change would.
Deletions deserve a thought too. Records deleted in Salesforce can be retrieved from the recycle bin on streams that support it, arriving marked as deleted rather than vanishing. On a search index you want those documents gone rather than flagged, so filter on that marker when querying, or handle removal in whatever layer sits between your application and the index.
Frequently asked questions
Why can I not find the Elasticsearch destination in Airbyte Cloud?
It is available on Airbyte Core and PyAirbyte, and not on the Standard, Plus, Pro or Enterprise Flex plans. This pipeline needs a self-managed deployment.
Why did my sync succeed but index fewer records?
It hit your org's daily API limit. The connector stops there, reports success, and resumes next run. On incremental sync this is self-correcting, but monitor document counts because a partial index looks like a failing search.
Do I need to define an index mapping first?
Strongly recommended. The analysed text versus keyword decision determines whether search and filtering work, and changing it later means reindexing everything already written.
Why are some Salesforce fields missing from the index?
The connector sees what the authenticating user sees, so field-level security and sharing rules on your integration user decide what reaches Elasticsearch.
Can I do this without writing code?
The Airbyte setup is entirely UI-driven. You will want to define the index mapping in Elasticsearch, which is short JSON and the most important part of the build.
Get your Salesforce data into Elasticsearch
Check your Airbyte plan first, because that decides everything else. Then design the mapping before indexing anything, use a scoped integration user knowing that its visibility becomes your index's contents, run incremental append and deduped so partial syncs resume, and monitor document counts rather than only sync status.
Airbyte's connector catalog includes 600+ pre-built connectors, so the same Salesforce source can feed a warehouse alongside the index. For that pairing, see Salesforce to Snowflake, and for the same destination fed by a relational source, MySQL to Elasticsearch.
Integrate with 600+ apps using Airbyte
Move data from 600+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.