Agentic Data Engineering Resources

Resource

What Is Hybrid Search, and How Is It Used?

Learn how hybrid search combines BM25 and vector retrieval, when it improves relevance, and when fusion can reduce retrieval quality.

Pedro Lopez

June 24, 2026

Last updated September 3, 2026

Summarize with AI:

Hybrid search runs keyword-based retrieval and vector-based semantic retrieval in parallel against the same corpus, then merges both result sets into a single ranked list through a fusion algorithm. Its strength is broader coverage: BM25 handles exact-match queries like product codes, dates, and proper nouns, while dense retrieval captures paraphrased or conceptually related matches.

That broader coverage matters only when both retrieval signals add useful information. Hybrid search is not automatically better. Fusion can reduce retrieval quality when one retriever adds little signal or performs substantially worse than the other, especially on short RAG chunks. Teams should therefore evaluate hybrid search on their own corpus and query distribution rather than treating it as a default.

TL;DR

Hybrid search combines BM25 keyword retrieval and dense vector retrieval, then merges both result sets with a fusion algorithm.
It improves coverage by handling both exact-match queries like product codes, dates, and proper nouns and semantic queries like paraphrased questions.
The main implementation choice is between RRF, which avoids score normalization, and convex combination, which can outperform RRF when tuned.
Teams should test hybrid search on their own corpus, especially for short Retrieval-Augmented Generation (RAG) chunks and cases where one retrieval method performs substantially worse than the other.

Why Does Combining Keyword and Vector Search Improve Results?

Combining keyword and vector search improves results because each method covers the other's blind spots. A BM25 query for "make my database faster" may fail to retrieve strong matches if the relevant document says "query performance tuning," because there is zero token overlap. A vector search for "ERROR 1045" can return results about database errors generically. That can lead to confusion between similar MySQL errors in retrieval systems.

Query Characteristic	Keyword Search (BM25) Advantage	Vector Search (Dense Retrieval) Advantage
Product codes, SKUs, IDs	Exact-match precision on alphanumeric strings	Misses non-semantic tokens entirely
Dates and proper nouns	Direct string matching	Loses specificity by embedding as generic concepts
Paraphrased questions ("make my database faster")	Fails if query terms differ from document terms	Captures semantic similarity across vocabulary gaps
Conceptual queries ("customer retention strategies")	Returns only documents containing those exact words	Surfaces related concepts like churn prevention and loyalty programs

A document that ranks highly in both lists is often relevant. A document that ranks highly in only one list may still surface, but with lower confidence.

This pattern shows up most clearly on out-of-domain datasets, where lexical and dense retrieval often fail in different ways. On BEIR, dense retrieval models fine-tuned on MS MARCO, the most common training set, underperform on datasets with a large domain shift. BM25 remains a competitive baseline on the same evaluation, while late-interaction or reranking models achieve the best average results.

That matters for enterprise search because enterprise corpora often look more like BEIR's specialized datasets than open-domain web QA.

How Does Hybrid Search Combine Keyword and Vector Retrieval?

Hybrid search combines two ranked candidate lists through either rank-based or score-based fusion. Each retrieval path returns a ranked list of candidates, and a fusion algorithm merges those lists into a single ordering. The core design problem is score incompatibility: the two systems produce scores on different scales.

BM25 drives sparse retrieval

BM25 is the standard keyword retrieval algorithm, used as the default in Elasticsearch, OpenSearch, and Lucene-based systems. It scores documents based on three signals: how often query terms appear in a document, with diminishing returns for repeated occurrences; how rare those terms are across the entire corpus, via inverse document frequency; and how long the document is relative to the average, through length normalization. The BM25 parameters k₁ and b control the saturation curve and length penalty respectively, with common defaults of k₁=1.2 and b=0.75.

The output is an unbounded positive score. A BM25 score of 15.0 on one query is not comparable to 15.0 on a different query, because scores are meaningful only relative to other scores in the same result set.

Dense retrieval uses vector embeddings

Dense retrieval encodes both queries and documents as vector embeddings, then ranks documents by similarity in that vector space, often using metrics such as cosine similarity or dot product. The embedding model compresses semantic meaning into a fixed-dimensional vector, so documents with similar meaning cluster together even when they share no vocabulary.

Cosine similarity produces scores bounded between -1 and 1 (or 0 and 1 for non-negative embeddings). Similarity scores are typically used to rank results for a given query, and their practical interpretation should be validated on your own data.

Score incompatibility shapes fusion design

Fusion design follows from that score mismatch. BM25 produces unbounded, corpus-dependent scores, while cosine similarity produces bounded scores in a fixed range. Without explicit weighting, the sparse component dominates the combined score because BM25/sparse scores are unbounded, while dense similarity scores are on a different scale. A static weight does not solve the problem because BM25 score ranges vary by query.

Most teams therefore choose between two fusion approaches. Rank-based fusion, Reciprocal Rank Fusion (RRF), discards raw scores and uses only rank positions. Score-based fusion, convex combination, normalizes both score sets to a common range before combining them with a weighted sum.

How Do Fusion Algorithms Merge Two Result Sets?

Fusion algorithms merge two result sets either by combining rank positions or by combining normalized scores.

Characteristic	Reciprocal Rank Fusion (RRF)	Convex Combination (Weighted Sum)
Input type	Rank positions only	Normalized scores
Requires score normalization	No (rank-based)	Yes (BM25 scores are unbounded; vector scores typically 0-1)
Tuning parameters	k constant (k=60 is a common default)	Alpha weight (0 = pure keyword, 1 = pure vector)
Common perception	"No tuning required"	"Requires tuning alpha per corpus"
Empirical finding (arXiv:2210.11934)	RRF is parameter-sensitive	Convex combination can outperform RRF when tuned; outcomes depend on dataset and evaluation setting
Production adoption	Common default in several major systems; other platforms use weighted score blending	Less common as default; stronger for teams that can run evaluation
Caveat	Can decrease retrieval quality vs. pure semantic search on specific datasets	Requires normalization (e.g., Min-Max scaling) before scores can be combined

Reciprocal Rank Fusion uses rank positions

RRF, introduced by Cormack, Clarke, and Büttcher at SIGIR 2009, scores each document by summing the reciprocal of its rank position across all result lists:

RRFscore(d ∈ D) = Σ_{r∈R} 1 / (k + r(d))

As an illustration, with k=60, the score difference between rank 1 and rank 2 is 1.6%. With k=2, that difference is 25%.

RRF sidesteps the score incompatibility problem entirely. Its disadvantage is that it throws away magnitude information: a document with a BM25 score of 100.0 receives identical treatment to one scoring 0.001, if both occupy the same rank position.

The original SIGIR work characterized k=60 as near-optimal but not critical. More recent work shows that RRF sensitivity can be high as a function of RRF parameters, which contradicts the "parameter-free" characterization common in vendor documentation.

Convex combination uses normalized scores and alpha tuning

Convex combination multiplies each system's normalized scores by a weight, alpha, and sums them:

score_final = α × score_dense_normalized + (1 − α) × score_BM25_normalized

Alpha ranges from 0, pure keyword, to 1, pure vector. Min-max normalization, which maps each result set's scores to [0, 1], is a common preprocessing step for score fusion.

Convex combination can outperform RRF when tuned, and replication on production search stacks confirms that the optimal weight is model specific.

The main cost of convex combination is the need for labeled relevance judgments to tune alpha. Tuning typically converges with a relatively small set of annotated queries as a one-time investment. For teams with no labeled data, RRF is often the starting default.

Post-fusion reranking improves final ordering

Post-fusion reranking improves final ordering by re-scoring the fused candidate set with a cross-encoder. Unlike bi-encoders that encode queries and documents independently, cross-encoders process the query-document pair jointly through a transformer. This captures fine-grained token interactions. The tradeoff is latency: each candidate requires a separate forward pass.

On financial QA evaluations, Hybrid BM25 produces top results, Hybrid RRF improves Recall@5 over dense-only retrieval, and adding reranking on top of hybrid retrieval improves it further.

Candidate count and model size largely determine that tradeoff. Reducing the candidate set too aggressively can sharply reduce recall.

When Does Hybrid Search Underperform Single-Method Retrieval?

Hybrid search underperforms single-method retrieval when one retriever adds little signal or introduces noise.

Scenario	What Happens	Why It Fails
Short RAG chunks with low term frequency variation	BM25's term-frequency saturation and document-length normalization break down on short text fragments	BM25 was designed for full documents, not the short chunks RAG preprocessing produces
High-quality domain-specific embedding model	Dense retriever already captures nearly all relevant documents; BM25 introduces noise from irrelevant exact-match hits	Sparse signal dilutes a strong dense signal
Queries on benchmark datasets like TREC-COVID	RRF can score lower than pure semantic search on some evaluation setups	Fusion pushes down strong semantic matches that rank poorly in keyword results

When one retrieval method performs substantially worse than the other on a given corpus, fusion can pull down the better-performing retriever.

BM25 weakens on short chunks for a specific reason. On fixed-size chunks of 256 to 512 tokens, every query term either appears once or not at all. That turns term frequency into binary presence or absence, and because the chunks are approximately the same length, length normalization contributes little.

At those chunk sizes, only IDF in the BM25 formula remains relevant for RAG. Teams should verify that BM25 adds signal at their actual chunk sizes before committing to hybrid retrieval.

How Is Hybrid Search Used in Retrieval-Augmented Generation (RAG) and AI Agent Workflows?

Hybrid search in RAG and agent workflows replaces a single retriever with parallel retrieval and fusion, then feeds the fused results into the next system step.

How Does Hybrid Search Change Retrieval in RAG Pipelines?

In a standard Retrieval-Augmented Generation (RAG) pipeline, hybrid search replaces the single-retriever step with a parallel retrieval and fusion stage. The fused results feed into the large language model (LLM)'s context window.

On financial QA evaluations, dense-only retrieval reaches 0.587 Recall@5, hybrid retrieval improves on that, and adding a cross-encoder reranker improves it again. Downstream RAG benchmarks show hybrid retrieval improving generation accuracy, with RetHyb-RRF reaching 62.9% accuracy on FinanceBench and showing gains on CovidQA. Retrieval quality directly affects generation quality.

Hybrid search serves as one agent tool among many

Within agentic RAG systems, hybrid search is one tool among several that the agent can invoke. The agent decides at each reasoning step whether a query is best served by hybrid search, for unstructured knowledge retrieval; structured SQL, for tabular lookups; graph traversal, for relationship queries; or direct application programming interface (API) calls, for live data. The agent's ability to select the right retrieval method per query type is what distinguishes agentic from static RAG. Teams working with the Context Store can treat retrieval as one step inside a broader agent workflow rather than the whole architecture.

Data freshness constrains operational retrieval

For agents working with operational business data, freshness limits retrieval in ways algorithm choice alone does not solve.

Agents reasoning over CRM deals, support tickets, or billing records need vectors searchable immediately after insertion, not after a batch reindexing cycle. That requirement applies to both retrieval paths, so the inverted index for BM25 and the vector index for dense retrieval both need current data. In the Context Store layer for AI agents, that means keeping data fresh, accurate, and permissioned so the agent can reason correctly at query time.

Retrieval also fails in other operational ways. Source auth can expire, permissions can be missing from the retrieval layer, live API calls can fail at runtime, and too many cross-system results can create token-window pressure during agent reasoning. Those failures are separate from retrieval relevance, but they still determine whether an agent can answer correctly.

What Should You Do Before You Deploy Hybrid Search for AI Agents?

Before you deploy hybrid search for AI agents, evaluate it against your actual corpus and query distribution rather than defaulting to it.

For agents working across multiple business systems, hybrid search within a single index only solves half the problem. The data must first be unified, kept fresh, and made permission-aware. Two execution patterns address the freshness spectrum: pre-indexed retrieval uses an indexed replica for speed, while just-in-time retrieval calls live source APIs when indexed data may be stale. This dual-mode architecture covers cases that hybrid search algorithms alone do not handle, especially when data lives in multiple systems and changes on different schedules.

Airbyte's agentic platform pre-materializes data from connected sources into the Context Store, a search-optimized index of business systems that agents query directly. This gives agents a live searchable index tied to fresh, accurate, and permission-aware business data rather than a stale standalone retrieval layer. Teams can use that context through the Web app, Agent MCP, Agent SDK, or API, so retrieval fits the interface their agent workflow already uses. In that setup, retrieval can be one step inside a broader agent workflow rather than the whole architecture.

Get a demo to see how Airbyte's agentic platform powers production AI agents with fresh, permission-aware data from every system your team depends on, or try Airbyte's agentic platform today.

Frequently Asked Questions

Does hybrid search always outperform pure vector search?

No. Results vary by dataset, and fusion can reduce retrieval quality instead of improving it in several common scenarios. Teams should evaluate hybrid retrieval on their own corpora with representative queries before committing to it as a default.

What is the difference between RRF and convex combination?

RRF merges result sets using rank positions and requires no score normalization, which makes it a practical starting point. Convex combination uses a weighted sum of normalized scores, controlled by an alpha parameter. Convex combination can outperform RRF when tuned, but it requires labeled relevance judgments.

How does BM25 behave on short RAG chunks?

On short chunks typical of RAG preprocessing (256 to 512 tokens), term frequency reduces to binary presence or absence and similarly sized chunks limit the effect of length normalization. That weakens the parts of BM25 designed for longer documents. Teams should verify that BM25 still adds signal at their actual chunk sizes.

Can hybrid search work across multiple data sources?

Standard implementations assume a single, pre-ingested index. For agents that need to reason across data from CRM, support, billing, and project management systems, the data must first be unified into a common searchable layer. This is a data infrastructure problem, not only a retrieval algorithm problem.

What role does reranking play after hybrid fusion?

Reranking applies a cross-encoder model to the fused result set and re-scores each candidate pair against the original query. Reranking on top of hybrid fusion typically improves final ordering. The tradeoff is latency, because each additional candidate requires more model work.