Ask a vector index for an exact error code, a SKU, or a contract’s clause number, and it will often hand back several confident, well-written answers, and zero documents containing the string you actually needed. That failure is quiet, because the response still reads fluent. It just isn’t right. That gap is what led us to hybrid search.
Why Semantic Search Fails
An embedding model compresses a piece of text into a vector of numbers that represents its meaning. Documents about similar things end up with vectors that sit close together in that space, regardless of the exact words used, which is precisely what makes semantic search good at synonyms and paraphrasing, and precisely what makes it blind to the things that don’t carry “meaning” in the way the model understands it.
Think of it like a librarian who organises every book purely by subject instead of by title. Ask her for books about space travel and she’ll bring you a great pile, sorted by theme, even if you never used her exact words. But ask her for “the book with catalog number 4471-B” and she has no idea what you mean that’s not a subject, it’s a label, and her entire filing system was never built to notice labels in the first place.
This is exactly the blind spot that trips up real systems. A support ticket’s error code, a warehouse’s SKU, a contract’s exact clause number, none of these carry much meaning on their own, so an embedding model treats them almost like noise. Two unrelated error codes can look nearly identical to it, because both are just short strings of letters and digits sitting in the same tiny corner of meaning-space. The model isn’t wrong about what they mean semantically, it’s just answering a different question than the one being asked.
It shows up on ordinary queries too, not just technical ones. Ask a country-facts index which nation is best known for rice fields and paddies, and semantic search will surface reasonable, agrarian, Southeast-Asian answers, while missing the one document that actually contains the word “rice” the most literally. The model retrieved the right neighbourhood. It didn’t retrieve the right house. That turns out to be the default outcome any time exact wording matters more than the general idea behind it, which is surprisingly often once you’re grounding an agent on manuals, contracts, or support logs instead of clean encyclopaedia text.
What gets lost: anything that behaves like a label rather than an idea, an ID, a code, a name, an exact figure, tends to disappear into a fog of “similar enough” the moment it becomes a vector.
What Hybrid Search Is And Why It Beats Either Alone
Hybrid search runs two independent retrieval engines on the same query and merges the results. The first is the vector search itself, unchanged. The second is lexical search usually BM25, an algorithm that scores documents by how often a query term appears, weighted so rare terms count for more than common ones. BM25 doesn’t understand meaning at all; it understands exact and near-exact term overlap, which is exactly the strength semantic search lacks.
Figure 1: Hybrid search runs semantic and lexical retrieval in parallel on the same query, then merges the two ranked lists into one.
BM25’s logic is simple counting, dressed up with two adjustments. It counts how many times each query word shows up in a document, but a word like “the” appears everywhere and barely moves the score, while a rare word like “paddies” gives a real boost the moment it shows up. Matching several query words at once climbs the score further, but with diminishing returns — a document that repeats one word twenty times doesn’t automatically beat a shorter document that mentions it twice but actually covers everything else the query asked about.
Fusing the Scores: Reciprocal Rank Fusion (RRF)
The two rankings can’t just be added together a BM25 score and a cosine-similarity score live on completely different numeric scales, so summing them would let whichever engine happens to produce bigger numbers dominate for no principled reason. Reciprocal Rank Fusion sidesteps the scale problem entirely by ignoring raw scores and working off rank position instead:
rrf_score(doc) = Σ 1 / (k + rank + 1)
across every ranking the document appears in
# rank starts at 0 within each list
# k = 60 is the standard convention it damps how much
# any single list can dominate the fused score
A worked example makes this concrete. Say BM25 ranks document A first, B second, C third, while semantic search ranks B first, C second, A third. Document B picks up 1/(60+0+1) from the BM25 list plus 1/(60+0+1) from the semantic list the best combined score of the three, because it placed near the top of both rankings. Document A only earns credit from one list, so despite being BM25’s favourite, it ends up behind B once the two lists are fused. No raw scores are compared anywhere in that calculation — only where each document landed in each ranking.
Figure 2: A worked RRF example — each document’s BM25 rank and semantic rank are converted to scores and summed; the document strong in both rankings wins the fused result.
Why it wins: lexical search catches the exact term that semantic search glosses over. Semantic search catches the synonym that lexical search can’t see. Fusing them by rank, not raw score, means neither one has to win outright — they both get a vote.
Put side by side, the trade-offs — and why combining the two closes both gaps at once — are straightforward:
| Capability | Lexical Search (BM25) | Semantic Search (Vector) | Hybrid Search (RRF) |
| Exact term / code / ID matching | Strong — literal match | Weak — treated as near-meaningless noise | Strong — inherited from lexical |
| Synonyms & paraphrasing | Weak — no notion of meaning | Strong — native strength | Strong — inherited from semantic |
| Scoring basis | Term frequency, rarity-weighted | Vector distance (cosine similarity) | Rank fusion (RRF) — not raw scores |
| Best fit | Structured, ID-heavy content | Free-text meaning & context | Real-world docs mixing both |
| Typical failure mode | Blind to meaning and synonyms | Blind to labels and exact strings | Needs both engines indexed |
| Setup complexity | Low — single keyword index | Low — single vector index | Low in Azure AI Search — fusion is native |
Table 1: How lexical, semantic, and hybrid search compare across the query types that matter in production retrieval.
Hybrid Search in Azure AI Search
Azure AI Search doesn’t require hand-rolling the RRF loop. A single search request can carry both a lexical query and a vector query at the same time — a plain-text search parameter feeds the keyword engine, and a vectorQueries array feeds the vector engine — and the service fuses both rankings server-side, using the same reciprocal-rank logic, before returning results.
POST /indexes/{index-name}/docs/search?api-version=2024-07-01
{
“search”: “error code E402”, // feeds BM25
“vectorQueries”: [{
“kind”: “vector”,
“vector”: [ …embedding… ], // feeds vector search
“fields”: “contentVector”
}],
“select”: “content”
}
Leave the search text out (pass it as null) and the request runs pure vector search; include only search and it runs pure keyword search; include both and Azure AI Search runs hybrid automatically. The SDK route looks almost identical to the REST call — SearchClient.search(search_text=…, vector_queries=[vector_query]) — pass both arguments and the client handles the same fusion on the server, without a single line of ranking math written by hand. A semantic ranker can layer on top for a further cross-encoder re-rank of the fused list, re-examining the top handful of results more closely for precision — but the hybrid fusion itself is the default the moment both query types are present in the request, not an opt-in feature to assemble.
Worth remembering: Azure AI Search bakes RRF into hybrid queries automatically — you supply search text and vectorQueries, not a fusion formula. The interface to remember is simpler than the mechanism underneath it. An agent calling this as a search tool doesn’t need to know BM25 or RRF exist; it just passes a query string and an embedding, and the fusion happens on the other side of the call.
Where We Use This
The same fusion logic isn’t limited to whole documents it applies just as well at the paragraph or chunk level, to find which chunk within a document is most relevant to a query, not just which document is. That granularity matters most in retrieval-augmented pipelines, where the unit that gets returned to the model is a chunk, not a full file.
We’ve validated this directly in our own retrieval pipeline: the reference and citation feature we use to trace a generated answer back to its exact source chunk is built on hybrid search, and it measurably outperforms semantic search alone — particularly whenever the source text contains the specific terms, codes, or names that the query is actually anchored to.
Pure semantic search was never an upgrade over keyword search it was a trade: better on meaning, worse on exact terms. Hybrid search is what removes the need to make that trade at all.
This is the kind of applied AI engineering we do at 47billion.com. Our team works on real-world retrieval and document AI pipelines from the initial architecture design through prompt and index engineering, evaluation, and production deployment. We have built and measured these systems against real queries, which means we understand not just the theory but the failure modes that matter in practice.
