Hybrid Search (BM25 + Vector)

Hybrid search is a retrieval method that runs keyword search and vector search on the same query and merges their rankings into one result list. It combines the exact-term precision of BM25 with the meaning-based recall of vector search, usually through reciprocal rank fusion or a weighted combination of normalised scores.

• Lexical retriever: A BM25 or full-text index that identifies documents containing exact tokens, such as error codes and function identifiers.
• Dense retriever: A vector index that identifies semantically related documents through their embedding representations.
• Fusion: The combination procedure that consolidates the two ranked candidate lists into a single ordering.
• Reciprocal rank fusion (RRF): A fusion algorithm that scores each document by adding one reciprocal-rank contribution from every list.
• Weighted fusion: An alternative approach that normalises both score distributions and combines them with a configurable weighting.

For example, the query "pod memory exit code 137" needs BM25 to match the exact exit code and vector search to recognise memory pressure, and hybrid search returns the OOMKilled runbook first.

Key Characteristics of Hybrid Search

• Complementary coverage: Combining keyword and semantic search lets keyword matching handle rare identifiers, while embeddings handle paraphrased symptom descriptions, as compared in keyword search vs semantic search.
• Parallel retrieval: Both retrievers operate independently, frequently concurrently, over the identical document collection.
• Rank-based merging: RRF disregards raw score magnitudes, so the incompatible BM25 and cosine similarity scales require no calibration.
• Agreement reward: Documents positioned highly by both retrievers accumulate the largest combined relevance scores.
• Configurable balance: Weighted fusion allows an engineering team to prioritise keyword or semantic evidence for a particular corpus.
• Pipeline position: Hybrid retrieval typically generates the candidate set that a separate reranking model subsequently reorders.

How Hybrid Search Works

1. Dual indexing: Every document is inserted into an inverted index for BM25 and, after embedding, into a vector index.
2. Parallel querying: The query is tokenised for the BM25 retriever and simultaneously embedded for the vector retriever.
3. Candidate generation: Each retriever returns an independent top-n list, which typically contains a few dozen documents.
4. Fusion: For every list containing a document, RRF adds the value 1 / (k + rank), and the constant k is commonly set to 60.
5. Final ordering: Documents are sorted by their combined score, and the highest-ranked results are displayed or forwarded to a reranker.

Score Fusion Formulas

Hybrid search must convert two ranked lists with incompatible score scales into a single ordering. Two formulas are common: one uses only the rank positions, and the other rescales the raw scores before combining them. The reciprocal rank fusion formula is:

• $d$: one candidate document returned by at least one retriever.
• $R$: the set of ranked lists, here the BM25 list and the vector list.
• ${\text{rank}}_r(d)$: the position of the document in list $r$, starting at 1, where a list that omits the document contributes nothing.
• $k$: a smoothing constant, conventionally 60, which prevents a single first position from dominating the combined score.

In words, every list awards a document a small contribution that decreases slowly with its position. With $k=60$, first place earns 0.0164 and tenth place still earns 0.0143, so agreement between the retrievers matters considerably more than one excellent position.

Weighted fusion first maps each retriever's scores onto the interval from 0 to 1 with min-max normalisation, then combines them with a configurable weight:

• $s$: a raw score from one retriever, such as a BM25 score or a cosine similarity.
• $s_{\min },s_{\max }$: the lowest and highest raw scores within that retriever's candidate list.
• $\hat{s}$: the normalised score, where the weakest candidate receives 0 and the strongest receives 1.
• $\alpha$: the weight assigned to BM25 evidence, between 0 and 1, while the vector score receives the remaining $1-\alpha$.

The worked example fuses four candidate runbooks for the query "pod memory exit code 137":

• Rank fusion: The OOMKilled runbook rb-31 is first for BM25 and second for vector search, while the node-pressure runbook rb-44 is third and first, so their RRF scores are almost identical.

• Normalisation: BM25 scores range from 0.00 to 3.60, and cosine scores range from 0.52 to 0.83, so each raw score is shifted and divided by its own range.

• Weighted fusion: With a balanced weight, rb-31 ranks first, but a vector-heavy weight of 0.2 allows rb-44 to overtake it.

# Candidate scores for "pod memory exit code 137".
# BM25 scores from bm25_ranking() in the example below, rounded; cosine scores are hand-made.
BM25 = {"rb-31": 3.60, "rb-08": 3.33, "rb-44": 0.90, "rb-57": 0.00}
COSINE = {"rb-31": 0.71, "rb-08": 0.52, "rb-44": 0.83, "rb-57": 0.60}

def min_max(scores):
    lo, hi = min(scores.values()), max(scores.values())
    return {d: (s - lo) / (hi - lo) for d, s in scores.items()}

bm25, vec = min_max(BM25), min_max(COSINE)
for d in BM25:
    print(f"{d}: bm25 {BM25[d]:.2f} -> {bm25[d]:.2f}   cosine {COSINE[d]:.2f} -> {vec[d]:.2f}")

for alpha in (0.2, 0.5, 0.8):
    fused = {d: alpha * bm25[d] + (1 - alpha) * vec[d] for d in BM25}
    top = sorted(fused, key=lambda d: -fused[d])
    print(f"alpha = {alpha}: " + ", ".join(f"{d} {fused[d]:.2f}" for d in top))

# RRF for comparison: ranks only, k = 60.
print(f"RRF rb-31: 1/61 + 1/62 = {1 / 61 + 1 / 62:.4f}")
print(f"RRF rb-44: 1/63 + 1/61 = {1 / 63 + 1 / 61:.4f}")

rb-31: bm25 3.60 -> 1.00   cosine 0.71 -> 0.61
rb-08: bm25 3.33 -> 0.93   cosine 0.52 -> 0.00
rb-44: bm25 0.90 -> 0.25   cosine 0.83 -> 1.00
rb-57: bm25 0.00 -> 0.00   cosine 0.60 -> 0.26
alpha = 0.2: rb-44 0.85, rb-31 0.69, rb-57 0.21, rb-08 0.19
alpha = 0.5: rb-31 0.81, rb-44 0.62, rb-08 0.46, rb-57 0.13
alpha = 0.8: rb-31 0.92, rb-08 0.74, rb-44 0.40, rb-57 0.05
RRF rb-31: 1/61 + 1/62 = 0.0325
RRF rb-44: 1/63 + 1/61 = 0.0323

• Stable rank fusion: RRF assigns rb-31 and rb-44 almost equal scores, because each document appears near the top of both candidate lists.
• Weight changes the winner: Lowering $\alpha$ to 0.2 places the memory-pressure runbook first, and raising it to 0.8 lifts the exit-codes runbook into second position.
• Sensitive normalisation: A single outlier score stretches the min-max range and compresses every other candidate towards 0.

In practice, $\alpha$ is a tuning decision that must be evaluated on labelled queries from the team's own knowledge base, whereas RRF requires no tuning, which explains why it is the usual default.

Example: Reciprocal Rank Fusion in Python

The program below ranks five runbooks with BM25, takes a hand-made ranking from a vector index and merges both lists with RRF.

import math

RUNBOOKS = {
    "rb-08 exit-codes":    "exit code 137 means SIGKILL, exit code 143 means SIGTERM",
    "rb-12 disk-full":     "disk full on /var/log: rotate logs and expand the volume",
    "rb-31 oom-kill":      "container OOMKilled: exit code 137, raise the memory limit",
    "rb-44 node-pressure": "node memory pressure evicts pods, check requests and limits",
    "rb-57 slow-queries":  "slow queries: add an index and check the connection pool",
}
QUERY = "pod memory exit code 137"

def tokens(text):
    return text.lower().replace(":", " ").replace(",", " ").split()

def bm25_ranking(query, k1=1.5, b=0.75):
    docs = {d: tokens(t) for d, t in RUNBOOKS.items()}
    avgdl = sum(map(len, docs.values())) / len(docs)
    def score(terms):
        total = 0.0
        for term in tokens(query):
            df = sum(term in t for t in docs.values())
            if df:
                idf = math.log(1 + (len(docs) - df + 0.5) / (df + 0.5))
                tf = terms.count(term)
                total += idf * tf * (k1 + 1) / (tf + k1 * (1 - b + b * len(terms) / avgdl))
        return total
    return sorted(((d, score(t)) for d, t in docs.items()), key=lambda x: -x[1])

# Ranking returned by a vector index for the same query (hand-made for the demo)
vector_ranking = ["rb-44 node-pressure", "rb-31 oom-kill", "rb-57 slow-queries",
                  "rb-08 exit-codes", "rb-12 disk-full"]

def rrf(rankings, k=60):
    """Reciprocal rank fusion: sum 1 / (k + rank) over every ranking."""
    fused = {}
    for ranking in rankings:
        for rank, doc in enumerate(ranking, start=1):
            fused[doc] = fused.get(doc, 0.0) + 1 / (k + rank)
    return sorted(fused.items(), key=lambda x: -x[1])

keyword_ranking = [d for d, s in bm25_ranking(QUERY) if s > 0]  # matched only
print("BM25:  ", keyword_ranking)
print("Vector:", vector_ranking[:3], "...")
print("Hybrid (RRF, k=60):")
for doc, score in rrf([keyword_ranking, vector_ranking]):
    print(f"  {doc:<20} {score:.4f}")

BM25:   ['rb-31 oom-kill', 'rb-08 exit-codes', 'rb-44 node-pressure']
Vector: ['rb-44 node-pressure', 'rb-31 oom-kill', 'rb-57 slow-queries'] ...
Hybrid (RRF, k=60):
  rb-31 oom-kill       0.0325
  rb-44 node-pressure  0.0323
  rb-08 exit-codes     0.0318
  rb-57 slow-queries   0.0159
  rb-12 disk-full      0.0154

• Agreement wins: The OOMKilled runbook occupies first and second positions in the two lists, which produces the highest combined score.
• Exact-term recovery: The exit-codes runbook ranks fourth in vector search, but BM25 lifts it into the fused top three through the token 137.
• Single-list documents: The slow-queries runbook appears exclusively in the vector list, so it receives only one contribution and falls behind.

Applications of Hybrid Search

• Engineering knowledge bases: Searching runbooks, postmortems and API documentation with queries that combine identifiers and descriptions.
• Code search: Matching exact symbol names while simultaneously retrieving functionally equivalent implementations.
• RAG retrieval: Supplying higher-quality candidate passages before reranking in RAG.
• Log investigation: Correlating exact error strings with semantically similar historical incidents.
• AI search engines: Generating the candidate set described in how AI search engines work.

Advantages

• Improved recall: Documents overlooked by one retriever can still be identified by the other retriever.
• Identifier precision: Error codes, version numbers and function identifiers remain reliably searchable.
• Uncomplicated fusion: RRF requires no training data and exposes only one configuration parameter.
• Graceful degradation: If the embedding model represents a specialised domain poorly, BM25 still returns exact matches.

Limitations

• Operational complexity: Two separate indexes must be constructed, updated and kept consistent with the same documents.
• Additional latency: Two retrievers and a fusion step require more time than a single retriever, unless they execute in parallel.
• Evaluation effort: The constant k, the candidate list depths and any fusion weights require evaluation on representative queries.
• Shared blind spots: A relevant document that neither retriever positions highly cannot be recovered by any fusion method.

Many search engines and vector databases offer built-in hybrid queries with RRF or weighted fusion, as described in vector database products.