The short answer
Build hybrid search as the deliberate combination of lexical (keyword/BM25) and semantic (vector) retrieval, fused and then reranked — because each one fails exactly where the other is strong. Vector-only search quietly loses the exact identifiers investigations depend on; keyword-only search misses everything phrased differently. The engineering is in the fusion and the reranking, not in picking a side.
Why neither lexical nor vector wins alone
Keyword search is precise on exact tokens — names, handles, case numbers, hashes, domains — and useless the moment the query and the document phrase the same idea differently. Vector search is the opposite: it captures meaning and paraphrase, and it's unreliable on the exact-identifier lookups that investigations live on, because an embedding will happily return something 'similar' to an account number that is not that account number.
For investigation work that gap is disqualifying. You cannot ship a search that sometimes fumbles the exact string a case hinges on. So hybrid isn't a nice-to-have — it's the only configuration that's precise on identifiers and recall-strong on meaning at the same time. The reason people default to vector-only is that it demos well; it's in production, on messy real queries, that the missing lexical precision shows up.
Fuse the two rankings deliberately
The core of the stack is how you combine the two result sets. Run lexical (BM25) and dense vector retrieval in parallel, then fuse their rankings — reciprocal rank fusion is a strong, boring default, or a weighted score blend when you want to tune the balance per query type. The point is that fusion is an explicit, tunable decision, not an accident of whichever index you happened to query.
Getting fusion right is mostly about respecting what each retriever is telling you. An exact-identifier query should let the lexical signal dominate; a 'find me things like this' query should lean semantic. I build the fusion so those behaviors are configurable and observable, because the right blend for adverse-media narrative search is not the right blend for looking up a specific wallet address, and a single fixed weighting quietly serves one of them badly.
Rerank for real relevance
Fusion gives you a good candidate set; reranking is what makes the top results actually right. A cross-encoder reranker scores each candidate against the query directly, catching relevance that neither the sparse nor the dense first-stage retrievers could see on their own. It's the highest-leverage quality improvement in most stacks and the step teams most often skip because the first-stage results already 'look fine.'
The trade-off is cost and latency — rerankers are heavier — so you rerank a bounded candidate set, not the whole corpus. Retrieve broadly and cheaply with hybrid fusion, then rerank the top candidates precisely. That two-stage shape is what lets the system be both fast and genuinely relevant, and it's where most of the perceived 'search quality' actually comes from.
Chunk and index for the questions you'll ask
A lot of hybrid-search quality is decided before retrieval, at indexing time. Chunk documents too coarsely and a precise answer gets diluted by surrounding irrelevant text; chunk too finely and you shred the context that made a passage meaningful. The right granularity depends on the documents and the queries, and it's worth treating as a tuning decision rather than a default you inherited from a tutorial.
The same discipline applies to what you attach to each chunk. Structured metadata — source, entity, date, document type — lets the system filter and boost in ways pure similarity never could, which matters enormously in an investigation where 'about this entity, from this source, in this window' is a common shape of query. Index deliberately for the questions the team actually asks, and the retrieval layer has far less work to do to look smart.
Tune against the queries you actually get
A hybrid stack is only as good as the evaluation behind it. You need a real set of representative queries with judged results, so that when you change the fusion weights or swap a reranker you can see whether relevance went up or down instead of guessing. Without that, tuning is vibes, and vibes regress silently the first time the query mix shifts.
For an investigation context the query mix is distinctive — heavy on exact identifiers, entity names, and 'related to this' expansions — and the stack should be tuned to it rather than to a generic benchmark. I build the evaluation loop in from the start: representative queries, judged relevance, and observability on where results are weak, so hybrid search stays a system you improve deliberately rather than one you ship once and hope holds.
The other constraint that shapes the whole design is the latency budget. Reranking is the highest-quality step and the most expensive one, so how much you can afford depends on whether search is powering an interactive investigator's console or a batch enrichment job. For interactive use I keep the first-stage retrieval fast and rerank a tight candidate set, sometimes caching results for repeated queries; for batch use I can rerank deeper because nobody's waiting. The mistake is picking one configuration and applying it everywhere — the same stack tuned for a sub-second interactive query and for an overnight bulk pass are genuinely different systems, and pretending otherwise means one of them is either too slow or needlessly shallow.