Query Rewriting: HyDE, Multi-Query, and Decomposition
The query the user types is not the query your retriever wants. Rewriting bridges the gap before retrieval, so the index sees something closer to what the answer looks like. Implement Hypothetical Document Embeddings (HyDE): generate a fake answer, embed it, retrieve against that vector instead of the query vector. Implement multi-query expansion: rewrite one query into N paraphrases, retrieve with each, merge the union by reciprocal rank fusion. Implement query decomposition: split a complex question into sub-questions, retrieve per sub-question, merge. Compare the three rewriters head to head on a fixture and explain when each strategy wins. Wire a mock LLM that produces deterministic, on-fixture outputs so the rewriter loop runs offline. A user types "what does our team do when uploads fail and the budget is gone?". The corpus contains a doc that says "AbortMultipartOnFail aborts an in-flight S3 multipart upload and decrements the per-bucket retry budget when the upload fails". The query and the document do not share a noun phrase. BM25 misses. The bi-encoder ranks the document third or fourth because the query vector lands in a region of the embedding space that prefers the doc about cancelled jobs, not the doc about aborted uploads. The two-stage rerank from lesson 66 can salvage the answer if it sits in the top-N, but if it does not…
Query Rewriting: HyDE, Multi-Query, and Decomposition: The query the user types is not the query your retriever wants. Rewriting bridges the gap before…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.