Chunking Strategies for RAG
Chunking configuration influences retrieval quality as much as the choice of embedding model (Vectara NAACL 2025). Get chunking wrong and no amount of reranking saves you. You put a 50-page contract into a RAG system. User asks: "What is the termination clause?" The retriever returns the cover page. Why? Because the model was trained on 512-token chunks and the termination clause sits 20 pages in, split across a page break, with no local keywords tying it to the query. The fix is not "buy a better embedding model." The fix is chunking. How big? Overlap? Where to split? With surrounding context? Feb 2026 benchmarks show surprising results: Vectara's 2026 study: recursive 512-token chunking beat semantic chunking 69% → 54% accuracy. SPLADE + Mistral-8B on Natural Questions: overlap provided zero measurable benefit. Context cliff: response quality drops sharply around 2,500 tokens of context. The "obvious" answer (semantic chunking, 20% overlap, 1000 tokens) is often wrong. This lesson builds intuition for six strategies and tells you when to reach for which. Six chunking strategies visualized on one passage Fixed chunking. Split every N characters or tokens. Simplest baseline. Breaks mid-sentence. Good compression, bad coherence. Recursive. LangChain's RecursiveCharacterTextSplitter. Try splitting on \n\n first, then \n, then ., then space. Falls back cleanly. The 2026 default. Semantic. Embed each sentence. Compute cosine similarity between…
Chunking Strategies for RAG: Chunking configuration influences retrieval quality as much as the choice of embedding model (Vectara NAACL 2025). Get chunking…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.