RAG Chunking

Chunking is the step in a retrieval pipeline where documents are split into passages before being embedded and indexed. It sounds like plumbing and it is the single biggest determinant of retrieval quality in most RAG systems. The trade-off runs in both directions. Small chunks embed precisely and retrieve cleanly, but they arrive at the model stripped of the surrounding context that made them meaningful — a paragraph that says "this does not apply to enterprise plans" is dangerous without the paragraph above it. Large chunks keep context and dilute the embedding, so a page about twelve topics matches weakly on all of them and strongly on none. What works in practice is chunking on structure rather than character count: headings, list items, table rows, function definitions. Overlap of a sentence or two carries context across boundaries, and storing the parent document's title and section path with each chunk lets you re-expand at answer time. Measure it. Retrieval precision on a fixed question set is cheap to compute and will tell you more about your assistant's accuracy than swapping the model will.