Chunking and Embedding Strategies

Most teams debugging a bad RAG answer start with the prompt: they add instructions, add few-shot examples, switch to a bigger generation model. Sometimes that helps a little. It rarely fixes the actual problem, because the actual problem usually happened upstream, weeks earlier, when a document was cut into chunks and turned into vectors. If the right passage never makes it into the top-k retrieved results, no prompt engineering downstream can produce a correct, grounded answer — the model is being asked to answer a question using evidence it was never given.

Chunking and embedding are the most under-invested lever in RAG quality precisely because they are unglamorous and happen once, offline, before anyone is looking at outputs. rag-architecture-end-to-end frames the whole pipeline — ingest, chunk, embed, index, retrieve, rerank, generate — and shows where each stage can fail. This subject goes deep on two of those stages: how you split a document into chunks, and how you turn a chunk into a vector. vector-databases-and-hybrid-search picks up from here — how those vectors get indexed and searched at scale, once you already have good chunks and good embeddings.

The interview signal to hit: chunking and embedding decisions are not made by intuition or by copying a blog post's "512 tokens with 10% overlap" default. They are made by building a small gold evaluation set, measuring recall@k under a few configurations, and picking the one that wins — then re-measuring whenever the corpus, the embedding model, or the query distribution changes.

graph LR
    DOC["document"] --> PARSE["parse"]
    PARSE --> CHUNK["CHUNK (this subject)<br/>strategy, size,<br/>overlap, metadata"]
    CHUNK --> EMBED["EMBED (this subject)<br/>model choice,<br/>drift & re-index"]
    EMBED --> INDEX["index<br/>vector-databases-<br/>and-hybrid-search"]
    INDEX --> RETRIEVE["retrieve"]
    RETRIEVE --> RERANK["rerank"]
    RERANK --> GENERATE["generate"]