Intermediate
Open
Pro
Diagnosing a Recall Drop After a Chunk-Size Change
A team increased their RAG chunk size from 300 tokens to 1,200 tokens, reasoning that "more context per chunk means better answers." Two weeks after shipping, recall@5 on their gold set dropped from 0.84 to 0.69, and support tickets about "the bot gave a vague or off-topic answer" went up.
- Explain, mechanically, why increasing chunk size can reduce recall@5 rather than improve answer quality.
- What would you check first to confirm this diagnosis rather than guessing?
- Propose a fix that keeps the benefit they were after (more context per hit) without the recall regression.
Share this question