Intermediate
Open
Pro
Diagnosing a 'Lost in the Middle' Failure
Your team ships a document Q&A feature backed by a model with a 200k-token context window. For a specific customer complaint, a user asks a question whose answer is a single clause on page 40 of a 90-page contract (the whole contract, concatenated, is about 55,000 tokens — comfortably within the window). The model's answer misses that clause entirely, even though the same model correctly answers questions whose answer is on page 2 or page 89 of similarly-sized documents.
- Explain, using the tokenization/context-window concepts from this subject, the most likely mechanism behind this specific failure pattern (why page 40 and not page 2 or 89).
- Propose two concrete fixes, and explain the trade-off of each.
- What would you add to your evaluation set so this class of failure is caught before it reaches a customer again?
Share this question