Paths Subjects Questions Quizzes Pricing Search
Advanced Open Pro

Assembling a Turn's Context Budget

Your document assistant has a 200,000-token window. In early testing you notice two problems: (a) cost per query is higher than expected because the prompt-prefix cache almost never hits, and (b) on synthesis queries the model sometimes ignores clearly relevant passages that were retrieved and included in the context.

  1. Diagnose both problems in terms of what's in the prompt and in what order.
  2. Design the token budget for one turn: what segments make up the 200k window and roughly how large is each.
  3. State the ordering you'd use and explain why it fixes both problems at once rather than trading one for the other.

Share this question

← Back to Case Study: Design a Long-Context Document Assistant practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.