Paths Subjects Questions Quizzes Pricing Search
Advanced Open Pro

Pre-Loading vs Just-in-Time Retrieval for a New Agent Feature

You're designing an agent that helps engineers debug production incidents. Two designs are on the table:

  • Design A: at session start, load the last 90 days of incident postmortems (roughly 400 documents, ~600K tokens total) into context so the agent has "full institutional memory" available from turn one.
  • Design B: give the agent a search_postmortems(query) tool that returns the top 5 matching postmortem summaries on demand, and a read_postmortem(id) tool to pull the full text of one if needed.
  1. Explain why Design A is not viable as described, independent of whether it would theoretically be useful — be specific about the mechanism that breaks, not just "it's a lot of tokens."
  2. Design B is the just-in-time approach. Identify one realistic failure mode of Design B that Design A wouldn't have, and how you'd mitigate it without reverting to loading everything upfront.
  3. Is there a middle-ground element worth pre-loading here, even under a JIT-first design? Name a concrete candidate and justify why it clears the bar for pre-loading.

Share this question

← Back to Context Engineering for Agents practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.