Advanced
Open
Pro
Pre-Loading vs Just-in-Time Retrieval for a New Agent Feature
You're designing an agent that helps engineers debug production incidents. Two designs are on the table:
- Design A: at session start, load the last 90 days of incident postmortems (roughly 400 documents, ~600K tokens total) into context so the agent has "full institutional memory" available from turn one.
- Design B: give the agent a
search_postmortems(query)tool that returns the top 5 matching postmortem summaries on demand, and aread_postmortem(id)tool to pull the full text of one if needed.
- Explain why Design A is not viable as described, independent of whether it would theoretically be useful — be specific about the mechanism that breaks, not just "it's a lot of tokens."
- Design B is the just-in-time approach. Identify one realistic failure mode of Design B that Design A wouldn't have, and how you'd mitigate it without reverting to loading everything upfront.
- Is there a middle-ground element worth pre-loading here, even under a JIT-first design? Name a concrete candidate and justify why it clears the bar for pre-loading.
Share this question