A Coding Agent Degrades Badly Somewhere Around Turn 60

1. Mechanism behind the degradation

The verbatim run_tests output and unbounded read_file are the two largest unmanaged sources of context growth described in tool design — each call can dump thousands of tokens with no relationship to what the agent actually needs on that turn. As the session accumulates dozens of these calls, two compounding effects show up: attention dilution (the early "/legacy" instruction, stated once near the start, is now competing with tens of thousands of tokens of test output and file contents for effective attention, and loses) and a practical inability to "remember" prior attempts (a fix tried at turn 15 and confirmed failed is buried deep in the transcript by turn 60, so the agent's effective working memory of "what have I already tried" degrades even though the information is technically still present in the window). Both symptoms — forgetting the constraint, repeating a failed fix — are exactly what "lost in the middle" / instruction-following drift predicts: content is present but not reliably attended to once it's far enough back in a long, noisy transcript.

2. Two concrete tool-design fixes

• Redesign run_tests to return a compact summary by default (pass/fail count, names of failing tests, and only the first N lines of each failure's stack trace), with a way to request the full output for a specific named test if actually needed. Mechanistically, this removes the largest recurring token cost per call without removing the agent's ability to get detail when it actually needs it — most turns don't need the full stdout, they need to know what changed since the last run.
• Cap read_file at a reasonable line window with an explicit offset/ limit interface (matching how the agent should be reading large files incrementally rather than in full), so a single call can't dump an entire large file when the agent only needed one function. This directly reduces per-call token cost and also nudges the agent toward just-in-time, targeted reads instead of habitually re-reading whole files "to be safe."

Both fixes work the same way: they don't just reduce total tokens, they reduce the rate at which low-signal content is appended per turn, which is what determines how many turns pass before attention dilution and compaction pressure set in — a one-time reduction wouldn't help a 60-turn session nearly as much as a per-call reduction does.

3. Making the /legacy constraint survive

Don't rely on it having been stated once in chat early in the session — promote it to durable context that doesn't depend on staying within attention range of the current turn. Two complementary options: write it into the agent's externalized progress/scratchpad note the moment it's stated, so any future compaction step has a structured, explicit item to preserve rather than something to infer from a buried chat message; and, if this is a standing rule rather than a one-off for this session, it belongs in the repo's steering file (CLAUDE.md) so it's loaded automatically at the start of every session rather than depending on a user re-stating it, or a compaction step correctly preserving a single chat mention, every time. The chat-only version is exactly the kind of instruction compaction is most likely to lose, because nothing marks it as more important than the surrounding conversation.