The Cost of Skipping Diff Review
A developer runs Claude Code in Accept Edits mode for an afternoon, asking it to make a series of small refactors across a codebase: renaming a widely-used utility function, updating its call sites, and adjusting a few related type definitions. They don't look at any diffs during the session — they just keep issuing new requests and trust that "the tests passed, so it must be fine."
- Explain specifically what the agentic loop's "verify results" phase does and does not guarantee about a change, using this scenario.
- Give a concrete example of a plausible mistake that could survive an entire session like this — passing tests, no prompts triggered — and only surface later. Explain why the loop wouldn't have caught it.
- What's the minimal habit change that would have caught this kind of issue, and why does it not require abandoning Accept Edits mode?
1. What verification guarantees and doesn't:
The verify-results phase checks whatever Claude chose to check — typically, that commands succeed and existing tests pass. That's a behavioral check bounded entirely by the test suite's actual coverage: it can tell you the code the tests exercise still behaves the way the tests expect. It says nothing about the things tests commonly don't cover: code style and consistency with the rest of the codebase, whether a rename was applied to every call site rather than just the ones with test coverage, whether a type change is technically valid but produces overly broad or awkward types, or whether the refactor introduced a call site that's syntactically correct and passes types but is semantically wrong in a way no test happens to exercise. "Tests passed" is real signal, but it is a sample of correctness, not a proof of it — and the sample is only as good as the existing test suite's coverage.
2. A plausible surviving mistake:
Suppose the utility function being renamed had a rarely-used call site in an admin-only or feature-flagged code path that has no test coverage — maybe a script run manually once a quarter, or an admin panel a QA team rarely exercises. Claude's Grep-based search for call sites might miss a call constructed dynamically (e.g. the function name built from a string, or invoked through a level of indirection like a registry object) rather than referenced literally by name. The rename would apply everywhere Grep found a literal match, tests would pass because none of them touch that code path, and the session would report success. The break wouldn't surface until someone actually runs that quarterly script or opens that admin panel weeks later — at which point the connection back to "that refactor session" is much harder to make, because nothing in between flagged it as suspicious.
3. The minimal habit fix:
Reviewing git diff (or asking Claude to summarize the full diff) at
natural checkpoints — after each meaningfully distinct piece of work,
not necessarily after every single edit — is enough to catch this
class of problem, because a human skimming the diff is checking a
different, complementary thing than the test suite: "does this whole
set of changes make sense and look complete," not "does the code
execute correctly." This doesn't require abandoning Accept Edits mode,
because the two mechanisms are solving different problems: Accept
Edits controls when Claude pauses to ask permission before acting,
while diff review is a separate, human, after-the-fact check on
what already happened. You can keep the speed benefit of not being
interrupted for every edit while still reserving a few minutes to
actually look at what changed before calling the session done — the
two aren't in tension, and conflating "I don't need to approve every
edit" with "I don't need to ever look at what changed" is exactly the
mistake this scenario illustrates.
Share this question