Trimming Broke the Cache the Team Was Relying On

A team ships a cost-reduction change to their multi-turn assistant: instead of resending the full conversation history on every turn, they now summarize everything older than the last 3 turns into a rolling "conversation state" block and prepend it to the stable system prompt, ahead of the tool schemas, "so the model sees the current state right away." Token counts per request drop as expected. But cache-hit rate on the system-prompt-plus-tool-schema block, which used to be ~95%, drops to near 0%, and overall cost barely improves despite the lower token count.

1. Diagnose why the trimming change, despite genuinely reducing token count, failed to reduce cost by much.
2. Redesign the prompt layout so both the trimming win and the caching win are preserved simultaneously.
3. Is there a scenario where trimming and caching are genuinely in tension — where you can't fully have both — and if so, what's the right way to decide the tradeoff?