Tuning a Semantic Cache Without Shipping Wrong Answers

Your team wants to add a semantic cache in front of an FAQ-answering endpoint that serves 2,000,000 queries/day. Each query that reaches the model costs about $0.0020 (input + output combined). A quick experiment shows an 18% semantic-cache hit rate is achievable at a similarity threshold of 0.90.

1. If the cache is enabled at that threshold and hit rate, how much does it save per day, holding everything else constant?
2. During testing you find that "cancel my plan" and "cancel my order" have a cosine similarity of 0.93 — above your 0.90 threshold — but require different answers. Explain what this means for how you should actually choose the threshold, and what kind of eval set you need to do it safely.
3. Name two scoping rules (beyond the threshold) that are non- negotiable for this cache, and say why skipping either one is a correctness bug, not just a missed optimization.