Advanced
Open
Pro
Designing the Confidence Gate and 'I Don't Know' Path
You're adding a confidence gate to a RAG system that currently always answers. Product wants "the assistant should almost never say I don't know, that feels unhelpful." Engineering wants a hard threshold on the re-ranker's top score.
- Explain the trade-off product is missing, and how you'd frame it for them.
- Design the gate: what signal(s) it uses, what happens when it fires, and how it should be tuned.
- What ongoing monitoring would tell you the threshold is set wrong in either direction?
Share this question