Advanced
Open
Pro
Setting the Clarification Bar Quantitatively
Your deep research product logs show that 40% of incoming queries trigger a clarifying question, and 15% of users who are asked one abandon the request. Product proposes removing the clarification step entirely: "just plan for the most likely interpretation and note the assumption in the report." An engineer counters that the clarification check is a single cheap call and removing it is obviously wrong.
- Using this subject's worked cost allocation (about 75 relative units per completed report, with the clarification check itself costing 1 unit), estimate the compute wasted by a wrong guess on an ambiguous query and derive the break-even probability of misinterpretation above which asking pays for itself on compute alone. Then explain why that number is not the whole argument.
- Restate the subject's clarification bar in operational terms a cheap classifier could actually apply, and use it to sort these three queries into ask / don't ask: (a) "research our competitor's pricing strategy"; (b) "what did Competitor X change about their enterprise tier pricing in the last 6 months"; (c) "how are small on-device models affecting API LLM providers?" Propose one lever, other than removing the step, that would reduce the 40% ask rate without reintroducing the wrong-guess risk.
- Reconcile an apparent contradiction: Step 5's rule is to spend the reasoning tier where "an error compounds," and a wrong clarification decision wastes the entire downstream budget, yet the table keeps the clarification check on the cheap tier. Why is that consistent?
Share this question