Match a job Paths Subjects Questions Quizzes Pricing
Advanced Open Pro

Setting the Clarification Bar Quantitatively

Your deep research product logs show that 40% of incoming queries trigger a clarifying question, and 15% of users who are asked one abandon the request. Product proposes removing the clarification step entirely: "just plan for the most likely interpretation and note the assumption in the report." An engineer counters that the clarification check is a single cheap call and removing it is obviously wrong.

  1. Using this subject's worked cost allocation (about 75 relative units per completed report, with the clarification check itself costing 1 unit), estimate the compute wasted by a wrong guess on an ambiguous query and derive the break-even probability of misinterpretation above which asking pays for itself on compute alone. Then explain why that number is not the whole argument.
  2. Restate the subject's clarification bar in operational terms a cheap classifier could actually apply, and use it to sort these three queries into ask / don't ask: (a) "research our competitor's pricing strategy"; (b) "what did Competitor X change about their enterprise tier pricing in the last 6 months"; (c) "how are small on-device models affecting API LLM providers?" Propose one lever, other than removing the step, that would reduce the 40% ask rate without reintroducing the wrong-guess risk.
  3. Reconcile an apparent contradiction: Step 5's rule is to spend the reasoning tier where "an error compounds," and a wrong clarification decision wastes the entire downstream budget, yet the table keeps the clarification check on the cheap tier. Why is that consistent?

Share this question

← Back to Case Study: Design a Deep Research Agent practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.