Advanced
Open
Pro
Costing an Uncapped Tool Result
A research agent's average run is 20 steps. On roughly 1 run in 5,
a search_documents call matches an unusually large set and returns
about 6,000 lines of text (assume ~15 tokens per line). The oversized
result typically lands around step 6, so it is re-encoded in the
prompt for the remaining ~14 steps. Input tokens cost $3 per million.
The team runs about 500 of these tasks a day.
- Compute the extra input-token cost caused by one oversized result, and the daily cost across all affected runs.
- Compute what changes if the harness caps tool results at 2,000 tokens with a pointer to the full result on disk.
- An engineer objects that capping "hides information from the model and will hurt quality." Give the strongest version of that objection and then the design that answers it.
Share this question