Advanced
Open
Pro
Designing a Trace Schema for a Tool-Using Agent
You're building observability for an agentic support assistant that rewrites the user's query, retrieves from a help centre, calls tools (account lookup, refund), and streams an answer with citations. Support keeps escalating tickets where they can't explain why the bot called (or didn't call) a tool, or why an answer cited the wrong thing, because all you currently log is one line: timestamp, user id, final answer, latency.
- Design the trace schema as a tree of spans. What does each span type need to capture, at minimum, to make "why did it do that" answerable after the fact?
- What would you explicitly exclude or redact, and why?
- Give two concrete support tickets this schema would let you resolve in minutes that the current one-line log cannot resolve at all.
- How does sampling/retention policy differ between recent traffic and long-term storage, and why not just keep everything forever at full fidelity?
Share this question