Intermediate
Open
Pro
A Structured-Output Pipeline Is Failing Silently in Production
Your team has a prompt that extracts {"amount": number, "currency": string, "date": "YYYY-MM-DD"} from invoice text, using "ask nicely and
validate": the prompt instructs the model to output only JSON in that
shape, the response is parsed with json.loads, and on parse failure
the code catches the exception and falls back to {"amount": 0, "currency": "USD", "date": null} so the pipeline "doesn't crash."
Finance reports that a small but steady stream of invoices are showing
up with $0.00 amounts, and nobody noticed for weeks because nothing
errored.
- Identify the actual bug — not the parse failures themselves, but the design decision that let this go unnoticed for weeks.
- Propose a fix that addresses both the parsing reliability and the "silent failure" problem.
- Is switching to schema-constrained decoding sufficient on its own? What does it fix and what does it not fix?
Share this question