Paths Subjects Questions Quizzes Pricing Search
Intermediate Open Pro

A Structured-Output Pipeline Is Failing Silently in Production

Your team has a prompt that extracts {"amount": number, "currency": string, "date": "YYYY-MM-DD"} from invoice text, using "ask nicely and validate": the prompt instructs the model to output only JSON in that shape, the response is parsed with json.loads, and on parse failure the code catches the exception and falls back to {"amount": 0, "currency": "USD", "date": null} so the pipeline "doesn't crash." Finance reports that a small but steady stream of invoices are showing up with $0.00 amounts, and nobody noticed for weeks because nothing errored.

  1. Identify the actual bug — not the parse failures themselves, but the design decision that let this go unnoticed for weeks.
  2. Propose a fix that addresses both the parsing reliability and the "silent failure" problem.
  3. Is switching to schema-constrained decoding sufficient on its own? What does it fix and what does it not fix?

Share this question

← Back to Prompt Engineering practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.