A Decomposed Pipeline Is Worse Than the Single Prompt It Replaced

1. Why decomposition can score worse

Decomposition only helps when each subtask is genuinely easier and more checkable in isolation than the whole, and when errors don't compound faster than the decomposition's clarity benefit offsets them. Two concrete mechanisms for a worse outcome here: (a) error compounding without recovery — step 3 (risk assessment) only sees step 1 and step 2's outputs, not the original clause text with full context; if step 1 mis-extracts an obligation or step 2 misclassifies the clause type, step 3 inherits that error with no ability to notice the upstream mistake and reconsider, whereas a single CoT call at least has the full clause text available at the point it commits to a risk judgment; (b) loss of holistic context — legal risk often depends on how obligations, parties, and clause type interact as a whole (a broad indemnification clause is riskier paired with an unlimited-liability party than in isolation), and splitting the task into sequential steps can structurally prevent the model from weighing an interaction that only becomes visible when everything is considered together, which a single prompt naturally retains just by having everything in context at once.

2. Debugging process

Don't guess — instrument each step against ground truth on a subset of the golden set. Concretely: for each of the 60 clauses, have a human (or the existing golden-set labels, if step-level labels exist) check step 1's extraction and step 2's classification independently, not just the final risk label. This produces a per-step accuracy number (e.g., "step 1 extraction accuracy: 92%, step 2 classification accuracy: 78%") that tells you which step is actually the bottleneck, rather than only having the end-to-end 74% to reason from. If step 2 is the weak link, look at its confusion pattern (which clause types get confused for which) to see whether it's a systematic ambiguity in the taxonomy or scattered noise — this also directly tests the "loss of holistic context" hypothesis from part 1, since you'd expect classification accuracy in isolation to already be lower than what a single CoT prompt implicitly achieves by classifying with full context available.

3. The right fix

Neither "add self-consistency to every step" nor an unconditional revert is the right first move — both skip the diagnosis. Adding self-consistency to each step would increase cost further (now 3x calls times N samples per step) without addressing a context-loss problem, since voting across noisy samples of the same context-starved step doesn't recover information that step never had access to. If the debugging in part 2 shows the accuracy loss concentrates in one step (say, step 2 losing context that step 3 needs), the fix is more targeted: carry the original clause text forward as additional context into later steps rather than only the prior step's structured output, or reconsider whether this task is the multi-hop, dependency-chain shape decomposition is meant for at all. If, after that targeted fix, the decomposed pipeline still doesn't clear the 81% single-prompt baseline, reverting to the single CoT call is the right call — decomposition earns its complexity only when it measurably beats the baseline it's replacing, and here the measurement says it currently doesn't.