Choosing a Chunking Strategy for a Legal Contract Corpus
You're building RAG over a corpus of 20,000 signed legal contracts for an internal "ask a question about any contract" tool used by the legal team. Contracts have numbered clauses and sub-clauses (structure), but individual clauses are dense — a single clause can reference definitions from three sections earlier, and a question like "what's our liability cap under this contract?" may need the definitions section plus the liability clause plus an exceptions sub-clause to answer correctly.
- Which chunking strategy would you start with, and why?
- Legal wants citations down to the exact clause. How does that constrain your design?
- How would you handle the cross-reference problem (a clause that only makes sense alongside a definition several sections away)?
1. Starting strategy
Structure-aware chunking, because contracts have explicit, reliable structure (numbered clauses/sub-clauses) that a parser can walk deterministically — this is exactly the case where structure-aware beats semantic chunking on cost-for-value: you don't need to compute sentence-embedding similarity to find boundaries the document already declares. But given the cross-reference problem in the prompt, I'd go further and use parent-child: child = individual clause or sub-clause (the precise, citable unit), parent = the clause plus its immediate section (enough context to usually resolve local references). Fixed-size chunking is a poor fit here — an arbitrary token cut inside a clause is close to useless for a legal reading, and semantic chunking adds complexity without a clear win over structure the document already provides.
2. Citation constraint
"Citations down to the exact clause" means the child-chunk granularity
is the citation granularity — each child chunk should map to exactly
one clause or sub-clause, not a merged span, so section_path (e.g.
"Contract #4821 > Section 7 > Clause 7.3") can be surfaced verbatim as
the citation. This also means overlap should be minimal to zero at the
child level: overlapping clause text would make "which clause is this
citation pointing at" ambiguous. Overlap, if used at all, belongs at
the parent level where boundaries are looser.
3. Cross-reference problem
Three complementary approaches, in the order I'd try them:
- Retrieve more than one hit and let re-ranking/generation compose them — if "liability cap" retrieves both the liability clause and the definitions clause it references (because both are topically related to the query), the generator can compose the answer from multiple chunks. This is the cheapest option and often sufficient.
- Explicit cross-reference metadata at ingest time — many contracts
use consistent phrasing ("as defined in Section 3.2"); a parser can
extract these references during ingestion and store them as a
references: ["3.2"]metadata field, then expand retrieval to pull in referenced clauses automatically alongside a matched clause. This is more engineering but directly solves the problem the naive approach only solves probabilistically. - Widen the parent for defined-terms sections specifically — if a contract's definitions section is short, making it its own always-included parent (or boosting its retrieval rank) is simpler than reference-tracking and covers the common case where "cap", "affiliate", "confidential information" etc. need their definition.
Whichever is chosen, this is a case for building a legal-specific gold set (real questions from the legal team, paired with the clauses that answer them, including multi-clause questions) and measuring recall@k before deciding — the cross-reference cases are exactly where a naive chunking choice would silently fail without that measurement.
Share this question