Case Study: Design a Long-Context Document Assistant
A full AI-engineering interview answer for a QA assistant over a ~10M-token corpus: corpus math, long-context vs RAG, chunking, per-turn context assembly, multi-turn compaction, citation grounding, injection defense against untrusted documents, cost and evaluation
Model interview answer for designing the context pipeline behind an assistant that answers questions over a large document corpus (legal, policy or technical-docs, ~10M tokens): the arithmetic that rules out stuffing the corpus into a window, the long-context-vs-RAG decision and the hybrid retrieve-then-long-read pattern most candidates miss, structure-aware chunking and citation metadata, per-turn context budgeting and ordering, multi-turn history compaction that preserves the citation trail, quote-then-answer grounding and hallucinated-citation detection, defending against untrusted document content, cost/latency at scale, and evaluation with recall@k and faithfulness.