Budgeting a Two-Stage Retrieval Pipeline with Re-Ranking

Your RAG pipeline currently does dense + BM25 hybrid retrieval fused with RRF, returning the top 5 chunks straight into the prompt. Generation quality is inconsistent: sometimes the right chunk is retrieved but ranked 4th or 5th, dilutes the context, or gets dropped entirely by an upstream truncation step. You're asked to add cross-encoder re-ranking.

1. Explain why a cross-encoder tends to produce a better final ranking than the fused hybrid ranking alone.
2. Why must it be applied to a shortlist rather than the full corpus — what specifically makes that infeasible?
3. Design the two-stage pipeline: what shortlist size would you fetch before re-ranking, and how would you reason about the added latency if a single cross-encoder pass takes ~10 ms per pair on your hardware?