Debugging a Low-Quality RAG System

A RAG assistant over 20,000 internal engineering documents scores 0.62 on an LLM-judge correctness rubric. The team's proposed fix is "use a bigger model and a longer, more detailed system prompt." Retrieval currently uses dense vectors only, fixed 1,000-token chunks with no overlap, and passes the top-15 chunks to the model. Error codes and service names appear frequently in queries.

1. What measurement would you take before changing anything, and why?
2. Identify at least four likely retrieval-side problems in this setup and the fix for each.
3. Explain why passing top-15 chunks may be hurting rather than helping.