Practice — Evaluating RAG Systems (5 questions)

Your team's weekly offline eval dashboard for a RAG-based support assistant shows:

• recall@5 = 0.65
• judge correctness (vs. gold answer) = 0.90
• faithfulness = 0.80

Your manager sees the 0.90 correctness number and says "that's a great score, let's ship this to more traffic." Walk through how you would respond: what do these three numbers actually tell you together, what do you suspect is really happening, and what would you fix first before shipping wider? Be specific about what evidence would confirm or rule out your hypothesis.