AI-300 Practice Question 58: Retrieval Evaluator
Microsoft Machine Learning Operations Engineer Associate (AI-300) · Test 6 · question 8 of 10
A RAG application often gives poor answers. The team suspects the search step returns irrelevant chunks, and wants a built-in evaluator that isolates the quality of the retrieved context from the quality of the generated answer, without needing ground truth answers. Which evaluator should they run?
Show answer and explanation
Answer:
RAG evaluators split into retrieval and generation checks. The Retrieval evaluator uses an LLM judge to measure how relevant the retrieved context chunks are to the query, and it needs no ground truth. That isolates the search step from the model's answer (option D).
- A is wrong because groundedness judges the generated answer against the context it was given. It cannot tell whether the retrieved context was the right context. - B is wrong because fluency measures language quality of the response and says nothing about the search step. - C is wrong because relevance judges the final response. A response can address the query well even when retrieval was poor, so it does not isolate the retrieval step.