AI-300 Practice Question 79: Evaluators Without Ground Truth
Microsoft Machine Learning Operations Engineer Associate (AI-300) · Test 8 · question 9 of 10
A team has an evaluation dataset with a query, the model's response and the retrieved context for each row, but no reference answers. Which set of built-in evaluators can they run on this data?
Show answer and explanation
Answer:
Textual similarity evaluators such as F1 score, BLEU, GLEU, ROUGE, METEOR and Similarity compare a response with an expected response called the ground truth. AI-assisted quality evaluators such as groundedness, relevance and coherence use an LLM judge on the query, response and context, so they run without ground truth (option B).
- A is wrong because these are overlap metrics that compare the response with a ground truth answer, which the dataset does not have. - C is wrong because all three compare the response to a ground truth, so none can run on this dataset. - D is wrong because each of these needs a reference answer to compare against.
Reference: Textual similarity evaluators