AI-300 Practice Question 79: Evaluators Without Ground Truth

Microsoft Machine Learning Operations Engineer Associate (AI-300) · Test 8 · question 9 of 10

Implement generative AI quality assurance and observabilitymedium

A team has an evaluation dataset with a query, the model's response and the retrieved context for each row, but no reference answers. Which set of built-in evaluators can they run on this data?

Select one answer
Show answer and explanation

Answer:

  • B. Groundedness, relevance and coherence

Textual similarity evaluators such as F1 score, BLEU, GLEU, ROUGE, METEOR and Similarity compare a response with an expected response called the ground truth. AI-assisted quality evaluators such as groundedness, relevance and coherence use an LLM judge on the query, response and context, so they run without ground truth (option B).

- A is wrong because these are overlap metrics that compare the response with a ground truth answer, which the dataset does not have. - C is wrong because all three compare the response to a ground truth, so none can run on this dataset. - D is wrong because each of these needs a reference answer to compare against.

Reference: Textual similarity evaluators

Practice Test 8 ›

Study guides for AI-300