Databricks Generative AI Engineer Associate Practice Question 10: Judges Requiring Ground Truth
Databricks Certified Generative AI Engineer Associate · Test 1 · question 10 of 10
Before a release, a team builds an evaluation dataset of 300 real user questions from its production traces. The dataset has the inputs, the agent's outputs, and the retrieved context from each trace, but nobody has written reference answers yet. They plan to run mlflow.genai.evaluate() with built-in LLM judges.
Which built-in judge cannot produce a meaningful result until the team adds expected answers (expectations) to the dataset?
Show answer and explanation
Answer:
Correctness asks whether the response is correct compared with the provided ground truth, so it needs expectations (expected facts or an expected response) for each row. Without a reference there is nothing to compare against. RetrievalSufficiency and ToolCallCorrectness also need ground truth.
Option A is wrong. RelevanceToQuery only needs the inputs and outputs, since it judges whether the answer addresses the question.
Option B is wrong. Safety judges the output content itself and needs no reference answer.
Option D is wrong. RetrievalGroundedness compares the response with the retrieved context recorded in the trace, which the dataset already has. It measures faithfulness to the context, not correctness against a reference.
Reference: Built-in LLM judges