Microsoft Foundry Generative AI Chat App Practice Question 4: Groundedness Evaluation Dataset
Microsoft Applied Skills: Develop a Generative AI Chat App Using the Microsoft Foundry SDK · Test 1 · question 4 of 10
Before release, a team exports 500 rows from their retrieval-grounded chat app into a JSONL file. Each row holds the user query, the response the app already produced, and the context passages that were retrieved for it. Reviewers complain that some answers state facts the retrieved passages never contained, even though the answers read well and address the question asked. In the Foundry portal, which evaluation should you run?
Show answer and explanation
Answer:
The Dataset target "evaluates preexisting model or agent outputs from a test dataset" in CSV or JSONL format, which is what the team has; the responses are already generated, so there is no reason to re-run the model. The Groundedness evaluator "measures whether responses are grounded in provided context," and context is the optional field mapped for exactly this RAG scenario. Answers that invent facts absent from the retrieved passages are ungrounded, which is the symptom described.
- A is wrong twice over: a Model evaluation re-runs the model against generated prompts rather than scoring the exported answers, and Fluency "evaluates natural language quality." The complaint is that the answers read well, so fluency would score high. - B picks the right target but the wrong evaluator. Relevance "evaluates how relevant responses are to the query." The stem says the answers do address the question asked; the problem is their relationship to the retrieved context, not to the query. - D is wrong because Coherence "measures logical flow and consistency of responses," which a fabricated but well-written answer passes. A Model target also regenerates outputs instead of scoring the ones reviewers flagged.
Reference: Run evaluations from the Microsoft Foundry portal