AI-300 Practice Question 89: Evaluation Quality Gate

Microsoft Machine Learning Operations Engineer Associate (AI-300) · Test 9 · question 9 of 10

Implement generative AI quality assurance and observabilitymedium

A team changes the prompts of its assistant several times a week. They want to stop a change that lowers answer quality from being merged. Which approach should they adopt?

Select one answer
Show answer and explanation

Answer:

  • B. Run a batch evaluation on a fixed dataset in the pipeline, failing below set thresholds

Foundry observability supports integrating automated quality gates into CI/CD pipelines. Running the same evaluators on a fixed dataset for each change gives comparable scores, and failing the build below a threshold blocks the regression before it reaches users (option B).

- A is wrong because manual spot checks are inconsistent and cover very little of the behavior, so regressions can slip through. - C is wrong because this detects a regression only after users are already affected, and it does not stop a bad merge. - D is wrong because a delayed review is monitoring after the fact, not a gate in front of the merge.

Reference: Observability in generative AI

Practice Test 9 ›

Study guides for AI-300