Databricks ML Professional Practice Question 19: Validate Evaluation Thresholds
Databricks Certified Machine Learning Professional · Test 2 · question 9 of 10
A nightly retraining job must stop automatically, before any registration step, whenever the newly trained classifier scores below an F1 of 0.80 on the holdout set. No person is available overnight to inspect results, and a failed task must surface as a failed job run.
Which implementation meets this?
Show answer and explanation
Answer:
Evaluation and validation are two separate calls in current MLflow. mlflow.evaluate computes and logs the metrics, and mlflow.validate_evaluation_results checks that result against validation_thresholds, a mapping of metric name to MetricThreshold. When a threshold is not met the call raises, the notebook task fails, and the job run is marked failed, so the pipeline stops before registration with no human involved, which is exactly the requirement.
Option A uses a parameter that no longer exists. The baseline_model argument was removed from mlflow.evaluate; comparison against a baseline is now expressed through the separate validation call.
Option B detects the problem hours after the model has already been registered, and it puts a person in a loop the requirements say must be unattended overnight.
Option D adds a deliberate human gate, which contradicts the same constraint. An approval task also sits in the deployment job that runs after a version is registered, so it cannot prevent the registration the question asks to avoid.
Reference: Model evaluation and validation (MLflow)