AI-300 Practice Question 5: Data Drift Monitoring
Microsoft Machine Learning Operations Engineer Associate (AI-300) · Test 1 · question 5 of 10
A demand-forecasting model has been live on an Azure Machine Learning managed online endpoint for three months. Ground truth labels arrive months later, so accuracy can't be measured yet, but the team wants an alert as soon as the incoming feature values stop resembling what the model was trained on. How should the monitor be set up?
Show answer and explanation
Answer:
Azure Machine Learning model monitoring computes signals by comparing production inference data against reference data, so the first step for a model on an online endpoint is to enable production inference data collection. The data drift signal then compares the distribution of each incoming feature against a baseline and raises an alert when a statistical test or distance score crosses your threshold. Microsoft's own guidance is to use training data as the comparison baseline for data drift, which is what option B configures, and it needs no labels.
- A is wrong because latency is an operational metric about the service, not the data. A model can drift badly while responding in five milliseconds. - C is wrong because re-scoring a fixed validation set produces the same accuracy every week. The validation data never changes, so it can't reveal a shift in live inputs. - D is wrong on two counts: the model performance signal requires ground truth data, which this team doesn't have yet, and recent production data is not ground truth.
Reference: Model monitoring in production - Azure Machine Learning