MLA-C01 Practice Question 9: Model Monitor No Ground Truth
AWS Certified Machine Learning Engineer – Associate (MLA-C01) · Test 1 · question 9 of 10
A credit-risk model is served from a real-time endpoint. The true outcome for any given application, whether the borrower defaulted, is not known until at least 60 days after the prediction is made. The ML team wants an automated CloudWatch alarm within days if the distribution of the input features arriving at the endpoint starts to diverge from the distribution of the training data, and they want to configure it with as little extra setup as possible.
Which SageMaker AI Model Monitor capability should they schedule?
Show answer and explanation
Answer:
Data quality monitoring is the only one of the four Model Monitor types that answers the question being asked and works without labels. You enable data capture on the endpoint, run a baseline job over the training dataset to compute statistics and suggested constraints, and schedule a monitoring job that compares captured request data against those constraints, reporting violations and emitting CloudWatch metrics you can alarm on. Feature distributions in production are available immediately, so alerts arrive in days rather than months.
Option A cannot run. Model quality monitoring compares predictions with actual ground truth labels that you ingest into S3 and that Model Monitor merges with the captured inference data. With labels 60 days behind, a daily model quality job has nothing to compare against. It is worth adding later, once outcomes start landing, but it cannot be the early-warning signal.
Option C measures something else. Bias drift monitoring watches bias metrics of the model's predictions across defined facet values (a fairness question, not a general input-distribution question), and requires configuring facets and a Clarify bias baseline.
Option D is closer, because feature attribution drift is also label-free, but it tracks whether the *importance ranking* of features has shifted rather than whether the incoming feature values have moved away from the training distribution. It also requires a Clarify explainability baseline, making it the heavier setup the stem rules out.
A note on product status: AWS's documentation now says SageMaker Model Monitor is no longer open to new customers, though existing customers can keep using it. It remains in the MLA-C01 exam guide, which is why it is tested here.
Reference: Data and model quality monitoring with Amazon SageMaker Model Monitor