MLA-C01 Practice Question 18: Endpoint Target Tracking Scaling
AWS Certified Machine Learning Engineer – Associate (MLA-C01) · Test 2 · question 8 of 10
A retailer hosts a GPU-based recommendation model on a SageMaker AI real-time endpoint. Load testing showed that one instance handles about 400 invocations per minute within the latency target. Traffic climbs from about 300 to 3,000 invocations per minute every morning, but the timing and size of the peak vary from day to day with promotions. The team wants the instance count to follow demand automatically, and it has seen instances removed and re-added within minutes during brief lulls.
What is the most appropriate way to configure scaling?
Show answer and explanation
Answer:
Target tracking is the standard SageMaker AI endpoint scaling policy. You register the production variant as a scalable target and choose a metric and a target value; Application Auto Scaling adds or removes instances to keep the metric at that value. The predefined SageMakerVariantInvocationsPerInstance metric maps directly to the load test, so a target somewhat below 400 leaves latency headroom. Because the policy reacts to actual load, it follows peaks whose timing and size vary. Setting ScaleInCooldown longer than ScaleOutCooldown makes the policy slow to remove capacity and quick to add it, which stops the flapping during brief lulls.
Option A rebuilds auto scaling by hand. Deploying a new endpoint configuration for every change is slow and error-prone, and CPU is a weak proxy for load on a GPU model.
Option B rules itself out: serverless inference does not support GPUs, and the model is GPU-based.
Option D only works when demand follows the clock. Scheduled actions are useful alongside target tracking for known events, but on their own they cannot track a peak whose timing and size change every day.
Reference: Define a scaling policy