MLA-C01 Practice Question 29: Endpoint Latency Metric Diagnosis
AWS Certified Machine Learning Engineer – Associate (MLA-C01) · Test 3 · question 9 of 10
Users of an application in eu-west-1 report that predictions take about 1.2 seconds at the 99th percentile, against a 400 ms target. The SageMaker AI real-time endpoint is in us-east-1. CloudWatch shows p99 ModelLatency of 90 ms and p99 OverheadLatency of 40 ms for the endpoint, CPUUtilization steady near 25%, and zero Invocation5XXErrors. The instance count has not changed.
What is the most likely cause of the gap between the reported time and the endpoint metrics?
Show answer and explanation
Answer:
Total time as the client sees it has three parts: network latency to and from the SageMaker Runtime API, OverheadLatency (transporting the request to the model container and the response back), and ModelLatency (the container processing the request). ModelLatency plus OverheadLatency is 130 ms here, so roughly a second is unaccounted for, and the one component CloudWatch does not measure is the cross-Atlantic network leg (option C). The fix is to host the endpoint in the Region where the callers are.
- A is wrong because a CPU-bound container shows up as high ModelLatency and high CPUUtilization. Both are low, and 90 ms of model time is well inside the target. - B is wrong because queueing inside the endpoint would raise OverheadLatency and push CPUUtilization up. At 25% utilization the fleet is not saturated. - D is wrong because throttled InvokeEndpoint calls return HTTP 429 errors that surface in Invocation4XXErrors, and the client would see failures rather than uniformly slow successes.