MLA-C01 Practice Question 6: Multi Model Endpoint Tenants
AWS Certified Machine Learning Engineer – Associate (MLA-C01) · Test 1 · question 6 of 10
A SaaS provider trains a separate XGBoost model for each of its 4,000 tenants. Every model artifact is about 30 MB and uses the same serving container. Most tenants send only a handful of predictions per day in short bursts, but a few are busy all day. The provider currently runs 4,000 single-model real-time endpoints and the hosting bill is unsustainable. Occasional first-call latency for a rarely used tenant is acceptable.
What is the most cost-effective way to host these models?
Show answer and explanation
Answer:
Multi-model endpoints are purpose-built for hosting large numbers of models that share one ML framework and serving container. All models live under an S3 prefix; SageMaker AI downloads and caches a model in container memory on first invocation, keeps routing that tenant's traffic to the instance holding it, and unloads idle models when memory is tight. That turns 4,000 fleets into one, and it matches the described traffic profile, a mix of hot and cold models where a cold-start penalty on infrequently used models is tolerable. Adding or removing a tenant is just an S3 upload or delete; the endpoint does not need to be updated.
Option B does not scale to this shape. Serverless Inference is limited to 50 serverless endpoints per Region per account, so 4,000 of them is impossible without an extraordinary quota change, and the per-endpoint management overhead would remain.
Option C changes the product. Nightly batch predictions cannot answer a request about a transaction that arrives during the day, and the stem describes on-demand predictions, not a precomputed lookup table.
Option D misuses the feature. Production variants exist for A/B testing and gradual rollout of different versions of the *same* logical model, they are weighted rather than addressable per tenant, and an endpoint supports only a small number of them, nothing close to 4,000.
Reference: Multi-model endpoints