MLA-C01 Practice Question 77: Inference Components Shared GPUs
AWS Certified Machine Learning Engineer – Associate (MLA-C01) · Test 8 · question 7 of 10
A company hosts six fine-tuned language models, each on its own real-time endpoint backed by an ml.g5.12xlarge instance with four GPUs. The models differ in size: two need all four GPUs, and four need only one. Traffic varies by model, and two of the models get no requests for days at a time. GPU utilization averages under 15%. The company wants to share GPU instances, reserve a set number of GPUs and amount of memory for each model, scale each model's copies independently, and let the rarely used models drop to zero copies.
Which hosting design meets these requirements?
Show answer and explanation
Answer:
An inference component is a hosting object that deploys one model to an endpoint and states how that model uses the endpoint's resources, including CPU cores, accelerators, and memory. Several inference components can share an endpoint's instances. For each one you choose how many copies to run, scale those copies independently, and can scale a copy down to zero to make room for others. That meets every requirement: shared GPU instances, per-model GPU and memory reservations, independent scaling, and zero copies for idle models (option B).
- A is wrong because a multi-model endpoint loads and unloads models dynamically in one shared serving container. It does not let you reserve a specific number of GPUs per model or scale each model's copies independently, and a model needing all four GPUs fits that pattern poorly. - C is wrong because a multi-container endpoint places every container on each of its instances. Containers do not get their own copy counts or independent scaling. - D is wrong because serverless inference does not support GPUs, and these models need them.
Reference: Deploy models for real-time inference - Inference components