AI-300 Practice Question 14: Batch Endpoint Selection
Microsoft Machine Learning Operations Engineer Associate (AI-300) · Test 2 · question 4 of 10
Every night a retailer scores about 40 million customer records that sit as parquet files in a data lake. Nothing waits on an individual prediction (the run only has to finish before the morning reports start), and the team wants to pay for inference compute only while the run is happening. Which deployment should you create?
Show answer and explanation
Answer:
Batch endpoints "receive pointers to data and run jobs asynchronously to process the data in parallel on compute clusters", and Microsoft's own list of when to use them matches every constraint here: inference over large amounts of data distributed in multiple files, no low-latency requirement, inputs already in a storage account, and work that parallelizes. The cost shape fits too: Azure Machine Learning "automatically provisions compute resources when the job starts, and automatically deallocates them as the job completes", and the docs recommend scale-to-zero on the cluster "to ensure no resources are consumed when they're idle" (option D).
- A is wrong because an online endpoint is built for low-latency synchronous requests. Scoring the file set means tens of millions of HTTP calls, and an online deployment's instances are billed for as long as the deployment exists, including the 22 idle hours. - B is wrong because a compute instance is a single-node development target, not a target for online deployments; managed online deployments run on instance types that the deployment specifies. - C is wrong because a Foundry serverless API deployment serves foundation models from the model catalog, not a custom model registered in an Azure Machine Learning workspace, and per-record invocation has the same synchronous shape problem as option A.
Reference: What are batch endpoints? - Azure Machine Learning