GCP Professional ML Engineer Practice Question 6: Batch Versus Online Inference
Google Cloud Professional Machine Learning Engineer · Test 1 · question 6 of 10
A propensity model must score roughly 60 million customer rows held in BigQuery once per night, with the scores written back to BigQuery before the 06:00 marketing run. No application calls the model interactively, and the team wants to minimize cost.
How should the scoring job be run?
Show answer and explanation
Answer:
Agent Platform offers two ways to get inferences, and they map onto exactly this distinction. Online inference is a synchronous request to a model deployed on an endpoint, with compute resources attached so it can answer with low latency. Batch inference is an asynchronous BatchPredictionJob sent to the model resource itself, with no endpoint involved; you use it when you do not need an immediate response and want to process accumulated data in a single request. A nightly bulk scoring run with a deadline rather than a latency SLO is the canonical batch case, and it supports BigQuery as both source and destination.
Option B pays for a permanently deployed endpoint and then makes 60 million round trips against it. Per-request overhead dominates, autoscaling reacts to load rather than anticipating it, and the cost is far above a batch job that provisions resources only for the duration of the run.
Option C adds a Dataflow job whose only purpose is to fan requests at an endpoint, two services and two bills to reimplement what batch inference already does natively.
Option D is closer in spirit but is a hand-rolled version of batch inference. You now own the deploy/undeploy orchestration, the failure handling if scoring overruns the window, and the endpoint charges for the whole scoring period, all to reach a worse version of option A.