GCP Professional ML Engineer Practice Question 8: Serverless ML Pipeline Orchestration
Google Cloud Professional Machine Learning Engineer · Test 1 · question 8 of 10
An MLOps team has written its retraining workflow (data validation, training, evaluation, conditional model registration) as components in the Kubeflow Pipelines SDK. They want a serverless runner that tracks the lineage of the artifacts each step produces, skips steps whose inputs have not changed, and requires no long-lived orchestration infrastructure.
Which option should they choose?
Show answer and explanation
Answer:
Agent Platform Pipelines (formerly Vertex AI Pipelines) runs pipelines defined with the Kubeflow Pipelines or TFX SDK in a serverless manner; there is no cluster to provision or keep alive. It stores every artifact each task produces in ML Metadata, which is what gives you artifact-level lineage across runs, and it supports execution caching so a task whose inputs and definition are unchanged is skipped rather than re-executed. That is a point-for-point match with the three requirements, and the team's existing KFP components run as-is.
Option A gives you neither lineage nor caching. Pub/Sub-chained Cloud Run jobs are a hand-built DAG with no artifact model, no run graph, and no way to reason about which step produced which model without writing that bookkeeping yourself.
Option C is a real orchestrator but the wrong one here. Cloud Composer means an Airflow environment whose nodes run, and bill, continuously, and Airflow's task-level metadata is not ML artifact lineage. It also throws away the KFP components the team has already written.
Option D self-hosts what Agent Platform Pipelines provides as a managed service: the team would own the GKE cluster, the Kubeflow installation, and its upgrades, which is the opposite of the "no long-lived orchestration infrastructure" requirement.
Reference: Introduction to Gemini Enterprise Agent Platform Pipelines