Databricks ML Professional Practice Question 15: Unit vs Integration Tests
Databricks Certified Machine Learning Professional · Test 2 · question 5 of 10
A retraining pipeline is deployed with a bundle and its code lives in a Git repository. The team wants two things: a pull request must fail within seconds if someone breaks a feature-engineering transformation, and nothing may reach production until the full pipeline has been proven to run end to end against real tables.
Which testing strategy meets both requirements?
Show answer and explanation
Answer:
The two requirements describe the two ends of the testing pyramid, and each needs its own gate. Unit tests run against small fixtures with no cluster, so a broken transformation fails the pull request in seconds; that is the fast feedback the first requirement asks for. Integration testing means deploying the bundle to a staging target and actually running the job against representative tables, which is the only way to catch schema drift, permission gaps and wiring errors that pure functions cannot expose. Passing staging becomes the gate on promoting the same bundle to the production target.
Option A satisfies only the second requirement. Integration runs need real compute and real data and take minutes to tens of minutes, so putting them on every pull request destroys the fast feedback loop, and a failure points at the pipeline rather than the function.
Option B satisfies only the first. Unit tests exercise functions in isolation and cannot detect a renamed upstream column or a missing grant, so an untested pipeline reaches production.
Option D tests after the risky step has already happened. Finding a failure once the production deployment is live is exactly the outcome a gate is meant to prevent, and manual runs get skipped under deadline pressure.
Reference: CI/CD for machine learning on Databricks