Databricks ML Professional Practice Question 24: MLflow Dataset Lineage
Databricks Certified Machine Learning Professional · Test 3 · question 4 of 10
An auditor asks which exact table and table version produced version 9 of a production model. The training runs log hyperparameters, metrics and the model artifact, but nothing that identifies the input data, and the source Delta table is rewritten weekly.
What should the team add to the training code so this question is answerable for future versions?
Show answer and explanation
Answer:
MLflow's dataset tracking exists for exactly this question. Constructing a dataset from the training source and passing it to mlflow.log_input records the source location, the version and a digest on the run, so a model version can be traced back to the precise data that produced it. Because the run is the link between the model version and the dataset, the answer survives the weekly rewrite of the underlying table; the recorded version pins which state of it was used.
Option B captures the table's name but not its version, which is the part the auditor needs when the table is rewritten weekly. It is also applied after the fact by hand, so it drifts.
Option C looks at lineage from the wrong end. Delta history records the operations that *wrote* the table; it does not record readers, so it cannot say which run consumed which version.
Option D answers the question at enormous cost and still answers it poorly. Copying the full training set into artifacts on every run duplicates the data indefinitely, and a loose copy carries no reference to the version it came from.
Reference: Track datasets with MLflow