Databricks ML Professional Practice Question 7: Baseline Drift Statistical Tests
Databricks Certified Machine Learning Professional · Test 1 · question 7 of 10
A credit model's endpoint logs requests to an inference table. A job unpacks the logs into a Delta table, and an inference profile (Lakehouse Monitoring, now called data profiling) runs on that table with daily granularity. Ground-truth labels arrive about 30 days after each prediction. Risk officers want an alert within days when production inputs move away from what the model was trained on. Two inputs matter: the numeric income and the categorical channel. Drift must be measured against the training data, and it must also be caught when it builds up slowly over many days.
Which configuration meets these requirements?
Show answer and explanation
Answer:
The drift metrics table (<table>_drift_metrics) has two kinds of rows. CONSECUTIVE rows compare a window with the one before it. BASELINE rows compare a window with the baseline table, and they are only calculated when a baseline table is set. Setting the training data as the baseline measures each window against what the model learned from. That catches drift that builds up slowly, which consecutive comparisons miss because each day looks like the day before. The statistical tests depend on column type. ks_test (a statistic/pvalue struct) is computed for numeric columns. chi_squared_test is computed only for categorical columns. You then create a Databricks SQL alert on those p-values.
Option A compares only neighbouring windows and never looks at the training data. It fails both the "against training data" requirement and the slow-drift requirement.
Option B measures model quality, not input drift. With labels arriving 30 days late, it cannot alert within days.
Option D is close, but the Kolmogorov-Smirnov test is only computed for numeric columns. For the categorical channel it is null, so drift in channel would never trigger the alert.
References: Data profiling metric tables; Profile alerts