Databricks ML Associate Practice Question 22: AutoML Chronological Split
Databricks Certified Machine Learning Associate · Test 3 · question 2 of 10
A data scientist runs Databricks AutoML on two years of weekly demand records to forecast next quarter's demand. The best trial reports excellent validation error, but when the model is applied to the most recent quarter its error is far worse. The default random split placed rows from every period into all three sets.
What should they change on the AutoML run?
Show answer and explanation
Answer:
With a random split, rows from the most recent quarter land in the training set alongside rows the model is later validated on, so the model is scored on periods it has effectively already seen. The validation figure measures interpolation between known weeks rather than forecasting. Selecting a time column switches AutoML to a chronological split: the earliest points become training data, the next earliest validation, and the latest points the test set. That mirrors how the model will be used and produces an honest estimate. The time column may be a timestamp, an integer or a string.
Option A confuses row order with split logic. A random split samples rows regardless of how the table is sorted, so pre-sorting has no effect on which rows go where.
Option C treats a leakage problem as an overfitting-by-search problem. Fewer trials only means less searching; every trial is still scored against a validation set that contains future periods.
Option D removes a legitimate predictor and leaves the flawed split intact. The issue is not that the model sees time, it is that validation rows come from the same periods as training rows.
Reference: AutoML data preparation and splitting