Databricks ML Professional Practice Question 4: Point in Time Feature Lookup
Databricks Certified Machine Learning Professional · Test 1 · question 4 of 10
A churn model reaches 0.95 AUC offline but only 0.71 in production. The labels DataFrame has customer_id, event_ts, and churned. Features come from the Unity Catalog feature table ml.features.customer_activity. That table gets a new row per customer every day, with a ts column. Its primary key is currently declared as customer_id only. The training set is built with fe.create_training_set and a FeatureLookup on lookup_key="customer_id".
What change most directly removes the likely cause of the gap?
Show answer and explanation
Answer:
The big gap between offline and production scores points to data leakage. With a key-only lookup, training rows can pick up feature values recorded after the label event, which the production model never has. Point-in-time lookup fixes this. You mark the timestamp column as a time series primary key (timeseries_columns in Unity Catalog) and pass timestamp_lookup_key in the FeatureLookup. The FeatureEngineering client then joins, for each label row, the latest feature value recorded before that row's event_ts.
Option A turns the lookup into an exact-match join on the timestamp. Label events almost never land on the exact daily snapshot time, so most features come back null. The docs also say timestamp columns should not be used as lookup keys.
Option C does not work on its own. lookback_window only applies to point-in-time lookups against a table with a time series key. Without that key the join is still an exact match on customer_id, so the leakage stays.
Option D uses a single global cutoff instead of one per row. It prevents future values, but every later label gets a feature value from before the earliest event. Those stale features create a different mismatch with production.
Reference: Point-in-time support using time series feature tables