MLA-C01 Practice Question 61: Point in Time Feature Join
AWS Certified Machine Learning Engineer – Associate (MLA-C01) · Test 7 · question 1 of 10
A subscription company stores customer features, such as support tickets in the last 30 days and logins per week, in a SageMaker Feature Store feature group with the offline store enabled. A new record is written for each customer every day, with an event time. To build a churn training set, an engineer joined a labels table (customer_id, snapshot_date, churned_within_60_days) to the *most recent* feature record for each customer. Validation AUC was 0.95, but the deployed model scores 0.71 on new customers, and no data drift was found.
How should the engineer rebuild the training set?
Show answer and explanation
Answer:
Joining every label to the *latest* feature values leaks the future. A customer who churned in March has features recorded after March, such as a burst of support tickets and zero logins, that the model could never see at prediction time. The model learns from that leak, and validation looks far better than production. Feature Store's dataset builder can create a point-in-time accurate join from the offline store, which uses each record's event time to join only feature values that existed at the label's timestamp. The training rows then match what the model will see in production (option B).
- A is wrong because the online store holds only the *latest* record for each customer, which is exactly the leak that caused the problem. - C is wrong because an event time range filters which records are included but does not align them to each label's date. Features from after a snapshot still reach that snapshot's row. - D is wrong because the earliest record removes the leak by pairing labels with features that may be years stale. The model then trains on inputs that look nothing like what it sees in production.
Reference: Create a dataset from your feature groups