MLA-C01 Practice Question 53: Time Ordered Data Split
AWS Certified Machine Learning Engineer – Associate (MLA-C01) · Test 6 · question 3 of 10
A grocery chain trains a model that predicts next week's sales for each store. The dataset holds five years of weekly rows for 300 stores, sorted by week from oldest to newest. The team split it 80/20 into training and validation data at random. Validation error was excellent, but forecasts for the weeks after deployment were much worse. The team is preparing the data in Amazon SageMaker Canvas and wants the validation score to reflect how the model will actually be used.
Which split transform should the team apply?
Show answer and explanation
Answer:
A forecasting model is always used to predict weeks *after* the data it was trained on. A random split mixes future weeks into training, so the model learns from weeks that come after the ones it is validated on and scores unrealistically well. Canvas's ordered split keeps the rows' existing order: with the data sorted by week, the first 80% of rows go to training and the last 20% to validation. The model is then validated only on weeks later than anything it trained on, just as in production (option D).
- A is wrong because a different seed is still a random split. Future weeks still leak into training, whatever the seed. - B is wrong because stratifying balances sales levels across the sets but still picks rows without regard to time, so the same future-into-past leak remains. - C is wrong because splitting by store tests the model on *stores* it never saw, over the same weeks it trained on. The chain forecasts its existing stores into the future, so this measures the wrong thing and still mixes time periods.
Reference: Transform data - Split data (Amazon SageMaker Canvas)