AIF-C01 Practice Question 12: Overfitting Train Test Gap
AWS Certified AI Practitioner (AIF-C01) · Test 2 · question 2 of 10
A data scientist reports that a new demand-forecasting model scores 98% accuracy on its training data but only 64% on a held-out test set it has never seen. What is the most likely explanation for this result?
Show answer and explanation
Answer:
Overfitting is when a model predicts the training data well but not new data. The signature is exactly this one: low error on training data, high error on test data, and a large gap between the two. An overfit model has high variance. It has learned noise and quirks of its training set instead of patterns that generalize (option C). Typical remedies are more (or more diverse) training data, early stopping, regularization, and simpler features.
- A is wrong because an underfit model does poorly on the training data too. 98% training accuracy rules that out. - B is wrong because drift describes production data changing after deployment. This gap appears on a held-out test set before the model is ever deployed. - D is wrong because high bias is the underfitting pattern: consistently poor results everywhere. This model does very well on one dataset and poorly on another.
Reference: What is Overfitting?