MLA-C01 Practice Question 16: Overfitting Diagnosis
AWS Certified Machine Learning Engineer – Associate (MLA-C01) · Test 2 · question 6 of 10
An ML engineer trains a SageMaker AI XGBoost model with max_depth set to 12 and num_round set to 2000. The training logs show training AUC climbing steadily to 0.99. Validation AUC peaks at 0.87 around round 150 and then declines slowly, ending at 0.81 by the final round. The training and validation splits were drawn from the same time period, and the feature list has been reviewed for leakage.
Which change is most likely to improve the model's performance on unseen data?
Show answer and explanation
Answer:
This is textbook overfitting: training performance keeps improving while validation performance peaks and then degrades, leaving a widening gap between them. The model has started memorizing noise in the training data. The remedies all reduce effective model flexibility: shallower trees, stronger regularization (for XGBoost, lambda, alpha, gamma, or min_child_weight), and early stopping, which halts training near the point where validation performance was best. SageMaker AI XGBoost's early_stopping_rounds stops training when the validation score has not improved for the given number of rounds.
Option A pushes in exactly the wrong direction. More rounds and deeper trees give the model even more room to fit training noise, widening the gap between training and validation.
Option B removes the only instrument that detected the problem. Training AUC on data the model has seen is not an estimate of performance on unseen data.
Option C is the prescription for *underfitting*, where the model performs poorly on the training data too. Here training AUC is 0.99, so the model is not too simple; making it more flexible would make the overfitting worse.
References: What is Overfitting?; XGBoost hyperparameters