Databricks ML Associate Practice Question 18: Train Validation Gap
Databricks Certified Machine Learning Associate · Test 2 · question 8 of 10
A gradient-boosted classifier scores 0.99 accuracy on the training set and 0.72 on the validation set. As the data scientist adds trees and increases the maximum depth, the training score creeps toward 1.00 while the validation score slowly falls.
What should they do next?
Show answer and explanation
Answer:
A large and widening gap between training and validation performance is the signature of overfitting: the model is memorising noise specific to the training rows rather than learning structure that generalises. The remedy is to constrain it: fewer or shallower trees, a lower learning rate, stronger L1/L2 penalties, subsampling, or simply more training data, and then confirm on the validation set that the gap narrows and the validation score rises.
Option B applies the treatment for underfitting to a case of overfitting. The trend described already shows that added depth makes validation performance worse.
Option C destroys the only honest estimate the team has. Training on the validation set guarantees a higher score on it while saying nothing about new data, which is the number that actually matters.
Option D reports the one figure known to be optimistic. Training accuracy measures performance on data the model has already seen; row count does not make it a generalisation estimate.
Reference: Underfitting vs. overfitting (scikit-learn)