AI-300 Practice Question 100: Fine Tuning Overfitting
Microsoft Machine Learning Operations Engineer Associate (AI-300) · Test 10 · question 10 of 10
During fine-tuning, the training loss keeps falling, but the validation loss starts to rise after epoch 3 of 6. The model's answers on new prompts are getting worse. Which change is most appropriate for the next run?
Show answer and explanation
Answer:
A falling training loss with a rising validation loss is the sign of overfitting: the model memorizes the training set. Reducing the number of epochs stops training closer to the lowest validation loss, and a smaller learning rate multiplier helps to avoid overfitting (option D).
- A is wrong because more epochs on the same data would increase overfitting, and the validation loss would keep rising. - B is wrong because the validation set is what shows the overfitting. Removing it hides the problem but does not solve it. - C is wrong because a higher rate makes updates larger and usually worsens overfitting, while a smaller rate helps to avoid it.
Reference: Customize a model with fine-tuning