AI-300 Practice Question 13: Sweep Early Termination

Microsoft's guidance on choosing a policy names this configuration directly: "For a conservative policy that saves resources without terminating promising jobs, consider a Median Stopping Policy with evaluation_interval set to 1 and delay_evaluation set to 5. These settings are conservative and can provide about 25%-35% savings with no loss on the primary metric" (option C). Median stopping compares a trial against the median of the running averages of all trials, and delay_evaluation=5 gives every configuration five reported intervals before it can be judged, which is what protects the slow starter.

- A is wrong because the same guidance puts Bandit with a smaller allowable slack at the aggressive end. A slack_factor of 0.05 ends any trial whose best metric falls below roughly 95 percent of the leader's, and with no delay_evaluation that judgement happens at the very first reported metric, the premature kill the team already suffered. - B is wrong because truncation selection cancels the lowest-performing percentage at every interval, and Microsoft lists a larger truncation percentage as the aggressive choice. At 50 percent it kills half the field each interval, again with no delay. - D is wrong because it saves money by searching less rather than by stopping unpromising trials. Cutting a 60-trial sweep to 20 reduces the chance of finding the best configuration at all, and every surviving trial still runs to completion.

Reference: Hyperparameter tuning a model (v2) - Azure Machine Learning