Databricks ML Associate Practice Question 28: Random vs Grid Search
Databricks Certified Machine Learning Associate · Test 3 · question 8 of 10
A team is tuning a gradient-boosted model with six hyperparameters. Four are continuous, such as the learning rate and the subsample fraction, and previous projects suggest only two or three of the six matter much, but not which ones. The cluster budget allows about 60 model fits.
Which search strategy makes the best use of that budget?
Show answer and explanation
Answer:
Random search is the better use of a fixed budget whenever only a few hyperparameters drive performance and you do not know in advance which. Because every trial draws a fresh value for every hyperparameter, 60 trials explore 60 distinct values of each, including the ones that matter. A grid reuses the same handful of values for every hyperparameter, so its coverage of any single dimension is only as fine as that axis, and most of the budget is spent varying parameters that change nothing. Sampling from distributions also handles the four continuous parameters naturally rather than forcing them onto arbitrary discrete points.
Option A spends the whole budget to try just two values per hyperparameter. For a continuous learning rate that is a very coarse look, and the grid grows exponentially, so refining any axis is immediately unaffordable.
Option B guesses at the answer the team admits it does not have. Fixing four hyperparameters at defaults guarantees the search cannot discover that one of them was among the important ones.
Option C is the least efficient and the least reproducible. One-at-a-time adjustment cannot see interactions between hyperparameters and depends on the order in which they happen to be tried.
Reference: Tuning the hyper-parameters of an estimator (scikit-learn)