Databricks ML Associate Practice Question 17: CrossValidator vs TrainValidationSplit
Databricks Certified Machine Learning Associate · Test 2 · question 7 of 10
A Spark ML tuning job searches a 36-combination parameter grid with CrossValidator(numFolds=3) over a 40-million-row training set. It fits 108 models and overruns the nightly window. The dataset is large enough that a single held-out validation set of several million rows gives a stable estimate, and the cluster is already at its maximum size.
Which change most reduces tuning time while still selecting on held-out data?
Show answer and explanation
Answer:
TrainValidationSplit makes one split, governed by trainRatio, and fits each parameter combination exactly once, 36 fits here rather than 108, roughly a third of the work. It is the cheaper tuning strategy, and its weakness is that the estimate rests on a single split, which matters when data is scarce. With millions of validation rows that weakness is not binding, so it trades away variance reduction the job does not need. Selection is still made on data the model did not train on.
Option A moves in the wrong direction. parallelism controls how many models are evaluated concurrently; setting it to 1 serialises the search and makes the job slower.
Option B reduces work but misstates the arithmetic and weakens the estimate. Two folds instead of three gives 72 fits, a third fewer rather than half, and two-fold cross-validation trains each model on only half the data.
Option D misreads the flag. collectSubModels retains the per-fold models in memory so they can be inspected afterwards; nothing is reused, no fits are skipped, and memory pressure goes up.
Reference: ML tuning: model selection and hyperparameter tuning (Spark ML)