Databricks ML Associate Practice Question 29: RMSE vs MAE Outliers
Databricks Certified Machine Learning Associate · Test 3 · question 9 of 10
A model predicts parcel delivery time in minutes. Most deliveries fall between 20 and 90 minutes, but a small number of shipments are delayed for days by weather or customs incidents that no feature in the dataset records. Operations want the metric that best reflects how far off a typical delivery estimate is, and they accept that the rare incidents are not predictable.
Which evaluation metric should they select the model on?
Show answer and explanation
Answer:
Mean absolute error sums the magnitudes of the errors, so a delivery missed by four thousand minutes contributes four thousand. Squared-error metrics contribute its square, which lets a handful of unpredictable incidents dominate the total and drive model selection toward whichever model happens to hedge against events it cannot predict. Since operations explicitly care about the typical estimate and accept that the incidents are out of scope, the linear penalty is the metric that matches the goal.
Option A gets the consequence of squaring backwards. Squaring amplifies large errors, making RMSE *more* sensitive to the outliers and less reflective of the typical delivery.
Option B misstates what R² is robust to. Being unitless is not the same as being robust: R² is computed from squared errors, so the same extreme residuals dominate it.
Option D confuses the training objective with the selection criterion. A model may optimise squared error internally while still being chosen on the metric the business cares about, and MSE has the identical outlier sensitivity as RMSE.
Reference: Metrics and scoring: regression metrics (scikit-learn)