Databricks ML Associate Practice Question 9: Log Target Evaluation
Databricks Certified Machine Learning Associate · Test 1 · question 9 of 10
To reduce skew, a house-price regressor is trained on log(price). On the validation set, the data scientist computes RMSE between the model's raw predictions and log(price) and gets 0.18. The current production model reports an RMSE of $41,000 on the same validation set.
What must they do before comparing the two models?
Show answer and explanation
Answer:
The model predicts in log units, so its 0.18 RMSE is in log units too. To compare it with a dollar-denominated RMSE, or to interpret any prediction, apply the inverse transform (exp, or expm1 if log1p was used) to the predictions. Then compute RMSE against the untransformed prices. Both numbers are then in dollars and measure the same thing.
Option B is false. RMSE is expressed in the units of the target, so it changes with any transformation of the target. A log-scale error and a dollar error cannot be compared.
Option C does not produce a valid comparison. RMSE is an average over rows, and the log of an average error is not the average error on the log scale. The numbers would still not be comparable.
Option D is also false. R-squared depends on the scale of the target, so a model scored against log(price) and one scored against price generally get different values. Exponentiating first is still required.
Reference: Transforming the target in regression (scikit-learn)