Databricks ML Associate Practice Question 25: Feature Scaling Linear Model
Databricks Certified Machine Learning Associate · Test 3 · question 5 of 10
A pipeline fits an L2-regularised LogisticRegression on features that include annual_income (tens of thousands), age (tens) and visits_last_week (single digits). A colleague points out that the random forest they built last month performed well with no scaling at all, and suggests skipping the scaler here too.
What should the team do, and why?
Show answer and explanation
Answer:
The colleague's observation about the random forest is correct but does not transfer. A tree splits on thresholds within a single feature, so multiplying a column by a thousand changes nothing about which splits are chosen; trees are genuinely scale-invariant. A penalised linear model is not. L2 shrinks the sum of squared coefficients, and a feature measured in tens of thousands needs a tiny coefficient to contribute the same effect as a single-digit feature, so the penalty falls far more heavily on the small-scale features. Standardising puts every column on comparable footing so the penalty reflects predictive value rather than units. The same reasoning applies to distance-based methods such as k-nearest neighbours and to gradient descent convergence.
Option A states the opposite of the property that matters. Scale invariance is a property of trees, not of regularised linear models.
Option B addresses the wrong side of the equation. The penalty acts on the coefficients attached to features; transforming the target changes what is predicted without equalising the inputs.
Option D defers a fix that is needed now. Neural networks also benefit from scaled inputs, but the current model is already the kind that requires it.
Reference: Preprocessing data: standardization (scikit-learn)