GCP Professional ML Engineer Practice Question 3: Target Leakage Offline Online Gap

This is textbook target leakage: the training data contains predictive information that does not exist at inference time. manual_review_outcome is effectively a delayed restatement of the label, so the model learns to read it instead of learning fraud, which is why the offline metric is spectacular and the online behaviour is not. Google's tabular training-data guidance states the rule plainly: every value used to train the model must be present when you request an inference. Removing the leaked columns and re-evaluating gives you the first honest estimate of the model's real quality; only after that does tuning mean anything.

Option B misreads the symptom. Class imbalance depresses recall on the minority class and makes accuracy misleading; it does not produce a 0.98 AUC offline paired with chance-level behaviour online. Rebalancing a leaked dataset simply reproduces the leak on a different sample.

Option C treats this as overfitting, but the model is not overfitting: it generalizes perfectly to the held-out split, because the held-out split carries the same leaked column. Regularization constrains capacity; it cannot remove information that legitimately predicts the label inside the dataset but is absent in production.

Option D monitors the wrong thing. Prediction drift compares recent serving data to an earlier production baseline. Nothing here has drifted, the model was never valid, and retraining on the same leaked table would only regenerate the same broken model.

Reference: Best practices for creating tabular training data