AIF-C01 Practice Question 82: Precision Costly False Positives
AWS Certified AI Practitioner (AIF-C01) · Test 9 · question 2 of 10
A company uses a binary classifier to move incoming email to a spam folder. Only about 2% of messages are actually spam. After launch, customers complain that legitimate invoices are being silently filtered out. Reporting shows 98% accuracy. Which metric should the team prioritize to reduce the harm customers are reporting?
Show answer and explanation
Answer:
Legitimate mail landing in the spam folder is a false positive: a positive prediction ("this is spam") that is actually negative. Precision = TP / (TP + FP) measures how many of the messages the model flags are truly spam, and AWS notes that *"Precision is an important metric when the cost of a false positive is high."* Raising precision means fewer good messages are flagged (option D).
- A is wrong because recall = TP / (TP + FN) measures how many actual spam messages are caught. Optimizing it pushes the model to flag more mail, which usually creates more false positives. - B is wrong because with only 2% spam, a model that never flags anything would already score 98% accuracy. Accuracy hides the problem on an imbalanced dataset. - C is wrong because log loss evaluates the quality of predicted probabilities. It does not isolate the cost of false positives, and it is not the standard metric for this trade-off.
Reference: Metrics and validation