AIF-C01 Practice Question 11: Recall Missed Fraud
AWS Certified AI Practitioner (AIF-C01) · Test 2 · question 1 of 10
A bank's binary classifier flags card transactions as fraudulent so analysts can review them. A missed fraudulent transaction costs the bank far more than an analyst spending a few minutes on a legitimate one. The team wants the evaluation metric that best shows how many of the actual fraud cases the model catches. Which metric should they prioritize?
Show answer and explanation
Answer:
Recall is TP / (TP + FN): of all the transactions that really are fraud, how many did the model catch? A false negative here is fraud that slipped through, and that's the expensive mistake. So recall is the metric to prioritize (option B). AWS describes recall as important when you need to find all of the true positives, as in cancer screening.
- A is wrong because precision is TP / (TP + FP). It matters when a false positive is the costly error. Here, extra reviews of legitimate transactions are cheap. - C is wrong because fraud is rare. A model that labels every transaction "legitimate" can reach 99%+ accuracy while catching no fraud at all. - D is wrong because mean squared error is a regression metric for numeric predictions. It doesn't apply to a yes/no fraud label.
Reference: Metrics and validation