Practice — Case Study: Real-Time Payment Fraud Detection (Stripe / PayPal-style) (5 questions)
Advanced
Open
Free
Why PR-AUC Beats ROC-AUC at 0.1% Fraud Prevalence Permalink →
Your fraud model is evaluated on 2,000,000 transactions containing 2,000 fraud cases (0.1 %). Model A flags 12,000 transactions and catches 1,600 fraud. Model B flags 5,000 transactions and catches 1,400 fraud.
- Compute the recall, false-positive rate, and precision for each model.
- Explain why the two models look almost identical on an ROC curve but very different on a precision-recall curve.
- Which model would you ship if manual review capacity is 5,000 transactions/day, and why?
Share this question