Intermediate
Open
Pro
ROC-AUC vs PR-AUC for a Rare-Event Model
Two candidate models for a 0.2%-prevalence account-takeover detector are evaluated on the same 500,000-session test set:
| Model | ROC-AUC | Average precision |
|---|---|---|
| A | 0.962 | 0.18 |
| B | 0.948 | 0.31 |
- Explain what ROC-AUC = 0.962 means in plain language, and why it can be high while precision is poor.
- Why can B have lower ROC-AUC but much higher average precision? Where on the score distribution must B be better than A?
- Which model would you recommend for a system where flagged sessions go to a small human review team? Justify with the metric properties.
Share this question