Explain ROC-AUC vs PR-AUC tradeoffs

Question

Consider a binary classifier you have trained to emit predicted probabilities. Put ROC-AUC alongside PR-AUC (the second is usually reported as Average Precision, equivalent to the area under the Precision–Recall curve).

1. What each curve plots. For both the ROC and PR curves, identify the axis quantities, the essential formulas (TPR/FPR for ROC; Precision/Recall for PR), and the intuitive meaning of each AUC.
2. Class imbalance and baselines. Describe how a low positive base rate influences each metric, and state the “no-skill” baseline for each one.
3. When the two disagree. In what situations can ROC-AUC appear “good” even though PR-AUC appears “poor” (or the opposite)? Provide a concrete example.
4. Practical tradeoffs. For evaluation or model selection, when would you favor ROC-AUC over PR-AUC, and when the reverse?
5. Limited-review scenario. Suppose the business objective is to identify as many positives as possible while only a restricted number of cases can be reviewed (for example, fraud investigation). Which metric or curve should be prioritized, and why?
6. Pitfalls and edge cases. List at least two practical pitfalls, such as calibration versus ranking, choosing an operating threshold, train-to-production prevalence shift, and the relationship between AUC and ranking.

Overview: A comparison of ROC-AUC and PR-AUC for binary classification: the axes and formulas for each curve, how class imbalance changes the baseline (0.5 for ROC versus the positive rate for PR), situations where ROC appears strong while precision collapses, and which metric to emphasize for capacity-limited reviews such as fraud investigations. It also includes a worked imbalanced example and key pitfalls around calibration, thresholds, and prevalence shift.