PayPal · ML & AI Fundamentals
Optimize thresholds under fraud costs
TrueInterview
October 7, 2026 · 1 min read
Cost-sensitive thresholding: You have a binary ATO classifier that scores transfers. The initial prevalence (fraud rate) is 0.2%. The costs are: a false positive (blocking a legitimate transfer) costs $2; a false negative (letting a fraudulent transfer through) costs $120. Consider the following operating points measured on a large validation set:
| Threshold | TPR | FPR |
|---|---|---|
| 0.90 | 0.50 | 0.0010 |
| 0.80 | 0.65 | 0.0030 |
| 0.70 | 0.75 | 0.0060 |
| 0.60 | 0.82 | 0.0100 |
| 0.50 | 0.88 | 0.0180 |
Assume 1,000,000 transfers are evaluated.
Tasks: A) For each threshold, compute the expected total cost as given a prevalence of 0.2%. Choose the threshold that minimizes cost, and report PPV and NPV at that operating point. B) If prevalence falls to 0.1% because of seasonality, recompute the costs and discuss whether the optimal threshold changes. C) Using ROC theory, derive the slope of the cost-optimal decision rule, , and explain how it maps to selecting a point on the ROC curve; interpret how prevalence shifts move the optimal operating point without retraining.
Overview: This question evaluates a candidate's competency in cost-sensitive binary classification, threshold selection, and ROC-based operating-point analysis, including computing expected costs and predictive values under varying prevalence and asymmetric error costs.