Choose threshold under asymmetric costs

You operate a credit-card fraud model that outputs a probability score. Select an operating threshold under asymmetric costs and defend the choice with numbers.

For every 1,000,000 transactions, assume a base fraud rate of 0.20%. A false positive (declining a valid transaction) costs $15, and a false negative (missing fraud) costs $100. Evaluate three candidate thresholds with these operating points on a representative validation set:

• T1: TPR = 0.90, FPR = 0.020
• T2: TPR = 0.80, FPR = 0.010
• T3: TPR = 0.65, FPR = 0.004

Tasks:

• For each threshold, calculate the expected counts of true positives, false positives, false negatives, and true negatives, along with the expected total cost. Choose the threshold with the lowest expected cost and describe the business trade-offs.
• Describe how you would calibrate the scores (for example, Platt scaling or isotonic regression), watch for dataset and label drift, and periodically re-tune the threshold by segment such as country, merchant category, and transaction amount.
• Suggest guardrail metrics that would flag harmful side effects, such as a spike in declines for high-LTV users, and design an experiment to validate changes without inflicting unacceptable customer pain.

Overview: This question assesses cost-sensitive classification, operating-threshold selection, score calibration, drift monitoring, and experiment and guardrail design skills in the Machine Learning domain.