Tune fraud threshold under review capacity and costs

Suppose your fraud model produces a calibrated score $s \in [0,1]$ for each account. You may automatically block accounts with $s \ge t$, route accounts with $s \in [r, t)$ to manual review, and allow the rest. The constraints and costs are: 2,000,000 accounts per day, a 1% base fake rate, a manual review capacity of 100,000 accounts per day, manual review catches 95% of the fakes it examines, blocking a genuine user costs $5, letting a fake through costs $20, and each manual review costs $1.

(a) Write the expected daily cost as a function of the thresholds $r$ and $t$ under calibrated scores. Explain how you would estimate that cost from historical labeled data using isotonic or Platt calibration and the empirical score distributions.

(b) Optimize $r$ and $t$ subject to the review budget. Describe how you would select the operating point on the precision-recall curve and validate it with an online interleaved experiment.

(c) Outline a drift monitoring plan and a weekly threshold retuning procedure that includes backtesting and safety rails.

Overview: This question tests a data scientist's ability to perform cost-sensitive threshold tuning, calibrated probability modeling, and operational decision-making for fraud triage, including expressing expected costs under a review capacity constraint and monetary trade-offs.