Optimize threshold using confusion matrix and costs

A calibrated classifier estimates the positive class at 1%. On 10,000 held-out examples, the observed counts are: at threshold 0.50, TP=60, FP=40, FN=40, TN=9,860; at threshold 0.20, TP=85, FP=300, FN=15, TN=9,600. (1) Calculate precision, recall, and F1 at both thresholds. (2) Given a cost matrix with FP=1 and FN=20 (TP and TN have zero cost), compute the expected cost at the two thresholds, choose the cheaper threshold, and show your work. (3) Explain why ROC-AUC can be misleading here and why PR-AUC is more appropriate; give a brief numeric intuition using the counts above. (4) For a perfectly calibrated model, use cost-sensitive decision theory to derive the optimal probability threshold $t^*$ in terms of FP cost, FN cost, and the class prior.

Overview: This item tests classification metrics, calibration, threshold selection, and cost-sensitive decision theory in imbalanced binary classification: computing precision/recall/F1, comparing expected cost from confusion matrices, and deriving a cost-optimal probability threshold.