Choose Classification Metrics Under Asymmetric Costs

You are assessing a binary classifier built for a business use case. Describe how a confusion matrix can be used to calculate and interpret the following:

• precision,
• recall (sensitivity),
• specificity,
• false positive rate,
• false negative rate,
• ROC-AUC,
• PR-AUC. Then respond to these questions:

1. What distinguishes a Type I error from a Type II error?
2. How should your choice of metric shift for scenarios like spam filtering, medical screening, fraud detection, and ad click prediction?
3. Why might accuracy give a misleading picture when classes are imbalanced?
4. What impact do decision thresholds, prevalence drift, and probability calibration have on model evaluation?
5. When false positives and false negatives carry different business costs, how would you select an operating threshold? Overview: This question tests knowledge of binary classification evaluation metrics and operating decisions, spanning measures derived from the confusion matrix (precision, recall/sensitivity, specificity, false positive and false negative rates), aggregate metrics (ROC-AUC, PR-AUC), Type I and Type II errors, threshold choice, prevalence drift, probability calibration, and cost-sensitive trade-offs. It is often used to gauge a data scientist's capacity to reason about model behavior under class imbalance and uneven business costs, and it falls within the Statistics & Math / Machine Learning model-evaluation area, demanding both conceptual grasp and hands-on application.