How would you choose a classification threshold?

You have trained a binary classifier that outputs a probability score $p(y=1|x)$. You need to choose a decision threshold $t$ to convert those probabilities into class labels.

Answer the following:

• What is your general approach to selecting $t$ on a validation set?
• How do you incorporate different costs of false positives vs false negatives?
• How do class imbalance and probability calibration affect threshold choice?
• Give one example where you would prefer a high threshold and one where you would prefer a low threshold.

Overview: This question evaluates your understanding of threshold selection for probabilistic binary classifiers. It tests competence in choosing a decision threshold on a validation set, making cost-sensitive trade-offs between false positives and false negatives, and accounting for the effects of class imbalance and probability calibration.