Explain bias-variance and evaluate a classifier

You are in an interview for an Applied Scientist internship. Respond to the machine learning foundations questions below.

1) Bias–variance

• Give definitions of bias and variance in the supervised learning setting.
• Describe the bias–variance tradeoff and its connection to underfitting versus overfitting.
• Provide 2–3 practical ways to reduce:
  • high bias
  • high variance

2) Classification metrics

• Define accuracy, precision, recall, and F1.
• Explain the situations in which accuracy can be misleading.
• Given a confusion matrix with TP, FP, TN, and FN, show how you would calculate these metrics and decide which one to optimize for an imbalanced problem.

3) Confidence intervals

• Define a confidence interval (CI).
• Suppose you evaluated a binary classifier on a test set of size $n$ and observed accuracy $\hat{p}$. Explain how you would construct a 95% CI for the true accuracy, and state the assumptions this requires.
• Name at least one alternative way to build a CI when the assumptions are weak, such as with a small sample or correlated examples.

Overview: This question assesses core machine learning knowledge: the bias–variance tradeoff, underfitting versus overfitting, classification metrics (accuracy, precision, recall, F1), and statistical evaluation of classifiers through confidence intervals.