Resolve a Mismatch Between Regression and Classification Metrics

An interviewer asks you to train a regressor on imbalanced data and score it with precision and recall. Explain why that request is internally ambiguous and how you would clear it up before modeling. Give a complete plan for each plausible interpretation: a rare-event classification target and a genuinely continuous regression target.

Constraints & Assumptions

• Do not turn a continuous outcome into categories just so precision and recall can be computed, unless you also address the information lost.
• Any resampling has to happen inside the training set only, never before the validation split is made.
• The evaluation should line up with the decision the model is meant to support.

Clarifying Questions to Ask

• What kind of target is it: categorical, binary, ordinal, or continuous?
• What real-world action is triggered by a positive prediction or a numeric estimate?
• What do false positives, false negatives, and large numeric errors cost?
• Does “imbalanced” mean rare classes, or a skewed distribution of a continuous target?

Hint — Begin with the estimand: The model family and metrics should follow from the target and the decision; “regressor” and “precision” cannot jointly define the task until that is clarified.

What a Strong Answer Covers

• Spotting the mismatch between regression and classification.
• A rare-event classification plan with appropriate splits, thresholds, and metrics.
• A continuous-target plan using regression losses and diagnostics that pay attention to the tails.
• Leakage safeguards for weighting, resampling, tuning, and model selection.
• A recommendation linked to operational costs, not just one default score.

Follow-up Questions

• When does average precision give more useful information than ROC AUC?
• How would probability calibration affect the choice of threshold?
• How can a regression model be assessed on rare high-value outcomes without converting the target to binary?

Overview: Sort out an interview prompt that mixes regression with precision and recall. Build separate plans for rare-event classification and skewed continuous targets, with metrics tied to the actual decision.

Read the full interview experience this question came from.

Community answers: one response says, I would use cross validation.