Explain your ML project end-to-end

Select the most involved machine learning project on your resume and address every part with precision: (1) Specify the business objective, target variable, key constraints, and the primary success metric you selected, along with your reasoning (for instance, PR-AUC versus ROC-AUC versus cost-weighted error). (2) Describe the data: its sources, labeling strategy, and train/validation/test splits; if the data is temporal, specify a time-based split and how you prevented leakage (provide concrete examples of potential leakage you checked for). (3) Model selection: list candidate models and the exact hyperparameters you tuned; present an ablation plan that isolates the marginal contribution of two specific feature groups; explain one bias–variance trade-off decision with supporting evidence. (4) Class imbalance: explain your resampling or weighting approach and how you set decision thresholds. Now compute this scenario: on a 10,000-example validation set with 8% positives, the baseline model at threshold 0.50 has precision=0.70 and recall=0.45; after adding Feature Set X and performing probability calibration, at threshold 0.30 you have precision=0.58 and recall=0.66. Compute F1 for both, the expected counts of TP, FP, FN at each threshold, and decide which to deploy if FP costs 1 and FN costs 5—show your cost calculation. (5) Deployment: propose concrete monitoring metrics (at least: calibration, drift on three top features, alert thresholds), a rule for triggering retraining, and how you would guard against data pipeline schema changes. (6) Online validation: create an A/B test with guardrail metrics, sample-size/duration estimation, and a rollback plan in case long-tail segments regress. (7) Post-mortem: identify two plausible failure modes and how you would debug them using specific offline error buckets and online slices. Overview: This question assesses a data scientist's end-to-end machine learning skills, covering problem framing and metric justification, data sourcing and labeling, model selection and calibration, class-imbalance handling, deployment and monitoring, experimentation design, and post-mortem analysis; it belongs to the Machine Learning domain and checks both conceptual understanding and practical application across modeling and MLOps. It is often used to evaluate an interviewee's ability to justify trade-offs, reason about operational constraints like latency, fairness and cost, design valid evaluation and A/B testing strategies, and define measurable monitoring and rollback criteria.