Coinbase · ML System Design
Build and evaluate a conversion prediction model
TrueInterview
October 7, 2026 · 1 min read
You are given a CSV of email exposure events, with one row for each user-email send. The columns are user_id, send_ts, treatment_flag, opened, clicked, purchased_within_7d (the label), user_region, device_type, tenure_days, prior_sessions_28d, prior_purchases_180d, avg_cart_value_180d, categories_viewed_28d, email_personalization_score, deliverability_score, and other anonymized features. Task:
- EDA: Find leakage risks and remove them by making sure every feature can be computed at send_ts (for example, do not use post-send actions such as opened or clicked unless you are modeling an uplift chain). Look at class imbalance, missing values, outliers, and high-cardinality categorical variables. Suggest data quality checks.
- Modeling: Fit a baseline logistic regression and a gradient-boosted tree to predict purchased_within_7d. Use time-based splits: train=2025-06, valid=2025-07, test=2025-08. Run cross-validation on the training set, tune hyperparameters, and add monotonicity or regularization where appropriate.
- Evaluation: Report ROC AUC, PR AUC, calibration (reliability curve, Brier score), and incremental lift for the top 10% of scored users relative to control. Give bootstrap confidence intervals and check stability across region and device.
- Deployment: Pick a decision threshold that maximizes expected incremental revenue, given an email cost of $0.003 and a treatment effect estimated from a calibration experiment. Describe monitoring (data drift, performance drift, alerting), retraining frequency, and next steps for improvement (feature engineering, causal uplift modeling, de-biasing with IPS or DR estimators). Overview: This question tests skill in predictive modeling, feature engineering with target-leakage control, time-aware validation, uplift and treatment-effect estimation, calibration and uncertainty quantification, and end-to-end deployment considerations in the Machine Learning domain.
Loading comments…