DoorDash · Statistics & Data Analysis
Design and analyze a switchback experiment
TrueInterview
October 7, 2026 · 2 min read
You are evaluating a delivery marketplace feature that is believed to reduce cold-food incidents among bike couriers in dense zones. Plan a two-week city-level switchback experiment in which the feature is switched ON and OFF in equal-length time slots inside each city. Be specific and cover: (A) Randomization: Pick a slot length given an average order lifecycle of 45 minutes and a driver relocation/carryover horizon of 30 minutes. Justify the chosen to limit contamination, and describe a block-randomization scheme that balances day-of-week and peak-hour mix while keeping the sequence unpredictable. (B) Assignment versus exposure: Explain the distinction between slot-level assignment (intention-to-treat) and actual exposure when some units work during OFF slots but receive spillover demand from adjacent ON slots. State the numerator and denominator for the primary metric (cold-food rate among bike deliveries), and give two denominator variants: all deliveries (condition_label=0 and 1) versus only deliveries with condition_label=1. (C) Analysis model: Write the exact regression you would fit (formula notation is acceptable) with city fixed effects and slot-of-week fixed effects, using cluster-robust standard errors at the city-by-slot level. Explain how you would add pre-period baselines or covariates such as weather, surge, and courier mix to improve precision. (D) Power: Given a baseline cold-food rate of 6%, a target relative reduction of 10% (MDE = 0.6 percentage points), an average of 120 eligible orders per slot, an intracluster correlation (ICC) at the slot level of 0.02, and 14 days, estimate how many switchbacks (ON↔OFF transitions per city) are required for 80% power at . State your assumptions and show the core calculation or code you would use. (E) Diagnostics: List the specific randomization checks and balance tests you would run, and describe how you would test for carryover, for example with leading indicators or by excluding boundary intervals. (F) Robustness: How would you handle partial compliance, missing telemetry, or mid-test shocks such as major events? Describe a principled decision rule for stopping, extending, or rerunning the test. Overview: This question assesses skill in experimental design, causal inference, randomization and contamination control, regression modeling with fixed effects and clustered standard errors, power calculation, and handling of compliance and robustness issues.