Instacart · Statistics & Data Analysis
Compute duration and stopping rules correctly
TrueInterview
October 7, 2026 · 1 min read
You need to determine in advance how long to run a same-day delivery attach rate experiment. The baseline attach rate is 22%, the smallest relative lift worth detecting is 5%, (two-sided), and power is . Daily traffic is 120,000 eligible checkouts, but eligible traffic is only 20% of all site traffic; both eligibility and traffic fluctuate by weekday, with large Sunday peaks. a) Work out the required sample size per arm and convert it into calendar days, adjusting for day-of-week seasonality and a 10% household cluster correlation via the design effect. b) Specify a monitoring plan that is safe for peeking (for example, group sequential or alpha spending), and explain how it changes the expected duration. c) If the metric is noisy on Sundays (Miami), propose stratified randomization or CUPED to cut variance; estimate the variance reduction you would expect and how that affects duration.
Overview: This question tests a data scientist's ability in experimental design and applied statistics: power and sample-size calculation, handling within-household clustering and design effects, sequential monitoring to control peeking, and variance reduction or stratified randomization for day-of-week seasonality.