DoorDash · Statistics & Data Analysis

Design an experiment for order batching

TrueInterview

October 7, 2026 · 1 min read

DoorDash is considering a batching change that would allow a dasher to collect two nearby orders during a single trip in peak periods. Design an experiment to measure the causal effect on (a) order conversion rate, (b) actual-versus-quoted ETA accuracy, (c) dasher hourly earnings, and (d) restaurant prep-time congestion.

Answer with specifics:

The randomization unit and your reasoning for it (for example, zone-hour, store, or dasher) in light of interference/spillovers, plus how you will reduce cross-treatment contamination.
Your stratification and covariate adjustment approach (such as city, cuisine, distance band, or forecast demand) and how you will pre-register the guardrail metrics (late deliveries, cancellations, CSAT).
Sample size and experiment length: state the minimum detectable effect, baseline rates, variance assumptions, and a power calculation at 80% power; if early stopping is planned, justify sequential testing or a group sequential design.
Novelty and learning effects: how you will identify and downweight the first N days and assess the longer-run steady state.
Heterogeneity: how you plan to estimate treatment effects by city and distance band without p-hacking (for instance, with hierarchical models or shrinkage).
Decision rule: give a concrete promotion rule for the case where guardrails degrade but primary metrics improve, including the acceptable deltas and confidence thresholds.

Overview: This question tests skill in experiment design and causal inference for marketplace changes, spanning the choice of randomization unit and handling of interference/spillovers, stratification and covariate adjustment, power and sample size calculations, detection of novelty and steady-state effects, heterogeneity estimation, and pre-registered guardrail decision rules. It appears often in analytics and experimentation interviews because interviewers need to evaluate both conceptual knowledge of causal inference and the practical use of randomized designs and operational measurement to support robust, contamination-aware assessment of a policy change.

DoorDash · Statistics & Data Analysis

Design an experiment for order batching

TrueInterview

October 7, 2026 · 1 min read

Answer with specifics:

The randomization unit and your reasoning for it (for example, zone-hour, store, or dasher) in light of interference/spillovers, plus how you will reduce cross-treatment contamination.
Your stratification and covariate adjustment approach (such as city, cuisine, distance band, or forecast demand) and how you will pre-register the guardrail metrics (late deliveries, cancellations, CSAT).
Sample size and experiment length: state the minimum detectable effect, baseline rates, variance assumptions, and a power calculation at 80% power; if early stopping is planned, justify sequential testing or a group sequential design.
Novelty and learning effects: how you will identify and downweight the first N days and assess the longer-run steady state.
Heterogeneity: how you plan to estimate treatment effects by city and distance band without p-hacking (for instance, with hierarchical models or shrinkage).
Decision rule: give a concrete promotion rule for the case where guardrails degrade but primary metrics improve, including the acceptable deltas and confidence thresholds.