Google · Statistics & Data Analysis
Infer causal impact without an A/B test
TrueInterview
October 7, 2026 · 1 min read
Engineering released a new version intended to reduce disconnections, but there is no A/B holdout to compare against. Evaluate the effect rigorously: select one of an interrupted time-series design with seasonality, difference-in-differences using untreated regions or devices, synthetic control built from donor pools, or regression discontinuity if the rollout timing is sharp; state the identifying assumptions, checks for parallel pre-trends, placebo tests, and robustness to a staggered rollout; specify the outcome and model family (for example, binomial for drop rate, Poisson or negative binomial for drop counts), variance estimation (cluster-robust standard errors by account or region), and variance reduction through CUPED; define the primary metric as drops per 1,000 minutes and the guardrail metrics, then calculate confidence intervals and the minimum detectable change for 80% power at given a baseline of 3.0 drops per 1,000 minutes over 10 million daily minutes and realistic autocorrelation; explain how you would adjust for confounders such as shifts in network mix, seasonality, and changing user composition, and how you would communicate uncertainty to stakeholders.
Overview: This question assesses causal inference and observational study design skills, covering time-series and panel methods, identification strategies (e.g., ITS, DiD, synthetic control, RD), statistical modeling and power or sample-size estimation, confounder adjustment, and communication of uncertainty.