Uber · Statistics & Data Analysis
Compute A/B sample size under clustering
TrueInterview
October 7, 2026 · 1 min read
Calculate the sample size needed per arm for a two-arm signup A/B test under these conditions: baseline conversion , target relative lift (so ), two-sided , power , a 2:1 control-to-treatment traffic split, 15% of sessions identified as bots and removed after collection, sessions clustered by user with an average of 1.4 sessions per user and , and 8% expected loss from data-quality filters.
Give the formulas for two-sample proportion tests, incorporate the clustering design effect, correct for attrition and the allocation ratio, and translate the result into test duration assuming 120,000 sessions per day.
List any further assumptions, and explain how violations such as variance misspecification or sequential peeking would alter the plan.
Overview: This item assesses skill in experimental design and statistical inference for A/B testing, particularly sample size estimation for two-sample proportions with unequal allocation, clustering effects (ICC and design effect), and corrections for bot removal and data-quality attrition.