Cvs Health · Statistics & Data Analysis
Compute A/B significance, CI, and power
TrueInterview
October 7, 2026 · 1 min read
You conduct an A/B test over seven days. Control A records 520 conversions in 10,000 sessions. Variant B records 630 conversions in 11,500 sessions. A calculator may be used as needed. Treat the observations as independent Bernoulli trials. a) Run a two-proportion z-test of against a two-sided alternative. Calculate the pooled-proportion z statistic and the corresponding two-sided p-value. b) Using the unpooled standard error, compute a 95% confidence interval for . c) For sample-size planning with baseline , what equal per-arm sample size (in sessions) is required to detect an absolute lift of percentage points (that is, and ) at a two-sided and 80% power using the normal approximation? Use and ; state the formula you apply and the resulting rounded up. d) If you were instead testing 12 independent metrics, what Bonferroni-adjusted per-metric would keep the family-wise ? Overview: This question assesses skill in statistical inference for A/B testing: two-proportion z-tests, confidence intervals built with unpooled standard errors, sample-size and power calculations, and Bonferroni correction for multiple comparisons.