Determine A/B test sample size drivers

For an A/B test where the baseline conversion rate is $p_0 = 8.0\%$, you need to detect a relative minimum detectable effect of 8% (that is, an absolute lift of +0.64 percentage points) with a two-sided $\alpha = 0.05$, 80% power, and a 1:1 assignment ratio. (a) Derive and compute the sample size required per arm from a normal approximation to the difference in proportions, and show the z-values you use. (b) Keeping all other inputs unchanged, describe both the direction and the magnitude of the change in the required $n$ when: the MDE is cut in half; $\alpha$ is tightened to 0.01; power is raised to 90%; allocation becomes 75/25; variance increases because of user-level clustering with ICC $=0.02$ and mean cluster size $m=5$ (calculate the design effect); and you use group-sequential monitoring with two equally spaced looks and O’Brien–Fleming boundaries. (c) If the outcome is overdispersed count data following NB2, outline how you would re-estimate $n$.

Overview: This item assesses statistical power and sample-size planning for A/B tests, including two-proportion hypothesis tests, normal-approximation derivations, the influence of the MDE, $\alpha$, and power, allocation ratios, clustered designs through ICC and design effect, group-sequential monitoring, and adjustments for overdispersed count outcomes.