Design an A/B test; choose Z vs T

You are setting up an A/B test on a signup funnel. The baseline conversion rate is $p_0 = 0.050$, and the target minimum detectable effect (MDE) is a relative increase of 10%, giving $p_1 = 0.055$. There are 50,000 daily eligible visitors, but only 60% satisfy the targeting criteria, and 90% of those eligible visitors are actually assigned to a bucket. The test uses a 1:1 allocation, a two-sided $\alpha = 0.05$, and power of $0.80$. Treat users as independent observations.

(a) Calculate the necessary sample size for each variant and the expected number of calendar days required to reach it under the traffic constraints described. Give your formula and note any continuity correction or pooled-variance assumptions.

(b) State whether you would use a Z-test or a T-test for the primary proportion metric, and justify your choice. When do their results differ in a meaningful way? Address how data-estimated variance, small-sample corrections, and unequal sample sizes or variances influence the decision.

(c) If the product team wants to stop the test “as soon as it looks promising,” propose a sequential testing or alpha-spending method, such as group-sequential boundaries, that still controls Type I error. Define the stopping rules and explain how they affect the nominal sample size and expected duration.

(d) Assume randomization is not feasible for a related rollout. Outline a causal inference plan: draw a DAG to identify confounders, suggest an identification strategy such as difference-in-differences with parallel trends checks or an instrumental variable, and list the key assumptions you would need to defend. How would you validate those assumptions in practice?

Overview: This question assesses a candidate's ability in experimental design and statistical inference, including sample-size and power calculations, choosing between Z and T tests, sequential testing and alpha-spending strategies, and causal-inference planning for non-randomized rollouts.