Design and analyze pricing-page A/B test

You are planning an experiment on a redesigned pricing page. The primary outcome is paid conversion within seven days of a user's first page view, measured at the user level. Baseline conversion is 6.0%; the target minimum detectable effect is a 5% relative lift, to 6.3%; α is 0.05 two-sided; power is 0.80. Daily traffic is 200,000 unique users, split 60% new and 40% returning, with 10% on tablet, 30% on desktop, and 60% on mobile. Constraints include cross-device switching, bot filtering, and paid-ad visitors whose intent may differ. Tasks: (1) estimate the approximate sample size per variant and the minimum experiment length with and without CUPED, assuming CUPED reduces variance by 20%; (2) specify guardrail metrics such as sample ratio mismatch, bounce rate, and revenue per user, and describe near-real-time SRM detection; (3) choose the randomization unit among cookie, user, or account, explain how to handle cross-device identity, and describe contamination prevention; (4) plan sequential interim analyses with Type I error control, such as alpha-spending, and define a pre-registered stopping rule; (5) outline a heterogeneity analysis by device and traffic source, and state how you would proceed if the overall average treatment effect is neutral but mobile shows a significant lift; (6) provide a concrete analysis and communication plan, including effect sizes with 95% confidence intervals and a ship/iterate/halt decision framework. Overview: This question tests data-science skills in online experiment design and analysis, covering statistical power and sample-size estimation, variance-reduction techniques, guardrail and SRM monitoring, randomization and contamination control, sequential monitoring and stopping rules, heterogeneity or subgroup analysis, and communication of results in the analytics and experimentation domain. It is often used to evaluate applied statistical inference and experimental rigor in product and business settings, emphasizing practical application while also requiring conceptual understanding of the underlying statistical principles and operational trade-offs.