Amazon · Statistics & Data Analysis
Compute an A/B test p-value by hand
TrueInterview
October 7, 2026 · 1 min read
You're examining a game feature through an A/B test where the metric of interest is the conversion rate, a binary outcome.
- Control: users, conversions
- Treatment: users, conversions
Tasks:
- Calculate the two-sided p-value against by applying a two-proportion z-test; include the formulas and the intermediate numbers.
- Say whether the outcome reaches significance at .
- (Implementation) Explain how you would obtain this p-value in Python without SciPy, allowed to use only the
mathmodule.
Overview: This prompt tests grasp of proportion hypothesis testing—namely the two-proportion z-test, p-value calculation, and what statistical significance means for an A/B experiment with binary outcomes.
Loading comments…