Design fundraising experiment and guardrails

You are planning an A/B test for two online solicitation versions in a nonprofit email campaign, where the variants differ in subject line and suggested donation amount. Assume the baseline conversion rate is 5.0%, and Variant B is expected to lift conversion by 10% relative to that baseline. Reaching each person costs $1, and converters donate $80 on average; assume the variant does not change that average. There is no online capacity limit.

Tasks:

1. State the primary success metric and at least two guardrail metrics that protect long-term health (such as unsubscribe rate or complaint rate), and justify each choice.
2. Compute the minimum sample size per arm needed to detect the expected lift using a two-sided test at 95% confidence and 80% power. Show the formula and the numeric result, and note any approximations you make.
3. Explain how you would break down results by donor tier (H vs L) without inflating false positives. Include your approach to multiple-testing control or hierarchical modeling.
4. Outline a stopping rule and the risk of peeking. How would you handle uneven email deliverability across segments?
5. If Variant B increases conversion but reduces average gift by 5%, show how you would redo the decision using net revenue per reached recipient.

Overview: This question tests experimental design; statistical power and sample-size calculation; selection of primary and guardrail metrics; segmentation with multiple-testing control or hierarchical modeling; stopping rules for sequential analysis; and revenue-impact calculations for A/B testing email solicitations.