Stripe · Statistics & Data Analysis
Plan an experiment to validate targeting impact
TrueInterview
October 7, 2026 · 1 min read
You have a ranked list of merchants predicted to adopt Subscription. Design an experiment to measure the business impact of targeting these merchants with a sales or email nudge. Constraints: randomization must be at the merchant level, limited to the top 50,000 scored merchants. Primary metric: conversion to a first active Subscription within 30 days. Baseline conversion without outreach is 2.0%; the minimum detectable uplift is +50% relative, i.e. to 3.0%. Costs: outreach costs $5 per merchant; a converted merchant is expected to generate $120 in 90-day gross margin. (a) Propose an RCT design with a control vs. treatment split, stratification by score deciles and country, guardrails for refund rate and chargeback rate, and a staged ramp. (b) Calculate the required sample size per arm for 80% power and a two-sided when comparing proportions; show the formula and the numeric result. (c) Using the stated costs and margin, compute the break-even precision threshold for targeting, ignoring long-term spillovers. (d) Discuss contamination risks from multi-merchant customers, how to treat merchants already piloting Subscription, and how to attribute conversions when merchants receive multiple touches. Overview: This question assesses a candidate's skills in experimental design, causal inference, statistical power and sample-size calculations, handling contamination and attribution, and cost–benefit analysis for targeted marketing experiments.