Size opportunity and prioritize experiments

Before you spend engineering effort, estimate whether a new e-commerce product line is worth running a test on.

1. Carry out both top-down TAM/SAM/SOM sizing and bottom-up sizing based on current traffic and conversion, and explicitly estimate the share of users affected (above 20% versus around 5%);
2. calculate the expected incremental revenue and the smallest detectable lift that would make the experiment economically worthwhile, accounting for fixed costs and the opportunity cost of traffic;
3. lay out a test plan covering targeting, ramp-up, key metrics, counter-metrics, and a go/no-go threshold;
4. explain how you would reduce the risk of cannibalizing existing categories and spot leading indicators before revenue appears;
5. state what you would do if short-term engagement falls while long-term retention is likely to rise.

Overview: This question tests skill in market sizing, funnel-based revenue modeling, A/B test power and minimum detectable effect analysis, experiment design and guardrails, cannibalization risk assessment, and measuring short- versus long-term trade-offs in the Analytics & Experimentation area for a Data Scientist position.