Back to problems

Design an A/B test with guardrails and SRM checks

System Design · Google · Medium

You are preparing to launch a new personalized ranking model for the main product listing page. Please answer all of the following. (1) Pick the primary success metric. Define it precisely: provide the exact ratio, state what belongs in the numerator and denominator, and list all data filters such as included users, excluded events, bot removal, and the observation window. (2) Select at least three guardrail metrics. For each one, give a concrete alert threshold and explain…

Checking your access…