Diagnose a sudden metric spike or drop

On 2025-06-12, your daily Checkout Conversion Rate (CCR) rose from 3.2% to 4.5% and remained elevated for three days. CCR is computed as unique purchasers divided by unique sessions that include at least one add_to_cart event. Outline a concrete, step-by-step investigation plan that does the following: 1) confirm whether the change is real or an instrumentation artifact, with specific sanity checks (for example, event volume balance, missing data by logger version, null spikes, late-arriving events, bot and outlier filters), 2) localize the shift using a minimal set of segmentations that could expose Simpson's paradox (device, app vs. web, geography, acquisition channel, experiment arms, release version, traffic vendor, new vs. returning users), 3) list and rank at least five hypotheses (such as a pricing change, shipping promotion, advertising mix shift, experiment ramp, search relevance tweak, or fraud filter change), including a quick back-of-envelope impact estimate for each and the precise query or metric you would use to confirm or refute it, 4) separate product-causal effects from mix-driven effects by suggesting a counterfactual, holdout, or synthetic control and the fastest way to compute it using existing logs, 5) decide on next steps (rollback, continue, or ramp) and set guardrails with explicit thresholds (for example, revenue per user, refund rate, support tickets), and 6) specify the artifacts you would generate within the first 60 minutes (plots and tables) and the exact time-series tests or seasonality checks you would run to prevent false alarms.

Overview: This question assesses a data scientist's skills in anomaly investigation, metric instrumentation validation, causal inference, and product analytics, covering log-level querying, segmentation, hypothesis generation, and rapid impact estimation.