Recover causal effect without a control group

An intern started an A/B experiment but neglected to set aside a control group; every eligible user was exposed to Treatment for 5 days (the T period). You have 4 weeks of pre-period data (the P period) collected under the same eligibility rules with a stable product. The primary metric is 1-day retention; guardrail metrics are crashes per session, p95 latency, and purchase conversion.

Tasks:

1. Propose and compare at least two identification strategies for estimating the treatment effect with observational methods: (a) pre-post with CUPED; (b) synthetic control via matching or propensity-score weighting (PSW) against ineligible-but-similar users or delayed-exposure users; (c) difference-in-differences using a holdout geography. For (b), specify the covariates, overlap checks, and diagnostics (SMD, eCDF, weight trimming).

2. State the assumptions each method requires (e.g., parallel trends, no interference, ignorability) and design falsification/placebo tests to stress-test them.

3. Explain how you would compute ATT vs ATE, account for calendar effects and novelty/seasonality, and quantify uncertainty (cluster-robust SEs or bootstrap under weighting).

4. List pitfalls in the original A/B setup that caused this failure and propose a prevention plan (exposure checks, invariant metrics, automated power and allocation validation).

Overview:

This question tests a candidate's ability in causal inference, observational estimation, and experiment analytics—specifically identification strategies, causal assumptions, validation/placebo tests, diagnostics, and uncertainty quantification after an accidental full rollout.