ByteDance · Statistics & Data Analysis
Apply PSM rigorously for observational A/B analysis
TrueInterview
October 7, 2026 · 1 min read
Suppose you have observational user-level data in which users self-select into receiving a feature, and you need to implement propensity score matching (PSM) to estimate the average treatment effect on the treated (ATT) for 7-day retention. Provide: (a) the propensity model specification (logistic versus gradient boosting) and the reasoning behind it; (b) the matching strategy (1:1 nearest neighbor with or without replacement, caliper choice and computation) and how you would tune it; (c) formal balance diagnostics (standardized mean differences below 0.1, variance ratios, KS tests) and re-weighting or rematching when balance fails; (d) detection of and remedies for lack of overlap (trimming/restriction); (e) variance estimation (Abadie–Imbens versus bootstrap) and when each is valid; (f) Rosenbaum sensitivity analysis—report the at which conclusions flip and how you would interpret it to a PM. Overview: English summary: The item tests expertise in causal inference and observational treatment-effect estimation, including propensity score modeling, matching strategies, balance diagnostics, overlap assessment, variance estimation, and Rosenbaum sensitivity analysis.