Meta · Statistics & Data Analysis
Decide and experiment on Group Call feature
TrueInterview
October 7, 2026 · 1 min read
Suppose the current date is September 1, 2025. The only data available is a single table, calls_daily_agg(date, user_id, country, device_tier, one_to_one_calls_started, one_to_one_call_duration_sec, dropped_call_rate, group_intent_signals, concurrent_call_overlaps, inbound_group_invites_proxy, outbound_group_invites_proxy, p50_call_quality_score). Working from this table alone: (a) Suggest concrete proxy metrics and a quantitative decision rule for estimating unmet demand for a new Group Call feature before it is built. Give explicit thresholds (for example, concurrent_call_overlaps per 1,000 users) and describe how you would segment users to avoid Simpson's paradox. (b) Identify the most valuable additional resources you would add if permitted (at most 5), and state the distinct bias each one would reduce. (c) Design an A/B test to evaluate Group Calls post-launch: pick the randomization unit (user vs chat-group vs geo), explain how you would handle interference, define one primary success metric and at least three guardrails, lay out a ramp plan, describe novelty-effect mitigation, state cluster-adjusted power assumptions, and specify stopping rules. (d) After nine months of general availability (December 1, 2024 to September 1, 2025), explain how you would distinguish durable success from regression to the mean; if the overall effect on the company's North Star is null but some cohorts gain or lose, provide a go/holdback/sunset framework with quantitative cutoffs. (e) Post-launch, suppose metrics decline in certain regions or device tiers: outline a diagnostics plan covering measurement, seasonality, cannibalization, and QoS constraints, and describe the sequence of holdouts or rollbacks you would run. Overview: This question tests a data scientist's proficiency in product analytics, causal inference, metric engineering, experimentation design, and post-launch diagnostics within the Analytics & Experimentation domain.