How to measure product success?

You need to assess whether a product or newly shipped feature is performing successfully. Explain how you would define success through a data science and product analytics lens. Your response should include:

• The product goal and how success changes depending on whether the product is in launch, growth, or maturity.
• A single primary success metric or north-star metric.
• Supporting metrics along the user funnel, for example acquisition, activation, engagement, retention, monetization, and user satisfaction.
• Guardrail metrics to verify the product is not improving one result at the expense of another.
• How you would separate correlation from causal impact, for instance using A/B tests or quasi-experimental approaches.
• How you would account for confounding factors, seasonality, novelty effects, selection bias, and heterogeneous effects across user segments.
• Which decision framework you would apply to determine whether the product is successful. Overview: This question tests a data scientist's skills in product analytics, metric definition, experiment design, and causal inference, including stating a north-star metric, supporting funnel and guardrail metrics, and addressing confounding, seasonality, and heterogeneous effects.