How would you define and use retention metrics?

Scenario

You work as a data scientist on a consumer product, either an app or a website. A product manager asks you to investigate user retention in depth and suggests measuring 7-day and 28-day retention.

Task

1. Define retention precisely. Provide at least three standard retention definitions and describe how they are different:
   • N-day (classic/cohort) retention
   • Rolling retention (also called unbounded)
   • Return rate / weekly active retention (or another sensible variant)
2. Describe how you would calculate and read 7-day versus 28-day retention:
   • Which user cohorts would you build (for example, signup or install week)?
   • What aspect of user behavior does each metric capture?
3. Discuss the short-term versus long-term trade-offs:
   • Give examples of product changes that could lift 7-day retention while hurting 28-day retention, and the reverse.
   • Propose a metric hierarchy with primary, diagnostic, and guardrail metrics.
4. Identify at least five pitfalls or edge cases in retention measurement and how you would handle them (for example, right-censoring, seasonality, re-installs, bots, changing definitions, missing events, timezone issues).
5. If the product manager wants to run an A/B test to improve retention, outline your design and evaluation approach (randomization unit, experiment length, how to account for delayed effects, and any variance reduction or sequential testing choices).

Overview: This question tests a data scientist's product analytics and experimentation skills by checking knowledge of retention metric definitions, cohorting approaches, short- versus long-term trade-offs, common measurement pitfalls, and A/B test design considerations.