How would you investigate a metric decline?

Scenario

You are a data scientist supporting a consumer marketplace product. A core business metric—such as orders per day, conversion rate, revenue, profit, or retention—has fallen noticeably during the past 1–2 weeks.

Task

Describe a structured approach you would use to investigate the decline.

Requirements

Your answer should address these points:

1. Define the metric clearly
   • Exact definition, numerator and denominator, applied filters, and attribution rules.
   • Timezone handling, event time versus processing time, and whether the metric is computed on a cohort basis.
2. Determine whether the drop is real or a measurement issue
   • Data pipeline problems, logging changes, bot or traffic anomalies, and backfilled data.
3. Locate the change in time
   • When did it begin? Is it a step change or a gradual trend? Does it coincide with releases or experiments?
4. Decompose and segment
   • Break the metric down by funnel stage and by important segments such as new versus returning users, geography, device, acquisition channel, product surface, and supply side versus demand side.
5. Evaluate external and seasonal factors
   • Seasonality or holidays, competitor actions, macroeconomic events, weather, and outages.
6. Form hypotheses and propose follow-ups
   • Further analyses, dashboards, and—where appropriate—experiments or mitigations.

Output

Explain the investigation plan and what you would do in the first day versus the first week.

Overview: This question assesses a data scientist's skills in diagnostic analytics, understanding of metric instrumentation, causal reasoning, and segmentation-based problem localization for consumer marketplace KPIs.