Evaluate Last-Mile Product Metric Changes

Assume you are a Principal Data Analyst supporting Walmart's Last Mile delivery products. The customer journey spans browsing or adding to cart, selecting delivery, viewing available delivery windows, delivery fees, and estimated arrival time, submitting the order, fulfillment from a store or fulfillment center, driver assignment, and final delivery.

Treat the product analytics case below as if you were advising product and operations leadership:

1. A core metric, such as delivery checkout conversion rate, shifted materially last week. How would you diagnose what happened? Cover the clarifying questions, funnel decomposition, segmentation, hypotheses, validation steps, and how you would distinguish a genuine business change from a data or instrumentation issue.
2. The team is introducing a new last-mile feature, for example a delivery-slot recommendation or an improved ETA promise experience. Design a metrics framework to assess whether the feature is successful. Include adoption metrics, customer experience metrics, operational metrics, business impact metrics, and guardrail metrics.
3. Suppose there is no reliable historical benchmark and the feature was not launched through a clean randomized A/B test. Propose an analysis plan to estimate impact anyway. Describe the analytical dataset, proxy metrics, likely confounders, selection bias risks, and possible causal or quasi-experimental approaches.
4. Explain how you would communicate the result to senior executives and turn the analysis into a product or business recommendation.

Overview: This question assesses product analytics, experimentation design, causal inference, instrumentation validation, and executive communication competencies in the context of last-mile delivery metric changes.