Design Identity-Trust A/B Test

You are interviewing for a Data Scientist opening on the Identity & Trust team of a consumer product company. The team plans to ship a feature that improves identity verification and introduces trust signals like a verified badge, extra account checks, or warnings on suspicious profiles. How would you set up an A/B test to assess this launch? Cover the following:

• the randomization unit and whether randomizing at the user level is appropriate,
• main success metrics and guardrail metrics,
• how to measure trust when harmful events occur rarely,
• interference or network effects when treated and control users interact,
• likely sources of bias or confounding,
• how you would determine the test's sample size and interpret outcomes if fraud falls while engagement or conversion also drops. Overview: The question assesses a candidate's ability in experimental design, causal inference, metric choice, sample sizing, recognizing bias and confounding, dealing with interference/network effects, and measuring rare adverse events in A/B tests for identity and trust features, for a Data Scientist role.