Design An Account Takeover Detection System

A payment API platform manages accounts that store sensitive secrets — API keys, saved payment methods, and bank transfer permissions. When an attacker compromises an account through credential theft, phishing, or a SIM swap, they can drain funds, reroute payouts, or exfiltrate customer data before detection. The challenge is to catch the takeover at the instant the attacker attempts a hazardous action, without disrupting legitimate users who travel, switch devices, or alter their workflows.

I would first clarify the principal detection entry point. For a payments API company, the highest-return entry point is not login — it is the moment a sensitive operation is attempted: changing a payment method, initiating a high‑value transfer, rotating an API key. Attackers who obtain credentials via phishing or SIM swap may sign in cleanly and act hours later; scoring only at login misses them.

The business objective is not recall alone. The goal is to minimize expected business cost across four possible responses: allow, escalate to step‑up authentication, flag for analyst review, or block and lock the account. Incorrectly locking a high‑value merchant account is expensive: operations are abandoned, customers call support, and churn rises. Every later design choice lives within that cost envelope.

This is not transaction fraud detection. Account takeover (ATO) is about determining who is using the account, not whether a particular payment is legitimate. The signals are behavioral (device changes, geographic anomalies, login rhythms), the labels originate from customer reports rather than chargebacks, and the action space is account‑level instead of transaction‑level.

The machine learning task is: given an account identifier, current session, device fingerprint, IP, behavioral history, action context, and graph‑derived signals, output a calibrated P(ATO | action) and map it through per‑action‑sensitivity and per‑merchant‑tier thresholds to one of the four enforcement actions. The consumers are the API gateway, the authentication service, and the analyst review queue.

At a high level, the system has two loops. The online path scores each sensitive action within roughly 100 ms: a rule pre‑filter catches known‑bad signals, a feature fetch pulls session velocity and account profiles from Redis, a behavioral scorer produces a calibrated probability, and an action mapper applies the threshold ladder. The offline path closes the feedback loop: events, ATO reports, analyst verdicts, and chargeback‑linked incidents flow through Kafka into Flink for nearline aggregations and Spark for batch training, retraining, and threshold governance. Four pressures make this difficult: real‑time scoring under a strict latency budget, labels that arrive days to weeks late, severe class imbalance at roughly 0.01–0.1% ATO rate, and adversaries who rotate credential‑stuffing toolkits within days.

Online and offline loops of the ATO detection system: real-time sensitive-action scoring with graduated enforcement, and delayed-label feedback for retraining and threshold governance.

Requirements

Clarifying Questions

You: “Where is the main entry point for scoring — at login or deeper inside the session?”

Interviewer: “Centrate on the sensitive actions. If someone attempts to change their payout details or rotate API keys, we must stop them inline within 100 ms. Login serves only as a secondary signal to build the session risk score.”

Key Point: Sensitive‑action gating is primary because attackers who obtain credentials can sign in legitimately and act later; scoring only at login misses the most damaging ATO scenarios.

You: “Is this a binary allow‑or‑block decision, or does the system have graduated responses depending on the risk score?”

Interviewer: “Four graduated actions: allow, step‑up authentication, flag for analyst review, or block and lock. The cost of each action differs by account tier and action sensitivity. A hard block on first detection is not the default.”

Key Point: Graduated enforcement means the model must output a calibrated probability that drives per‑action and per‑merchant‑tier threshold bands, not a single global cutoff.

You: “Where do ATO labels come from, and how long after an incident do they typically arrive?”

Interviewer: “Customer ATO reports are the gold standard but arrive 7 to 14 days late on average. Analyst investigation verdicts come faster but only cover flagged accounts. Some ATOs are never reported at all.”

Key Point: Training and evaluation must use report‑mature time slices, and partial‑label‑aware loss is required to prevent the model from treating the not‑yet‑reported window as clean negatives.

You: “How fresh do the behavioral and graph features need to be? I’m guessing session velocity is time‑critical but account history can lag a bit.”

Interviewer: “Real‑time context is available at request time. Session‑level velocity features must reflect events within seconds. Account profiles and graph features can refresh hourly to daily.”

Key Point: Three distinct freshness tiers require three separate pipelines: inline request context, a Flink nearline job for session velocity, and a daily Spark batch for account profiles and graph signals.

You: “If we block a suspect action, we never see if it was actually fraud. Is that blind spot acceptable, or do we need a way to test our blocked decisions?”

Interviewer: “We definitely need to test them. Ignoring that blind spot is too risky because our model’s performance will decay as patterns change. We need active ways to gather feedback on what we block.”

Key Point: Step‑up exploration on borderline scores, shadow scoring on blocked actions, and a small held‑out allow slot are necessary to keep an unbiased label stream and prevent training distribution shift.

Functional requirements

• Score every sensitive account action (payment‑method change, high‑value transfer, API key rotation, password reset) in real time and return one of four verdicts: allow, step‑up, flag for review, or block and lock.
• Produce a calibrated risk score plus reason codes so analysts can triage flagged accounts and account holders can understand enforcement actions.
• Maintain per‑account behavioral profiles that accumulate signals (login cadence, device set, geo patterns, API usage) and update near‑real‑time.
• Close a feedback loop: confirmed ATO from customer reports, analyst verdicts, chargeback‑linked incidents, and false‑positive overrides feed into training and threshold tuning.
• Support threshold governance: business‑tunable action thresholds per merchant tier and action sensitivity, adjustable without re‑deploying the model.
• Train under severe class imbalance (0.01–0.1% ATO rate) using cost‑weighted loss that reflects dollar‑impact differences across account tiers.
• Handle structurally delayed labels: customer ATO reports arrive 7–14 days later on average, analyst investigations have partial coverage, and some ATOs are never reported.

Non‑functional requirements

• Online scoring latency within ~100 ms p99 for sensitive‑action gating and ~200 ms p99 for login‑path scoring.
• Sustained throughput of ~10k scoring requests per second with ~50k RPS peak capacity during credential‑stufing bursts.
• Scoring service availability >= 99.95% with regional failover and graceful degradation (rule‑only fallback for low‑sensitivity actions, fail‑closed step‑up for high‑sensitivity).
• Session‑level behavioral features reflect events within seconds. Account profiles and graph features refresh hourly to daily.
• Daily batch retraining with emergency hot‑fix retraining triggered by drift alerts.
• All enforcement decisions logged with model version, feature snapshot hash, and reason codes for compliance and customer appeal.

Out of scope

• KYC identity verification at account registration or onboarding.
• Payment transaction fraud detection (stolen card use, card testing) — a distinct ML problem with different signals, labels, and action space.
• Anti‑money‑laundering and sanctions screening.
• Bot detection at the WAF layer (rate limiting, CAPTCHA). We consume bot‑detection signals as features but do not design the bot‑detection system itself.
• MFA and OTP challenge delivery infrastructure. Our system decides when to trigger step‑up, not how the challenge is delivered.
• Password hashing, credential storage, and authentication protocol design.
• Merchant onboarding risk scoring.