Compute fraud probabilities with Bayes and Binomial

An online platform sorts accounts into fake or authentic. The prior probability that an account is fake is 3%.

Signals: every account recorded n = 5 independent sessions during the past week. Within a single session, a "suspicious action" happens with probability p_F = 0.5 for a fake account and p_A = 0.05 for an authentic one. An account is flagged once it shows at least k suspicious sessions.

(a) With k = 2, use the Binomial distribution to compute TPR = P(flag | fake) and FPR = P(flag | authentic). Give the formulas and the numeric results.

(b) Apply Bayes' Theorem to compute PPV = P(fake | flag) and NPV = P(authentic | not flagged) for k = 2.

(c) Manual review now runs only on flagged accounts. On its own the reviewer has sensitivity 0.90 and specificity 0.98. An account is actioned only when the rule flags it and the reviewer labels it "fake." Compute the resulting overall TPR and FPR, plus the updated PPV.

(d) For 1,000,000 accounts, compute the expected true positive, false positive, true negative and false negative counts under the process in (c).

(e) For k ∈ {1,2,3,4,5}, which k gives the highest F1 score at the stated prior, with the manual review step removed? Sketch the computation and give the numeric answer. Discuss how the best k would shift if the base fake rate rose to 10%.

(f) Point out which errors in (a)–(e) map to Type I and which to Type II errors here.

Overview: This question tests whether a candidate can do probabilistic modeling and statistical decision-making, centered on the Binomial distribution for session-level events and Bayes' theorem for posterior probabilities in a fraud-detection setting.

Community answers

Answer by SS

Accounts on a platform fall into two kinds: Fake (per-session probability of a suspicious action = 0.50) Authentic (per-session probability of a suspicious action = 0.05)

Every account had 5 sessions last week. You count how many of those 5 sessions contain a suspicious action.

Your detector flags an account when at least 2 of its 5 sessions are suspicious (k = 2).

The question is: how well does this rule catch fakes while leaving authentic accounts untouched?

What is TPR and FPR in plain English? TPR (True Positive Rate) = given that an account really is fake, how likely is your rule to catch it? You want this HIGH. FPR (False Positive Rate) = given that an account really is authentic, how likely is your rule to flag it by mistake? You want this LOW.

Why Binomial? Sessions are independent, and in each one the suspicious action either occurs or it doesn't. That is precisely the Binomial setup — comparable to flipping a coin 5 times and counting heads.

The Binomial formula for exactly k suspicious sessions out of n = 5: $P(X = k) = C(n,k) \times p^k \times (1-p)^{n-k}$ Here C(n,k) means "n choose k" — the number of ways to select k sessions from n.

"At least 2" means: $P(X \ge 2) = 1 - P(X = 0) - P(X = 1)$ Subtracting the "fewer than 2" cases from 1 is easier than summing every "2 or more" case.

Computing TPR — for a FAKE account (p = 0.50) Start with P(X = 0) and P(X = 1): $P(X = 0) = C(5,0) \times 0.5^0 \times 0.5^5 = 1 \times 1 \times 0.03125 = 0.03125$ $P(X = 1) = C(5,1) \times 0.5^1 \times 0.5^4 = 5 \times 0.5 \times 0.0625 = 0.1562$

Answer by SS

Let's work through it step by step, using the figures from part (a).

What we already know Prior fake rate: P(fake) = 0.03 (3%) So P(authentic) = 0.97 (97%) TPR = P(flag | fake) = 0.8125 FPR = P(flag | authentic) = 0.0226

Imagine 10,000 accounts to make it concrete Before touching any formula, this is the simplest way to see what is going on. Fake accounts = 3% × 10,000 = 300 Authentic accounts = 97% × 10,000 = 9,700

Now apply the detector: Fake accounts flagged (true positives) = 81.25% × 300 = 243.75 ≈ 244 Fake accounts missed (false negatives) = 300 − 244 = 56 Authentic accounts flagged (false positives) = 2.26% × 9,700 = 219.22 ≈ 219 Authentic accounts not flagged (true negatives) = 9,700 − 219 = 9,481

So out of 10,000 accounts: Total flagged = 244 + 219 = 463 Total not flagged = 56 + 9,481 = 9,537

PPV — Positive Predictive Value PPV = P(fake | flag) = "of all flagged accounts, how many are actually fake?" Using Bayes: $P(\text{fake} \mid \text{flag}) = \frac{P(\text{flag} \mid \text{fake}) \times P(\text{fake})}{P(\text{flag})}$ First find P(flag) — the total probability of being flagged: $P(\text{flag}) = P(\text{flag} \mid \text{fake}) \times P(\text{fake}) + P(\text{flag} \mid \text{authentic}) \times P(\text{authentic})$ $= 0.8125 \times 0.03 + 0.0226 \times 0.97$ $= 0.024375 + 0.021922$ $= 0.046297$ Now plug in: $PPV = \frac{0.8125 \times 0.03}{0.046297} = \frac{0.024375}{0.046297} = 0.5265 \ (52.65\%)$ From the concrete numbers: 244 / 463 = 52.7% ✓ In plain English: just over half of the flagged accounts are genuinely fake. Nearly half are innocent accounts flagged in error.

NPV — Negative Predictive Value NPV = P(authentic | not flagged)