Case Study: Real-Time Payment Fraud Detection (Stripe / PayPal-style)
"Design a system that decides, while the card is being charged, whether this payment is fraudulent" is a staple prompt at Stripe, PayPal, Adyen, Square, banks, and every marketplace that carries payment risk. It is a rich interview problem because almost every default ML habit is wrong here: accuracy is meaningless at a 0.1 % positive rate, labels arrive weeks late and are noisy, the adversary adapts to your model, the decision has to be made in tens of milliseconds inside someone else's request, and every prediction may need to be explained to a regulator or a merchant.
This subject is a model answer following the 7-step framework from the ML System Design Framework subject — clarify requirements → business objective to ML objective → labels and data → features → model → offline and online evaluation → serving and monitoring — with fraud-specific detail at each step. It leans on the general material from the ML Data Pipelines and Feature Stores, Model Serving and Deployment and ML Monitoring and Drift subjects rather than repeating it, and it contrasts with the ad-CTR case where useful (there the problem was calibration under an auction; here it is asymmetric costs under an adversary).
Aim to deliver the whole answer in ~40 minutes and then use the follow-up questions to stress-test it.
Step 1 — Clarify Requirements and Assumptions
Questions to ask first
- Which fraud? Assume card-not-present transaction fraud (stolen card details used online). Account takeover, merchant fraud and money laundering are related but different systems.
- Who is the customer of the decision — the payment processor (protecting merchants and the network) or a single merchant? Assume a processor serving many merchants.
- Where in the flow does the decision sit? Assume inside the authorisation path: the decision must be made before the charge is sent to the card network, so latency is bounded.
- What actions are available? Approve, decline, step-up authentication (3-D Secure challenge), or route to manual review.
- Is there a human review team and what is its capacity?
Assumptions to state
| Quantity | Assumption |
|---|---|
| Transactions per day | ~100 M (≈1,200 TPS average, ~5,000 TPS peak) |
| Fraud rate (by count) | ~0.1 % → ~100 k fraudulent attempts/day |
| Average transaction | 60; fraudulent transactions skew higher (~150) |
| Decision latency budget | < 100 ms p99 for the whole risk step, ~20–30 ms for feature fetch + model |
| Label delay | chargebacks arrive 2–12 weeks after the transaction |
| Manual review capacity | ~5,000 transactions/day (0.005 % of volume) |
| Cost asymmetry | fraud loss ≈ full amount + chargeback fee (~$15–25) + network penalties if fraud ratio is high; false decline ≈ lost margin (~2–3 % of amount) + customer churn |
| Adversary | actively probes thresholds, rotates cards, devices, IPs; attack patterns change in days |
The two facts that shape everything else: the decision is inside a synchronous payment call, and the ground truth arrives weeks later from a party (the card issuer) you do not control.