Payment Fraud Screening: 90%+ Accuracy With 40–70% Less Manual Review
Fraud detection is a threshold problem wearing a modelling problem's clothes. Getting the operating point right mattered more than the choice of classifier.
- Client
- Online payments operation
- Industry
- Payments
90%+
Accuracy, class-balance dependent
40–70%
Less manual review
ROC-AUC
Evaluated on risk metrics
Tuned
Precision / recall trade-off
The problem
Transaction screening was largely manual, which meant it was slow, expensive and inconsistent. The review team could not look at everything, so they looked at what rules flagged, and the rules were both too broad in some places and blind in others.
Fraud detection is severely imbalanced — the overwhelming majority of transactions are legitimate — which makes accuracy a misleading headline metric. A model that approves everything scores impressively and catches nothing.
What we built
Structured preprocessing and feature engineering over the transaction data, deriving the behavioural and velocity signals that actually separate fraud from unusual-but-legitimate activity, rather than feeding raw fields to a classifier and hoping.
Supervised models selected and evaluated on precision, recall, F1 and ROC-AUC together, because each alone hides a different failure. Accuracy was reported for completeness and ignored for decisions.
A threshold and validation workflow that turns model scores into an operating decision, with the operating point chosen against the real cost asymmetry — a blocked legitimate customer and an approved fraudulent transaction are not equally expensive, and the ratio between them belongs in the threshold.
Results
The classifier reaches 90%+ accuracy depending on class balance, with the risk metrics that actually govern the decision reported alongside rather than hidden behind it.
Manual review workload fell 40–70%: high-risk transactions are flagged automatically for human verification, and the clearly legitimate bulk stops consuming reviewer attention. The reviewers spend their time on the cases where judgement changes the outcome.
False positives fell through threshold tuning while detection strength held, which is the trade the business cared about — every false positive is a real customer being told no.
What we'd flag
Fraud is adversarial and non-stationary. A model fit on last year's patterns decays as tactics change, and this needs monitoring and periodic refitting rather than being treated as finished.
We deliberately did not automate the decline. The system flags and prioritises; a person decides on the high-risk cases. Automating a decline is automating the loss of a customer, and that is not a boundary we would move without a much stronger case.
