Rethinking False Positives

I’ve been thinking about performance, fraud, and anomaly detection lately. For a long time, I told myself that the nature of solving for fraud was one that should be seen as identifying where ‘expected behaviors’ go wrong. I feel like I need to modify that thinking. In the case of synthetic fraud, we know that there are many instances where the engagement of the identity with an FI does not involved extracting money from the FI. Sometimes synthetic identities are created for reasons outside of the FS ecosystem related to a CRA business.

A go-to signal for fraud is early delinquency. A problem here is that there are non-fraudulent instances that none-the-less go delinquent or default early. We shouldn’t be blind to the parallel idea that there are instances of fraud which do not go bad early. Given both of these aspects, there is some ambiguity pertaining to correctly measuring the effectiveness of solutions and designing those solutions.

There are two related topics to this line of thought: 1) the conceptualization of false positives; and 2) the bias toward building solutions where you are classifying something between one of two choices. These topics are entwined with each other as well. Discussion of false positives is nearly always discussed in the context of classification where the world is divided into True Positives, False Positives, True Negatives, and False Negatives. The framework can be extended to multivariate outcomes, but it is not the norm. Similarly, the industry standard for analytic approaches in Financial Services is based on distinguishing ‘bad’ from ‘good’. I suspect that it may make more sense to treat fraud framed by the possibility that it can be attempted or even carried out where, from the point of view of a particular FI, it may cause immediate, unplanned losses or not yield the expected benefit to the FI. An example of the latter would be a fraudster or fraud ring that needs multiple accounts that are used in some synchronous fashion where some of the accounts may not be used to extract money, but are vehicles (‘get-away cars’). Their use doesn’t align with a ‘normal’ consumer and the lender wouldn’t see the type of ROI as they usually do, even though there’s no immediate early, high value ‘loss’.

Having some ability to admit fraud that acts beyond immediate loss is potentially important because these fraudsters are potentially connected to those fraudsters who do extract immediate losses – it’s part of the same ecosystem of individuals that are the threat being addressed. Circling back to the False Positive aspect, it means that we should consider broadening what a false positive is, showing that many of those who do not lead to early or high-value losses, are none-the-less, not valuable partners to FI’s. Solving to minimize this type of ‘false positive’ will potentially eliminate important signals of the true ‘bad’ fraudsters.