Developing a scoring model for managing money laundering transactions using machine learning

Money laundering is defined by the 1998 UN Vienna Convention (Article 3.1) as “the conversion or transfer of property knowing that such property is derived from any offense(s), for the purpose of concealing or disguising the illicit origin of the property”, or put another way, money laundering is the act of making dirty money look clean.

To detect such activity, banks use transaction monitoring (TM) systems to analyse patterns of transactions within their institutions. However, TM systems are renowned for creating high volumes of false positives, which require large numbers of human resources in the form of financial analysts to review. These ever-increasing volumes are forcing banks to innovate to meet demand.

This article outlines and examines the performance of a “bolt-on” alert scoring model used by a major bank. For commercial privacy, both the country and institute involved remain anonymous throughout this study. Using machine learning (ML), the model automatically ranks suspicious activity as low, medium or high-risk enabling large volumes to be managed more efficiently through auto-hibernation or escalation for priority attention. This article examines the design and development in detail and provides a robust evaluation of performance and a blueprint for the financial industry to consider implementation of such models more widely.

According to the United Nations Office on Drugs and Crime, a staggering $1.6tn is laundered each year (Global Coalition to Fight Financial Crime, 2021). Money laundering has significant negative macroeconomic and social consequences. The potential consequences of money laundering include increased exposure to organized crime; undermining the legitimate private sector; weakening of financial organizations; dampening of foreign investments; and economic distortion and instability (Chaikin and Sharman, 2009).

While money laundering itself is a crime in most areas, it is preceded by other illegal activity that is the foundation (aka predicate offense). For example, the illicit act of tax evasion is a predicate offense, and the illegal proceeds generated would be laundered. Predicate offenses may vary from country to country and are usually codified in a country’s criminal code. For instance, in the USA, these offenses were initially created by the Bank Secrecy Act of 1970 and have been expanded by the 2001 USA’s Patriot Act (Chitimira and Munedzi, 2023). In 2021, the European Union (EU) issued the Sixth EU Anti-Money Laundering Directive which defines and standardizes 22 predicate offenses including narcotrafficking, tax evasion, murder, grievous bodily harm, corruption, fraud, smuggling, human trafficking, illegal wildlife trafficking and forgery, among others (EU Directive 2015/849, 2021).

The act of money laundering is traditionally recognized as being conducted through a three-stage process consisting of placement, layering and integration (Cassella, 2018). In stage one, placement, funds derived from criminal activity are introduced into the financial system (Cassella, 2018). Often, this is accomplished by placing the funds into circulation through financial institutions, money service bureaus, or cash-based businesses (Cassella, 2018). Examples of placement include mixing illegitimate and legitimate funds together, such as placing the cash from illegal drug sales into a restaurant; purchasing significant numbers of stored value cards with cash; dividing cash into lesser amounts and depositing it into numerous bank accounts (Cassella, 2018).

In stage two, layering, the funds are repeatedly moved through the financial system to obfuscate the source of the funds, which makes it more difficult to identify the origin (Cassella, 2018). Examples of layering include moving funds from one financial institution to another; moving funds within accounts at the same institution; placing funds in stocks, bonds or insurance products; reselling high-value goods, prepaid store cards or other stored value products (Cassella, 2018).

The final stage, integration, is used to re-integrate the newly “cleaned” funds so that they can be used for normal purposes (Cassella, 2018). For example, the launderer might choose to invest the funds in real estate, financial ventures or luxury assets.

Formed in 1989, the Financial Action Task Force (FATF) is an intergovernmental body created by the group of seven industrialized nations, to set standards and foster international action against money laundering. In total, more than 200 countries and authorities have committed to implement the FATF’s standards as part of a coordinated global response to preventing organized crime and corruption. As a result, countries worldwide have established frameworks that obligate financial institutions to identify and report suspicious financial activity (Kern, 2000).

Since its inception, the FATF has undertaken the study of money laundering typologies (i.e. methods and trends) as a key component of its work. In doing so, they have identified that common money laundering typologies include rapid movement of funds, structuring and use of dormant accounts (Gallo and Juckes, 2005). Rapid Movement of Funds involves the swift transfer of funds between various accounts or financial institutions, often without a clear business or legitimate reason. This behaviour can indicate attempts to obscure the origin of funds. Structuring involves the use of deposits or withdrawals just below regulatory reporting requirements to evade detection, and nay indicate suspicious activity (Gallo and Juckes, 2005). Furthermore, dormant accounts suddenly being used for elevated levels of activity, or multiple accounts used to move funds in a potential effort to obscure the source of money, are both common money laundering typologies (Ai and Tang, 2011; and Lai, 2010). When institutions suspect the presence of money laundering, often by identifying the outlined customer behaviour, they are required to submit a suspicious activity report (SAR). Failure to do so may make them liable to criminal or civil penalties (Pavlidis, 2023).

SARs are a tool provided by the Bank Secrecy Act of 1970. Originally called a criminal referral form, the SAR has become the standard form to report suspicious activity. The purpose of SARs is to initiate the pro-active investigation of potentially illegal activity (Ping, 2005). SARs are the key vehicle that financial institutions use to highlight suspected money laundering to the authorities and Financial Intelligence Units (FIUs) (Ping, 2005; Stalcup, 2015). FIUs analyse SARs, and if they suspect criminal conduct, the information is then shared with the appropriate law enforcement agencies for further investigation (Pavlidis, 2023; and Ping, 2005).

Spotting suspected money laundering is difficult as large institutions can process millions of transactions per day. Identification is heavily reliant on internal practices that are set up to identify potential indicators that suspicious activity is occurring (Hopkins and Shelton, 2019). The process of generating SARs starts with the detection of unusual activity by TM programmes, systems and processes designed to highlight behaviour that meets predefined typologies (Repin et al, 2017; Sapozhnikova et al., 2017, 2018). Large financial institutions deploy automated TM systems to detect money laundering (Chau and van Dijck Nemcsik, 2020). These TM systems flag transactions as “alerts” if they meet the criteria of the typologies/rules/scenarios configured (Harvey, 2020).

The task of reviewing alerts falls to employees trained in analysis and examination of transactions for signs of illicit activity. The number of analysts required to review alerts can be considerable and large financial institutions can employee hundreds of analysts. These financial analysts are employed within alert management teams, who are typically structured into three sub-teams known as level 1, level 2 and level 3. The level 1 review is sometimes referred to as an “alert Investigation”. To conduct a level 1 review, an analyst may utilize various sources of internal and external data, including customer records, payment data, public domain searches, government and third-party databases. If the analyst can determine that the alert poses no concern, then they can close the alert. If not, the alert would be escalated to level 2, which results in the creation of a “case”.

This level 2 review, referred to as a “case investigation” is where an analyst conducts further analysis of the activity. These reviews typically require the procurement of additional data from the customer, other internal functions, or use of tools that level 1 analysts may not have access. If a level 2 analyst is unable to obtain the required information to close the alert, then the analyst will recommend escalation to level 3 for the filing of a SAR.

As part of the level 3 review, a SAR is prepared and submitted to the FIU. As an example of scale, in the UK, over 460,000 SARs are submitted every year. Collectively, these reports contribute a great deal of information towards the anti-money laundering (AML) efforts of financial authorities. A typical SAR is less than 10 pages but a SAR for a complex transaction could be as long as 50 pages.

A limitation of TM systems is that they may not always efficiently detect money laundering, which leads to a high volume of false-positive alerts. A typical TM system may have an alert-to-SAR ratio of just 5%, and to address this issue, it is necessary to refine their performance (Ratanachu-ek, 2021). Two techniques traditionally used to achieve this are segmentation and tuning (Ratanachu-ek, 2021).

Segmentation involves dividing customers into homogeneous groups based on similar transaction behaviours or risk profiles, allowing for threshold settings for distinct groups (Ratanachu-ek, 2021). This ensures banks have homogenous groups who possess similar behaviour, products, and/or services, and therefore, unusual activity is easier to identify. This method has inherent risks if not done correctly. For example, if all banking customers were placed in the same group, then this could result in comparisons between ordinary customers, students, high-net-worth individuals or even businesses. Clearly, these groups have little in common in terms of their financial transactional behaviour and if compared against one another may negatively affect the rates of false positives.

Tuning involves regularly analysing transactional data to set and adjust parameters, which minimizes false positives, while also ensuring significant alerts are not overlooked (Ratanachu-ek, 2021). For instance, what constitutes a “large transaction” for one customer, may be modest for another, necessitating different monitoring parameters for both. Parameters can be set for a range of features and examples include “transaction size”, “maximum monthly balance” and “number of cash deposits”. Setting the parameter values for certain scenarios can be quite simple, as some are defined by law. For instance, the threshold value for what constitutes a large transaction is usually outlined in localized legislation. However, other parameters must be set using local expertise or derived from a statistical approach that focuses on behaviours calculated to identify anomalies.

To help alert management teams assess risk, TM systems often assign risk scores to alerts based on a broad range of factors that are indicative of suspicious activity (Simpson, 2018). Risk scoring allows the institute to identify high-risk alerts quickly (Simpson, 2018). In recent years, the efficiency of TM systems has been significantly enhanced by technologies such as artificial intelligence (AI) (Gupta et al., 2023; Landge et al., 2024; Zhang and Chen, 2024), including ML (Gerlings and Constantiou, 2022; Oztas et al., 2022), and has enabled large organizations to deploy complex data mining techniques to improve the accuracy of detection.

The ease of digital banking has caused the volume of transactions monitored by financial institutions to dramatically increase in recent years (Oztas et al., 2022). The availability of digital data has also significantly enhanced the TM process as digital data provides many opportunities for identifying patterns and trends in activities that humans may overlook (Lokanan and Maddhesia, 2023). Studies on ML techniques applied in this domain have historically focused on detecting unidentified anomalies using historical AML cases (Lv et al., 2008), graph analysis (Alshantti and Rasheed, 2021; Desrousseaux et al., 2021; Rui and Wunsch, 2005), anomaly detection (Gao, 2009; Liu et al., 2008; Muhammed Shokry et al., 2020), behavioural analysis (Ketenci et al., 2021; Jullum et al., 2020) and risk rating (Larik and Haider, 2011). As such, the effectiveness of TM systems has been enhanced in recent years, especially with the use of AI and ML (Gupta et al., 2023; Landge et al., 2024; Zhang and Chen, 2024; Gerlings and Constantiou, 2022; Oztas et al., 2022).

Despite these studies, a review of ML techniques that improve TM has identified significant gaps in literature (Oztas et al., 2022). Specifically, there is a lack of research in the use of ensembled learning, reinforcement and deep learning methods; studies lacking information or reproducibility; minimal information on data sets used; and limited evaluation (Oztas et al., 2022). The review concluded that solutions to detect money laundering remain an ongoing research area, with insufficient literature available regarding ML methods for transactional monitoring, especially using real data sets (Oztas et al., 2022).

The purpose of this study is twofold. Firstly, we outline the creation of an ML “bolt on” TM risk scoring model. This ML model uses predictive classification to score the likelihood of an alert being a false or true positive based on historical data (Ratanachu-ek, 2021). Secondly, we explore the results of implementing the model in a real-world environment. This is valuable as it explores the potential to reduce workload on human analysts responsible for reviewing alerts, by “auto-hibernating” low-risk alerts and “auto-escalating” high-risk alerts.

Exploring these two objectives is important for several reasons. First, it may provide a blueprint of how to develop similar models. If effective, then these may help financial institutions increase capacity to detect money laundering. Second, it could also reduce the time taken to investigate suspicious activity, enhancing the institutions AML response. Finally, the use of this method could refine TM systems and extend this under researched area by outlining one of the first examinations of the alert scoring method for optimizing a TM risk assessment process.

“Bolt on” technologies that sit on top of rule-based systems have been cited as a method to improve efficiency of TM systems by using ML to risk-score and auto-hibernate or escalate alerts (Oztas et al., 2024). We achieved this by developing a model to mimic the analysts alert review process with the aim of improving efficiency, reducing investigation timeliness and improving the time-to-SAR duration. The model did so by scoring AML alerts to help conduct three actions:

1. automatic hibernation of low-risk alerts;

2. review of medium-risk alerts by the level 1 team; and

3. automatic escalation of high-risk alerts to the level 2 team for detailed investigation.

To score and provide a rank-ordered list of alerts, the bank used an ensemble ML technique called boosting which constructs, evaluates and tests the model to create a stronger and more accurate prediction model. The model used the LightGBM gradient boosting framework for its fast-training speed, low memory usage, accuracy and ability to handle large data sets. Comparative studies have demonstrated that LightGBM performs competitively with other ensemble learning algorithms like XGBoost and Random Forest when used for classification tasks (Li and Chen, 2020). LightGBM was also selected due to its ability to impose monotonic constraints on variables. This functionality to iteratively over-penalize erroneous decisions at each tree-building stage contributed to improved model performance.

The data used to build and train the model was collected from three years of TM alerts, compliance flags, SAR data, customer demographic details and transactional data. The model was trained on the relationship between analysts’ historical decisions and the associated customer, account and transactional behaviour. This was achieved by conducting iterative experimentation and model training on the historical data.

While all alerts are considered suspicious, those leading to SARs send a much stronger signal. To leverage this, the model oversamples alerts that produced SARs by three times, allowing the estimator to emphasize this signal during the learning process. The goal of doing so was to ensure that the estimator iteratively over-penalized erroneous false negative decisions for SARs at each tree build, which reduced false negatives.

A predictive binary classification model was developed to risk score alerts. Initially, the data contained approximately 2,000 predictive variables (features), which were assessed based on their contribution towards performance. These were later reduced to improve and hyper tune the model. The hyperparameter grid related to this process can be found in  Appendix Table A1.

The feature selection methodology involved multiple sequential steps. While the initial model performed satisfactorily, further simplification was pursued to reduce complexity. Feature importance scores generated by LightGBM help identify the most influential features in integrated classification models (Huang and Wang, 2023). Using LightGBM feature importance method, the lowest 10% of features were iteratively removed until the model performance was compromised. This enabled us to reduce the initial set of 2,000 features to 356 that could be assessed based on contribution towards performance.

At this point it became evident that most feature variables represented customer transactional behaviour calculated over different lookback windows, leading to a set of highly multicollinear variables. Once this point was reached, we incorporated insights from the banks compliance experts to further narrow down the feature variables retaining only those with the highest correlation to the target variable and reducing the final list to 91. A summary description is provided in Table 1.

Table 2 provides the training and out-of-time (OOT) testing sample periods, sample counts and positive class rates for the model. The model was trained using 16 months of data from January 2021 to April 2022 and validated using 12 months of data from May 2022 to April 2023. The training sample size was 59,685, while the validation sample size was 105,589. Positive class rates were consistent across training and validation periods, with the model showing 34% positive cases and 30% in validation. This consistency indicates stable data distribution over time. In addition, the training sample included 3,281 SARs, while the validation sample included 3,190 SARs. This substantial number of SARs in the training and validation datasets ensured that the model could learn and validate effectively. Overall, the table highlights that the model has adequate sample sizes and maintains performance consistency across different time periods, ensuring robust training and evaluation.

We also used receiver operator characteristic (ROC) curves and the area under the curve (AUC) to evaluate the model performance. ROC curves help assess how well the model distinguishes between true and false positives, determine the optimal threshold for classification and compare performance of different models. LightGBM’s hyperparameters provided a mechanism for maintaining balance between model underfitting and overfitting. RandomizedSearchCV from scikit-learn was used to search the parameter grid.

The difference between training and test ROC AUC scores (diff_roc_auc) was selected as the objective function of the optimization analysis. We aimed to minimize diff_roc_auc to ensure consistent model performance across training and test data sets. For the model, the selected hyperparameter configuration resulted in a diff_roc_auc of approximately 0.03, with a mean training ROC AUC of around 0.86 and a mean test ROC AUC of around 0.83. These results, shown in  Appendix Figure A1, indicate that the model maintained reliable performance with minimal overfitting, as evidenced by the minimal differences between training and test ROC AUC scores.

As the model must rank alerts, this was also calibrated. The ROC, and AUC are again selected as performance evaluation metrics. The ROC curve is used as it plots the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings. The AUC provides a single-number summary of the curve, ranging from 0.0 to 1.0, where 0.5 represents random model performance and 1.0 represents perfect model performance. The model was back-tested over four OOT test periods. Each OOT sample contained three months of alert data generated from May 2022 to April 2023. The Model Classification report in  Appendix Table A2 lists the model AUC values for all OOT tests, with ROC curve figures provided in  Appendix Figure A2. Metrics for validating TM alerts include precision, TPR of level 2 cases and TPR and false negative rates (FNR) of SAR at top percentiles of prediction scores, such as the top 10, 20 and 30 percentiles.

The first phase of the model implementation was conducted as a six-month project to ensure that the model could be tested using real-time customer data without any associated risk from false negatives and positives.

During the pilot, for low-risk alerts, analysts were instructed to expedite their investigations using a “light-touch” review process which lasted 5 min versus the typical process which is considerably longer. For high-risk alerts, investigators were instructed to skip the level 1 review entirely and immediately escalate to level 2.

Throughout the pilot, results were monitored to track alignment between the model’s risk scores and decisions made by human analysts. On conclusion of the pilot, and after establishing successful performance of the model, the full implementation was conducted, and new working practices were put into place. The light-touch review was replaced by auto-hibernation of low-risk alerts and the threshold was increased to 20% of alerts and subsequently further increased to 30%.

In addition, overrides were incorporated into the auto-decisioning process that meant certain alerts could not be hibernated. For instance, if a customer had previously been the subject of a SAR, then future alerts could not be hibernated. Likewise, if a customer had two low scoring alerts hibernated in the previous 90 days, then the third alert could not be hibernated.

To assess the results, we used 11 months of continuous data, covering both the pilot and implementation period to assess performance. To understand improvements in efficiency, we use the data to derive time saved in analyst reviews.

Results indicate that the implementation of the risk scoring model for managing money laundering detection has been successful. Tables 3 and 4 outline the descriptive results obtained for low- and high-risk alerts, respectively, and indicate that the model performed extremely well.

With respect to low-risk alerts, we can see in Table 3, on average, the model categorized approximately 13% of alerts as low-risk (n = 15,696). During the first 2 months the number was approximately 9%, but this gradually increased to 19% in subsequent months. There were no SARs missed during the six-month pilot during the light-touch review process. After five months the light-touch review process was replaced with automatic hibernation of “low-risk” alerts and the model continued to perform extremely well, not missing any SARs.

With respect to high-risk alerts, we can see that on average the model categorized approximately 15% of alerts as high-risk (n = 17,954). During the first 3 months the percentage was approximately 7%–9%, and this was gradually increased in subsequent months to over 18%. As the percentage of alerts tagged as high risk was increased, the percentage of total SARs covered also increased.

In this context, we see the model performed excellently throughout the entire period, with an average of 57% of the total SARs reported during this period being covered by top 15% of alerts.

As discussed, the aim of the model was to enhance the efficiency of investigations into suspicious activity. The best way to measure this is the timeliness of investigations, as delays result in missed opportunities for investigators to potentially freeze assets and funds held by the bank, and secure vital evidence. Figure 1 illustrates the lead time between generating an alert and submitting a SAR has dramatically reduced by 61%.

To help understand if the implementation of the model improved the efficiency of the bank, we equated the time saved into human resources. This is achieved by calculating the number of alerts processed by employees. For example, without the model, an employee usually processed approximately 120 alerts per month. In February 2024, the model raised 14,540 alerts, which would ordinarily require around 121 individuals to process. However, by using the model this was dramatically reduced by the equivalent of approximately 45.6 full-time employees (FTE) (23.4 FTE for low-risk alerts and 22.2 FTE for high-risk alerts).

As business volumes increase further, the total alerts are also predicted to increase, and hence the number of FTEs saved is also forecast to increase. This underscores the significant impact the new model can continue to have on operational efficiency, potentially serving to help mitigate the increases in alerts generated by higher transaction volumes.

This study had two aims: firstly, to outline the creation of a “bolt on”, ML hibernation model for managing money laundering transactions; Secondly, to examine the results of implementing the model in a real-world environment to assess the model’s efficiency. Both have been met and, in this section, we discuss the implications.

We began this article by detailing how the risk scoring model was developed and implemented. In doing so, we have provided a blueprint for other financial institutions to consider similar approaches to AML. This is significant because previous studies of ML applications in the domain of AML (Alshantti and Rasheed, 2021; Desrousseaux et al., 2021; Gao, 2009; Jullum et al., 2020; Ketenci et al., 2021; Larik and Haider, 2011; Liu et al., 2008; Lv et al., 2008; Muhammed Shokry, 2020; Rui and Wunsch, 2005) have not explored this type of use before, particularly in the context of ML ensembled techniques (Oztas et al., 2022). It also addressed several cited limitations of previous research by increasing the depth of information in the data sets used and providing a formal evaluation (Oztas et al., 2022). Importantly, the blueprint and evaluation metrics used in this study ensure its reproducibility, addressing another previously identified issue (Oztas et al., 2022).

In our literature review we outlined how industry experts considered alternative methods to improve the efficiency of TM systems, such as segmentation and tuning, to be limited due to the need to regularly adjust thresholds, which resulted in high numbers of false positives (Oztas et al., 2024). This study demonstrated that ML models can complement this function effectively, relieving operational teams of significant work and creating additional capacity for other activities.

Another benefit of the study is confirmation of viability of the “bolt-on” approach. Industry experts have highlighted this as a way financial institutions can reap benefits of ML without abandoning traditional measures, such as the “rules-based” approach presently used (Oztas et al., 2024). Thus, financial institutions could integrate the models with assurance they will not only maintain but enhance the status quo. This should provide reassurance for compliance units and regulators that this approach is safe.

In addition to technological and methodological contributions, this article demonstrated the models were highly efficient at categorizing suspicious activity. Their success in consistently identifying low-risk alerts correctly significantly reduced the volume of false positives that needed manual review. Prior to implementation, 100% of alerts had to be reviewed, by the end of the examined period, 19% of alerts were hibernated and 18% of alerts were auto-escalated resulting in a marked improvement in the effective alert rate at level 1, demonstrating that the “bolt-on” model performed significantly better than the TM systems alone.

Furthermore, this study identified that the model contributed to a significant reduction in time taken to investigate and submit SARs, which reduced by 61%. This greatly enhances the timeliness of investigations, which is crucial for money laundering prevention as research indicates that timely responses enable institutions and law enforcement agencies to make better use of the investigation timeframe after an offence is identified, enabling them to more rapidly address threats, freeze assets when necessary, and secure vital evidence before it is lost (Zolkaflil et al., 2019).

This study has several limitations that should be acknowledged and can be addressed through future research. Firstly, its scope is limited to a single major bank. This means that the model created was not validated across different financial institutions. As it was developed bespoke for the bank in question, this limits the generalizability of the findings, albeit we suggest the methodology can be universally applied. In this regard, application of the method in other AML areas, such as to assist in sanctions screening or monitoring of politically exposed persons (PEP) would broaden literature on the application of AI and ML in AML. Second, the reliance on historical transaction data and analysts’ decision-making may have introduced potential bias. This is a flaw inherent to all AI but one worthy of further examination to ensure the model does not unjustifiably or disproportionately impact any one demographic. Finally, we used a single ML methodology; therefore, incorporating other techniques, including deep learning, reinforcement learning and graph-based models, could enhance or improve predictive performance and capture more complex relationships in the data, and should therefore also be explored further in the context applied in this study.

Together, these findings echo previous studies that have outlined the potential for improved efficiency to reduce operational costs associated with AML compliance (Khan et al., 2018). The implications of these contributions are that they can help banking institutions effectively keep pace with the volume of transactions, which has a positive effect on reputation through enhancing their AML response (Nicknora, 2024).

In conclusion, improved efficiency is vital for banks as those institutes who are better at AML compliance also outperform competitor’s (McCarthy et al., 2019). This should serve as a commercial motivator for others to consider the blueprint we have presented.

Figure A1 

Figure A2 

Table A1 

Table A2