Lexicon surveillance is not dead
An explosion in data volumes, proliferation of platforms, and evolving regulatory requirements has made effective monitoring of employee communications much more difficult. The remote and hybrid working environments financial firms have adopted due to the coronavirus (COVID-19) pandemic have driven increased use of emails and chats, and accelerated use of collaboration and web conferencing channels such as Slack, Microsoft Teams, WhatsApp, Twitter and Zoom.
Amid this rapid change, there is a debate among compliance teams over the efficacy of traditional lexicon surveillance – using simple keywords and phrases to monitor electronic communications – versus artificial intelligence (AI) technologies. In fact, firms that use both methods – as appropriate, and complementing a robust surveillance operating model – are best positioned to manage regulatory and internal risk.
The case for lexicon surveillance
Financial firms are required to perform surveillance on their employees’ electronic communications to meet both regulatory obligations and internal compliance mandates.
Regulators have gotten much more aggressive in assessing large fines to financial market participants, for relatively simple and clear violations of surveillance policies. Though no surveillance system is perfect – and a truly malicious actor can find ways to circumvent a policy – well-defined lexicon-based surveillance systems remain a critical component to flagging potentially damaging communications for further investigation. As various transcripts of legal record filings illustrate, some firms could have avoided financial and reputational damage by optimizing their lexicons.
In addition to external regulatory requirements, it is becoming increasingly important for firms to monitor electronic communications to identify non-compliance with internal policies and codes of conduct which can impede goals to provide inclusive work environments for all employees.
Companies must also be concerned with data loss prevention and compliance with privacy laws such as the European Union’s (EU’s) General Data Protection Regulation (GDPR). For example, an employee sending a spreadsheet containing personally identifiable information (PII) or PDFs containing internal intellectual property to their personal email address can put a firm at significant risk.
Lexicon-based surveillance policies can be very simple – just a single word or a phrase – or very complex. Sophisticated scoring models, wildcards, Boolean, proximity and emoji operators can be used to increase or reduce the likelihood that a match is found. In the area of data loss prevention, lexicon surveillance can be very effective, through simple rules that identify patterns of specific data, such as driver’s license numbers, credit card numbers, IP addresses, document identifiers or any other recurring pattern.
For reviewers already familiar with lexicon surveillance, high numbers of false positives – an alert flagged in the system which is not a true issue – are a recurring problem. The percentage of false positives produced by lexicon surveillance is frequently over 90 percent, leading to many lost hours reviewing unnecessary alerts and reviewer fatigue, along with data storage and processing costs. Coupled with random sampling, which is a popular but inefficient method to determine true risk, the high number of false positives inherent in lexicon-based surveillance has forced the industry to look at novel approaches.
The benefits of machine learning, AI and NLP
To reduce false positives, and therefore compliance costs, banks and other financial institutions (FIs) are turning to AI technologies, including machine learning (ML) and natural language processing (NLP). The promise implied is a significant reduction in false positives, automated surveillance, identification of intent and improvement in detection of complex market abuse scenarios – a potential panacea to the most pressing challenges facing compliance departments.
However, ML introduces significant complexity. One of the advantages of lexicon surveillance is that policy administrators can quickly and easily make modifications to rules if needed. Responding to new incidents or a market event can be simplified by modifying lexicons and providing evidence of those modifications to regulators.
When it comes to ML, modifications and evidencing are not as straightforward. For example, it may be challenging for a traditional lexicon policy administrator to simply modify a ML model by editing keywords. ML models involve complex algorithms, which must be trained and fine-tuned by data scientists, and small changes in precision versus recall can alter the likelihood of an alert being generated.
ML also requires significant resources. Many large banks have invested millions of dollars in ML as a surveillance solution, seeking out engineering talent and taking the time to build the process itself. Data scientists are needed to build scenario models which are fed by massive data sets of electronic communications, as well as subject matter experts to help train models. A firm must also review and label thousands – potentially hundreds of thousands – of messages to improve the efficacy of its algorithms. Each analyzed communication must be tagged as a true or false positive, or with specific scenario attributes, so the model can begin to recognize patterns in the data. This exercise can be time-consuming and expensive.
ML also presents challenges around bias and explainability. Like humans, ML can be biased, with prejudiced results due to false assumptions. The old adage ‘garbage in, garbage out’ is especially relevant, with the best results dependent on good quality data. Because ML models are initially trained by humans, those inherent human biases can be passed onto the machine. Explainability is an equally important issue. Unlike lexicon-based systems, which are relatively simple to understand and explain, ML models are very complex and cannot be simply ‘read’ by a layperson. This barrier affects both policy administrators and regulators when trying to judge the efficacy of an ML system.
A hybrid approach
Lexicon-based surveillance works well within specific scenarios but comes at the cost of high false positives. ML offers new approaches, using complex models to predict whether new communications are true issues. However, ML will never completely replace lexicon-based surveillance, particularly with respect to data loss prevention or in instances when firms are searching for specific lexicons or patterns.
A hybrid solution can offer the best of both worlds. For example, if a firm is using both technologies, it could use NLP and ML to determine whether a message is business related or personal, with increased accuracy. A lexicon policy combined with entity extraction, such as a company name or a stock ticker, could be used to focus surveillance on only business communications, thereby reducing false positives.
Regulatory concerns
It is important that firms have a robust compliance foundation in place before experimenting with ML. Firms need to first make sure they are adequately covering the basics. Are they reviewing an appropriate number of alerts? Are written supervisory procedures documented, and are they being followed? Are they performing random sampling as one approach to surveillance?
Are they running regular reports on what has been reviewed? Are they reviewing their lexicons on a regular basis and updating them in response to incidents and market events? If they are audited by a regulator like the SEC or FINRA, can they prove that they have met their regulatory obligations? Many companies are not doing this today, and ML and NLP will not be able to fix these structural problems. Even worse, they will likely compound existing issues.
Considering increased regulatory scrutiny and pressure to drive down internal costs, financial firms must proactively and successfully monitor their employees’ communications and activity. It is vitally important for these organizations to expand their arsenals by partnering with a vendor that excels in incorporating ML into surveillance, while still providing robust lexicon tools.
This article was written by Bloomberg Product Manager of Compliance Solutions Eugene Semetsky and is reproduced from Risk & Compliance.