By Marcin Kacperczyk, a Professor of Finance at Imperial College Business School, and teaches on the AI & Machine Learning in Financial Services virtual Executive Education programme

Like most industries, the financial services sector is being rapidly redefined by emerging new technologies. No longer limited to operating at the transactional level (think mobile banking and digital payments), increasingly advanced tech is encroaching into the more human-dominated roles within the sector. A Bank of England survey recently reported that financial services firms expect to see significant growth in their use of technologies such as Machine Learning over the next three years.

These technologies are expected to be brought in to automate processes such as decision making – relying on algorithms to reach conclusions in a quicker, more accurate manner than a human. Aside of these obvious benefits which serve to boost company and industry performance and profitability, there is the added expected advantage that, by gathering the data necessary for this process to occur, the company can also discover new insights that can be exploited for commercial gain.

On paper the decision for financial services firms on whether to invest in such technologies is a no-brainer – however all is not as straightforward as it seems. Such optimism contrasts with additional reports that many firms are facing significant challenges when it comes to effective implementation of the technology necessary to make these benefits possible. In fact, global research and advisory firm Gartner has predicted, rather dramatically, that “through 2022, 85 percent of AI projects will deliver erroneous outcomes due to bias in data, algorithms or the teams responsible for managing them.”

The fact of the matter is that, as the technology has advanced the level of understanding needed for implementing them successfully has not kept pace, meaning that despite costly and time-consuming investments many financial services firms will fail to reap the benefits they are hoping for. A dire outlook indeed.

But all is not lost. Whilst we cannot halt technological revolution and industrial progress, we can extend greater efforts to ensure that the steps we are making towards advancing our own organisations are made correctly and with a clear benefit from the outset.

The key to making good decisions about AI and Machine Learning – and avoiding expensive failures – is to understand the current state of the technology, where it works well and, most importantly, where it doesn’t.

Through my own research, and in working with financial services executives on the AI & Machine Learning in Financial Services Executive Education programme at Imperial College Business School, I have identified five fundamental lessons for effective AI and machine learning deployment – essential for any financial services professionals to know.

• AI and Machine Learning are not the same

It’s easy to see why the two technologies are often seen as one and the same – after all, they provide similar functions. But they do so in wholly different ways and have different requirements for their success. It is vital for financial services professionals to know the differences between the two in order to understand how they can be used and where their best applications lie. Being able to make this distinction will also help determine if there is real cause to employ such technology in the first place.

AI, in its simplest form, creates computer programmes which can mimic, or even enhance, the ways in which the human mind works. But it has its limitations. Though no doubt clever, AI is actually routine-based. It learns from common data patterns it is exposed to and makes decisions based on the typical actions and results of similar scenarios it has previously recorded. However, it cannot forward think or innovate. Despite pop culture giving such technology an intimidatingly clever persona, and appearing as if it can think for itself, in reality we are still far away from achieving that level of capability.

Machine Learning, whilst similar, remains a sub-category of AI’s capabilities. Such technology is designed and implemented with a set specific purpose – to either fit models or identify patterns in the data it is provided with without being explicitly programmed and with little-to-no human intervention, and to report its findings back to the user. A good example of Machine Learning at work is search engines ranking results for the words and phrases we type into them. The lack of need for human guidance means it is perhaps, our most advanced step so far towards true artificial intelligence.

• Train your Algorithm

You may well have read about super-brain AI systems capable of beating world champion Chess players at their own game – one such example is Google’s AlphaGo system which was able to swiftly beat the world’s top human player of Chinese strategy game “Go!”. However, these machines are not outsmarting humans by being creative or possessing some higher intellect. They are simply detecting patterns in the game’s data, having been fed all the possible information there is to know about it by humans, and making their moves by selecting the most potentially successful option each time.

Why does this matter? Well, it’s a fantastic example of how, with the right lever of detail and understanding, Machine Learning can be applied to great success. Games like Chess or Go! are no doubt difficult to master, but they have very clear rules and set outcomes. This sort of structure is where Machine Learning can really thrive.

And for the Finance sector? If trained effectively, Machine Learning algorithms can help firms make sense of messy, complex data sets quickly and efficiently, taking the pressure off human staff with a slimmer margin for error. For example, Machine Learning has been employed successfully to help analysts distil hundreds of potential indicators of future investment returns to a few, more robust, measures. Such tasks have previously been not only difficult for human finance professionals to tackle effectively because of the sheer size of the data and variants but also relentlessly dull work due to it repetitiveness.

• Machines still cannot beat human rationale

Whilst such technologies certainly excel at identifying relationships and patterns within set data, they are not designed nor are they capable of doing this in more uncertain environments – i.e., when any need for human decision making or rationale comes into play. They cannot evaluate whether the correlations they find can be classed as legitimate so it is not just unsuitable but also potentially dangerous to use a tool such as Machine Learning to solve problems where discretion or special consideration may need to be made, or where there is no clear-cut way of defining right from wrong.

The finance sector, by its very nature, involves working in tune with your instincts and taking calculated risks – something which technology may not always be the best assessor of. The challenge of employing such technologies in these scenarios become apparent when we try to establish deep economic associations from data and answer questions such as “At what point do highly leveraged companies stop investing?” or “How might a rise in interest rates impact economic growth?”

Whilst an algorithm might well do a good job of identifying patterns in historic data to help answer these questions, its intellect is limited to exactly that – what has happened before. And past performance is not always an accurate indicator of future potential. In this instance, and in so many others, human intellect still remains superior.

• Good data is vital

It might sound obvious, but you need good data to get good results. Machine Learning is only as reliable as the data it has been presented with. Many of the finance practitioners I have spoken with have expressed a shared frustration of finding it difficult to generate enough data of the right quality in order for the technology to generate useful output. It is, I believe, a misconception that the finance industry is awash with data. Whilst this may well be true for transactional areas such as payments, data on company performance, for example, is compiled very differently and as such does not meet the same standards. It is typically very hard for analysts producing quarterly reports to collate sufficient data to build a statistically robust machine learning model. Similarly, going back to the previous point it, is difficult to also produce algorithms that do not create implicit biases in which fairness is important.

• Machine Learning can be applied to more than just numbers

Whilst there are certainly many challenges to its application, I believe there are good opportunities for us to use Machine Learning to enhance financial decision making – when the technology is applied correctly.

For example, the development of software that can conduct accurate textual analysis means it will soon become possible to apply Machine Learning technology to data which is not only numeric but textual too. I am currently working on a research project to explore exactly this prospect – analysing millions of words of analysts’ reports to compare what they have said to various audiences to what the numbers actually tell us.

And it’s not just text that machine learning can now begin analysing, but also images, which could prove to be a real asset to the financial sector in the future by creating more timely and reliable data for analysts. For example, an analyst could use real-time satellite imagery to record the number of cranes being put up in a city and use this information to help measure the levels of construction activity over a given time-period, which could then help produce information in advance of industry surveys. Another use could be viewing and analysing the intensity of lighting in a specific urban area in order to help estimate the level of economic growth – supplementing information where financial reports may be lacking.

Machine learning for competitive advantage

If the financial services sector can move beyond the hype of AI and machine learning, and take the time to truly understand the realistic and practical applications of such technologies, there will be a lot to be gained. As an industry, there needs to be a period of reflection and realism, not just focusing on the how to use but also the why to use such technologies – and, most importantly, should it be used in the first place. Aside of the technical hurdles there are certainly ethical and regulatory issues to consider. A good example would be the use of so-called “robo-advisers” or AI chatbots. As their use grows there is a discussion that needs to be had around whether their algorithms can be truly objective, and how these services should be presented to customers. Additionally, as Machine Learning’s ability to digest different types of data expands there are questions which need to be asked about how to ethically use data from other sectors, particularly when it comes to using this data as part of the decision-making process.

Those that can face those discussions and master the technological application process effectively stand to win big. The firms that gain a competitive advantage over the next few years will do so because they have been able so access and utilise novel data sets.

But, for the immediate future, it has never been more vital for those making technology investment decisions within the financial services sector to truly understand what machine learning and AI can offer right now, and what may be possible further down the line. Mastering this first challenge will set them on a path to future prosperity.