Editorial & Advertiser Disclosure Global Banking And Finance Review is an independent publisher which offers News, information, Analysis, Opinion, Press Releases, Reviews, Research reports covering various economies, industries, products, services and companies. The content available on globalbankingandfinance.com is sourced by a mixture of different methods which is not limited to content produced and supplied by various staff writers, journalists, freelancers, individuals, organizations, companies, PR agencies Sponsored Posts etc. The information available on this website is purely for educational and informational purposes only. We cannot guarantee the accuracy or applicability of any of the information provided at globalbankingandfinance.com with respect to your individual or personal circumstances. Please seek professional advice from a qualified professional before making any financial decisions. Globalbankingandfinance.com also links to various third party websites and we cannot guarantee the accuracy or applicability of the information provided by third party websites. Links from various articles on our site to third party websites are a mixture of non-sponsored links and sponsored links. Only a very small fraction of the links which point to external websites are affiliate links. Some of the links which you may click on our website may link to various products and services from our partners who may compensate us if you buy a service or product or fill a form or install an app. This will not incur additional cost to you. A very few articles on our website are sponsored posts or paid advertorials. These are marked as sponsored posts at the bottom of each post. For avoidance of any doubts and to make it easier for you to differentiate sponsored or non-sponsored articles or links, you may consider all articles on our site or all links to external websites as sponsored . Please note that some of the services or products which we talk about carry a high level of risk and may not be suitable for everyone. These may be complex services or products and we request the readers to consider this purely from an educational standpoint. The information provided on this website is general in nature. Global Banking & Finance Review expressly disclaims any liability without any limitation which may arise directly or indirectly from the use of such information.

Data and Analytics – how can event streaming help

By Lyndon Hedderly, Director Customer Solutions at Confluent.

Conjure up the frustration, anger and feeling of helplessness you get when stuck in traffic trying to get home, to a meeting or flight, or some other pressing deadline. Driving around most cities, it quickly and depressingly becomes apparent the transport infrastructure is not optimised for the motorist. Traffic congestion isn’t just highly annoying – it’s costly. We see lost productivity, pollution and overcrowding in major cities due to transport links. In short, congestion is bad for the economy and people.

What’s this got to do with finance and regulation? Congested road traffic is analogous to data flows in an enterprise. Cities have often seen huge increases in demand for transport infrastructure, without the luxury of long-term planning required to provision for this. For example, just remember the last time you were stuck in traffic because of a major sporting event or rush hour. Similarly, most enterprises have applications and architectures that have evolved over time. These complex, inflexible legacy architectures struggle to meet increasing demand, new digital business models, greater customer experience expectations and fast evolving operational requirements including, regulatory requirements.

We see a ‘spaghetti mess’ in both road layouts and enterprise application integration diagrams. Suggesting alternatives is often met with objections based on cost, practicality and scale of change. With both a city and technology architecture, it’s nearly impossible to rip-and-replace the existing architecture or infrastructure. Instead, we make do with small fixes and the associated delays, gridlocks and missed deadlines.

That said, frustration often results in innovation. Lack of transport options in San Francisco famously led to Uber – now the de facto adjective for innovation. Companies want to be the Uber of banking, or the Uber of retail, the list goes on. More recently, we’ve seen burgeoning alternative modes of transport, such as electric scooters and eBikes.

The data hurdles faced by financial institutions 

Lyndon Hedderly
Lyndon Hedderly

Before explaining event streaming, let’s take a look at how the current pain of moving data around an enterprise impacts financial services organisations. Most readers will know this story well. After the 2008 global financial crisis, regulators grip on banking institutions tightened with the introduction of a raft of new and necessary regulation. As an example, financial institutions are now being asked to report on the Markets in Financial Instruments Directive (MiFID II) and Markets in Financial Instruments Regulation (MiFIR), the second Payment Services Directive (PSD2) and the Common Reporting Standard (CRS) amongst many other regulations.

Each regulation often requires accessing and exposing massive amounts of data from numerous data sources and reporting on this on a regular basis. And regulations change. This has made regulatory reporting a time consuming and error-prone task. Like a city planner being asked to enable the free flow of all traffic through a congested city, many banks have found this feat quite impossible and have simply failed at the task.

These failures are real. They have cost the world’s top investment ‍banks $43bn in fines over the past seven years, making it the single most expensive compliance issue in banking today. Processing data accurately and in real-time has become such a big issue that Deutche Bank has set aside £3 billion to cover potential fines incurred. That’s a lot of frustration, anger and helplessness.

Unpacking the 3Vs of data

So, how does event streaming help?

According to the 3Vs model, the challenges of big data management result from three properties of data; Volume, Variety and Velocity. This is true for dealing with data for regulatory compliance purposes.

  • Volume – with the rise of digital and mobile banking, the sheer amount of data is mind boggling. There used to be just a few transactions in bank branches per day. Now we have a combined effect of transactions becoming much easier and therefore greater in volume and we are collecting much more data in general. Apps are now the biggest route to checking bank balances and executing transactions in Britain – being used over 2.6 million times a day. This requires additional compute resources impacting the aging, costly and maxed out mainframe systems – the same systems which own the processing cycles for regulatory compliance and reporting.
  • Variety – we now have many different types of data, from many different sources. Services are evolving rapidly, with KYC (Know Your Customer) initiatives. These include new tech and innovative services. For example, we’ve seen some banks implement automated services to apply for a mortgage through a video link, or wristbands that can be loaded with credit, which work with contactless readers. All the time, these systems have to comply with the General Data Protection Regulation (GDPR).
  • Velocity – the speed at which we need to analyse data is increasing all the time. Many regulatory requirements have specific time deadlines. For example MiFiD II requires firms to timestamp trades down to the microsecond and report basic details of their trades almost immediately, so that the information can be circulated in the market. The near real-time broadcasts of trade information is set to improve the transparency of pricing and offer greater insight into how prices are quoted and formed. Historically, it takes many financial firms weeks or even months to pull this data together, from multiple sources and systems.

Event streaming allows financial institutions to cope with the 3V’s of data in order to remain compliant in an efficient and cost-effective way. Many organisations are now implementing event streaming as the backbone, or central nervous system of their enterprise, partly driven by regulatory requirements.

Event streaming: The magic wand for data analytics 

There are many additional benefits of working with events in an event streaming platform. Reporting, or D&A (Data & Analytics) is one of them. Among IT and digital decision-makers, more than half (or 54% to be precise) of UK financial services organisations struggle to draw valuable insights from data that they collect according to a report by Claranet. And according to Forbes, an estimated 60 percent of a data scientist’s time is spent cleansing their data before being able to do any analysis.

An event streaming platform can help address these challenges.  Event streaming architectures help take the human out of the process, as they’re designed for services (and micro services) to ‘talk’ to each other. Rather than the traditional 3-tier request-response style architecture which serves a UI, which in turn serves a human or creates a report, the event streaming platform is really a step forward for the software defined business. This, of course, is a great foundation as we move into the age of Machine Learning (ML) and Artificial Intelligence (AI) – which are sure to play a part in future regulation.

Back to our city analogy, imagine a sci-fi scenario, whereby a city planner could magically implement a fast transport platform to off-load all those people stuck in gridlocked cars, buses, even trains and transport them to their chosen destination in near real-time. At the same time, the cars autonomously drive to their destination in a fast efficient manner without any human interaction. In our enterprise scenario, by using event streaming, this scenario is possible for data flow in an enterprise.

More and more I’m seeing business cases built for global banks to implement an event streaming platform, where the case is made on the back of risk-cost avoidance. That is, implementing an event streaming platform can help prevent fines potentially incurred by failing to meet regulatory requirements. Is this magic? No, it’s real and it has the potential to save FS organisations millions of dollars.