Stream Processing Driving Financial Services Digital Transformation

By Konstantin Knauf, Solutions Architect at Ververica 

Modern financial services organisations are looking to transform their operations into real-time and software-operated businesses of tomorrow. According to IDC, by 2025, nearly 30 percent of the so-called “global datasphere” will consist of real-time information. What role does technology play in this? Well, stream processing is destined to offer a paradigm shift for data processing in the finance industry of the future. Banks and other financial institutions are going through an extensive digital transformation in the face of market volatility, consumer behaviour and a shift in regulatory requirements. Changes in the finance industry can be witnessing as they transition from largely un-digitised and traditional businesses to becoming organisations adopting highly-automated largely software-operated operations. According to analysts, financial services spend on big data technologies will surpass $14 billion in 2021, a 55% increase since 2018.

Stream processing, and the associated streaming data architecture environment, allows financial services organisations to react to information in a completely different manner, able to benefit from real-time insight with 24/7, data-driven applications that allow businesses to react on changes and alerts instantly. As financial services organisations adopt stream processing, what innovations can we expect in this industry?

With a streaming data architecture you can process data at the very moment it is generated, the exact point when it’s most valuable. From a cybersecurity perspective, stream processing empowers the finance industry to build real-time fraud detection systems with powerful machine learning algorithms. Such a collaboration enables fraudulent activity to be detected in real-time to head off potential business losses, not to mention the potential to avert a negative customer experience. As such tools become more and more sophisticated, financial services organisations will benefit from a more powerful response in the face of the cybersecurity threat spectrum — credit card fraud, identity theft or fraudulent transactions.

One example of a bank using stream processing to power its real-time fraud detection engine is ING. They were able to build an Apache Flink-powered risk engine that allows the company to respond to new, previously-unknown threats instantly. ING’s fraud detection systemsupports multiple goals for the business, such as rule-based alerting as well as creating and scoring machine learning models.

 With the introduction of so-called ‘open banking’ (under the PSD2 legislation), banks’ customers, whether they are consumers or businesses, are now able to use third-party providers to manage their finances. This new crop of fintech companies is starting to exhibit a significant competition to the finance industry. They can innovate faster and provide a more personalised experience for their customers. By adopting stream processing, the finance industry has the ammunition to stay ahead of this by building a 360° customer view program analysing data in real-time. Financial services organisations have millions of customers, generating billions of transactions on a daily basis. These customers are also engaging with the company’s web and mobile portals, as well as customer services tools and agents. Now imagine if these organisations could process and respond to any of these data transactions in real-time. Such an agile approach allows for significantly improved monitoring of customer health and loyalty. Additionally, real-time data processing enables a company to create a more tailored product offering for customers, depending on what transaction types they favour or how they interact with the website, mobile app, or customer service portal of the organization. The modern financial services industry faces many challenges — being able to completely understand the customer viewpoint and respond to it in real-time is a great opportunity to cope in this fast-paced landscape.

As an example, Capital One uses stream processing and Apache Flink for real-time monitoring of customer activity data to ensure that issues are detected and resolved proactively while providing an enhanced digital customer experience.

Financial services organisations operate in a highly regulatory and complex environment. Stream processing allows reducing the work involved in reporting to regulatory bodies by automating and streamlining the process. It can enable technology-led, real-time compliance that moves away from manual checks. Stream processing offers always-on, data-driven systems that alert and report instantly on the current state of the business to different regulatory bodies. It can also be used to maintain a real-time market position across the organisation that provides a current view of the bank’s risk exposure. Streaming data architectures allow compliance departments to continuously process information in real-time thereby avoiding any potential fines by regulatory bodies due to non-compliance.