Dr. Giles Nelson, CTO, Financial Services at MarkLogic

The recent growth of financial regulations, such as the introduction of MiFID II in January 2018, has made it extremely difficult and time-consuming for financial services companies to integrate, analyse and report data collected from many years of operations. This accumulation of historic data can be collected from multiple sources in many different formats, meaning that the task of achieving transparency and compliance to complex regulations can be problematic because firms can never be certain of the data history – or lineage.

As financial services companies may face up to 374 new legislative initiatives by 2021, it’s critical that they implement an adequate data management strategy and invest in the appropriate technology sooner rather than later. Not only will this help firms to remain compliant now and in the future, but having a comprehensive overview of your organisation’s data lineage can be used to realise many other benefits across the operation of the business. It’s therefore important to gain the buy-in required from senior management and key stakeholders throughout the business to initiate a data management strategy that is fit for the 21^st century.

What is data lineage?

A business’ data tells its story. Analysing historical information can show you where the firm started and how it has transformed over time to get to its present state. This is true for not only financial statistics and customer information but also metadata about how the company has organised and managed all of this information over time.

Data lineage is essentially the history of information, tracked through its whole lifecycle from the creation of a record through any changes, dissemination or uses, up until the information is no longer deemed necessary for analysis or reporting. It is a critical component of any data-driven business, but especially so for firms operating in heavily regulated industries such as financial services.

The cost of compliance

Regulatory compliance requirements are putting greater transparency demands on firms to trace and audit data. For capital markets trading firms, data lineage must be implemented to support risk management, data governance and reporting for various regulations such as BCBS 239 and MiFID II. Although these regulations have been in effect for a while, financial firms are still struggling to meet the ongoing regulatory requirements as ad hoc reporting requests are exposing the limitations of existing compliance systems.

This is because the volumes of data stored by financial firms can often exist in individual silos and be stored in different formats, making it difficult for legacy technologies to integrate. As such, when an organisation is asked to present a 360 view of their trading process it’s difficult and time-consuming to achieve, particularly when dealing with a short notice request. This has already led to a number of fines by the FCA, with suggestions that yet more are still to come.

To resolve this issue, a process of data integration needs to occur with the objective of creating a single, unified data management system built for agility and flexibility. The aim of this is not to replicate the siloed data sets but to create a data hub, which sources data from its original systems and integrates it to give one, 360, view. Once complete, this will mean a firm’s data can be quickly accessed, queried and searched when faced with ad-hoc regulatory requests.

Operational efficiency

One of the biggest challenges to adopting data lineage is gaining management buy-in for initial data management projects. To increase the likelihood of project funding, data managers should not only make the case at the technical level, but also at an operational level. The business opportunities of sustainable data lineage more than justify the associated costs.

Through improved data quality and reliability, business leaders can harness insights from their data lineage, enabling more accurate analytics and informed decision-making. Data lineage projects also have the potential to reduce the costs associated with redundant data and systems as well as identification of reusable data and processes. Moreover, the insights that data lineage provides expand the ability to more easily identify new business opportunities, such as the creation of new products or services, through better understanding of data and visualisation of data and processes.

The future of digital regulations

As the regulators lead from the front by digitally transforming their own systems and operations, requirements will increase for improved data integration and lineage between financial firms and regulatory agencies to automate reporting, lower costs and facilitate greater transparency.

Many regulators are modernising their reporting systems from legacy mechanisms that require physical copies of reports to digital forms and streamlined databases. In order to co-operate effectively with the regulators, it makes sense for businesses to be one step ahead and implement a digital data lineage strategy up front. This will save costs and time further down the line, as firms are able to seamlessly meet the requirements of different regulations in the future.

The effective governance of customer information, financial trade records and data usage threatens to become a crippling burden unless technology can keep pace. Data lineage plays a vital role in meeting complex regulatory requirements and should be utilised towards making compliance easier. Businesses may need to reassess their data management capabilities as regulations become more complex and digital, in order to sustain data lineage now and in the future.