David Mackay, Senior Director, at Ness Software Engineering Services

The financial services industry is evolving at an exponential rate in this age of digital transformation. Shifting customer expectations, disruptive technology, and demanding regulatory requirements are constantly reshaping the sector. Many firms are finding that their current infrastructure and platforms are simply not capable of supporting the proliferation of user-centric, omni-channel capabilities that customers have come to expect from interacting with consumer-friendly software and apps created by internet giants like Apple and Google.

As banks and other firms continue to modernise IT systems, like moving to the cloud and virtualising data centres, addressing infrastructure and platform depreciation is often overlooked and contributes to ‘technical debt’ (also known as design debt or code debt). This debt is the additional cost of future development or maintenance of software attributable to structural problems which remain in or have been introduced into the software over time. It generally relates to how the software has been designed and built, rather than to functional or performance defects within IT systems. It typically crystallises at the point where software needs to be updated.

Many financial organisations allow technical debt to accrue because they are reluctant to upgrade legacy systems and risk introducing defects; they would rather just leave things running as they always have done. However, at the end of last year we saw several big banks experience billion dollar losses as they found themselves in positions where they could no longer delay a technology overhaul. Many experts say banks must clear these hurdles if they are going to survive in a sector where consumers expect to manage their money and other financial services from any device, with a seamless experience. Institutions will need to address accrued technical debt and its associated costs when contemplating digital enhancements aimed at driving growth and achieving business objectives.

In order to help financial services teams understand technical debt and minimise the total cost of ownership of software, it’s worth looking at three distinct stages of debt occurrence:

Stage 1: New software development

Often new software development takes place in a commercially-demanding environment, where quality can take third place after cost and time. Software is often created as a minimum viable product (MVP) to serve as a pilot system to deliver baseline capabilities. During development, many options are usually explored, and the code may be written in haste.

MVPs can be a very productive approach, as they encourage innovation to flourish. However, sometimes they are developed in a manner that doesn’t comply with development standards, such as going through a design review or adhering to naming conventions, and this isn’t best practice. There is often some work still to be done to refactor the code, tidy it up, and make it the foundation for subsequent development, as well as documenting it sufficiently. This is especially important in order to capture some of the design decisions that were made to aid subsequent development. At this stage, the technical debt is the difference between the cost incurred to deliver the project such that the functionality works, and the additional cost required to complete the project so that subsequent enhancements can easily be made to it at a later date. Normally a lack of budget prevents the technical debt from being addressed, which leads to problems in the future.

While later stages of debt are more likely to be encountered, this explains the creation of the original debt.

Stage 2: Ongoing software maintenance

Once software has been deployed to production, it usually needs to be maintained to keep track of evolving third-party interfaces, protocols, standards and other dependencies. Some of these updates are mandatory, such as changes due to regulatory compliance. This can require parts of the software to be enhanced, extended, refactored or rewritten. But often, the people charged with making these changes may not be that familiar with the code they are changing. Frequently the original code (or most probably a subsequent iteration) has functionality that has likely been subsequently replicated (normally through a “cut-and-paste” type copy) rather than re-used and adjusted based on the current context.

All of this leads to complications when upgrading software. Subsequent updates may be made without an understanding of the original design, which can add unnecessary complexity that itself needs to be maintained. In addition, customisations are often made to the code that then makes it harder to upgrade at a later date. This increases the cost and time to maintain and enhance the code.

Stage 3: Legacy software maintenance

These days, most enterprise software is created by large teams, with sharing of knowledge acquisition and transfer spread across a team. Many years ago, when older systems were being written that are still in use today – especially in huge organisations like banks – the nature of software engineering was very different. Software was created in antiquated languages by very small teams and sometimes even by individuals. These people were responsible for designing, developing, testing and releasing their code and were familiar with all aspects of the development lifecycle. This included an innate knowledge of the build process, build tools and scripts, deployment environments and supplementary information. Many of the original team members may have left an organisation – taking with them their knowledge – or simply forgotten how software they created 20 years ago actually worked and was built.

This is a common problem which results in the software having to be treated as a “black box” – viewed and analysed as a closed system in terms of its inputs and outputs. This means that when updates do need to be made, they can be expensive both in terms of finding people with the requisite skills, and in making and testing changes to avoid causing issues in other parts of the software.

Another challenge with legacy code is that while processes and tools have been created for more modern software languages to analyse non-compliance or bad-practice at the source code level, the tools simply do not exist for many legacy programming languages. These processes were not deemed important at the time and so associated tools were never created.

Addressing Technical Debt

There are established software engineering metrics that can be used to analyse technical debt. These include the ‘Maintainability Index’, normally calculated from lines of code measures, and ‘Cyclomatic Complexity’ which looks at the number of linearly independent paths through the source code, as well as statistical measures.

Many financial services in-house software development teams are striving to address technical debt and are turning to partners for help in this area, leaving them to focus on new software creation. Experienced software product engineering firms are ideally positioned as debt reduction partners. They are used to working on multiple engagements for multiple clients, have broad experience of many different platforms, and have invested in tools, repeatable processes and engineering talent to ensure a uniform and consistent approach. This allows technical debt to be minimised both in the current code and in subsequent enhancements to that code.

While technical debt may not seem, on the surface, to be a major problem, it accrues and compounds over time so that maintenance and updates to software become vastly more complex and costly than they need to be. Technical debt should be minimised throughout the software lifecycle as the nature of technical debt changes over the lifetime of a software platform. Putting strategies and partnerships in place to remediate this debt can help ensure the cost of the debt remains low in the future too.