What is ModelOps – and why should banks care

By Paul Jones, Head of Technology at SAS UK & Ireland

Does your bank manager know who you are? Unless your net worth is unusually high, the answer is probably no, and that’s been the case for many years. Banks have been using statistical models to inform credit-related decisions since at least the 1970s, and today almost every aspect of operational decision making is driven by sophisticated real-time analytics. So, while customers may hear about the outcome of their loan application from a bank employee, the decision itself has already been made in the background by computers without any human input.

Model-based automation has unlocked huge benefits for banks and customers alike because credit decisions can now be made in minutes or seconds, rather than hours or days. However, managing models at scale creates significant challenges of its own, and designing efficient model operations (ModelOps) is still largely an unsolved problem for most financial institutions.

Big banks, big problems

For example, the more established banks tend to have adopted model-based approaches piece by piece over the years. The use of models now extends far beyond the retail credit risk marketing function, and different parts of the business are using different methodologies, tools and techniques for managing the model life cycle. Both the data and the models themselves are isolated in departmental silos, so the bank often ends up making decisions in unconnected ways or based on only a fraction of the information it possesses on each customer. In simple terms: If the mortgage department’s model doesn’t have the same information as the loan team’s model and isn’t aware that a customer just took out a large loan, it may not make the best decision.

Similarly, the size and complexity of these banks tend to sap the agility of their model deployment processes. Going live with a new model involves surmounting countless organisational and regulatory hurdles. SAS research indicates that it can take three months to get a model deployed, while Gartner has found that over 50% of models never make it into production. The opportunity cost of failing to have the right models in place can be significant. I recently spoke with a CRO who estimated that during the credit crisis in 2007, delays to model deployment had cost the CRO’s bank around £500,000 per month.

New challengers, same old issues

The newer digital banks and fintechs tend to do better at agile model management. Since the burden of legacy systems doesn’t apply, they can potentially start from scratch and adopt a more joined-up approach. And because they have fewer customers and their failure poses less of a systemic risk to the economy, they attract less scrutiny from regulators, which means they can have lighter processes.

However, as these smaller banks grow, the weight of regulation they must shoulder will grow too – a burden they may lack the infrastructure and expertise to sustain. While they may currently be able to get away with a simpler approach to model management, that’s not going to work in the long term. To compete at the scale of the larger incumbents, they will need to tighten up their governance and industrialise their processes.

Model management as a key battleground

Whether the established banks maintain their dominance or the challengers prevail, ModelOps will play a key part in shaping the industry over the next few years. As artificial intelligence opens up new possibilities for even smarter cross-channel, real-time decisioning, the ability to design, train, deploy, monitor, update, audit and explain models will separate the wheat from the chaff.

Currently, almost all banks are struggling with model life cycle management, and especially with deployment. A recent McKinsey study found that less than 6% of companies had the ability to easily embed AI into formal decision making and execution processes, and less than 15% had the technological infrastructure to support deployment. This jibes with a recent article where Gartner Vice President and analyst Jim Hare stated: “Where organizations need help is how do [they]scale and operationalize and really handle an increasing number of models in production.”

At SAS, we believe that the inability to integrate analytic solutions into workflows and achieve front-line adoption is the No. 1 reason why data and analytics initiatives fail. That’s why we’ve concentrated efforts on industrialising the deployment of AI.

Why is model management so hard?

To start moving in the right direction, banks first need to understand the problem. Why are model management and deployment so hard? One of the biggest reasons is more human than technical: It’s is a place where two different traditions meet.

On one side, data science, which comes from an academic background and aims to turn groundbreaking research into game-changing business value. On the other, IT operations, which focuses on delivering reliable services within technical, regulatory and business constraints. These two traditions work in different ways, move at different speeds, and target different goals – so it’s not surprising that there’s often a clash of cultures.

How ModelOps can help

The promise of ModelOps is that it provides a robust workflow that acts like a set of intermediate gears between the data science and IT operations teams, enabling the smooth transmission of models from development into production while allowing both teams to work productively and at the right pace. By automating handoffs between teams throughout the model life cycle and providing end-to-end traceability and governance, a ModelOps approach can turn a misfiring modelling pipeline into a well-oiled machine.

At SAS, we’ve had firsthand experience of the challenges of moving to a ModelOps approach. We’ve always been both a data science company and an IT operations company, so we’ve had a foot in both camps for over 40 years. But it’s only relatively recently, with the maturity of cloud technologies and the widespread adoption of DevOps practices such a continuous integration and deployment (CI/CD), that we’ve really cracked the problem.

For example, we’ve learned how to use the cloud to break the model life cycle out of departmental silos and provide a commercial model that suits the experimental, fail-fast approach that data scientists need. Meanwhile, modern DevOps tooling provides common ground for data scientists and IT operations teams to collaborate effectively and ensure proper governance while managing models at scale.

Real-world results

We’re now applying the insight we’ve gained to help clients throughout the financial services sector adopt a ModelOps approach. For example, Covéa Insurance chose SAS to enable the deployment of complex machine learning models in a high volume real-time scenario, while Standard Chartered Bank gained more efficient model deployment in support of IFRS 9 and won the Asian Banker’s Enterprise Technology Implementation of the Year award. Find out more about ModelOps or take a deeper dive.