By Craig Beddis, CEO and Co-Founder, Hadean

Financial models struggle to deal with the chaotic nature of our world: How do you account for a ship blocking the Suez Canal? What is the Brexit variable or the impact of new emerging markets? How do you prepare for Redditors taking on Wall Street? An influx of new data, new agents and new emerging factors has necessitated that financial simulations reflect the complexities of an inherently unpredictable environment. 

The success of many fintech startups is built upon empowering outsiders of traditional financial institutions. Financial inclusion is growing across the globe; from modern banks reducing transaction fees and encouraging the flow of cash across borders, through to disruptive startups using big data to redesign lending models and risk assessment. We are even beginning to see AI used as a means to provide both financial insight and services. 

The cumulative effect is a significant increase in the number of agents – individuals, companies, systems – in the financial world. At the same time, greater connectivity is providing more data and a greater flow of communication. The Gamestop fiasco showed that even casual social media users could act as powerful agents when given a medium to collude upon. With more agents interacting with each other, understanding emergent phenomena is increasingly important.

Emergence is a ‘sum of its parts’ property that the individual agents do not possess. Oftentimes, it happens intentionally, such as in ant colonies, where the individual actions of the ants show a collective intelligence and shared goal. But unintentional, random and even counterintuitive emergence can also occur. Adding more lanes to a road often increases the traffic stuck on it due to drivers individually choosing to swap lanes more. It can help explain how prices are set in a free market, how clerical mistakes result in huge losses and even how the actions of individual Reddit users can bring down hedge funds.

Handling Unpredictability 

Financial modelling techniques such as regression analysis generally take a ‘top down’ approach, by applying macroeconomic factors and equations to situations. Agent-based modelling, in contrast, works by starting from the bottom and models the actions of individual entities. In this sense, it is far better suited to uncovering emergent phenomena, otherwise hidden in the myriad individual actions. 

With the plethora of agents in today’s world, simulating their actions is key to providing greater insight into the direction of financial markets. But providing the infrastructure for agent-based modelling comes with its own set of challenges. Agent-based models analyse large data sets from multiple disparate sources – data from social media, stock prices and supply chain information needs to be collated and synthesised into single, large-scale simulations. The unpredictable nature of its volume and velocity results in a highly volatile demand for computational resources. 

For financial analysts, the introduction of load balancing mechanisms that effectively handle surges in computation will be pivotal in encouraging widespread adoption of agent-based simulation. Current architecture is ill equipped to deal with unpredictable surges in demand

common to agent-based models, restricting scale and detail of simulations. At the same time, large data sets cannot be effectively aggregated, limiting scope and accuracy. 

Moreover, such setups present a high barrier to entry both in terms of cost and accessibility. As it stands, creating and deploying models is a complex engineering feat that requires constant maintenance and management. As workloads continue to migrate to the cloud, there is a significant risk of expensive overprovisioning, or the alternative whereby the underlying infrastructure is overwhelmed and crashes. By contrast if infrastructure concerns are automatically taken care of, much of the engineering burden is alleviated. Smaller, agile teams can create repeatable simulations where variables are adjusted in order to find the optimal strategy. The real-world information proving more meaningful than abstract estimations. 

Clearly finding a way to solve the infrastructure problem will open up new opportunities within the financial sector. The social benefits of financial inclusion are clear, with technology driving its inevitability. Its presence is rapidly expanding the number of agents and prevalence of emergent behaviour in economies. This is changing the foundations on which traditional financial modelling is based, which previously only accounted for markets with a smaller number of influential actors. Adopting agent-based models provides a window into this new world, allowing you to make sense of the chaos.