The next stage goes beyond producing answers
Financial firms have spent several years experimenting with artificial intelligence tools that summarise documents, draft correspondence or help employees find information. A new question is emerging as systems become capable of carrying out sequences of tasks. An AI agent may gather data, select a workflow, call software tools, prepare a recommendation or initiate an action within boundaries set by its operator. The difference is not simply more fluent text. It is the possibility of delegated execution.
For a bank, insurer or asset manager, that capability could improve service and reduce repetitive work. It could also create a new operational risk if a system acts on an incorrect assumption, follows a misleading instruction or exceeds its intended authority. The sector's challenge is to decide which tasks can be delegated, how the delegation is controlled and what must remain subject to human approval.
What the evidence does and does not establish
The Financial Stability Board has assessed how AI adoption may affect financial services through operational efficiency, advanced analytics and more personalised products. It also identifies model risk, cyber threats, third-party concentration and correlated behaviour as potential sources of vulnerability [1][2]. These findings concern AI broadly; they do not establish that autonomous agents have already caused a particular financial crisis or will necessarily do so.
The practical implication is to avoid judging a technology only by demonstrations. A successful pilot may prove that a model can answer a question or complete a controlled workflow. It does not prove the system will behave reliably under unexpected data, conflicting instructions or a disruption at a connected service provider. Financial organisations need evidence from testing, supervision and real operational performance before expanding delegated authority.
Autonomy is a spectrum, not a switch
It is helpful to separate AI use into levels of operational authority. At one end, a system prepares information but makes no changes. Next, it proposes an action that a person must approve. Further along, it may execute a narrow permitted task, such as categorising a document or updating an internal record, while recording its steps. The most sensitive tier would allow autonomous action involving money movement, customer eligibility or material risk decisions.
Treating all these activities as equivalent would produce either excessive restriction or inadequate control. The appropriate safeguards depend on the consequences of an error. A draft meeting summary and a payment instruction should not share the same approval threshold. Firms should document what an agent may read, what it may change, which tools it may use and when the system must stop and hand control to a responsible employee.
Authority and accountability must remain visible
A financial institution cannot transfer legal or regulatory responsibility to a software model. Even where a system completes work autonomously, management must know who approved its design, who monitors it and who can intervene when something goes wrong. Access permissions should follow the principle that the tool can do only what its task requires. Strong identity management and clear separation of duties become more important as agents interact with internal systems.
Institutions should also distinguish between the human who requests an outcome and the employee accountable for the process. A user may ask an AI system to resolve an exception, but authority to release a payment or alter a customer record may sit elsewhere. Preserving these controls requires systems design, not just a reminder in a policy document. Audit trails should record relevant instructions, tool calls, approvals and outcomes without unnecessarily collecting sensitive information.
Third parties become part of the control environment
AI workflows can involve a model provider, cloud platform, data vendor, orchestration software and internal applications. An outage, policy change or compromised component in one layer may affect services elsewhere. The FSB has identified third-party dependency and concentration as potential financial stability vulnerabilities associated with AI [1]. The Basel Committee's operational resilience principles likewise call attention to critical operations and third-party dependencies [3].
A bank assessing an AI agent should therefore map the entire service chain. What data leaves the institution? Which supplier stores prompts or outputs? Can the firm continue serving customers if a model becomes unavailable? Is there an alternative procedure that employees know how to use? Outsourcing a technical capability does not remove the obligation to understand its effect on business continuity, confidentiality and service delivery.
Testing should include hostile and ordinary errors
Model testing is often framed around factual accuracy. For agents, firms must examine behaviour as well as answers. An employee could give a vague instruction; a customer-provided document could contain manipulative text; an external data source could be outdated; or one of the tools available to the agent could return an unexpected response. Each scenario could push a workflow outside its intended scope.
Meaningful tests include ambiguous requests, permission boundaries, false data, system timeouts and attempts to persuade the agent to ignore the operator's rules. Teams should measure not only whether a task was completed, but whether the system asked for clarification, stopped when necessary and preserved a reliable record. The objective is a system that behaves safely when it cannot complete the assignment, rather than one that always produces an answer or takes action.
The danger of correlated decisions
AI systems may be deployed by many financial firms using similar models, data sources and optimisation objectives. That creates the possibility that institutions respond to the same signals in similar ways. The FSB has highlighted market correlations as a potential source of AI-related systemic vulnerability [1]. It is a risk to monitor, not evidence that every AI system will produce identical trading or credit decisions.
Organisations can reduce dependence on a single technique by testing diverse scenarios, applying limits and retaining independent review of material decisions. Investment firms should understand whether automated recommendations could reinforce concentrated trades or respond poorly to unusual liquidity conditions. Lenders should assess how data quality and model changes affect decisions across customer groups. Governance needs to include the wider market context as well as individual model performance.
Measure value after the cost of control
An agent that completes a task in seconds may appear to deliver major productivity gains. The full economics are more complicated. Firms must include integration, supervision, testing, exception handling, cybersecurity, vendor charges and the cost of correcting errors. If an automated process generates frequent reviews or customer complaints, its apparent speed advantage may not translate into sustainable value.
Measurement should compare the entire workflow before and after deployment. Useful indicators include successful resolutions, employee time released for higher-value work, escalation accuracy, customer outcomes and incidents avoided or detected. Firms should also check whether human staff can explain decisions and override the system without excessive delay. The objective is not maximum autonomy. It is the best combination of service quality, control and cost for a given business process.
The human capability automation still needs
Delegating more work to an AI agent changes the job of the people supervising it. Employees who once entered information may instead need to verify outputs, investigate exceptions and decide when the system is not suitable for a task. That requires training in the underlying business process as well as the technology. A person cannot meaningfully approve a recommendation they do not understand, particularly if the system presents its conclusion with unwarranted confidence.
Organisations should create clear routes for employees to challenge an agent's behaviour without being penalised for slowing a process. Internal teams need simple mechanisms to pause access, correct records and report suspected model or data failures. Those controls must survive staff turnover and vendor updates. A workflow that works only because one engineer remembers an undocumented limitation is not a mature financial service.
There is a longer-term governance question as well. If multiple business lines use the same external model, failures may arrive simultaneously and appear in different forms. An institution should retain central visibility over provider dependencies, permissions and high-impact incidents while permitting local teams to own their business outcomes. This combination of shared infrastructure controls and specific accountability can help avoid both fragmented risk management and one-size-fits-all approval processes. In a regulated industry, investing in human competence remains a necessary part of investing in machine capability.
A pragmatic adoption agenda
Financial institutions can begin with well-defined tasks where the data is reliable, consequences of failure are limited and results can be checked. They can set explicit permissions, require approval for sensitive actions and develop procedures for withdrawing or suspending an agent. Compliance, technology, risk and business teams should agree on what successful performance looks like before a system is placed into production.
The emerging trend is not simply that AI will become more capable. It is that organisations will increasingly have to decide which decisions and actions they are willing to delegate. The firms that benefit most may be those that make responsibilities, evidence and escalation paths clearer as they automate. In finance, reliable autonomy depends on enforceable limits and accountable people, not on the confidence with which a system describes its own actions.
References
1. FSB, The Financial Stability Implications of Artificial Intelligence. Source link
2. FSB, AI Financial Stability Assessment. Source link
3. BIS Basel Committee, Principles for Operational Resilience. Source link
4. BIS, Core Principles for Effective Banking Supervision. Source link