As businesses move from AI assistants that suggest answers to agents that can carry out tasks, operational discipline is becoming as important as the technology itself.
From assistance to action
An artificial intelligence assistant that drafts an email is fundamentally different from a system authorised to send it, update a customer record, place an order or transfer information between applications. In the first case, a human can examine a suggestion before deciding what to do. In the second, software may perform several connected steps before a person sees the result. This shift towards agentic AI is attracting attention because it promises to remove routine handoffs from business processes. It also creates a sharper question for leadership: who is accountable when an automated decision triggers a costly action?
The opportunities are tangible. A procurement team might use an agent to compare purchase requests against policy, locate approved suppliers and prepare an order for review. A bank could use a similar approach to gather supporting documentation for an account query. A manufacturer might automate the first response to an equipment alert by looking up a maintenance history and drafting a service ticket. None of those examples requires software to act without limits. The value comes from reducing repetitive work while retaining defined responsibility for exceptional decisions.
According to the NIST AI Risk Management Framework, organisations need to manage AI risks across governance, mapping, measurement and management. Its guidance is voluntary, not a certification of an individual product. The practical implication is that a useful AI deployment needs more than a vendor demonstration: it requires an explicit description of the task, intended users, foreseeable failures and mechanisms for detecting problems. The same principle applies whether a company builds an internal agent or buys access through a software provider.
Further reading: NIST AI Risk Management Framework
A new kind of operational exposure
Automated systems can make ordinary mistakes quickly and consistently. An agent might misunderstand an instruction, select the wrong customer account, rely on outdated information or follow a misleading document. If it has permission to take action, a small interpretation error can become an accounting, contractual or privacy problem. The greater risk is not necessarily an extraordinary AI failure. It is a plausible mistake made at scale, with insufficient opportunity for someone to interrupt the process.
Agentic systems also create security questions. Instructions may arrive from emails, websites, attachments and shared records rather than a trusted operator. A hostile instruction embedded in one of those materials can attempt to redirect the agent away from its original task. OWASP identifies prompt injection and related vulnerabilities as risks for applications built around large language models. This is a reason to treat external content as untrusted input, rather than as an instruction source, and to limit the actions any system can perform by itself.
Businesses should distinguish between access to information and authority to change it. A customer-service agent may need to read an order status, but it does not necessarily need permission to issue refunds. A finance workflow might be allowed to propose a payment but not approve the beneficiary details and release funds in the same step. Separating these privileges can prevent one faulty action from becoming a wider incident. Familiar cybersecurity ideas, including least privilege, authentication and logging, remain relevant even when the user interface looks conversational.
Where human approval still matters
The strongest design may be neither complete automation nor manual review of every trivial action. It may be a tiered model in which low-risk work proceeds automatically while consequential decisions require confirmation. An agent could classify incoming documents and identify missing fields, yet seek approval before changing customer entitlements or sending funds. Thresholds could reflect monetary value, legal significance, data sensitivity or uncertainty in the available evidence. Clear escalation rules make the system predictable for employees and auditable for management.
Human approval is only meaningful when the reviewer understands what the software proposes to do. A screen that says “approve task” offers little protection if it hides the beneficiary, amount, source documents or assumptions. Reviewers need a concise action summary, relevant evidence and the ability to correct or reject a proposal. For more complicated processes, an independent second check may be justified. The goal is not to make employees click through additional boxes; it is to reserve human judgement for points where it can genuinely alter the outcome.
Companies also need to know when an agent has encountered uncertainty. A system that produces fluent language can sound confident while relying on incomplete facts. Useful safeguards include validation against approved records, explicit confidence or exception indicators where appropriate, and a requirement to stop when information is missing. The NIST Generative AI Profile discusses the distinctive risks of generative systems and offers a framework for identifying and managing them. That guidance supports evaluation rather than a blanket assumption that any model is reliable in every setting.
Further reading: NIST Generative AI Profile (AI 600-1)
Measure outcomes, not the number of bots
The number of agents deployed is a poor measure of digital transformation. A more informative scorecard asks whether work is completed accurately, how often humans must intervene, whether customers receive better service and what incidents or near misses occur. For a document-processing workflow, the meaningful measure may be the percentage of cases completed correctly without rework. In customer service, resolution quality and the rate of inappropriate actions matter alongside response time. An apparently efficient system that increases corrections may have simply moved costs elsewhere.
Before rollout, management should establish a baseline against which performance can be compared. Trials should include normal cases, ambiguous cases, unexpected data and attempts to manipulate instructions. Teams should record what the agent did, which tools it called and when a human intervened. Those logs should be protected in line with the organisation’s data rules, because the record itself may contain commercially sensitive or personal information. Monitoring needs to continue after launch: an updated application interface or changed supplier procedure can alter how an agent behaves.
Accountability must also reach vendors. Contracts should clarify data processing, service levels, incident notification and what evidence the customer can obtain after a failure. A vendor may operate the model while a business defines the tasks and grants access to internal systems. Responsibility can therefore be shared in practice but cannot be assumed away. Procurement, security, legal and operational teams should agree which controls they own instead of treating AI deployment as only an IT project.
How organisations can scale responsibly
A sensible deployment begins with a bounded process whose inputs are understood and whose mistakes are recoverable. Document classification, internal knowledge searches and draft preparation may offer early benefits. More sensitive activities, such as payment instructions, eligibility decisions or access-control changes, warrant stronger testing and approval. Organisations should inventory agents much as they inventory important software: who owns each one, where it operates, what information it sees and what actions it can initiate.
Governance also needs an exit plan. If an agent behaves unexpectedly, staff must be able to pause it, revoke its credentials and continue the underlying process manually. Resilience depends on understanding how work gets done without the automation. This matters when a model provider changes its service, an integration fails or an incident requires investigation. A dependable agent is not one that never fails; it is one whose failure is contained, visible and recoverable.
The emerging business distinction is not between companies that use AI and companies that do not. It is between organisations that can explain the limits of their automated systems and those that cannot. Speed will attract attention, but trust will determine which deployments endure. As AI agents gain a practical role in finance, retail and industrial operations, the lasting competitive advantage may come from knowing precisely when a machine should act—and when it should stop.
For boards, one useful discipline is to require an owner to sign off on every agent operating in a consequential workflow. The owner should be able to state its purpose in plain language, identify the systems it can access and explain how an error would be discovered. A cross-functional review can then test whether permissions are proportionate to the task. If a system generates purchase orders, for example, the team might cap amounts, prohibit changes to bank details and require separate approval for unfamiliar suppliers. It should also define who receives alerts and how quickly the agent can be suspended. These are operational controls, not promises that every automated judgment will be correct.
Employees need training that goes beyond writing effective prompts. They should understand when generated information requires confirmation, how to challenge an automated recommendation and how to report a suspicious result without fearing blame. Organisations can learn from near misses, including cases where a human stopped a faulty action before damage occurred. A culture that measures only time saved may discourage people from raising such concerns. A culture that rewards reliable outcomes makes employees active partners in responsible adoption.
There is also a strategic question about dependence. As more workflows rely on one model, cloud platform or integration provider, a temporary outage can affect several departments simultaneously. Contingency planning should include manual alternatives, data portability where feasible and clear priorities for service restoration. Companies do not need to reject outside technology to manage concentration. They need to understand which provider failures would stop essential activities and build proportionate alternatives before a disruption occurs.
Further reading: NIST AI RMF Playbook
References
1. NIST AI Risk Management Framework. https://www.nist.gov/itl/ai-risk-management-framework
2. NIST Generative AI Profile (AI 600-1). https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence
3. NIST AI RMF Playbook. https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook
4. NIST Cybersecurity Framework 2.0. https://www.nist.gov/cyberframework
5. OWASP Top 10 for LLM Applications. https://genai.owasp.org/llm-top-10/