An AI assistant that drafts a report creates a task for someone to review. An AI agent that changes a customer record, sends the report or approves a transaction creates a different management problem. The company must decide what the software is authorised to do, whose access it uses and how an action can be investigated afterwards.
That distinction matters as organisations connect AI to operational systems. The commercial appeal is understandable. Employees spend time moving information between applications, checking records and preparing routine responses. Software that completes part of that work could reduce delays. Yet the same connections can expose information or allow changes that exceed the purpose of the original task.
The practical question is how to give an agent enough access to be useful while limiting the consequences of error. This article examines that question for ordinary business workflows. Its central argument is that permission design, testing and operational ownership belong inside the investment decision. They affect whether a promising demonstration becomes a dependable service.
When assistance becomes action
The word agent covers a wide range of products. In this discussion, it means an AI system that can select and use tools to complete a task, potentially over several steps. Those tools might retrieve documents, query a database or update an application. The important distinction is the authority available through those connections.
OWASP identifies excessive agency as a security vulnerability associated with excessive functionality, permissions or autonomy. Its description helps separate three business choices: which tools a system receives, which resources those tools can reach and which actions it may complete without intervention. These choices can be reviewed independently.
A purchasing assistant might need approved catalogue data to compare products. It may not need the ability to change supplier bank details. A service assistant might need to propose a refund while lacking permission to issue one. These are illustrative design choices rather than claims about a particular vendor. They show why a general promise to automate a department is too broad for a useful access review.
Permissions belong outside the conversation
An instruction telling a model to avoid sensitive records can be helpful, but it should not be the sole barrier protecting those records. A more robust design restricts access in the connected application. If a tool has no permission to retrieve a file, an incorrect model response cannot create that permission by itself.
The principle is familiar from established cybersecurity practice. NIST describes zero trust architecture as a resource-focused approach that does not grant implicit trust merely because an account or device is inside a network. Applying that principle to AI workflows is an analytical extension: the organisation should evaluate the identity and authority behind each connection.
This becomes difficult when a demonstration uses a shared account with broad privileges. The prototype may work quickly because it can see everything. Moving it into production without changing that arrangement can obscure whose authority is being exercised. A narrower service identity, or a connection that respects an individual user's permissions, creates a clearer basis for accountability.
Reading information can also create risk
Access reviews often concentrate on whether software can change data. Reading can carry substantial consequences too. A system that combines documents from different departments could reveal information to someone who was never entitled to see the underlying files. The sensitivity of the final answer may depend on several sources rather than one obvious confidential document.
Consider a hypothetical assistant assembling a customer briefing. Contract notes, pricing records and internal complaint assessments may each have different audiences. Combining them into one convenient response does not remove those restrictions. A responsible design needs to preserve the relevant access boundaries through retrieval and into the resulting output.
Data minimisation also affects operating costs. Retrieving unnecessary material can increase processing, complicate review and make it harder to identify the evidence behind an answer. A smaller, clearly defined information set can improve traceability. That does not establish a universal relationship between smaller inputs and better model performance; it gives teams a more manageable workflow to test.
Untrusted content can influence behaviour
Connected systems encounter material that the organisation does not control. Customer emails, supplier documents and public websites may contain text that resembles instructions. A model can misinterpret that content as a direction to change its behaviour. The problem becomes more consequential when the same system can take action.
OWASP's prompt injection guidance explains this risk and notes that retrieval augmented generation and fine-tuning do not fully resolve it. Better access to company knowledge can improve relevance without making every retrieved document trustworthy. Businesses therefore need to assess both the information supplied to the model and the authority available afterwards.
For an invoice workflow, an unexpected instruction inside an attachment should not be able to change the destination of a payment. Practical controls could include separate validation of payment details, a restricted set of permitted operations and additional review when a request falls outside normal conditions. The exact controls depend on the system and transaction; no single technique removes every exposure.
Human review needs a specific purpose
Adding an approval button does not automatically provide effective oversight. Reviewers need enough information to understand what will happen. A vague request to confirm that an agent should continue offers less protection than a clear description of the record being changed, the proposed values and the supporting evidence.
Review also consumes attention. If every minor operation requires approval, staff may begin accepting requests mechanically or abandoning the workflow. If too few operations require review, consequential errors can pass unchecked. Organisations need to define review around the significance and reversibility of an action, while considering applicable obligations and their own risk tolerance.
The reviewer must have authority to stop the process and a practical route for escalating uncertainty. A person who is expected to approve hundreds of unfamiliar transactions under time pressure may not provide the scrutiny the design assumes. Staffing and training costs therefore belong in the project budget alongside software licences and technical implementation.
Testing must reflect the actual task
An agent can produce a convincing answer and still fail operationally. It may use an outdated record, choose the wrong customer or repeat an action after a connection timeout. Evaluating only the quality of its language misses the reliability of the surrounding process.
NIST's Generative Artificial Intelligence Profile provides a cross-sector resource for considering generative AI risks. Its relevance to business deployment is that risk assessment should accompany the use case. A company still has to translate general guidance into tests that reflect its own data, tools and consequences.
For a proposed service workflow, teams can test ordinary requests alongside ambiguous records, missing information and conflicting instructions. They should also examine what happens after a partial failure. A duplicate update may be more damaging than a failed answer. These suggested scenarios are practical analysis, not a claim that passing them certifies a system as safe.
A pilot should include the awkward cases
Small trials can overstate readiness when they rely on clean data and cooperative users. Production work includes inconsistent naming, stale documents and customers who describe the same problem differently. A pilot that excludes those cases may measure a demonstration rather than the work the business intends to automate.
Companies can limit the initial deployment to a defined population while preserving a representative range of difficulties. Performance should be assessed by completed tasks, corrections, escalation and the time staff spend checking results. Where practical, a comparison with the previous process helps distinguish a real gain from a change in reporting.
The pilot also needs a decision rule. Management should know what evidence would justify expansion, further adjustment or cancellation. Without that rule, a project can remain in a permanent trial, accumulating integrations and costs without demonstrating a stable commercial benefit. A narrower deployment may be the sensible outcome even when the underlying technology performs well.
Records make accountability possible
When an agent changes a system, the company needs a reliable record of the event. That record should identify the relevant user or service identity, the operation and its outcome. Depending on the workflow, it may also need the source information and the version of the configuration used.
Logging everything indiscriminately creates its own difficulties. Records can contain personal or commercially sensitive information, and long retention periods can increase exposure. The organisation must determine what evidence is necessary, who can inspect it and how it is protected. Traceability and confidentiality should be designed together.
An audit record should not depend entirely on the model's explanation of its own behaviour. The connected system can record the actual operation, while the agent's explanation supplies additional context. Keeping those roles distinct helps investigators compare what the software said it intended to do with the change that actually occurred.
Ownership continues after deployment
An agent's operating environment can change after approval. A connected application may introduce new fields, a supplier may change an interface or staff may widen the task. Even if the model remains unchanged, the resulting workflow can require a fresh assessment.
The NIST AI Risk Management Framework is intended for voluntary use and supports consideration of AI risks throughout use. It does not replace sector requirements or local law. For a business, the useful implication is that governance cannot end when a pilot receives approval.
Someone needs responsibility for access reviews, incident handling and changes to the workflow. Operational staff may own business rules while technical teams maintain integrations, but the boundaries should be clear. A vendor can provide documentation and controls; the customer still needs to decide whether the deployment fits its activities and obligations.
The economics of controlled automation
The financial case depends on the complete workflow. Time saved in drafting may be offset by verification, corrections or specialist support. Integration and monitoring costs can remain significant even when the model itself is inexpensive. Benefits should be evaluated against these recurring obligations.
Some tasks may justify automation because faster completion improves service rather than reducing headcount. Others may free staff for more complex work. Those outcomes require different measures. A project intended to shorten response times should not be judged solely on staff reductions, while a project justified by lower costs needs evidence that those costs actually fall.
Reversibility also has economic value. A system that can be paused without interrupting essential operations gives management more room to respond to problems. Maintaining an alternative process can cost money, but that cost should be compared with the consequences of dependence on an unreliable workflow.
Questions businesses are asking
Should every AI agent have write access
No. Some useful workflows require only retrieval and drafting. Write access should follow the requirements of a defined task and a review of the consequences. Giving software additional authority simply because an integration supports it increases exposure without necessarily improving the service.
Does a human approval step remove the risk
It can reduce risk when the reviewer has the evidence, time and authority to assess the action. It cannot compensate for every weakness in data quality, permissions or testing. Businesses should evaluate whether review works in practice under realistic workloads.
What indicates readiness to expand
Evidence should include useful completion rates, manageable correction costs and reliable handling of exceptions within the proposed scope. Expansion also depends on whether access boundaries and operational ownership remain appropriate. Success in one department does not establish readiness for a different task with different consequences.
AI agents offer businesses a way to automate parts of work that previously required repeated human coordination. Their commercial value depends on the authority surrounding that automation. Companies that define tasks precisely, enforce access boundaries and investigate failures will have a clearer basis for deciding where these systems earn a place in daily operations.
References
1. OWASP Gen AI Security Project. LLM06 2025 Excessive Agency. Source.
2. NIST. Zero Trust Architecture. Special Publication 800 207, 2020. Source.
3. OWASP Gen AI Security Project. LLM01 2025 Prompt Injection. Source.
4. NIST. Artificial Intelligence Risk Management Framework Generative Artificial Intelligence Profile. NIST AI 600 1, 2024. Source.
5. NIST. AI Risk Management Framework. Source.