Published by Jessica Weisman-Pitts
Posted on February 21, 2024
6 min readLast updated: January 31, 2026
Add as preferred source on Google
ByNeil Serebryany, Founder and CEO, CalypsoAI
Deploying generative artificial intelligence and large language models across the enterprise presents opportunities for both increased productivity and innovation, but also creates challenges for managing organizational risk.
Where We’ve Been
Approximately 10 years ago, artificial intelligence (AI)-dependent tools became an established feature in the business landscape. Financial organizations were among the earliest and most enthusiastic adopters, leading the development and deployment of innovative, productivity- and revenue-driven AI solutions for issues that have long plagued the field, such as anticipating market trends and fluctuations, ensuring compliance in a dynamic regulatory environment, improving client support services, and deterring fraud. More recent challenges include multi-channel customer options and collecting and analyzing data on consumer behaviors and preferences.
Where We Are
The latest additions to the AI toolbox are generative AI (GenAI) and large language models (LLMs), which are being deployed throughout the enterprise for tasks as diverse as assessing risk exposure, developing investment strategies, improving competitive advantage, and generating and executing narrowly targeted marketing campaigns. Across the banking, insurance, and financial services industries, these models are increasing productivity, driving innovation, ensuring compliance, and reducing fraudulent activity. However, operational and security challenges balance these large productivity gains.
Organizations adopting these models must acknowledge and prepare for the new risk layer that accompanies–and can disrupt–successful model adoption and deployment. The list below identifies top considerations that accompany LLM deployments and ideas for addressing them.
Cost Controls
The cost of model deployment can vary dramatically and includes variables that must be carefully considered before being decided upon. At its most basic, the spend is a cost vs performance issue. LLM providers’ fees are typically a cost-per-thousand-token calculus, with a token being the equivalent of around three-quarters of a word. The amount of information the LLM can process in a query (the model’s “context window”) and the model’s performance characteristics also affect pricing. Models with longer context windows can provide responses of greater depth and nuance, but increase the compute spend.
Observability and Visibility
The typical enterprise deploys multiple models, including multimodal GenAI models that use voice and images, that operate in parallel, leaving security teams with a fragmented view of the overall system when what they need is one tool that provides full visibility into and across all models in use.
The solution is to deploy an automated tool spanning all models in use, providing observability at a per-model, system-wide, level. When able to audit and secure models, administrators can leverage insights about usage and user behavior, enhance and streamline decision-making processes, overcome inherent limitations, and provide stability, reliability, efficiency, and added security across the organization.
Data Security
Both fine-tuned LLMs and retrieval-augmented generation (RAG) models access large amounts of proprietary data. They must be protected from accidental, as well as deliberate, data leakage. The most common data leak is an unintended exposure via a user query to the model, such as an internal memo containing detailed information regarding a merger or acquisition under consideration sent to the LLM to improve the structure and verbiage more professional. The security issues here are threefold:
The solution is also threefold:
AI Security
“AI security” is used more and more frequently, but often without explanation. The term refers to the strategic implementation of robust measures and policies to protect an organization’s AI systems, data, and operations from unauthorized access, tampering, malicious attacks, and other digital threats. It goes well beyond traditional cybersecurity because every AI-driven or AI-dependent component linked to an organization’s digital infrastructure adds to the sprawl of pathways into the system.
While many technical solutions exist to address technological vulnerabilities, an organization’s most commonly exploited vulnerability is the user who, as mentioned above, inadvertently includes sensitive information in a prompt or acts on information received in a response unaware that it’s malware, a hallucination, a phishing campaign, or a social engineering attempt. An insider who takes deliberate actions, for instance, by trying to outwit security features via prompt injection or “jailbreak,” not realizing they could put the organization at risk, is another unfortunately common threat vector.
The solution is to apply strong filters on outgoing and incoming channels to identify content that is suspicious, malicious, or otherwise misaligned with organizational policies and industry standards.
Where We’re Going
The challenges of deploying AI systems, specifically Gen AI and LLM systems, are growing in number and sophistication on par with the number and sophistication of the models themselves. The risks presented by poor or incomplete adoption and deployment plans are also expanding in scope, scale, and nuance. This is why identifying, evaluating, and managing every potential risk is vital for maintaining the models’ integrity, security, and reliability, as well as the organization’s reputation and competitive advantage.
While having a strong AI security strategy and deployment plan, which includes employee education and training about the role they play in mitigating risk, are very important, incorporating the best tools for the situation is also critical to ensuring a safe, secure adoption and rollout. A “weightless” trust layer built into the security infrastructure, which enables full observability into and across all models and detailed insights about their use, is the ideal. When system and security administrators can see who is doing what, how often, and with which models, they are afforded both wide and deep user and system insights that support a strong, stable, transparent deployment posture.
About Author:
Neil Serebryany is the founder and Chief Executive Officer of CalypsoAI. Neil has led industry-defining innovations throughout his career. Before founding CalypsoAI, Neil was one of the world’s youngest venture capital investors at Jump Investors. Neil has started and successfully managed several previous ventures and conducted reinforcement learning research at the University of South California. Neil has been awarded multiple patents in adversarial machine learning.
Generative AI refers to algorithms that can generate new content, such as text, images, or music, based on training data. It is increasingly used in various industries, including finance, to enhance productivity and innovation.
Risk management is the process of identifying, assessing, and controlling threats to an organization's capital and earnings. In finance, it involves strategies to mitigate potential losses from various risks.
Large language models are AI systems trained on vast amounts of text data to understand and generate human-like text. They are used for tasks such as customer support, content creation, and data analysis.
Data security involves protecting digital information from unauthorized access, corruption, or theft. In financial services, it is crucial to safeguard sensitive customer and transaction data.
Compliance refers to the adherence to laws, regulations, and guidelines governing financial practices. It ensures that organizations operate within legal frameworks to avoid penalties and maintain trust.
Explore more articles in the Technology category
