Unified Data Layer + Hybrid Search Engine Conquering a common issue that causes financial service AI deployments to fall short

By Steve Mayzak

With every paradigm-shifting technology comes a reality check, and artificial intelligence is no exception.

In a report released in August 2025, MIT found that despite collectively investing $30 to 40 billion in generative AI, 95% of organizations are getting zero return on their investments. All told, 60% of organizations have evaluated enterprise-grade genAI copilots, according to the report, yet just 20% of those deployments reached pilot stage and just 5% reached production. “Despite high-profile investment and widespread pilot activity [involving genAI], only a small fraction of organizations have moved beyond experimentation to achieve meaningful business transformation,” the authors of the report assert.

That lack of ROI from AI is an issue for most industries, financial services included; tech and media are the only two segments where genAI is showing clear signs of structural disruption, according to MIT. What’s more, as McKinsey noted in a separate 2025 report, significantly increased technology spending specifically by asset management organizations (almost 9% annually over the past five years) has yet to deliver ROI. McKinsey blames this “tech ROI challenge” on fragmented systems, siloed data environments and the lack of a comprehensive, fit-for-purpose, front-to-back platform for integrating diverse data sources.

Why So Many GenAI Deployments Fail to Deliver ROI

These findings offer two important messages to financial services firms. First, as much value-creation potential as AI shows, there are limits to its transformative powers — limits that have much to do with the data feeding AI models. The quality of output from AI, and the quality of the overall outcomes that AI implementations produce, depend largely on the quality of the data inputs supporting those implementations.And second, financial services companies that are able to harness the transformative power of AI at scale position themselves to seize a measurable competitive advantage over their peers. AI shows great promise in a wide range of financial services use cases across the back, middle and front offices.

For a financial services firm to realize strong ROI and a competitive edge from their AI implementations, then, it must have a strong data foundation that includes a unified layer of structured and unstructured data, plus a genAI search tool that can consistently cull reliable, fresh and contextually appropriate insight from both types of data (a hybrid search tool). In many genAI use cases that fail to deliver ROI, the culprit is two-fold: a search engine that struggles to handle both structured and unstructured data, as well as a fragmented or siloed IT landscape in which data is scattered across platforms, databases and systems — trading platforms, CRM systems, finance systems, risk management and compliance systems, even regulatory filings, call transcripts and email. It’s the old garbage-in, garbage-out dynamic at work.

But when all that data is accessible to a search engine that knows how to normalize and process it, that gives the engine the raw material it needs to produce consistently valuable, actionable outputs. Rather than relying on data that has been moved from its original location into some type of data lake, which can strip data of important context, the search engine instead takes data that resides in its original source systems, then indexes it with its context preserved, yielding fast, accurate and relevant results. Ifyour company’s data is a book, think of the search engine as that book’sindex.

The result: a comprehensive view across an organization's entire IT ecosystem, minus the siloed data blind spots. That in turn enables the genAI search engine to produce faster, richer and more reliable insights, and to scale with growing data volumes, new data sources and use cases.

Unleashing GenAI

This combination of a unified, ready-to-indexdata layer and hybrid AI search can be a powerful one for a financial services firm, opening up all kinds of possibilities for value-creating, ROI-generating genAI use cases, including:

Automating manual tasks. Look for low-hanging fruit: tasks that are normally performed by human beings to get internal and customer questions answered or to deliver a service. Many of these tasks could be candidates to automate with GenAI. In the front office, that could mean using a hybrid AI search engine as an AI agent to conduct daily market research and analysis on behalf of your wealth managers. The AI search tool and agent essentially become a digital extension of those wealth managers, performing time-consuming tasks to enhance their productivity and free them for higher-value work.

Ultimately, this can save wealth managers hours of work each week. Whether your firm has dozens, hundreds or even thousands of wealth managers, these hours can quickly translate into significant efficiency and productivity gains. If you start to measure the ROI of AI in terms of revenue per employee, as I believe financial services firm could and perhaps should, the payback on an enterprise-level genAI use case like this is compelling.

Fraud detection and diagnosis. Fraud detection during onboarding of new clients is another area where the combination of genAI and agentic AI can be a true difference-maker for a financial services firm. A client onboarding team could task AI to scan and validate the authenticity of onboarding documents, then report back with any red flags, including seemingly harmless anomalies (mismatched fonts, odd document formatting, minor data inconsistencies, etc.) that suggest a potential customer may not be who they say they are and thus warranting additional screening. Not only can the AI agent provide an alert about such a customer, it can diagnose the issue and recommend appropriate remediation steps. Eventually — though we’re not there yet given valid concerns related to compliance and the trustworthiness of AI —a firm might even entrust the AI agent to execute the steps it recommends, with humans in the loop to sign off on those actions. More on that in a moment.

Identifying opportunities with new fee structures, business models, and markets. A simple plain-language query and dialogue with a genAI assistant could provide valuable strategic insight about how a potential fee change or a re-bucketing of customers could impact revenue. Drawing from modeling and analytics applied to historical data, that digital assistant could also provide insight into how a potential move into a new geographic market might play out in terms of cost (expected vs. actual), regulatory ramifications and HR impacts.

As much as search-related use cases like these can move the needle for a financial services firm, they only scratch the surface of genAI’s potential to deliver ROI back to the business.

The Foundation for Successful GenAI Implementations

To tap that immense potential, a few critical foundational pillars have to be in place, starting with a data layer capable of handling both structured and unstructured data, along with a hybrid search engine that can combine keywords and semantic search to deliver outputs with consistently high relevancy.

Ensuring relevancy is a big issue with many AI search engines. So you also want to be sure the search engine you’re using has built-in relevancy tuning capabilities so it provides the most probabilistically true answers. Observability is also critical with whatever genAI search capabilities you’re using, so you can actively monitor the output of those capabilities, and, for compliance purposes, report transparently on how they’re being used.

With any and all AI use cases, it’s vital to have an AI governance program with systems and processes that put human beings squarely in the loop as the ultimate decider in evaluating the behavior of AI and the quality of its output. Ultimately, AI should be judged on the value it provides back to your organization. And that value depends on consistently high-quality output.

Steve Mayzak has been a developer, architect, dev manager and global presales VP in his 25+ years in the technology industry.He went from building eCommerce websites for Nike and Roland in the late 1990s through the early 2000s to building world-class technical presales teams at companies like Elastic, Pivotal and Grafana.In the last few years, Steve has been focused on AI and ML and currently leads a team of genAI Specialists at Elastic. Steve’s curiosity has driven him to stay on the cutting edge of technology, be it in software development build/deploy techniques with Pivotal, bringing search to observability and security at Elastic, and bringing two disparate worlds together through ML/AI.

Connect with Steve on LinkedIn here.