Getting AI right: why financial services organizations must focus on the foundation first

Financial services organizations across banking, capital markets, insurance, and wealth and asset management are entering a new phase of AI adoption, one that requires structure, scale, and deep integration across the enterprise. Initial artificial intelligence (AI) efforts often centered on quick wins and isolated experiments. But the landscape is shifting. Organizations are now realizing they need to embrace structured programs that align with long-term goals, recognizing that AI is no longer a stand-alone initiative confined to side projects or innovation labs but rather a core part of enterprise strategy.  

AI is being woven into broader transformation programs, becoming part of the infrastructure rather than a separate add-on. Across financial services, AI is being used to personalize customer experiences, enhance operational efficiency, and unlock insights from vast data sets. Domain-specific models trained on proprietary data are helping firms anticipate client needs, tailor services, and build trust through relevance and reliability. This change reflects a deeper understanding of AI’s role in the enterprise.  

It is not just a chatbot or a clever application. It is a capability that, when embedded into the right places, can drive significant value. But none of it will work without IT modernization, a foundation of high-quality data and strong teams executing on the organization’s AI strategy.  

As AI moves from isolated experiments to enterprise-wide transformation, firms must rethink how they approach data readiness and integration. Here are three major challenges companies must overcome to get their data, and their institutions, ready for scalable AI. 

1. ITmodernization:the foundational hurdle 

Modernizing core IT systems remains one of the biggest challenges facing financial services organizations. Many are finding that most of their AI investment does not go into the AI models or agents themselves. Instead, it goes into preparing their infrastructure to support those models and agentic frameworks. This foundational effort is often underestimated, and so are the time, cost and operational changes required to generate meaningful returns. 

Moreover, many legacy platforms were not built for the cloud, real-time analytics or machine learning. This outdated architecture makes it difficult to access and use these systems in ways that AI demands. Institutions that began modernizing their infrastructure early are now in a stronger position. They can more easily integrate AI into their systems and processes, giving them a clear head start. 

The explosion of AI and data processing demands robust infrastructure. Financial services organizations must evaluate the balance between cloud and on-premises systems, ensuring they have the scalability, flexibility and security to support AI at scale. Migrating from legacy tools to cloud-native platforms, overhauling data pipelines and implementing governance frameworks are critical steps. This process requires substantial investment and time, but it is critical for building an infrastructure that can support AI at scale. 

1. Strengtheningdata foundations for scalable AI 

Even with modern systems in place, firms must make significant investments in how they manage, test and use their data. However, testing AI is fundamentally different from traditional software testing. Because AI models are dynamic and probabilistic, their behavior can change based on new inputs or data shifts. As a result, financial services organizations need new types of testing frameworks and tools that safeguard accuracy, fairness, compliance and performance. This has become a major area of focus for banks seeking to meet regulatory requirements and ensure reliability. 

Another important investment area is the underlying data itself. The truth is that AI cannot perform well if the underlying data is not ready. Increasingly, institutions are discovering that their existing data foundations, particularly metadata, taxonomies, document management systems and data catalogs, are insufficient for AI agents to fully interpret and act on structured data. Historically, firms prioritized structured data for compliance, liquidity tracking and financial reporting. But AI solutions often require a broader range of inputs, including unstructured data from PDFs, scanned documents, emails or internal notes.  

Take the example of vendor contracts or commercial loan agreements. These documents can contain valuable insights, but they are often stored in unstructured formats across fragmented systems. Before AI can surface that intelligence, companies must identify, organize and secure this data. This requires more than just integrating APIs. It often means centralizing data on platforms where it can be properly curated and governed.  

Additionally, the shift from large language models (LLMs) to domain language models (DLMs) is gaining traction. DLMs leverage proprietary institutional knowledge to build models tailored to specific domains, offering differentiated capabilities and deeper relevance. This evolution reinforces the importance of having data that is not only clean and accessible but also structured in ways that AI can consume and learn from effectively. 

Most importantly, security must also be embedded into this foundation. As AI systems become more autonomous and data volumes grow, firms must establish that data privacy, access controls and ethical safeguards are in place to protect sensitive information and maintain trust. Getting the data into the right platform is only the beginning. From there, companies must establish processes to verify that data is usable, trustworthy, and accessible for AI use cases. 

1. Buildingteams that bridge strategyand execution 

Even with strong systems and solid data, the success of AI ultimately depends on people. There is intense competition for talent in AI and data science, especially in financial services. While tools are becoming more user-friendly, the need for individuals who can design, implement and manage AI systems within a complex regulatory environment remains extremely high. Experts who understand both the technical and compliance requirements of financial services are in especially short supply. 

To address this, financial services organizations must rethink how work is done and empower employees across the institution, not just specialized teams, to use AI in their daily tasks. AI should be something everyone in the company can use. Achieving this requires not only accessible tools but also a cultural shift in how teams collaborate and innovate.  

Of course, technical specialists are still needed. Data engineers, machine learning experts and governance professionals are critical to building and maintaining AI systems. But assembling a team that balances institutional expertise with specialized skills is what makes AI work in the real world. The most effective teams are those that can bridge strategy and execution by combining deep knowledge of the business with the technical capabilities to deploy AI responsibly and at scale. 

Unlocking the true value of AI will require significant investment in data architecture and governance so that AI agents can understand the context and meaning behind the numbers. AI has the potential to deliver real business transformation. But to realize that potential, financial services institutions must begin where it matters most: the underlying systems and the data. Those organizations that will succeed with AI are not necessarily those with the most sophisticated tools, but those that have done the work to modernize IT systems, strengthen data capabilities and build the right teams. 

The views reflected in this article are the views of the author and do not necessarily reflect the views of Ernst & Young LLP or other members of the global EY organization.