3 Steps for Financial Institutions to Modernise Legacy Systems with GenAI

Modernisation is Key to Unlocking Insights

In today’s rapidly evolving digital landscape, financial institutions are increasingly recognising the critical need to modernise their data infrastructure. For many, the choice is no longer whether to modernise, but how to do so effectively. Aging legacy systems, once the backbone of operations, are now costly to maintain and inefficient in meeting the complex data demands of today’s financial markets. These outdated systems trap valuable data within organisational silos, leaving what could be transformative insights untapped.

A prime example is a large African bank based in Morocco, which faced like many significant challenges with its legacy systems, including Oracle databases and SAP Business Objects. The bank's modernisation journey involved upgrading to an advanced on-premise Cloudera platform, integrated with Tableau for enhanced data visualisation and reporting. This project was not just about adopting new technology but was aimed at fundamentally transforming the bank’s ability to access and leverage its data more effectively.

The migration focused on five essential operational dashboards, aiming to streamline report generation and enhance data accessibility throughout the organisation. Once implemented, the state-of-the-art data platform enabled the development of business use cases. All data transformations are now automatically performed within Cloudera, with reports generated automatically in Tableau. This modernisation effort, rooted in a comprehensive understanding of the existing data landscape, has become a pivotal step in maximising the bank's data potential.

Case Study: Migrating a large African bank to Cloudera On-Premises

Understanding the Data Landscape

At the heart of the bank's data management were legacy systems like the Oracle Database, SAP Business Objects, and various Microsoft Access databases used by business users for local data enrichment. This setup, common in many financial institutions, resulted in a fragmented and opaque data landscape. Analysts interacted with the Oracle database through SAP Business Objects, while Data Services handled necessary transformations. However, the reliance on Microsoft Access for further enrichment led to significant inefficiencies and a lack of transparency in data processes.

To gain a deeper understanding of user preferences and the operational challenges faced by the bank, Theodo conducted a series of interviews with key stakeholders. These interviews provided valuable insights into how business users interacted with the data, their needs, and the limitations of the existing tools. The head of business intelligence played a crucial role in this process, offering detailed knowledge of the systems and identifying critical pain points and inefficiencies within the infrastructure. This collaborative approach allowed Theodo Data and AI to tailor their recommendations to address the most pressing issues effectively.

Data Governance and Quality

In response to these challenges, the bank recognised the importance of establishing robust data governance frameworks as a cornerstone of their modernisation efforts. A new digital factory was established to streamline data processes, supported by a data office organisation inspired by data mesh methodologies created by Theodo Data & AI. This structure allowed different teams to focus on specific sections of the data pipeline, leading to faster development and easier data integrity checks.

For example, when discrepancies arose during the migration—such as differences in data between the old and new platforms—these were quickly identified and resolved within the governance structure. This not only fostered trust but also ensured that the bank was well-prepared to take full ownership of the new data platform from the outset. This approach ensured stronger data access policies, enhancing both security and compliance with regulatory requirements.

Mapping Business Objects

Another crucial aspect of the bank's modernisation was the standardisation of business objects across the organisation. The existing fragmentation of tools and processes had led to inconsistent definitions of these objects, undermining trust in the data. To address this challenge efficiently, the bank implemented a comprehensive data mapping and migration process, with some assistance from modern tools and technologies.

The bank's approach involved several key steps:

1. Providing overviews of certain transformations within Data Services, which contributed to improved clarity in data lineage.
2. Defining schemas for the new Cloudera platform, using the legacy Oracle schemas as a reference.
3. Expediting the documentation of new Tableau reports by analysing report files, enhancing their accessibility to users.

These efforts were supported by various tools and methodologies, including some AI-assisted processes suggested by Theodo Data & AI. While not the primary driver, these tools helped accelerate certain aspects of the work.

This comprehensive approach not only standardised the data model across the bank more effectively but also empowered business users and analysts to interact with the data more efficiently. As a result, the bank laid the groundwork for improved decision-making and analytics maturity in a timely manner.

Streamlining Report Generation

Before the migration, the bank’s report generation process was characterised by significant inefficiencies. Business users had to request data extracts or database refreshes from IT teams, as these processes were not automated. Delays were further compounded by issues with data completeness, particularly when reports depended on data from core banking systems that were not promptly updated.

The bank’s modernisation efforts aimed to overhaul this process by automating data extraction, ensuring timely updates, and leveraging Cloudera’s robust processing capabilities. This shift enabled faster report generation, freeing up valuable time for business users to focus on analysis rather than data wrangling.

Key Outcomes

While the project is still ongoing, the migration was already successful in delivering four our of five dashboards with an average explainable discrepancy in data between 0 and 1%, ensuring high accuracy and reliability.

Data ingestion and transformation processes were automated and orchestrated using Airflow, enabling daily updates and improving the overall efficiency of data operations.

The bank's team, in collaboration with Theodo Data & AI, was thoroughly trained and became proficient in their new roles within the data factory, following a well-defined skills matrix. This preparation ensured that the bank was fully equipped to manage and optimise its new data platform moving forward.

Conclusion: Three Steps to Unlocking the Full Potential of Modern Data Systems with GenAI as an Enabler

This experience illustrates the challenges and opportunities that come with modernising legacy systems. While the journey may seem daunting, with the right approach, strategies, and enabling technologies like GenAI, financial institutions can transform these challenges into significant opportunities. Here are three critical steps, drawn from the bank's experience, where GenAI can serve as a valuable enabler:

1. Understand and Document Legacy Systems

• Challenge: Legacy systems are complex, with undocumented transformations and processes that have evolved over decades, making them difficult to understand and migrate.
• Solution: Conduct thorough stakeholder interviews and system analyses to gain deep insights into existing processes and user needs.
• GenAI as Enabler: While not central to this case, GenAI tools can potentially accelerate the analysis of legacy codebases and assist in generating documentation, complementing human expertise.
• Outcome: This clarity allows institutions to make informed decisions about what to migrate, modify, or discard, reducing risks and accelerating the modernisation process.

2. Establish Robust Data Governance

• Challenge: Data silos and inconsistent governance lead to poor data quality, hindering analytics and regulatory compliance.
• Solution: Implement a comprehensive data governance framework, such as the data mesh methodology, to standardise data management across the organisation.
• GenAI as Enabler: GenAI can support this process by helping to identify patterns in data usage and suggesting optimised data models, enhancing the human-led governance efforts.
• Outcome: Improved data governance facilitates better data management and ensures that the migration to modern systems is successful and compliant with industry regulations.

3. Streamline and Automate Reporting Processes

• Challenge: Manual, time-consuming reporting processes drain resources and delay access to critical business insights.
• Solution: Leverage modern data platforms like Cloudera and visualisation tools like Tableau to automate data transformations and report generation.
• GenAI as Enabler: While not explicitly used in this case, GenAI can potentially enhance automation by suggesting optimisations in data pipelines and assisting in the creation of more intuitive dashboards.
• Outcome: Automated reporting processes free up valuable time for business users to focus on analysis rather than data wrangling, leading to faster and more informed decision-making.

By focusing on these three steps—understanding legacy systems, establishing data governance, and streamlining reporting processes—financial institutions can overcome modernization challenges. The strategic use of GenAI as an enabling technology can further enhance these efforts, although it's important to note that successful modernisation, as demonstrated in this case study, can be achieved through well-planned strategies and human expertise.

The journey to a modern data infrastructure, while challenging, can lead to unlocking the full potential of an institution's data, paving the way for future growth and success. As technology continues to evolve, the integration of tools like GenAI may offer additional avenues for optimization and innovation in data modernisation projects.

Looking to upgrade your data platform or improve your business intelligence setup? We can help. Don’t hesitate to contact us! At Theodo Data & AI, we're here to make your migration smooth and successful.