From blue sky to the Cloud—a journey to data modernization
How Deloitte helped a global financial services powerhouse power up to the cloud
The situation
One of the world’s leading financial services organizations was moving tens of millions of futures and options contracts around the world via its many subsidiaries.
Given the high velocity of this trading, the company needed to navigate several complexities:
- As a regulated exchange, they are required to report out on the technology and business.
- Their existing stack of intricate applications and supporting data lakes is complex.
- The knowledge base across the organization in cloud engineering, security, architecture, cost optimization, management, and maintenance continues to evolve.
- There was significant business pressure to migrate to the cloud quickly and at scale within certain deadlines.
- Coordinating program outcomes and risks would require a serious effort.
Clearly, the scale here was daunting. And help was needed to coordinate a program that resulted in a successful migration and mitigated potential risks.
The company turned to Deloitte to help drive the Data and Analytics modernization effort—a multi-year transformation to Google Cloud. Deloitte's invaluable experience in data strategy, data engineering, financial exchanges, and proficiency in designing data governance operating models played a pivotal role in supporting them as they navigated through this transformative initiative. Deloitte’s collaborative working approach further proved to be a significant asset to the overall success of this effort.
Tens of millions of futures and options contracts a day. One cloud platform.
The solve
The goals were clear: to operationalize the Chief Data Office, architect a program for the next generation of data and analytics products, and engineer the deployment of data services through Google Cloud.
Several Deloitte accelerators were explored by the project team before and during the engagement, including our:
- Data & Analytics Modernization Strategy and Integration Office POV
- Target State Financial Exchanges Data Domain Model
- Google Cloud Target State Reference Architecture
- Data Governance Operating Model accelerator
- Financial Exchanges Data & Analytics Use Case Inventory accelerator
- DevOps Playbook
- PM templates
- Agile standards and templates
- Pod based delivery.
For the Data & Analytics transformation, the company was looking to build out a unified data architecture and platform that would house several BI tools consolidated into the target state.
Deloitte also deployed a group of Google Cloud domain specialists to serve as a central point of contact for the application teams migrating to Google Cloud. The Google Cloud technologies were required to be enveloped as data services within the cloud operating model, including specific deployment and test tools. Deloitte facilitated engineering services, facilitating the successful deployment of key data services and enabling the solution to go live:
- Spark on GKE and Dataflow as part of the effort to containerize and migrate batch and streaming workflows
- Google Cloud Data Fusion to support migration away from a legacy ETL solution
- Google Cloud Apigee X to manage the full-scale API lifecycle for target APIs
- Google Cloud Dataflow to deploy Apache Beam jobs via a repeatable user workflow
- Google Cloud Dataplex with Data Catalog enabled to serve as a single data fabric for data lineage and metadata tracking
In addition to Deloitte’s engineering support of the core data services in the Data Platform, Deloitte also helped the company adopt the industry-leading DevOps metrics (DORA) to provide insights into their end-to-end software delivery through its GitOps platform on GKE.
Getting to the cloud securely, safely, and with speed.
The Impact
The result? An end-to-end data solution centered on Google Cloud’s leading technologies that leveraged Deloitte’s extensive experience with Data Governance Operating Models, Google Cloud, and financial exchanges. As the transformation progressed, the company established a new Chief Data Office as a centralized data organization. This involved architecting and engineering on-premises data to native Google Cloud services, enabling supporting data services. The overarching goals included facilitating the migration of data workloads to the cloud and the creation and commercialization of new data products.
Key achievements:
Improved data management and governance:
- Developed data management, data certification and governance processes and capabilities to organize, store, and manage data.
- Enabled a single pane-of-glass view into the entire organization’s data and associated metadata and lineage.
- Onboarded over 10 data domain owners.
Designed data and analytics architecture:
- Designed a domain-driven data ecosystem with decentralized ownership while centralizing and democratizing the enterprise data leveraging Google Cloud native services.
Enabled revenue-driving business outcomes:
- Helped the company capture and prioritize enterprise-wide revenue generating (via data monetization) use cases and gathered detailed requirements for implementation.
Engineered data services:
- Enabled repeatable data capabilities via six core Google Cloud data products that serve as building blocks for accelerated cloud adoption.
Engineered delivery solutions:
- Enabled Data Foundations ahead of schedule, including comprehensive data services with documentation and streamlined user onboarding.
Building a future that makes data accessible, understandable, and effective.
