Deloitte AI Institute
Bridging the Gaps in Enterprise AI
Four important questions for business leaders and technologists
Measuring AI ROI and impact
One important ingredient in value-driving AI programs is collaboration. How stakeholders work together in this AI era can make all the difference between AI use cases that deliver intended ROI and those that come up short. This could be particularly true for technologists and business leaders, who may face a gap in understanding and communicating AI endeavors.
To be sure, connecting business leaders and technologists has been important for a variety of transformational endeavors. When it comes to AI, however, the stakes may be somewhat higher. AI application can fundamentally change how organizations engage customers and differentiate products and services in the marketplace. Given that impact, technologists, and business leaders should speak the same language, measure the same outcomes, and collaborate across the enterprise.
Yet, there is often some tension. The business leader’s focus might be on delivering excellence to the customer, while the technologist’s priority could be innovation. And as technologists explore dynamic new AI capabilities, the C-suite may contend with the implications for risk management. The task today is to help harmonize these complementary priorities. To that end, consider four primary questions that can begin to help bridge the gaps in collaboration and support AI programs that deliver the intended outcomes.
Where do we start?
In a marketplace full of options, the first step could be identifying how to balance high-risk, high-reward differentiating projects against AI use cases that can drive near-term outcomes for the business. When selecting and building a portfolio of offerings, there are three primary areas where AI use cases may sit. First, there are the “out-of-the-box” solutions. These are mature applications offered by established vendors selling to a variety of customers. Because these solutions are highly commoditized and available to any enterprise, they likely do not yield differentiating capabilities. There is often also an opportunity cost with these applications, since the scientific process of innovation with AI helps the workforce learn and improve.
The second category of solutions are those developed and refined in-house using enterprise data assets. In this, the organization trains a proven model with business and production data, perhaps by leveraging a vendor accelerator, and produces something that is specific to the business needs. These types of use cases may constitute the majority of an organizations AI projects, and the goal is to rapidly show viability and a path to ROI that justifies the investment.
The third use case area includes solutions that could be truly transformative and differentiating. It may take sophisticated organizations a number of years to build the internal capabilities needed to confidently pursue a transformational project. Before diving into these areas, the organization should prove its methodology, possess the right competencies, and hold rich experience with use cases driving near-term returns.
One point of caution concerns vendors whose off-the-shelf products supposedly drive near-term results while also enabling enterprise-wide transformation. This sounds attractive, but it is often too good to be true. A ready-made product that is available to any business will likely not deliver the kind of capabilities and coordination that are needed for true transformation.
How do we define and measure success?
In assessing AI programs, stakeholders should have a single source of truth for understanding and interpreting results. Business outcomes with AI are frequently framed as efficiency and cost savings by virtue of automation. From a technical standpoint, success with AI may also be gauged by the accuracy and usefulness of algorithmic outputs. Yet, there is a larger arena of business benefits that are important factors in whether an AI program is successful, and there are many ways to ascribe value.
Stakeholders are called to collaborate in defining what success looks like and how to measure it. One of the pitfalls for technologists can be overpromising short-term cost-saving ROI. Technologists may feel compelled to promise significant returns because they seek business funding for their endeavors. If the technologists and business leaders are only weighing success against a financial metric, they may be missing other significant outcomes that are just as important. What’s likely needed is a way to measure process changes and understand what is working.
For example, in many AI projects, even those that do not reach deployment, the workforce learns from the experience. Technical skills can improve while necessary changes in AI processes and workflows become clearer. The delivered value for an AI project thus also includes intangible, qualitative returns, such as mastering a new programming language, published papers, and thought leadership in the marketplace. These downstream returns can allow the business to improve over time and equip stakeholders with the necessary experience to pursue truly transformational and differentiating use cases. Taking this broader view of success could allow the enterprise to holistically evaluate, track, and measure the impact and outcomes from its AI programs.
How should metrics be communicated and consumed?
In thinking about ROI beyond the financial impact, one important element is determining how to communicate the results of a proof of concept (POC) to technical and non-technical audiences alike. The results should be presented in a way that does not obscure insights and relevance, even for stakeholder groups with a limited role in the POC. In the effort to share meaningful metrics with the enterprise, consider ROI in three distinct areas.
Measurable ROI is the impact on process efficiencies and revenue. This is often the only kind of ROI that is referenced, and for better or worse, there are few use cases that can be fully measured in that way. For example, what is the ROI of research and development? In truth, ROI can be found across the entire value stream. Thus, a second kind of return is strategic ROI. These may be the near- and long-term strategic anchors for the organization’s AI vision and how the enterprise is progressing toward AI maturity. And alongside this is capability ROI, which can reflect improvements in methodologies and in-house competences.
Overall, results can be conceived as a system of ROI. This can resonate with business stakeholders, as they gain a clearer, more robust understanding of how the enterprise is building AI maturity.
What is the role of IT in data science and AI?
Data science and AI is a distinct domain from the organization’s IT stakeholders. Yet, IT can be impacted as those professionals contend with deployed AI and confirm it can function within existing IT assets and processes. The challenge is in part that IT can be deterministic. IT has been on a 30-year journey building credibility and maturing capabilities, and there are fixed ways to manage servers, security protocols, software upgrades, and more.
Yet, AI can inject risk for IT because AI is often inherently probabilistic. If data scientists and AI engineers operate in a vacuum and only consider IT when it is time to deploy, the results may fall short of the potential. Indeed, one of the reasons POCs fail is because not enough stakeholders were brought in from the beginning to make the endeavor successful. To cross the chasm between POC and deployed AI that delivers value, business stakeholders, technologists, and IT professionals should collaborate throughout the AI lifecycle to determine if the program can be deployed and managed at scale in a sustainable way.
The path to AI maturity is likely winding and exploratory. Answering these four questions can help set the enterprise on a trajectory toward AI maturity and impactful business outcomes.
There are important lessons to learn and resources to access, but as a foundation for value-driving AI, an indispensable asset is collaboration and communication between the business and technology stakeholders.
Get in touch
Rich Nanda
Principal | US Strategy& Analytics Offerings Leader
Deloitte Consulting LLP
+1 (312) 486 2761
Let's talk