Bridging measurement divides in AI, cloud, and cyber
Investments in AI, cloud, and cybersecurity are on the rise. An alignment in how business and technology leaders see tech value can elevate funding and support.
Tim Smith
Chris Thomas
Rohit Tandon
Ian Fleming
Iram Parveen
Diana Kearns-Manolatos
Joseph Price
As organizations increasingly invest in generative AI, cybersecurity, and cloud solutions to power their digital transformations, business leaders are eager to demonstrate the value of every one of these investments. Without the right measurements in place, however, individual tech investments can be undervalued or underfunded, wasting valuable resources, missing opportunities for growth, and fueling misaligned expectations.
Our research into the various measurements that organizations use to evaluate tech investments draws insights from two surveys—a 2023 survey of 1,600 global business and technology leaders and a 2023 survey of 2,835 generative AI decision-makers—as well as interviews with 10 C-suite executives from varied industries and geographies. This research revealed disparities in the value business and technology leaders expect from AI, cloud, and cyber—and how that value is measured.
The link between value expectations and measures can’t be underestimated, as leaders typically focus first on what they want from a technology before selecting key performance indicators to track against those expectations. Therefore, discrepancies in expectations can evolve into systemic measurement challenges, jeopardizing the ability to direct capital accordingly and achieve returns. Indeed, our past research has shown that when leaders have too narrow a view of digital value, they can put up to 20% of digital investment returns at risk.
Measurement behaviors and expectations vary between business and tech leaders
To ensure a broader view of digital value across your organization, consider bringing business and technology leaders together. In our research, these groups showed clear gaps in alignment—warranting regular collaboration to align strategic expectations and define shared success measures across AI, cloud, and cyber investments. Reconciling business and tech leaders’ value priorities can help maximize returns from emerging technology capabilities.
In a previous article, we explored how business and tech leaders measure the success of their tech investments across 10 broad categories of KPIs. The categories ranged from traditional financial metrics (earnings before interest, taxes, depreciation, and amortization [EBITDA] and operating margins, for example) and customer engagement metrics (like net promoter score and customer acquisition cost) to emerging and non-traditional metrics like social media sentiment and share price volatility. (For a complete list of 46 KPIs, see “Mapping digital transformation value”).
Broadly, we found business and technology leaders differ in how they’re evaluating the success of their tech investments:
- AI: Generative AI is driving demand for investments in “traditional” AI capabilities like machine learning and predictive AI, and cloud, and cyber, but currently, technology leaders are more optimistic than business leaders about the benefits their organizations can reap from generative AI. Our analysis suggests business leaders may be undervaluing the potential for generative AI to detect fraud, uncover new ideas, and increase the speed and ease of developing software.
- Cloud: Business and technology leaders often see cloud investments through different lenses. Many business leaders focus on back-office metrics like process effectiveness and utilization. However, tech leaders tend to recognize cloud's larger potential to directly boost new revenue growth.
- Cybersecurity: When it comes to cybersecurity, the misalignment is about a growth versus risk aversion focus. Business leaders have a broader perspective on the added strategic value of cyber. In contrast, technology leaders tend to concentrate more narrowly on defensive outcomes like the number of security incidents processed and attack attempts detected.
Business leaders could learn more about AI’s true potential from tech leaders
Given generative AI’s potential to reduce costs, grow revenue, accelerate innovation,1 and more, the market is poised to grow to US$1.3 trillion by 2032 according to estimates from Bloomberg Intelligence.2 The opportunity is huge, and the expectations are high, but business and technology leaders expect different sets of benefits (figure 1), as revealed by Deloitte’s state of generative AI in the enterprise Q1 report, which had 2,835 respondents. Business leaders are less likely to be targeting strategic benefits such as innovation and growth.3
Alignment in these areas could help enhance the quality of strategic decisions about how, when, and where to invest—maximizing returns for the organization, customers, and shareholders.
Of the two groups, business leaders may have more work to do to expand their understanding of generative AI’s value potential. Both groups expect generative AI to improve efficiency and productivity, but technology leaders anticipate greater benefits from this technology in all areas.
Two value-adds that business leaders may not be fully accounting for showed significant disconnect between the groups: Technology leaders are 34 percentage points more likely than business leaders to expect that generative AI will help companies detect fraud and manage risk and 30 percentage points more likely than business leaders to expect generative AI to increase the speed or ease of developing new software and systems. Perhaps this is because tech leaders have more firsthand knowledge of where there are strong data foundations in place: a technical prerequisite to getting generative AI programs off the ground.
Insight to action: Business leaders should recalibrate with tech leaders’ generative AI value expectations. With generative AI offering exponential growth and innovation possibilities, business leaders should consider recalibrating their thinking to close this expectation gap with tech leaders. With trillions of dollars expected to be spent on generative AI, the stakes are too high for business leaders to be focused only on efficiency improvements or cost reduction. Technology leaders should clearly articulate these inherent capabilities of the technology so that business leaders formulate effective financial plans and don’t risk missing its strategic potential. An essential investment, even a prerequisite, will be having a strong data foundation in place to power generative AI solutions. Technology and data leaders should advise on the existing AI processes, the tech stack, and data maturity.
Past AI investments in traditional AI point to leadership differences over KPIs
Are leaders on the same page about generative AI KPIs? Given that the technology is rapidly evolving, and its adoption is exponentially accelerating, a study aimed at this information would likely produce tenuous insights. Expectations make sense to measure now, but the KPI landscape may not have matured enough to get a clear picture of how leaders are measuring value. There are clues, however, in how business and technology leaders measure value for traditional AI that, when taken together with their expectations, give us a sense of what may be developing. After all, generative AI capabilities are part of a much larger class of AI investments that include machine learning, deep learning, and conversational AI that have established measurement behaviors.
For insights on how business and tech leaders were measuring traditional AI prior to the current rush to adopt generative AI, we turned to data from our February, 2023 survey of 1,600 respondents. Leaders investing in these more traditional AI capabilities—like machine learning, deep learning, and conversational AI—according to this data, have tended to focus more on the same traditional financial and customer KPIs we observed from digital investments broadly. However, we can see that business leaders and technology leaders have established—and not fully aligned—KPI priorities (figure 2).
When it comes to AI, tech leaders look outward at sales, brand, and intangible measures. We see signals in the data that technology leaders are tracking some AI KPIs more broadly than business leaders. Technology leaders are 12 percentage points more likely than business leaders to be using sales of new digital products as a KPI and seven percentage points more likely to be focused on sales through new digital platforms.
They also use net promoter scores and intangible assets more than business leaders. These strategic measures suggest technology leaders expect the true value potential of AI to be tied to core business applications. In other words, they view AI as a reusable asset. For instance, an oil and gas company could use the same learning model and data for predictive maintenance to proactively diagnose and correct faults in machinery as it does to manage risk related to supplier performance.4 Generative AI models can similarly be repurposed for different types of tasks within an enterprise.
Business leaders appear more interested in the process benefits associated with traditional AI investments. As AI is increasingly used in mission critical, customer-facing applications, however, they should also consider new KPIs more closely aligned with product revenue and customer success.
Insight to action: Tech leaders should educate business leaders on AI’s strategic innovation potential. It can add value to customers, create new revenue streams, manage risk, enable sustainability strategies, and more.
Secondly, when it comes to all forms of AI, business and tech leaders alike may be collectively missing opportunities to consider innovation measures and long-term value creation. Among those leaders who measure traditional AI, only 31% to 32% use innovation KPIs like tolerance for experimentation and intelligent failure and number of agile pods or teams.
Connecting the dots between cloud’s strategic and tactical value
While already essential to most digital transformations, generative AI is expected to dramatically accelerate cloud consumption and investment. Public cloud services are predicted to grow 16% from 2023 to 2024, to US$690 billion in 2024, according to forecasts from Statista Market Insights.5 And while, in our February 2023 survey, 70% of cloud investors said they believe they’re getting value to a large or very large extent, leaders will likely need to build strong business cases to secure future investments while managing ROI, technical debt and the potential for ballooning costs based on surges in cloud consumption.
Leaders, however, may be too heavily focused on back-office measures like “process performance.”6 Our data indicates this larger, core issue is driven by business leaders. They generally focus more on factors like procurement value for money, employee utilization, and EBITDA than technology counterparts when making cloud platform investments and are focused on process effectiveness 13 percentage points more than tech counterparts for cloud-native investments (figure 3).
For both cloud-native applications and cloud platforms in the 2023 tech value survey, tech leaders pay closer attention to sales of new digital products, customer retention rate, and net promoter score at levels well above business counterparts.
Notably, business leaders in the 2023 tech value survey are more likely than tech counterparts to measure organizational mission fit by eight percentage points when it comes to cloud-native solutions. This is a sizeable gap, perhaps a positive sign that business leaders are effectively communicating the larger strategic purpose for the digital applications related to these investments.
Insight to action: Technology leaders see the strategic potential of cloud to generate net new revenue. However, there’s an opportunity for tech leaders to understand how cloud investments may also contribute to mission and organizational purpose.
Additionally, leaders should think about where and when traditional cloud and native solutions have shared outcomes. If they can identify where they interoperate as part of their solution design, they can reduce complexity across systems, as well as technical debt—an important ROI pain point. For example, business leaders might task tech teams to look for efficiencies across cloud platform and cloud native investments as part of multi-cloud programs.
Cyber investors measure a wider range of KPIs
As organizations continue to innovate, a robust cybersecurity and risk-mitigation infrastructure is critical to an organization’s digital portfolio.7 Broadly, business and technology leaders are aligned on what constitutes a successful cyber investment, but when it comes to certain cyber technologies, they’re not.
Business leaders investing in cyber capabilities measure almost all top cyber KPIs to a higher extent than tech leaders—even indicators like innovation and number of agile pods that fall well within the technology domain (figure 4).
This is a notable difference compared with other technologies and could be due to several factors. First, over the last decade, cyber has been elevated to a board level issue with chief information security officers and other business leaders having a strong voice in cyber strategies. Second, business leaders’ broader focus is on risk mitigation, and they understand that an early investment in cyber would result in a lower total cost of ownership for the asset. Third, business leaders are prioritizing cyber KPIs that yield positive outcomes from the technology, while tech leaders tend to focus on negative outcomes they want to avoid, like number of incidents processed, number of attack attempts seen and others.
On the flipside, cybersecurity rating—an independent scoring of the state of an organization’s IT security—is not among the most used cyber KPIs for either group. Tech leaders measure it slightly higher than business leaders, but the usage is still lower than advisable. This could be an area of potential risk, given leaders may have a more optimistic perspective of the state of cybersecurity in their own organizations, overestimating their strengths and overlooking their weaknesses. Hence, as necessitated by the US Securities and Exchanges Commission, organizations can seek independent evaluation to understand where they can improve security, mitigating the need for reactive responses and higher costs later.
Among these cyber investments, we also analyzed four cyber capabilities (figure 5). Cryptography shows greater differences between business and tech leaders. Business leaders value cryptography for ensuring business continuity, prioritizing it 13 percentage points higher than tech leaders. It’s still a new technology, so they may benefit from tech leaders’ knowledge of its additional benefits. Technology leaders emphasize organizational resilience, EBITDA, employee productivity, and other measures that business leaders may be paying lesser attention to.
On the other hand, when investing in zero-trust security and identity and access management (IAM), business leaders use a broader set of KPIs than tech leaders, touching on more workforce, customer, and financial measures.
Insight to action: While business and technology leaders are more aligned on cyber investments overall, each capability brings new and different elements to a cyber program that both groups should ensure they understand.
Business leaders tend to have a more mature understanding of how zero trust and IAM capabilities can proactively minimize the risk of unauthorized access. They emphasize KPIs related to how these capabilities safeguard customer and employee trust and ensure regulatory compliance. Business leaders are clear on foundational outcomes to expect from zero trust and IAM investments across the digital ecosystem.
Secondly, as advanced cryptography techniques become more important to secure cloud solutions, AI, quantum and more, leaders are focused on a different set of KPIs. While business leaders’ top measure is business continuity, tech leaders emphasize employee productivity, an organization’s resilience, and EBITDA. Here tech leaders may be missing the bigger picture. They should look beyond day-to-day processes, recognizing how strict data encryption measures enable business continuity.
Lastly, to avoid becoming complacent and overvaluing their internal security systems, both business and tech leaders should pay attention to their organization’s cybersecurity rating by undertaking independent cybersecurity assessments. While business leaders might be skeptical of exposing sensitive internal systems to an external risk assessment, technology leaders can help them understand the value of these assessments to strengthen their risk mitigation systems and comply with SEC incident reporting requirements.
Where we go from here
As generative AI continues to gain steam, leaders have an opportunity to better align on investment expectations to build stronger business cases and direct capital to the most strategic initiatives. Given these investments will be part of larger business transformation programs, many organizations are having these strategic discussions at the board level and across executive teams. Business leaders need to be equipped with information that speaks to the true potential of today’s technology. Technology leaders can be their partner in these strategic discussions to connect the dots between tech investments and the larger enterprise value goals.
Methodology
The data for this research is derived from two already published studies:
- Mapping digital transformation value: This analysis is based on a survey of 1,600 global business and technology leaders that are director-level and above, fielded in February 2023 across six industries: consumer; energy, resources, and industrials; financial services institutions; government and public services; life sciences and health care; and technology, media, and telecommunications. Respondents were from organizations of all sizes and structures—including both public and private companies—and 14 countries: The United States of America, Canada, Mexico, the United Kingdom, the Netherlands, Spain, Germany, France, Ireland, Australia, China, India, Japan, and Singapore. We also interviewed 10 C-suite leaders across industries and geographies. Based on the data, we developed a framework with five value categories: financial, customer, process, workforce, and purpose, 10 subcategories, and 46 KPIs.
- Deloitte’s State of generative ai in the enterprise Q1 report: Additionally, to understand the impact and adoption of generative AI within organizations, Deloitte is conducting a series of quarterly surveys. The series is based on Deloitte’s State of AI in the enterprise reports, which have been released annually for the past five years. The first generative AI survey was fielded to 2,835 director to C-suite–level respondents across six industries and 16 countries between October and December of 2023. Industries included: consumer; energy, resources, and industrials; financial services; life sciences and health care; technology, media and telecommunications; and government and public services.
