Analytics and AI-driven enterprises thrive in the Age of With
The culture catalyst
​More than a decade after the concept of big data became part of the lexicon, only a minority of companies have become insight-driven organisations—and culture may be the culprit.
THE amount of data available to organisations every day continues to proliferate at a staggering volume. But technologies such as analytics and artificial intelligence (AI) have the potential to help businesses make better use of these massive volumes of data. In an age of collaboration between humans and machines—what we call the "Age of With"1—organisations can gain advantage by designing systems in which humans and machines work together to improve the speed and quality of decision-making.
But not every organisation is optimising the opportunities available in the Age of With. Some do little or nothing with data to aid their decision-making. Others carry out analytics projects in pockets of the business. Far fewer consistently embed analysis, data and evidence-based reasoning into their decision-making process.
Most large companies fall into the last two categories, as do all the companies surveyed recently by Deloitte (see sidebar, “Survey methodology: Becoming an insight-driven organisation”).
It’s understandable why many companies feel they are far down the path of becoming an insight-driven organisation. Many have invested in creating the requisite data initiatives, analytics, or data science groups. Many have created chief data officer (CDO) or chief analytics officer (CAO) organisations. The vast majority have invested in tactical solutions.
These evolutions seem natural. It has been more than a decade since the term “big data” became part of the lexicon. Many legacy issues that traditionally posed barriers have now been eliminated or reduced. These include the high cost of data storage, expensive proprietary software and the need to devote capital to expensive data centres.
But what is the reality? How many companies have truly evolved into insight-driven organisations?
To find out, in April 2019, Deloitte posed questions to more than 1,000 executives working at large companies (500+ employees) who interact with, create, or use analytics as part of their job. The goal of the survey was to see how many identified their company as being in the top two categories of the Insight-Driven (IDO) Maturity Scale (see figure 1)—analytical companies and analytical competitors2—as well as discovering how fully they leverage data and tools to make decisions and what role culture and talent play in IDO maturity. Among our findings:
- Most executives do not believe their companies are insight-driven. Fewer than four in 10 (37 per cent) place their companies in the top two categories of the IDO Maturity Scale and of those, only 10 per cent fall into the highest category. The remaining 63 per cent are aware of analytics but lack infrastructure, are still working in silos, or are expanding ad hoc analytics capabilities beyond silos.
- Culture can be a catalyst or a culprit. Establishing a data-driven culture is harder than acquiring the right tools or hiring the right talent. But it pays off. Organisations with the strongest cultural orientation to data-driven insights and decision-making were twice as likely to have significantly exceeded business goals. Among the 37 per cent of companies in the survey with the strongest analytics cultures, 48 per cent significantly exceeded their business goals in the past 12 months, making them twice as likely to do so compared to the 63 per cent that do not have as strong an analytics culture.
- Aim high for analytics champions. Executive sponsorship is vital to this level of organisational change and the best champion sits in the corner office. According to the survey, the CEO is the lead champion of analytics in 29 per cent of companies surveyed and these companies are 77 per cent more likely to have significantly exceeded their business goals. They are also 59 per cent more likely to derive actionable insights from the analytics they are tracking.
- Most executives are not comfortable accessing or using data. Fully 67 per cent of those surveyed (who are senior managers or higher) say they are not comfortable accessing or using data from their tools and resources. The proportion is significant even at companies with strong data-driven cultures, where 37 per cent of respondents still express discomfort. This points to a major opportunity for companies to provide more education and improve the user experience if they want every employee to use insights as part of their work.
Survey methodology: Becoming an insight-driven organisation
To obtain a cross-industry view of how organisations approach business analytics and where they fall along an analytics continuum, Deloitte conducted an online survey of 1,048 executives (senior managers or higher) who interact with, create, or use analytics as part of their job and work for US-based companies with 501+ employees. Thirty-six per cent are senior vice-presidents and above, and 13 per cent are from the C-level. We eliminated 928 potential respondents representing approximately 35 per cent of the potential survey base because they do not have any interaction with analytics in their companies. The survey was fielded in April 2019 with a margin of error of ± 3.03 per cent at the 95 per cent confidence level.
Business analytics becoming mainstream
Three-quarters of survey respondents report that their organisation’s analytical maturity has increased over the past year and nearly as many—70 per cent—expect business analytics to be more important in the next three years than it is now.
Accompanying these indicators of increased organisational awareness is the finding that over the next few years, business analytics as an organisational priority is expected to be on par with such critical drivers of business value as risk management, reputation management, product/service innovation and managing growth expectations (see figure 2). In other words, analytics is becoming an established fact of business life and no longer the sole domain of the IT or finance department.
AI, the more technology-intensive relative of business analytics, is not yet used as commonly as some other business and management tools—though other Deloitte surveys suggest that its use is growing even more rapidly than business analytics.3
As figure 3 shows, analytics is being used to support a number of strategic areas representing a broad spread of fairly mainstream use cases with no low frequencies. The top use—identifying business process improvements—aligns with one of the top organisational priorities among respondents shown in figure 1—overall cost efficiency. Using analytics to improve processes, such as optimising pricing or stock levels, represents a classic use of analytics.
Slightly less common is the use of analytics to guide product and service development or monitor competitors. This is not surprising. Typically, there is less structured numeric data available in these functions. Also, as is discussed later in this report, companies are far less likely to rely on unstructured data such as product images or customer comments gathered from interacting with a chatbot while shopping online.
The insights trifecta: Data and tools, talent and culture
Being an insight-driven organisation is not the result of any single factor; it is multidimensional. For organisations to fully leverage the insights they derive and embed them into decisions and actions, a combination of three drivers is required: data and tools, talent and culture.
Beyond BI and spreadsheets
Although it’s only one component of success, investment in and adoption of data and tools is perhaps the most visible and easily measured manifestation of where a company is on its journey to becoming an insight-driven organisation.
The traditional workhorses of the data analytics universe—spreadsheets such as Microsoft Excel and business intelligence tools such as Microsoft Power BI or IBM Cognos—are the most commonly used tools. Yet 67 per cent of the companies surveyed also use at least one advanced tool such as SAS, an open source tool such as R, a programming language such as Python, or an AI tool.
This range of tool sophistication is encouraging. However, most organisations in the survey are at risk of hindering the success of their analytics by limiting the types of data they employ and limiting the chance to adopt a cross-enterprise approach.
According to our survey, most organisations (64 per cent) today report relying on structured data from internal systems/resources. Far fewer (18 per cent) have taken advantage of unstructured data (such as product images or customer audio files) or comments from social media. These data types can be difficult to put in the row-and-column relational format characteristic of traditional data storage, but over the last decade several new technologies have emerged to address that, including Hadoop and other open source projects, cloud-based architectures, approaches to managing streaming data and new storage hardware environments.4
These forms of data are often more challenging to interpret but can deliver a more comprehensive and holistic understanding of the bigger picture—particularly of the world outside an organisation. A look at companies that exceeded their business goals shows a connexion with this appreciation for unstructured data: Executives who say unstructured data is one of the most valuable sources of insights are 24 per cent more likely to have exceeded their business goals, according to our survey.
The majority of companies today also adopt a fragmented, siloed approach to analytics tools and data, which correlates with diminished business success. Sixty per cent of companies in the survey that use different tools and systems across different teams or business units exceeded their goals last year. But among a much smaller group—the 26 per cent that use a single, common set of tools and methods across the enterprise for accessing and analysing data—an impressive 80 per cent exceeded their business goals last year. The absence of one enterprise approach to finding insights is a common barrier to effectiveness.
Data terms decoded
Analytics: The systematic quantitative analysis of raw data to draw conclusions that help drive business strategy and performance.5
Artificial intelligence (AI): The development of computer systems that can perform tasks previously performed only by humans.
Big data: Big data is structured and unstructured data generated from diverse sources in real time, in volumes too large for traditional technologies to capture, manage and process in a timely manner.6
Machine learning (ML): A method of data analysis that automates the building of analytical models. These algorithm-based models are primarily built from statistical techniques and theoretical computer science and leverage large datasets to continuously learn and improve.7
Case study 1: A common data and analytics environment at Procter and Gamble
Procter & Gamble (P&G) has long maintained a common, enterprisewide approach to data management and analytics as part of its Global Business Services organisation. It established global standards for data type and quality; with these standards, aggregating and comparing data across product lines and regions became significantly less complicated with the data being stored in standard formats in the central data warehouse. Over time, managers found that the single, companywide database played a much more strategic role in aiding decision-making by serving as the “one truth” for the entire corporation. The company’s IT organisation also automated the generation of reports that were used across multiple business units. In addition to simplifying the task of retrieving data and performing some basic analyses, the reports served to standardise the way data was visualised across the company.8 With consistent visuals, analysts and managers from one unit could step into a role, or even a meeting, with a different product or region and quickly understand the situation. The IT organisation also developed scaled and standard solutions for accessing analytics, including the Decision Cockpit, a web-based customizable dashboard that tracked the most relevant data and news for each individual employee. For management teams, P&G developed a patented physical environment for information-based executive decision-making called the Business Sphere. Leadership teams would typically gather in over 50 Business Spheres around the world at regular intervals to review product and market performance for the previous periods and make decisions about how to proceed going forward.9
Data science is a team sport
Talent can be spread broadly across an organisation or concentrated among a select few. D.J. Patil, appointed as the US government’s first-ever chief data scientist in 2015, liked to say that data science is a team sport.
But if data science comprises teams within organisations, the teams are generally small and homogeneous. The survey data shows that two-thirds of organisations rely on a select group of employees who have been trained on analytics or data science, versus 27 per cent who say that all employees are trained (quite rare in our experience) or the small minority that trains no employees.
Organisations need to embrace a diversity of roles and skills. Instead of relying on siloed teams of highly technical quantitative experts, companies would do well to cultivate a wide variety of people throughout the organisation who are curious, numerate and capable of translating between analytics/data science methods and business requirements. This might be called the “democratisation” of data science.”10
Data from the survey confirms the merit of this approach. In companies where executives report that all employees have been trained on analytics, 88 per cent exceeded business goals, compared to just 61 per cent of companies in which only select employees have been trained on analytics.
Concomitant with the need to involve all employees in the use of analytics for decision-making is the need for user-centred design and stakeholder analysis. Decision-makers, data scientists and business analysts, all must care about the business outcomes and be consulted frequently on analytics projects. Effective users of analytics are those who understand what matters most, prioritising the right questions and zeroing in on the right decisions to improve. They need to understand where to find the insights and quickly know what to discard, to avoid being overwhelmed by the massive volumes of data that businesses now routinely generate or receive.
The issue of improving companywide access and use of analytics is not restricted to employees at lower levels in the organisation. The survey shows that 67 per cent of those surveyed (who are senior managers or higher and interact with data) say they are not comfortable accessing or using data from their tools and resources. Surprisingly, comfort level rises with title level, which could show access to resources also rises with level in the organisation. In any case, the fact that two-thirds of executives at large organisations are not comfortable navigating what is now the lifeblood of every business is a gap that could yield huge dividends if addressed.
Case study 2: Building an analytical talent ecosystem at Principal
Principal Financial Group, based in Des Moines, Iowa, offers insurance, retirement and asset management products. Recruiting data scientists and AI specialists is difficult anywhere and especially away from hotspots such as the Bay Area, Boston and New York; so Joseph Byrum, the company’s chief data scientist, has been forced to be creative about building a talent ecosystem. In addition to hiring data scientists, he’s explored crowdsourcing and close relationships with university analytics programmes. Over the course of his career, Byrum has worked with thousands of crowdsourcing projects and he strongly believes that they generally yield more innovative solutions than working with internal domain experts. He works with several different crowdsourcing firms. In addition to crowdsourcing, Byrum and Principal also rely heavily on an even less common talent source—student capstone projects in analytically oriented programmes at universities. There are now hundreds of such programmes in the United States alone,11 and many of them have “capstone” programmes in which students are expected to work on real business problems with corporate sponsors. Byrum has set up over 60 capstone projects at a variety of universities—many involving important problems for Principal to solve. Of course, Principal does need to hire some internal people as well to help develop AI and analytics solutions and Byrum has thus far built a team of about 20. He hires often through the capstone projects, which Byrum finds much more useful than interviews. He says that the staff that he does hire are a little different than most data science types in other companies—they have quant skills, but also design thinking and systems engineering skills. He also puts a strong emphasis on their ability to communicate and collaborate and uses psychological profiles to ensure that members of his team complement each other.12
A culture that acts on insights
Among the key drivers that help companies scale from carrying out analytics projects in pockets to becoming an IDO, a data-driven culture is the most difficult to establish. It also appears to be the one factor holding back many organisations.
A data-driven culture is one in which important decisions are made based on data and analytics (assuming that data is available). It is important to note that data, technology, and solid statistical and machine learning capabilities are facilitators but not necessarily drivers of insight-driven decision-making. There has to be a willingness to act on analytically derived insights—to make decisions, change processes and adapt behaviours based on insights rather than intuition.
The survey highlights this powerfully by revealing a strong correlation between culture and business performance: Organisations that reported having the strongest cultural orientation to data-driven insights and decision-making are twice as likely to have reported exceeding business goals in the past 12 months. Forty-eight per cent of these businesses say they outperformed their target versus just 22 per cent of those with a more diluted analytics culture (see figure 4).
A twofold difference in not just meeting but exceeding business goals is a significant difference and it contributes to the dividing lines between the insight haves and have-nots we describe in the introduction.
Creating any form of desired culture is a challenge for most organisations and analytics is no exception. Only 39 per cent of respondents say their company has a strong cultural orientation to data-driven insights and decision-making, and a similarly low percentage (37 per cent) feel employees in their company are aware of the importance of data analytics.
But, as with other areas in this survey, organisations that have a strong analytics culture and employee awareness of analytics are more likely to have exceeded business goals, illustrating the bottom-line benefits of investing in creating the right culture and elevating employee awareness of the importance of analytics.
How companies assign responsibility for analytics is a critical factor in moving up the maturity scale. Here the results are encouraging: Analytics is deployed companywide at the majority of organisations (57 per cent), while one in three deploy across specific company groups and only one in 10 deploy them in an ad hoc manner.
As with talent, the survey data indicates diffusing responsibility across organisational lines is more effective than localised responsibility (see figure 5). However, the survey shows most rely on a pool of identified experts rather than charging all employees with responsibility for developing insights or learnings from data analytics. In other words, analytics is not yet a team sport.
The concept of necessary-but-not-sufficient is a critical one for insight-driven maturity. It is easy to mistake the necessary for good enough; but building tools and acquiring data are not sufficient to reach the top levels of IDO maturity. As figure 5 illustrates, spreading responsibility for analytics across the enterprise and making analytics-gathering and decision-making a team sport are crucial to success. Acquiring tools without engaging teams will not lead to better business outcomes.
The IDO Maturity Scale: Five kinds of organisations
As stated earlier, when companies in the survey were asked to rank themselves on the IDO Maturity Scale (see figure 1), 63 per cent placed themselves in the bottom three categories—analytics aware, localised analytics and analytics aspirations, meaning they are not insight-driven organisations (see figure 6).
The methodology used in the survey eliminated approximately 35 per cent of the potential survey base because they did not have any interaction with analytics in their companies. This means the insight-driven organisations in the survey make up 37 per cent of a group that is, at a minimum, analytics aware. Given that the overall population of companies includes those that are not even analytics aware, the number of insight-driven organisations in real life is very likely even smaller than this survey indicates.
In addition to measuring where companies placed themselves on the maturity continuum, we also wanted to examine what patterns emerge when we looked at how they approach analytics culture, data and tools, and talent. To analyse this, Deloitte cross-referenced respondent’s self-ranking on the continuum with other factors.
In general, development in key input (tools, talent, culture) and outcome (business success) metrics rises across the maturity continuum.
An interesting exception is the group that is lowest on the continuum, those that are analytics aware. They rated themselves higher (close to the top performers’ rankings) in all the key factors making up analytics maturity. Given that they represent only 6 per cent of respondents, this anomaly represents a very small percentage of the data. It is possible that their atypical responses may be a layman’s appraisal of the factors driving analytics maturity. In other words, they may be an example of the adage, “You don’t know what you don’t know.”
The otherwise steady progression of best practises on the continuum below illustrates the importance of leadership and culture in becoming an insight-driven organisation. These two factors are most likely to be the drivers of this segmentation, in our experience. For example, committed leaders can provide data and tools, and hire the necessary people.
Making culture a catalyst
In our experience—reinforced by this survey—the vast majority of companies do not have initiatives in place to address data-driven culture issues. Initiatives around data and technology, and even hiring specialised talent, are not enough to bring about the cultural changes needed to help companies evolve to being insight-driven.
Here are some recommended steps to bring about the cultural changes needed to become an IDO:
- Hire or promote leaders with a strong orientation to analytics-based strategy and competition.
- Educate employees at all levels and in all functions about the role of analytics in business decision-making.
- Implement individual performance assessment tying the use of analytics to incentives.
- Encourage leaders to model examples. In meetings, for example, demonstrate asking for data points to back up business decisions.
- Make it easy for employees to act on data and analytics through nudges, an effective way to motivate desired actions.13
- Use social proof, a concept explored by the prominent social psychologist Robert Cialdini and a standard part of the “nudge” toolkit, to inspire action by discussing how other companies are doing this.14
- Reward trying and risk-taking, even if efforts fail. Create a culture that respects the notion of honourable failure.
- Know the limits of analytics: If you can’t get the data, you can’t gain the insights.
- Enlist an executive sponsor, ideally the CEO, so that you can melt away the “permafrost” of change-resistant middle management.
Final thoughts
Our 2019 survey results clearly show insight-driven organisations represent a minority of businesses today, despite the number of years technologies such as big data and analytics have been readily available. The most common culprit is culture. Buying and using analytics tools is not hard—changing behaviours is. By emphasising education, enlisting executive sponsors, and modelling and rewarding the right insight-driven behaviours, organisations doing business in the Age of With can benefit from human collaboration made greater with AI and analytics.
Deloitte Analytics and AI
​Achieving your business outcomes, whether a small-scale programme or an enterprisewide initiative, demands ever-smarter insights—delivered faster than ever before. Doing that in today's complex, connected world requires the ability to combine a high-performance blend of humans with machines, automation with intelligence, and business analytics with data science. Welcome to the Age of With, in which Deloitte translates the science of analytics—through our services, solutions, and capabilities—into reality for your business.
