Skip to main content

Data management barriers to AI success

Is a holistic data modernisation approach the answer?

Companies are modernising their data infrastructure as part of AI initiatives to gain competitive advantages. How can technology providers help businesses solve data management challenges?

BUSINESSES are pursuing a range of AI initiatives and modernising data infrastructure tops the list. But current data practises are an issue, as several companies haven’t attained a high level of sophistication with crucial data-related aspects. A Deloitte study of AI adopters finds businesses face challenges in critical aspects of data management: preparing and cleaning data, integrating data from diverse sources, training AI models and ensuring data governance.

In Deloitte’s latest State of AI in the Enterprise survey, at least 40% of adopter organisations reported “low” or “medium” level of sophistication across a range of data practises.1 Moreover, nearly a third of executives identified data-related challenges among the top three concerns hampering their company’s AI initiatives.

Is cleaning data really that difficult? Research suggests that, indeed, cleaning crude and inaccurate data before feeding it into an AI model is a cumbersome process. For instance, companies now routinely spend six to 12 months cleaning the data.2 Cleaning data is vital, as the cost of adjusting poor data rises dramatically later in the process.3

Data preparation demands persistence and disciplined execution. Data specialists spend a large part of their workweek preparing data for analytics and AI/machine learning (AI/ML) initiatives.4 As more organisations shift their AI workloads to a cloud environment, data integration challenges are intensifying. Some of the most common barriers to access third-party data sources include dealing with disparate data that exists on different systems and merging data from diverse sources.5 For all these efforts, the right talent and expertise can be critical. Often, AI/ML initiatives fail primarily due to lack of expertise, besides other major factors that include unavailability of production-ready data and integrated development environment.6

To add to these issues, data governance is fast gaining prominence as a problem spot. A 2019 study found that more than half of organisations lacked a formal data governance framework and a dedicated budget to address the issue.7 And even as regulatory scrutiny has intensified worldwide, Deloitte’s State of AI survey finds leaders “highly concerned” about a lack of data policy for personal data use. A shortage of specialists and difficulty in building a comprehensive data strategy are among the top challenges impeding data governance efforts.8

If these various data management and governance issues are not addressed early on, deeper issues could emerge later to fracture AI initiatives. AI technology providers can play a role in supporting businesses to navigate shortcomings related to data practises by:

  • Aligning data strategy with business outcomes, working with adopter organisations to clearly understand the business needs, the AI business case and data management needs
  • Bringing a multidisciplinary team comprising AI/technical, business, regulatory, and domain specialists to establish data practises and strategy that align with the customer organisation’s business goals and outcomes
  • Building a scalable data-based AI solution, considering all potential end users (employees, customers, business partners) of AI systems and the overall IT infrastructure (on-premise, cloud, proprietary IT, open-source)

These steps can enable the adopter organisations to develop a holistic data-based AI strategy that scales with and adapts to their changing needs and demands. As Deloitte LLP US technology sector leader Paul Silvergate notes, “Gleaning information from data—and then a competitive advantage from that information—requires a clear view of where the business is going, coupled with a tight link between the business and IT.”

Technology, Media & Telecommunications

Deloitte's Technology, Media & Telecommunications (TMT) industry practice brings together one of the world's largest group of specialists respected for helping shape many of the world's most recognised TMT brands—and helping those brands thrive in a digital world.

Learn more

Thanks to David Jarvis for his guidance and support with survey data, and to Divya Tewari for her research support.

Cover image by: Viktor Koen

  1. Beena Ammanath, Susanne Hupfer, and David Jarvis, Thriving in the era of pervasive AI: Deloitte’s State of AI in the Enterprise, 3rd Edition, Deloitte Insights, July 14, 2020.

    View in Article
  2. David Roe, “Why organizations need to clean their dirty data,” CMS Wire, December 13, 2019.

    View in Article
  3. AI Multiple, “The ultimate guide to data cleaning in 2020,” June 21, 2020.

    View in Article
  4. An online survey conducted by Trifacta covering 600-plus US-based data professionals (from analysts to C-suite levels) revealed that 46% of respondents spent 10-plus hours every week to properly prepare data for analytics and AI/ML initiatives. Some data professionals spent upward of 40 hours a week. See Trifacta, “Obstacles to AI & analytics adoption in the cloud,” January 2020.

    View in Article
  5. Ibid.

    View in Article
  6. In a survey covering more than 2,000 IT and line-of-business decision-makers, IDC noted that 28% of AI/ML initiatives failed due to lack of expertise, unavailability of production-ready data, and absence of integrated development environment. See IDC, “IDC survey finds artificial intelligence adoption being driven by improved customer experience, greater employee efficiency, and accelerated innovation,” June 10, 2020.

    View in Article
  7. Gartner, “Gartner says data and cyber-related risks remain top worries for audit executives,” November 7, 2019.

    View in Article
  8. Business Application Research Center, “How to rule your data world: The role of data governance,” March 2020.

    View in Article

Did you find this useful?

Thanks for your feedback

If you would like to help improve further, please complete a 3-minute survey