Skip to main content

Bridging the data-sharing chasm

Although public sector data-sharing has increased, there’s more to do to balance its upside with challenges

Adita Karkera
Mahesh Kelkar
Joe Mariani

Expanding the canvas for safe, interoperable data-sharing

Data is a coin of the realm. Many have seen how companies—in fact, entire sectors—have come to base their business models and value propositions on effectively using information, analyzing data to meet customers’ rising expectations and gain a competitive advantage.

Agencies, too, are finding ways to tap the power of information to both serve citizens and improve operations. Thanks to democratization of data, there is more data about public interests than even before. The boost that data-sharing has received across government, companies, and individuals is driving innovation, enhancing public services, and improving people’s lives.1 But in collecting and using data, the public sector faces different challenges and perhaps higher stakes, dealing with more sensitive and personal information—and, often, mandatory participation.

While government leaders increasingly recognize data’s importance to improving mission performance, many are learning how to balance the upside with the dangers: The more data in circulation, the higher the privacy and security risks can be.

How can agencies alleviate the tension, balancing efforts to break down data silos with protecting citizens’ data? The answer lies in effective data governance. One way is developing interoperability frameworks that allow cross-jurisdictional and cross-sector organizations and databases to safely interact and share information. It requires governments to not only secure stakeholder buy-in—especially when dealing with external citizen and private-sector data—but also showcase the meaningful change that expanded data use can bring.

Walls coming down

  • Between agencies and government: There is an increase in data-sharing between agencies and levels of government to improve operational efficiency and decision-making (see infographic, By the numbers: Bridging the data-sharing chasm to know about the data types being used for operational efficiency and decision-making).
  • Between public and private sectors: Broader data-sharing with the private sector is creating business value and benefitting the public interest.
  • Between citizens and government: Calls for heightened data privacy and security are increasingly being heeded in government circles.

Trend in action

Data is increasingly central to government operations, from back-office activities to delivering services to constituents, and more leaders are recognizing the value of effectively sharing information. For example, by funneling data from many different city agencies into a single command center, the Portuguese city of Cascais has made mobility, construction, waste management, law enforcement, and emergency management more efficient.2

Data as a tool for public good

Businesses, individuals, and agencies increasingly view data as a tool for public good rather than a commodity to be monetized. Government can derive value by convening and collaborating within and outside its boundaries; businesses need to willingly relinquish control over some data and work with policymakers to design viable solutions. There were glimpses of this during the pandemic when governments shared data widely to facilitate research and discovery, pandemic response, and even contact tracing.3 Also, for some of the most complex policy issues of our times, the best possible evidence to inform government decisions could come from multiple departments and agencies.

Finland’s Carbon Neutral Tourism project aims to use data collaboration to improve energy efficiency in the tourism industry and move toward carbon neutrality.4 The cities of Helsinki, Turku, and Tampere are working together to balance business and societal benefits, collecting information across jurisdictional boundaries, including data on air traffic, road traffic, environment, and hotel bookings.5 Insights based on such integrated data can help drive industry stakeholders to make more sustainable choices and reduce carbon emissions, from forward-looking hotel investments to city services nudging tourists toward environment-friendly options.

Data-sharing among government, nonprofits, the private sector, and social enterprises can also help address stubborn human services challenges. To cite a particularly visible example, homelessness—worsened by the COVID-19 pandemic and the housing crisis in many cities worldwide6—demands an ecosystem approach that taps into real-time, integrated data for more effective prevention and mitigation strategies.

In the United States, the Department of Housing and Urban Development collects a great deal of information on homelessness, but this does not give a complete picture, with much data residing in jurisdictional silos, making solutions harder to develop.7 Some promising initiatives, based on local and regional data collaboration, suggest a way forward for others. For instance, Los Angeles County aims to better target prevention programs by identifying people at risk of homelessness, using a tool developed by researchers at UCLA’s California Policy Lab. The tool combines and analyzes data from eight county agencies—with an algorithm incorporating some 500 data points—to pinpoint the most effective intervention points. The tool identifies critical and “at-risk” areas where services currently reach few people.8 Similarly, the National Alliance to End Homelessness has created a “State of Homelessness” portal that aggregates data from across the nation and represents it in visual form.9

Improved data fluidity

If data can help serve the public good, more data sources can multiply the benefits—assuming usage is safe, responsible, and effective. The last two decades have seen agencies and organizations gain access to vast volumes of data previously stored in government silos with real results. As AI technology has improved organizations’ ability to process volumes of information, many agencies have moved toward storing and presenting data in formats that are FAIR: findable, accessible, interoperable, and reusable. The FAIR principles emphasize machines’ ability to assimilate and use data—in other words, making information machine-actionable.10

Interoperability is an important FAIR concept, but discussions generally revolve around semantic interoperability—data exchanges, data formats, and data vocabularies—basically the structuring and codification of data. For instance, the Australian Department of Finance’s framework—a critical step toward transforming the nation’s digital records—focuses on semantic interoperability to allow humans and machines to clearly understand information’s context.11 A similar effort in India since 2015 through the Interoperability Framework for e-Governance (IFEG) has aimed to improve open data standards and constituent services.12

Human-machine communication is only one factor that impacts data interoperability among agencies, departments, and levels of government. Discussions should also include cultural and legal interoperability.

In 2020, the European Union refreshed its 2017 interoperability framework to include common principles, which cities and communities can use to deliver better constituent services. The European Interoperability Framework for Smart Cities and Communities project aimed to tap into the knowledge from other EU initiatives and projects,13 looking beyond technology to consider the human, social, and ethical aspects of large-scale data usage. The organization developed a governance model relevant at the local level, enlisted both public and private stakeholders, and helped to engage different communities. While the earlier framework made government agencies responsible for providing integrated public services, the new version includes the private sector, nonprofits, and community organizations.14

This seemingly minor framework change has massive implications, demanding a more mature cultural interoperability: understanding social, cultural, political, and ideological differences between individuals and organizations. It requires debates and discussions between different stakeholders to find ways to build consensus on interoperability challenges and solutions while operating in different legal and regulatory environments—particularly with regard to the tricky area of data-sharing.15

Building trust, transparency, and ethics

Data interoperability frameworks can lower technical challenges to data-sharing, but they may not necessarily illustrate the benefits to stakeholders. Agencies, companies, and other parties can be far more willing to participate if they see clear value and trust their would-be collaborators. Government can help build that trust, and encourage information sharing, by adopting a clearly articulated set of ethics to guide transparent sharing of how stakeholders’ data is used and what’s in it for them.

For more than a decade, the city of Boston’s Mayor’s Office of New Urban Mechanics (MONUM) has convened an ecosystem of technology vendors, start-ups, academia, and other government agencies to create technology solutions to benefit city residents. Since its launch in 2010, MONUM has successfully deployed a range of digital applications aimed at assisting citizens with street parking, flu shots, and more; labs continue to generate prototypes and solutions in housing, playgrounds, public spaces, and education.16

Transparency has been a key element in MONUM’s success. One project involves using cameras to collect data on street usage, to improve safety for pedestrians, bikers, and other traffic. But since such broad data collection raised obvious privacy concerns, the team focused on noninvasive sensor data, making sure to neither collect nor store visually identifiable images of faces and license plates.17 And to alleviate public wariness, the team placed signs with scannable QR codes at data collection spots, offering information on the project, an explanation of the eventual data use, and a site for people to share comments.18

Getting public buy-in for data-sharing can be challenging, but business leaders are often even more wary, citing low trust in data platforms and the inability to demonstrate business or social value. A World Economic Forum study in 34 cities globally found limited private-sector participation in citywide open-data platforms, due in part to a lack of confidence in government’s ability to address data misuse and prevent breaches.19

A lack of trust in the broader data-sharing ecosystem can slow down advances dependent on collaboration. For instance, the technology has long been in place to scale smart-kilometer or road-user charging solutions to urban areas, but data governance remains a stubbornly thorny issue for the transportation ecosystem. Vehicle manufacturers, government agencies, and other players have struggled to agree how best to protect drivers’ privacy when collecting data about where and how much they drive.20 Similar problems continue to hinder progress on data-sharing in a number of domains.

Brussels has aimed to solve the problem by building data governance and privacy into its SmartMove smart kilometer-charge program from the beginning. The city developed SmartMove to adhere to GDPR guidelines and with privacy-by-design principles in mind; the program uses a federated data system in which the city’s tax agency is made aware of the vehicle miles driven within a geographic zone only—not where or on which route. Only the SmartMove app user can see all of this information in one place. The city is live-testing the application with 1,500 residents and plans to scale it to the whole city in the next couple of years.21

Moving forward

  • Go beyond semantic interoperability: Enabling semantic interoperability—functioning data exchanges and common data formats—is foundational to improving data-sharing. But government leaders should also focus on building cultural and legal interoperability to enable collaboration between agencies, the private sector, and citizens.
  • Be laser-focused on data privacy and security: Constituents’ and businesses’ willingness to provide consent and share more data with the broader ecosystem will depend on their trust in the data ecosystem. But creating that trust is beyond the scope of any individual government leader. Rather, chief data officers, chief privacy officers, and chief information security officers, may need to blend their data and their ethical and security expertise to provide an environment that stakeholders can trust.
  • Develop a value proposition for the private sector: In creating plans for a data-sharing ecosystem generating public benefits, government leaders should showcase business value to encourage private-sector participation.
  • Identify where to get started: Government leaders should not boil the ocean but rather be more strategic in choosing the right policy area and solutions. They should proactively identify leading practices and success stories in different priority areas and test them further to show broader benefits on the ground.

My take

Ren Essene, chief data officer, US Consumer Financial Protection Bureau

A good data strategy is key to mission success

The US Consumer Financial Protection Bureau (CFPB) is a relatively new organization born in the digital age, giving us a unique advantage. While other public sector agencies may have legacy system issues or may have their data in silos across the agency, the CFPB has had a centralized approach to data since our inception. Data is at the heart of our mission and is in our DNA.

The CFPB’s mission is to implement and enforce federal consumer financial law and to ensure that markets for consumer financial products are fair, transparent, and competitive, a task that is impossible without data about how those markets function.

Therefore, from the beginning our charge was to make data available to regulators and the public to bring greater sunlight to market practices and improve competition and encourage innovation. To achieve that, our data strategy is built around five priorities:

  1. Increase the accessibility and volume of public data 
  2. Create a modern data platform by migrating to the cloud, managing costs, and enabling analytical capabilities
  3. Improve the quality of data and leverage modern tools to support research and analysis
  4. Increase staff’s awareness and understanding of CFPB data assets 
  5. Foster data literacy and upskill staff

We have built a series of campaigns, one for each priority, to drive the progress across a three year horizon. We have engaged our business and technical stakeholders throughout the process and have campaign leaders from multiple offices. We are focused on incremental wins and delivering value on mission priorities. By improving our data practices, we can help to create more fair, transparent, and competitive markets to ultimately improve the lives of consumers.    

Show more

Deloitte Center for Government Insights

The Deloitte Center for Government Insights shares inspiring stories of government innovation, looking at what’s behind the adoption of new technologies and management practices. We produce cutting-edge research that guides public officials without burying them in jargon and minutiae, crystalizing essential insights in an easy-to-absorb format. Through research, forums, and immersive workshops, our goal is to provide public officials, policy professionals, and members of the media with fresh insights that advance an understanding of what is possible in government transformation.

Learn More

  1. Adita Karkera, Data-fueled government: Breaking down silos with turbo-charged data, Deloitte Insights, March 24, 2022.

    View in Article
  2. Deloitte Cloud, A city sees what’s possible for its citizens and visitors with a cloud-based command center, 2019.

    View in Article
  3. Juergen Klenk et al., Fluid data dynamics: Generating greater public value from data, Deloitte Insights, March 4, 2021.

    View in Article
  4. 6Aika, “Results: Carbon Neutral Tourism,” April 20, 2022.

    View in Article
  5. Maxim Chantillon et al., “Final study report: Proposal for a European interoperability framework for smart cities and communities,” Deloitte and KU Leuven for the European Commission, July 2021.

    View in Article
  6. German Lopez, “Homeless in America: The homelessness crisis is getting worse,” New York Times, July 15, 2022; Kirsten McRae, “The global housing crisis: A crisis unlike any other,” Urbanet, October 25, 2022.

    View in Article
  7. Tiffany Fishman and, Disrupting housing insecurity and homelessness, Deloitte Insights, January 5, 2022.

    View in Article
  8. Emily Alpert Reyes, “A computer model predicts who will become homeless in L.A. Then these workers step in,” Los Angeles Times, June 12, 2022.

    View in Article
  9. National Alliance to End Homelessness, “State of Homelessness: 2022 edition,” accessed December 12, 2022.

    View in Article
  10. Klenk et al., Fluid data dynamics: Generating greater public value from data.

    View in Article
  11. Australian Government Department of Finance, “Australian Government Records Interoperability Framework,” July 2, 2021.

    View in Article

    Government of India Department of Electronics and Information Technology, “Interoperability Framework for eGovernance, version 1.0


    View in Article
  13. European Commission, “Proposal for a European Interoperability Framework for Smart Cities and Communities (EIF4SCC), accessed October 2, 2022.

    View in Article
  14. Ibid.

    View in Article
  15. Author interview with Deloitte Portugal team, September 9, 2022.

    View in Article
  16. City of Boston, “New Urban Mechanics,” accessed December 13, 2022.

    View in Article
  17. New Urban Mechanics, “Numina street sensors,” November 10, 2021.

    View in Article
  18. New Urban Mechanics, “Digital transparency in the public realm,” November 10, 2021.

    View in Article
  19. World Economic Forum and Deloitte, Governing smart cities: Policy benchmarks for ethical and responsible smart city development, July 2021.

    View in Article
  20. Tiffany Fishman et al., Transportation trends 2022–23: Making the most of a huge infusion of federal funds, Deloitte Insights, November 8, 2022.

    View in Article
  21. Author interview with Deloitte SmartMove team, August 20, 2022; SmartMove, “SmartMove sets Brussels on the right track!,” accessed December 13, 2022.

    View in Article

The authors would like to thank Apurba Ghosal from the Deloitte Center for Government Insights for driving the research and development of this trend and Meenakshi Venkateswaran for helping design the graphics of the article. They also thank William D. Eggers for his insights and thoughtful feedback.

Cover image by: Natalie Pfaff

Did you find this useful?

Thanks for your feedback

If you would like to help improve further, please complete a 3-minute survey