How data centers are evolving to shape the future of digital government

Bill Eggers: This is Government’s Future Frontiers, the podcast from Deloitte that asks questions today to help create tomorrow. I’m Bill Eggers, the executive director of Deloitte’s Center for Government Insights, and in this episode, we’re going to dig deep into the fascinating subject of data centers and AI: it’s what we’re calling “the invisible backbone of the digital age.” To unpack that, I sat down with Charbel Aoun, the Smart Spaces and local government leader at Nvidia.

Well, why don’t we start off by telling the audience a little bit about yourself?

Charbel Aoun: I [have been] working for Nvidia for the last nine years. I run [the] Smart Spaces and local government business[es] for EMEA. I’ve been into smart cities and digital transformation since 2006.

Eggers: Nvidia has talked about data centers evolving into “AI factories.” Can you tell us more about what that means, and the role they might play in sovereign AI, which we’re hearing so much about today?

Aoun: Well, in a traditional context, data centers were meant to house data and store data. Moving forward, in a world of generative AI, that’s not going to be good enough. Data centers are going to become, what we call, AI factories—the same principle where a factory takes raw materials and produces products. AI factories are going to be following the same path, where the raw materials are all types of data, and they’re going to manufacture intelligence, insights, value, and economic development—and that is the big transformation.

So, now, infrastructure and data centers are not just a compute; it’s a critical strategic infrastructure that ties directly to economic development and to foreign direct investment and to talent enablement and to the startup community.

Now, these factories combine the best of the best of AI, in terms of networking and accelerated compute, because a traditional data center used to be a general-purpose compute. But now, we need accelerated compute to be able to deal with all these large language models, digital twins, and large amounts of inferencing.

Now, in terms of sovereignty, which I think you were talking about, I think it’s very important to talk about sovereignty. It’s not about closing. It’s about the jurisdiction, the city, the nation’s ability to own and control and secure their AI infrastructure, their data, and their models, as well as their own talent—so made locally, dependent on local data, for the purpose of local.

Now, inside this, there are three components I want to talk about—a local compute power—your sensitive data never leaves your border. That’s very important. That’s sovereignty. The second one is control over model training and fine-tuning. So basically, here we want to ensure compliance with regulations and retain the cultural values of the community. This is a critical part of sovereignty. And the last one is about an ecosystem engine.

Because you want to enable your local university, your local talent, your local startups, your local enterprises, to produce and be part of this AI-factory journey. Now, what’s happening is that we are seeing cities taking the concept of the AI factory and layering on top different components, from engaging universities through educational programs for getting workforce out of the university, experts in technology. We’re taking inception programs where we support and nurture and help startups, so we can have a two-way directional exchange with the cities and jurisdictions to build their own community of startups—homegrown companies—and then bringing [in], basically, the cities’ departments and their priorities. Each city, each jurisdiction has certain priorities and [require to] put this data in a compliant way in this service—and that’s the motion that’s happening today.

Eggers: Right. And I think it’s so critical when you look at city workforces, which you mentioned about universities as a way of bringing in a lot of that tech talent.

My alma mater is the University of California, San Diego. And they actually just started a major for undergrads in artificial intelligence, and they have a data science institute, and they’re graduating hundreds, if not thousands, of students a year right now with deep experience now in [machine] learning and data science and AI. And that can be part of the future workforces of cities.

Now, you also mentioned around the compute power and how important that is. Now, we’re reading so much about the Blackwell chip. Could you tell the audience a little bit more about that chip?

Aoun: So when we talk about AI and the need for large compute capacity, that’s relevant to three main workloads: generative AI, which requires a massive amount of data. That’s how all these LLMs or agents that you see are being fine-tuned. It’s about large digital twins, and it is about inferencing at scale.

So, these three workloads are compute-intensive, or could be compute-intensive. And Nvidia is on a journey. As we always do, we innovate once or twice a year on improving and whatnot. And Blackwell is our latest and greatest. It’s the best of the best of what we have.

It’s the most efficient, accelerated-compute platform that we have; it incorporates all our latest software stack. It incorporates our fastest NVLink networking components.

Eggers: Now, as enterprises build ever-larger AI centers, how should we think about cost and sustainability, you know, between energy and carbon? We’re reading so much about how much energy use data centers are taking up. Could you talk a little bit more about that?

Aoun: So, when you think of such a technological asset as a strategic national critical infrastructure, you cannot but think about cost, sustainability, and risk. These are tightly linked, and you cannot treat any critical infrastructure strategically without thinking about this. So, on a cost perspective, we take pride in our accelerated compute, where it offers a significant advantage when you look at the total cost of ownership—when performance is measured per unit of useful work, not per server.

Now, what we’ve done here is we’ve taken large spaces where you had thousands of racks of CPUs, and we replaced them with a much smaller physical unit—with multi-x, tens and twenties and 50x and 100x performance and footprint on energy […]. So, when we measure it, we measure it on the output device. So that’s the cost. And on the total cost of ownership, it’s such an advantage to have accelerated compute. When you look at sustainability, a modern data center has to be designed around efficiency and reuse.

This is what is at the heart of Nvidia’s reference architecture. In the past, a certain task used to require hundreds and hundreds and hundreds of CPUs, general-purpose compute. Now we have brought a parallel processing, accelerated compute GPU [graphics processing unit], which replaces all this massive amount with a few, very few, a handful of [the] amount.

Now, what we are seeing also is automatically when you do this, the footprint is reduced on energy, on space. But we are seeing a phenomenon now that a lot of cities and regions and nations are using renewable energy, turning bio waste into a source of cooling. Now, when we look at the GPU energy consumption on a unit basis, you may think it consumes more than a traditional server. But when you look at what you are replacing in terms of quantity of old general-purpose compute, it’s significantly efficient and power-efficient. Now, when you talk about risk and resilience, it’s just like parallel accelerated compute is running mission-critical workload.

So resiliency and security are very essential to build up. So, basically, that’s where Nvidia has built the AI factory to remain in your sovereign space, under your sovereign control, to process your sovereign data.

Eggers: Why don’t we move on to looking at new operational models? So you’ve got abstraction, hybrid clouds, hyper scales, AI infrastructure management, and all kinds of new models that we’ve been hearing about. Which ones will become mainstream in the next three to five years?

Aoun: I believe that, in the future, AI won’t just run on infrastructure. AI will run the infrastructure. And we are talking about smart infrastructure because it’s more about data centers of the future should anticipate demand, manage their own resources, and optimize energy use in real time.

Now, Nvidia is the provider of a lot of technology that’s being used to simulate and optimize. And we are going to put our own technology in the use of our own data center, basically. So data centers of the future will operate as hybrid sovereign ecosystems. And it’s about blending local control with global innovation, with global agility, with global power.

We take the Nvidia reference architecture, and we make it a local sovereign asset. That is where the data center is moving in. So the real meaning of intelligent infrastructure is about new systems that think, adapt, and sustain themselves. So data center will become, in a way, cognitive.

And this means your workforce that runs your data center will move to focus on innovation and value creation rather than basic administration. So we take the administration tool, and we automate it. So, that is the philosophy and the thinking process.

Eggers: Well, that’s fascinating because we’ve heard so much about digital twins and cognitive in various areas. But actually, like seeing data centers themselves as cognitive and predictive is, you know, how far away are we from that?

Aoun: So, now, we’re talking about what we call physical AI a bit. So digital twin is a critical component because right now we are in a place where we can take the real physical-world “data” and incorporate it in a high-fidelity replica of the physical space: Being a factory, being a stadium, being a shopping mall, being a city, being a district, it doesn’t matter. It’s a scale issue. Now, when you do that, you have the ability to, in a safe environment, to try, play, destroy, move around, flood, burn, or deal with disaster. You can now simulate and optimize, and then when you reach a point where you are comfortable with the outcome being the optimal outcome, now you can go take it into the physical world and improve the physical world. How far is that? That’s happening around hundreds and hundreds and hundreds of factories and airports. Now it’s coming into cities. So, we have a couple of examples that are interesting you can look into—like the SNCF, the national rail of France. So, all the train stations of the SNCF are integrating. They’re using digital twin and real-time data from their stations, train stations, to improve operation, deliver significant sustainable targets, save energy, and so on, and so forth. At the same time, they’re using it to simulate different levels of footfall into the train station.

How would my asset work? How is the flow working? How can I improve it and improve the physical space to accommodate the safety and well-being and the experience of people? That’s becoming now a common theme in cities—Like the city of Gothenburg is actually flooding their cities as many times as they want in the virtual world and getting ready for the eventuality of this happening. We are better prepared. So this is becoming more and more a reality.

Eggers: Well, it’s I think really exciting. Thank you so much for your time.

Aoun: Thank you very much for having me.

Eggers: Fascinating conversation and very, very helpful, I think, about the future.

Aoun: Thank you. Appreciate it.

Thanks so much to Charbel for joining me in this episode of Government’s Future Frontiers from Deloitte.

All previous episodes are available now from your favorite podcast provider, and if you haven’t already, be sure to follow us so you’ll get the next episode as soon as it’s available.

This podcast is produced by Deloitte. The views and opinions expressed by podcast speakers and guests are solely their own and do not reflect the opinions of Deloitte. This podcast provides general information only and is not intended to constitute advice or services of any kind. For additional information about Deloitte, go to Deloitte.com/about.