#105: Can we trust something we don’t fully understand?

– Intro music of The Green Room begins followed by an introduction from our host. –

Steph Dobbs (Host)

Trust is both delicate and hard earned. It can also be the difference between experimentation and scale. As organisations race to adopt AI, the challenge isn't just about capability, it's about confidence. Because the companies that benefit most from AI, may not be the ones with the most advanced technology, but those who create enough trust for people to use it, at scale.

Today we're asking the big question: Can we trust something we don't fully understand?

– The podcast transitions into showcasing highlight clips from the upcoming episode. –

Simon McDougall (Guest)

AI has become distributed and widely used very quickly. We haven't had time to build all of these components up and we're going to have to kind of build some of them as we go along.

Avtar Benning (Guest)

I would argue trust right now is probably more important than more intelligent models.

Simon McDougall (Guest)

The good organisations aren't just doing this stuff for the sake of it.

Avtar Benning (Guest)

If you get the trust angle right, it really becomes a transformation enabler.

– The podcast music ends and it transitions into the main episode, starting with our hosts introduction. –

Steph Dobbs (Host)

Hello and welcome to The Green Room by Deloitte. I'm Steph Dobbs and I'm joined today by my co-host, and another new face to the team, Andrea Boxall. How are you feeling, Andrea?

Andrea Boxall (Host)

I am absolutely thrilled. You could even say I'm Deloitte-d!

– Laughter from hosts and guests. –

Steph Dobbs (Host)

Oh, gosh!

Andrea Boxall (Host)

Sorry, had to get that in.

Steph Dobbs (Host)

Good start to the episode. So for those of you who are watching the episode on YouTube or on Spotify, you may actually notice that our ‘Green Room’ has turned a shade of pink this week. But don't worry, the screen isn't broken. We'll actually be back in our normal studio soon.

So as we continue our series looking into the impact AI could have on businesses, individuals and society, there is one topic which has come up in every episode. Trust. For the last couple of years, it's felt like capability has been the main question asked about AI. But as we move into a new phase of adoption, the question of trust and confidence in the systems becomes even more important.

Trust can be seen as a brake on innovation. But is it actually what allows AI to scale past experimentation and into transformation? For organisations, trust is central to any AI adoption, and it's a non-negotiable that needs to be built in from the start, not added later. So as the pace of change shows no sign of slowing down, how can we keep up with technology and build confidence at scale?

Today we're answering the big question: Can we trust what we don't fully understand?

Andrea Boxall (Host)

Today we're joined by Simon McDougall, who's the Chief Strategist for Privacy and AI at ZoomInfo. And Avtar Benning, who's a director in Deloitte's trustworthy AI offering, it is a pleasure to have you on the sofa.

Avtar Benning (Guest)

Thank you.

Simon McDougall (Guest)

It's a pleasure to be here.

Andrea Boxall (Host)

So I wanted to begin with a tension point we often hear. So it's this idea that because individuals are adopting AI at pace that we haven't seen before, it's putting pressure on businesses to move quicker. So is there any truth in that thought and what are the differences in what individuals and organisations have to think about when it comes to AI adoption?

Simon McDougall (Guest)

I think that's very true. I think that one of the key stats that I've picked up in the last year is, is that it took ChatGPT just a couple of months to get to 100 million users, when it took TikTok well over half a year to get there, it took Instagram well over two years to get there.

The adoption of this technology by individuals is faster than anything we've had before. And that, I think, is where often you see a bit of a trust gap because people are getting hold of this technology and using it and then worrying about trust later. And that's a big challenge for businesses because businesses have different interests and different concerns and different obligations. And so they're going to have to try to keep up with that.

Andrea Boxall (Host)

Yeah, totally, I know that... I think once you understand what it can do as well, you kind of find yourself coming back to it and asking certain questions. And that trust, as you say, doesn't tend to be thought about. How are the questions that companies need to ask about AI changing as it develops, and as we get more used to it being part of our lives?

Simon McDougall (Guest)

In terms of the questions that companies are asking - a lot of those are well-worn. So before I was at ZoomInfo, I was a regulator. I was Deputy Commissioner at the Information Commissioner's Office, and we were doing work on AI going back to 2018, 2019. And then it wasn't called, you know… AI is machine learning, it was synonymous with machine learning. And this whole wave of AI now wasn't really envisaged. But if you look at the guidance we produced at the time, a lot of the risks were the same. We discussed explainability a lot. We discussed transparency a lot and how AI could be used to make decisions about people, hiring decisions, decisions on credit or immigration. So those things are well-worn risks.

And then you have other risks on top of that, not least just how widely spread this technology is now. But also there are other unique risks that weren’t there before. I mean, deepfakes, was not something we delved very heavily into, for instance. But I think one of the challenges for organisations is saying, “Well, what are the actual risks represented by this new technology? How do they affect me, and what is the risk of harm? The risk of harm to our consumers, to society, to our staff, to our customers - if they're corporate customers - and how do we manage those risks of harm?”

Steph Dobbs (Host)

Definitely. And I think that's the thing because you often see companies obviously, sort of promoting the power of an AI tool, don't you? And sort of all the capabilities that it's able to do, but actually that trust and transparency point is something that isn't often widely promoted or it hasn't necessarily been until now. I suppose - Avtar let’s come to you first - is AI advancing faster than public confidence in it is? Is that why?

Avtar Benning (Guest)

Yeah, absolutely. Actually, I would argue trust right now - where we are in time - is probably more important than more intelligent models. I think trust is becoming the real barrier from going from simple POCs to full scale, enterprise-wide adoption.

And I see this with clients a lot. There's no shortage of examples of going to a client, seeing a very long list of great and wonderful, exciting things they would like to do, but then prioritising that and then figuring out where are you going to get the most ROI? Where are they more safe and reliable? I think all of these questions come a bit later, and then there is a bit of a bottleneck going from that to full enterprise roll out.

Simon McDougall (Guest)

If you look at how innovation works, you're going all the way from innovation being an idea that that some people have - whether it's in a lab, if it's in a garage, in a university - all the way through to actually being fully scaled and actually distributed. You need trust to get from one to the other. So very often the people who are building the cool things underestimate how hard it's going to be to get everybody else to adopt those cool things.

Steph Dobbs (Host)

Yeah, I know this sounds like a really fundamental question, but why is that trust actually so critical?

Simon McDougall (Guest)

Well, I think that's very context specific because when we're saying trust, the issue with trust is sometimes you say it, and everyone just nods.

Avtar Benning (Guest)

Yeah.

Simon McDougall (Guest)

And so yes, trust is very important. But are you talking about trust among a consumer group? Are you talking about trust among a group of citizens? Are your customers a bunch of corporates? Is it B2B? Is it societal trust? There’re all these different things flowing through and the importance of trust I think is very context specific there.

In the end, what does unify all of those cases is that people, or corporates, or whatever aren't going to fully engage with your solution if they don't trust it. They might play with it, they might experiment with it, but they're not going to actually invest and use it on a day-to-day basis. And that becomes more and more critical the more sensitive the usage.

Steph Dobbs (Host)

I suppose, Avtar, in your world as well, are there times where trust actually matters more than it does in other decisions? Are there times where the trust actually matters even more than the AI system does in some ways?

Avtar Benning (Guest)

Yeah, absolutely. I think there are certain scenarios and I like to sort of think of it in the classifications in the way the EU AI Act classifies in high, medium, low risk categories. And when you're in the sort of high-risk bucket where there is repercussions, or material repercussions, of it going wrong, I think trust outweighs, say accuracy by some level of percentage points.

I've worked across a range of different risk categories and actually a lot of them are falling into the high-risk categories. And so often the conversation is around: “we're excited about this, but we can't get this wrong.” And so, indeed, very much so, I think trust is quite rightly at the top of everyone's list.

Andrea Boxall (Host)

And Avtar, I’m interested to know, what do you think is the kind of one thing that's holding businesses back from having trust in AI systems and capabilities?

Avtar Benning (Guest)

I think the challenges, you know, with these AI models - some of these closed black box models - there's a misconception there that you need to… well, firstly, you can't understand each and every component of those systems. But you don't need to understand each and every individual component of these models to gain trust. You need to be able to understand certain components of these models but enough confidence around reliability, accuracy. So there are a lot of things you could do around the peripherals of these models to gain trust.

It's the classic aeroplane analogy. You don't understand how a jet engine works, but you're confident enough to get on an aeroplane. And that's because there's a lot of testing that's been performed. There's a lot of controls and risks around it that are well understood. So I think it comes back down to not necessarily needing to know all the individual details, but enough confidence that it will perform reliably that gives you trust.

Simon McDougall (Guest)

And I think one of the key things there is that this is not a new conversation. We have had trust conversations for ever and ever.

One of my favourite stories around this is going back to 1891 - I think the US President’s name was Benjamin Harrison, I'm doing it from memory so fingers crossed on that one - they electrified the White House. They put lights into the White House, very exciting at the time. Revolutionary. The President and the First Lady refused to touch the light switches because they were worried they were going to get electrocuted. To the point where there was somebody employed at the White House, one of their jobs was to turn the switches on and off on behalf of the president.

And now, obviously, we have all, probably a number of times today used electric switches and turned them on and off. And we don't worry about that. Now, why is that? That is not because we've all become electricians and we haven't tested these particular circuits very well. But there are layers of assurance and testing and common use and life experience - all these things come together to build trust. So we use it unthinkingly. And if somebody does get electrocuted by a light switch, that's something you tell your friends about, and it becomes a newspaper story, and everyone wonders what happened, and there's an investigation into it. So there's all these components - and we'll talk about bits and pieces, I'm sure, as we go through - that come together.

Again, going back to my first point, one of the challenges we have here is that compared to other technologies, AI has become distributed and widely used very quickly, there’s this post-ChatGPT AI in particular. So we haven't had time to build all of these components up. And we're going to have to build some of them as we go along. But we need all of those components before we really get to a stage of saying we are going to trust AI.

Avtar Benning (Guest)

And the interesting thing is when you then go and speak to clients about this problem, it’s exactly what you said [Avtar points to Simon]. Some of these risks are well understood, but many of them aren't. So there's a big educational challenge. And then they're quite often faced with the dilemma that they don't necessarily always have the resources or even the technology to be able to perform the testing, or assurance, or whatever is required to give them trust. The technology is moving faster than they can keep up. It's a very challenging time, I think.

Simon McDougall (Guest)

And one of the other challenges there - I think for a lot of businesses - is that on the other side of the coin, there is real commercial pressure to innovate in AI. And this is not really a full trust point, but it's the other side of that, it’s what's pushing these businesses. The boards are challenging CEOs, CEOs are challenging the business saying, “well our peers are doing X, Y and Z, and I've read this thing over here, and we should be adopting AI here. How do we go about that quickly?” So there is a pressure to move fast. And I think a lot of risk management professionals, whether it's chief risk officers, or general counsel, or people in corporates are caught between a business that's clamouring for rapid deployment of these solutions and a low trust environment over here. And they've got to kind of balance those two things out.

Andrea Boxall (Host)

There’s a real tension in those two, isn't there? Real tension.

Steph Dobbs (Host)

Bringing us very well onto the point around what actually creates trust itself. So, you know, we've got the example of the light switch - which I'll be thinking about next time I turn a light on, thank you for that. And as you say, there's the example around planes, the fact that we trust planes despite the fact that very few of us probably know how they actually work. So what actually makes people trust a system? Does it have to be a positive experience that drives that? Simon, let's start with you.

Simon McDougall (Guest)

So the good news is that there's a whole world of research and academia about this. And if you want to go down a rabbit hole, then you can go down a trust rabbit hole. Again, the trust conversation itself is not new. AI is the new thing and it's raising trust questions. And one of the traditional models talks about gaining trust through three different values you have to display. And that's: ability, benevolence and integrity.

And so when they say ability, it’s, do you believe that this, that the thing you’re going to trust, the organisation or the person, you know, actually has the ability to do what they say they're going to do, you know, are they competent in this? Benevolence is, are their interests, the same as mine? So, it's an emotional thing. Are our values aligned? Or are they always going to be a bit sneaky about things? Integrity is, are they actually going to follow through? Do they have principles? Will they deliver on something that they say they're going to do? Benevolence is being well meaning, integrity is saying, “Yeah, we will always deliver.”

And to have trust in something, you have to build up a confidence in those three values, which are all slightly different. So that's one way to think about it. But there's a lot of other different ways of going around it. In the end, if there's just one strapline, it’s people need to just believe that you're going to do what you say you're going to do. I think that that is kind of the baseline for these things.

Avtar Benning (Guest)

Yeah I’d agree. I think almost rolling that up into there needs to be some kind of evidence, right? You know, I come at this from a sort of a bit more of a technical testing angle and my conversations very quickly always boil down to: “have you done this test and what is the outcome of this?” And that's great. But exactly to your point, there’s a much bigger kind of trustworthy AI framework out there which spans across many different angles. And I think all of that is what forms your evidence base.

Back to my earlier point. It is difficult to explain each component of the neural network models underneath, and that just won't happen. If AI providers did more in the transparency side of things, I think that will help the pace of this space. But they also have a motive to get these models out there very quickly. And so it's on us to make sure we come in and assure it.

Simon McDougall (Guest)

I think that supply chain point is super important.

Avtar Benning (Guest)

Yeah.

Simon McDougall (Guest)

It's got better along the way. I think in that immediate post-ChatGPT 3.5 rush, there was this sense I think among the hyperscalers, the model providers that, well, you know, just take it or leave it. And then over time, model cards have got better. And explanations of how the models work have got better, but they’re still by no means perfect.

And very often what you see corporates doing, corporates and governments, and all organisations is having to buy solutions that they have a partial understanding of. And going back to that point around, you know, around ability, benevolence, integrity - if you're buying solutions you don’t fully understand and then you're talking to your customer base and they're asking for detailed explanations. You're caught in the middle. And I think the worst thing you can do then is actually bluff your way through this. You've got to find a way to square that off and be quite upfront about what you know and what you don't know.

Andrea Boxall (Host)

And I think we've talked on previous podcasts about how our ability to think critically as humans is so important when it comes to AI, and being able to ask those questions of what it produces and how we use it. And I know there'll be people out there that would love to know, that would want to see behind, you know, and see how it works and understand and kind of have those three things covered. But then there's also people like me, who like to use it and see what happens and kind of maybe have a positive experience of it. So then we return to it and we kind of see good results, and it provides us with what we need. Is there a link between that kind of positive experience and our ability to trust it?

Simon McDougall (Guest)

I think again, context is everything here. And if I go back again to my days as a regulator, we did work with Manchester University in 2018, 2019 on explainability and trust, and we were running these large workshops called Citizens Juries with lots of people who are representative of the UK, so not specialists, people who are just representative of the UK. And at the same level of understanding.

And what we found was that people were willing to trust AI in certain contexts much more than others. So, for instance, if it was a diagnostic tool which was approved and used by the NHS, where there was already a huge level of trust and is being used for their benefit, people would say, “fine, we're going to go with that because we can see the benefit and we trust the NHS.” If it was a hiring decision by a faceless corporation, they were less willing.

I think actually people are much smarter in this area than we give them credit for. We assume people are just blithely going in and trying things just for fun. I think people are making smart decisions. If it's a really cool toy and you’re not sharing much personal data and it's just on your phone, it's doing some cool things, well then great, you know, if it makes your dog look like a cat... Fine! But if it's giving you a guess on your medical diagnosis with some symptoms, then you're going to go, “Okay well, who's telling me this and why are they telling me this, and why are they directing me towards this particular supplier for a solution?”

So I think people make lots of these little decisions almost intuitively. And I think normally they're pretty smart about it. So I think yeah, I think if something gives you a bit of joy and it's fairly low risk, you're going to go and do it.

Avtar Benning (Guest)

I think there's another interesting angle to this as well, which I've encountered a couple of times. This kind of overreliance on trust almost. It was a very specific example where, a client had developed an AI tool in their financial crime space, it was doing really well, exceptionally well, to the point where its accuracy was far greater than what was previously done by humans.

And over time, they trusted it so much that eventually people left the team and skill set retention was an issue. And fast forward a couple of years later when it started deteriorating in performance, they didn't have the skills around that team to fix the problem. And it was all a bit of a mess. And so, you know, I think there's also a bit of a danger on what happens when you trust it too much and you become over reliant on it. So it is threading that needle a bit carefully as well.

Steph Dobbs (Host)

Definitely, I suppose thinking about applications of AI within firms themselves. So coming back to that workforce point in particular, how important is it to have almost like an audit trail of understanding, you know, particularly if you’re talking about something like agentic workforce, you know, how AI is actually being worked, where it's being used, why it's being used, etc... How important is it to have an understanding built in of what the agentic workforce is actually up to and how you understand it?

Avtar Benning (Guest)

Yeah, I think the agentic problem sort of extrapolates the whole problem by a factor of ten or more. You know, we’re sort of moving into this task orientated system to now a goal orientated system where you have multiple agents all autonomously having the power to make decisions, read/write access tools. And so in that world, I've experienced many firms expressing nervousness about letting these agents go loose in their systems without the necessary risks and controls.

But I think the, you know, part of the answer to that problem is, as you say, the kind of auditability, the kind of traceability of these systems. I think what you really need is a kind of real time monitoring system in place that enables you to pull out the kind of telemetry of all of the interactions in real time, and having in place the right guardrails, risks and controls. That way you're able to at least monitor these actions.

And to the extent possible, I think safety by design should always be in the first principles as well. I think when you're setting up an agentic system, there may be certain agents that you restrict access to or you make them more deterministic by the way you construct them. So I think there are definitely ways to make agentic systems less sort of adventurous almost, and a bit more deterministic. But to your original question, yeah, I think, that traceability, auditability of it is super important. Otherwise you don't really know what's going on.

Simon McDougall (Guest)

And I think one of the challenges there is that these new technologies, whether we’re talking about AI, or we’re talking about LLMs, or agentic, or we move into world models, whatever. They do and they will fail in different ways to the way humans fail. And one of the challenges I've seen again and again is organisations try and apply risk models, which are based on deterministic technology and humans, to AI. And it's going to fail in different ways. And so if you're not actually thinking through how things could go wrong and just using old world controls, then you're going to get caught out along the way. So you've got to be thinking through how this works. And Avtar, to your point, I think with agentic where you could have like a sequence of decisions being made without any human supervision and an outcome in the real world, I think we're going to see, sadly, some real interesting but maybe messy examples of failure, which will seem very strange to us because we’re just used to kind of like the old world of failures. These are going to be different control failures.

– The podcast transitions into an episode break for a Social Impact message from our host, with background music. –

Andrea Boxall (Host)

Looking for a way to make a difference while you shop? Well, with Scope you can. Scope is one of the national charities Deloitte supports as a part of our Social Impact programme. Visiting any of their retail stores or online shop helps to support Scope’s work to create an equal future for disabled people. Every pre-loved item purchased, or donated, funds vital services and helps keep millions of items out of landfills. It's good for people and it's good for the planet. Just visit scope.org.uk to find out more.

– The podcast transitions back to the main episode as the music ends. –

Steph Dobbs (Host)

If we maybe move on to talk about how trust itself is actually built, leadership is obviously such an important thing here. So who actually should own trust in an organisation? You know, is this board level who needs to take ownership?

Simon McDougall (Guest)

I can tell you who it shouldn't be by themselves, and it shouldn't be the chief technology officer by themselves. It shouldn't be the general council by themselves. It shouldn't even be the chief risk officer by themselves. What we're talking about here is general purpose technologies. And we're talking about often, as we’ve said, rapid adoption of those technologies across the enterprise. So it has to be a cross-functional play.

I think what's more important than whether it sits with any particular individual is having clear allocation of responsibilities across the board. Personally, I am more in favour of trying to actually allocate responsibility across existing roles than creating brand new AI risk individuals because in the end, this is just another technology or another set of technologies. It's very exciting. It's fast moving, but the risks to consumers are going to be the same kind of risks you've always had.

You know, if it's a bank, it's going to be unfair credit decisions, for instance. If it’s a social media firm, it might be about protecting children. It's just another iteration technology. So I'd be challenging all the usual risk management functions to say, in this new world, how are you managing these new risks?

Avtar Benning (Guest)

AI is quite pervasive in an organisation, it touches a lot of different areas of a firm. It's not just constrained to one area. So, that friction is proving to be challenging to get an aligned ownership. But I completely agree. I think there's some shared responsibility there across all the different areas.

Andrea Boxall (Host)

I'd love to hear from both your experiences about what you've been seeing in organisations that are getting it right, if you've got any examples?

Simon McDougall (Guest)

So I'll start with a point that I wanted to cover - and this come in here I think, which is around use of standards and frameworks and those kinds of areas. I think my heart slightly sinks - even though I'm a regulatory nerd, I should love these things - but my heart slightly sinks sometimes when we get on talking around these areas. We're moving to a more mature world where use of standards and frameworks is starting to become appropriate for AI.

The big international standard is ISO 42001. That seems to be being adopted by a lot of organisations now and has come on leaps and bounds in adoption in the last couple of years. There's also the NIST AI Risk Management Framework, which touches on some of these points we already spoke about building trust and actually discusses trust in it. So that's also helpful.

And there's many other things out there, as I say, those are the two main frameworks out there - so I wouldn't recommend one or the other in particular. I think what I've seen among the better managed firms is they are looking at those frameworks and either taking a best of both – they are quite different ways of doing it - or going down one road or going somewhere else. But they are now saying, “okay, how do we actually have something which encompasses a level of maturity around how we're using AI in the organisation?”

Avtar Benning (Guest)

I think the firms that are doing it well, definitely some common characteristics there. Safety by design, not thinking about trust at the end, but at the offset whilst they're initiating and producing their use cases. If you get the trust angle right, it really becomes a kind of transformation enabler rather than the thing that slows you down. If you've got good governance frameworks, you've got good testing, you've got good risk and controls, actually helps you go quicker because you move through the process quicker. If you don't have that, you just end up sort of a bit of a bottleneck at the end.

But I think again - sort of to your question - I think also taking a risk based approach has been I think, a really good way of putting in proportionate risks and controls because you don't want to be spending the same length of time testing a PowerPoint AI tool versus something that has medical implications. You know, I think having a risk-based approach, enables you to allocate your time more effectively.

And I think good governance, I think that - you've mentioned it a couple of times - but I think it's a no brainer. If you've got good governance frameworks in place, risks, policies, roles and responsibilities with authority to make decisions, I think, again, that will help you move through the process quicker. And so some of the firms that I've seen doing exceptionally well and have use cases in production, I think they've ticked most of those boxes.

Simon McDougall (Guest)

And I think one of the key things is joining up the good practice, which is obviously a good thing to do. It's often the ethical thing to do. We all like doing these things. Tying this up with the commercial realities of what you're doing. ZoomInfo is a B2B company, we focus on enterprise customers. They have high expectations around what we do, and we use things like certification to actually articulate that we do this well. But the good organisations aren't just doing this stuff for the sake of it – worthy though that that is - they can show how having good governance in place and good controls, and maybe one of these frameworks, is building trust with their customer base and enabling them to sell more stuff, whatever the stuff is.

Andrea Boxall (Host)

So we've talked about kind of ways of building trust and we've mentioned assurance. So I wondered if you could just talk a little bit, maybe either of you take this one about what is AI assurance?

Simon McDougall (Guest)

I'll go first because I think it's actually a really interesting question for the whole of the UK, because AI assurance is something that the UK government wants to be a leader in and for good reason. We have a very vibrant market in the UK around AI assurance. If there's anyone who wants to go down the rabbit hole, there was a great UK government report on this a couple of years ago, sizing the market and looking at how it works.

I think the key thing to say is that AI assurance is not one technique. AI assurance is the whole suite of stuff we can use to get confidence and ultimately to build trust in AI. So it covers technical things such as model audits. It covers standards and certifications. It covers looking at the overall process and the lifecycle. And with all these areas, you're looking to kind of build up this patchwork of areas of assurance, to finally build some trust.

Avtar Benning (Guest)

I think that's a great summary. And I think just building on that, the UK and R&D are doing a lot in this space and you've got DSIT and NPL and they're all working towards some common understanding of the definition of AI assurance. But also to some extent some protocols, guidelines around this.

But I think it's probably one of the key answers of building trust. And assurance in the traditional sense many may interpret as assuring to some form of a standard, but I think we all know that there are some standards out there, NIST, ISO, but it's not uncommon to go to a client and they ask, “I'm not really sure how to test for bias or hallucinations”. And I think that has resulted in a bit of a sort of wild old west scenario where no one's quite sure what to do.

But AI assurance is definitely a big piece of that jigsaw, which is making sure you've got a good governance framework in place, but also good testing protocols in place. Whether that extends from accuracy, bias, fairness, explainability. You know, we've got a good sort of trustworthy framework which we align our principles to, but I think if you work your way around the principles of what makes trustworthy AI and assure it towards that, I think you then build an ecosystem that is trustworthy. So I think AI assurance there is super important to building that trust in an organisation.

Steph Dobbs (Host)

And I suppose it's how do you measure that public confidence piece? So, as you say, there are obviously standards, and let's take another example from another industry - food hygiene for example. There are standardised inspections which have gradings, which are clearly displayed, which we can understand even if we're not specialists. How do you measure public confidence, though, when it comes to AI? That's trickier, isn't it?

Simon McDougall (Guest)

It is. We've got a long way to go. You've got things like food safety and airlines as these posterchilds elsewhere for how this should all work in terms of trust. And when you have that kind of environment, it's taken decades to get there. And whenever anything goes wrong, you know, food poisoning or heaven forbid, you know, a kind of plane accident then that is headline news and often takes a long time to unravel.

And they dig deep to find those root causes and they improve. The maturity of what we have over here is nothing near that. I hate to say it, and I very much hope that it's not organisations that any of us are working with, but there will be many bumps in the road before you get to that level of maturity and trust with AI and technology. Without being doomsayer about it - but we should be realistic - right now, trust in AI, at least among the general public, but the same among some corporates as well is plummeting.

As we record this, a few weeks ago, Eric Schmidt was giving a commencement speech in the US and was booed when he mentioned AI. If we walked out of the studio and ask people about AI in the street, they’d say, “I'm worried of what it is doing to my kids and I'm worried about whether my kids or indeed I will have a job in a few years time.” They're not going to say, I'm really excited about AI.

So we’ve got some headwinds to work with before we get to this kind of high trust environment. We will get there because, in the end, it's the only way any product or any technology succeeds. So we'll get there through some bumps in the road, but in the end, we'll get to a mature, high trust environment where we have regulators and standards and certifications and people just do it without say it without asking questions around it. But we're nowhere near that right now.

Steph Dobbs (Host)

So actually, as we start to come to a close, let's maybe sort of round off by starting to think about maybe some takeaway advice for some of our listeners. So Avtar, let’s come to you first, what is the first step that firms actually need to take when they're ensuring that they're embedding trust in AI in order to be able to scale with confidence?

Avtar Benning (Guest)

There are many things, but one of the first things firms should try and do, is get an AI inventory in place that also classifies the use cases and risk categories. So if you've got a sense of where the entire organisation and you have a sense of high, medium, low – or whatever the categorisation is - you're then able to start putting in the appropriate set of proportional controls, risks and the subsequent actions. But I think that's one of the first steps, and you would be surprised at how few firms out there still don't have a sense of where AI is being used in the firm. I think only once you've once you've got that, then you can start thinking about the next steps.

Simon McDougall (Guest)

And that I think is a very fast evolving discipline. On the one hand, you have shadow AI - which we haven't touched on too much, but I'm sure is well known to the listeners. Going back to the very first point you made Andrea, are the staff just using their phones because it's a better solution than what is in the firm? Then there's quite a good ecosystem now of discovery tools to actually identify when AI is being used in the organisation. But also, I've heard of corporates getting some quick wins just by looking at corporate credit card spend. You'd be amazed how often, especially if the centre is being a bit slow, your business will just go and use their corporate credit cards to get a subscription to a new model, and suddenly you've got a whole new service you didn't know about. So running through these different ways of discovering where the tech is being used is helpful.

Andrea Boxall (Host)

So I think I'm really struck by, I think the two takeaways for me are around the benevolence point and the bumps in the road - not to kind of centre on the negative thing. But I think, from what you have said, we are needing to build trust as organisations in AI, but we also have to live with the fact that it won't always be right and there will be bumps, and we have to live with that tension. But we have to maybe trust in the kind of benevolence factor that actually our intentions and what we want to use it for align with, you know, we’re all on the same page with that. I think I'm really struck by what you said there.

Simon McDougall (Guest)

Honesty and transparency here is really key. I completely agree, and I think very often - and this might be me being optimistic - but if you are honest and transparent around what you're doing, if you're clear about what the risks are and you are clear you're striving to manage them, then whether it's consumers or whether it's other businesses that you're working with, they'll often accept a few bumps in the road within reason, if they know that you are striving to manage it. Conversely, if you sit there and you bluff your way through, then the first time you get caught out, you've lost all trust.

Steph Dobbs (Host)

So let's round everything up then and come back to the big question: Can we trust something we don't fully understand?

Simon McDougall (Guest)

You can, if you trust everything else around it. So in terms of what's building this trust - we're just speaking about the ability, benevolence, integrity points - is it whether you trust the institution that is using the AI? Is it you're trusting their track record? Are you trusting the fact that they've explained things to you and they're transparent? Is it because everyone else is using it and you trust some of the people who are using it? You can do all these things, which means you don't need to understand the model - in the same way you don't understand the Dreamliner flying on or the entire supply chain for the burgers you’re buying in the supermarket - but you need to trust something if you're going to trust the thing.

Steph Dobbs (Host)

Ok, Avtar?

Avtar Benning (Guest)

Yeah, I’d agree with that. I mean, if you can trust another human, you can trust an AI system. I think if you've got the right governance in place, if you can put the right risk and controls in place, solid testing, enough humans involved where there needs to be, you've got an ecosystem where you're building an evidence base that enables you to understand the system more. And if you can understand the system more, you can then make informed decisions on how to use it safely. And sometimes the answer may be it's not appropriate in this scenario, but other cases it may be. And I think if you do that, you can take a risk-based approach to it.

Steph Dobbs (Host)

Well, both thank you so much for joining us in The Green Room that is pink today - you can trust that it's normally green - but thank you so much for joining us.

Simon McDougall (Guest)

Thank you.

Avtar Benning (Guest)

Thank you.

-- Outro music of The Green Room begins followed by our host talking --

Steph Dobbs (Host)

Thanks for listening to this episode of The Green Room by Deloitte. We release a new episode every other Tuesday with another big question. So don't forget to hit follow or subscribe to this podcast. Wherever you're listening or watching and make sure your notifications are on. So that way you'll be alerted whenever a new episode drops. This podcast is produced by our very own pod squad. Original music by Ali Barrett.

-- The podcast music fades up and the episode ends --