WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:0

2ae6a700-8772-484d-9619-1101db19f08e-0
00:00.040 --> 00:02.800
Allow allow me to introduce the
two speakers.

adb7c5b4-90b2-4788-a7bc-5c8a1a202eb4-0
00:03.280 --> 00:07.270
We have Robbie Yalamanchili and
he is the Associate Vice

adb7c5b4-90b2-4788-a7bc-5c8a1a202eb4-1
00:07.270 --> 00:10.840
President of Digital Quality and
Labs at Deloitte.

2d868497-636b-44d3-bb76-34cec54d7072-0
00:11.120 --> 00:15.188
And we also have Matt Humphries
and he's also seen a principal

2d868497-636b-44d3-bb76-34cec54d7072-1
00:15.188 --> 00:16.480
at Deloitte as well.

0a144fb7-b5eb-4210-974e-8c365adb6a5d-0
00:19.760 --> 00:20.680
The mic is yours.

29b727e7-cce6-44c3-b449-31774fc8dc8d-0
00:33.600 --> 00:34.560
Thanks for the introduction.

4383ae6d-3990-4ba2-a777-d0573ff6dab2-0
00:34.640 --> 00:36.400
I'm Raviola Munchley at Merck.

a3955608-b995-4b09-99ce-a5759914a91e-0
00:37.600 --> 00:43.040
So today, I think we're still
trying to work out the video.

01b818ef-0977-40c8-b71a-c01699c708df-0
00:43.760 --> 00:45.880
OK, sounds good.

f4d5315f-2f1d-42cd-9672-7d8df4e988da-0
00:46.160 --> 00:47.360
So it looks like we figured that
out.

3345aa89-5aa3-4a72-996e-c8883685b6d5-0
00:48.040 --> 00:52.266
So we're going to cover a few
things about deviation

3345aa89-5aa3-4a72-996e-c8883685b6d5-1
00:52.266 --> 00:57.210
management and how we look at AI
in specific when it comes to

3345aa89-5aa3-4a72-996e-c8883685b6d5-2
00:57.210 --> 01:00.480
innovation in quality and
manufacturing.

86591c8f-09f9-4add-af61-15802bd3c0f6-0
01:06.560 --> 01:12.568
So Merck has rolled out a new
digital strategy or or relaunch

86591c8f-09f9-4add-af61-15802bd3c0f6-1
01:12.568 --> 01:15.960
of a digital strategy around
2022.

0a93d286-5764-4037-ab5f-4d41d0c21824-0
01:16.400 --> 01:20.286
And when we did that, we were
very intentional about using AI

0a93d286-5764-4037-ab5f-4d41d0c21824-1
01:20.286 --> 01:24.298
and, and, and, and, and thinking
the next two years, five years

0a93d286-5764-4037-ab5f-4d41d0c21824-2
01:24.298 --> 01:27.996
and 10 years and how AI will
play a big role in our system

0a93d286-5764-4037-ab5f-4d41d0c21824-3
01:27.996 --> 01:30.880
strategy and, and our digital
transformation.

f0eb4cc3-7852-4219-8ca8-17a64f1c8147-0
01:31.640 --> 01:36.150
So what one of the key pillar
for us when we did this was all

f0eb4cc3-7852-4219-8ca8-17a64f1c8147-1
01:36.150 --> 01:40.733
the components of your strategy
should really fit together for

f0eb4cc3-7852-4219-8ca8-17a64f1c8147-2
01:40.733 --> 01:44.080
you to be able to build your
models at scale.

4732e159-5674-4166-9929-5732be0737cc-0
01:44.520 --> 01:45.880
What am I mean by that?

d6103593-f950-4d52-b50d-412043b282c8-0
01:46.560 --> 01:50.250
All your digital strategy
starting from picking your

d6103593-f950-4d52-b50d-412043b282c8-1
01:50.250 --> 01:54.080
software vendors, your systems
design, your continuous

d6103593-f950-4d52-b50d-412043b282c8-2
01:54.080 --> 01:57.840
improvement, all have to be
based on certain very key

d6103593-f950-4d52-b50d-412043b282c8-3
01:57.840 --> 02:02.088
principles that will allow you
to maximize the value that is

d6103593-f950-4d52-b50d-412043b282c8-4
02:02.088 --> 02:03.760
coming out of your data.

9de1a30d-5747-4a0c-969a-611e79d959d4-0
02:04.480 --> 02:07.798
So this strategy was endorsed
and, and, and, and we operate

9de1a30d-5747-4a0c-969a-611e79d959d4-1
02:07.798 --> 02:11.228
towards the strategy right now
and we're starting to see that

9de1a30d-5747-4a0c-969a-611e79d959d4-2
02:11.228 --> 02:14.381
the first experiments are
starting to come out and we'll

9de1a30d-5747-4a0c-969a-611e79d959d4-3
02:14.381 --> 02:16.760
talk about one of those
experiments today.

31aa1b77-4302-4720-af07-67d4052fadad-0
02:16.760 --> 02:17.720
Very specifically.

c3aa5b04-1b87-4a88-a261-2b758c42ef14-0
02:18.440 --> 02:24.277
We also developed a way for us
to measure ourselves against

c3aa5b04-1b87-4a88-a261-2b758c42ef14-1
02:24.277 --> 02:25.640
this strategy.

a024cd78-fa5e-47d2-a99a-8042070908c9-0
02:26.040 --> 02:29.309
We took the industry standard
model or our industry model and

a024cd78-fa5e-47d2-a99a-8042070908c9-1
02:29.309 --> 02:31.840
then we started customizing it
for Merck needs.

cb61364e-a12c-44fc-a108-b8b1c9302308-0
02:32.080 --> 02:34.560
We call it the digital 30-70
hundred model.

6f200c1d-9a28-481e-aa78-cd24a1db9a48-0
02:36.080 --> 02:39.833
And, and one of the things that
that I want you guys to take

6f200c1d-9a28-481e-aa78-cd24a1db9a48-1
02:39.833 --> 02:43.464
away from it is when you try to
move from digital silos to

6f200c1d-9a28-481e-aa78-cd24a1db9a48-2
02:43.464 --> 02:47.280
connected plants, digital silos
is your digital 30, connected

6f200c1d-9a28-481e-aa78-cd24a1db9a48-3
02:47.280 --> 02:48.880
plants is your digital 70.

c342b873-470a-4b18-8d41-e7493a4f37f0-0
02:49.360 --> 02:53.127
There are a few foundational
investments you got to start

c342b873-470a-4b18-8d41-e7493a4f37f0-1
02:53.127 --> 02:57.284
making, especially around data,
which is very important in this

c342b873-470a-4b18-8d41-e7493a4f37f0-2
02:57.284 --> 03:01.247
context of use of AI and, and
that those investments include

c342b873-470a-4b18-8d41-e7493a4f37f0-3
03:01.247 --> 03:04.495
your data standards, data
contextualization, data

c342b873-470a-4b18-8d41-e7493a4f37f0-4
03:04.495 --> 03:05.080
taxonomy.

265dc5ef-bfed-4152-94c6-7c8d3c54a9d9-0
03:05.120 --> 03:09.053
There there are various things
you got to get right, or at

265dc5ef-bfed-4152-94c6-7c8d3c54a9d9-1
03:09.053 --> 03:13.186
least start to get right for for
your AI investments to start

265dc5ef-bfed-4152-94c6-7c8d3c54a9d9-2
03:13.186 --> 03:13.919
paying off.

a6bae007-e0cc-4fd6-be90-fc1b4e220e4d-0
03:15.520 --> 03:19.417
Your whole goal is to get your
digital 100, which is a

a6bae007-e0cc-4fd6-be90-fc1b4e220e4d-1
03:19.417 --> 03:23.881
predictive plant and, and the
more investments you make in the

a6bae007-e0cc-4fd6-be90-fc1b4e220e4d-2
03:23.881 --> 03:28.204
standardization and, and, and,
and, and interconnectivity of

a6bae007-e0cc-4fd6-be90-fc1b4e220e4d-3
03:28.204 --> 03:32.243
the systems, the ability to
build models at scale and at

a6bae007-e0cc-4fd6-be90-fc1b4e220e4d-4
03:32.243 --> 03:34.440
speed will be that much faster.

20f9a3c0-579d-43f5-9ce8-63126bef1d82-0
03:35.040 --> 03:38.600
So that that's one of the way we
measure ourselves today.

4e1dc6fc-ff4d-4415-ab89-f25dbbfd3437-0
03:38.600 --> 03:41.364
How connected are we and are,
are these systems talking to

4e1dc6fc-ff4d-4415-ab89-f25dbbfd3437-1
03:41.364 --> 03:41.880
each other?

1589b3ce-015b-4ba2-84cd-9696bfc06c1e-0
03:42.120 --> 03:45.686
And when they are talking to
each other, are they talking by

1589b3ce-015b-4ba2-84cd-9696bfc06c1e-1
03:45.686 --> 03:49.018
force or very we call it the
friction, frictionless data

1589b3ce-015b-4ba2-84cd-9696bfc06c1e-2
03:49.018 --> 03:49.720
flow, right?

ccc68d83-6ac2-437b-a216-8c1551bbdbe2-0
03:50.640 --> 03:53.901
Are, are those connected based
on very specific data standards

ccc68d83-6ac2-437b-a216-8c1551bbdbe2-1
03:53.901 --> 03:56.800
that will make it easy for us to
build models later on?

33891687-dad3-4e6c-bdce-dd5c6ee3884b-0
03:56.800 --> 04:01.160
And, and, and why is this
important?

138162b7-270f-4256-9731-b0af3866a597-0
04:01.640 --> 04:05.498
This is where I invite Matt to
talk to why this is all

138162b7-270f-4256-9731-b0af3866a597-1
04:05.498 --> 04:06.200
important.

11e72a82-e2e3-415c-8036-2c95e4a9fdb9-0
04:08.880 --> 04:09.200
Thanks, Ron.

e0a373f3-3658-4e1f-8004-04124022f137-0
04:09.200 --> 04:12.585
So, you know, at Deloitte, I
mean, similar to Ravi and

e0a373f3-3658-4e1f-8004-04124022f137-1
04:12.585 --> 04:16.340
similar to Merck, we see, you
know, significant potential in

e0a373f3-3658-4e1f-8004-04124022f137-2
04:16.340 --> 04:19.910
the adoption of AI and Jin AI
specifically in the quality

e0a373f3-3658-4e1f-8004-04124022f137-3
04:19.910 --> 04:20.280
space.

956bfd55-58ed-4007-aa51-9f288e20fa7b-0
04:21.280 --> 04:24.111
You know, we've, we've looked
over the last couple years at

956bfd55-58ed-4007-aa51-9f288e20fa7b-1
04:24.111 --> 04:26.848
the impact these technologies
are having across biopharma

956bfd55-58ed-4007-aa51-9f288e20fa7b-2
04:26.848 --> 04:27.840
companies in general.

6fad7d7b-b63b-43b0-84df-34072909052a-0
04:27.840 --> 04:31.504
And what I would say is that,
that based on our, our findings,

6fad7d7b-b63b-43b0-84df-34072909052a-1
04:31.504 --> 04:35.168
we, we predict that there would
be a 1 to $2 billion impact in

6fad7d7b-b63b-43b0-84df-34072909052a-2
04:35.168 --> 04:37.959
operations, mostly derived from
cost reduction.

914e1df8-538d-4a47-bc82-3b2293eab421-0
04:38.240 --> 04:40.200
So significant opportunity.

3ccce604-78b2-4c74-b070-d4f30ece8736-0
04:40.200 --> 04:43.160
When we assessed how companies
are facing off with that

3ccce604-78b2-4c74-b070-d4f30ece8736-1
04:43.160 --> 04:46.331
opportunity and where they are
in the journey, there were a

3ccce604-78b2-4c74-b070-d4f30ece8736-2
04:46.331 --> 04:49.080
couple things that that, that
really come to light.

632c68f3-1d3e-46d5-9fae-c1e212120333-0
04:49.080 --> 04:52.712
One, first, there's still a lot
of uncertainty from a regulatory

632c68f3-1d3e-46d5-9fae-c1e212120333-1
04:52.712 --> 04:56.289
perspective and while you know
most of our clients are speaking

632c68f3-1d3e-46d5-9fae-c1e212120333-2
04:56.289 --> 04:59.530
with regulators and working
through those collaboratively

632c68f3-1d3e-46d5-9fae-c1e212120333-3
04:59.530 --> 05:02.827
with the regulator so they can
bring capabilities, digital

632c68f3-1d3e-46d5-9fae-c1e212120333-4
05:02.827 --> 05:04.560
assets in this regulated space.

4611d61b-bc33-47d7-87de-5f9b153d1e1e-0
05:05.280 --> 05:09.118
The second is many clients are,
you know, advanced in terms of

4611d61b-bc33-47d7-87de-5f9b153d1e1e-1
05:09.118 --> 05:12.896
developing POC's, MVP's, you
know, pick your acronym and are,

4611d61b-bc33-47d7-87de-5f9b153d1e1e-2
05:12.896 --> 05:16.369
are, and some are moving them
into scaled solutions into

4611d61b-bc33-47d7-87de-5f9b153d1e1e-3
05:16.369 --> 05:17.040
production.

f48406a4-3cb4-4922-bb86-c7dcdcda05aa-0
05:17.400 --> 05:20.837
And so it, it's clear that
there's no dearth of good ideas

f48406a4-3cb4-4922-bb86-c7dcdcda05aa-1
05:20.837 --> 05:24.449
or no lack of appreciation of
the significant value that sits

f48406a4-3cb4-4922-bb86-c7dcdcda05aa-2
05:24.449 --> 05:27.478
within specifically operations
and the impact these

f48406a4-3cb4-4922-bb86-c7dcdcda05aa-3
05:27.478 --> 05:28.759
technologies can have.

f627133a-b0aa-4129-a3f7-a4fd1ca559cf-0
05:29.480 --> 05:32.266
And then lastly, what I would
say is that there there there's,

f627133a-b0aa-4129-a3f7-a4fd1ca559cf-1
05:32.266 --> 05:35.097
there's a big investment in the
underlying data as as Ravi just

f627133a-b0aa-4129-a3f7-a4fd1ca559cf-2
05:35.097 --> 05:37.795
said, you know, data maturity is
something that's recognized

f627133a-b0aa-4129-a3f7-a4fd1ca559cf-3
05:37.795 --> 05:38.680
across the industry.

e74b89bd-9a3f-4cf3-a397-321a9cecc0d1-0
05:38.920 --> 05:42.301
And we see a lot of investments
in the underlying taxonomies,

e74b89bd-9a3f-4cf3-a397-321a9cecc0d1-1
05:42.301 --> 05:45.410
ontologies at whatnot, so that
that data can be used and

e74b89bd-9a3f-4cf3-a397-321a9cecc0d1-2
05:45.410 --> 05:47.320
enabled in these GIN AI
solutions.

3c7f6b1d-5ecc-46d9-9ed5-eecf3973c1cb-0
05:47.320 --> 05:50.860
And so when we look specifically
at quality, deviations and

3c7f6b1d-5ecc-46d9-9ed5-eecf3973c1cb-1
05:50.860 --> 05:54.578
complaints are the top 2 areas
of focus we see, you know, more

3c7f6b1d-5ecc-46d9-9ed5-eecf3973c1cb-2
05:54.578 --> 05:56.880
broadly across the, the, the
industry.

19bc6e9d-ff45-4cd2-9544-372385a34f64-0
05:56.880 --> 05:59.458
And I think that was good and,
and, and confirming for, for

19bc6e9d-ff45-4cd2-9544-372385a34f64-1
05:59.458 --> 06:01.994
Merck to see that, you know,
they're not alone in terms of

19bc6e9d-ff45-4cd2-9544-372385a34f64-2
06:01.994 --> 06:04.358
seeing the opportunity and
prosecuting this, this high

19bc6e9d-ff45-4cd2-9544-372385a34f64-3
06:04.358 --> 06:04.960
priority area.

d961daaf-ed6a-4bc6-83b4-a8207f3cfa4a-0
06:05.000 --> 06:11.404
So deviations forms a very
interesting case for us to

d961daaf-ed6a-4bc6-83b4-a8207f3cfa4a-1
06:11.404 --> 06:18.520
experiment with and there are a
couple of reasons for that.

9f5f2214-c099-4c38-9af2-8409a993728e-0
06:20.240 --> 06:24.018
The foundational reason why we
chose to go down this path is

9f5f2214-c099-4c38-9af2-8409a993728e-1
06:24.018 --> 06:27.672
your ability to understand your
unstructured data and your

9f5f2214-c099-4c38-9af2-8409a993728e-2
06:27.672 --> 06:30.954
ability to interpret in in for
intelligence from the

9f5f2214-c099-4c38-9af2-8409a993728e-3
06:30.954 --> 06:34.732
unstructured data will form the
basis or the algorithms will

9f5f2214-c099-4c38-9af2-8409a993728e-4
06:34.732 --> 06:37.953
form the basis for you to
understand a lot of other

9f5f2214-c099-4c38-9af2-8409a993728e-5
06:37.953 --> 06:41.483
unstructured data you have
across the company because it

9f5f2214-c099-4c38-9af2-8409a993728e-6
06:41.483 --> 06:44.704
it's the most non routine
routine thing you do in a

9f5f2214-c099-4c38-9af2-8409a993728e-7
06:44.704 --> 06:45.200
company.

37d3c0f3-889f-4d25-97d5-968da7b15109-0
06:45.240 --> 06:48.944
You got deviations routinely,
but investigating them is very

37d3c0f3-889f-4d25-97d5-968da7b15109-1
06:48.944 --> 06:50.280
non routine in nature.

bbb91473-c377-4612-a109-35c3636604fe-0
06:51.200 --> 06:55.000
So we all understand the basic
flow of a deviation.

c7043355-5660-4adc-ac4c-e8752edd1119-0
06:55.040 --> 06:59.804
You do intake, you do initial
assessment, you try to gather

c7043355-5660-4adc-ac4c-e8752edd1119-1
06:59.804 --> 07:04.806
data, you try to investigate,
apply root cause, do your Kappa,

c7043355-5660-4adc-ac4c-e8752edd1119-2
07:04.806 --> 07:05.680
do your EV.

7160c429-f5a1-4ce3-ab52-60b350f3934f-0
07:06.680 --> 07:12.193
So when we looked at this
workflow, our ability to, to

7160c429-f5a1-4ce3-ab52-60b350f3934f-1
07:12.193 --> 07:18.408
resolve or identify repeat root
causes becomes very important

7160c429-f5a1-4ce3-ab52-60b350f3934f-2
07:18.408 --> 07:22.919
for you to understand the repeat
root cause.

0bb35b19-ae2c-4eb5-927c-48ff78a8a116-0
07:23.840 --> 07:27.144
It is it's imperative that your
AI algorithm understands the

0bb35b19-ae2c-4eb5-927c-48ff78a8a116-1
07:27.144 --> 07:30.720
root cause and only then you can
determine the repeat root cause.

0d05b78b-7bf1-482a-9507-7913a3616add-0
07:31.320 --> 07:35.513
And, and you don't need me to
say that if you can avoid repeat

0d05b78b-7bf1-482a-9507-7913a3616add-1
07:35.513 --> 07:39.573
root causes in any plant, let
alone pharma manufacturing, it

0d05b78b-7bf1-482a-9507-7913a3616add-2
07:39.573 --> 07:43.034
shows you that your, your
ability to investigate is

0d05b78b-7bf1-482a-9507-7913a3616add-3
07:43.034 --> 07:47.228
investigate and put in the right
campus is, is working, you're

0d05b78b-7bf1-482a-9507-7913a3616add-4
07:47.228 --> 07:48.160
under control.

4c61e6cf-46dc-4c6b-8372-959493aec82c-0
07:48.480 --> 07:53.600
And that's why this, this, this
use case was very important for

4c61e6cf-46dc-4c6b-8372-959493aec82c-1
07:53.600 --> 07:53.840
us.

cbec6764-42bc-498d-b168-a351e523e320-0
07:54.480 --> 07:59.064
The way we went about it is we
took all of our existing data

cbec6764-42bc-498d-b168-a351e523e320-1
07:59.064 --> 08:03.573
sets and and looked at what
language models, large language

cbec6764-42bc-498d-b168-a351e523e320-2
08:03.573 --> 08:08.307
models can be used to train our
algorithms to say, look, I now

cbec6764-42bc-498d-b168-a351e523e320-3
08:08.307 --> 08:12.967
understand the relevance in the
context of this deviation and

cbec6764-42bc-498d-b168-a351e523e320-4
08:12.967 --> 08:17.701
based on some of your previous
occurrences, I can suggest some

cbec6764-42bc-498d-b168-a351e523e320-5
08:17.701 --> 08:21.760
root causes that is at the basis
of of the algorithm.

b6b7558c-f968-4731-b5fd-6af0be0ef2a1-0
08:22.840 --> 08:25.840
We had to, I think we had to go
through a few iterations.

a9e2e7ae-3979-479a-87fe-b5d514b52dbc-0
08:25.840 --> 08:29.956
We went through 14 models before
we landed on something that will

a9e2e7ae-3979-479a-87fe-b5d514b52dbc-1
08:29.956 --> 08:33.698
work for us and, and, and what
I'll do is we'll jump into a

a9e2e7ae-3979-479a-87fe-b5d514b52dbc-2
08:33.698 --> 08:37.128
demo of how this was solved and
then I'll get into the

a9e2e7ae-3979-479a-87fe-b5d514b52dbc-3
08:37.128 --> 08:39.000
importance of how we solve it.

15d27db1-51b9-4e77-b306-47d981fcdcff-0
08:41.360 --> 08:44.591
One of the use cases that is
getting a lot of traction in the

15d27db1-51b9-4e77-b306-47d981fcdcff-1
08:44.591 --> 08:46.520
marketplace is deviation
management.

e66c46b7-43f2-4374-a533-ee6f04e2af64-0
08:46.640 --> 08:49.560
To those unfamiliar with this
use case, let me explain.

32f0ccb7-d5db-43a6-9c38-d2dc9ce5e77e-0
08:49.880 --> 08:53.172
In the context of manufacturing,
deviation management is a

32f0ccb7-d5db-43a6-9c38-d2dc9ce5e77e-1
08:53.172 --> 08:56.185
systematic approach to
investigate and correct events

32f0ccb7-d5db-43a6-9c38-d2dc9ce5e77e-2
08:56.185 --> 08:58.920
that deviate from standard
operating procedures.

29a265ef-9737-4317-8f79-363ade120d61-0
08:59.120 --> 09:02.772
Deviations can pose significant
risk to operations, affecting

29a265ef-9737-4317-8f79-363ade120d61-1
09:02.772 --> 09:06.131
quality, compliance and
efficiency, so it's important to

29a265ef-9737-4317-8f79-363ade120d61-2
09:06.131 --> 09:07.840
pay proper attention to them.

a622dbd7-94c8-45e6-a470-3a9ca0f62a83-0
09:08.200 --> 09:11.326
The traditional approach to
deviation management is to

a622dbd7-94c8-45e6-a470-3a9ca0f62a83-1
09:11.326 --> 09:14.965
resolve manufacturing deviations
manually, for example, dealing

a622dbd7-94c8-45e6-a470-3a9ca0f62a83-2
09:14.965 --> 09:18.547
with a sudden equipment failure
or an accident at the site due

a622dbd7-94c8-45e6-a470-3a9ca0f62a83-3
09:18.547 --> 09:19.400
to human error.

b27d6d8b-c959-4a27-921b-17fedb574425-0
09:19.520 --> 09:22.600
There are some key challenges to
manual deviations management.

2045af23-7f9d-438d-837f-27111d4da3ec-0
09:22.800 --> 09:26.542
It's labour intensive in nature,
requiring time and effort on

2045af23-7f9d-438d-837f-27111d4da3ec-1
09:26.542 --> 09:29.983
multiple people to capture the
deviation, which may mean

2045af23-7f9d-438d-837f-27111d4da3ec-2
09:29.983 --> 09:32.760
subjectivity or factual errors
and omissions.

178beca4-33ab-488a-b09a-30a18931f11d-0
09:33.000 --> 09:37.225
These errors in turn can trigger
further deviations, creating a

178beca4-33ab-488a-b09a-30a18931f11d-1
09:37.225 --> 09:38.480
*********** effect.

0daa7f2a-1896-4c01-96d4-0f3ecad1cd25-0
09:38.920 --> 09:41.677
Another issue is the
inconsistency in root cause

0daa7f2a-1896-4c01-96d4-0f3ecad1cd25-1
09:41.677 --> 09:44.884
mapping 2 Similar deviations
might be logged and handled

0daa7f2a-1896-4c01-96d4-0f3ecad1cd25-2
09:44.884 --> 09:47.360
differently, leading to
different outcomes.

17c65164-e6f4-48d2-8c17-48b650150c67-0
09:47.960 --> 09:51.864
This inconsistency hampers
ability to spot patterns or

17c65164-e6f4-48d2-8c17-48b650150c67-1
09:51.864 --> 09:55.840
learn from previous mistakes or
standardized processes.

b7f2e2a0-e208-44ee-b1af-d65c02ed63e1-0
09:56.520 --> 10:00.387
Moreover, the subjectivity and
inconsistency elevate the risk

b7f2e2a0-e208-44ee-b1af-d65c02ed63e1-1
10:00.387 --> 10:03.320
of repeated non compliance and
quality issues.

6cb9ac0f-88ad-44c3-9f07-1cb7abe7d657-0
10:03.960 --> 10:06.765
Deloitte has developed a
transformative artificial

6cb9ac0f-88ad-44c3-9f07-1cb7abe7d657-1
10:06.765 --> 10:07.920
intelligence and Gen.

ece2ffc0-c77e-411a-a606-721f3f097492-0
10:07.920 --> 10:11.787
AI solution to aid in the
identification of true root

ece2ffc0-c77e-411a-a606-721f3f097492-1
10:11.787 --> 10:15.797
causes and assisting in
eliminating repeated deviations

ece2ffc0-c77e-411a-a606-721f3f097492-2
10:15.797 --> 10:19.808
and improve productivity by
automating manual processes

ece2ffc0-c77e-411a-a606-721f3f097492-3
10:19.808 --> 10:24.319
leading to a more streamlined,
efficient and compliant system.

bb90a033-a1a3-43e3-90bf-005f484711de-0
10:25.200 --> 10:29.320
All this is done by keeping the
SME at the center of it all.

8f9c6bbf-4849-45fd-97ab-e373dbcb838e-0
10:29.840 --> 10:33.573
The solution has been designed
to assist the SME in taking the

8f9c6bbf-4849-45fd-97ab-e373dbcb838e-1
10:33.573 --> 10:36.240
next best action and aid in
decision making.

22b6e849-2a5e-4cae-b8b8-d30464656180-0
10:36.680 --> 10:38.600
The solution has four
capabilities.

eeda452d-02ff-4f42-bd4c-37115bbfbe70-0
10:39.080 --> 10:42.246
Real time visibility of
deviations via a dashboard to

eeda452d-02ff-4f42-bd4c-37115bbfbe70-1
10:42.246 --> 10:44.240
support informed decision
making.

fdb98a01-7ad3-40fe-974b-aebd98fb4998-0
10:44.760 --> 10:47.827
NLP and classification
algorithms that provide

fdb98a01-7ad3-40fe-974b-aebd98fb4998-1
10:47.827 --> 10:50.569
recommendations on the
classification and

fdb98a01-7ad3-40fe-974b-aebd98fb4998-2
10:50.569 --> 10:54.616
categorization of the deviation
to guide the user on the next

fdb98a01-7ad3-40fe-974b-aebd98fb4998-3
10:54.616 --> 10:55.400
best action.

a25b1378-2f34-436a-9c0a-c983fc1326b3-0
10:55.760 --> 10:56.120
Gen.

e5b07b62-02a5-403b-9fbc-0ee98fc78d00-0
10:56.120 --> 11:00.209
AI is leveraged to generate
preliminary investigation report

e5b07b62-02a5-403b-9fbc-0ee98fc78d00-1
11:00.209 --> 11:03.360
and the virtual agent or chatbot
which is Gen.

ac3fe41a-bbcf-4ece-8d70-409d9aed1553-0
11:03.360 --> 11:06.840
AI enabled that helps the user
with the ongoing investigation.

40b4b8ed-30fa-423d-996b-e16665aa1f5a-0
11:07.400 --> 11:15.043
1st to the dashboard which
provides real time visibility

40b4b8ed-30fa-423d-996b-e16665aa1f5a-1
11:15.043 --> 11:21.614
into deviations, enabling
stakeholders to access

40b4b8ed-30fa-423d-996b-e16665aa1f5a-2
11:21.614 --> 11:29.928
comprehensive and up to Here is
how traditional AI algorithms

40b4b8ed-30fa-423d-996b-e16665aa1f5a-3
11:29.928 --> 11:37.303
are leveraged to make a
recommended to the operator on

40b4b8ed-30fa-423d-996b-e16665aa1f5a-4
11:37.303 --> 11:40.120
the next best action.

fe7c2172-1c39-48b8-a428-4f03149ccd98-0
11:40.280 --> 11:44.611
The models use narratives of
deviations as inputs to predict

fe7c2172-1c39-48b8-a428-4f03149ccd98-1
11:44.611 --> 11:48.587
classifications or categories
that can be assessed by a

fe7c2172-1c39-48b8-a428-4f03149ccd98-2
11:48.587 --> 11:49.440
quality SME.

9fd3a413-d103-492c-a416-20e9292defba-0
11:49.760 --> 11:52.651
Each step requires the user to
accept or override the

9fd3a413-d103-492c-a416-20e9292defba-1
11:52.651 --> 11:53.240
prediction.

dcf22c23-462a-4878-b6a6-a4d3a301f3ac-0
11:53.440 --> 11:56.877
For example, if a given
deviation is impacting product

dcf22c23-462a-4878-b6a6-a4d3a301f3ac-1
11:56.877 --> 12:01.002
quality, a deviation manager can
adjust the severity of the event

dcf22c23-462a-4878-b6a6-a4d3a301f3ac-2
12:01.002 --> 12:04.690
and give a custom preventative
action to close out the non

dcf22c23-462a-4878-b6a6-a4d3a301f3ac-3
12:04.690 --> 12:05.440
conformance.

4eff6530-0b16-4ab0-a5cc-bb20586bb360-0
12:06.120 --> 12:09.224
The model learns from user
feedback to update predictions

4eff6530-0b16-4ab0-a5cc-bb20586bb360-1
12:09.224 --> 12:09.920
in real time.

53eae006-3d3d-48b6-959d-cea76c9cddec-0
12:10.560 --> 12:14.322
Once all six steps have been
reviewed, the event moves to the

53eae006-3d3d-48b6-959d-cea76c9cddec-1
12:14.322 --> 12:15.840
Processed events section.

3fa0ab36-e781-46d6-b07c-7d4ab409c5f2-0
12:16.080 --> 12:19.280
This enables users to view each
past event.

9e5b2a22-ef59-4d71-97ca-33146fcdcd7a-0
12:19.640 --> 12:23.780
Upon processing a deviation, the
system generates a comprehensive

9e5b2a22-ef59-4d71-97ca-33146fcdcd7a-1
12:23.780 --> 12:27.543
preliminary investigation aid
report using a large language

9e5b2a22-ef59-4d71-97ca-33146fcdcd7a-2
12:27.543 --> 12:27.920
model.

3b500693-2be4-4f8c-a149-237d4fbfeaf9-0
12:28.080 --> 12:31.540
These large language models,
intelligently guided by our

3b500693-2be4-4f8c-a149-237d4fbfeaf9-1
12:31.540 --> 12:35.122
designed prompts, create reports
that include an executive

3b500693-2be4-4f8c-a149-237d4fbfeaf9-2
12:35.122 --> 12:38.582
summary, historical instances of
similar deviations, the

3b500693-2be4-4f8c-a149-237d4fbfeaf9-3
12:38.582 --> 12:41.860
underlying cause of the
deviation, impact assessment,

3b500693-2be4-4f8c-a149-237d4fbfeaf9-4
12:41.860 --> 12:45.200
resource allocation details, and
a conclusive summary.

f3eba18d-191b-456c-9843-8e166d35611e-0
12:46.520 --> 12:49.951
And the fourth component, the
quality Buddy chat bot, also

f3eba18d-191b-456c-9843-8e166d35611e-1
12:49.951 --> 12:53.149
driven by the same large
language model, leverages the

f3eba18d-191b-456c-9843-8e166d35611e-2
12:53.149 --> 12:56.638
retrieval augmented generation
approach to address not only

f3eba18d-191b-456c-9843-8e166d35611e-3
12:56.638 --> 13:00.128
user inquiries pertaining to
deviations but also systematic

f3eba18d-191b-456c-9843-8e166d35611e-4
13:00.128 --> 13:01.640
issues within the process.

a2cce9ed-96c4-4626-93c6-d0ca9f67a2f5-0
13:01.800 --> 13:06.056
By analyzing historical data and
providing insights into the root

a2cce9ed-96c4-4626-93c6-d0ca9f67a2f5-1
13:06.056 --> 13:10.054
causes of deviations, the chat
bot helps to identify systemic

a2cce9ed-96c4-4626-93c6-d0ca9f67a2f5-2
13:10.054 --> 13:11.279
process weaknesses.

37892777-03d6-450d-bb48-0485a75474ff-0
13:11.520 --> 13:15.031
Furthermore, users can engage
with a chat bot to brainstorm

37892777-03d6-450d-bb48-0485a75474ff-1
13:15.031 --> 13:18.250
potential improvements or
directive actions to prevent

37892777-03d6-450d-bb48-0485a75474ff-2
13:18.250 --> 13:21.469
deviations from reoccurring,
thus fostering continuous

37892777-03d6-450d-bb48-0485a75474ff-3
13:21.469 --> 13:22.640
process improvement.

02fee793-ab62-499e-97a8-80cffd4c405c-0
13:23.000 --> 13:26.882
I hope this practical demo has
brought to life a clear example

02fee793-ab62-499e-97a8-80cffd4c405c-1
13:26.882 --> 13:27.560
of how Gen.

f94e2faf-5000-4cfb-8b5d-b8270c7f7488-0
13:27.560 --> 13:31.640
AI and AI can help in day-to-day
supply chain operations.

97782c5f-fbce-4252-b29c-22b06b9e4c3b-0
13:40.600 --> 13:43.835
Just a disclaimer, a lot of the
data sets that you have seen are

97782c5f-fbce-4252-b29c-22b06b9e4c3b-1
13:43.835 --> 13:45.080
very synthetic in nature.

dcb92b2e-e0f5-4614-b01d-61fd28a5a2e8-0
13:45.120 --> 13:47.200
That's not live production
system.

ff175288-cca1-459c-ad6f-a85d50d439c7-0
13:48.000 --> 13:52.325
So as you can see, a lot of the
companies that you've seen in

ff175288-cca1-459c-ad6f-a85d50d439c7-1
13:52.325 --> 13:56.790
that video have been tried, have
been solved or or attempted to

ff175288-cca1-459c-ad6f-a85d50d439c7-2
13:56.790 --> 14:00.000
be solved in different scenarios
in the past.

55abcb9e-cb14-4c66-a3c7-2c4698813ae7-0
14:00.400 --> 14:04.066
Intelligent search of deviations
is one of them, especially when

55abcb9e-cb14-4c66-a3c7-2c4698813ae7-1
14:04.066 --> 14:07.564
you're gathering data and is
your data set that you gathered,

55abcb9e-cb14-4c66-a3c7-2c4698813ae7-2
14:07.564 --> 14:09.200
is that comprehensive enough?

3aecef83-8c67-407a-9f1f-6ece125dbc96-0
14:09.520 --> 14:13.542
And after that, can you consume
all the data and in for the

3aecef83-8c67-407a-9f1f-6ece125dbc96-1
14:13.542 --> 14:16.760
outcomes of the data in a very
cohesive manner?

78d90d17-fa16-45d5-90b1-3dd873c3a7c2-0
14:17.600 --> 14:22.266
All of those problem statements
were tried to were solved in the

78d90d17-fa16-45d5-90b1-3dd873c3a7c2-1
14:22.266 --> 14:26.790
industry one way or the other,
but never put together in a way

78d90d17-fa16-45d5-90b1-3dd873c3a7c2-2
14:26.790 --> 14:31.241
where you start with the basic
description and then have your

78d90d17-fa16-45d5-90b1-3dd873c3a7c2-3
14:31.241 --> 14:35.118
model understand the root cause
based on the previous

78d90d17-fa16-45d5-90b1-3dd873c3a7c2-4
14:35.118 --> 14:39.785
occurrences and then collect all
the relevant data and summarize

78d90d17-fa16-45d5-90b1-3dd873c3a7c2-5
14:39.785 --> 14:40.360
for you.

b0530e5f-93b5-4110-a4bf-b0da9739c80c-0
14:40.760 --> 14:44.347
What is the right amount of
information for you to consume

b0530e5f-93b5-4110-a4bf-b0da9739c80c-1
14:44.347 --> 14:45.320
in this context?

c9c2b1dc-dfcd-4be3-90fc-13cf46932c57-0
14:46.400 --> 14:51.325
That is the hypothesis we went
after and and when when we came

c9c2b1dc-dfcd-4be3-90fc-13cf46932c57-1
14:51.325 --> 14:56.017
out of it, the the the logic
that we applied to this is are

c9c2b1dc-dfcd-4be3-90fc-13cf46932c57-2
14:56.017 --> 15:00.786
we doing this classification and
and summarization on par or

c9c2b1dc-dfcd-4be3-90fc-13cf46932c57-3
15:00.786 --> 15:03.680
about our current human
performance?

b4b46a58-d467-4bc1-a256-557a626851d8-0
15:03.960 --> 15:07.960
That was our bar because it's a
hypothesis.

0c469e00-0f27-476c-a36f-9c2258f90c5c-0
15:07.960 --> 15:08.720
It's an experiment.

8b52e132-ae31-4017-882d-da6df73918c7-0
15:09.320 --> 15:11.600
You never know what you what
your outcome will be.

595eb551-2a68-4f9c-8d93-707e84e6613b-0
15:12.200 --> 15:15.304
And that's where Deloitte
partnership was very useful to

595eb551-2a68-4f9c-8d93-707e84e6613b-1
15:15.304 --> 15:18.898
us because they were able to put
in the right data scientists and

595eb551-2a68-4f9c-8d93-707e84e6613b-2
15:18.898 --> 15:21.840
the team that we needed at scale
to make this happen.

36110787-300f-47fb-b2cf-ae66bb9b6511-0
15:23.360 --> 15:26.040
But I want to, I want to leave a
message here, right?

85bf2de5-c6f4-4bf4-84bd-f27e5a3a6a34-0
15:26.400 --> 15:30.255
Large language models alone is
not a not the answer for what

85bf2de5-c6f4-4bf4-84bd-f27e5a3a6a34-1
15:30.255 --> 15:31.520
you're trying to do.

999f61e3-72e6-465d-a58c-2c059f037b1e-0
15:31.880 --> 15:35.384
Traditional AI will take you
months, if not years to solve

999f61e3-72e6-465d-a58c-2c059f037b1e-1
15:35.384 --> 15:38.710
something like this because
you're limited by your data

999f61e3-72e6-465d-a58c-2c059f037b1e-2
15:38.710 --> 15:42.333
sets, the size of them and the
investment, the compute power

999f61e3-72e6-465d-a58c-2c059f037b1e-3
15:42.333 --> 15:45.600
you would need to get to what,
what we need to get to.

9129ff04-aa71-497f-aef8-24a90524fd46-0
15:46.040 --> 15:50.229
So a combination of all of this
helped us get here and, and for

9129ff04-aa71-497f-aef8-24a90524fd46-1
15:50.229 --> 15:54.221
us, the, the outcome is can I
identify the root, repeat root

9129ff04-aa71-497f-aef8-24a90524fd46-2
15:54.221 --> 15:58.410
'cause that's where we started,
but everything else that you've

9129ff04-aa71-497f-aef8-24a90524fd46-3
15:58.410 --> 15:59.720
seen is a byproduct.

a8b84d26-1791-47f8-ab50-cf3b54c8d02f-0
15:59.800 --> 16:02.632
Oh, now that I understand, I can
understand the text and I

a8b84d26-1791-47f8-ab50-cf3b54c8d02f-1
16:02.632 --> 16:05.320
understand the repeat root
'cause I can do all of this.

ffd1584a-a08f-4e28-a9f3-261c989312d5-0
16:05.400 --> 16:08.600
I can do the intelligent search,
I can do do the Q&A bot.

8a095700-3110-41a1-a5fb-804fc0d1c9ed-0
16:08.920 --> 16:11.320
Everything comes as a package
with it, right?

c368434a-6032-4b50-8cfa-a0bb0546093e-0
16:12.200 --> 16:15.992
You can see at the bottom right,
we, we had to do multiple

c368434a-6032-4b50-8cfa-a0bb0546093e-1
16:15.992 --> 16:19.591
workshops, multiple models
before we landed on a couple

c368434a-6032-4b50-8cfa-a0bb0546093e-2
16:19.591 --> 16:21.520
models that solve this for us.

1c51dc51-f882-4065-9733-adde4862381f-0
16:23.680 --> 16:24.520
I'll hand it over to Matt.

c2f5c9e3-adce-4a53-b71c-a7db551ed865-0
16:27.360 --> 16:30.476
So in terms of actually, let's
go back for one, one minute,

c2f5c9e3-adce-4a53-b71c-a7db551ed865-1
16:30.476 --> 16:30.840
please.

9e33a818-98c6-48df-af63-c8344515cce4-0
16:31.400 --> 16:34.552
So in terms of the, the results,
I just wanted to highlight, you

9e33a818-98c6-48df-af63-c8344515cce4-1
16:34.552 --> 16:37.560
know what, what were, So what
was the result of, of, of this?

7d49f8f1-e555-4849-bf56-332b358baa67-0
16:37.560 --> 16:39.916
And this is, you know,
preliminary result based on

7d49f8f1-e555-4849-bf56-332b358baa67-1
16:39.916 --> 16:40.840
preliminary results.

44d46eda-37f2-4483-aa37-25cccf9ffdf6-0
16:40.840 --> 16:44.403
I mean, the first, as Robbie
said, was we saw improvement in

44d46eda-37f2-4483-aa37-25cccf9ffdf6-1
16:44.403 --> 16:48.026
the, the classification as, as
you can imagine, there's human

44d46eda-37f2-4483-aa37-25cccf9ffdf6-2
16:48.026 --> 16:51.589
variation, right, whether it's
language countries, different

44d46eda-37f2-4483-aa37-25cccf9ffdf6-3
16:51.589 --> 16:54.920
jurisdictions in terms of how
deviations are classified.

78410116-571c-4934-890c-b441b4720502-0
16:54.920 --> 16:57.895
And the model was able to
actually improve consistency in

78410116-571c-4934-890c-b441b4720502-1
16:57.895 --> 16:59.280
classification by over 50%.

692a5780-9dd8-4ec2-ac44-2525bffed9ae-0
16:59.560 --> 17:01.800
So, you know, significant
improvement there.

3331e55c-81fe-4ee6-8596-d32df4f79d0b-0
17:02.160 --> 17:06.218
The other, the other area of
benefit was really around, as

3331e55c-81fe-4ee6-8596-d32df4f79d0b-1
17:06.218 --> 17:10.689
Ravi spoke to about identifying
repeat root causes and, and, and

3331e55c-81fe-4ee6-8596-d32df4f79d0b-2
17:10.689 --> 17:15.092
both the, the ability for it to,
to identify additional repeats

3331e55c-81fe-4ee6-8596-d32df4f79d0b-3
17:15.092 --> 17:19.013
that weren't able to be
identified by humans really stem

3331e55c-81fe-4ee6-8596-d32df4f79d0b-4
17:19.013 --> 17:20.319
from 2 two factors.

7f550a16-e4ee-4f4c-bf40-5c318f855ad8-0
17:20.320 --> 17:23.588
One, I mean, the, the, the
machine and learning and, and,

7f550a16-e4ee-4f4c-bf40-5c318f855ad8-1
17:23.588 --> 17:27.138
and Jedi algorithms could ingest
a significant amount of data,

7f550a16-e4ee-4f4c-bf40-5c318f855ad8-2
17:27.138 --> 17:29.900
far more data that can be
consumed by any single

7f550a16-e4ee-4f4c-bf40-5c318f855ad8-3
17:29.900 --> 17:30.520
individual.

90dcb76f-e6de-4115-b14c-a22d36ee1e5a-0
17:30.520 --> 17:33.888
And they looked over a historic
time frame that was longer than

90dcb76f-e6de-4115-b14c-a22d36ee1e5a-1
17:33.888 --> 17:36.520
many quality professionals are
actually in roles.

6779b84b-21dd-4c9c-9fec-1c8aaf2f8f53-0
17:36.520 --> 17:40.784
And so it was it, you know, it
used, you know, a far different

6779b84b-21dd-4c9c-9fec-1c8aaf2f8f53-1
17:40.784 --> 17:42.680
approach than a human would.

7ed6e6d0-47e7-4fc1-a5f5-33646813eab9-0
17:42.680 --> 17:45.400
And that's what really drove
those improvements.

2191967d-d4ea-47e9-b5d3-b7df94865180-0
17:45.400 --> 17:47.720
And so I, I just want to kind of
under_that.

961f78d7-b03c-400b-a1a4-3dbe06cad2a3-0
17:47.720 --> 17:49.952
The other thing I, I think in
terms of the repeats was, you

961f78d7-b03c-400b-a1a4-3dbe06cad2a3-1
17:49.952 --> 17:51.440
know, it, we took a different
approach.

1b51a205-1df5-4a93-964a-38be184550e2-0
17:51.440 --> 17:54.864
We actually as, as Robbie was
alluding to with the the LLMS,

1b51a205-1df5-4a93-964a-38be184550e2-1
17:54.864 --> 17:58.345
we were able to interrogate the
investigation reports and use

1b51a205-1df5-4a93-964a-38be184550e2-2
17:58.345 --> 18:01.938
that to inform the, the identity
modification and, and, and and

1b51a205-1df5-4a93-964a-38be184550e2-3
18:01.938 --> 18:04.240
pattern recognition in terms of
repeats.

d24930c0-83b5-45ab-a6f6-ab1724445a3f-0
18:04.360 --> 18:07.721
Whereas historically trending a
metadata structured data versus

d24930c0-83b5-45ab-a6f6-ab1724445a3f-1
18:07.721 --> 18:10.400
the unstructured data was used
for categorization.

25bfd994-7b34-4504-83b3-3889e88ae13c-0
18:10.400 --> 18:13.349
So it, it fundamentally used a
different technique than has

25bfd994-7b34-4504-83b3-3889e88ae13c-1
18:13.349 --> 18:14.480
historically been used.

1654bdd4-7647-4c86-97af-6c39afd58b81-0
18:14.480 --> 18:18.206
And so these benefits along with
the the reduction in

1654bdd4-7647-4c86-97af-6c39afd58b81-1
18:18.206 --> 18:20.000
investigation time, right?

25c77358-31ea-4586-95d8-6753eca785f9-0
18:20.000 --> 18:22.990
We didn't quantify this, but as
you can imagine with what you

25c77358-31ea-4586-95d8-6753eca785f9-1
18:22.990 --> 18:25.787
saw in the video in terms of
that first draft generation,

25c77358-31ea-4586-95d8-6753eca785f9-2
18:25.787 --> 18:28.439
significant time saved in terms
of just generating it.

0fcc7f63-bf00-41cf-a40d-6d5104a29381-0
18:28.440 --> 18:31.771
But as the model gets adopted,
opted as it gets better use and

0fcc7f63-bf00-41cf-a40d-6d5104a29381-1
18:31.771 --> 18:34.891
the data gets better, you could
we can you know, we expect

0fcc7f63-bf00-41cf-a40d-6d5104a29381-2
18:34.891 --> 18:37.747
better Kappa effectiveness,
which Ravi alluded to and

0fcc7f63-bf00-41cf-a40d-6d5104a29381-3
18:37.747 --> 18:40.920
overall fewer investigations and
and quicker investigation.

37d1cc0b-0631-47d0-aa19-1c063689d5f8-0
18:40.920 --> 18:43.842
So you know, our estimate when
we look across the that, that

37d1cc0b-0631-47d0-aa19-1c063689d5f8-1
18:43.842 --> 18:46.908
the end in process, we estimate
that could be as much as 25% of

37d1cc0b-0631-47d0-aa19-1c063689d5f8-2
18:46.908 --> 18:48.920
the time it takes to do an
investigation.

643a59d3-f426-4431-96f8-22582beb4606-0
18:48.920 --> 18:51.400
So significant impact, you know,
across the board.

638c17a9-9ebb-4f21-accf-a6a163582f31-0
18:51.680 --> 18:56.880
So I'm going to the next slide
here.

b23f4042-ea07-4533-92a3-2d6913b09dc5-0
18:56.960 --> 18:58.680
Sorry, go back, please.

a49363a1-5082-411b-9a90-53a31aee23d2-0
19:00.000 --> 19:03.022
So you know that I I think the
video and can you go back one

a49363a1-5082-411b-9a90-53a31aee23d2-1
19:03.022 --> 19:03.320
slide?

54e7341e-59ab-4be5-8006-719183f0b160-0
19:03.400 --> 19:04.160
Sorry, yeah, there you go.

07b37e70-cf1d-4afb-af7c-a63d25353562-0
19:05.400 --> 19:09.063
You know, this, this slide
simply deconstructs what it took

07b37e70-cf1d-4afb-af7c-a63d25353562-1
19:09.063 --> 19:11.200
to actually develop that
solution.

02720085-f333-4c5e-b8d8-3862a249b2eb-0
19:11.200 --> 19:14.518
And on the left, what you'll see
is we we broke the overall

02720085-f333-4c5e-b8d8-3862a249b2eb-1
19:14.518 --> 19:17.836
business problem into a finite
number of objectives and and

02720085-f333-4c5e-b8d8-3862a249b2eb-2
19:17.836 --> 19:21.321
then related those and, and use
those to define a specific set

02720085-f333-4c5e-b8d8-3862a249b2eb-3
19:21.321 --> 19:22.039
of use cases.

b479e4d0-a072-4210-8da6-1ba799745dcb-0
19:22.040 --> 19:25.558
And, you know, at, at, as, as
the video showed, this was

b479e4d0-a072-4210-8da6-1ba799745dcb-1
19:25.558 --> 19:28.645
around, you know,
classification, this was around

b479e4d0-a072-4210-8da6-1ba799745dcb-2
19:28.645 --> 19:32.411
triaging, this was, was around
report generation and around,

b479e4d0-a072-4210-8da6-1ba799745dcb-3
19:32.411 --> 19:35.621
you know, Q&A support for
the, the, the quality

b479e4d0-a072-4210-8da6-1ba799745dcb-4
19:35.621 --> 19:38.400
professionals in the, in future
generations.

a1542989-b7f3-4ef1-ba04-90d8336912d3-0
19:38.400 --> 19:41.680
That would include things around
dashboards for deviation,

a1542989-b7f3-4ef1-ba04-90d8336912d3-1
19:41.680 --> 19:44.571
performance management,
automated workflow and, and

a1542989-b7f3-4ef1-ba04-90d8336912d3-2
19:44.571 --> 19:47.796
alerting, as well as greater
insight generation to inform

a1542989-b7f3-4ef1-ba04-90d8336912d3-3
19:47.796 --> 19:51.243
things like process improvement,
training and, and regulatory

a1542989-b7f3-4ef1-ba04-90d8336912d3-4
19:51.243 --> 19:51.800
reporting.

dd551e14-c63e-46da-9c50-4251c5c2adb7-0
19:51.800 --> 19:55.676
So, so that was really important
in terms of breaking the problem

dd551e14-c63e-46da-9c50-4251c5c2adb7-1
19:55.676 --> 19:59.200
into a set of use cases that
could be solved independently.

e5eadc83-d37d-46fa-a0b5-aca5f340437c-0
19:59.200 --> 20:02.762
And then as, as Ravi alluded to,
we, you know, we started out and

e5eadc83-d37d-46fa-a0b5-aca5f340437c-1
20:02.762 --> 20:04.760
explored more than 14 different
Gen.

d150039f-bc0a-47a9-89cf-7a05eb70c062-0
20:04.760 --> 20:07.120
AI and, and machine learning
models.

5ae64bec-be5d-4662-9286-d567c3f3fb0e-0
20:07.520 --> 20:09.240
It wasn't, you know, we didn't
hit it.

da31af67-ee59-4db1-8281-e6f1da789b9a-0
20:09.760 --> 20:13.066
You know, correctly on the first
time, it was a, it was a, a

da31af67-ee59-4db1-8281-e6f1da789b9a-1
20:13.066 --> 20:16.535
repeated process where we, you
know, fine-tuned the models and,

da31af67-ee59-4db1-8281-e6f1da789b9a-2
20:16.535 --> 20:19.950
and required extensive testing
of alternative models to select

da31af67-ee59-4db1-8281-e6f1da789b9a-3
20:19.950 --> 20:23.420
what you see up on the page as
where we ultimately ended from a

da31af67-ee59-4db1-8281-e6f1da789b9a-4
20:23.420 --> 20:26.726
Gin AI and machine learning
perspective, concurrent with the

da31af67-ee59-4db1-8281-e6f1da789b9a-5
20:26.726 --> 20:30.033
build out of the, the actual
solution, we, we, we mapped out

da31af67-ee59-4db1-8281-e6f1da789b9a-6
20:30.033 --> 20:33.177
what data was going to be
required and what systems those

da31af67-ee59-4db1-8281-e6f1da789b9a-7
20:33.177 --> 20:35.400
data that data needed to
originate from.

10240e4d-1a6d-459d-9fc3-27dfca413580-0
20:35.400 --> 20:38.362
And what I would say you see on
the page here is more of an in

10240e4d-1a6d-459d-9fc3-27dfca413580-1
20:38.362 --> 20:38.880
state view.

2734d433-9318-4017-82de-031d5eedc5ac-0
20:38.880 --> 20:41.687
So we did not build out all
these integrations and, and, and

2734d433-9318-4017-82de-031d5eedc5ac-1
20:41.687 --> 20:44.080
bring in all this data for
what's been built today.

02a865e8-9845-4768-ad81-27ba48adf4f0-0
20:44.080 --> 20:46.930
But this is the vision that we
have in terms of having a data

02a865e8-9845-4768-ad81-27ba48adf4f0-1
20:46.930 --> 20:49.597
rich input coming from multiple
source systems across the

02a865e8-9845-4768-ad81-27ba48adf4f0-2
20:49.597 --> 20:52.586
enterprise and, and the linchpin
and making all of this work was

02a865e8-9845-4768-ad81-27ba48adf4f0-3
20:52.586 --> 20:55.620
really, you know, down the, down
the middle that, that, that data

02a865e8-9845-4768-ad81-27ba48adf4f0-4
20:55.620 --> 20:57.000
modeling and data engineering.

e4eb0d85-5e0c-45a2-830b-e3fd6abbda11-0
20:57.400 --> 21:00.475
You know, a simple example is,
is that when we were looking at

e4eb0d85-5e0c-45a2-830b-e3fd6abbda11-1
21:00.475 --> 21:03.550
classification, you know, Mark
does a great job of classifying

e4eb0d85-5e0c-45a2-830b-e3fd6abbda11-2
21:03.550 --> 21:05.600
deviations based on multiple
root causes.

baa14f26-4ae8-4dfe-a2ac-49653c6babaf-0
21:05.960 --> 21:08.773
And there are, there are many
categories they used to, to

baa14f26-4ae8-4dfe-a2ac-49653c6babaf-1
21:08.773 --> 21:09.840
identify a root cause.

1a7197bc-b5c7-4797-8777-3a851579ca71-0
21:09.840 --> 21:12.797
Well, the, the preliminary
models that we used were unable

1a7197bc-b5c7-4797-8777-3a851579ca71-1
21:12.797 --> 21:15.605
and, and didn't perform very
well in terms of doing the

1a7197bc-b5c7-4797-8777-3a851579ca71-2
21:15.605 --> 21:18.613
classification given the, the,
the distribution, the uneven

1a7197bc-b5c7-4797-8777-3a851579ca71-3
21:18.613 --> 21:21.120
distribution across all the
different categories.

d6d67ff7-cd15-4e01-a09a-9110443efc76-0
21:21.120 --> 21:23.906
And so we had to go back and
actually working with the

d6d67ff7-cd15-4e01-a09a-9110443efc76-1
21:23.906 --> 21:27.098
quality professionals, come up
with a taxonomy that related to

d6d67ff7-cd15-4e01-a09a-9110443efc76-2
21:27.098 --> 21:29.986
the low frequency root causes so
that we could get model

d6d67ff7-cd15-4e01-a09a-9110443efc76-3
21:29.986 --> 21:33.229
performance to a level that was,
was adequate and met the needs

d6d67ff7-cd15-4e01-a09a-9110443efc76-4
21:33.229 --> 21:34.040
of the business.

75995e50-f553-42df-8739-1fa87d347259-0
21:34.040 --> 21:37.269
And so, you know, on that last
point, you know, I, I, I can't_I

75995e50-f553-42df-8739-1fa87d347259-1
21:37.269 --> 21:40.449
mean, we talked a lot about the
technology and we're showing a

75995e50-f553-42df-8739-1fa87d347259-2
21:40.449 --> 21:43.426
lot of, you know, things here,
but it's, it's actually the

75995e50-f553-42df-8739-1fa87d347259-3
21:43.426 --> 21:44.840
people that are so critical.

ab087230-e7af-4d37-8b36-c9d74e256b3f-0
21:45.440 --> 21:47.959
You know, the individuals we had
involved, the subject matter

ab087230-e7af-4d37-8b36-c9d74e256b3f-1
21:47.959 --> 21:50.560
experts were the prior process
owners for deviation management.

b12cab2d-a01d-45e3-b9be-f7e227cb894b-0
21:50.920 --> 21:54.035
They were involved in helping us
understand the Indian workflow,

b12cab2d-a01d-45e3-b9be-f7e227cb894b-1
21:54.035 --> 21:56.814
contextualizing the data,
reviewing the results and model

b12cab2d-a01d-45e3-b9be-f7e227cb894b-2
21:56.814 --> 21:59.594
performance, and ultimately
helping us do the, the prompt

b12cab2d-a01d-45e3-b9be-f7e227cb894b-3
21:59.594 --> 22:01.320
engineering and the tuning of
that.

9078d555-c390-4d5f-980a-6f2e69b46bc4-0
22:01.480 --> 22:04.333
So there was a really good user
experience in terms of the

9078d555-c390-4d5f-980a-6f2e69b46bc4-1
22:04.333 --> 22:07.379
chatbot and, and, and how it can
be prompted for from AQ and a

9078d555-c390-4d5f-980a-6f2e69b46bc4-2
22:07.379 --> 22:07.960
perspective.

653dcb09-4983-4f7e-a43a-a851f7f50b81-0
22:07.960 --> 22:11.504
So, so in the end, this is a, a,
a decomposition of how we built

653dcb09-4983-4f7e-a43a-a851f7f50b81-1
22:11.504 --> 22:14.721
the solution, but it was the,
the, the talent and the team

653dcb09-4983-4f7e-a43a-a851f7f50b81-2
22:14.721 --> 22:17.993
that really came together to
deliver the results that I, I,

653dcb09-4983-4f7e-a43a-a851f7f50b81-3
22:17.993 --> 22:18.920
I, I spoke about.

6e813b62-58b5-4c61-8ca2-5ef51f656c96-0
22:22.680 --> 22:22.880
OK.

3eabad5f-0369-4d84-892e-0ff08562ed0c-0
22:23.520 --> 22:27.033
So you know, one of the things
that we, I get asked a lot and,

3eabad5f-0369-4d84-892e-0ff08562ed0c-1
22:27.033 --> 22:30.490
and we had to look at asked a
lot is really around how do you

3eabad5f-0369-4d84-892e-0ff08562ed0c-2
22:30.490 --> 22:33.000
go from APOC or an MVP to a
scaled solution?

a3fd2fbc-63dc-4700-8621-d63cc94bae6d-0
22:33.000 --> 22:36.621
And a lot of times as we unpack
what's really underneath that

a3fd2fbc-63dc-4700-8621-d63cc94bae6d-1
22:36.621 --> 22:39.600
question is how do you drive
transformative value?

8f5ccf61-7169-4186-8871-2ffe9771caac-0
22:39.680 --> 22:43.040
We see time and again where
digital solutions are are built

8f5ccf61-7169-4186-8871-2ffe9771caac-1
22:43.040 --> 22:46.120
that don't pull through the
value, that don't have the

8f5ccf61-7169-4186-8871-2ffe9771caac-2
22:46.120 --> 22:47.240
adoption as desired.

be4e33ca-aed0-447a-8e52-00e206fc7c6a-0
22:47.520 --> 22:50.874
And as companies are spending
increasingly sums of money both

be4e33ca-aed0-447a-8e52-00e206fc7c6a-1
22:50.874 --> 22:54.175
in developing and maintaining
solutions like this, you know,

be4e33ca-aed0-447a-8e52-00e206fc7c6a-2
22:54.175 --> 22:57.475
senior executives are asking,
you know, operations and other

be4e33ca-aed0-447a-8e52-00e206fc7c6a-3
22:57.475 --> 22:59.640
entities to really show them the
money.

bed8f50d-7869-4c88-b995-1b668b850016-0
23:00.120 --> 23:03.497
And so at Deloitte, what we like
to talk about is a is a string

bed8f50d-7869-4c88-b995-1b668b850016-1
23:03.497 --> 23:06.451
of pearls and and really this is
a simple concept about

bed8f50d-7869-4c88-b995-1b668b850016-2
23:06.451 --> 23:08.826
interconnected digital
capabilities to drive

bed8f50d-7869-4c88-b995-1b668b850016-3
23:08.826 --> 23:10.040
transformative results.

46028bbe-ff80-4acc-aeca-872a677619d8-0
23:10.040 --> 23:13.086
And so as you can imagine in the
deviation example, deviations

46028bbe-ff80-4acc-aeca-872a677619d8-1
23:13.086 --> 23:14.440
aren't managed in isolation.

476cc306-a730-4119-9a6c-ba6da19b8460-0
23:14.800 --> 23:17.651
You can imagine a world where
you have a digital asset and

476cc306-a730-4119-9a6c-ba6da19b8460-1
23:17.651 --> 23:20.357
deviations and complaint
management that provide an end

476cc306-a730-4119-9a6c-ba6da19b8460-2
23:20.357 --> 23:23.160
to end seamless user experience
and better orchestration.

353c6173-2d9e-4339-a41f-452df6e8ac13-0
23:23.520 --> 23:27.159
You can imagine enriching data
across deviations, asset raw

353c6173-2d9e-4339-a41f-452df6e8ac13-1
23:27.159 --> 23:30.313
material, manufacturing
performance to drive better

353c6173-2d9e-4339-a41f-452df6e8ac13-2
23:30.313 --> 23:33.953
insights across operations or
how improvements and insights

353c6173-2d9e-4339-a41f-452df6e8ac13-3
23:33.953 --> 23:37.835
from complaints can improve and
drive continuous improvement in

353c6173-2d9e-4339-a41f-452df6e8ac13-4
23:37.835 --> 23:41.050
in SO PS that then reduce
overall deviations and the

353c6173-2d9e-4339-a41f-452df6e8ac13-5
23:41.050 --> 23:43.720
synergistic outputs you can you
can create.

7d3d832e-bda4-437d-a391-a85f1359e820-0
23:43.800 --> 23:46.296
That's what we mean by a string
of pearls and driving

7d3d832e-bda4-437d-a391-a85f1359e820-1
23:46.296 --> 23:47.360
transformative results.

e60c0aa7-d2ac-4b59-80b1-ab1e4a5b9065-0
23:47.360 --> 23:48.680
And it really requires two
things.

822e9863-f211-400d-a8e8-50aecf563cad-0
23:49.240 --> 23:52.280
In its simplest basis, it
requires understanding the

822e9863-f211-400d-a8e8-50aecf563cad-1
23:52.280 --> 23:55.778
component part and and from the
get go building the solution

822e9863-f211-400d-a8e8-50aecf563cad-2
23:55.778 --> 23:58.360
patterns that with with the
intent of reuse.

8db38b20-6f84-4d73-915b-663f2fc9adf5-0
23:58.360 --> 24:01.451
In other words, what what we've
built here for deviations can

8db38b20-6f84-4d73-915b-663f2fc9adf5-1
24:01.451 --> 24:04.294
easily be used some of the
components and complaints and

8db38b20-6f84-4d73-915b-663f2fc9adf5-2
24:04.294 --> 24:06.040
other use cases across
operations.

3a34d5f6-d0ee-42e9-a9b0-713cf12242ff-0
24:06.040 --> 24:09.064
And similarly as as Robbie
talked about at the beginning,

3a34d5f6-d0ee-42e9-a9b0-713cf12242ff-1
24:09.064 --> 24:11.880
the data maturity and data flow
is vitally important.

3d3029f0-5553-4188-88a6-13baa4dc97b7-0
24:12.000 --> 24:15.271
So how do you look at the end to
end flow and draw insights from

3d3029f0-5553-4188-88a6-13baa4dc97b7-1
24:15.271 --> 24:16.480
one solution to another?

9aff4950-d077-47e2-890c-d53b84abf2a2-0
24:16.640 --> 24:19.497
It's that combination, that
intentional build of specific

9aff4950-d077-47e2-890c-d53b84abf2a2-1
24:19.497 --> 24:22.306
solutions and how they're
combined that really can drive

9aff4950-d077-47e2-890c-d53b84abf2a2-2
24:22.306 --> 24:23.440
transformative results.

65f9f8cb-1a9c-4fee-8422-f813566f419d-0
24:25.800 --> 24:29.072
So just in conclusion, you know,
as, as I said, I mean, the

65f9f8cb-1a9c-4fee-8422-f813566f419d-1
24:29.072 --> 24:32.181
possibilities not only in
deviation management, but more

65f9f8cb-1a9c-4fee-8422-f813566f419d-2
24:32.181 --> 24:33.600
broadly around AI and Gen.

41493815-f20b-4e20-a784-283309a13fc5-0
24:33.600 --> 24:36.697
AI is significant, but it's not
only about delivering value,

41493815-f20b-4e20-a784-283309a13fc5-1
24:36.697 --> 24:38.120
it's about sustaining value.

ee188fca-d8a2-47d6-abeb-c3b4419ab2bc-0
24:38.360 --> 24:40.839
And so one of the things I would
just touch on, I, I mentioned

ee188fca-d8a2-47d6-abeb-c3b4419ab2bc-1
24:40.839 --> 24:43.240
previously in the development,
the importance of the people.

b8afc451-9ca1-460c-99bf-34fceba1fb02-0
24:43.520 --> 24:45.800
It's also the importance of the
people in terms of adoption.

004d220b-a86d-402a-88b9-aebb2eba289b-0
24:46.000 --> 24:48.693
This isn't just about, you know,
changing the roles and

004d220b-a86d-402a-88b9-aebb2eba289b-1
24:48.693 --> 24:51.771
responsibilities of individuals,
but actually changing the very

004d220b-a86d-402a-88b9-aebb2eba289b-2
24:51.771 --> 24:54.079
fabric of how work gets done
across operations.

fddbf741-910e-47d6-af5e-21e64ac88828-0
24:54.360 --> 24:57.553
And that is essential if you're
going to drive the results and

fddbf741-910e-47d6-af5e-21e64ac88828-1
24:57.553 --> 24:59.480
sustain the results over time,
right?

38c0057e-8dbb-4b5d-9ad9-23cc7a6ab11d-0
25:03.040 --> 25:03.160
Good.

df749b5b-2143-4564-90c1-b17890825e1c-0
25:03.160 --> 25:07.953
So couple things I want to
emphasize before we close out

df749b5b-2143-4564-90c1-b17890825e1c-1
25:07.953 --> 25:10.560
now to, to what matter you did.

90f88c37-27fa-4076-909a-6a45b88bdfe1-0
25:10.840 --> 25:14.565
The ways of working is very
important when you get into this

90f88c37-27fa-4076-909a-6a45b88bdfe1-1
25:14.565 --> 25:18.352
use of AI, especially it, it
becomes extremely important that

90f88c37-27fa-4076-909a-6a45b88bdfe1-2
25:18.352 --> 25:21.039
you're not 100% reliant on the
tool, right?

fe09bde8-d4c7-4d02-8a37-a506d2379167-0
25:21.480 --> 25:24.360
You, you understand the how how
the tool is adopted.

87a2289e-43fd-455b-9b5a-5e7da5ff7c95-0
25:24.560 --> 25:27.140
Are you putting the right belts
and suspenders, which is the

87a2289e-43fd-455b-9b5a-5e7da5ff7c95-1
25:27.140 --> 25:28.240
phase we are in right now?

780ce558-babc-4bf9-aae9-39afa499bb9f-0
25:28.720 --> 25:32.613
And are you actually overall
improving your current control

780ce558-babc-4bf9-aae9-39afa499bb9f-1
25:32.613 --> 25:34.560
and compliance posture, right?

27be6582-3d51-4d96-8c8c-9bab3286d540-0
25:35.320 --> 25:38.395
Your body said for today's
compliance posture and your AI

27be6582-3d51-4d96-8c8c-9bab3286d540-1
25:38.395 --> 25:39.880
should improve on it, right.

98669f6e-4679-47dc-8724-bb82e58619b7-0
25:41.200 --> 25:46.078
Another point that I want to
emphasize here is when, when you

98669f6e-4679-47dc-8724-bb82e58619b7-1
25:46.078 --> 25:51.035
build AAI tool or, or or or AI
based systems, your your, your,

98669f6e-4679-47dc-8724-bb82e58619b7-2
25:51.035 --> 25:55.520
your traditional SDLC kind of
fails you in a way, right?

0dabdb52-8dcd-4c0e-982b-48a77c5ce1df-0
25:55.720 --> 25:59.268
When you go with your current
SDLC model, you basically go

0dabdb52-8dcd-4c0e-982b-48a77c5ce1df-1
25:59.268 --> 26:02.937
live and then you basically
react to any bugs or enhancement

0dabdb52-8dcd-4c0e-982b-48a77c5ce1df-2
26:02.937 --> 26:04.080
requests in, in AI.

6245302b-31d4-48a8-b7fc-6fbdef8b14ba-0
26:04.400 --> 26:08.105
You have to continually feed it
and, and and re look at your

6245302b-31d4-48a8-b7fc-6fbdef8b14ba-1
26:08.105 --> 26:09.320
model, fine tune it.

6e5e078b-fe13-406c-96e3-bc94eb4fad74-0
26:09.640 --> 26:13.130
You get new data sets, you get
events that the model hasn't

6e5e078b-fe13-406c-96e3-bc94eb4fad74-1
26:13.130 --> 26:14.120
seen in the past.

30a4e4ac-fc63-4fa8-841c-aa2a65a122e9-0
26:14.560 --> 26:15.600
So you got to account for it.

110ea8c8-eace-49f7-9023-3cc75b437b95-0
26:15.600 --> 26:19.158
So your ML OPS becomes extremely
important and you want to make

110ea8c8-eace-49f7-9023-3cc75b437b95-1
26:19.158 --> 26:21.160
sure that you thought that
through.

13962900-df22-47c6-a1d3-f951a2d5cf88-0
26:21.400 --> 26:25.432
You staffed it, you resourced it
and you made sure that you're

13962900-df22-47c6-a1d3-f951a2d5cf88-1
26:25.432 --> 26:29.336
very intentionally about it
before you release your A models

13962900-df22-47c6-a1d3-f951a2d5cf88-2
26:29.336 --> 26:30.360
into production.

d27f7f6e-0b46-4db3-82b5-302e6930f6f7-0
26:30.640 --> 26:34.405
So that's one item I want you
guys to take away from this

d27f7f6e-0b46-4db3-82b5-302e6930f6f7-1
26:34.405 --> 26:35.120
slide here.

2f1b8544-6f34-4e17-b234-b538bdf6a121-0
26:35.560 --> 26:38.458
And sustained innovation is
another thing that that Matt was

2f1b8544-6f34-4e17-b234-b538bdf6a121-1
26:38.458 --> 26:41.357
talking about and we are very
passionate about the 14 models

2f1b8544-6f34-4e17-b234-b538bdf6a121-2
26:41.357 --> 26:42.640
that we talked about there.

595c1060-a2fc-47c7-a31e-8c650ca9a6d3-0
26:43.600 --> 26:46.556
If, if I could understand how a
deviation is written in the

595c1060-a2fc-47c7-a31e-8c650ca9a6d3-1
26:46.556 --> 26:49.561
context and the relevance, I
could do that for complaints, I

595c1060-a2fc-47c7-a31e-8c650ca9a6d3-2
26:49.561 --> 26:51.680
could do that for 100 other
things, right?

9b455270-e669-4ac8-a55f-75c863b6e444-0
26:52.560 --> 26:55.784
So when you build an algorithm,
you're really not solving for

9b455270-e669-4ac8-a55f-75c863b6e444-1
26:55.784 --> 26:56.720
that one use case.

0d4e6ad6-edf9-43a8-afa7-c3fe9120476c-0
26:57.320 --> 27:01.600
You could apply that to five,
6-7 other use cases.

b74a264f-b6e3-4889-aa9f-581c42432051-0
27:01.600 --> 27:06.429
So as you build your library of
algorithms, your speed of

b74a264f-b6e3-4889-aa9f-581c42432051-1
27:06.429 --> 27:10.760
innovation and deployment
becomes that much faster.

6834f78c-30ec-4c67-8f6d-bdf4581773a5-0
27:10.960 --> 27:14.495
The initial inertia will be
high, but eventually it'll

6834f78c-30ec-4c67-8f6d-bdf4581773a5-1
27:14.495 --> 27:18.160
smooth out and you'll you'll be
able to deploy at scale.

0fe07944-accd-48e6-bdb2-a5b494fa9744-0
27:19.320 --> 27:19.640
Thank you.

76a060e1-e928-40bc-ade1-d8433ee2960c-0
27:19.640 --> 27:37.660
Oh, sorry.

8b970612-7e81-4b40-929c-fd76c5fcb583-0
27:37.660 --> 27:37.980
There you go.

561577a6-98c6-40e7-9753-80c93dad39f2-0
27:38.300 --> 27:38.460
Hello.

d3643997-a21e-4bd8-aa14-3bcc64ed8ef7-0
27:38.460 --> 27:42.740
Sorry, I'm Victor from quarter
group.

f6464317-4cb0-40b9-9247-8da426799c82-0
27:43.540 --> 27:47.690
I'm was wondering how when
people start using these

f6464317-4cb0-40b9-9247-8da426799c82-1
27:47.690 --> 27:52.478
deviation help us or whatever
you want to call them, how do

f6464317-4cb0-40b9-9247-8da426799c82-2
27:52.478 --> 27:57.666
you continually ensure that they
are using their critical senses

f6464317-4cb0-40b9-9247-8da426799c82-3
27:57.666 --> 28:01.976
and are aware of the
responsibility when making these

f6464317-4cb0-40b9-9247-8da426799c82-4
28:01.976 --> 28:07.083
deviations instead of slacking
off maybe and just accepting the

f6464317-4cb0-40b9-9247-8da426799c82-5
28:07.083 --> 28:08.760
prepared suggestions.

47f6e4df-28bf-4855-a7ad-36bb595b29c8-0
28:12.960 --> 28:17.591
Excellent question, right that,
that's something that from day

47f6e4df-28bf-4855-a7ad-36bb595b29c8-1
28:17.591 --> 28:22.076
one when we started this, right,
companies spend millions of

47f6e4df-28bf-4855-a7ad-36bb595b29c8-2
28:22.076 --> 28:26.193
dollars training trained
investigators and they spend a

47f6e4df-28bf-4855-a7ad-36bb595b29c8-3
28:26.193 --> 28:30.972
lot of money, time and resources
to make sure that your outcomes

47f6e4df-28bf-4855-a7ad-36bb595b29c8-4
28:30.972 --> 28:33.839
of those investigations are top
notch.

12732376-3074-4141-b8ed-725659752337-0
28:34.440 --> 28:38.637
So your your scale for
implementation of AI tools in

12732376-3074-4141-b8ed-725659752337-1
28:38.637 --> 28:43.784
this space has to be the bar has
to be at least at or about that

12732376-3074-4141-b8ed-725659752337-2
28:43.784 --> 28:46.240
level of trained professionals.

01ee118b-411c-447b-bbbf-daa8e1788340-0
28:46.720 --> 28:48.040
That's your first pass, right?

15f388fe-c009-434a-9957-46bdb2d8ed56-0
28:48.560 --> 28:52.513
That tells you that if this goes
into production, when this goes

15f388fe-c009-434a-9957-46bdb2d8ed56-1
28:52.513 --> 28:55.797
into production, you're
minimally able to replicate a

15f388fe-c009-434a-9957-46bdb2d8ed56-2
28:55.797 --> 28:57.440
trained investigators eyes.

736611cb-99b4-49d7-97e5-f56b39237fd4-0
28:58.000 --> 29:00.618
The second one is the belts and
suspenders you put into it,

736611cb-99b4-49d7-97e5-f56b39237fd4-1
29:00.618 --> 29:00.880
right?

1a852ce9-6f21-43cb-a33e-c35b05ffc355-0
29:02.000 --> 29:04.600
You, you don't, you're not.

6bbe40d4-fcc7-4471-a712-285c03e7a1f1-0
29:05.400 --> 29:09.070
When you build tools that are
data rich, you can monitor from

6bbe40d4-fcc7-4471-a712-285c03e7a1f1-1
29:09.070 --> 29:09.840
the back end.

9bbcccc6-69af-4973-aa30-6f3bd4ca4fa6-0
29:10.520 --> 29:12.440
How well are those tools being
used?

ac56e151-52ef-43b7-921a-4cc0f7770c92-0
29:12.440 --> 29:13.760
Do you have Kpas around it?

00acefd7-a15d-437a-a35a-51db844f92ff-0
29:14.080 --> 29:17.383
Let's just say if a person
accepts all six root causes in

00acefd7-a15d-437a-a35a-51db844f92ff-1
29:17.383 --> 29:20.800
six seconds, you know there
wasn't enough time spent on it.

5df84b30-eea2-4730-a231-794608268292-0
29:21.240 --> 29:24.507
And do you wait for three months
to monitor it or do you flag it

5df84b30-eea2-4730-a231-794608268292-1
29:24.507 --> 29:26.720
immediately before the deviation
is closed?

0ac3d73b-4aed-40a0-9526-7f4ec25d8473-0
29:27.280 --> 29:30.755
You got to be very intentional
in monitoring of your process

0ac3d73b-4aed-40a0-9526-7f4ec25d8473-1
29:30.755 --> 29:34.117
and these tools are are are
designed from day one for your

0ac3d73b-4aed-40a0-9526-7f4ec25d8473-2
29:34.117 --> 29:35.200
ability to do that.

1d67da24-6924-40e8-8afd-b79a0c97c0fa-0
29:35.600 --> 29:37.680
In traditional systems, that
becomes an afterthought.

93083fed-79ae-469e-bd44-fdbb536372f3-0
29:37.920 --> 29:41.040
In AI, the data is rich and you
can capture that from day one.

a2e252dd-1720-47a5-9376-7721acf5853c-0
29:41.280 --> 29:45.443
So we're thinking through all of
those aspects and, and designing

a2e252dd-1720-47a5-9376-7721acf5853c-1
29:45.443 --> 29:49.040
those guardrails right now
before it goes to production.

5be9d16c-35fe-44f8-8e33-16b3ea28e1c1-0
29:49.360 --> 29:51.120
We are extremely sensitive to
that aspect.

f90fbbdc-f09a-46d6-9c6a-1c51d1af240a-0
29:55.480 --> 29:56.200
1st, thank you.

0041d913-5b84-4fb3-91e6-ad72e5222219-0
29:56.200 --> 29:59.680
A very good presentation
gentleman Nikolai Makaranka from

0041d913-5b84-4fb3-91e6-ad72e5222219-1
29:59.680 --> 29:59.920
BMS.

443ca11c-7297-49ca-8e8c-f8b32224bf6e-0
30:00.440 --> 30:02.240
I have two questions that are
related.

d6bcd0d3-d92a-4f1d-817d-961f9871e307-0
30:02.240 --> 30:06.411
The first one, if you could
maybe give more details about

d6bcd0d3-d92a-4f1d-817d-961f9871e307-1
30:06.411 --> 30:11.158
your evaluation approach because
I think from my opinion and from

d6bcd0d3-d92a-4f1d-817d-961f9871e307-2
30:11.158 --> 30:15.329
what we've seen with a similar
solution at BMS is that it

d6bcd0d3-d92a-4f1d-817d-961f9871e307-3
30:15.329 --> 30:19.213
unless it can be, you know,
fully integrated into the

d6bcd0d3-d92a-4f1d-817d-961f9871e307-4
30:19.213 --> 30:23.672
process when you mention it in
SOP and give people like clear

d6bcd0d3-d92a-4f1d-817d-961f9871e307-5
30:23.672 --> 30:27.772
guidance that it's not just
auxiliary business tool, but

d6bcd0d3-d92a-4f1d-817d-961f9871e307-6
30:27.772 --> 30:31.440
something that becomes part of
the quality system.

f0c59ff9-000c-4e2d-bd1f-bc6ca353c22c-0
30:32.200 --> 30:36.019
It's not yet fully adopted, You
know, the adoption sort of

f0c59ff9-000c-4e2d-bd1f-bc6ca353c22c-1
30:36.019 --> 30:37.120
becomes optional.

d61372a7-d7c8-4d47-92dc-616e2fa76516-0
30:38.360 --> 30:41.400
So how do you see this, this
tools in the future?

0c4d64bb-7ba7-4fd4-997d-7a56534136e9-0
30:41.400 --> 30:43.960
Do they become part of the
quality system?

01c8bceb-bdf3-45f9-b31a-681b42e34a0d-0
30:44.440 --> 30:49.400
And with that, like how would
you, how would you get there?

f79cbf0d-20b7-468d-81df-79c20e598f2d-0
30:49.680 --> 30:53.697
What's your sort of vision to
evaluate it to ensure it works

f79cbf0d-20b7-468d-81df-79c20e598f2d-1
30:53.697 --> 30:57.517
correctly and to, to make it
part of the quality, quality

f79cbf0d-20b7-468d-81df-79c20e598f2d-2
30:57.517 --> 31:01.601
management system or maybe keep
it sort of as this exhilarate

f79cbf0d-20b7-468d-81df-79c20e598f2d-3
31:01.601 --> 31:05.487
tool that people can use in
addition to what they have as,

f79cbf0d-20b7-468d-81df-79c20e598f2d-4
31:05.487 --> 31:07.200
as their main flow in SOP?

52301a48-31bf-46ce-a30d-cd8cc2aae6b5-0
31:08.880 --> 31:09.400
Good question.

14112725-1a7e-4280-8fe0-67476b222cb0-0
31:10.400 --> 31:11.160
Thanks for the question.

171ff5ed-0478-490e-80d3-5125855bb9b7-0
31:12.080 --> 31:16.516
It's, it's right now, it's not a
vision, it's a requirement that

171ff5ed-0478-490e-80d3-5125855bb9b7-1
31:16.516 --> 31:20.817
we put in place at Merck that it
has to be within the workflow

171ff5ed-0478-490e-80d3-5125855bb9b7-2
31:20.817 --> 31:25.186
of, of, of which regardless of
which AI tool we develop, it has

171ff5ed-0478-490e-80d3-5125855bb9b7-3
31:25.186 --> 31:29.145
to be within the workflow when
it comes to our ability to

171ff5ed-0478-490e-80d3-5125855bb9b7-4
31:29.145 --> 31:30.920
incorporate this at scale.

e82067bb-1745-4fdc-8d68-71ca128c3ab9-0
31:30.920 --> 31:32.520
That, that is very important,
right.

ba5a4617-44d6-4a0f-a5fb-7a307dd6a505-0
31:33.720 --> 31:37.668
Your, your other question about
ability to validate it, this is

ba5a4617-44d6-4a0f-a5fb-7a307dd6a505-1
31:37.668 --> 31:41.000
a question that the whole
industry is grappling with.

d51b1c94-07b9-49ac-8b57-81f5aa8a2aa4-0
31:42.000 --> 31:46.641
We are extremely interested in
perspectives of our regulators

d51b1c94-07b9-49ac-8b57-81f5aa8a2aa4-1
31:46.641 --> 31:47.240
as well.

412e3051-16c4-459c-8021-ef9859c84dd4-0
31:47.760 --> 31:49.440
We are starting to engage on
that.

4b6bdab6-4033-4831-8652-671eb027c09e-0
31:50.280 --> 31:53.832
And our threshold for this, like
I said, is whatever our

4b6bdab6-4033-4831-8652-671eb027c09e-1
31:53.832 --> 31:57.635
performance today is, the tool
has to be better than that in

4b6bdab6-4033-4831-8652-671eb027c09e-2
31:57.635 --> 32:00.440
the workflow before we put it
into full use.

2ec2fe32-56ab-40ae-ba19-f6a50352b0bd-0
32:02.200 --> 32:04.880
Right now, what you have seen is
MVP one.

0c145d38-5927-4cd7-92f4-59f705b24d12-0
32:04.880 --> 32:08.887
There's MVP 2, which is to
integrate within the workflow

0c145d38-5927-4cd7-92f4-59f705b24d12-1
32:08.887 --> 32:09.520
and MV 3.

35bd4fe3-20b4-4304-84b9-bee1c24f2e03-0
32:09.520 --> 32:13.748
MVP 3 is when you're ready to
production lies it since this is

35bd4fe3-20b4-4304-84b9-bee1c24f2e03-1
32:13.748 --> 32:15.360
in quality space, right?

f853a4cc-1ceb-4d60-a8ed-0f2a16d61621-0
32:16.080 --> 32:22.734
It's not, it's not in spaces
where you could, you could take

f853a4cc-1ceb-4d60-a8ed-0f2a16d61621-1
32:22.734 --> 32:25.680
things close to perfection.

7c4ca992-2319-4eff-88f8-33842298c82a-0
32:25.920 --> 32:27.440
We want perfection on this.

c07bf89f-c6b1-44b9-9328-3c419b0c140b-0
32:28.360 --> 32:30.880
And then that's the expectation
from our executives as well.

9761ef5b-a964-47d5-92aa-b9c06da2046a-0
32:31.320 --> 32:34.568
So we'll share more as we
continue to learn on how we get

9761ef5b-a964-47d5-92aa-b9c06da2046a-1
32:34.568 --> 32:35.520
to that endpoint.

5e614211-3317-4a5b-adb3-4aab97012a35-0
32:38.600 --> 32:42.620
Hi, my name is Wendy, I am from
Novnodisk and I'm very curious

5e614211-3317-4a5b-adb3-4aab97012a35-1
32:42.620 --> 32:46.257
about to hear how does the two
work regarding root cause

5e614211-3317-4a5b-adb3-4aab97012a35-2
32:46.257 --> 32:49.895
analysis because one of our
biggest challenges we need a

5e614211-3317-4a5b-adb3-4aab97012a35-3
32:49.895 --> 32:52.319
structure way to find the root
cause.

e61bd103-710b-4bcf-8a20-0c08597fec8c-0
32:52.560 --> 32:55.120
We need different inputs,
different people.

10dcf529-4cd0-4c52-8f35-80f45e616062-0
32:55.400 --> 32:59.271
Does the two ask some previous
questions when you find the

10dcf529-4cd0-4c52-8f35-80f45e616062-1
32:59.271 --> 33:01.240
deviation or do they just get?

54123aea-03e3-4b53-b7d7-6a169055c80a-0
33:01.520 --> 33:05.912
Does the two just gather the
previous deviations and and try

54123aea-03e3-4b53-b7d7-6a169055c80a-1
33:05.912 --> 33:08.000
to make possible root causes?

a030d26d-b09c-41d9-a368-fd0ddf396ac9-0
33:08.800 --> 33:09.240
Good question.

4e092984-a109-40f8-882e-d1332ccc99a7-0
33:10.400 --> 33:13.000
So I've I've been in the
industry 20 years.

796d4dd7-e165-44a9-973a-6661242d9977-0
33:13.000 --> 33:17.855
I've seen quite a few deviations
myself, almost the three or four

796d4dd7-e165-44a9-973a-6661242d9977-1
33:17.855 --> 33:19.400
companies I work for.

b1e00f27-7243-4c54-a7d5-548ce82da23f-0
33:19.960 --> 33:23.990
Everybody has a deviation
process that has drop downs or

b1e00f27-7243-4c54-a7d5-548ce82da23f-1
33:23.990 --> 33:27.878
some form of root cause
assignment and you trend based

b1e00f27-7243-4c54-a7d5-548ce82da23f-2
33:27.878 --> 33:30.000
on that root cause assignment.

bef4c5e4-a5a0-4569-8266-0d9eecf836aa-0
33:31.120 --> 33:33.558
In this algorithm, the way we
look at it is we go to the

bef4c5e4-a5a0-4569-8266-0d9eecf836aa-1
33:33.558 --> 33:35.440
unstructured data, not the
structured data.

6418853b-840a-4819-a4ed-8351cc03b2fc-0
33:35.520 --> 33:38.758
First we go to the unstructured
data, the actual description,

6418853b-840a-4819-a4ed-8351cc03b2fc-1
33:38.758 --> 33:42.153
the actual root cause write up,
what were the events that led to

6418853b-840a-4819-a4ed-8351cc03b2fc-2
33:42.153 --> 33:43.720
that root cause determination?

6a8a32c5-fc3f-4302-80d1-50ed03b19f05-0
33:44.080 --> 33:45.680
What were the root cause
considered?

efdca9bb-d7c2-41a9-9999-a5d5aa3b6ab1-0
33:45.680 --> 33:47.360
What were the root causes
eliminated?

2adfcfa8-259a-454c-ab61-32d3e620c0cf-0
33:47.760 --> 33:52.072
And based on that we infer what
does the algorithm think it

2adfcfa8-259a-454c-ab61-32d3e620c0cf-1
33:52.072 --> 33:56.600
should be and then you relate it
back to your structured data.

e30f8d15-1370-4d62-8691-9e5d936168f3-0
33:57.280 --> 34:01.680
So that is the difference in how
we approach this hypothesis.

33b56aac-addf-4862-9e88-690d242d066e-0
34:01.680 --> 34:07.640
OK, hi.

a100d8be-75d6-4d8e-b59c-450732e9aa22-0
34:07.920 --> 34:11.751
Hi, my name is Videl, I'm
quality control pharmacist from

a100d8be-75d6-4d8e-b59c-450732e9aa22-1
34:11.751 --> 34:12.280
Tunisia.

cac0be6f-d255-4c18-b157-259866f89d3d-0
34:13.120 --> 34:14.560
Thank you for the presentation.

1b26de31-580f-4a3a-b55e-0e19e59bae2c-0
34:15.160 --> 34:16.520
I actually have two questions.

7dea5a75-d542-4765-9de0-e559a0e1d22e-0
34:17.200 --> 34:19.618
First question about the
training data set user to

7dea5a75-d542-4765-9de0-e559a0e1d22e-1
34:19.618 --> 34:20.520
validate the model.

858f2d81-4b47-47fa-b1a4-00c65f9502ea-0
34:20.520 --> 34:24.120
Is it the same data set for all
the clients or it is adapted for

858f2d81-4b47-47fa-b1a4-00c65f9502ea-1
34:24.120 --> 34:24.840
every client?

df8df89b-31f3-438e-8c75-95f931fb5f38-0
34:26.520 --> 34:31.368
Second question, since this soft
software or solution is in the

df8df89b-31f3-438e-8c75-95f931fb5f38-1
34:31.368 --> 34:36.064
core of quality operations, the
validation of the software, I

df8df89b-31f3-438e-8c75-95f931fb5f38-2
34:36.064 --> 34:40.231
mean the validation metrics for
the software accuracy,

df8df89b-31f3-438e-8c75-95f931fb5f38-3
34:40.231 --> 34:44.700
sensitivity, specificity is
taken from which regulation or

df8df89b-31f3-438e-8c75-95f931fb5f38-4
34:44.700 --> 34:49.397
reference to see it to say if
that the classifications of the

df8df89b-31f3-438e-8c75-95f931fb5f38-5
34:49.397 --> 34:52.200
prediction of this model is
correct.

47d2d93d-13f0-452a-bad8-cc0079000b95-0
34:53.040 --> 34:53.400
Thank you.

3f91abe3-f589-40e2-a40a-7cad8149791f-0
34:55.280 --> 34:58.968
So no, the training data set,
we, we were very intentionally

3f91abe3-f589-40e2-a40a-7cad8149791f-1
34:58.968 --> 35:02.476
bought it from day one that the
training data set will be

3f91abe3-f589-40e2-a40a-7cad8149791f-2
35:02.476 --> 35:06.225
different a subset of of your
whole set, but it won't be used

3f91abe3-f589-40e2-a40a-7cad8149791f-3
35:06.225 --> 35:08.040
for training the model, right.

957368c2-1052-4839-910a-730190b0fce4-0
35:08.400 --> 35:10.240
So those are two separate sets.

36628e8c-f8f2-4b9a-afa5-8f35fbd73ffc-0
35:10.800 --> 35:15.974
As far as validation of to which
specific criteria, again, the,

36628e8c-f8f2-4b9a-afa5-8f35fbd73ffc-1
35:15.974 --> 35:20.905
the regulations expect us to do
a thorough investigation and

36628e8c-f8f2-4b9a-afa5-8f35fbd73ffc-2
35:20.905 --> 35:26.080
it's accurate and, and and the
investigation stands on its own.

f4392045-0ee9-4e4c-9ed6-c7776358b02b-0
35:26.240 --> 35:27.440
So that's your scale.

c66c05a5-0592-4065-a8f7-9db105a67dc2-0
35:28.480 --> 35:32.514
I do not believe there is a
99.9% accurate scale on on these

c66c05a5-0592-4065-a8f7-9db105a67dc2-1
35:32.514 --> 35:33.440
things, right.

e8fa730e-659a-46d6-8f65-14701364bc08-0
35:35.320 --> 35:38.869
Like I said, we invest so much
money in getting these right the

e8fa730e-659a-46d6-8f65-14701364bc08-1
35:38.869 --> 35:39.480
first time.

6628fa58-0751-4d11-8b08-b3d40bebb003-0
35:40.560 --> 35:43.080
Every company does at least
every major pharma does.

3e33c03b-002a-49c2-a250-9ef39d84b517-0
35:44.480 --> 35:47.633
So if you can cross that bar,
that means you're already

3e33c03b-002a-49c2-a250-9ef39d84b517-1
35:47.633 --> 35:50.280
crossing a pretty high bar in,
in our opinion.

9a2b681f-da9b-4668-b7dd-ffc79bc3a4e2-0
35:50.640 --> 35:54.307
And and what we're seeing the
results are we are crossing that

9a2b681f-da9b-4668-b7dd-ffc79bc3a4e2-1
35:54.307 --> 35:57.160
bar not by a point or two by
significant margin.

382b5d86-90fd-4288-97f0-1cc431eaa539-0
35:57.400 --> 36:00.985
Just one more question you
afterward if you still have

382b5d86-90fd-4288-97f0-1cc431eaa539-1
36:00.985 --> 36:04.440
questions to follow up with our
speakers separately.

31fb3ccd-c857-431b-8f8b-400687d8c1de-0
36:06.440 --> 36:12.241
So, so you mentioned about the
output, it should be equal or

31fb3ccd-c857-431b-8f8b-400687d8c1de-1
36:12.241 --> 36:17.663
more than the best trained
resource who can write a best

31fb3ccd-c857-431b-8f8b-400687d8c1de-2
36:17.663 --> 36:19.280
deviation, right.

3bdf98d3-80b3-49d8-8234-8961225e17aa-0
36:20.680 --> 36:25.513
Then how, what did you do to
train the model with that kind

3bdf98d3-80b3-49d8-8234-8961225e17aa-1
36:25.513 --> 36:26.400
of an data?

888af408-f606-4b1c-9175-be5ebb114030-0
36:26.960 --> 36:29.460
Because when you look into the
data, it's like for so many

888af408-f606-4b1c-9175-be5ebb114030-1
36:29.460 --> 36:31.240
years, so many people would have
done it.

a477441a-06c6-4fb6-acb8-b25948217393-0
36:31.800 --> 36:34.840
Did you put any kind of an
quality?

eb52851a-ea40-4eb6-8758-92468caa70be-0
36:34.880 --> 36:37.323
Because right now we're talking
about good augmentation

eb52851a-ea40-4eb6-8758-92468caa70be-1
36:37.323 --> 36:39.985
practices, but then the area
that good quality data practice

eb52851a-ea40-4eb6-8758-92468caa70be-2
36:39.985 --> 36:41.600
is something that we need to
evolve.

12109af7-a40a-4a8e-b474-0bd397c48fb6-0
36:42.000 --> 36:44.600
Did you put that as a baseline
for you to select?

1791627c-8d3b-429a-a3f5-bdca2bf68764-0
36:44.600 --> 36:48.000
This is the best that you want
your model to be trained on.

0636bef3-ea62-4f72-b9f2-92b2166ad288-0
36:48.960 --> 36:51.720
So look, when we do
investigations, I go back to my

0636bef3-ea62-4f72-b9f2-92b2166ad288-1
36:51.720 --> 36:54.640
statement, they're done by very
trained professionals.

a0da80bb-bd71-4645-ae0a-e4d714de8fe6-0
36:54.640 --> 36:58.920
There will be variation because
of like jurisdictions.

f2ba2881-7333-4c5e-9726-ac999a5ec54c-0
36:58.960 --> 37:03.438
The controls in place at that
plant site is the plant in a

f2ba2881-7333-4c5e-9726-ac999a5ec54c-1
37:03.438 --> 37:08.220
country where your inference is
slightly different of the same

f2ba2881-7333-4c5e-9726-ac999a5ec54c-2
37:08.220 --> 37:08.600
word.

ab03d360-71f7-48fe-a0c2-80e557296adf-0
37:09.880 --> 37:13.075
You account for that variation
by saying, look, when you

ab03d360-71f7-48fe-a0c2-80e557296adf-1
37:13.075 --> 37:16.495
account for all of this, your
accuracy rate is I'm picking a

ab03d360-71f7-48fe-a0c2-80e557296adf-2
37:16.495 --> 37:17.280
random number.

abb57e45-a491-4f95-8861-c7d8e930ee02-0
37:17.280 --> 37:19.160
This is not reflective of what
we found.

25e6632d-e2bb-4424-8455-77d966a046bc-0
37:20.120 --> 37:21.800
Your accuracy rate is say 95.

e9ea384f-befd-4726-b4d2-9960b43f58b8-0
37:22.080 --> 37:25.440
Then that becomes your bar that
that 5% that you didn't get

e9ea384f-befd-4726-b4d2-9960b43f58b8-1
37:25.440 --> 37:28.968
accurate has accounted for that
variation and you want that as

e9ea384f-befd-4726-b4d2-9960b43f58b8-2
37:28.968 --> 37:30.200
part of your data set.

bb83b7ba-c911-48ac-9686-d4a76dcf8f16-0
37:31.040 --> 37:34.938
You can't train your data set on
perfection and then expect it to

bb83b7ba-c911-48ac-9686-d4a76dcf8f16-1
37:34.938 --> 37:37.360
perform better in your live
environment.

af8a29a7-bf17-43a1-a4ee-a155174e9528-0
37:37.600 --> 37:40.452
You it needs to know what is
bad, what is good for you to

af8a29a7-bf17-43a1-a4ee-a155174e9528-1
37:40.452 --> 37:41.240
learn perfectly.

c08f3aa6-f577-400b-838c-1e196856909b-0
37:41.600 --> 37:43.480
So you want that variation in
your data set.

65b96e1c-f12e-438e-a83b-aac9f6ed5311-0
37:50.040 --> 37:50.320
Thank you.

90ff8724-e8ae-473e-9955-bc7b81291e39-0
37:50.320 --> 37:50.680
Thank you guys.