This site uses cookies to provide you with a more responsive and personalized service. By using this site you agree to our use of cookies. Please read our cookie notice for more information on the cookies we use and how to delete or block them.

Bookmark Email Print this page

Technology Assisted Review: Leveraging Advancements to Improve Efficiencies

Deloitte Insights video

The cost of legal document review has risen due to the significant increase of data that employees generate and store on company servers. This trend has led to the adoption of sophisticated technologies, most often referred to as machine learning, supervised learning or active learning. Watch this episode of Deloitte Insights to learn how this new technology is being leveraged for eDiscovery, and the importance of having specialized knowledgeable people and processes that allow companies to benefit from these advancements.

The latest episode of Deloitte Insights features Maura Grossman, Counsel at Wachtell, Lipton, Rosen & Katz and Carmen Oveissi Field, Principal at Deloitte Financial Advisory Services LLP . Tune in to learn more about the importance of technology assisted reviews.


Maura Grossman, Counsel, Wachtell, Lipton, Rosen & Katz

Carmen Oveissi Field, Principal, Deloitte Financial Advisory Services LLP


Sean O’Grady (Sean): Hello and welcome to Insights, where today, we will be discussing “Technology Assisted Review,” and how companies are using new technologies to reduce document review costs. With us in New York for this discussion are Maura Grossman, Counsel at Wachtell, Lipton, Rosen & Katz; and Carmen Oveissi Field, a principal within Deloitte Forensic and Dispute Services at Deloitte Financial Advisory Services. So ladies, if we go with the premise of Moore’s law that technology gets better over time we would think that eDiscovery costs should be going down, but in preparing for this segment, we found that that is not the case. That eDiscovery costs are going up and so my first question to you Carmen is how could that be?

Carmen Oveissi Field (Carmen): The short answer is it is an issue of volume. So, if you think back to when you got your first computer, and every time you installed an application or put data on it, you really paid attention to how much space you had, and now frankly it is probably not something you ever think about. And if you think about big company with hundreds and thousands of people who are just storing everything that they want to forever that when they get to a point where legal hold comes around, and people actually have to review these documents, there are a lot more to review and while technology may get more efficient, people pretty much stay the same.

Sean: So the costs have gone up because the amount of data has gone up. Maura, my thought to you is, is there something that technology can do to make this easier for all parties involved.

Maura Grossman (Maura): Well technology was part of the problem and technology needs to be part of the solution. What we are seeing today is the use of sophisticated computer algorithms to help with some of the heavy lifting. So, instead of having a room full of attorneys reviewing hundreds of thousands of documents one at a time, we have lawyers review a small subset of the documents and then the computer does the heavy lifting by applying what it learns from those documents to the larger dataset.

Sean: Now, is there a specific software or technology that is being used, could you elaborate a little bit on it.

Maura: Well these technologies are often called machine learning or supervised learning or active learning. And these technologies have actually been around for quite some time. They have been used in other applications, such as spam filtering or in the law enforcement surveillance realm. So they are really not new, what is new is the application in the eDiscovery world.

Sean: So you are mentioning training, Carmen how do you go about training a computer.

Carmen: Well think about it this way. So when I log on to the New York Times, there is a section at the bottom that is called – recommended for you, and so every time I log on and I click on an article it learns about me. It learns what types of things I would like to read, and so over time it creates a profile on me and then starts recommending articles. And I think one of the key pieces here and the same way it is important on the doc review side, is that it is not a one-step thing, every time I log on, every day that I go back it is learning more. So, it is an iterative process. So, the computer is learning what kind of articles I like and then serves them up for me.

Sean: I guess the next question is why now. So, if we are seeing the application of this technology in other areas like the example you just gave, why is it new in the eDiscovery space.

Carmen: Short answer it is kind of a complicated stuff, you know what we are used to using with technology is kind of more simplistic, keyword searching. So, you type in a word and it finds every instance of that word in all the documents. Frankly, it is pretty easy to explain and if you are the person who has to get up in front of a judge, and explain that technology – probably not so hard, but when you get into these really complicated searching algorithms and heavy math and they talk about the black box, it is much more intimidating to try to explain that and get people to really understand why you should be doing that instead of using old school keyword technology.

Sean: So Maura, if it’s been intimidating in the past what’s changed, why is now the time to use this type of technology?

Maura: Well two things, first the technology is now being applied in eDiscovery and we have had some time to experiment but the primary reason I think is that there are now two cases where courts have actually approved the use of this kind of technology. We have one at the Southern District New York and one from a state court in Virginia where the courts have actually said this is a reasonable way if you put in place a sound process with quality control and testing that it is perfectly appropriate to use this kind of technology. So, nobody has to be that first guinea pig anymore.

Sean: Now, earlier Carmen had said that keywords aren’t going to get you there. So, how do you establish that kind of a comfort for attorneys to use this technology and to feel more comfortable with it.

Maura: Well partly, there have been a number of studies that have shown that this technology is actually not only better than keywords but it is better than actual attorneys looking at documents. There are several studies, I did one with Professor Gordon Cormack that is in the Richmond Journal of Law and Technology where we actually took attorneys who were either third-year law students or professional contract reviewers and we put them back-to-back with the technology and not surprisingly the technology won. It found not only more of the documents; it made fewer errors at 50 times the efficiency. So, this is a no-brainer.

Sean: A no-brainer, so does that mean we can just hit the button and folks have what they need.

Carmen: I wish it were that easy. I mean, think about it like these gentlemen here with the cameras. It is incredible technology, but they are very skilled at their trade. They know how to use the technology. So, it is not just about the technology but really specialized knowledgeable people who know how to use it, along with process that works in concert with the technology. If you don’t have the right people and process you know that great camera isn’t going to get great shots of us.

Sean: Thank you very much that makes sense to me. I guess from the last question, it is a clarifier, and that is, is this technology just about discovery or there other components to it?

Carmen: Well I think in the context of discovery we can use it in some other ways other than just trying to find the magic documents. You know if you think about maybe for quality control. So for example, maybe a person is only going to be comfortable with human reviewers, okay fine but let us just see if the technology can maybe find the flaws in what people did. So, from a quality control perspective it can come in and look for things where maybe the humans didn’t get it right, or frankly if you don’t have a lot of time and you are on a really limited schedule you can use technology frankly to tell you which documents to look out first, so kind of prioritization as well.

Sean: Very interesting, it sounds like all technology that we discussed on this program, it sounds like it is growing and budding. So, thank you for helping us to (?)(06:28) down a bit here today.

Carmen: Thank you.

Sean: You are welcome. Okay we have been discussing “Technology Assisted Review” with Carmen Oveissi Field, a principal within Deloitte Financial Advisory Services, and Maura Grossman, counsel at Wachtell, Lipton, Rosen & Katz. If you like to learn more about Carmen, Maura, or any of the topics discussed on today’s broadcast, you can find that information on our website. It is For all of the good folks at Insights, I’m Sean O’ Grady. We’ll see you next time.

Join the Conversation


Share this page

Email this Send to LinkedIn Send to Facebook Tweet this More sharing options

Stay connected