This site uses cookies to provide you with a more responsive and personalized service. By using this site you agree to our use of cookies. Please read our cookie notice for more information on the cookies we use and how to delete or block them.

Bookmark Email Print this page

Leveraging predictive coding to reduce document review costs and increase efficiency

How a company saved money & improved review efficiency by leveraging an analytical process that helped to prioritize documents


DOWNLOAD  

Abstract

A global sports and entertainment company needed to comply with a Foreign Corrupt Practices Act investigation. Using initial search terms that were negotiated between counsel and the Department of Justice resulted in more than 235,000 documents for review. In addition, linear review on the search term hits initially achieved a responsiveness rate of just 3 percent. Faced with the prospect of spending significant client resources to review material mostly unresponsive to the needs of the matter, counsel needed a solution to achieve better results and improve efficiency.

The Deloitte Discovery project team leveraged Dynamic Review, Deloitte's proprietary advanced analytics process, to help prioritize the review of documents and increase efficiency.

Using Dynamic Review, Deloitte was able to increase the initial responsiveness rate from 3 percent to 52 percent among the materials indicated as most likely to be responsive. Focusing on the materials likely to be responsive allowed counsel to set aside a significant portion of the materials that otherwise would have been reviewed, resulting in a linear review savings of approximately $100,000. Additionally, Dynamic Review demonstrated that significant or “hot” documents for the matter were escalated for review ahead of non-responsive materials.

The Challenge

The client was faced with an ongoing linear review, but became convinced that technology could help improve the review process. Upon deciding to use technology, the client sought an analytics process that would effectuate review efficiencies and achieve linear review cost savings, all without stopping or interrupting the ongoing linear review.

How We Helped

Client's counsel commenced linear review on the keyword search term hits and initially achieved a responsiveness rate of just 3 percent. At its height, counsel was able to increase its linear review responsiveness rate to 7 percent. Faced with the prospect of spending significant client resources to review material mostly unresponsive to the needs of the matter, counsel met with Deloitte to discuss options.

Deloitte's discovery team leveraged previously-reviewed documents to create a training set for its Dynamic Review process. From this training set, our proprietary software created a model for the remainder of the document corpus. To measure the efficacy of this model, we compared counsel's coding of a sample set of documents, known as the “validation set,” to the Dynamic Review model and identified the documents where the model disagreed with counsel's coding. After ongoing collaboration with counsel through a series of iterations, agreement was reached on the ideal model. The refined model was then used to score the entire remaining review population. The Deloitte team used the latest scores to target the documents being reviewed, prioritizing the documents that the model indicated were most likely to be responsive.

At the height of its modeling effectiveness, Dynamic Review achieved a 52 percent responsiveness rate among the materials indicated as most likely to be responsive.  This enabled counsel to focus on the responsive materials first, relegating an ever-growing group of documents to a batch that did not merit review. Upon exhausting the documents with a reasonable likelihood of being responsive, counsel and Deloitte collaborated to vet the remaining group, statistically validating a responsiveness rate of less than one tenth of one percent (0.1%). With a predicted linear review cost of over $1,000 per responsive document, counsel was armed to discuss setting this group aside with the regulatory authorities.

By not reviewing this material, which constituted 38% of the documents responsive to search terms, counsel saved its client approximately $100,000 of linear review cost.

Several key factors contributed to the success of the project:

  • Ongoing consultation with counsel and iteratively training the model succeeded in prioritizing the most relevant documents earlier in the review - without interrupting the ongoing review process.
  • By using Dynamic Review to prioritize the manual review effort, Deloitte's document review team was able to escalate “hot documents” to counsel's attention more quickly than they would have been able to by just leveraging linear review.
  • Implementing a statistical sampling process on the population of documents the model predicted were the least likely to be responsive eliminated the need for “eyes on” document review for more than 90,000 documents, reducing the number of documents requiring review by 38 percent.

Solution

Through collaborative consultation between counsel and Deloitte, the client was able to achieve approximately $100,000 in cost savings. In addition, there was no interruption of the existing linear review, but instead the incorporation of Dynamic Review was able to bolster the review efforts and improve the efficiency of the overall process.

Contacts

Steve Shebest
Director
Deloitte Transactions and Business Analytics LLP
+1 202 220 2631
sshebest@deloitte.com

Derrick C. Lee
Manager
Deloitte Transactions and Business Analytics LLP
+1 212 937 8231
derrlee@deloitte.com


As used in this document, “Deloitte” means Deloitte Transactions and Business Analytics LLP, an affiliate of Deloitte Financial Advisory Services LLP. Deloitte Transactions and Business Analytics LLP is not a certified public accounting firm. Please see www.deloitte.com/us/about for a detailed description of the legal structure of Deloitte LLP and its subsidiaries. Certain services may not be available to attest clients under the rules and regulations of public accounting.

Related links

Share this page

Email this Send to LinkedIn Send to Facebook Tweet this More sharing options

Stay connected