Posted on: December 28, 2016in Blog
Predictive Coding vs. Search Terms: Who Determines the Method of Review?
This blog was originally published on The Daily Record.
Two recent cases in two different courts dealt with the issue of technology assisted review (“TAR”) and its use in discovery. At issue in each case was the plaintiffs desire for defendants to use TAR instead of relying on search terms to identify relevant documents. Both courts denied the requests.
Technology Assisted Review in eDiscovery
Before we go further I will provide a brief refresher on TAR and its place in the electronic discovery process. TAR, or predictive coding as it’s sometimes called, is a technology and a process that involves a subject matter expert (a.k.a. human) coding documents as responsive or non-responsive. Those decisions are fed into predictive coding software and after a number of iterations the software is able to determine the responsiveness level of the remaining documents.
Yes, this is more of the “machines are replacing humans” you have been reading about, which has been happening since the Luddite riots. TAR has been around for a number of years and has been widely used by parties on all types of matters. Its application has been proven in many studies and in practice to be a more efficient and effective manner of identifying and producing relevant documents compared to reliance on key terms. Many courts have advocated and celebrated the use of predictive coding. One of those courts is the same who denied the plaintiffs request to force defendants to use it.
Hyles vs. New York City
In Hyles vs. New York City (S.D.N.Y 08/01/2016), Plaintiff requested the Court to force the City to utilize TAR. Plaintiff proposed that the City should use TAR as a “more cost-effective and efficient method of obtaining ESI from Defendants.” The City claimed that using TAR would not be cost-effective and due to previous disagreements on discovery issues, felt that coming up with an agreeable set of seed documents would be difficult. They staunchly opposed the use of TAR and wanted to utilize keywords and conduct a traditional document review for responsiveness and privilege.
Judge Peck, Predictive Coding, and the Obligation to Determine Document Review Workflow
The judge in the matter, Andrew Peck, is a strong proponent of TAR and agreed with the plaintiff that it is far superior to key terms. Judge Peck stated: “Hyles absolutely is correct that in general, TAR is cheaper, more efficient and superior to keyword searching.”
However, Judge Peck also strongly believes in the Sedona Conference Principles and referenced the sixth principle that states: “[r]esponding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.” He went on to state that “it is not up to the Court, or the requesting party (Hyles), to force the City as the responding party to use TAR when it prefers to use keyword searching. While Hyles may well be correct that production using keywords may not be as complete as it would be if TAR were used, the standard is not perfection, or using the ’best’ tool, but whether the search results are reasonable and proportional.”
If it’s a fact that TAR is a more efficient and cost-effective process for discovery, then why would any party refuse or be reluctant to use it? Maybe the value of the case is too small to warrant the use of TAR. While TAR is more cost-effective, those savings typically occur during the review phase. The processing and application of predictive coding technology is often pricier than indexing a document set in preparation for keyword searching.
If you have a small dataset or access to inexpensive review resources, then TAR may not be the path for your matter. It also may be that TAR is seen as black-box technology or an esoteric process that attorneys don’t trust or understand. Maybe they want eyes on every documents and using keywords is like a warm blanket and hot cocoa on a cold winter’s night. Or perhaps a party already has what it believes to be a sound process for identifying and reviewing documents... A process similar to Pfizer’s.
A Responding Party Can Choose its Method of Document Selection
Pfizer’s Viagra products liability litigation, an order was issued by the Court (N.D. California) in October of this year regarding the use of TAR. Once again the plaintiffs requested that the Court force “Pfizer to use technology assisted review (“TAR”) and/or predictive coding with Plaintiffs' input to identify the locations of relevant information and the responsive ESI from those locations. Plaintiffs argue that TAR and/or predictive coding is a more sophisticated tool than the traditional search term or search query approach, and that using that suggested approach would save time and money for both sides.”
This sounds like a lot like the Hyles matters and it warms my heart that the plaintiffs are concerned with Pfizer saving money. However, there is a cost saving potential for parties on the receiving end as well. Pfizer’s use of TAR may result in a smaller number of documents produced to the plaintiffs, which lessens their cost of review. But maybe Pfizer has found a better way or one that it feels works better than TAR. Their process is outlined in the order:
“Pfizer describes its preferred methodology an iterative process—albeit not the same iterative process as TAR and/or predictive coding—in which Pfizer tests search terms and validates them using rigorous sampling of potentially responsive documents. Initially, Pfizer tests and validates a number of search terms by applying those terms to a sample set of documents, reviewing the documents for responsiveness, and verifying that the search terms yield high rates of response.”
Effective perhaps, old-school perhaps, but a process nonetheless. And one that seems to work for Pfizer.
In the end the Court denied the Plaintiff’s request and cited the Hyles matter.
“As the court reasoned in Hyles, the responding party is the one best situated to decide how to search for and produce ESI responsive to discovery requests. The responding party “can use the search method of its choice. If [the propounding party] later demonstrates deficiencies in the ... production, the [responding party] may have to re-do its search. But that is not a basis for Court intervention at this stage of the case.”
Making TAR the Standard
Whatever the reason(s) the defendants in these two matters had for not using TAR, both courts decided that a party can’t be forced to use one method over another. I think these decisions are sound ones. Absent proof that there has been a deficiency in discovery, a party should be able to choose its method of document selection, review and production.
However, I do feel that more parties should consider TAR and other technologies to assist with document review. The future is now and it may not be long before TAR is the norm. Judge Peck is certainly someone who would like that day to come sooner rather than later.
“The Court would have liked the City to use TAR in this case. But the Court cannot, and will not, force the City to do so. There may come a time when TAR is so widely used that it might be unreasonable for a party to decline to use TAR. We are not there yet.”
What do you think? Should courts be able to force a party to use TAR or any other method it deems superior?
D4 Weekly eDiscovery Outlook
Power your eDiscovery intellect with our weekly newsletter.
Posted March 16, 2017
7 Best Practices for a Defensible Legal Hold Process
Posted March 10, 2017
BYOD Privacy: 6 Considerations to Protect Your Business
Posted March 08, 2017
3 Benefits of Managed Review
Posted March 02, 2017
How Data Repurposing Can Lower the Cost of Discovery
Posted February 23, 2017
Women in eDiscovery Atlanta | New Data Technology Trends
Posted February 23, 2017
Corporate Internal Investigations Best Practices
Posted February 13, 2017
4 Key Internal Roles Involved with Conducting Corporate Investigations
Posted February 09, 2017
Corporate Internal Investigations: A Legal & IT Love Story [Webinar]
Posted February 09, 2017
Intellectual Property Theft: How to Ensure a Defensible Investigation
Posted February 02, 2017
Could the Amazon Echo be a New Source of ESI?