Legility was challenged to quickly and cost-effectively conduct a legal review of more than 500,000 documents in response to requests for production of a branded pharmaceutical maker in a matter arising under the Hatch-Waxman Act. The documents were collected and reviewed in two sequential phases – loose electronic document (eDoc) collection (80,000) and email collection (420,000) – and produced on a rolling basis. For this matter, the 500,000-plus culled and deduplicated documents were an even, subtle mixture of 1.) responsive, 2.) responsive – do not produce per objection or special production decision (R-DNP), and 3.) non-responsive documents pertaining to the brand drug at issue. The Legility review needed to be completed under a very tight time window (less than one month) to meet the substantial production deadline.
Because of the tight deadline and the need for lead counsel to start mounting its case quickly, Legility attorneys performed an early case assessment (ECA) to identify culling terms for non-linear review and to prioritize likely responsive documents to review first.
Legility’s process consisted of a multi-step strategy in identifying documents for review to get attorneys moving into important documents as soon as possible. First, Legility performed sampling to determine the expected relevancy rate/overall richness of the collection. Second, working with supervising counsel, Legility identified search terms most closely correlating to responsive, responsive
– do not produce (R-DNP), and non-responsive documents. Third, Legility evaluated the source metadata of the documents along with search term culling and sampling to eliminate large groups of documents from substantive review.
Other Legility Tasks:
Performed a 95/5 sample (less than 400 documents) to determine the overall richness of the corpus was a 35%/35%/30% split between responsive/R-DNP/non-responsive documents
Identified useful search terms for likely (1) responsive; (2) R-DNP (3) non-responsive documents at multiple tiers of richness from the sample and in consultation with counsel
Executed domain and metadata analysis, particularly on the electronic document collection, to prioritize different groups of documents for review
Created and validated searches for non-production level documents and after confirming that the documents were removed from the workflow prior to manual review
Used Relativity Analytics across the email collection to provide a threaded review of the emails
Performed comprehensive quality assurance by sampling and targeted searches of responsive documents for each rolling release; this was so supervising counsel could check Legility’s coding accuracy; samples of non-responsive documents were also provided to supervising counsel.
Tracked progress via a team dashboard and rate metrics throughout the review, which allowed Legility to provide daily reporting. The daily reports highlighted review progress, nature of discovered materials and projected completion rates.
The full scale of the review was not known until well after the review began. An unmovable deadline required Legility to act quickly to scale the team (to a peek size of 45+ review attorneys). Because of Legility’s detailed metrics and knowledge keeping, the company was able to move quickly and precisely in upscaling the team to meet the new demand. The team was able to scale from 10 to 45 people in just 48 hours and seamlessly bring the additional reviewers up to speed on the matter without burdening an already stretched supervising counsel. The full team was fully executing at capacity within three days of scale up.
Faster case strategy development.
Outside counsel was able to develop case strategy faster because the most likely responsive documents were promoted for review first.
Due to the application of threading analytics, concept clustering and the elimination of documents pre-review, Legility was able to keep the overall cost of review within budget estimates, despite the high richness.
The Legility team documented the process, placed eyes on every production as required by outside counsel, and provided expert defense in the methodology and validity for the review.