Legility was challenged to quickly and cost-effectively review 62,400 documents (1.1 million pages) produced by the opposing party to assist counsel in preparing for depositions and pre-trial motions in a matter arising under the Hatch-Waxman Act. Due to the large page count, high richness (90%+), and large volumes of substantively duplicative material, a linear page-by-page review of the document set would be extremely costly and unlikely to result in the focused set of documents needed by counsel to support its case effectively.
Those terms helped a small team of Legility review attorneys with Hatch-Waxman experience identify the best set of documents (generally 50-100 documents) to inform these legal theories. This team developed tiered terms for each issue and evaluated each on the following factors:
- Value level (or heat) of the term
- Number of documents identified by the term, and, if the terms resulted in more than 1,000 documents
- The likely richness of the documents for the particular theory or issue.
Additionally, metadata was limited (many of the documents were scanned copies) to the extent the source path/file location of the collected document was used to assist our attorneys in “soft” categorizing the corpus to further focus and refine the nature of the review. In sum, Legility worked with outside counsel to tailor the review to provide several deliverables:
- The top 50-100 significant document per issue/theory identified by counsel
- Short document summaries for each of these documents
- Bulk issue categorization for all other documents
- Summary memo, factual timelines and indices on demand
For the significance review, Legility worked with supervising counsel to create and prioritize three tiers of search terms:
Tier 1 terms are generally understood as “Hot” terms. These terms are, on their face, likely to be very focused toward specifically identifying the particular theory/issue and unlikely to identify extraneous materials. These search terms also bring back relatively low document counts. If counts are high, the terms are refined or demoted to lower tiers. (In many matters, Legility will identify “Tier 1A” terms, which are terms that are so specific they need little to no pre-validation and can be used to push documents into review immediately.)
Tier 2 terms are strong terms that more likely than not bring back documents related to the particular theory/issue (i.e. > 50% richness). These terms usually pull in document sets over 500-1,000 documents, but are usually worth capping at under 2-3% of the overall corpus.
Tier 3 terms are generally terms that exist as part of a “deep dive” approach toward an issue. These terms have significantly lower richness (<50%) and often as a result are broad and bring back large document counts (up to 10% of the production corpus). Legility recommends only reviewing documents for these terms if an issue is particular sensitive or higher tier term identification is difficult.
Legility attorneys ran and updated search-term hit reports throughout the term development and provided those to supervising counsel throughout the tier term development process. For this matter, and because of the massive page count and lack of metadata, Legility recommended and executed an initial review of a 98/2 sample of the production set (resulting in 4,000 documents initially reviewed). This sampling allowed Legility to develop a heat map of issue clustering throughout the production and develop a set of supplemental search terms based on the opposing party’s own terminology. Simultaneously to performing initial sampling, Legility attorneys worked with supervising counsel to develop an initial set of Tier 1 (Hot) terms, resulting in approximately 3,000 documents for initial significance review.
As the Tier 1 term review began to yield results, Legility further adapted and supplemented the Tier 2 key terms. Ultimately, the Tier 2 terms resulting in approximately 11,000 additional documents. In consultation with supervising counsel and the client, Legility reviewed these documents as well.
Concurrently with the significance review, Legility worked to bulk categorize the overwhelming majority of documents in the production corpus as to issue/legal theory. This bulk categorization was performed by applying broad issue tags using search terms, metadata analysis and limited sampling. Specifically, this categorization allowed supervising counsel to understand in general what was produced by the other part from a volumetric perspective and to focus future searches within these categories. The categories included clinical studies, marketing-sales, adverse event reports, formulation, patents, prior art, whether documents had no relevant information, and more.
Throughout the review, Legility provided daily reports to the client and supervising counsel, tracking progress and document counts by issue. Legility also met daily with supervising counsel to address questions, provide substantive finding updates and update progress. Ultimately, it was determined at the conclusion of the Tier 2 review that review of the Tier 3 terms was unnecessary, and Legility was able to complete the review. Overall, only one-third of the corpus was attorney reviewed (i.e. <20,000 of the 60,000+ produced), and the review was completed in only 2 1/2 weeks (ahead of schedule and under budget).
Faster case study development.
Outside counsel was able to develop case strategy faster because the most useful documents were prioritized for review.
By compiling target search terms, Legility provided supervising counsel with the most useful set of documents for its case strategy very quickly. Supervising counsel determined that the most useful documents identified using the Tier 1 and Tier 2 key terms were sufficient, and additional searching and review was not needed to supplement its strategy. Ultimately, Legility reviewed less than one-third of the production set to identify a sufficient number of useful documents for case strategy and deposition preparation.
Three tiers of search terms were developed to reduce the corpus - making for more manageable review and significant cost savings.