Black Logo

Our site uses cookies to give you the best user experience and to collect and share information for analytics, advertising and personalization on this and other sites. Please select whether you consent to our use of cookies and related technologies (“Cookies”), as described in our Cookie Policy. You can return at any time from the same web browser to update your preferences. Please note that resetting your browser’s cookies will reset your preferences. You can control the use of some types of cookies through the Cookie Settings below, but note that if you choose to disable certain cookies, it may limit your use of certain features or functions on our services and websites.

Privacy Policy
Strictly Necessary Cookies

These cookies are required to enable core site functionality.

Functionality Cookies

Functionality cookies allow us to provide enhanced and more personalized content and features. In order to permit your connection our website, our servers receive and record information about your computer and browser, potentially including your IP address, browser type, and other software or hardware information. All of these features help us to improve your visit and assist in navigation of the sites’ features.

Analytics Cookies

We and our service providers may use analytics cookies, which are sometimes called performance cookies, to collect information about your use of our website, for instance, which pages you go to most. The information allows us to see the overall patterns of usage, help us record any difficulties users may have while using our website and can show us whether or not our advertising is effective.

Advertising And Targeting Cookies

We may use third party advertising and targeting cookies to correlate your use of our website to personal information obtained about you so that we may more clearly target the information we provide you to the specific items we think you will find interesting, based on your prior online activities and preferences. We also may use these cookies to deliver ads that we believe are relevant to you and your interests.

For more information, view our Cookie Policy

Client Successes

Pharma Case Study: Large, Subtle Review with Tight Deadline

The Challenge

Legility was challenged to quickly and cost-effectively conduct a legal review of more than 500,000 documents in response to requests for production of a branded pharmaceutical maker in a matter arising under the Hatch-Waxman Act. The documents were collected and reviewed in two sequential phases – loose electronic document (eDoc) collection (80,000) and email collection (420,000) – and produced on a rolling basis. For this matter, the 500,000-plus culled and deduplicated documents were an even, subtle mixture of 1.) responsive, 2.) responsive – do not produce per objection or special production decision (R-DNP), and 3.) non-responsive documents pertaining to the brand drug at issue. The Legility review needed to be completed under a very tight time window (less than one month) to meet the substantial production deadline.

The Solution

Because of the tight deadline and the need for lead counsel to start mounting its case quickly, Legility attorneys performed an early case assessment (ECA) to identify culling terms for non-linear review and to prioritize likely responsive documents to review first.

Legility’s process consisted of a multi-step strategy in identifying documents for review to get attorneys moving into important documents as soon as possible. First, Legility performed sampling to determine the expected relevancy rate/overall richness of the collection. Second, working with supervising counsel, Legility identified search terms most closely correlating to responsive, responsive
– do not produce (R-DNP), and non-responsive documents. Third, Legility evaluated the source metadata of the documents along with search term culling and sampling to eliminate large groups of documents from substantive review.

Other Legility Tasks:

Performed a 95/5 sample (less than 400 documents) to determine the overall richness of the corpus was a 35%/35%/30% split between responsive/R-DNP/non-responsive documents

Identified useful search terms for likely (1) responsive; (2) R-DNP (3) non-responsive documents at multiple tiers of richness from the sample and in consultation with counsel

Executed domain and metadata analysis, particularly on the electronic document collection, to prioritize different groups of documents for review

Created and validated searches for non-production level documents and after confirming that the documents were removed from the workflow prior to manual review

Used Relativity Analytics across the email collection to provide a threaded review of the emails

Performed comprehensive quality assurance by sampling and targeted searches of responsive documents for each rolling release; this was so supervising counsel could check Legility’s coding accuracy; samples of non-responsive documents were also provided to supervising counsel.

Tracked progress via a team dashboard and rate metrics throughout the review, which allowed Legility to provide daily reporting. The daily reports highlighted review progress, nature of discovered materials and projected completion rates.

The full scale of the review was not known until well after the review began. An unmovable deadline required Legility to act quickly to scale the team (to a peek size of 45+ review attorneys). Because of Legility’s detailed metrics and knowledge keeping, the company was able to move quickly and precisely in upscaling the team to meet the new demand. The team was able to scale from 10 to 45 people in just 48 hours and seamlessly bring the additional reviewers up to speed on the matter without burdening an already stretched supervising counsel. The full team was fully executing at capacity within three days of scale up.


Faster case strategy development.

Outside counsel was able to develop case strategy faster because the most likely responsive documents were promoted for review first.

Cost savings.

Due to the application of threading analytics, concept clustering and the elimination of documents pre-review, Legility was able to keep the overall cost of review within budget estimates, despite the high richness.

Defensible results.

The Legility team documented the process, placed eyes on every production as required by outside counsel, and provided expert defense in the methodology and validity for the review.

Unmovable deadline required Legility to scale the client's team from 10 to 45 in just 48 hours.


Number of documents to be reviewed

The right people, process and technology deliver the right results.

About the author


Legility, a leading provider of technology-enabled legal services, provides consulting, technology, managed solutions, and flexible legal talent to corporations and law firms. The company has more than 1,000 lawyers, engineers, consultants, technology and data specialists, and operational experts serving more than one-third of the Fortune 100 and one-quarter of the Am Law 200. Legility helps its clients improve operational efficiency. By combining people, processes, and technology, Legility offers innovative and bundled solutions that align with how the legal market is increasingly looking to engage.

Subscribe to Insights

Your one-stop shop for the Legility logo & more.

Your one-stop shop for the Legility logo, brand guidelines, photography assets, and more.

Subscribe to insights