Managing ESI Burdens

Managing ESI Burdens

Rule 26(b)(1) puts electronic data through the same proportionality test as paper. A party may discover nonprivileged matter that is relevant to a claim or defense and proportional to the needs of the case, and proportionality turns on the importance of the issues, the amount in controversy, the parties' relative access to information, their resources, the importance of the discovery, and whether its burden or expense outweighs its likely benefit.¹ Rule 26(b)(2)(C)(i) adds a second check: discovery is open to limitation when it is unreasonably cumulative or duplicative, or when another source can supply it more conveniently, less burdensomely, or less expensively.² The earlier sections applied both checks to sources that are hard to reach. They matter just as much for data that is fully accessible, because a live e-mail server can hold so much that an unfocused request imposes a heavy burden although no source is out of reach.

Three tools keep a request proportional in a large electronic collection. Search terms decide what gets reviewed, predictive coding shrinks the review itself, and metadata and database requests raise the question of what form of data a requester actually needs. All three work best under good project management, which includes planning, cooperation, technological tools, sound supervision, clear communication, and accountability, and which is itself a recognized way to reduce cost and maintain control.³

Many decisions below predate the amended Rule 26(b)(1), and the amendment changes how their reasoning is stated. Older opinions quote the cost-benefit comparison as a free-standing limit in Rule 26(b)(2)(C)(iii).⁴ That clause now directs the court to limit discovery outside the scope that Rule 26(b)(1) permits, and the proportionality comparison sits inside that scope.⁵ The amended rule also drops the phrase “reasonably calculated to lead to the discovery of admissible evidence,” and one commentator cautions that many earlier cases might have come out differently under the new text.⁶ Read each opinion for its reasoning about burden, and restate the weighing under the current rule.

Search Terms and Protocols

A party that holds a large body of e-mail seldom reads all of it. It runs a list of words against the collection, treats each matching document as a hit, and has lawyers review the hits for relevance and privilege. The terms therefore set the cost: each word either narrows the review to documents that matter or widens it to documents that do not. That is why Rule 26(b)(1)'s question, whether the burden or expense of the discovery outweighs its likely benefit, is often answered in practice when the list is drafted.⁷

Courts begin from a simple allocation of the burden. One judge put it this way: when parties request relevant information in good faith and the requests are not unduly burdensome, discovery proceeds, and the party resisting production bears the burden of establishing lack of relevancy or undue burden.⁸ A responding party that objects to a search must therefore bring facts about what the search would cost and what it would return.

Hit counts show how fast a poorly built search grows. In one dispute the defendant complained that the plaintiff had insisted on searching nineteen employees' e-mail with common words as search terms, and that the searches had produced 59,635 documents, only a minuscule portion of them relevant.⁹ The two choices compound each other. Every added custodian enlarges the pool, and every common word matches routine messages in that pool.

Da Silva Moore v. Publicis Groupe states the working standard for lawyers who rely on keywords. At a minimum they must craft the keywords carefully, with input from the custodians about the words and abbreviations those custodians actually use, and they must quality-control test the methodology to assure accuracy in retrieval and to eliminate false positives, meaning documents that match a term but have nothing to do with the dispute. The court added that every lawyer, including those who did not come of age in the computer era, should understand this.¹⁰ Custodian input matters because people write in shorthand and nicknames that a lawyer working from the pleadings would not guess. Testing matters because only a run against real data shows whether a term catches the right documents. Testing on a sample has precedent in the backup-tape cases: the court in Zubulake limited the first round of discovery to a sample of five tapes, which gave the parties and the court a basis for deciding whether more e-discovery, and more cost, was likely to turn up relevant evidence.¹¹

I-Med Pharma Inc. v. Biomatrix, Inc. supplies the most useful checklist for judging a protocol. The court there asked five questions:¹²

1. How broad is the search, and is it restricted to specific computers, file systems, or custodians?
2. Does the search carry date restrictions?
3. Do the terms include proper names, uncommon abbreviations, or other words unlikely to occur in irrelevant documents?
4. Do operators such as “and,” “not,” or “near” restrict the universe of possible results?
5. Could the number of results practically be reviewed, given the economics of the case and the amount of money at issue?

The first four questions are levers a drafter can pull, and the fifth is the test the levers serve. Limiting custodians, sources, and dates shrinks the pool before any term runs. Distinctive terms and operators shrink what each term catches. The fifth question then asks whether what remains can be read at a price the case can carry, which is the comparison of burden to benefit applied to a hit list.¹³

A search protocol is also an agreement, and a party can be held to it. In In re Fannie Mae Securities Litigation, a stipulation gave the individual defendants the sole right to specify search terms. The D.C. Circuit read it that way, with no limit on the number or scope of the terms, rejected the agency OFHEO's argument that the stipulation's reference to “appropriate search terms” qualified that right, and affirmed the finding of contempt.¹⁴ The case shows the danger of letting one side designate search terms with no check on whether they are proportional to the needs of the case.¹⁵

SEC v. Collins & Aikman states the responding party's side of the duty. The Commission raised legitimate concerns about the burdens of particular requests, but the court held that it could not unilaterally decide that those burdens outweighed the defendants' need for discovery. It had to negotiate in good faith and craft a search protocol that would retrieve responsive information without an unduly burdensome expense disproportionate to the size and needs of the case, and the court directed the parties to cooperate in setting the scope and design of a search of the Commission's e-mail.¹⁶

A responder that offers only the conclusion that review would cost too much has done none of that. It has proposed no terms, tested nothing, and documented no cost. Nor has it made the showing that Rule 26(b)(2)(B) requires of a party that calls a source not reasonably accessible because of undue burden or cost, which the responder must make on a motion to compel or for a protective order.¹⁷ The cost of reading the hits from a source that can be searched is a burden objection under Rule 26(b)(1), to be measured with facts. It is not proof that the source cannot be reached.

The duty to negotiate has a practical reason: each side holds half of what a good list needs. The responder knows its systems, which custodians hold what, and how its documents are labeled. The requester knows what it is looking for and which words its theory depends on. A responder that may simply refuse never learns what a narrower search would cost, and a requester that may dictate terms makes the responder pay to review words chosen without knowledge of the data. Requesters carry their share of the duty. One court described the discovery rule's aim as eliminating “kneejerk discovery requests served without consideration of cost or burden to the responding party.” It observed that lawyers customarily serve requests far broader, more redundant, and more burdensome than necessary to get the facts that resolve a case, and it said that lawyers who meet and confer before serving discovery can discuss the amount in controversy and how much discovery, of what type and in what sequence, the case needs, so that the cost to all parties is proportional to what is at stake.¹⁸

Negotiation also runs out of time. In Oxbow, a party asked to renegotiate after the court had found production appropriate, and the court called the proposal untenable. The parties had had months to negotiate without resolution, further delay was not justified once production was found appropriate, and the request to renegotiate contradicted Oxbow's own earlier position that renegotiation would not be productive and that attorney time spent on it might exceed any marginal savings.¹⁹ A protocol is easiest to reshape before the parties agree to it and before production begins, and a party that proposes renegotiation only after losing on production may find the court unwilling to reopen the question.

Predictive Coding

Predictive coding is a form of computer-assisted review aimed at the cost of reading every document. Counsel code a seed set of documents, software trained on that coding classifies the rest, and the parties sample the results, including the documents the software treats as irrelevant, to check whether it is working and whether it needs retraining.²¹ Courts compare it with the alternatives, which are linear manual review and keyword searches, and the comparison is the starting point for deciding when the method is appropriate.

Da Silva Moore v. Publicis Groupe is the leading approval. The court concluded that computer-assisted review appears better than the available alternatives and should be used in appropriate cases, conceded that it is not perfect, and noted that the Federal Rules do not require perfection.²² The same court warned that it is “not a magic, Staples-Easy-Button, solution appropriate for all cases,” and that what courts must examine is the process used and the interaction of man and machine.²³

Two measures give “better” a meaning. Recall is the fraction of the relevant documents that a review identifies, so it measures completeness. Precision is the fraction of the identified documents that are relevant, so it measures accuracy. A method succeeds when it produces higher recall and higher precision than another method, at a cost proportionate to the value of the case.²⁴

The court approved the method after weighing five circumstances:²⁵

1. The parties agreed to its use.
2. The volume of ESI was vast, more than three million documents.
3. Computer-assisted review was superior to the available alternatives, linear manual review and keyword searches.
4. The case called for cost effectiveness and proportionality under Rule 26(b)(2)(C).
5. The producing party proposed a transparent process.

Each circumstance does separate work. Agreement made the decision relatively easy, since the parties disagreed only about how to implement the review and not about whether to use it.²⁶ Volume matters because the cost of manual reading rises with every document added, while the effort of training and checking the software is largely fixed, so the savings grow with the size of the collection. Superiority supplies the comparison without which no method can be called proportionate, because a technique is justified only to the extent that it beats the next best one. Proportionality was framed under Rule 26(b)(2)(C), and the same cost-benefit comparison now sits in Rule 26(b)(1), to which clause (iii) points.²⁷

Transparency does the subtlest work. The software learns from counsel's coding, so a coding error repeats across the collection, and a requester who sees only the output cannot detect it. The producing party, MSL, therefore proposed a transparent process that included a meet-and-confer to resolve any disagreements about the coding applied to the documents in the seed set.²⁸ The court said such transparency lets the opposing party and the court be more comfortable with computer-assisted review and reduces fears about the so-called “black box” of the technology. It acknowledged that not every experienced ESI lawyer thinks so much openness is necessary, and it highly recommended that counsel in future cases at least be willing to discuss it.²⁹

Approval of the method does not approve every choice made while using it. MSL proposed to stop review at 40,000 documents, and the court rejected the proposal as a “pig in a poke.” It explained that where the line is drawn depends on what the statistics show for the results, because proportionality requires consideration of results as well as costs, and a cutoff that left a tremendous number of likely highly responsive documents unproduced would not work.³⁰ The cost side of the comparison could be priced in advance, but the benefit side could not, because no one yet knew how many likely responsive documents lay beyond the 40,000th. Cost remains a factor under the rule, but the court held that it cannot be considered in isolation from the results of the predictive coding process and the amount at issue in the litigation.³¹

The same reasoning keeps quality questions open until real results exist. Whether a final sample of the documents the software rejected contains any “hot” or “smoking gun” documents matters. One highly relevant document may require retraining the software or switching to another search method, while several documents that add nothing to the case might not matter. The court said such questions are better decided down the road, when real information is available to the parties and the court.³²

Dynamo Holdings Ltd. P'ship v. Commissioner of Internal Revenue carried the question into the Tax Court. The court noted that the Tax Court rules at issue resemble Federal Rules 34(a)(1)(A), 34(b)(2)(E), and 26(b)(2)(B), and, because it had not previously addressed computer-assisted review tools, it took the question up. It added that a requesting party who believes the final response is incomplete, and can support that belief, may file another motion to compel.³³ Because the Tax Court applies its own rules, the decision persuades a federal court but does not bind it.

Metadata and Database Requests

A request for metadata passes through the same gate as any other: the information must be nonprivileged, relevant to a claim or defense, and proportional to the needs of the case.³⁵ Metadata is information stored about an electronic file, such as when it was created, when it was last changed, and who changed it. One court explained that metadata is discoverable if it is relevant to a claim or defense and not privileged, and that its discovery is also subject to the balancing test of Rule 26(b)(2)(C), which requires the court to weigh the probative value of the proposed discovery against its potential burden.³⁶ That opinion speaks in the language of the rules before the amendment. A modern argument restates the balance as Rule 26(b)(1) proportionality, with clause (i) of Rule 26(b)(2)(C) still available against cumulative requests.³⁷

Form of production is where metadata fights usually begin. If the requester names no form, the responding party must produce ESI in a form in which it is ordinarily maintained or in a reasonably usable form. A party may choose another reasonably usable form, but it may not convert the information into one that makes it more difficult or burdensome for the requester to use efficiently, and ESI kept in an electronically searchable form should not be produced in a form that removes or significantly degrades that feature.³⁸ The form-of-production dispute in EEOC v. SVT, LLC turned on the same standard: whether the format the responding party chose left the information reasonably usable.

Courts weighing metadata have often looked to the Sedona Conference, whose published Principles and commentary they treat as instructive. The first edition leaned against production. Williams v. Sprint/United Management Co. quoted two of its Principles. Principle 12 said that, unless metadata is material to resolving the dispute, there is no obligation to preserve and produce it absent agreement of the parties or order of the court. Principle 9 said that, absent a showing of special need and relevance, a responding party should not be required to preserve, review, or produce deleted, shadowed, fragmented, or residual data.³⁹ The commentary went further and described a modest legal presumption against producing metadata, on the reasoning that most metadata has no evidentiary value and that the time and money spent reviewing it would be wasted, though it conceded that relevant metadata should be produced.⁴⁰

The second edition reversed course on the presumption. Principle 12 and its commentary were revised to remove any presumption against the production of metadata, and the commentary now tells parties to consider what metadata is ordinarily maintained, how relevant it is, and how important reasonably accessible metadata is to facilitating the parties' review, production, and use of the information. In choosing a form of production, the two primary considerations are the need for and probative value of the metadata and the extent to which it will “enhance the functional utility of the electronic information.”⁴¹ A decision that relies on the first edition's presumption states an earlier position, and you should not present it as the current view.

The change matters because relevance and usability are different questions. Relevance asks whether the metadata proves anything about the dispute, and for most word-processing files the answer is no, which is why a presumption against production once seemed sensible. Usability asks whether the recipient can work with what it receives, and for structured data the answer often depends on the metadata. A request is strongest when the metadata is needed to use the data and weakest when it merely describes documents the requester can already read.

Databases show the usability point most clearly. In Aguilar v. Immigration and Customs Enforcement, a civil-rights action alleging unlawful home searches by the government, the plaintiffs requested production of ICE databases, and ICE responded by offering text-searchable PDF documents. The court observed that the more interactive the application, the more important the metadata is to understanding its output, so that metadata may add little to a word-processing document but is often critical to a database application.⁴² The way the requester intends to use the database information is an important factor in choosing the form of production.⁴³ The reason follows from what a database is. A printout records what the database displayed on one occasion, while the database's value lies in its fields and the links among its records, which let a user sort, query, and cross-check. Without them the recipient holds a picture of an answer that it cannot test.

Cost objections to a database request are judged against the stakes. In a dispute over insurance charges under Best Buy's leases, the Odom database was no longer in a searchable format and would have to be restored from original sources at a cost of at least $124,000, plus $27,823 a month in storage. On review, the court determined that, because of that cost, the database was not at present reasonably accessible.⁴⁴ The magistrate judge had reached the opposite conclusion on accessibility. In the magistrate judge's view, $124,000 was reasonable given potential contract damages above $800,000, the potential for enhanced damages on the fiduciary-duty and fraud claims, and the long-term economic impact of the outcome on all parties, and Best Buy had not challenged the database's relevance.⁴⁵ The same figure read differently depending on what it was compared with: the reviewing court looked at the cost of restoring and maintaining the database from original sources, and the magistrate judge looked at the size of the case.

A burden claim about a database also weakens when the producing party uses the data itself. In LaBrier v. State Farm, State Farm said it could not search its payment data because the data sat in a “picture format” reachable only through its ECS system, and that calculating labor depreciation for a claim required downloading Xactware estimates one at a time and recalculating each by hand. The court found it incredible that there was no cost-effective way to match information in one database with information in another, observing that data sorting is what computers do in much higher volumes in very short amounts of time. Even if the matching had to be done claim by claim, the time and cost did not justify keeping LaBrier from critical information.⁴⁶ The record undercut the burden claim in other ways. State Farm's own employee, Guevara, had used Xactware software at the start of the lawsuit to identify the putative class members and estimate the damages alleged, and the employees who knew the systems, State Farm's Jangda and Xactware's Stoddart, described no burdensome process, let alone an unduly burdensome one.⁴⁷

A request fails when the metadata does not connect to the claim. In Aguilar the plaintiffs sought system metadata, including the date created, date modified, and modified-by fields, for all Word, Excel, and PowerPoint documents. The court found the information of limited relevance to the question at the heart of their Bivens claim, which was whether ICE agents violated their constitutional rights by entering their homes without seeking consent. The plaintiffs' example, an operation plan, listed no targets or residential locations, so learning whether it was modified before or after the operation would not show who was targeted.⁴⁸ Timing also counted against the plaintiffs. The same court declined to order a retracing of steps to recover metadata for harvested e-mail. It said it would have entertained a formal request made before the harvest was largely complete, but the plaintiffs had delayed for a considerable period, and it was far from clear that exhaustive retracing would yield useful information beyond what they already had.⁴⁹

LaBrier and Aguilar differ in what the data does for the requester. In LaBrier the data was critical to the claim, and the producer's own use of it undercut its burden argument. In Aguilar the fields had limited bearing on the claim, and the request came late. An exam answer that treats metadata as categorically protected or categorically discoverable skips the analysis that decides these requests.