Strategic Legal Technology

What will it mean in the future to be a lawyer? 

For an excellent, sweeping view of the future, read Richard Susskind’s The End of Lawyers?. On his website, Susskind asks “what elements of their current workload could be undertaken more quickly, more cheaply, more efficiently, or to a higher quality using different and new methods of working.” He answers

“that the market is unlikely to tolerate expensive lawyers for tasks that can be better discharged with support of modern systems and techniques [and that] the legal profession will be driven by two forces in the coming decade: by a market pull towards the commoditisation of legal services, and by the pervasive development and uptake of new and disruptive legal technologies.”

Two recent E-discovery (EDD) items instantiate the issues that Susskind discusses. My colleague Babs Deacon wrote a great Integreon blog post, Early Case Assessment - The Emperor Has No Clothes. She explains that early case assessment (ECA) is a process, not a software application, and a process that takes significant skill and thought. She notes

“If litigators are beginning to agree that e-discovery is really just part of discovery, then all litigators will have to have some level of comfort with e-discovery management — from Information Management at the far left of the EDRM framework through Presentation. No litigator, or even an attorney in another practice area, should be truly free from the responsibility of attaining basic competence in this area.”

She notes that comparatively few lawyers have the requisite skill. This means that lawyers (A) should learn more and (B) rely on professionals who know ECA. She continues “don’t sideline ESI attorneys as ‘nerds’ and push them off of the partner track — a fear I’ve heard articulated more than once from junior associates.” I too have heard this and seen how lawyers who branch out from skills beyond the most basic, core, traditional law practice are penalized, not praised.

Well-known EDD commentator Craig Ball comments similarly in Show No Fear - Lawyers need to — and can — learn the language of e-discovery (Law Tech News, 1 June 2010). Ball asks “How many times have you heard a lawyer tell a court that he or she doesn’t ‘understand computer stuff’?”. He acknowledges the challenge of learning how to understand digital data relative to paper but argues lawyers must do so.

I agree with Babs and Ball that lawyers should learn EDD. A handful of AmLaw 100 firms also agree, the ones that have e-discovery practice areas. Lawyers in these practices do understand the law as well as the bits and bytes.

The challenge with lawyers learning more about EDD, however, as I see it goes to skills, training, and perhaps most importantly, mindset. Lawyers learning EDD is not the only solution to the problem fortunately. As Susskind and other commentators point out, we are in age of unbundling and disaggregating legal services. Just because lawyers have been the putative masters of discovery in the past does not mean that must be true in the future.

Perhaps there is a better solution than lawyers learning EDD, especially if the following points are true:

• The mindset of those attracted to law school does not favor the skills and ability to handle EDD; these skills include technology, process, and quantitative analysis.
• Law schools have little interest interest in teaching EDD. Granted, there are signs of change but progress in the first decade of EDD has been glacial.
• BigLaw does not have the cultural or organizational mechanisms to evaluate much less truly value skill sets not taught in law school, at least as attended by the partners who now matter. If law firms continue to focus on rainmaking on the one hand and reading cases and writing briefs or transaction documents on the other hand, wither EDD?

Richard Susskind emphasizes the question mark in the title of his book; he recognizes lawyer will have a continuing role. The future role of lawyers, however, depends on what lawyers and the institutions training and supporting them choose to do today. In my view, individually and institutionally, with respect to EDD, they have until very recently put their heads in the sand. That may be fine; it just may mean the future scope of what lawyers do is that much further narrowed to its original, pre-20th century (much less 21st) conception.

One of the most interesting ongoing electronic discovery questions is the role and importance of search tools in document review. I recently spoke with Jonathan Nystrom and Dick Oehrle of Cataphora about this and related issues. 

Cataphora develops powerful language analysis software. Its pattern recognition engine is designed to detect deviations from the norm, which makes it useful in investigations and compliance. The company is also an established EDD provider. I first saw Cataphora technology in 2003 and was impressed from the first time.

We agreed that while the choice of tool is important, the bigger considerations for effective e-discovery are (1) the fit of the tool to the document set and (2) the overall process a litigation team follows, which includes the training and skill of the people using the tool.

We left unresolved the tension between standardizing on a platform (the goal of many law firms and departments) and choosing the best tool for the matter. Optimizing tool choice by the case is a challenge: unless an organization has an entire process and infrastructure built around the tool, it unlikely to be the “best choice” for that case. So, as a practical matter, it seems likely that all but the largest legal organizations and EDD vendors will standardize around one tool. Or, if they need a specific tool for a specific matter, find a vendor that has optimized around it.

As for tool selection itself, we agreed that the answer to the question of “best tool” is empirical, not theoretical. Comparing tools on the basis of features has limited value because (1) different features work better for some collections than for others and (2) most lawyers simply cannot evaluate competing algorithms (much less how those algorithms are instantiated in a specific application).

Empirical testing requires sampling and statistics. We bemoaned the fact that most lawyers are not comfortable with either. I speculated that aversion to sampling and stats is more than just a lack of training or even familiarity. Rather, I suggested that sampling – and probability more generally – is fundamentally an anathema to most lawyers.

Lawyers dislike uncertainty; they hate being wrong. In their eyes, sampling can never be certain - it is thus suspect. This honestly held but mistaken believe leads to skewed (I am being charitable) outcomes. For example, we agreed lawyers’ faith in manually reviewing documents is misplaced. Those of us with experience in the trenches know humans make many mistakes. And sampling tests would undoubtedly prove that. Whoops, lawyers don’t do sampling, so they continue to have faith in human accuracy. The challenge is that e-discovery is largely science and math and some art; but it definitely is not religion. So blind faith is necessarily misplaced.

My conversation with Cataphora management was stimulating; they clearly have a sophisticated view of the market and the tools. And I was intrigued to learn that this summer, the company will release a consumer tool that will tell Outlook users what Outlook says about you. For those who are open-minded, those data could well give rise to another instance of having to question honestly held views. So, with apologies to the Talking Heads, “Goes to show what a little DATA can do”.

This is a live report from Legal Tech NYC on the session E-Discovery - Considerations for In-Sourcing and Outsourcing. 

The panelists:
Marilyn Caldwell - Director of Practice Support, Lowenstein Sandler
Michelangelo Troisi - Senior Counsel and Director of Risk Managment, Samsung Electronics America
Scott Cohen - Proskauer
Craig Ball - Consultant and court-appointed Special Master for EDD

Introduction. Most everyone is already outsourcing at least some EDD work, so this panel is focused on considerations for bringing work back inhouse. A recent poll of inhouse counsel found that 3/4 said law firms are not doing enough to respond to cost pressures, especially on EDD. The volume of ESI keeps growing, so the problem is not going away. And ESI is becoming more complex with the advent of new media such as Facebook and Twitter.

The law is now clear - companies have a duty to preserve records. Doing this poorly, without expert advice, can lead to problems. So for organizations taking EDD inhouse, how do you make sure you meet your obligations?

“The reality of e-discovery is it starts off as the responsibility of those who don’t understand the technology and ends up the responsibility of those who don’t understand the law” - Craig Ball

Identification and Preservation. Craig discusses the EDRM reference model. Prime candidates for insourcing are RM, identification, preservation, collection, and processing. Questions latest Judge Scheindlin Pension Committee decision, suggesting that custodians must be involved in preserving records. Custodians need to be involved but that is not enough. Mike says that it is not practical, at outset of litigation, to immediately go out in the corporation and interview all the custodians. Samsung sends out notices to employees. He says that this ends up in over-preserving. Talking to custodian to determine relevance just takes a lot of time. Custodians preserving their own docs is really the only practical solution. But acknowledges that not all employees read all notices.

Craig suggests some institutional / enterprise steps to help ensure preservation. For example, before sending out preservation notice, create and save back-ups of key systems such as an Exchange. Samsung does this with Exchange and Enterprise Value (again, he says, this results in over-preservation).

Collection. As lawyers and legal staff acquire skill, they will be able competently collect data. But today, lawyers deal with intermediaries. Craig argues that lawyers need to be closer to the data earlier in the case than is currently the case. Marilyn points out that lawyers are nervous about collection because they are aware that they may change the data. So some debate over whether collection is merely ministerial. Craig: why didn’t these lawyers know the answer to the data change issue. Marilyn: IT staff knew meta data needs to be preserved but business people think only about document content. This leads to a discussion of what credentials or training is required to collect data appropriately. Craig points out that there are no readily available certifications / training for lawyers. He says he could teach most lawyers in 2 to 3 days. But he is not aware of any courses. And he’s not sure how many lawyers would sit for such a course. Craig: in all the time lawyers spend rationalizing why they can’t or shouldn’t learn this, they could actually learn it. [Editor note: reminds me of mainframe computer days, when I sat though meetings where glass box IT staff spent hours explaining why data I needed was impossible to get instead of spending that time writing code to get it.]

Scott: many tools purport to help with collections. But you first have to understand requirements. Once you understand the legal requirements, then you can match up with actual capabilities of tools. Emphasize actual, meaning having tested the tool and not relying only on vendor representation.

Craig: bring as much as you can inhouse so that corporation can competently and defensibly reduce data volumes. This will yield smaller data set that gets turned over to vendor for processing and that needs review. A show of hands: many companies have hired vendors to forensically collect data. Craig thinks that with one competent and properly trained inhouse person, this is not necessary. Marilyn counters that you need someone who can testify appropriately and this may not be the typical inhouse IT person. Craig responds that if the collection is competent, the likelihood of having to testify is very low.

For companies outsourcing some of their IT infrastructure, they need to consider the real total cost of ownership, which includes the cost of collection and preservation.

Audience question: Should law firms offert IT training to clients? Scott would be more comfortable pointing clients to appropriate third parties. Marilyn would also turn to 3rd party vendors.

Steps for Insourcing - Corporate Perspective. (presented by Mike)

Questions :
1. Do steps comply with law?
2. Do you have the right processes?
3. Do you have an action plan for when litigation arises?
4. Has the action plan been tested and used?
5. Is the process defensible? Has counsel signed off on the plan?
6. Is the process repeatable? Does it work for all or at least most cases?
7. Who owns and manages the process and plan?

Forming an internal EDD team:
1. Is it cross-functional?
2. Who leads the team?
3. Does the the team have the necessary resources?
4. Is IT fully involved and committed?
5. Is the team truly cross-functional?
6. Which department manages ESI? (At Samsung, law department manages ESI. Company pays IT for support it needs.)

Technology:
1. How can tech help the process and people work better?
2. Is the tech modular or upgradeable?
3. Will the vendor continue to suppor the tech?
4. Can the company survive without the solution?
5. Can the company live with the tech becoming obsolete?

Steps for Insourcing - Law Frim Perspective. (presented by Marilyn)

Evaluate and Plan
- Need personnel with right expertise, technical and consulting
. Keep in mind that training is a continuous process
- Need the right hardware and software
. Is your system scalable
. Does your tool have the right feature set
. When there are problems, what do you do?
- Determine admin requirements
. Keep track of what you have done and who did it
. Record action in case challenged in court
- Assess risk
. What happens when something goes wrong?
. Will the firm accept this risk?
- ROI
. There are a lot of moving parts - is the firm ready for all these costs?

SWOT analysis
Strengths
- Flexibility, efficiency, deeper understanding, transparency, control
Opportunities
- Cohesive work with case teams
- Integrate tech efficiencies into work flows
- Expand knowledge
Weaknesses
- Limited exposure to varying data types
- Staffing constraints
Threats
- Difficult to decline work once you’ve started
- Scaling up
- Data volumes growing
- Software bugs

Considerations for In Sourcing
Do you have the people?
Do they hae the time, expertise, and interest?
Do you have the budget?
Can you monetize?
Do you have the right tools and infrastructure?
Can you stay current with tools, with the right training, and keeping methods current
Are you lawyers willing to be looped in?
When in doubt, farm it out

The Posse List, an increasingly important resource for e-discovery generally and contract lawyers specifically, is publishing a series of interviews with EDD professionals. My interview appears today, An interview with Ron Friedmann of Integreon; the legal paradigm shift, predictive coding, document categorization, and more . 

In it, I explain my views of why the new normal will require law firms to deliver more value to clients. This will require more technology. In e-discovery, this means substituting technology for humans where possible. That said, my view is that it is “man and machine", not “man versus machine”. I comment on early case assessment (ECA) and predictive coding, suggesting that they are more points on a continuum than different approaches. Both require significant human input and share the goal or automating document review.

Several other interviews appear this week at the Posse List and are worth reading.

Tom O’Connor and I recently wrote a joint blog post about concept search software for e-discovery. Subsequently, we received comments from Herb Roitblat of Orcatec, an expert in information management, data mining, statistics, and eDiscovery processes. I share his comments here. 

Tom posted at his docNative Paradigm Blog Herb’s comments on Xerox CategoriX and Musings on the Best Approach to EDD Search (29 Oct 2009) by Tom and Ron:

• Herb Roitblat on ED Searches (12 Nov 2009)
• Herb Roitblat On Search: Part 2 (23 Nov 2009)

I publish here, with permission, additional comments from Herb, who wrote these in response to a message I sent him with my “take aways” from his first comments.

Summary
My summary and interpretation of Herb’s comments below and in the posts at Tom’s blog is that while concept search is a useful tool for e-discovery, the selection of the specific “flavor” of concept search tool matters less than smart application of it. Tool selection needs to be case specific because a “bake-off” among concept search tools only tells you how well a tool does against a specific set of documents. Since it’s not economically feasible to use multiple tools per case, you need to make a reasonable tool selection at the outset of a the case. As important, you need a reasonable and defensible process (which means documenting tool selection and process). The reasonableness standard depends on the stakes of the case.

Herb and Ron Exchange by E-Mail

Ron: So it sounds like what you are saying is that the difference in e-discovery concept search tools is probably overwhelmed by differences in document sets and in process / control.

Herb: I agree with this, but it has to be said carefully. Clothing does not make the man and high-powered tools do not make the builder, but they do help a good builder do better work. No matter how good your tools are, if they are not used well, you get a questionable result.

Ron: Concept search is not a magic bullet but helps expand the universe of documents to consider because it finds docs with words you would not otherwise think of as search terms.

Herb: It helps you think, but it is not a substitute for thinking. It is, as you say, not a magic bullet, just an amplifier.

Ron: Concept search can also help speed review by clustering similar documents.

Herb: Concept search expands queries to return results that are the best match to the expanded query. Thus, the top results are those that best match the query term and its context. (See green search on Truevert.com for an example, search for meat and get organic meat, not Omaha Steaks). There the context is given by green documents.

Ron: I back away from my initial assertion of the need to use multiple tools. I argued that to spur thinking among EDD professionals. Upon further reflection, what I really meant to say is that lawyers should focus more on industrial processes and controls, statistics, and metrics than on software features.

Herb: That’s what I think.

Ron: So that means we have no magic bullets. The legal profession has hard work ahead to industrialize its processes.

Herb: It’s actually not that hard. You just have to be thoughtful about what you are doing. It is not even terribly burdensome if you are realistic about the levels of accuracy that you can really achieve (see below).

Ron: We still don’t seem to have an objective standard by which to judge if a process is ‘good enough’.

Herb: There are lots of ways of deciding whether a process is good enough and lawyers are used to making reasonableness judgments and arguing about them. What are the consequences of different types of errors (e.g., retrieving too many documents, retrieving too few)?

Scientists, by tradition, usually use a standard of .95 confidence. For example, if two treatments are different with 95% confidence, then we accept them as different. That does not tell us how different they are or that the difference is practically important or useful, only that the difference is statistically significant. Scientists often report higher confidence levels than that, but the minimum is usually .95. That tradition has worked well in science where subsequent research can correct the relatively few times when the difference does not really exist, but resulted from sampling (luck of the draw).

As an analogy, if you play slot machines, the things return only about 95 - 98% of the money that gets pumped into them, but that does not mean that some people don’t actually win large amounts. It happens sometimes. The luck of the draw usually returns less than you put in, but sometimes it returns more.

Back to good enough. Engineers typically use confidence levels to tell them how well to build a bridge. They consider the consequences of different kinds of failure (think of the Tacoma Narrows Bridge). NASA uses confidence levels to determine the quality of their systems. Where the consequences are severe, they require higher confidence.

In eDiscovery, we are familiar with proportionality arguments and the like for determining things like cost shifting. The same thing applies here. A bet the company litigation may merit a higher level of confidence than a run of the mill litigation. Different types of errors may be weighted differently depending on the consequences of that kind of error.

None of this is hard nor does it require very much mathematical background. I published some tables a while back showing how many documents you should sample if you want to achieve a certain level of confidence and you are willing to accept the possibility of missing a certain proportion of responsive documents.

As I think I’ve said, I think that another part of reasonableness is transparency. Be able to describe what you did. A scientific publication is intended to describe enough of the methodology so that another scientist can replicate the observations. I don’t think that you necessarily have to publish to the other side what you did, but you should be able to provide that information if required (think Victor Stanley).

This is a two-part, joint blog post. I recently spent some time looking at Xerox’s new CategoriX EDD tool and writing a post about it. After reading it, I realized it would be helpful to set my discussion in a broader context. So I turned to my friend and e-discovery expert Tom O’Connor and author of the docNative Paradigm Blog . What follows is a combined post; we wrote each section individually and are cross posting this. 

Xerox CategoriX and Musings on the Best Approach to EDD Search
by Ron Friedmann

In early October, Xerox Litigation Services released a new e-discovery search and review tool called CategoriX. How should EDD professionals think about this and other new search technologies?

A Xerox PR firm offered me phone time with the CategoriX product manager, Svetlana Godjevac. Always curious about new litigation document review tools, I accepted. I also read the CategoriX product sheet and a statistics-heavy Categorix white paper explaining how Xerox tested the product.

The CategoriX approach sounds interesting and useful. Xerox R&D in Grenoble developed the product and the company appears offers it beyond the litigation market (see the page
Text Categorization and Clustering housed under Xerox Technology and Brand Licensing.) The product combines ‘probabilistic latent semantic analysis’ (document clustering) with iterative machine learning.

It sounds powerful but I can’t evaluate its effectiveness. This is by no means a criticism. Both search approaches have been around for years so it’s hard for me to assess how they work in CategoriX. Learning more about Caterorix confirms what I’ve suggested before: mere mortals can no longer evaluate EDD platforms, at least not by assessing the underlying algorithms.

I lament that I don’t know enough statistics to fully comprehend the white paper but Xerox appears to have tested the product (though the nature of the 2 document sets studied and human reviewer groups is not described). One finding I did focus on is that Xerox used this tool to quantify inter-reviewer variability. Not surprisingly, humans are not all that consistent, a fact that lawyers routinely overlook. In my conversation, Ms. Godjevac reports that Xerox does explain the statistics to lawyers and works with them to understand the problems of human review.

How a litigation team should choose among the available advanced tools is a real quandary. The investment to run a “bake off” among competing choices is enormous; moreover, the outcome may well depend on the nature of the documents. What does this say about defensibility in general? Would it be defensible to use product A if an objective study showed that product B was 20% better? And what exactly does 20% better mean anyway?

Courts seem a long way off from considering this question but the leap from the current standard to one that requires comparing tools seems more a matter of degree than of kind. Are litigation support professionals obliged constantly to evaluate new tools to make sure what they now use is adequate?

Of course, I may be way off base here. Which is why I am surprised and dismayed that I haven’t found much commentary on this tool. Many other bloggers comment on EDD but I did not find much blogging (or Tweeting) about CategoriX. I would like to see more discussion of products, comparisons of them, and the future standard of what courts will rule is defensible.

[I felt this did not stop at quite the right spot so am glad Tom stepped in….]

The Challenges of Evaluating EDD Search Tools
by Tom O’Connor

Ron, your comments about the problems facing anyone attempting to evaluate ED applications are right on target. First of course is the fact that one needs an engineering degree to even read some of the white papers in this field. But it seems to me that the problem starts even before that with several fundamental problems.

The first, as you mention, is that there is never enough detail given about the document sets being studied. Understanding the documents is a crucial part of any automated litigation process and evaluating products which don’t sufficiently describe the universe of documents they are working with is simply impossible. This is not a failing of Xerox alone but really all the reviews I have seen. It is nearly impossible to cross compare applications if they are “tested’ on widely divergent data sets.

In addition, some search engines use a standardized thesaurus such as the publicly available WordNet Lexical database, an open source thesaurus from Princeton University. It has over 100,000 English words and associations. As an open source resource, the WordNet database is available for download and examination if needed for litigation validation purposes. If, however, the comparison is between one program using this database and another one that uses an internal or closed database, does that really help us?

Even the widely touted TREC (Text Retrieval Conference) study suffers from this failing in my opinion. The TREC study used a test set of 7 million documents available to the public pursuant to a Master Settlement Agreement between tobacco companies and several state attorneys general. Attorneys assisting in the study drafted five test complaints and 43 sample document requests (referred to as topics). The topic creator and a TREC coordinator then took on the roles of the requesting and responding counsel and negotiated over the form of a Boolean search to be run for each document request.

The problem is those documents were not in native format and did not include attachments. Given that typical collections today consist largely of massive volumes of e-mail, many with attachments (and attachments to attachments), this is , a huge issue when evaluating search capability for email.

A second problem I see concerns what type of search is best. We all agree that computer searching is more accurate than human review. The Sedona Conference Commentary on the Use of Search and Information Retrieval Methods in E-Discovery, released in August 2007, states that “Human review of documents in discovery is expensive, time consuming, and error-prone. There is growing consensus that the application of linguistic and mathematic-based content analysis, embodied in new forms of search and retrieval technologies, tools, techniques and process in support of the review function can effectively reduce litigation cost, time, and error rates.” So the assumption that concept search is better than Boolean searching, although widespread, may be wrong.

In Disability Rights Council of Greater Wash. v. Wash. Metro. Area Transit Auth., 2007 WL 1585452 (D.D.C. June 1, 2007) Federal Judge Facciola stated that “concept searching, as opposed to keyword searching, is more efficient and more likely to produce the most comprehensive results.” Judge Grimm made a similar statement in Victor Stanley, Inc. v. Creative Pipe, Inc (Civil Action No. MJG-06- 2662 (D. Md. May 29, 2008).

The TREC study results, however, don’t seem to support these judicial positions. In that study, computer scientists from academia and other institutions attempted to locate responsive documents for a number of topics using 31 different automated search methodologies, including concept searching. The result? Boolean searches located 57 percent of the known relevant documents. None of the alternative search methodologies had better results.

In fact, a Boolean search generally equaled or outperformed any of the individual alternative search methods, but those alternative searches also captured at least some responsive documents that the Boolean search had actually missed. The lesson? Manual review misses many documents but so does keyword searching, Boolean searching and concept searching - but they all miss different documents. The best approach is to use multiple applications to do iterative searches which winnow down to the best possible results.

This isn’t late breaking news. Ron, you started a discussion in May 2008 in Concept Searching in E-Discovery. Some of the info above I gleaned from reports on web sites and reports by people like Herb Roitblatt or Gene Eames who know a whole heck of lot more about this than do I. But the point is, one product isn’t going to do the job, no matter how good the product or convoluted their documentation. And irrespective of the tool, the “operator” better be well trained or who knows what the results will be.

I share Ron’s concern about emerging standards of defensibility. Given the technical complexities and the lack of statistical certainty, I don’t see how a clear, stable defensibility standard will emerge other than what we’ve seen, namely, have a plan, apply some smarts, and document what you do. As we’ve seen in other arenas, developing standards by judicial opinions is a long and messy process. Well, I suppose the upside is that consultants will stay busy!

This morning at the The Masters Conference, one of the leading e-discovery conferences, taking place in Washington, DC, I sat in on the first half of Early Case Assessment: Looking to the Future - From Early Assessment to Early
Awareness. Here are notes from the first half and some comments.  

What is ECA? It’s getting an early look at the facts of your case and at the scope of discovery. Goal: understand the matter risk profile and the cost to defend so that counsel make informed decision about settlement or moving forward?

Where are the immediate savings; how is this different than past? It’s not a tool per se, it’s a method (a process, the right people, and technology). ECA does not generate savings very early - it’s not about upfront savings. You have to invest at the outset to learn about the case. Clients typically want to delay spending, so ECA is counter-intuitive to many lawyers. Lawyers need to spend more upfront but this creates significant savings over the life of the matter. As important, it prepares lawyers to argue their case more effectively.
The vendor challenge here is that clients are reluctant in this environment to spend upfront. So you need to educate your client - let them know that ECA will reduce the volume of documents that require human review. For a DC practice, dealing with agencies, lawyers have cordial relationship with adversaries. ECA works better with agencies than private parties because of the cordial relationship.
Another benefit of ECA is as an “early warning system.” But from an antitrust perspective, this is less of a factor. Benefit may be greater in securities or fraud matters.
When lawyers offer search terms, they often don’t know the impact on the case. ECA let’s you understand the impact of the search terms that you select. But ECA is still aspirational - uptake is still slow.
Need to emphasize the doc review cost savings.

Other than cost, are there other limits that hold back ECA? Clients are the main barrier. We are moving beyond linear review and search terms to a more subjective approach. If you can do that quickly and iteratively - as you can with ECA - you have a powerful approach, one that is much better and cheaper than collecting too much data. So ECA is less about the tool than getting real insight into data and let that drive decision making and strategy. It gives you more control over the process, with objective insights into the data, so it’s more defensible.
The early adopters are likely to be lawyers or clients who regularly deal with key word searching (RF: suggesting serial litigants I think).

Tools that are better at ‘understanding’ data are in the ECA bucket… how do these emerging technologies affect the process, especially if clients bring the tools in-house? To start, who should operate the tools? It depends on organizational structure. It can be IT, Legal, Info Security, other corporate departments. Depends also on budget. Companies must make conscious and considered decisions about the process. Legal is the ultimate stake holder and should drive the process. Purchasing is beginning to fall into the model of enterprise software acquisition. As for running the software, this is still an emerging area because the processes are still evolving. This too requires work, for example, defining workflows.
There are risks in moving away from lawyers supervising the process. First, you can lose attorney-client privilege. And second, without lawyer supervision, problems in data collection or custodian selection are more likely.

[End session notes]

[Begin comments]

Unfortunately, I had to leave at the half-way mark. What struck me most about this discussion is how much education is required. Already 20 years ago, it was obvious that upfront investment and planning in assessing discovery materials was a good decision. That is, it always makes sense to invest and plan. Sometimes cases do settle and it feels like you’ve over-invested. But that confuses decisions and outcomes. Frankly, I just don’t understand the reluctance to invest. Perhaps if lawyers thought more carefully about the full life-cycle cost of litigation and kept historic data (imagine that!), they would understand.

Further, it was also obvious two decades ago with the earliest full-text databases, built form scanning and OCR’ing documents, that iterative search was an absolute necessity. By 1992, there was a decent selection of conceptual and Boolean search tools. Tool selection did not matter - iterative search and analysis was critical and obviously necessary to anyone who gave the matter any thought.

I find it distressing that there is still a need to discuss this. This is not a knock on the panel, which, unfortunately appears to be a necessary discussion. It does not speak well, however, of lawyers and the legal profession that two decades is not enough time to ‘get it.’

This is a near-real-time post from the The Masters Conference, one of the leading e-discovery conferences, taking place in Washington, DC. This session is How Defensible is Your E-Discovery Process? 

Panelists:
Dean Gonsowski, Esq., Vice President of E-Discovery Services, Clearwell Systems
John Loveland, Managing Director, Navigant Consulting, Inc.
Bruce Markowitz, Director of Litigation Support with McKenna Long & Aldridge LLP
Sonya Thornton, Manager, Legal Discovery / Compliance, Sprint Nextel Corporation

Defensibility is not a recipe, it’s a process. What is defensibility? It is based on a “reasonableness” standard for the entire EDD process. Several triggers raise this issue:
1. Preservation: When did you know litigation was likely and what did you preserve.
2. Search: Lawyers cannot just design key word search by the seat of their pants - that’s not defensible.
3. Inadvertent privilege

How Does Collaboration Play into Defensibility? Law firms need to understand client structure and staff and where information resides. To have collaborative conversation - with vendor, client, third party, opposing party - lawyers must understand what data clients have and don’t have and what is reasonably accessible. If collaboration is handled right, it makes process more defensible. Where parties are in agreement, judges rarely will intervene. Of course, reaching consensus is hard. Need to focus on the “end result:” lawyers need to start from what they want to prove and work backwards to search terms and negotiation over process. Then collaborate to try to achieve your desired end result.

Question: In collaboration, how much should you educate opposing side about your data?
Answer: If you don’t educate other side, they will make mistakes and then seek continuances. So it can work against you if your opponent is not smart enough about search and process. Absent education, you face too many iterations. Giving other side a bunch of data that neither helps nor hurts does not serve anyone’s purpose. Doing Early Case Assessment (ECA) helps refine search terms. A collaborative approach to term selection does not give away that much. Not dumping useless data on opposition is probably ultimately more defensible because it is more reasonable.

Question: Should each side give other side the “word wheel,” which is the list of all unique words (aka inverted index)?
Answer: Panelist says this does give away too much. Another panelist took opponents proposed term and gave stats on percent responsive and percent non-responsive. This can get to consensus faster.

Law firms can gain client “stickiness” once they learn their clients’ IT infrastructure and where data are stored. This lowers the cost, makes the process repeatable, and improves defensibility.

Question: How do you define one or more workflows?
Answer: It takes time; at Sprint, it’s taken well over a year to map data, gather retention schedules, assessing which firms really understand e-discovery. Sprint will have distinct workflows for data management based on type of law (e.g., employment v IP). There may be variations, but company starts with a baseline process.

Question: What is responsibility of inhouse counsel v law firm with respect to responding to a subpoena? Law firms may change scope based on less complete understanding.
Answer: Outside counsel is on the hook in court, their license is on the line. So firm has to be very educated about client’s data and the issues in the case. If firm is educated, inside and outside counsel should be very closely aligned in responses. In practice, firms tend to add, not subtract terms. Also, firms need to show what search terms will yield and discuss results (iterate) with the client.

Question: Is there a single, perfectly defensible process?
Answer: Unanimously: no. Need to understand, you can never spend enough to eliminate all risk. Clients should be conscious about what level of risk they are willing to accept. Also, with case law changing, this is not a static question. Lawyers must constantly re-evaluate law, case, and risk level of current process.

Question: Are there specific practices that are an especial danger or risky to defensibility?
Answer: Sprint: one big concern is data mapping. We have 1000s of apps running and to figure out where all data resides is hard. We have created system profiles for various apps: one for judges, one for lawyers, one for the IT staff. These all help define reasonableness. Plan in advance to define reasonableness (e.g., you can’t stop your billing system).
Law firm: what are risks of self-collecting - this depends on nature of case but can be risky. Clients must way costs of self-collection versus vendor collection with the risks of self-collection.

Question: How transparent should collection and hold be? Are clients in danger if they present data collection as a “block box”
Answer: Yes, every judge wants process to be completely transparent. To be transparent, lawyers must be sure to understand how IT, HR, and other departments operate and manage data.

Last week top law firm O’Melveny and information retrieval firm H5 announced an alliance to provide clients with innovative search methods for electronic document discovery (EDD). This is a legal market watershed in my view. I was fortunate to interview Richard Goetz, Chair of O’Melveny’s Class Actions practice group about this agreement and the firm’s thinking behind it. 

On 3 August 2009, the two organizations issued the press release O’Melveny and H5 Form Strategic Alliance to Offer Cutting-Edge Model for Document Review. I understand why blogosphere and Twitter chatter about this was limited. E-discovery software vendors and service providers sometimes persuade law firms to issue press releases announcing the use of their products or services. These releases are about the vendor, not the firm. As far as I can tell, only Law.com’s Legal Technology Blog noticed something different here, observing the firm likely sees competitive advantage in the announcement.

I agree and go further. Unlike the typical press release announcing partnerships with e-discovery software vendors or service providers, O’Melveny stakes out a clear position: it wants to provide “clients with a cost-effective, efficient and accurate alternative to traditional search methods for document discovery, review and analysis.” The release quotes Goetz acknowledging client concern about the cost of e-discovery and document review. Everyone in the legal market knows this is true but few BigLaw firms say so; fewer still take public steps to address the problem. So I was pleased be able to learn more by speaking with Mr. Goetz.

Goetz explained that the firm started an initiative several months ago to identify a means to help address what the firm knew was urgent client concern about the spiraling cost of document review. O’Melveny spoke to clients about the problem and sought potential solutions. The firm put aside any consideration of their own economics and legacy approach. After evaluating options, they chose H5.

I asked Goetz if the firm conducted a competitive analysis of how other firms handle EDD, mentioning Dorsey & Whitney’s LegalMine offering (my blog post about LegalMine here). O’Melveny did not consider how other law firms handle doc review; rather, the firm wanted to figure out the problem from a client perspective. Their idea was to solve the problem by offering a completely different model. So the diligence was on e-discovery providers and Goetz says the firm spent much time looking at available technology and process before reaching out to H5.

So will it work? I have long been impressed with H5 technology and process and have mentioned the company in several posts over the years. So I have no doubt that H5 is a good approach. The question is really better stated as, “will lawyers adopt this new way of working?”

In my two decades of legal technology experience, I know that it’s one thing to come up with an innovative approach, another to sell it inside a law firm. Goetz says that O’Melveny focuses on what clients need and the rest flows from that. The firm will not require partners to use H5 and will still offer traditional doc review approaches for those clients who want it. But Goetz reports that in internal discussions, partners are very receptive to the idea of a better way and recognize the importance of offering desperate (my word, not his) GCs an alternative. One internal selling point was the potential to get litigation matters that the firm might not otherwise get.

I pressed him on the internal pitch, pointing out that many firms profit hugely from staff or contract attorneys and consequently don’t want to disturb the model. Goetz’s view is that if you want to be a leading law firm and litigation powerhouse, you cannot ask how to keep high margins. Instead, you have to ask how to deliver the best service for clients and solve problems in the best manner possible, at the best cost.

I agree - this point seems obvious yet sorely missing at many firms. When I was a law firm manager discussing client service issues, I often said - to partners and to staff - let’s have this discussion as if the GC of our biggest client were in the room. He or she does not necessarily have to agree with everything we say but should not be embarrassed, appalled, or angry at anything we say. I have heard many a discussion in BigLaw about attorney review that fails this simple test.

I wonder though, if in breaking the mold, O’Melveny has created yet higher expectations for itself. After all, many other methods exist to control and reduce litigation cost. Two of my favorites are budgets (based on rigorous analysis of aggregated data from prior matters) and risk analysis using decisions trees. So I asked Goetz if clients may now expect O’Melveny to announce additional cost control measures. Goetz hopes so. He agrees that more rigorous budgeting is a good candidate. Plus he thinks a really big opportunity is better collaboration with in-house counsel to make business-based risk decisions about litigation.

The jury is, as they say, still out. It will be interesting to watch O’Melveny and H5 to see if either discusses their future joint experience, especially cost savings they achieve. Irrespective of outcome, however, I view this partnership as a great step forward for the market. Goetz talked about gaining the “first mover” advantage - words seldom heard in BigLaw. O’Melveny may very well gain the benefits accruing to a first mover. Moreover, with luck, the entire market will benefit: the firm’s move may force the rest of BigLaw to talk about the elephant in the litigation room in new terms.

The litigation support and e-discovery market has felt chaotic for years now: rapid growth, technological shifts, emerging judicial reviews, and changing roles of the the main players (law departments, vendors, and law firms). EDD SHOWCASE: Strange Times by George Socha & Tom Gelbmann (Law Technology News, August 2009) provides good market insight. 

For anyone in EDD land, this is a must read. My key take-aways:

• Law firms nationwide purport to harbor deep EDD expertise - much of this is a convenient fiction
• No more than 30 companies have deep EDD expertise; others struggle to build it
• The fear of EDD cost and risk drives the market; “seldom mentioned, is every litigator’s fundamental need to find the data necessary to tell a compelling tale…. Finding the right data and figuring out what to do with it should be primary drivers, yet all too often these considerations are not even part of the conversation.” [See my June post , E-Discovery Goal: Win or Avoid Disaster?]
• “Increasingly document review has become the purview of specialized review shops — some domestic, others partially or predominantly offshore.”
• Survey data show a 9% dollar contraction in EDD yet market participants expect growth around 25%. My take: volumes continue to grow; unit costs are shrinking (and will continue to do so); more reluctance to share data so results may not be as reliable; and costs are shifting to earlier in the EDRM cycle where measuring them is harder.
• “In a crowded, noisy market, too many providers are making unsubstantiated claims and creating consumer confusion, while consumers lack effective means to compare technologies and methods. ”

Over at the EDD Update blog, there is a very good discussion of defensible EDD / ESI search. It’s worth reading and prompts me to ask what we mean by reasonable. 

Eric Mandel of Zelle Hofmann posted Searching for the Definitive “Search” Tool on July 23, 2009. As of now, there are comments by Herbert Roitblat, Craig Ball, Tom O’Connor, Eric (in rebutal), and Chris Spizzirri. This is one of the more interesting dialogues I’ve read on e-discovery in sometime. (The discussion gets a bit heated in places but I ignored the polemics.)

Eric suggests the current discovery system is broken, that humans are not reliable reviewers, that machines may not be reliable, that the introduction of statistics may confound matters, and concludes that “So the question remains, how do we find a definitive search tool that is affordable, manageable and defensible in 94 district courts, never mind who knows how many state courts? ”

The comments all discuss what makes e-discovery defensible and are worth reading. My take away of the comments is that a “reasonableness” standard will prevail, though one is left with the question of just what is reasonable.

I do find it hard to anticipate what courts will consider reasonable or defensible. For example, lawyers assume that lawyers must review documents. Why? I know of instances where paralegals have reviewed documents. And I’ve not found any lawyer or lit supp professional who can point to legal authority requiring that lawyers be the ones to review documents. It’s possible that some community college grads with a few days of doc review training could do as good if not better job than some contract lawyers. Of course, I don’t know any lawyers who would risk using non-lawyers in a doc review, even if they had test results showing no statistical difference in review designations between lawyers and non-lawyers.

My more general point is that I have a hard time believing in a standard of reasonableness in a system with unstated and untested assumptions. Is it fair to say that reasonableness in doc review is judged primarily (solely?) by prior practice rather than by any objective standard? And, if so, is that a good way of approaching the definition of reasonable?

A new e-discovery study by Equivio statistically compares human versus software performance in designating responsive documents. The results are worth reading. 

I’ve argued that litigators and judges should rely on statistical analysis to determine the the most reliable and accurate approach to reviewing documents. In my March 2007 blog post, The Gold Standard for E-Discovery Document Review, I argued that lawyers’ belief in the accuracy of human review is likely misplaced. In that post, I described an empirical study H5 conducted.

The Equivio>Relevance ™ study is available by registration. It compares human review results to Equivio>Relevance ™ results and found that

“Out of the 4,107 documents in which the review analyses differed, the Oracle ["Topic Authority” in TREC lingo, aka “subject matter expert"] reviewed a statistically drawn sample of 190 documents, or slightly less than 5% of the documents in dispute. Of those 190 documents, the Oracle determined that Equivio>Relevance was correct in 147 of the cases. The human review team, by comparison, prevailed in only 43 of the disputed sample documents.”

In my view, that means the computer performed better than humans. Equivio is, however, rightly more cautious and draws this conclusion:

“computer-assisted review can dramatically increase the efficiency and accuracy of a document review team’s work”.

Instead, the study suggest litigators can use the tool for earlier and case understanding and more consistency, among other benefits. The study also notes that “Equivio>Relevance ultimately identified more than 1,000 additional responsive documents that had been overlooked or mis-categorized by the human review team.” (The H5 study written up in my ‘Gold Standard’ post also reported that software found more relevant docs than the humans.)

I think about the Equivio analysis by way of analogy. Let’s say the corpus of documents were a bunch of cells in your body. And let’s say the responsive docs represented cancerous cells. Further, the more cancerous cells you can find and segregate, the higher your chances of survival. If you, as the patient, had to choose the diagnostic test you wanted, which one would you chose? I’d go with the computer because it’s finding more cancer cells. And if the FDA were evaluating whether to approve human search or computer search, the statistics would strongly favor the latter.

Of course, we are lawyers and not doctors. And lawyers must appear before judges and defend their discovery methods. And we know judges don’t like “black box” approaches to discovery. That however, may have to change. We can learn from medical clinical trials and how the FDA approves new drugs or devices. It’s all about statistical outcomes (technically, the “mechanism of action” for a treatment does not have to be understood to obtain approval).

Anyone care to do the stats to compare smart keyword searching against concept searching?

Have lawyers forgotten the goal of e-discovery? EDD articles and professionals seem exclusively focused on avoiding disaster. The real EDD goal is winning the case. 

In 1997, I characterized the goals of discovery as: [see note below for source]
A. Achieve the best outcome for the client
B. Minimize cost
C. Learn the facts of the case
D. Prepare offense or defense
E. Fulfill obligations to produce documents

I don’t think that has changed. Producing documents is a requirement of discovery; the goal, however, is learning case facts, telling the best story and winning. We hear countless stories of EDD gone awry. Where are the stories about how a team found the best documents, put together the best chronology and witness binders, and did the best job marshaling the facts?

Winning a case by “doing great EDD” should be newsworthy – but can you cite a single article in the last five years on this topic? Are litigation teams so paranoid about blowing e-discovery that case prep has suffered? If yes, that’s bad. If no, let’s hear the success stories, not just the disasters.

In an earlier discovery era, some were more concerned about winning. I remember the transition from paper + objectively coded databases to scanning + OCR + full-text + coded data.

Sure, we were careful to avoid pitfalls. For example, in serial litigation, inconsistent privilege designations across cases was a big problem. Corporate defendants sometimes did not turn over relevant documents in one state when they had in another. Why? Reviewers did not properly identify duplicates so a document could be in the database multiple times with different privilege designations. We mitigated this problem by using “document footprints” to de-dup.

The real excitement, however, was better case prep and winning. For example, we asked the lead partner to write his “dream smoking gun document”. The paragraph he drafted was literally copied and pasted into the search field of Personal Library Software (PLS was one of the earliest concept search tools). The search surfaced many useful documents – many that we never would have found using the then-standard coded database search and paper pulls or even Boolean searches on a full-text database. The partner estimated that finding the documents so quickly added $250,000 to the settlement value.

Then there was a time a partner was eager to find a document he felt sure must be in the collection. Many a search failed to find it. Then he had an idea: search for the fax number of the person who likely wrote it. Sure enough, even though the fax number was in small type and fuzzy, it had OCR’d well and we found a cover sheet, which quickly led to the document he sought.

Lawyers might do EDD better if they saw as much upside as downside. If anyone has good stories about how they used modern EDD tools to find the best documents and tell the best stories, please comment or send me e-mail to ron at prismlegal dot com.

[You can read more about my experience with this earlier generation of technology in an outline I wrote for a PLI presentation available on this website at Lessons Learned in Litigation Support.]

Update (23 June 2009): EDD consultant Tom O’Connor responds to this post in What Is the Goal of E-Discovery?. He writes:

“I disagree. I believe that there are two major additions to the goals of discovery since the amendments to the FRCP and they are both significant new obligations. The first is establishing the preservation of electronic documents at a point before the litigation actually commenced, that is that a proper ESI litigation hold strategy was employed and the second is the duty to cooperate in order to better facilitate FRCP 1.”

Raise your hand if you’ve read the TREC 2008 Legal Track study released in March. TREC is a great project to help legal professionals understand the accuracy and process of document review. To improve doc review and gain the value of this on-going study, however, TREC organizers and the legal profession need more dialogue. 

Background and Articles about TREC Study. The TREC study “focuses on evaluation of search technology for discovery of electronically stored information in litigation and regulatory settings.” Two recent articles help explain the TREC study and its importance. TREC 2008 Stresses Human Element in EDD by Jason Krause at law.com (1 May 2009) does an excellent job explain the TREC study. Krause also authored another article about TREC, In Search of the Perfect Search in the April 2009 ABA Journal.

The Study is a Hard Read. After the release of Overview of the TREC 2008 Legal Track, it generated little immediate discussion in blogs and at Twitter. The limited coverage is not surprising; I found reading it a tough slog. At risk the risk of being immodest, I have a good background to understand it: lawyer, 20+ years experience with full-text retrieval, three years of college majors-sequence math, and four years of hands-on econometrics experience. So if I have trouble understanding the report, that does not augur well for other legal professionals. It’s a tough read for three reasons: (1) long, (2) academic, and (3) too much passive voice, which makes understanding who did what hard. So I was glad to see Kruse’s articles.

TREC “Interactive Task” Focused on Replicating Decisions of Experienced Litigator. The part of the study I found most interesting focuses on how best to automate or replicate responsiveness decisions an experienced litigator would make. This was the TREC Interactive Task in contrast to the Ad Hoc and Relevance Feedback tasks. The study “gold standard” was the document designation a “Topic Authority” would make. “Gold standard” is a term of art that means the best-established or best-accepted solution. “Topic Authority” means an experienced litigator who best understands the facts and issues of the case and who, time permitting, would personally review all the documents. Topic Authority seems a variant on the more generic “domain expert” or “subject matter expert”.

Potential Issues with “Gold Standards”. The TREC study assumed that the Topic Authority’s judgment is the gold standard. More on that in a minute. Contrast that to the apparent assumption many litigators make. In a March 2007 blog post, The Gold Standard for E-Discovery Document Review , I wrote

“Many lawyers appears honestly to believe that human review is accurate, the “gold standard” for document review. “Honestly held” and “right” can diverge. I, for one, have never seen data to support the commonly accepted “gold standard.”

My concern then was that too many lawyers seem ready to dismiss software designations of documents in favor of lawyer designations, even if the lawyer has no case-specific know-how. I might accept the Topic Authority’s judgment as the gold standard but I don’t accept any old lawyer’s judgment as such (especially not a contract lawyer new to case who might have had only an hour or two of training about the matter).

Even the Topic Authority as Gold Standard raises issues for me. The accuracy and reproducibility of Topic Authority review has not been established. That said, it is not clear what standard is better. On the one hand, this standard acknowledges we must rely on the subjective opinion of an expert. On the other hand, reasonable experts can and do disagree. In contrast to doctors conducting clinical trials, lawyers cannot establish an objective and reliably reproducible standard. This conundrum is perhaps best summed up by the TREC Topic Authority, Maura Grossman, quoted in Krause’s law.com article:

“The decision process that determines what is responsive, and therefore what must be produced, is inherently a task involving human judgment. The concept of responsiveness in e-discovery is not an objective standard, it is ultimately a judgment found in the senior attorney’s head.”

TREC Compared How Closely Different Approaches Came to Replicating Human Judgment. The study examined how accurately different teams – where a team could be human or software or both – came to replicating the Topic Authority’s judgment. Each team had an opportunity to interact with the Topic Authority to refine or tune its approach. After all teams finished the review, researchers compared doc designations and, where results differed, set up an “appeal process” to decide on the correct designation.

I think it would be helpful to consider explicitly how Topic Authority judgment variability might influence thinking about e-discovery issues. We have no data on whether similarly situated Topic Authorities would agree on most document designations. (My doctor friends tell me that in clinical trials, this is the inter-rater variability problem). For example, suppose a future study employs two Topic Authorities per topic and finds a 20% variance in doc designation between them? We need parameters around the variance among Topic Authorities otherwise comparison to software approaches seem suspect. More specifically, if software approach A compares well to Topic Authority A and software approach B compares well to Topic Authority B but A and B vary by 20%, what does that mean? If the variance were under 5%, I think everyone would agree “so what.” If the variance exceeded 25%, I think many would find that troubling.

Conclusion: Need More Dialogue to Make TREC Useful to Legal Professionals. The potential variation among Topic Authorities notwithstanding, TREC offers what may be the best possible approach to evaluating the reliability of alternate approaches to document designation. The study as it stands, meaning a rather academic work, may not provide enough guidance to real-world lawyers, e-discovery professionals, and perhaps most importantly, judges. That said, I think that TREC could be instrumental to the future of EDD. Having a disinterested but highly knowledgeable third party evaluate alternative approaches based on scientific principles (meaning based on valid statistics and reproducible) is the only way to establish reliable standards.

To achieve traction in the real world of EDD, the profession needs to discuss the TREC theory and findings more widely. Organizations such as EDRM and the Sedona Group, which are already deep into e-discovery, have the potential to help digest, explain, and disseminate the findings in a way that make the work more relevant to everyday practitioners. A dialogue with practitioners would not only inform litigators and EDD professionals how best to use study results but might also influence study design to make it more useful to the legal profession.

[Note: I want to express my appreciation to Nicolas Economou, Sandra Song, and Dan Brassil of H5, who helped me parse out some of the finer points of the study.]

The e-discovery market is consolidating and its growth slowing according to a recent Gartner report. 

Gartner, in a December 2008 report, Gartner MarketScope for E-Discovery Software Product Vendors, reports several findings:

• Spending on e-discovery software technology and services will grow between 25% and 35% annually, down from 50% growth. Corporate insourcing is one reason for the declining growth.
• By year-end 2009, there will be 25% fewer EDD vendors.
• It remains difficult to compare products.
• “[W]e have also seen companies and law firms changing their tactics and the overall volume of litigation decreasing due to the excessively high cost of e-discovery.”
• Assessments of EDD software vendors include Anacomp, Autonomy, CA, Clearwell, FTI, Guidance Software, IBM, Interwoven, Kazeon, PSS, and Recommind.

The Gartner report is available for registration at the Clearwell site or for purchase at Gartner.

What I find interesting is that the litigation support and EDD market has long been fragmented. When I arrived at a law firm in 1989 I analyzed the lit supp vendors, which was exactly the type of work I had just done at strategy consulting firm Bain & C0. I was amazed at how fragmented the market was.

The explanation I came up with then is the same that I think applies today: purchasers are very fragmented. That is, few law firms - or law departments for that matter - buy EDD services on an institutional basis. Individual lawyers at firms and departments typically select vendors. There is a slow movement by corporations to institutionalize buying. Consolidating the number of buyers likely would do more than any other single factor to drive further vendor consolidation. Doing so would likely also continue to drive prices lower.

Law firm lay offs have cut wide and deep. At a legal knowledge management lunch today, we were wondering how law firm EDD staff have fared. Hours later, I had an answer - and it surprised me. 

I’ve gotten to know David Cowen of the Cowen Group and the blogger at Opportunity Knocks over the last year. He is a search and recruiting expert for litigation support, practice support and Electronic Data Discovery and someone who thinks deeply about this market. His blog post today (reproduced with permission) E-Discovery dodges the bullet…:

Of the 2100 layoffs since January 2009, less than 3% (60) have impacted E-Discovery and Litigation Support professionals. See Layoff Tracker for specific details.

Here’s a quick overview of how we see E-Discovery staffing in the first quarter…

#1 - Most AmLaw 200 firms are holding off on permanent staffing. This will increase the demand for contract employees and outside vendors for their E-Discovery work.

#2 - Corporate E-Discovery headcount remains flat as they continue to wrestle with budget cuts, layoffs and in-house E-Discovery planning. This should be good news for vendors and outside counsel.

#3 - Vendors are proactively and in some cases aggressively looking for talented sales and client services professionals. Good news for talented individuals that find themselves with a shaky firm.

The use of contract and temporary workers is typical during recessionary periods. If permanent staff cuts go deeper then contract staffing and outsourcing will continue to increase.

While this is not great news if you’ve been laid off it does offer firms the opportunity to hire experienced talent that is highly motivated and affordable.

I am surprised that even 3% of the lay offs are EDD staff. I know that litigation is down and the cuts are deep but I think it’s harder to find good EDD staff than lawyers. Firms are either

• short-sighted and will soon regret not having experienced EDD staff or
• are converting fixed costs to variable (meaning outsourcing EDD) or
• the litigation downturn is deeper and longer-lasting than many of us hope or expect.

Which is right?

[By the way, the Layoff Tracker David cites is a spectacular resource for those who want to follow the grim reaper.]

I’ve attended Legal Tech since 1990. Legal Tech 2009 reminds me of Legal Tech 1990. 

Back then, vendors had a plethora of new software versions, almost all based on DOS (the PC operating system). The forwarding thinkers understood that Windows loomed and would likely take over the market.

Legal Tech 2009 has many evolutionary EDD technology and law developments. The distinct feel is incremental evolution, nothing dramatically new. So in this regard, it’s like 1990. Unlike 1990, however, a looming change is not so visible; I’m not sure there’s a “Windows equivalent” that will take the market by storm.

And perhaps that’s the way it should be. The current EDD 1.0 is simply Litigation Support 2.0. That is, in spite of the hoopla, e-discovery is not all that different than paper-based discovery - the volumes are higher and the technology more complex.

EDD 2.0, whatever it ends up being, likely will not be revolutionary. It will entail more sophisticated analytic software, better statistical modeling and QC, and more rigorous processes. A magic bullet would be great but we all need to prepare for a lot of hard work in the trenches to improve what we’ve got.

Money pours into e-discovery. Courts rule on it. Lawyers debate it. Seemingly missing though are statistics. 

Is e-discovery and document review today all that different from litigation support 20 years ago? Sure, technology has changed. The underlying process, however, has not. The biggest difference is the volume of data.

There’s a field dedicated to dealing with high data volumes: statistics. EDD debates rarely includes a serious discussion about statistically valid sampling and comparisons of review approaches.

If the US system of litigation had a mechanism to pay for statisticians to assess alternative approaches to e-discovery an document review, I suspect EDD discussions - and judicial decisions - would be very different.

The February 9-13, 2008 Georgetown Law E-Discovery Training Academy has an outstanding faculty and well-conceived syllabus. As I read the session descriptions, I don’t see that statistics will enter the discussion. I hope that future e-discovery training programs will include a session devoted to statistics.

As a one-time econometrician (statistical economist), I think litigators should have enough knowledge to make an informed decision about whether statistics would help manage the EDD and review process. If lawyers doing EDD were pilots, including stats is not like making the leap from visual to instrument flight rules. It’s more like flying with your eyes open rather than closed.

Keyword searching is the e-discovery buzz. It may surprise many legal market professionals that search and keywords is really an old issue. 

Ari Kaplan, in We’re All in the Keyword Business (law.com, 15 Dec 2008), reports on a recent EDD conference. He writes:

“the recurring issue throughout the two-day exposition was the growing importance of search… There were various suggestions for refocusing keyword protocols, which included: (1) using misspellings in your list of search terms; (2) conducting sampling; (3) developing an overall search process; … develop a more sophisticated means of searching, which may include Boolean practices”

All necessary but not new. Or only new to the newbies. Some litigation support professionals grappled with similar issues throughout the 1990s. By 1991, some firms were scanning paper, OCRing, and creating full-text searchable litigation support databases. They had to deal with many of the issues we face today: under- and over-inclusive searches, finding related words, selecting the right conceptual search engine, and techniques such as sampling to perform QC.

Anyone who thinks that learning from history can be useful can have a look at my 1997 PLI outline, Lessons Learned in Litigation Support. Search the page for “search” - you may be surprised how often it occurs. And by the way, this outline describes work that was started in 1990-91 and in production by 1992-93. The money quote - from the introduction - applies equally today:

“Perhaps the greatest lesson is that the human element and human processes are more important than technology in achieving cost-effective results. The best technology does not help unless the litigation team plans for its use and changes the way it works.”

Will Uppington of Clearwell has a good post called Concept Search Versus Keyword Search in Electronic Discovery . 

It offers a helpful explanation and comparison of concept search versus categorization. He concludes that for concept searching “use to become widespread, it will need to become more transparent. But that’s a topic for another day.” I am eager to see him develop this theme.

Concept search software that I evaluated in the early 1990s relied on approaches such as statistical co-occurrence of words (e.g., the SIRE algorithm), vector analysis, state space analysis, tuples, n-grams, and thesaurus-based. I have more math training than most but generally did not understand the computational linguistics behind these.

Given the sophistication and the complexity of concept search software, I am not sure how transparent it can be. I think we need to rely on empirically testing and comparing various tools and approaches.

One measure of market size is the advertising for it. In a possible first, the Wall Street Journal today has a full-page ad for an e-discovery product. 

Turn to B5 to see (at least in the editions delivered to DC and NYC areas) a full-page ad by Autonomy for its eDiscovery product. Now we know EDD has really arrived!

The ad states “10 of 10 top law firms rely on Autonomy for eDiscovery” and that Autonomy is the “Fastest Growing EDD Provider”. I was surprised by both but suspect that as a FTSE 100 company, Autonomy carefully vets its claims. On the lighter side, I wonder how Crowell & Moring feels about the rubber ducks pictured, which feature prominently in some C&M ads.

How should lawyers deal with the ever growing volume of digital data in e-discovery? The answer likely involves far more extensive use of statistics. 

Do Your Searches Pass Judicial Scrutiny? by Wayne C. Matus and John E. Davis of Pillsbury Winthrop Shaw Pittman (New York Law Journal, 31 Oct 2008) offers an excellent paradigm for how to use concept search in EDD. Both implicit and explicit in their suggested approach is heavy reliance on statistical analysis.

The authors suggest sampling data: “Counsel should isolate a random and statistically significant sample of the relevant datasets…” This is not as easy as it sounds. Philosophically and mathematically, “random” is a difficult concept; choosing a truly random sample turns out to be difficult.

Practically, a “statistically significant sample” requires an appropriate sampling technique and defining the sample size. The latter is easier than the former; at least for well-distributed populations, the “central limit theorem” says that sample sizes do not scale linearly with the size of the population. In English: valid samples are not a fixed percent of the population. (Note that many polls of US citizens sample around 1500 people.)

The problem is that most populations are not well-distributed; instead, they are lumpy. Huh? Think of these questions: how do you know if you should sample uniformly across a data set or more heavily in key custodian files? What date range should the sample cover and might it be appropriate to over-weight certain ranges? Should you sample more from e-mail than from Word files? How do you sample Excel files, which may consist mainly of numbers. [In this political season, consider just one of the problems of sampling voters: more younger than older voters have only mobile phones and there is no way to sample mobile phones. What gottchas like this lurk in EDD?]

In all the recent cases and articles about EDD and search techniques, I believe that judges and lawyers have overlooked a critical question: how do we know if a document is responsive or privileged. Medical trials compare a new therapy a proven one, the so-called gold standard. Reproducible tests such as blood chemistry or imaging establish standards. How do lawyers know what the right designation for responsiveness and privilege is for any given document?

I’ve not seen any data I’ve seen on how reproducible lawyer designations are. Do we rely on the judgment of a contract lawyer who may not know the case that well? Is the partner in charge the authority on document designations? How many lawyers have run a document through two or more lawyer reviews and compared results? Until we can agree on a method to reproduce reliably document designations, it’s not obvious to me how we can compare search techniques. [This goes back to my prior blog posts pointing out that it’s a mistake to assume that human review gets the designations right.]

To untangle this mess, lawyers should call in the statisticians.

I am at the Masters Conference, a leading e-discovery conference. This a real-time report on the session Challenges Faced by Inhouse Counsel in the Pharmaceutical & Healthcare Industry. 

Panelists:
- Bernard Ford, Navigant [moderator]
- Peter S. Spivack, Hogan & Hartson
- David Z. Seide, WilmerHale
- Geoffrey R. Kaiser, Navigant

[Notes on this session: (1) the one in-house counsel schedule for this panel could not be present; (2) most of this session focuses more on regulatory and litigation issues than EDD so I have limited my reporting to elements of particular interest to EDD community.]

Q: What are the top three challenges inhouse counsel fact today?
A:
Kaiser: (1) The cost of managing discovery, including outside counsel review fees, data collection costs, and integrating records in acquisitions. This will continue to be - and the trend has been - to rely less on outside counsel to review docs that were over-collected in the first instance. They will invest a bit more upfront to scoping the collection, the GC can control costs. (2) Dealing with strict liability, including criminal liability, for arguably ordinary corporate actions. Cites Oxycontin case as example of execs pleading guilty to a mis-branding charge [ed. note - stemming from what is frequently called off-label use]. Thinks the misbranding issue net will expand to medical device makers.
Spivack: (1) Agrees that misbranding liability is a huge issue.
Seide: (1) Risk management and assessment is the biggest challenge. GCs have to figure out “what the hell happened”. It’s often to figure out what happened given the numbers of custodians and volume of data. Need to manage the story over the course of what may be year+ investigation as understanding the data evolves. (2) Cost is another big issue. It’s very expensive to figure out what’s in the mountain of data and documents. There is no magic way to understand data and docs - someone has to read the documents and data. (3) GCs want closure; they don’t want multi-year investigations. But EDD is very hard to do quickly and efficiently.

Q: I ask: what are GCs doing to address, other than whining?
A: Review counsel bills. Compliance systems. Active monitoring and internal audit program that goes beyond the finances (e.g., audit against procedures). Need to recognize the challenge of managing risks in globally diverse companies with so many employees. There is a lot of fear among GC of off-label use. Sales team has to be trained for this but company needs an active monitoring program to control this. Selling prevention internally is not easy.

I am at the Masters Conference, a leading e-discovery conference. This a real-time report on the session Helpful Insights & Observations from the Corporate & Government Perspectives. 

Panelists:
- David Shonka, FTC
- Paul J. Bohr, SEC
- Fred Block, SEC
- James Maroulis, Oracle
- Wendy Butler Curtis, Orrick
- Monica Palko, BearingPoint

Q: Do companies treat government requests differently?
A: It depends. Factors: is it an investigation or about a contract, which agency, scope of request. A company may more willingly turn over certain docs to government to than to commercial party. With government, less likely to fight to not turn over certain docs.

Q: Government, please comment on above.
A: As investigators, we don’t always know what we need at outset. Being understaffed, we may start with broad requests to start because we don’t have resources to narrow. Call us to discuss - we will negotiate our document requests, in all types of matters (antitrust second requests, investigations, and commercial matters). Companies should make available their IT experts, lay out how they store date, provide org chart, and a plan of how to approach the matter. Agencies look for helpful guidance from litigants. Especially in HSR matters, companies need to cooperate to close deals quickly.

Q: How many companies come to government meetings properly prepared?
A: Some staff won’t come to a meeting unless company brings the IT people.

Q: To gov’t lawyers: are you receptive to cost considerations in doc productions?
A: Yes. SEC is supposed to consider cost-benefit in its investigations. SEC will look both at dollars at stake and the programmatic goal in an investigation. SEC will provide resources in some instances to get access to data (e.g., if it is investigating a “pump and dump” operation, it might offer to image drive and review it, segregating any privileged docs). But Oracle has found that as third-party witness to insider trading cases has found some government document requests extremely burdensome.
At FTC, proportionality is a big consideration. For a mega-merger, agency cares less about production costs. The best way to control cost of production is with upfront negotiations.

Q: What happens to data once turned over to the government?
A: FTC: We provide instructions on how to produce. For example, Excel in native file format. Sometimes we have agree to having a hosting company provide access to documents. Internally, we use Summation and Concordance. Once matter ends, we return or dispose of materials unless they have become a government record. We work with parties on disposition. But there are regs that do allow for other agencies to gain access to records.
SEC: We have standard data delivery guidelines. For review, we use Concordance. Then we start reviewing. [Ed. note: Why doesn’t the government use concept search tools?]

Q: How important is it for private parties to document their production efforts?
A: FTC: It’s critical. The requirement is reasonable steps. You have to show you were reasonable. This includes litigation holds, production efforts, and chain custody. Oracle: document your “blind alleys” and why you might not have collected certain documents. Know why you did what you did. Have an audit trail of all your actions. That can be critical to show, for example, that you might have overlooked a critical doc because it was reviewed early in case before anyone could have understood its true import. Consensus of both private parties and agencies: document as much as you can about the entire discovery process (including lit hold notices and search terms).
The more you can document, the easier it will be to show that you took reasonable steps. Monica of BearingPoint points out that this is one value of using a vendor: the vendor is accustomed to documenting; moreover, vendor can testify with an expert who knows how to do this.
SEC rep says that it makes him nervous when a private party uses more than vendor because vendors take different approaches. So it’s not enough just to document, consider who is doing the work.

Q: Does government have tools beyond Concordance and Summation?
A: SEC: in certain instances, we can use systems that do more than Concordance and Summation.

Q: Do you struggle with ongoing preservation costs when cases go quiet.
A: Audience member: we had a case go quiet when we were spending $80k/month on preserving data. We went to an archiving solution. Outside counsel did communicate with the government, which was comfortable with the archive solution
FTC: Don’t assume that government will just forget about the case. But sometimes attorney may forget to send letter to close matter. So private side should check.
Oracle: We don’t want to take chance of reminding agency that a matter may still be open.
SEC: We may decide not to take action against the company. But that could put at risk ongoing investigation of an individual involved. We consider company requests to archive or change how it saves data to reduce cost as long as it’s preserved for the ongoing investigation.
Oracle: If you do get a closing letter, then take the opportunity to destroy records. But audience member suggests that matter could still come back to life.

Q: What is gov’t perspective on advanced searching tools?
A: It goes back to early discussions about data production. FTC is not likely agree to search terms at the outset because it does not have a good sense of the playing field. But if party is prepared to talk about searching custodians, specific terms, and sample results and can then show how this has been tested, then we might sign off on concept or Boolean searching. FTC is looking at new tools.

I am at the Masters Conference, a leading e-discovery conference. This a real-time report on the session The Positive Impacts of Records and Information Management on E-Discovery. 

Panelists:
- Courtney Ingraffia Barton, Crowell & Moring
- Laura A. Zubulake, Consultant and Author
- Donna Vitalie, AOL
- Maura Dunn, Duff and Phelps
- Christina Ayiotis, CSC [moderator]

Donna Vitalie presents:

• Business runs on information: decision making, develop new products, collaboration, customer information, staff management, etc.
• An active records management (RM) program helps move from the ocean of information to meeting e-discovery requirements
• RM is critical to EDD preparedness; in addition, it can help business with quick access to operational information, reduced costs of IT, consistent treatment of recrods (including destruction), and legal hold management.
• To illustrate how RM can work at its best, consider a structured scenario for procuring vendor services:
  - Check a contract management system (CMS) for approved vendors, MSAs, other useful docs or information
  - Use the CMS to build the request for proposal, distribute to vendors, and collect responses online
  - Capture final contract in CMS - now vendor work is underway
  - The actual work and invoices will be tracked in an ERP (enterprise resource planning) system
  - As long as workers use these two systems, managers and auditors only have to look in two places for all relevant information [As a records manager, you have to make sure above systems work or managers will work outside the systems, leaving records in many different places]
  - In this scenario, the “records footprint” is limited to two systems.

• In reality, some work will probably occur out of these two systems. But with a structured solution like this, in e-discovery, you can limit where you have to look.

Maura Dunn presents on RM under litigation, audit, or investigation:

• In a structured program as described above, you know where data and docs reside

• In reality though, there may be many places to look. Business applications include desktop apps, e-mail, ERP, CRM, and imaging systems. At the enterprise level, there are many separate systems.
• In the old days, RM was an afterthought. But now, with all these systems in use, RM had to occur in real time. But in stealth manner because workers don’t want actively to participate in RM. So RM has to fit in seamlessly with the varied systems in use.
• Records destruction is hard today. Paper is easy - just shred it. But what is a record today? Do we delete just one record in a database? Records managers need to understand all the systems in use. And you need a plan for holds and destruction.
• EDD has a series of separate tools. These are not designed for daily business operations. Nor should they be. You need a strategy of holds or freezes to use these appropriately.
• Once an e-discovery need arises and you need a hold, you must identify custodians and storage locations. Then collect, preserve, and review.
• One challenge is that data moves over time. Between archival processes, employees leaving, and normal operations of IT migration processes, data created in one place may eventually reside elsewhere. Typical stats: software upgrades very 18 months and hardware upgrades every 3 years. Another rule of thumb: you can’t open files after two major versions forward. Even if you can, you are likely to lose data and/or formatting. So to say 5 years later that you have an authentic representation of the original is very hard.
• It’s one thing to start a litigation hold, it’s another to release it. You need a plan of what to do with records post-release.

Courtnoy outlines three main challenges for clients:

• 1. Getting money to fund RM programs. But consider that a cost of EDD is the possibility of court sanctions if you don’t do RM right. Unfortunately, proving cost of this is hard. It’s easier to quantify the cost of dealing with the volume of data. By having a good RM program, you can demostrate savings in processing

• 2. Where do you start? Just start - put a program in place. Improvise as you go is better than not starting.
• 3. How do you implement and enforce? You need good program and managers. If not, you have to think how your organization works best at enforcing other policies. Training will be key, as will the right technologies.

Laura Zubulake (plaintiff in eponymous case). How knowing how your business actually runs can help you locate important records.

• From various records management requirements imposed by regulators, she knew that records had to exist (or were supposed to).
• Product v. country reporting lines. Knowledge of the management structure help illuminate where records would lie.
• From daily business operations, knew how reliant entire organization was on e-mail. So knew that there was a large corpus of message. Similarly, there was instant messaging (IM), then called chat.
• Chat tools were known not to be very private. So understood that any written communication about here would almost certainly be conducted in e-mail, not IM / chat.
• Understood a bit about back-end technology. Limited server space meant limits on inbox controls. Being away two weeks would fill inbox and restrict sending outgoing messages. Seeing this helped figure out back-end RM.
• There was a “double delete” process. User delete did not really delete files. There were additional steps required

A few items about e-discovery caught my eye recently: corporate insourcing, a high profile move, a tech item on MD5 hashes, and comments on the cost of privilege review. 

Insourcing EDD Keeping EDD In-House Could Contain Costs (Law Technology News, October 8, 2008) by Patrick Oot of Verizon presents the case for companies to insource EDD. I see the merits in his arguments but suspect that few companies have the scale generally or depth of expertise in EDD specifically to do this successfully.

Another High Profile EDD Move. “Julia Brickell, former associate general counsel of Altria Client Services and deputy general counsel of Philip Morris USA, has joined H5 as executive managing director and general counsel” according to an H5 press release.

Lurking E-Discovery Nightmare? The MD5 hash is detects duplicate files in EDD. Blogger Greg Duncan wonders, however, if someone could force an “MD5 collision”. A collision means identical hash values for different files. Someone with bad intent could, perhaps, in theory, make a “bad” file look like a common file such as notebook.exe. Will a judge rule now that we cannot rely on MD5 hashes to de-dup? Let’s hope not.

The Cost of Privilege Review. My Integreon colleague Chris Egan blogs on the cost of privilege review. Citing Fulbright’s Litigation Trends Survey, he points to a survey finding that suggests priv reviews may cost less than commonly thought.

There’s an old adage “the great thing about standards are there are so many to choose from.” Similarly, it seems there are more than a few questions you should ask when selecting an e-discovery vendor. 

Finding the Right Five Questions for EDD Vendors (Law Technology News, 6 Oct 2008) asks numerous EDD experts to answer the question “What are the five most important questions that any organization should ask of vendors before signing the contract?”

The range of suggested questions is staggering. Some overlap, especially if you abstract to a higher level, but I saw less commonality than I would have expected. Topics include pricing, vendor capabilities, capacity, references, project management approach, software features, resumes of key players, turnover among executives, and very specific tech questions.

My take-away is that selecting an EDD vendor cannot be reduced to five easy questions. The list of varied questions explains why EDD RFPs are often quite long. Given the complexity of e-discovery, I suppose this is not surprising. Were I novice though, I think I’d read this article and conclude that I need an expert to help me select a vendor.

The Wall Street Journal today reports on an upcoming survey on the growing cost of e-discovery costs. 

Digital Data Drive Up Legal Costs (WSJ, 6 Sep 08) reports on a survey by American College of Trial Lawyers and the Institute for the Advancement of the American Legal System that will be released on Tuesday. That survey of “1,400 lawyers who are members of the American College of Trial Lawyers” finds that 87% “said electronic discovery is too costly and is driving up litigation costs.” If so many trial lawyers, for whom, arguably, discovery is a profit center, express concern over e-discovery cost, then society better take notice.

On that note, I was not previously familiar with the Institute for the Advancement of the American Legal System, which is a “a national, non-partisan organization dedicated to improving the process and culture of the civil justice system” and a part of the University of Denver. IAALS this year (probably in July) published two e-discovery studies: The Emerging Challenge of Electronic Discovery: Strategies for American Businesses and Electronic Discovery: A View from the Front Lines (link to download both). These are long and thorough reports. I was surprised I had not come across these previously. Together, they paint a bleak picture: skyrocketing costs, unprepared companies, ignorant lawyers, and unscrupulous e-discovery vendors.

Another e-discovery acquisition. 

Access Litigation, a DC-based litigation support and e-discovery provider plus developer of NatiView Analytics, has been acquired by Edmond Scientific Company. I could not find press releases on either company’s web sites but the Washington Business Journal reports this in Edmond Scientific buys Access Litigation (29 August 2008).

I was not previously familar with Edmond, not to be confused with Edmund Scientifics, a really cool catalog company for kids, geeks, and gadget and toy seekers.

On Friday, the Wall Street Journal ran a long story on e-discovery. It highlights some of the current issues but also, in my opinion, oversimplifies them. 

Tech Firms Pitch Tools For Sifting Legal Records (22 August 2008) by Justin Scheck opens with a provocative statement: “A growing number of tech companies are riding the rising flood of corporate email and electronic records by pitching software to sift them – and meeting resistance from lawyers who want a piece of the action.”

Implicit in this article is a topic about which I have blogged regularly: the appropriate role of software versus humans in identifying and reviewing documents in discovery. This is a difficult and nuanced issues; see, for example, my posts Computer v Human Search Revisited (22 June 08), Concept Searching in E-Discovery (20 May 08), Lawyers’ Duty to Learn E-Discovery Search Techniques (18 April 08), Improving E-Discovery with Smart [Humans] [Technology] (16 March 08), and The Gold Standard for E-Discovery Document Review (18 March 07).

Another issue implicit in the article is the difference between archiving / preservation tools and collection / processing tools. See, for example, my post Preservation versus Collection in E-Discovery (8 Aug 08), which also discusses e-discovery “convergence.”

Rather than explain the difficult “humans versus computers” review question or the “convergence” issue, the article frames current e-discovery issues, in essence, as lawyers trying to protect their billable hours. This framing oversimplifies the issue but does alert corporate managers - think CFOs and CEOs - who ultimately pay BigLaw bills that perhaps there are more cost-effective ways to approach litigation. So for that reason alone, I was glad to see the WSJ cover the topic.

And the timing of the article is perfect for the many EDD managers who will be in Dallas this coming week for ITLA. It should be good grist for discussion.

AmLaw 100 law firm Dorsey & Whitney last week announced LegalMine, an integrated e-discovery service at a fixed price per document. 

LegalMine is “a powerful tool to review huge quantities of documents – efficiently, thoroughly and accurately.” According to the FAQ page of LegalMine: “Dorsey is unaware of any other law firms currently offering a per document or per-page pricing model for document review.”

This is indeed the first I’ve heard of a law firm doing this though I know that at least three e-discovery vendors announced in 2008 fixed price services: Huron, LawScribe, and Integreon. It will be interesting to see if Dorsey’s move influences other law firms or the overall EDD market. In my January 2007 blog post, Coming E-Discovery Battle between Vendors and Firms?, I discussed potential competition between vendors and law firms for EDD consulting dollars. The dimensions of the competition seem to be expanding. That said, my guess is that Dorsey’s move will have a greater impact on other law firms than on vendors. Let the document review market share war begin!

If corporations succeed in managing and preserving information, will e-discovery collection problems disappear? 

I’ve previously written about ”e-discovery convergence,” the idea that enterprise “information life cycle” management products may reduce or eliminate the need for EDD collection.

Is Preservation in E-Discovery Overrated? at the Clearwell blog (23 June 2008) argues that because litigation holds are inevitably imperfect, corporations should instead focus on rapid collection of relevant documents: “the best way to take the risk out of the legal hold process is to move very rapidly from preservation to collection.”

I view this conclusion, as I do many others in EDD, as a hypothesis that requires empirical testing. Since I lack data, I can only respond conceptually. Litigation holds are more of a “flow” than a “stock” (flows and stocks are standard ideas in accounting and economics). Collecting data at one point in time is often not enough; corporations frequently have an obligation, arising from one or more cases, to preserve data over an extended period - a “flow”. If the obligation continues as a flow, I don’t see how rapid collection helps very much. If, in contrast, there is no ongoing hold - the collection is a “stock” - then the proposed strategy seems to have merit.

One reason to take an empirical approach is that answers may change over time. Consider that companies have multiple reasons to control information. Beyond litigation holds, factors such as storage costs, knowledge management, compliance requirements, and protecting trade secrets drive preservation considerations. With multiple reasons to preserve - or destroy - data, it seems inevitable that companies will focus more effort on preservation and destruction. If so, some of the current risk of lit holds may diminish. That said, I think the points in the Clearwell post about rapid collection to assess requirements and narrow the focus has much merit given where information management is today.

E-discovery is hot. As well it should be with the explosion in data volume and relatively swift (for the legal market) transition from paper to digital data. A just-released salary survey of law firm EDD and litigation support staff puts some parameters on the market demand. 

The Cowen Group is a “leading search firm specializing in staffing and recruiting for Litigation Support, Practice Support and Electronic Data Discovery from coast to coast and abroad.” This week they released survey data on law firm salaries for litigation support and EDD; also included is the number of searches by position type that the firm has conducted since 2005. Register here to get the The Cowen Group 2008 Salary Survey.

Some highlights I found interesting:

• Job descriptions and salary ranges for positions from lit supp analyst up to firm-wide director of litigation and practice support. The salary range, for major east coast cities, is from $60k to $325k
• Eyeballing the time series data presented graphically, I’d say salaries for most positions have increased 50% over the last 3 or 4 years
• The number of searches for most positions has doubled in the same time period. Searches for senior positions in 2008, however, seem to be slowing down. This makes sense: firms have hired more senior managers over the last couple of years and asked them to build out their departments.
• No surprise that NYC positions pay more than DC or Philadelphia. My guess though is the difference does not suffice to really cover the higher cost of living in NYC.

Granted, the above are based on one firm’s experience and we have no data on Cowen Group’s relative market share for search. Nonetheless, given the paucity of good data in this market, I will take what I can get and what I see supports the more anecdotal evidence I’ve heard.

On July 24, Interwoven, best known in the legal market as a leading developer of document management software for large law firms, announced it will acquire e-discovery player Discovery Mining, Inc. 

According the Interwoven press release, “The acquisition of Discovery Mining extends Interwoven’s offering – the legal industry’s de-facto standard for organizing, finding, and governing matter content – by adding a service for managing eDiscovery. Most organizations do not have the infrastructure to handle large volumes of information, and are turning to companies like Discovery Mining to manage the eDiscovery process for them.”

I’ve previously written about EDD convergence, the idea that enterprise content management, storage, and “big iron” companies will end up “owning” a good part of the discovery market. This continues the trend. The twist here is that Interwoven, unlike most other enterprise content management companies, already has deep legal market domain expertise.

[For reference, here is a list of e-discovery acquisitions that I have tracked.]

The American way is “innocent until proven guilty.” The American Lawyer way seems to be “guilty by association”. 

The July 2008 American Lawyer, in the “Bar Talk” section, has an article Irony, Thy Name is Guidance sub-titled “Howrey’s EDD provider has EDD problems of its own.” The short article reports that discovery practices of Guidance Software, Inc., makers of popular EDD software EnCase, “have come under attack” for how it produced e-mail in an employment matter.

That an EDD vendor may have its own EDD problems is perhaps newsworthy. But the article opens by asking “Did Howrey bet on the wrong horse when in linked up with electronic data discovery provider Guidance Software, Inc. in May?” Two paragraphs later: “The same month that Guidance announced its alliance with Howrey, an arbitrator in Los Angeles admonished the Pasadena-based company for its discovery practices in an employment suit in which it is a defendant.”

So what? The implication seems that Howrey is somehow at fault. I don’t get it. The magazine, instead of analyzing the business value of Howrey’s alliance with Guidance, implies the firm has chosen poorly. What is the standard of care for a law firm (or any entity for that matter) in choosing products or business partners? Should customers not use a leading product because the company has what may be a minor legal problem?

For the many BigLaw partners I’ve met over the years who believe that American Lawyer has an axe (or two or three) to grind, this article will certainly confirm their views.

Can lawyers rely on search to find documents in e-discovery? Recent decisions suggest perhaps not. 

Craig Ball’s blog post, Grimm Prognosis for ESI Search (6 June 2008) recaps decisions by Magistrate Judge John Facciola in U.S v. O’Keefe and Equity Analytics v. Lundin and by Magistrate Judge Paul Grimm in Victor Stanley, Inc. v. Creative Pipe, Inc.. Ball captures a central issue: “It’s assumed that lawyers are qualified to review documents and decide their relevance, responsiveness and privileged character. But are we qualified to craft proxies for our judgment in the form of keyword searches?”

Ball quotes Judge Grimm’s ruling that testing ("utmost care in selecting methodology") is required to rely on software searches. I’m all for process and testing and the utmost care - if applied uniformly, including to lawyer performance on document review. Most judges and lawyers seem to assume that lawyers are (1) qualified to review documents and (2) better at doing so than software. The truth of either is purely an empirical question; the presumption of lawyer accuracy may well be false.

I’ve seen no evidence that passing the bar qualifies lawyers to review documents accurately. Even if it does, I’ve seen no evidence that lawyers can maintain accuracy reviewing documents hours on end (which is the norm). In fact, I’ve seen contrary evidence. Even if lawyers are qualified and, almost miraculously, do not falter as the hours roll by, do they always agree on how to classify a document? I have seen many instances where lawyers disagree. Of course, if qualified and alert lawyers can disagree about document designations, where does that leave software?

I’ve raised a similar point previously in response to a Law Tech News October 2006 article proposing to place a high burden on EDD vendors. LTN published my reader response arguing that the same burden should apply to lawyers. My logic there applies here.

The legal profession seems ready to set high standards for vendors and for software. Let’s be consistent and apply the same to lawyers reviewing documents. If we do, will any approach meet the standard of care implicit in Judge Grimm’s decision?

As the profession ponders how to conduct e-discovery, consider some practical ramifications. Blogger-lawyers Jim Beck and Mark Hermann of the Drug and Device law blog wrote a great post called Will Technology Increasingly Favor Plaintiffs?. The gist is that the long-standing gap between the discovery effort for plaintiffs and defendants has grown dramatically in the age of digital data. They note that

“We fear that search technology will increasingly permit plaintiffs to identify with relative ease the specific information that they need, but the technology will be far less helpful to defendants trying to separate the wheat from the chaffe.”

They rightfully fret that search technology “assume[s] that you know what you’re looking for” but observe the defense lawyer’s job is to put information in context and that context may not be knowable in advance.

As a profession, we need to
1. Develop consistent, evidence-based standards for how we manage discovery and review and
2. Consider whether the “e” in “e-discovery” has tilted the playing field unacceptably and, if so, whether there are policies we can create to fix the problem.

If National Public Radio reporting is a sign, then e-discovery is breaking out of the legal press and into the public consciousness. 

E-Mail, the Workplace and the Electronic Paper Trail, a Morning Edition segment on 18 June 2008 provides a short, lay person view of some of the many EDD challenges. The text and audio appear identical.

The report seems accurate to me. If you are an EDD or litigation support professional, there is no news here but that’s fine. The value to you of this piece is (1) to help educate the many lawyers who still have their head in the sand about EDD and (2) to show something to your friends that is digestible and professionally produced so they can begin to understand what you do (Hey, listen to this Morning Edition segment - it’s all about what I do at work every day!).

FTI Consulting announced on Tuesday that it is acquiring Attenex. 

FTI is already a major player in e-discovery; it also owns popular litigation support software Ringtail. The press release, FTI Consulting, Inc. to Acquire Attenex Corporation, dated 10 June 2008, enumerates several expected strategic advantages, including “Accelerates Penetration of Corporate Market for in-house eDiscovery Processing.” There will be a public conference call on Wednesday (6/11/08) at 9am EDT.

Over the last 15 months, Lextranet, MetaLINCs, Stratify, and now Attenex have been acquired. MetaLINCs is the only one where the company acquiring was not already an obvious EDD player. Who’s next?

[See my list of EDD acquisitions; includes dates and links.]

Update (15 June 2006) For a financial analysis of this deal, including deal multiples and a comparison to the Seagate acquisition of MetalLINCS, see FTI Consulting Acquires Attenex for $88 million over at Clearwell’s blog.

Should courts accept e-discovery productions generated by concept searching? 

That’s the question addressed in When E-Discovery Is Put to the Test (Pennsylvania Law Weekly, 14 May 2008). This article takes a deep dive into a potential Daubert challenge to using concept searching. Some lawyers will find it tough reading but, as I argued in Lawyers’ Duty to Learn E-Discovery Search Techniques, that may be the price of admission for some litigators.

This article adds to an important professional discussion about e-discovery tools and it is worth reading. I take issue, however, with the author’s analysis, which contends that courts cannot evaluate concept search tools because the tools are proprietary. I see three issues with this assertion:
1. In my experience, only computational linguists can understand advanced concept search algorithms. I’ve tried reading academic or vendor literature on how the algorithms work and cannot get past the first paragraph. (Let the record show I have three years of college math major classes plus I worked professionally as an econometrician.) So even if you could get at the source code, it’s not easy to evaluate. Plus, my past conversations with computational linguists and software developers suggest that they typically do not agree on the efficacy of algorithms.
2. Even if everyone agrees an algorithm works, that does not mean the software works. You still have to show that it was translated into error-free computer code. It’s a rare piece of complicated code that is error-free.
3. Even if the code can be vetted, it may have to be vetted on a specific operating system or even on specific hardware. And modern software often interacts with other pieces of code - does all associated code need to be examined as well? (Along these lines, remember the problem Applied Discovery experienced.)

So the proprietary nature of the software seems only a piece of the problem. I would try to side step the whole Daubert issue . In my view, Judge Waxse’s comments at a Legal Tech session - legal system requires reasonableness, not precision; plus costs must be reasonable - offer a better framework for thinking about the most appropriate tools for e-discovery.

Benchmarking is not the same as analyzing a process. 

In Law departments rarely benchmark processes, law department consultant Rees Morrison explains that general counsel benchmark ratios but “very few law departments formally benchmark key processes, such as how to manage large numbers of contracts.” In my experience, you can say the same for outside counsel. This explains why it’s hard for me to find topics for my legal best practices blog category.

Paying more attention to process might accelerate the halting change in how lawyers practice and help address risk aversion. Take a look at one process: e-discovery nee litigation support. A chronology of developments - poetic license taken - illustrates what happens when lawyers don’t confront process:

• We can’ t use OCR (optical character recognition) because it’s not 100% accurate. Never mind that the alternative, objective coding, is at best about 98% accurate. [1992]
• We can’t use Boolean search of full text instead of objective coding because we will miss key documents. Never mind that there is good evidence that Boolean search, with all it limitations, typically finds more relevant documents than bibliographic coding. [1993-94]
• We shouldn’t ask for opposing party e-mail because then they might ask for ours. Never mind that it was obvious a decade plus ago that in many organizations, e-mail was critical. [1996]
• We can’t use a hosted repository, it might not be secure. Never mind that few firms had performed security audits on their own systems. [2000]
• We have to produce documents in fixed file formats such as TIFF or PDF with Bates numbers. Never mind that spreadsheets have no natural pagination and that many PowerPoint decks have crucial animation sequences. [2003]
• We can’t use software to screen for documents, it might not find all of them. Never mind that the alternative, human review, has errors and there is little data to compare the relative error rates. [Now]
• We can’t use offshore lawyers, they might make mistakes. Never mind that the alternative, armies of domestic lawyers, make mistakes. [Now]

For a profession that loves to dissect problems into issues, it’s surprising lawyers are typically unwilling or unable to dissect and examine processes. This reduces their effectiveness and the only reason they get away with it is that the clients don’t know any better.

I’ve long thought that litigators should more frequently use litigation risk analysis using formal decision trees. 

I am swimming against the tide. In Making the Case for Change (ABA Journal, April 2008), the sub-article Case Evaluation: Too much em­phasis on a quantitative ap­proach often misses key factors by Louis M. Solomon of Proskauer criticizes over-reliance on decision trees: “An overly numerical approach to case evaluation has serious limitations. Formal case evaluation seeks to achieve an optimal identification of possible outcomes, but perfection in the process cannot be achieved. An unthinking use or overuse of mathematically driven case evaluations does not serve clients well.”

Beyond complexity, Solomon lists three limitations of decision trees:
1. Overemphasis of quantifiable factors
2. Discounting small probabilities
3. Inadequate consideration of the client’s risk tolerance

He closes, however, by acknowledging its potential value, especially to dispel fuzzy thinking and statements like “you have a good chance of winning.”

I’m not a litigator and I don’t know the right answer. But I view this technique as I do drugs, devices, or medical procedures. What does the evidence say? I’m not sure enough litigators and clients have used decision trees properly and compared alternative risk assessment methods to reach an empirically sound conclusion. Sure decision trees are hard and have limits. But for cases about money (and most are), what is the better approach to assess risk and set up a framework for discussing decisions? If achieving “perfection in the process” is the standard, we may as well give up at the outset. The fact that doing risk analysis is hard may mean lawyers need more training, not that they should ignore a potentially valuable technique.

A Rees Morrison blog post alerted me to this article. In a separate post, Law firms, law departments and the asymmetry of concern about money Morrison concludes that “firms and departments exhibit an imbalanced focus on money. Where many people outside focus on increasing money, relatively few people inside – and only part of the time and reluctantly at that – focus on holding the line” Perhaps if inhouse counsel were more concerned about holding the line, they’d eagerly employ decision trees. Have they even used risk analysis enough to say it’s not worth the effort?

Finding and filtering relevant documents in e-discovery is a big challenge. What steps must a lawyer take? Just how far does a lawyer’s ethical obligation go? 

In Improving E-Discovery with Smart [Humans] [Technology] (16 March 2008) I suggested that lawyers must learn enough about e-discovery software to use it effectively to search for and find relevant documents. (But I agree with Will Uppington’s comment that the software could be easier to learn.)

Craig Ball, in his always excellent Law Technology News column, addresses this issue in The Science of Search (April 2008). He concludes that lawyers

“need to learn more about the science of search as part of our legal and continuing education. We need to become skilled at tools and methods that help us refine searches and routinely test them against representative data so we can distinguish noisy terms from effective ones and learn to zero in on relevant ESI.”

Does a litigator’s ethical duty of zealous representation extend to learning search software. As I understand it, difficulty does not excuse a lawyer from a duty but may be a basis for delegation with supervision. Ethics aside, I agree with Ball that as a practical matter, lawyers must learn how to search. Unfortunately, this challenge is bigger than meets the eye.

Ball also writes that “lawyers believe themselves adept at keyword search in e-discovery because they’ve mastered keyword search in online legal research.” Lawyers may honestly believe they know how to search, but I suspect this is another honestly held but wrong belief. My conversations with law librarians over two decades suggest many lawyers have not mastered online legal research search techniques.

To clarify a lawyers’ obligation, all we need is a malpractice case. It would allege that a lawyers failure competently to search an e-discovery database led to a bad outcome. Nothing like a malpractice case to cause lawyers to pay attention.

Does finding the best way to search through huge volumes of e-data sometimes feel like the quest for the Holy Grail? 

I’ve previously written about applying smart search tools and using offshore lawyers to manage e-discovery reviews more effectively. In thinking about process improvement, we should not forget the importance of human expertise in conducting search. Better Search for E-Discovery by Will Uppington at the E-Discovery 2.0 blog is a good summary of recent TREC research. A key finding: one of the most effective techniques to reduce the volume of docs to review is to have expert searchers iteratively conduct searches. As the post notes, this may be obvious but that does not mean it’s widely followed.

I think this has been true for decades and will likely remain true absent a software revolution. So I disagree with Uppington’s explanation for why iterative search by experts does not occur in all cases: “the single biggest reason is that the technology used to perform searches for e-discovery has simply not been easy enough for legal experts to use.”

Until computers can read our minds (and I hope that days never comes), lawyers and their colleagues must “communicate” complex information to software. I’m not sure that this process can be so simplified that lawyer-experts don’t have to learn something about the search tool.

The seeming simplicity of Google lulls many into an expectation that one word searches suffice. There are good and bad Google searchers. Laziness in or fear of learning to conduct good Google searches should not excuse the same when it comes to EDD. We all expect our doctors and surgeons to learn their tools - why should we expect any less from lawyers?

The WSJ Law blog recently wrote about “a horrific tale of document review drudgery.”  

Sound of the First Year Workin’ on the . . . Doc Review Gang (15 Feb 08):

“the Law Blog was regaled with a horrific tale of document review drudgery… Reportedly, the doc review team — sedentary for so long — would conduct late-night cartwheel contests … The story was neither easy to tell nor easy to hear…. It reminded the Law Blog of our own doc review war story… What followed was four weeks of sixteen-hour-days clicking away on a computer program whose name we’ve blocked from memory. In fact, we’ve tried hard to block the whole four weeks from memory.”

While I was at NYU Law (’86) and as a summer associate in three law firms over two summers I often heard about the travails of document review work. I went to work as a management consultant at Bain when I graduated. In 1989, a now AmLaw 20 firm created a position for me that today would be the equivalent of Director of Practice Support Systems. I think I was the first non-practicing lawyer in that type of position.

When hearing about my job, most junior litigation associates said something like “Wow, that is really cool. I’m envious. I can’t believe how much time I spend reviewing documents and how boring it is.” It was obvious then that associates did not like doing document reviews.

Little has changed since then. Whether you spend your day flipping through stacks of paper or hitting the “next doc / page” button makes little difference. The WSJ blog post reminds us that though doc review is work that needs doing, many who do it would rather be practicing a different type of law.

More consolidation in e-discovery. 

“FTI Consulting, Inc. (NYSE:FCN), the global business advisory firm dedicated to helping organizations protect and enhance their enterprise value, today announced the acquisition of Strategic Discovery, Inc. (“SDI”), a premier firm known for its market and thought leadership as well as its steadfast commitment to advancing rational and cost effective responses for corporate clients facing Electronically Stored Information (ESI) demands.” 12 Feb 2008 press release.

I’ve known SDI founder and president Adam Bendell for many years. He is one of the best and brightest in this field. Early in my prior Prism Legal Consulting practice, I was fortunate to be able to work closely with Adam and his team. FTI is very fortunate to have SDI’s outstanding team and I wish SDI the best in its new home.

Live from Legal Tech NYC, a session on empirical research on e-discovery, specifically the reliability and value of using computers to review document. 

The session: “The Electronic Discovery Institute is a 501©(3) non-profit corporation dedicated to resolving the legal community’s electronic discovery challenges. The Institute’s study compares the time, cost and accuracy of traditional, manual document review processes with computer assisted categorization tools.”

The panelists:
The Honorable David Waxse, Federal Magistrate Judge District of Kansas
Craig Ball,Esq. Attorney & Computer Forensic Examiner
Julia Brickell, Esq., Associate General Counsel, Altria
Peter Gronvall, Esq., Managing Director, AdamsGrayson
Anne Kershaw, Esq., EDI President & founder of A.Kershaw PC/Attorneys & Consultants
Laura Kibbe, Esq., Senior Corporate Counsel & Managing Director, Pfizer, Inc.
Jonathan Nystrom, EDI Study Participant & Vice President, Cataphora
Patrick Oot, Esq., EDI Vice President & Director of Electronic Discovery, Senior Counsel, Verizon [MODERATOR]
Herb Roitblat, Ph.D., EDI Chairman & Principal, OrcaTec LLC
Rich Tobey, CPA, EDI Study Participant & Managing Partner,Vmax Consulting

Oot opens by pointing out that the real goal in e-discovery is justice. Rule 1 of the FRCP references securing “the just, speedy, and inexpensive determination of every action and proceeding.”

Start with the notion that assessing relevancy is difficult. Oot references his involvement in Verizon acquisition of MCI. They used traditional 2nd request review process with much manual review. 83 custodians, 2.3 million documents, 2 law firms involved with one deploying 115 lawyers and the 2nd deploying 110 lawyers to conduct privilege and relevance review. It took four months of long days. The cost of document review was just shy of $13.5 million. Note that this matter was not big by today’s standards. FTC would not allow the parties to use key word searches to narrow the document review. “There’s got to a better way to do this than all the human review.”

Oot and Kershaw started the eDiscovery Institute (EDI) to study if there is a better way to conduct document review. Kershaw now summarizes the Institute: The idea started a few years ago with a private review Kershaw did comparing two approaches to document review. Judges and others wanted more data to compare approaches. Work today has just scratched the surface - much remains to be done. Institute is a not-for-profit and is set up to do additional studies to ease the pain of conducting litigation.

EDI’s first study compared traditional doc review with an electronically assisted process. EDI will publish a white paper in early 2008; it will be peer-reviewed and available freely. Views EDI as unique organization to provide factual information (Sedonna focuses on princicples). Pfizer and Verizon are current sponsors but EDI seeks additional sponsors. EDI will not be a vendor or process certification organization - it will report on factual findings.

QUESTIONS EDI WILL ADDRESS
- Should a party consider alternative methods to brute force review?
- Is computer assisted relevancy assessment reasonable under the Rules?
- Is any process reasonable?

The study dataset: The MCI-Verizon acquisiton data for antitrust 2nd request - 83 custodians in 10 states, 13. terabytes, over 2 million documents.

Roitblatt describes study: Quantitative measurement is key. References the seminal Blair-Moran 1985 study that found that researchers are only 20% accurate in finding docs but thought they were 80% accurate. The way to measure accuracy is to measure actual performance against the “the truth.” You have to approximate the truth. [Editor: in medicine, this might be called the gold standdard.] How do you define the “baseline” of the objectively or widely accepted definition of relevance of each document. Must consider both false positives and false negatives. Precision is percent of docs selected that are truly relevant. Recall is percent of relevant docs actually retrieved. Elusion is percent of docs not retrieved that are relevant.

Key question is what we can actually measure? What are the appropriate “power tools” for e-discovery (versus manual review)? To answer, start by looking at ESI review process: training, case background, examples combined with experience lead to judgments of whether a document is responsive or not. In a 2nd tier review, typically reviewers only look at what first round designated as responsive. So two tier review has problem that relevance calls on first round are not necessarily carefully reviewed.

How does a computer get experience to separate responsive from non-responsive docs? It’s all just mathematics. The competition among vendors is who has the better math. The process with computers is based on rules, text, and math applied to docs. Computer approach may “recurse,” that is, adjust its process based on feedback from human reviewers. For study, “true” designation of document is based on original work of MCI-Verizon team.

Roitblatt describes the famous Turing test for artificial intelligence: can a human tell the difference between a computer and a human in a text interface, interactive conversation. By extension, a computer aided review should be comparable to a human review.

PROVISIONAL RESULTS OF STUDY: 4 computer systems agreement with original attorney review ranged from 72% to 88%. (For this comparison, the original review is considered as the “truth.") Note that in human reviews, where there are multiple human reviewers, rate of agreement among the humans is typically lower.

Question to Judge: what happens when issues of best method come to the court. The Judge says it’s better for the parties to collaborate to come to a shared view on this topic. Disagreement should be aired at 16b conference. Legal system requires reasonableness, not precision. Plus it requires reasonable cost. [Editor: this begs the question of how precise is precise enough to be reasonable.]

Brickell: it is no longer reasonable to presume humans should review all the documents.
Craig Ball: If parties cooperate, they can agree on a reasonable method. Cautions that studies show that human review, by some measures, are only 40% accurate. Should computers be designed to make the same errors that humans make? Compares issues here to Google ranking, where links, which are made by humans (at least in theory), are a form of group voting.
Kershaw: Many discovery requests presume way too much is relevant. These studies may help narrow scope of what we generally consider as relevant. Low accuracy of human review reflects inconsistent judgment and fatigue.

Panel discussion continues at 1140am but other Legal Tech events beckon…

[Editor’s note: In Thoughts on Full Text Retrieval (a KM and litigation support topic) (July 2003), I noted that “What we need as a profession is a mechanism to perform real-world tests, both on how the search tools perform under the most favorable conditions and how they work when actual users operate them. Unfortunately, this is costly and the incentives and structures to do so just do not exist.” It’s great to see that this is finally beginning to happen.]

Some industry commentators speculate that e-discovery as a separate product category (if not discipline) will disappear as enterprise software vendors get in on the act. 

I’ve previously written about EMC’s foray into EDD. Now, HP joins the game. A Clearwell press release (29 Jan 2008) announces that HP enters e-Discovery market by reselling Clearwell. Now that Seagate, EMC, and HP have irons in the fire, can IBM be far behind? Five years from now, big companies may well have most of the e-discovery features they need built into corporate enterprise systems. If so, it will be interesting to see the impact on the current EDD leaders.

I’ve previously posted about the convergence of corporate information life cycle management systems and the e-discovery market, including the entry of EMC into the market in February 2006. Now, the next move. 

In the current (Jan 2008) issue of American Lawyer magazine EMC has a white paper as a 4-page inserted advertisement ("A Practical Approach - Ten-step framework for effective policy management and in-house eDiscovery"). It’s targeted toward inhouse counsel with a message about managing information to avoid EDD problems. The same insert also appears in the January Law Tech News.

Ads do not equate to market share but can serve as an indication of intent. EMC may be poised to invest more in the discovery market. That said, SAP placed an 8-page essay in the September 2005 American Lawyer and I did not see a subsequent push in legal.

Speaking of ads as indicators, CRA International also has ads in the current American Lawyer offering “legal business consulting” for “running a corporate legal department.” That’s the first I’ve seen of this company targeting corporate law departments for legal operations and other services.

Alas, we may never know the legal standard of care for an e-discovery vendor. 

A pair of lawsuits that could have made for some great drama and established a vendor standard of care has been settled. I previously blogged about a dispute between Sullivan & Cromwell and EED. Both S&C’s initial suit in NY state and EED’s countersuit in Washington state have been settled according to Sullivan & Cromwell, E-Discovery Vendor Settle Lawsuits (National Law Journal 18 Jan 2008): “The short-lived clash was believed to be the first of its kind between a major law firm and a leading vendor in the growing area of electronic discovery services.”

Fear not - given the volume of EDD, I suspect it’s only a matter of time before we see another such suit.

Consolidation of the e-discovery (EDD) market continues apace. 

Press Release excerpt: “Deloitte Financial Advisory Services LLP (Deloitte FAS) announced today that it has acquired certain assets of Barrasso Consulting LLC (Barrasso), a firm specializing in planning and executing litigation discovery programs, to further enhance Deloitte FAS’ existing national Forensic & Dispute Services practice…. Barrasso Consulting will be fully integrated within Deloitte FAS’ Forensic & Dispute Services practice as a new service line, called Document Review Services. Approximately 80 Barrasso Consulting professionals will form the foundation of this new service line, including Barrasso founder and president Diane Barrasso. Barrasso will join Deloitte FAS as a principal and national service line leader of the Document Review Services team.”

Everyone who works in e-discovery knows it is a messy process and prone to errors. How many mistakes are too many though? We may soon find out. 

Sullivan & Cromwell Suit Against Vendor Highlights Problems With E-Discovery (National Law Journal, 7 Jan 07) reports that S&C is suing e-discovery vendor EED:

“In a complaint filed Dec. 28, 2007, in the Southern District of New York, Sullivan & Cromwell said, “untimely and inaccurate” work by Electronic Evidence Discovery Inc. (EED) hindered the law firm’s staffing arrangements and caused it to expend extra resources on discovery. The firm asked for a ruling that EED was not entitled to collect $710,000 in outstanding bills.”

In my experience, large law firms are not quick to become plaintiffs. While they don’t like to deal with vendor mistakes, they tolerate some errors and deadlines slipping generally, even more so for e-discovery. So the fact that a large law firm is suing a vendor suggests, in my opinion, particularly bad vendor problems, action to protect a client relationship, or heading off a malpractice claim.

Update (9 Jan 2008) On January 8th, EED filed suit against Sullivan & Cromwell. “EED seeks to enforce the terms of its contract with Sullivan & Cromwell and to recover outstanding fees for services rendered during early- and mid-2007.” The press release also quotes the CEO saying that this is “dispute regarding payment of a valid receivable.”

Just how far can concept searching and semantic analysis go in automating document review in discovery? 

No one knows yet. The answer depends in part on another question: Is the ability of software accurately to assess responsive documents in a large collection constrained by linguistics (algorithms) or by computational limits (the cost of raw processing power)?

Advanced e-discovery software has proliferated recently. My sense is that these products are limited by the underlying techniques of full-text search and analysis, not by limits on affordable processing power. (See my July 2004 post, Developments in Full Text Searching, in which I share comments of a computational linguist on this issue).

If, however, the limit is processing data in a reasonable amount of time, then advances in computing speed would improve results. A recent article suggests that the moral equivalent of supercomputing will soon be available to the mass market. Google and the Wisdom of Clouds in the December 24th issue of Business Week explains that Google and IBM are developing “cloud computing,” the ability to use the vast networks of servers as a virtual supercomputer. If this succeeds, then lawyers will be able to run search software “in the cloud.” (I am assuming confidentiality issues can be resolved.) At that point, we would know better just how much massive computing power can replace human review.

Holiday toast overheard tonight: “Long live e-discovery confusion”. EDD confusion is good for service providers, vendors and law firms. Not so for General Counsels, who should do more to reduce confusion. 

CommVault, a provider of data archiving, protection, and resource management systems with e-discovery solutions, recently released a a press release on a survey of EDD preparedness. The key finding: of 375 IT managers responding “only 20 percent are aware of the [FRCP] amendments governing e-discovery, and only 6 percent of the total respondents have actually implemented a solution to address the issue.”

The amended rules are old news. Even eWeek and Information Week have carried numerous stories. So IT managers have no excuse, but the real responsibility lies with GC. They should educate IT, which would do a lot to reduce confusion.

The press release implicitly chastises IT managers: “Nearly as startling… only 14 percent of IT departments had made any effort to engage the legal department on the new standards.” Even IT managers with dim awareness of EDD have little motivation to learn more; they have enough else to do. Of course, GC are also busy, but it’s hard to argue that establishing policies to deal with data doesn’t deserve attention.

Are you surprised that disk drive maker Seagate Technologies acquired e-discovery software developer MetaLINCS last week? 

According to the press release, this and prior acquisitions allow Seagate to “offer corporations, law firms, and litigation service partners a technology leading analytics engine along with one stop sourcing for archive, recovery and collection, review tools and services”.

This acquisition continues the active e-discovery deal-making (see my October 2007 list of EDD acquisitions). By now, it should not surprise anyone that a disk drive maker is getting into e-discovery. First, storage system company EMC entered the EDD market almost 2 years ago (see EMC eDiscovery Services and my Feb 2006 blog post E-Discovery Convergence at Hand? ). And second, noted industry analyst Michael Clark of EDDix has long predicted the convergence of e-discovery with enterprise solutions providers: see his comments in my post Oracle and E-Discovery?, Nov 2006.

Those interested in how MetaLINCS as part of Seagate will play out can read Mark Reichenbach’s blog, On the Mark. Mark is an experienced EDD professional and MetaLINCS VP, Client and Industry Development.

Disclosure: My employer Integreon offers MetaLINCS services as part of its e-discovery and offshore document review services. That said, I blogged about MetaLINCs long before joining Integreon: see Another E-Discovery Tool (Aug 2005) and my listing it as one of my 5 legal tech picks for 2006 in January 2006.

Does your firm have a clear policy to check and clear conflicts for temp or contract lawyers? 

A lawyer / e-discovery expert asked me this yesterday. With all the articles and conferences about EDD, I don’t recall seeing much on this topic. Nor have I heard anything in private discussions. Some tough issues need to be addressed. Do you need to treat contract or temp lawyers the same as any lateral lawyer entering your firm? How well do agencies or the temps themselves keep track of all the past matters on which they’ve worked? What happens if a temp’s memory is not reliable? Does it matter that a temp might only have worked on a project for a day? Does it matter who pays the temp lawyers (law firm versus the client)? Or where the temps work (your space versus the temp agency’s)?

Does anyone know of published material on these questions or have a policy they are willing to share? If so, please let me know at ron at prismlegal dot com.

Who would you rather have review litigation documents: contract (temp) lawyers or full-time lawyers in India? 

In Document Review in the US versus India (8/07) I suggested that Indian lawyers likely do better reviewing documents than US contract lawyers.

For a well-written, colorful, amusing, and depressing article about the life of temp lawyers doing doc review, read Attorney at Blah (Washington City Paper, 11/7/07). Those skeptical of offshore review should read this for good anecdotal insight into the typical domestic alternative.

I found this article referenced at The Secret Life of Contract Lawyers at Legal Blog Watch, where Carolyn Elefant comments on the article and provides links to other resources re document review and contract lawyers.

Should law firms do e-discovery data collection and processing inhouse? 

Most large firms have - and should have - the capacity to process and collect some amount of data. The question is how much. As far as I know, only a handful of large firms have significant in-house EDD collection / processing and run it as a significant profit center.

Personally, I would avoid the risk of mistakes and challenge of keeping up with ever-changing technology. A more general issue is the role of businesses beyond pure practice inside of or connected to a firm. Law Firms Backing Away From Affiliate Businesses (National Law Journal, 11/2/07) reports that many firms are selling affiliates and few are opening new ones.

Whether a big e-discovery processing center is run as a legally separate affiliate or not, I think it raises the same issue: should law firms be in businesses other than law practice? I think that this article adds to the reasons to avoid turning EDD processing into a business venture.

“McDermott, Will & Emery plans to create a new tier of attorneys – think of them as permanent contract associates – to handle lower-end tasks at lower billing rates." 

So reports McDermott Will to Add Lower-Paid Associates (The Recorder, 11/2/07). A big focus of these new attorneys will be document review. I am all for any step a large law firm can take that reduces legal costs. Some comments on three points in the article:

1. “While hiring contract attorneys is nothing new, creating a second class of full-timers is.” Fact checking please! Look back two decades: Law Firms Add Second Tier (New York Times, March 11, 1987) reports on Jones Day hiring “staff attorneys.” Or just one year ago, read in sibling ALM publication American Lawyer, Temporary Lawyer (Sep 06), “Some [large firms], like Skadden, Arps, Slate, Meagher & Flom and Sullivan & Cromwell, have created a new category of full-time attorneys outside the associate track to handle document review in place of temps.”

2. The new McDermott Will attorneys will “put in more like 30 to 40 hours and be paid something like 25 percent less, though an exact pay range hasn’t been decided.” Work what sounds like 1/3 to 1/2 the hours as many associates for 25% less… will some associates want “down grades?”

3. A Quinn Emanuel Urquhart Oliver & Hedges partner is quoted on the alternative of conducting reviews offshore: “In high-stakes litigation, it would be giving up too much control. A document that might seem harmless might be the key to victory, and we’d be really fearful that sort of thing would be missed.” As I’ve written many times (see, e.g., Onshore v Offshore Pivot Point - Part II (9/13/06)), this is an empirical question. Lawyers assume - usually without evidence - that whatever they are doing now works and works well. In my experience, US document reviews have fewer controls and QC processes than those conducted offshore.

More consolidation in the e-discovery (EDD) market. 

Iron Mountain today announced it will acquire Stratify. Between acquisitions and start-ups, it’s hard to track exactly, but I think the number of free-standing review platforms is going down.

For reference, from my The Best Home for an E-Discovery Vendor (another acquisition) post in June 2006, here are some acquisitions that I’ve tracked, separated pre- and post that date:

June 2006 or earlier:
- Xerox acquired Amici Xerox press release
- The Phillipine phone company acquired SPI Tech
- Pitney Bowes acquired Ibis, and Compulit before that
- Integron acquired Bowne’s EED services
- Xiotech acquired Daticon
- Epiq acquired nMatrix
- FTI Consulting acquired Ringtail
- Marsh & McLennan acquired Kroll
- LexisNexis acquired Applied Discovery

Since June 2006:
7/25/06: LexisNexis announced yesterday (7/24/06) that it is acquiring DataFlight, maker of Concordance software. See the press release
8/1/06: Huron Consulting announces acquisition of Aaxis Technologies and of Nextra
9/27/06: Techbooks announces (9/18) acquisition of Whitmont Legal Technologies.
9/28/06: Thomson acquires LiveNote
10/6/06: Océ Business Services acquires CaseData (press release, PDF)
11/20/06: Kroll Ontrack acquires e-discovery software provider Engenium (press release, PDF)
3/1/07: Merrill Corporation Announces Acquisition of Lextranet
5/31/07: Anacomp acquires CaseLogistix
6/4/07: Syngence acuired by private investors
7/16/07: Autonomy and Zantaz merge July 3, 2007
10/31/07: Iron Mountain today announced it will acquire Stratify
12/9/07: Seagate Acquires MetaLINCS E-Discovery Software
1/14/08: Deloitte Acquires Barrasso Consulting
6/10/08: FTI Consulting, Inc. to Acquire Attenex Corporation
7/24/08: Interwoven to Acquire eDiscovery Innovator Discovery Mining, Inc.
8/29/08: Access Litigation acquired by Edmond Scientific Company per Washington Business Journal in Edmond Scientific buys Access Litigation

Every litigation practice may soon need its own e-discovery attorney. 

I recently wrote a white paper for Renew Data called for Renew Data discussing the emerging role of “e-discovery lawyer” (see my prior blog post, Managing E-Discovery (EDD) . You can now read it here at primslegal.com: 4 Ways an eDiscovery Attorney Can Make Your Firm More Successful

The four reasons are
1. Build the Right Team to Handle a Complex Problem
2. Master the Intersection of Law and Technology
3. Develop EDD Best Practices to Minimize Risk and Make Life Easier
4. Advise Clients on E-Discovery and Litigation Readiness

Price umbrellas and inefficiencies create market opportunities. At least so it seems when it comes to reviewing discovery documents in litigation. 

General counsels complain about law firm costs to review documents flowing from e-discovery (EDD). The market is listening. One year ago, I blogged about offshore options for document review. Recently I’ve come across some US companies that review discovery documents. The list here is likely not complete (identifying review companies by web searching is hard):

• Barrasso Consulting
• Black Letter Discovery
• De Novo Legal
• DiscoverReady
• Digital Mandate
• Huron Consulting Group (which acquired Nextra)

There are probably lines to be drawn in the market among consulting companies that focus on managing the discovery process, staffing agencies that also provide facilities (and I understand more and more are doing so), and companies focused primarily on document review.

Wherever the line my lie, it’s good news for clients who don’t want to pay law firm rates and mark-ups for high volume document review. They can investigate a range of on- and off-shore options.

E-discovery (EDD) press coverage suggests problems galore. Have we lost sight of the real purpose of discovery in litigation? 

E-discovery means more than producing relevant, non-privileged documents. It means winning by telling the best story possible. So much of recent EDD discussion revolves around doing it right and avoiding disaster. Reported judicial opinions self-select for problems because absent a problem, no opinion issues. There’s no equivalent judicial mechanism to learn about winning discovery strategies. Nor is there any other apparent mechanism: clients and vendors seldom share details of their victories.

I would like more press and conference coverage on how to use the available tools to achieve outstanding results. Stories along the lines of “I used the text of the ideal incriminating e-mail as my search and that led me to a treasure trove of documents that helped win my case. Or “Mapping documents, e-mail, and calendar entries on a date line proved a perfectly innocent explanation for events that plaintiffs alleged constituted intent to defraud.” EDD defeats are well-documented; it’s time to start telling victory stories.

It’s hard to keep up with all the developments in e-discovery. Some news and articles of recent note: firms let go of EDD expertise, Autonomy and Zantaz merge, and Information Week provides guidance for corporate IT departments on EDD and the amended FRCP. 

The Lost Treasure of EDD by Alan Cohen in Law Firm Inc. (June 2007) is an exploration of why “many big firms missed electronic data discovery opportunities because of a fixation on billable hours.” Cohen notes that while law firms do plenty of EDD work, they are focused only on a single piece of the pie. Their limited view of EDD has led to the formation of at least one law firm specialized on EDD (Redgrave Daley Ragan & Wagner) and the departure of other experienced litigators from BigLaw to consulting companies. Cohen laments the departure of EDD experts from law firms, concluding that they have lost a valuable substantive and competitive asset. This sounds historically true, but contrast this to my conclusion in June about “E-Discovery (EDD) Attorneys:” “I suspect more [firms] will hire “e-discovery attorneys” or start e-discovery practices.”

Merger news: Autonomy to acquire Zantaz. “The combination of Autonomy and ZANTAZ, market leaders in enterprise search and content archival, will redefine Information Risk Management by automating the full spectrum of consolidated archiving, e-discovery, analytics and real-time policy management which is fully integrated with day-to-day operational information systems.” Looks like more evidence of convergence of information management with EDD. [Spotted at the Orange Rag, blog of the Legal Technology Insider.]

E-Discovery: New Federal Rules Require A Proactive Strategy (Information Week, July 2/9, 2007) is a good overview of the issues facing and available solutions for corporate IT departments in dealing with e-discovery and the FRCP.

How should law firms manage e-discovery?  

The Data Boom: Can Law Firms Profit? in LawFirmInc. (May 2007) offers answers to this question, covering economic questions such as whether EDD should be a profit center and how to conduct document reviews.

My favorite excerpt:

“Fenwick and Kirkpatrick & Lockhart Preston Gates Ellis-have adopted polar opposite models. [Fenwick limits] the number of associates needed. Cases in which hundreds of gigabytes of raw data were collected require six to 12 lawyers for review, according to Fenwick’s Brownstone: ‘We apply radical de-duplication and statistical analysis to keep what the reviewers see to the absolute minimum of data.’ Kirkpatrick, on the other hand, staffs no associates on review. Indeed, its document analysis technology group doesn’t even have associates. Instead, the firm draws on a pool of some 200 in-house project attorneys who do nothing but review documents.”

Care to guess which approach is more cost effective? Last time I looked, technology, even expensive tech, costs a lot less than lawyers, even contract lawyers or staff attorneys.

Wherever firms come out on these questions, I suspect more will hire “e-discovery attorneys” or start e-discovery practices. I recently wrote a white paper for Renew Data called 4 Ways an eDiscovery Attorney Can Make Your Firm More Successful (registration required to download). I interviewed several lawyers in this position. The article quotes Thomas Barnett, Special Counsel - Sullivan & Cromwell LLP, Laurie Weiss, Partner - Fulbright & Jaworski LLP, Charles Cohen, Partner - Hughes Hubbard & Reed LLP, and Mira Edelman, Counsel - Hughes Hubbard & Reed LLP.

I have long argued that deploying armies of domestic contract lawyers to review documents in e-discovery is not a sustainable practice. 

Search Software Gets Boost From New Rules in the Wall Street Journal (5/16/07) explains that the December 2006 amendments to the Federal Rules of Civil Procedure are driving new corporate spending on e-discovery software and services.

The article quotes a Gartner Group analyst: “Electronic-discovery software eliminates the need to have lawyers conduct extensive data reviews.” Really? Can we send home the thousands of contract lawyers reviewing documents at this very moment? I think not. Perhaps this quote was taken out of context, but even with foreseeable software advances, lawyers will still spend much time reviewing documents. (I’ll avoid semantics over what “extensive” means.)

In my opinion, the WSJ is the best newspaper in the US. So it disturbs me to read an article about a market I know well and see what I consider significant inaccuracies. We have to evaluate information on web sites and blogs carefully; it turns out the same is true for mainstream media (MSM).

A new National Law Journal (4/19/07) article profiles the Cisco law department and general counsel Marc Chandler. The article reiterates many interesting legal technology applications at Cisco. I was most struck by Chandler’s comment on document review in e-discovery. 

Using Technology to Cut Legal Costs quotes Chandler:

“The discovery process in litigation can be very, very expensive. We have built electronic tools that mine all the documents that exist in the company, collect them and eliminate the stuff that is not responsive. Most outside counsel embrace that. Some still prefer to do everything by hand, using armies of associates. We have not seen quality differences between those two models, but there is a giant cost difference.”

I’ve written previously that the choosing the best document review approach - US contract lawyers, offshore lawyers, or software - is an empirical question. Chandler adds anecdotal evidence that computers do better than humans, at least considering cost.

The rest of the article is well worth reading to learn more about other Cisco legal department innovations.

If I ask the titular question, you’d rightly think me odd. But if I ask “how’s e-discovery going?", what’s the problem. Do names mean anything? 

For centuries, the horse and carriage was a key way to get around. It took years before the horseless carriage moniker faded in favor of cars (or autos, SUVs, etc.).

The name “e-discovery” suggests something special, that there is some other kind of discovery. Sure, paper may still play a supporting role, but with the newly amended FRCP, E has the lead role.

At some point - maybe now - we need to lose the E. Dropping E might help de-mystify the new electronic world and signal that the fantasy of non-digital discovery must die. (Even if not all discovery is quite yet E, lawyers better know about it. See Conrad Jacoby’s Separating E-Discovery Myths from Realities at LLRX.com for more on this point.)

Is e-discovery the tail wagging the dog? 

Consider the stages of EDD, each costly:
1. identify sources and custodians,
2. preserve data,
3. collect data appropriately,
4. convert data into usable formats,
5. review data for responsiveness and privilege,
6. produce data to opposing parties, and
7. manage the overall effort.

Previous posts have discussed reducing review costs with offshore lawyers or more reliance on software. A better way is re-think the steps leading to review (identifying, preserving, etc.) Doing so is a double win: direct cost savings for those steps plus indirect savings by ending up with fewer documents requiring review.

Enter corporate records management (RM). Companies seek RM solutions that simplify tracking digital data and help prepare for litigation. I’ve suggested such convergence (E-Discovery Convergence at Hand? and Compliance and E-Discovery: Long Term Convergence?). Now, The Forrester Wave™:Records Management report (Q1 2007), available courtesy of CA here, observes that litigation is the “hot driver” of records management (RM). Forrester writes that today

“RM vendors do not have the capabilities necessary to truly solve the eDiscovery challenge. In an effort to claim a portion of the lucrative eDiscovery market, look for RM vendors to first partner with — and then buy — Discovery pure-play vendors such as Attenex, CaseCentral, MetaLINCS, Stratify, and ZANTAZ.”

If the leading RM vendors (the top three the report identifies are CA MDY, Open Text, and IBM FileNet) succeed in expanding their reach, then e-discovery becomes cheaper and easier. For example, one person operating a console can replace armies of data collection technicians; destruction and retention occurs in real time, reducing data volumes; integrated tools enable selecting relevant documents during collection rather than afterwards; and loading a review repository occurs with a few mouse clicks. Separately, automatic policies to enforce litigation holds reduce the risks of spoliation and sanctions.

In this world, RM will become the dog and EDD merely the tail.

My thanks to Peter Pepiton II, Esq., Discovery Product Manager, CA, Inc. for helping educate me on the corporate perspective and his comments on a draft of this post.

Last week I attended a great e-discovery forum: “And Justice for All: How the Electronic Information Explosion is Transforming the American Legal System.” H5 Technologies assembled an outstanding panel (listed below), moderated by Prof. Arthur Miller. 

The 2-hour long panel discussion explored the impact of the explosion in electronic data on litigation, government record keeping, and privacy. I will focus on a few highlights from the litigation portion.

A big theme was the ability of lawyers to find relevant documents. One panelist thought that business should organize information anyway, so it shouldn’t be that hard to find documents if they just did this. Another replied “no,” if organizing data had value, then business would do so (a view I share).

Professor Miller suggested that perhaps lawyers have always been bad at finding documents and now still are, so what? A classic Socratic question but I was surprised to hear one panelist agree, saying what documents are found matters not since only a tiny percent are used at trial: we only need the important items, collections have much duplication, and litigators have always had nagging doubts about missing something critical. I can’t square that view with mounting the best case possible and fulfilling production requirements.

That challenge of finding documents led Justice Breyer to express concern about the cost of EDD. If reviewing documents is so expensive, then we limit access to justice to those who can afford EDD, namely just big companies. An audience member shared this concern, saying that his small business clients can no longer litigate ordinary employment matters because of EDD costs. (See also Rising Costs of E-Discovery Requirements Impacting Litigants, Fulton County Daily Reporter, 3/20/07.) A possible solution is more cooperation in the meet and confer process. Others took issue with this suggesting it is utopian; one panelist noted that the “gotcha” is no longer about the case, it’s about the discovery.

Expecting a panel to resolve these issues is not realistic. I think it’s a good sign that a diverse group of prominent lawyers came together to discuss everything from the nitty-gritty of search technology to the loftier issues of equal access. Many such discussions will be required to reach a new and better equilibrium.

Panelists:
Professor Arthur R. Miller (Moderator), Harvard Law School
Jason R. Baron, Director of Litigation, National Archives and Records Administration
Ron Brachman, VP of Worldwide Research Operations, Yahoo! Research
Richard Brachman, Executive Director, The Sedona Conference
Hon. Stephen G. Breyer, Associate Justice, United States Supreme Court
Julia Brickell, Associate General Counsel, Altria Corporate Services
Nicolas Economou, CEO, H5
Hon. John Facciola, Magistrate Judge, U.S. District Court for the District of Columbia
Anne Kershaw, Founder, A. Kershaw PC // Attorneys and Consultants
Patrick Oot, Director of Electronic Discovery, Verizon Communications
Marc Rotenberg, Executive Director, Electronic Privacy Information Center
Hugo Teufel III, Chief Privacy Officer, U.S. Department of Homeland Security
David C. Vladeck, Associate Professor of Law, Georgetown University Law Center

Two recent e-discovery webinars suggest that the legal profession has yet to settle on the best approach to e-discovery document review.  

E-Discovery: Search Engines & The Lawyer’s Role in Review (law.com, 2/28/07) addressed using key words versus concept searching to speed document review. Implicit in the discussion was that it’s not a matter of if but when most litigants will rely on software assisted review. I share that assumption.

The panelists had a spirited discussion about simple Boolean search versus concept searching and what directions the courts would take. Some panelists took what I consider the only sustainable position: determining the best approach is an empirical question. They also pointed out that we will need a series of published opinions to settle the question.

Information Risk Management and E-Discovery: Alternative Approaches to Document Review (H5 Technologies, 3/1/07) included a presentation describing just such an empirical test: about 50,000 documents reviewed by both humans and an advanced concept search tool. The software performed better than the reviewers. Someone in the audience asked about the risk of the software missing a document. The panelist rightly answered that the question was backwards given that her data showed the software did better than the humans.

Here’s my take after listening to both. Many lawyers appears honestly to believe that human review is accurate, the “gold standard” for document review. “Honestly held” and “right” can diverge. I, for one, have never seen data to support the commonly accepted “gold standard.” I suspect that the proponents of it are the same ones who postponed dealing with e-data as long as they could.

Absent a well-controlled study with widely accepted findings, we seem doomed to years of costly litigation and a trickle of published decisions to establish a new standard. If I seem jaded, well, I am. Boolean search has been available as early as 1980. A seminal scholarly article (Blair and Moran, 1985) showed that Boolean search alone found less than 20% of the relevant documents. In 1989, I used a concept search engine (PLS) that generally found more documents than Boolean searching alone. In 1992, there was good anecdotal evidence that machines did better than humans in objective coding.

At least there is a debate today and at least some EDD thinkers are at least talking about an empirical approach.

The legal press buzzes with news about a discrimination law suit between an associate and his BigLaw employer. A development today - destruction of a hard disk drive - offers lessons on both e-discovery and records management. 

Destroyed Hard Drive Becomes Focus of Hearing in Sullivan & Cromwell Suit (NLJ, 2/9/06) reports ex-associate Aaron B. Charney, who is suing Sullivan & Cromwell for sexual orientation discrimination, “recently destroyed a personal hard drive to which he had e-mailed firm documents.” Those documents are a factor both in Mr. Charney’s suit and in S&C’s counter suit for “stealing documents and publicly disclosing confidential information.”

Lesson one: if litigation has commenced or is reasonably anticipated, destroying relevant evidence may be spoliation. The reported facts don’t sound favorable for Mr. Charney.

Lesson two is less obvious. To work outside the office, many lawyers e-mail documents to themselves or copy documents to portable storage media. A records management policy should address this fact. An RM policy here would not change the legal issues. The facts, however, remind CIOs and records managers to deal with document copies outside the walls and firewalls of the office. Absent a digital rights management system or draconian limitations, controlling such copying is virtually impossible. A well-crafted RM policy can at least put lawyers on notice concerning how the firm expects them to manage such copies.

Several large law firms and EDD vendors have announced new or enhanced e-discovery consulting services. So law firms and EDD vendors may compete for e-discovery consulting business. Who should clients choose and what limitations do each face? 

Clients should consider carefully who offers the better set of skills and experiences. BigLaw brands may offer comfort, but some vendors have hired experienced lawyers and may offer the better bundle of skills and experiences.

EDD vendors risk competing against large law firm customers. Were I on the BigLaw buy side, I would not give my EDD business to vendors competing for the same consulting dollars.

BigLaw may suffer from the self-imposed caste system. They typically seeem not to list non-lawyer tech and other experts on EDD practice pages on public web sites. This lacuna could hurt marketing: as a client, I’d want to know about who is on my team. Furthermore, the caste system takes more than just a psychic toll; it can reduce effectiveness by hampering teamwork. Inhouse counsel often buy into the caste mentality but that is balanced by greater awareness of the importance of team work.

I can’t predict who will win the battle for EDD consulting dollars, but it will be interesting to watch the emerging competition.

(Disclosure: no sour bones from me and my NYU JD and two state bar (inactive) memberships.)

Many articles lately cover the new e-discovery rules but don’t answer some nitty-gritty issues. 

For example, enterprise databases (e.g., SAP or Oracle) are decades old yet preserving or harvesting data from them can still be a struggle. Separately, I previously wrote about the potential EDD challenges of dealing with software as a service.

Now consider the rise of Web 2.0 tools that enable intra- and inter-enterprise collaboration via the web. Culture of Sharing Is Possible in eWeek (12/18/06) concludes that that “in 2007, more applications will allow simultaneous editing of content with good mechanisms for apprising participants of changes to that content.”

So, what happens when work moves from traditional applications (e.g., MS Word) to web-based systems (e.g., Google DOCs)? How often are data on third-party servers backed up? How long are those back-ups kept? These are the easy questions.

Preserving and harvesting issues grow potentially more complex with truly dynamic systems where multiple users simultaneously edit text or data. How often do such systems take snapshots? How many individual users take snapshots and then how do these compare to what others may save or what’s saved centrally? What happens when the system sends e-mail to alert users of updates, especially if the message contains content that is subsequently altered? Who “owns” or “controls” the data on these systems, especially if two separate companies have agreed to use the same third party tool? What if you need to discover data from a dynamic system such as Second Life?

Whoever addresses these questions on first impression may have a tough job. Likewise the corporate managers who have to establish and enforce record keeping policies concerning these systems. Legal Tech NYC next week is a good place to seek answers.

All litigators should read the front page of the New York Times today. There’s a public pronouncement of an e-discovery challenge. 

Firms Fret as Office E-Mail Jumps Security Walls discusses the challenges of managing corporate e-mail when so many employees use personal accounts on services such as Google, AOL, Yahoo, and Hotmail. Beyond concerns about trade secrets leaking

“companies could run afoul of federal laws that require them to archive corporate mail and turn it over during litigation… Lawyers in particular wring their hands over employees using outside e-mail services. They encourage companies to keep messages for as long as necessary and then erase them to keep them out of the reach of legal foes. Companies have no control over the life span of e-mail messages in employees’ Web accounts.”

This issue should not be news to litigators; it’s certainly not to electronic discovery or legal technology consultants. In the past, lawyers seem successfully to have pleaded ignorance on issues like this. With the issue landing on the front page of the Times, the “dog ate my homework” types of excuses will no longer work.

The new federal e-discovery rules take effect on December 1st. How will technology changes affect the application of these rules? 

A recent software demo caused me to ponder this question. The “reasonably accessible” standard of amended Rule 26(b)(2)(B) guides allocating the burden and cost of discovery. New technology may alter that burden and cost. For example, magnetic tapes, which contain enormous quantities of corporate data, are often considered relatively inaccessible. But what if the restore cost drops dramatically?

I saw a demo of Index Engines, which provides an appliance that reads multiple back-up tape formats and indexes them at “wire line” speed. The result is a full-text searchable, de-duplicated database that is 5% to 8% the size of the original materials. Lawyers can do searches to identify which specific tapes should be restored.

I am neither a backup tape nor FRCP expert. But it seems that the easily restoring tapes across formats, without requiring original systems, and compressing them significantly may well change what courts consider “reasonably accessible.” This product seems to lower the cost of figuring out what’s on back-up tapes.

I think the broader implication is that lawyers must keep current on technology options. Inaccessible media today may well be all too accessible tomorrow. Reading tech trade journals may be as important as reading legal publications.

We may see a “reasonability ratchet.” Acceptance by just a few courts of a new approach may spawn a standard other courts find hard to ignore. In a loose opposite of Gresham’s Law, good technology will drive out bad. Whether this is good probably depends on your perspective. The technology cost may go down but the lawyer review time and costs up.

[Note: Index Engines may well have direct competitors – I did not research this. I have no relationship with this company. When vendors offer possibly interesting demos, I sometimes accept and then sometimes blog about it. My analysis may be wrong but I don’t have a vendor- or technology-driven bias.]

Kroll Ontrack today acquired concept-search engine maker Engenium. 

From the press release (11/20/06): “Kroll Ontrack…. today announced the acquisition of Engenium… a leader and pioneer in intelligent search and search-related products. This acquisition signals Kroll Ontrack’s continued commitment to advancing technology that addresses the growing demand for cost-effective methods to locate relevant information from large volumes of data in support of investigations, litigation and regulatory compliance matters. Terms of the transaction were not disclosed.”

The EDD consolidation we are seeing means more competition. With competition comes price pressure and the need to differentiate offerings. So it does not surprise me that a top EDD vendor would buy a software product that allows it to differentiate. It will be interesting to see how current Engenium EDD customers react (and note that one listed customer is Fios, another large player).

I’m also glad to see a leading vendor emphasize the need to make document review more cost effective.

Last week, Oracle acquired Stellent. BigLaw CIOs should keep their eye on this longer term. 

Stellent is a content management company, though in legal, it may be better known for its Outside In viewing technology, which is integrated into several e-discovery platforms. The merger (press release here) could have two distinct impacts. Oracle might go after the document management market, competing with OpenText (formerly Hummingbird) and Interwoven (formerly iManage). But DM is small potatoes relative to compliance.

EDD is exploding in more ways than one. Continued EDD vendor growth does not seem sustainable, at least from the corporate side, which pays the bills. The eventual solution is in the maturing of tools for corporations to manage their documents and content. With better enterprise content management (including retention, destruction, categorization, and search), many of today’s e-discovery challenges go away.

As noted EDD market observer Michael Clark of EDDix observed in an e-mail exchange:

“Not to minimize the complexity but, at the enterprise level, litigation support (including electronic discovery) is essentially a workflow problem that cuts across records policy, storage topology and content (document) management tools … as is compliance and reporting. Today, large organizations in highly litigious and highly regulated categories are beginning to look for holistic workflow solutions that can address both the prescriptive needs of compliance as well as the reactive needs of litigation.

I fully expect to see enterprise players like Oracle developing and marketing converged solutions … which will have a non-trivial impact on the EDD marketplace over the next several years. I think the large enterprise players will be strategic buyers of EDD vendors (to gain the legal domain knowledge, specific e-discovery tools and workflow patterns) and I see them as ultimately dominating the large organization marketplace. Although I don’t think that EDD was a major driver of the Oracle-Stellent deal, by virtue of the document viewing tools Oracle has acquired, Oracle is well-positioned to enter the market for converged solutions”

EDD companies currently sell at high multiples. With the dollars chasing deals and the entry of new players, competition may take a toll on pricing. If so, then I suspect that valuations will drop. That would would make it cheaper and easier for the Oracles of the world to acquire the legal domain know-how. And that could have a major impact on EDD.

Should law firms have records management policies? Risk management partners say yes but many firms struggle to define or implement a policy. Not having one can have consequences. 

Church Group Suing Weil Gotshal Claims Firm Withheld E-Mails (NYLJ, 10/23/06) reports that a Weil Gotshal client “is accusing the firm of withholding from discovery e-mail correspondence.” The allegation focuses on a big discrepancy in the number of e-mail messages produced by different lawyers at Weil Gotshal. The firm’s general counsel says the discrepancy is due to one partner who did not save e-mail messages while the others did.

Whether this will cause a problem for Weil Gotshal remains to be seen. But the fact that a client raises this issue and the press reports it illustrates the need for a records management policy and enforcing it. Aside from the potential to create the appearance of inconsistency, haphazard retention may make it impossible for a firm to produce a complete record that would help in its defense. A partial record can be worse than no record or a full record.

The current issue of Law Technology News has an excellent roundtable discussion on what lawyers must do to ensure that EED processing works. Not addressed however, is an equally interesting question: the duty of care for lawyers reviewing documents. 

EDD Showcase: Worst Case Scenario (Oct 2006, free registration required) features a discussion among EED experts Michael Arkfeld, Craig Ball, and J. William Speros answering “Who is responsible when you delegate EDD and things go awry?” Arkfeld holds lawyers to the highest standards of due diligence concerning vendors. He argues that

“the attorney must be reasonably confident that the processing and searching methodology is sound and that the evidence was disclosed… Among the many things a vendor should do is to provide affidavits, testing data, etc., detailing how a particular piece of software processes electronically stored information (ESI)… This would also apply to search software: lawyers must use reasonable care to ensure accuracy… you need to include quality control standards that assure disclosure of all responsive evidence to the opposing side.”

Were Arkfeld’s approach applied to how lawyers designate privilege and responsiveness, I suspect that many lawyer review processes would fail. Let’s start by looking at a simpler task: objective coding of data such as author and date. Vendors do not promise 100% accuracy. Historically, accuracy much above 98% required double keying.

So how accurate is the lawyer review process? In my view, ensuring accuracy requires statistical techniques. Subsets of documents must be compared to the “correct answers.” The correct answer probably requires a team of experienced lawyers to agree on the designation for the selected documents. A statistically significant sample for each reviewer should be analyzed. The problem is, the statistics only tell you how bad the problem is - they don’t help fix it.

As law firms wake up to the potential problems vendors can cause, they should not shirk from the problems inherent in typical document reviews. The implications of this article for vendor management are startling enough. Perhaps a future roundtable will discuss “Who is responsible when lawyer review goes awry?” And I hope the answer does not end up holding vendors to a higher standard than lawyers.

I recently wrote that choosing onshore v. offshore review is an empirical question. Here’s some good reading to help think about answering this. 

In Onshore v Offshore Pivot Point - Part II I said in passing that software might be an even better choice for document review than lawyers, onshore or off. “Of Litigators and Butterflies: The Quest for a Quantum Leap in Large-Scale Document Review” in Pike & Fisher’s Digital Discovery and e-Evidence (July 2006) and available at H5 Technologies’ web site as a PDF is a good analysis of the inherent error in human document review.

Author Nicolas Economou, H5’s CEO, sets forth a mathematical equation for assessing document review error. Though the equation is conceptual for now, he also presents provocative empirical data. A study found that the H5 automated approach correctly identified relevant documents 98% of the time.

Of course, many lawyers will immediately react that this is simply unacceptable - how can we live with a 2% error rate. Those lawyers, however, most likely lack empirical data about the accuracy of their army of lawyers.

The article also reports that “for each relevant document missed by the H5 system, the control review process missed 32 documents. That is, the risk of failing to flag relevant documents for litigator review was 32 times greater under the traditional review process.”

If this is true and can be further substantiated, at what point will lawyers commit malpractice by continuing to rely primarily on human review?

In my prior post, I summarized two recent articles about discovery document review: DuPont’s going offshore and BigLaw’s move to contract lawyers. Which is better - onshore or offshore review? 

The choice is purely an empirical question, meaning a decision driven by hard data: the cost to review each document, adjusted for accuracy. You can argue over how best to measure cost and accuracy, but it’s hard to see how else to decide. (I assume that ethical issues, if any, of going offshore can be resolved. I also leave for another day whether software is not an even better approach than lawyers, onshore or off.)

So the rational general counsel should ask firms for statistically reliable cost and accuracy measures. Firms not already tracking these are probably not effectively managing the review process.

Lawyers often don’t consider alternatives and, when they do, seem to debate endlessly. But why argue when you can measure? Sure, going offshore may require more supervision; maybe even travel costs to send US lawyers overseas to supervise. But finding contract lawyers and paying agencies is not cheap. I could go on about many costs. The point is, stop debating and start measuring.

If inside counsel lack the data or analytic horsepower, then retain legal technology or other consultants or borrow financial analysts from other corporate departments. With the millions of pages to review and enormous costs, production- and factory-like discipline is essential. What are we waiting for?

The current costly approach to e-discovery document review is not sustainable. I’ve often suggested that offshore lawyers are a cost reducing option. At last, we have a public instance of offshore review - and a big one. At the same time, we have new insight into the use of contract lawyers in BigLaw. So which will win - the offshore option or ongoing armies of costly contract lawyers? 

Let’s Offshore The Lawyers in Business Week (9/18/06) reports that DuPont is offshoring document review. It reports that on “the outskirts of Manila, 30 Filipino attorneys, including three who have passed U.S. bar exams, are seated elbow-to-elbow with 50 other staff at long tables crammed with PCs. Working in three shifts seven days a week, they read, analyze, and annotate digital images” of documents. DuPont hopes to save 40% to 60% by going offshore and “figures 70% of the labor in a typical insurance or liability case can be outsourced.” Moreover, “DuPont hopes it can slash the discovery process in insurance cases to three months from an average of 18 months.”

So finally, a public example of offshore document review. The article acknowledges that this is a test of sorts for DuPont and OfficeTiger, which provides the service. But now the way is paved for others to try, absent a problem.

At the same time, the cover story of the current issue of American Lawyer (Sep 2006) describes in detail how large US law firms increasingly rely on contract lawyers for document review. Temporary Solution reports that the sheer volume of electronic evidence requires armies of lawyers. Some points that stood out for me:
- The working condition of many contract lawyers is not good.
- Firms are turning a tidy profit on contract lawyers.
- BigLaw management is, at least in some instances, troubled by reliance on so many temps.
- Not all contract lawyers have passed the bar.
- Some firms have created (well, maybe re-established is a better word) a new lower class of lawyers for document review.

So, by chance, in the same week, two articles in two prominent publications describe two paths: onshore and offshore. In my next post, I propose how to decide which approach is better.

In June I wrote about the emerging position of EED attorneys in law firms. Corporations are also creating such positions. 

Discovery Trailblazers in InsideCounsel (Aug 2006) reports that many large companies are looking for someone to manage e-discovery. One motivation is cost control: “An e-discovery manager works directly with vendors, ensuring that the company gets the best possible rates.” Though vendor pricing is important, I suspect that even bigger savings lie in actively managing how outside counsel conducts the entire discovery process. After all, large law firms are not reknown for their project management skills.

“It’s also important to have someone bridge the communication barriers between legal and IT. Without a translator in place, e-discovery can go painfully wrong.” Clearly something went wrong for Morgan Stanley in the Coleman case (the $1.4b judgment is on appeal); having someone inhouse to shepard document collection and establish a records management policy is a good idea.

The article reports compensation for this emerging position ranges from $100,000 to $300,000. If I were a GC, I would not want someone at $100k (which buys about two-thirds of a first year associate in NYC); I think $175k is a more realistic minimum, especially given the demand for the right qualifications.

IBM announced today that it will acquire Enterprise Content Management vendor FileNet. Over time, this could change the EED landscape. 

Enterprise content management, which increasingly includes records management, is becoming the arena of major corporations. (For example, see my blog post about CA acquiring MDY.) As big companies gain control over the life cycle and storage locations of their information, the e-discovery landscape is sure to change. Exactly how, however, remains to be seen but I suspect pure e-discovery providers, especially those focused on collecting and processing data, will eventually have to develop new offerings to thrive and grow.

The American Lawyer today reports that a Software Glitch May Have Erased E-Mail Text in Enron Suits. 

Quoting from the article:

“The company [Applied Discovery] handling electronic document production in the Enron civil suits says a software bug may have erased text in e-mails produced for discovery in the case over an 18-month span…. lawyers handling the Enron litigation said it was too early to predict the potential impact. … The e-mail bug delivers messages with only the subject lines and sender information intact, but does not capture the body text of the e-mail. … Nagel [of Applied Discovery] described the problem as “a Microsoft issue.” He said the problem is not e-discovery specific, meaning that anyone who uses an unpatched version of Microsoft Outlook 2003 to open a type of file called a .PST from Microsoft Outlook 2000 might find that the e-mail opens blank…. Craig Ball, a computer forensics and electronic discovery consultant… said he is surprised to learn that a market-leading company like Applied would use Microsoft Outlook to search e-mail. He said Outlook is good about opening files, but inaccurate when it comes to searching file content and is limited in its ability to do Boolean and other searches.”

The entire article is worth reading. Time will tell if this problem could easily have been avoided or reflects inadequate care. Meanwhile, it’s a good lesson that litigators and lit supp managers should make sure the tools they use work appropriately. For those who might think “time to return to reliance on humans,” remember that people on balance make more mistakes than machines.

This development may usher in a new age where not understanding how critical tools work is unacceptable. If so, the question still remains how much you need to know and investigate and what the appropriate standard is or will be.

Thanks to Monica Bay over at the Common Scold for pointing this out and sharing the news.

Update: See the August 16, 2006 Common Scold blog post reporting on law firm Bartlitt Beck’s letter concerning the above.

Update: See the August 11, 2006 LexisNexis response here. I came across this via a Google search; I did not find a link to this on the L-N site. See also, however, a L-N letter and a response by Craig Ball, both at the Common Scold blog.

The e-discovery market continues to grow explosively and evolve. Who are the leaders in service, software, and by stage of e-discovery? 

The fourth annual Socha-Gelbman E-Discovery Survey answers these and other important e-discovery market questions. A summary of the results is at Socha Consulting. Here are some highlights:
- The EED market continues to grow at 35+%
- The top 5 e-discovery providers are Fios, Kroll Ontrack, LexisNexis Applied Discovery, Renew, and Zantz (top 20 listed)
- Among the top 20, there is (as covered in two separate tables) significant variation in
     . experience, capacity, and law firm ranking
     . capability by stage of e-discovery processing
- The top 10 e-discovery providers include both some “new generation” players such as Attenex and some long-standing players such as Summation and Concordance.

For law firms evaluating service providers and software, this is a useful resource (purchase information here).

For me, the big surprise is that there are still so many players in a market that is not that large ($1.3 billion in 2005). I suspect that there is still room for significant additional consolidation (see my list of recent consolidations).

Document review in e-discovery is costly. Contract lawyers are less expensive than associates. Going offshore can lower costs yet more. 

I await published case studies detailing offshore document reviews. In the mean time, several offshore companies or US companies with offshore operations now offer discovery document review services. Here is what vendors say on their websites:

Aphelion (formerly BPL Services): “Aphelion’s management team has been involved with substantial document review projects in litigation matters handled by the firms DLA Piper Rudnick Gray Cary and Womble Carlyle.”
ITELS: ” ITELS document review team includes top-tier US and Indian lawyers experienced as litigators and corporate attorneys for international and domestic corporations…. Review of documents in complex matters and corporate due diligence… Review for relevancy, privilege, and issue coding.”
Lumen Legal: “Lumen Legal dramatically lowers the cost of document review typically associated with major litigation…. as a pivotal provider delivering the right mix of on-shore and offshore talent, we can provide 24/7 workflow at cost savings of 50% or more over traditional models.”
Pangea3: “Analyze documents for materiality, privilege, and responsiveness to discovery requests, subpoenas, informal and formal regulatory requests… Conduct large-scale electronic file reviews.”
QuisLex“Our highly trained and specialized litigation support team works closely with your legal team and performs the following services… Subjective coding (creating substantively coded indexes etc)… Relevance and privilege analysis”. (See also SQ Global Solutions.)
Tusker Group: “Tusker Group has the ability to align domestic and international attorneys to provide high-quality, cost-effective first level document review: relevancy, privilege, confidentiality, issue content and responsiveness.”

I have included in the above list only those vendors whose web site clearly indicate that they perform document review in litigation. Other vendors may also do so (see the list of outsourced legal services).

LexisNexis announced yesterday (7/24/06) that it has acquired DataFlight, maker of Concordance software and CaseSoft, maker of CaseMap. 

From a L-N press release: “The addition of Los Angeles-based Dataflight (www.dataflight.com) enables LexisNexis to offer attorneys evidence management capabilities via the Concordance solution. Law firms, corporations, and government agencies use Concordance as the repository of choice to organize, identify and produce critical documents in litigation.” The company will unify “data repository and workflow tool for Total Litigator, linking its own capabilities with those of LexisNexis’ other litigation products such as LexisNexis® Applied Discovery®, LexisNexis® File & Serve and LexisNexis® Court Link®”. The press release also has details on the acquisition of CaseSoft.

Another leading lit supp product, Summation, was acquired by CCH a while back. With the continuing consolidation of e-discovery generally and the moves of big legal information providers CCH and L-N to bulk up in EED and litigation support, it will be interesting to see if West or Thomson Elite makes a move.

What’s the best home for an e-discovery company? Yet another acquisition last week raises this question. 

“To expand its document management services business, Xerox… announced that it has agreed to acquire privately-held Amici LLC for $174 million. Amici is a leading provider of electronic-discovery (e-discovery) services, primarily supporting litigation and regulatory compliance.” The Xerox press release goes on to report that ”Amici was founded in 2002 and currently employs about 125 people.”

As pure-play EED vendors are scooped up, we need to think about how EED fits with the new parent’s existing business. Recent acquisitions include:
- The Phillipine phone company acquired SPI Tech
- Pitney Bowes acquired Ibis, and Compulit before that
- Integron acquired Bowne’s EED services
- Xiotech acquired Daticon
- Epiq acquired nMatrix
- FTI Consulting acquired Ringtail
- Marsh & McLennan acquired Kroll
- LexisNexis acquired Applied Discovery

It’s not obvious what the best long term fit for e-discovery players is. Is it a “pure play” legal company like LexisNexis? A large company like Pitney Bowes or Xerox with selected legal market services? Is it a more specialized legal provider like Epiq or FTI? How about an insurer and financial services company? And where is HP, EMC, IBM, and CA (which just acquired records management software and services company MDY)? It seems that e-discovery would fit well with storage management and information life cycle companies.

At this point, it’s too early to answer. I don’t think BigLaw CIOs and lit support managers need to worry. There are plenty of vendors from which to select. Acquisitions often take a long time to affect operations and service offerings. Longer term though, those buying EED services will likely need to consider ownership and its implications as one of the selection criteria.

Updates
7/25/06: LexisNexis announced yesterday (7/24/06) that it is acquiring DataFlight, maker of Concordance software. See the press release
8/1/06: Huron Consulting announces acquisition of Aaxis Technologies and of Nextra
9/27/06: Techbooks announces (9/18) acquisition of Whitmont Legal Technologies.
9/28/06: Thomson acquires LiveNote
10/6/06: Océ Business Services acquires CaseData (press release, PDF)
11/20/06: Kroll Ontrack acquires e-discovery software provider Engenium (press release, PDF)
3/1/07: Merrill Corporation Announces Acquisition of Lextranet
5/31/07: Anacomp acquires CaseLogistix
6/4/07: Syngence acuired by private investors
7/16/07: Autonomy and Zantaz merge July 3, 2007

Does every big law firm need it’s own e-discovery attorney? That’s the question posed and thoughtfully answered by a good new article. 

E–Discovery Attorneys: Hot or Not? in LawFirm, Inc. (June 2006) by Seth Davis, a recruiter with Major, Lindsey & Africa, examines the emerging trend of law firms and law departments “seeking e-discovery managers, e-discovery consultants, e-discovery engineers, e-discovery gurus, e-discovery and document-retention specialists and, finally, attorneys who understand e-discovery. ”

According to Davis, the challenges, uncertainties, and frustrations of electronic discovery have led many legal organizations to want to hire an attorney who specializes in e-discovery. “The Am Law 100 has been bringing back former technology and litigation associates to run their internal e-discovery departments.” But Davis points out the supply of qualified lawyers is limited - probably no more than 200.

Firms looking for a quick fix to their discovery challenges should, however, be cautious. They need to “recognize that the technology your firm uses will be no different than what the vendors are using. As a result, a new e-discovery department may find itself working through problems that many vendors dealt with years before. My advice is for firm leaders to be patient with a new e-discovery attorney.”

As a consultant, I advised some of my clients to consider this type of role a couple of years ago already, so I agree with Davis’ assessment.

Another sizable e-discovery vendor announced it is being acquired.  

SPI Tech, a litigation support and e-discovery provider, announced the pending acquisition by ePLDT, Inc., a wholly-owned subsidiary of the Philippine Long Distance Telephone Company (“PLDT”). A public disclosure document indicates the agreement was reached on May 25, 2006.

It’s hard to say what it means when a telephone company acquires an e-discovery vendor. The transaction looks like it’s about business process outsourcing generally. So perhaps the e-discovery business fits under the new corporate umbrella.

Pricing may be collapsing. No, not large law firm billing rates; I mean e-discovery processing. 

Lately, my legal technology consulting friends have reported that the price umbrella was collapsing. Now comes “official” confirmation. Publicly traded Marsh & McLennan Companies, which owns Kroll, reports in a May 3rd press release on Q1 results: “Significant pricing pressures in the marketplace adversely affected Kroll’s electronic discovery business, reducing both its revenues and profitability.”

Ok, the title is not true. But it may as well be given the mind set of some bar regulators. 

What’s a Little Metadata Mining Between Colleagues? on law.com discusses lawyers’ use of metadata and that some state bars are considering making it unethical to mine metadata.

In my view, banning lawyers from viewing metadata in documents received is like banning reading a sticky note that a lawyer forgot to remove from hardcopy. The rules should work to discourage bad practices (e.g., not deleting metadata when necessary) instead of protecting those who can’t be bothered to understand the technology they use.

I have advocated formal risk analysis and decision trees to assess and manage litigation risk. Learning decision trees is, however, not that easy. A new automated approach overcomes this barrier for some cases. 

Predix, from case management software provider Bridgeway, “is a sophisticated decision analysis system that lawyers can use to evaluate their inventory of commodity litigation – small to medium size cases of a repetitive nature” according to the Predix press release. An article in the March issue of Metropolitan Corporate Counsel explains

“Predix software is simple in terms of the information needed to be loaded into the tool. The user plugs in the claims, any hot button issues, the track record of the other side against the company in similar cases, and the defenses that the company is thinking of raising… This gives users the opportunity to project probabilities, which can be modified, tweaked and aggregated based on any of a variety of scenarios…. The tool helps speed decision making because it provides a visual depiction of a decision tree without the user having to know anything about the tree’s underlying sophisticated structure.”

For background on decision trees, see an article I co-wrote with David Post, which also explains an earlier approach to automating decision trees that Wilmer Cutler co-developed with Litigation Risk Analysis, Inc.

To learn more about using decision trees, you can attend seminars by Litigation Risk Management Institute, TreeAge software, or Litigation Risk Analysis, Inc.

Consolidation in the e-discovery market continues. Pitney Bowes announced today the acquisition of e-discovery company Ibis Consulting. 

According to the press release,

“Pitney Bowes Inc. (NYSE: PBI) today announced that its Management Services subsidiary has signed a definitive agreement to acquire 100% of the stock of Ibis Consulting, Inc. for approximately $67 million. Ibis is a leading provider of electronic discovery (eDiscovery) services to law firms and corporate clients.”

Pitney Bowes has long been in the business of operating mail rooms and other back-office services for law firms. Last year, the company acquired litigation support company CompuLit. With the acquisition of an e-discovery vendor, it appears that Pitney is making a major push to become an integrated provider to law firms. In the press release, the Chairman is quoted: “One of our strategies for growing customer and shareholder value is to leverage our mailstream management expertise in transaction-intensive verticals like the legal market.”

One challenge for any integrated service provider will be the law firm purchasing process:
- Getting to “yes” at a law firm takes time and effort.
- The purchasers of back-office services typically have little say in purchasing litigation support and e-discovery services.
- The buy decision for lit supp and e-discovery is typically at the partner level; it’s usually not an institutional decision.
In other words, leveraging existing relationships can be hard. It will be interesting to see how Pitney Bowes now approaches the legal market.

“EMC Corporation, the world leader in information management and storage, today introduced the EMC eDiscovery Solution, an integrated offering of EMC information and content management software, networked storage and professional services designed to help customers of all sizes better manage the legal discovery process.” [emphasis added] 

So opens a February 27th EMC press release (spotted on Lit Support Blog). EDDix recently predicted a convergence of archival/retention systems with e-discovery solutions (details here). EMC reports that it is the first “storage company to deliver an integrated solution for proactive e-discovery.” It seems EMC is positioning itself to be a turnkey storage, archival, and e-discovery solution for corporate America, including professional services.

Corporations may now face an interesting choice: deploy a single, integrated system or assemble a best-of-breed solution. A huge consideration: the biggest e-discovery cost is typically lawyer review time. A best-of-breed approach with the right advanced search tool might yield a lower total cost. But even if that is true, many companies will likely find the integrated solution attractive. In any event, it probably would not be very hard for EMC to integrate other tools into its solution - and the company has made numerous acquisitions over the last few years.

Certainly this is news for law departments and corporations. But it is also news for law firms, which should closely track the evolution in how their clients manage information.

A newly published survey suggests that the e-discovery market is consolidating. 

Growth Curve, the AmLaw Tech column in the February 2006 issue of American Lawyer Magazine, presents the e-discovery portion of the results of a recent survey administered in connection with the Litigation Department of the Year article in the January issue. The magazine surveyed 81 litigation department chiefs at large law firms. They asked about e-discovery vendors, scanning and coding vendors, and software used to manage discovery documents and cases.

“[T]he industry fragmentation that we spoke of [2 years ago] appears to be waning.” The article points out that 2 years ago, 24% of respondents listed Kroll as their primary EED provider; this time 46% did. It also quotes me as saying “It is more efficient to have a few big players than to have many smaller ones. A larger vendor can offer more capacity, higher quality, faster turnaround.”

Consistent with other recent surveys, this one finds among the top five e-discovery vendors - all with mentions by more than 20% of respondents - are Kroll, Applied Discovery, EED, and Fios.

Legal Tech is beginning. The Hilton in NYC is already buzzing with attendees on Sunday night. Even before the show, the elevators tell a story. 

For the last few years, a Legal Tech vendor has an ad running on continuous loop on the closed circuit TV in the elevators. (Otherwise it’s a news channel.) Three years ago, knowledge management was the topic; this year and last, it’s e-discovery. I don’t know if the “elevator index” is a reliable predictor (leading or lagging for that matter), but take it for what it’s worth.

Of course, the great thing about the elevator index is that it goes up and down. You just have to figure out which!

Earlier today I wrote that legal offshoring is no big threat according to a new report. But that does not mean the legal outsourcing market is standing still. An Indian company has just acquired electronic evidence processing assets from Bowne. 

Integron (already featured in my outsourced legal services list), yesterday announced in a press release “the acquisition of the electronic discovery services business from Bowne (NYSE: BNE) for undisclosed terms.” Integron will provide e-discovery consulting and processing from US offices and document coding from its facilities in India.

The e-discovery market is booming. A boom does not, however, guarantee success. According to a press release, big player Daticon has declared bankruptcy and been acquired. 

In the release dated Jan 17, 2006, acquiror Xiotech writes that “today it has reached agreement with Daticon Incorporated to purchase the assets of that company. Daticon is a Norwich-based litigation support and document conversion services company that filed for Chapter 11 bankruptcy today. Xiotech is pledging to keep Daticon’s operations based in Norwich, Conn. protecting the company’s 170 local jobs.” (Press release spotted at Litigation support industry news and information blog.)

Xiotech is a storage solution company. The release quotes industry analyst, Michael Clark of EDDix: “To capitalize on growing opportunities around EDD, existing e-discovery companies will need more expertise in data storage and management. Combining a company like Daticon with Xiotech brings together a lot of the right capabilities.” I agree with Clark and suspect that we will see further consolidation.

EDDix provides objective, in-depth research about the e-discovery market. Its latest study, Corporate Litigation Readiness, presents several interesting findings. 

EDDix reports are typically based on research conceived, executed, and analyzed by EDDix. Here, an e-discovery vendor commissioned a detailed survey of in-house counsel and turned over the results to EDDix to analyze. Working from a less-than-perfect research instrument, EDDix nonetheless teased out interesting findings:

• Corporate focus on compliance will cause companies to acquire archival/retention systems that ultimately handle more aspects of e-discovery. And corporations will play a bigger role in setting e-discovery strategy. EDD vendors will need to re-focus the services they offer. But the process will likely take a full decade.
• For the moment, however, the data suggest that many general counsels still do not have a handle on electronic discovery details. The survey found that 62% have an “e-mail archiving system.” Would that it were so. EDDix points out that many technology surveys find that the penetration of e-mail archive systems is under 30%. The interpretation: in-house counsel may be confused about the systems their companies use and their capabilities for e-discovery. But compliance and risk managers will likely take the lead role, so it doesn’t really matter that much.
• Fear of sanctions, despite the widely publicized Zubulake and Morgan Stanley decisions, is not driving change in corporations; rather, it is compliance mandates, particularly SOX. This drive will ultimately lead corporations to adopt enterprise content management systems.

More information on the Corporate Litigation Readiness: free summary for members, which requires free registration; full-study costs $798 for members and $1450 for non-members.

Can law firms rely on software rather than lawyers to review huge volumes of documents in discovery? In Concept Searching in Discovery I wrote that this is an empirical question. 

Now, a well-known litigator has gone on record saying the answer is yes. Fred Bartlit of Bartlit Beck Herman Palenchar & Scott

“says that his firm has run blind tests pitting document discovery software against a document review team of lawyers, and the software performed as well or better than its human counterpart. Bartlit concedes the system may not be perfect – but then neither are associates, and they cost a hell of a lot more.”

Reported in The Billable Hour: Are Its Days Numbered? on law.com.

Merger activity continues in the US electronic evidence market. 

EPIQ Systems, “a national leader in the market for fiduciary management and claims administration systems and provides an advanced offering of integrated technology-based products and services,” has acquired nMatrix, a “a leading provider of case management and document management products and services for electronic discovery and litigation support” according to an EPIQ press release. The acquisition is for $125 million ($100m cash and $25m stock); nMatrix’s projected operating revenue for its fiscal year ending December 31, 2005 is approximately $20 million, more than double the revenue for its fiscal year ending December 31, 2004. Sounds like a pretty good multiple to me. (Spotted this news on Litigation support industry news and information.)

Many have and are predicting consolidation in the e-discovery market. I’ve been expecting a storage or information life cycle management company to acquire a pure-play e-discovery company. EPIQ and its business are new to me; it will be interesting to see what synergies emerge over time.

David Snow, legal technology editor for law.com/tech, assembled a panel to discuss “Experts’ EDD Trend Watch and Product Picks.” The recorded conversation was excellent. (Easy registration required; sound takes about 30 seconds to start once controls display.) 

The action in e-discovery is unprecedented. The last big change in discovery occurred some 15 years ago when the market moved from paper files and coded databases to scanning, OCR, and full-text. That transition raised difficult issues but relatively few lawyers or firms paid much attention. Today, the e-discovery story is different. With billion dollar plus judgments stemming from discovery problems, pending rule changes, and the enormous cost of processing and reviewing electronic evidence, lawyers and law firms are very much engaged.

The attention e-discovery commands is reflected in the outstanding panel law.com assembled to discuss a range of timely topics. Participants were:
-Jerone English, Litigation partner and co-chair of e-discovery committee, Pillsbury Winthop Shaw Pittman
-Ron Friedmann, President, Prism Legal Consulting, Inc.
-Jonathon Hughes, Partner, director of litigation department, Howard Rice Nemerovski Canady Falk & Rabkin
-Jim McKenna, Litigation Technology Manager, Morrison Foerster
-Lori Ann Wagner, Partner, Redgrave Daley Ragan & Wagner

Highlights of our discussion include:
- Challenges of comparing vendor costs
- Proposed Federal Rule changes and Sedonna Principles and implications
- The value of early EDD planning
- The degree to which law firms should rely on vendors versus do EDD work in-house
- How firms are staffing to manage e-discovery
- Selecting vendors and products
- The role of concept searching in document review

Yesterday, I summarized several ALM articles on EDD published in connection with the webinar.

ALM’s law.com published today (on the web) a collection of stories on E-Discovery Trends of 2006. 

A recorded webinar, “Experts’ EDD Trend Watch and Product Picks,” will soon be available. I was one of five panelists for this interesting discussion. Once it’s live, I will post some thoughts about. In the mean, time four articles cover a range of EDD topics:

The Forecast for EDD ("Chief among developing trends, the EDD market is rife with confusion and ready for consolidation.")

Piecing Together EDD’s In-House vs. Outsource Puzzle ("Most law firms and corporate legal departments turn to outside companies to help them manage and process portions of the data. Doing this can save time, money and energy – but experts say it’s important to have established relationships with reliable vendors before taking on large cases.")

EDD Training: A Growth Industry ("Law firms and corporate counsel are dealing with EDD in different ways, but the fears are the same: sanctions such as those in the Morgan Stanley and Philip Morris cases… So let’s say you are a lawyer who is chosen to be the firm maven on EDD. Or let’s say your company wants to train a core group of lawyers. Where do you go?")

More Than OneNote: Will Microsoft Break Into EDD? ("Microsoft is not currently offering products specifically designed for e-discovery processes, but it’s just a matter of time before it does, says Laura DiDio, research fellow at industry analyst firm The Yankee Group.")

A boutique law firm has just formed that will focus on e-discovery. 

New Law Firm Targets E-Discovery Angst on law.com (in The Recorder) reports that a new, four lawyer, virtual firm “will offer strategic advice to businesses on improving the management of their digital information and records, how best to respond to electronic discovery requests and the risks involved in the integration of different information systems in the aftermath of a merger or acquisition.” The four lawyers met at as members of the Sedonna Conference.

I have previously suggested that large law firms appoint a senior lawyer as the e-discovery expert or “go to” person for e-discovery. This report just reinforces this idea and growing importance of firms getting a grip on e-discovery, from the legal issues, to the technology, to the management of projects.

A leading electronic evidence discovery vendor may be changing ownership structure. 

The August 26th Transaction Granted Early Termination published by the FTC (PDF) lists private equity investor Welsh, Carson, Anderson & Stowe, e-discovery vendor Electronic Evidence Discovery Inc., and EED founder and CEO John Jesson.

Under the Hart Scott Rodino act, parties to certain proposed transactions must file a “Hart Scott Rodino” notice and provide the government an opportunity to review the transaction for potential antitrust problems. The FTC web site states that the agency grants “a request for ‘early termination’ … only after compliance with the rules and if both the Federal Trade Commission and Department of Justice Antitrust Division have completed their review and determined not to take any enforcement action during the waiting period.”

I am neither a dealmaker nor merger expert, but I suspect this suggests a transaction between Welsh Carson and EED is imminent. Welsh Carson says that it “believes that its value-added investment strategy is best executed when it acquires operational or board control of companies.”

We in the legal market know about the e-discovery explosion. The shock waves now reach outside legal. 

MetaLincs Debuts Discovery Software in the 8/22 issue of eWeek magazine describes new e-discovery software. I’ve been reading eWeek for years and recall only one or two prior feature story on discovery software. Interestingly, the featured product is new to me (other interesting search tools are described here and here)

The article notes that with “more and more enterprises finding themselves under intense scrutiny due to regulatory compliance mandates and litigation pressures, e-mail and document discovery is fast becoming a labor-intensive proposition laced with inconsistency and a small margin of error.” eWeek describes the company’s software as including features for “indexing, processing, search, review, reporting and production.” Visual displays are a key aspect; the company CTO explains that the “software presents customers with a visual representation of the relationships between people, time, documents, events and communication patterns based on appropriate context and search criteria.”

More information at the company web site.

The August Corporate Legal Times reports on ways to reduce litigation costs. Technology plays a big role and one of my favorite approaches - decision trees – is featured. 

GCs Offer Strategies for Reining in Litigation Costs reports on a recent general counsel roundtable. The sub-title is “Decision Tress Play a Key Role in Settling Complex Cases.” My former colleague and now law professor David Post and I explained the basics of risk analysis and decision trees in a 1990 article (the concepts have not changed since then) and I recently wrote about software for doing decision trees.

A sidebar to the article lists “Controlling Costs in 10 Easy Steps” by FMC general counsel Jeff Carr. Two involve technology: (1)"Require that your law firms create a decision tree or other early case assessment within 90 days of retaining them.” (2)"Once you are in litigation, use an artificial intelligence engine instead of people to triage electronic documents for discovery relevance. Seek to reduce the documents people need to review to the important ones as opposed to using people to review documents to determine which are important.”

And two of Carr’s steps relate to knowledge management techniques: (1) “After it’s all said and done, do an ‘after action’ or a ‘lessons learned’ review.” (2) “Remember the whole process is circular… learn from the process to avoid it again.”

“IBM plans to give away key search technologies for corporate data retrieval that use concepts and facts instead of simpler “keyword” searches” according to a press release. This could have implications in the e-discovery market. 

An IBM spokesman notes that major players such as Google and Yahoo have stayed focus on searching the Internet, not the enterprise. Now “IBM plans to openly offer other software developers its Unstructured Information Management Architecture (UIMA), a technology that can analyze text within documents and other media to understand latent meanings, relationships and facts”. Some fifteen companies, including Attensity, Inxight, and ClearForest plan to incorporate the UIMA framework. UIMA will be available on an open-source basis.

It’s not clear what impact, if any, this will have on search in e-discovery, but it sounds promising. IBM has spent 4 years researching this framework. After the ABA TechShow in April, I spoke to an IBM spokesperson who said IBM had no interest in the e-discovery market. But by opening up the technology, e-discovery vendors will have access to it. (For more background on UIMA, see my prior posts here and here.)

Separately, for law firms running WebSphere, this announcement also has potential knowledge management implications: “IBM is also offering its WebSphere OmniFind software for helping users perform searches on unstructured data in a variety of formats or languages, be they located in databases, e-mail files, audio recordings, pictures or video images.”

Update, 1/25/06: Articles: IBM releases UIMA source code as open source; IBM Turns Over Search Project To Open Source Community
Update, 8/11/05 at noon eastern: Some additional detail at this eWeek article

Ten days ago I wrote about some interesting articles. This summer is proving to have more than the usual number of useful legal technology articles, including an intriguing report in Information Week about e-mail visualization software that could help in e-discovery. 

E-Discovery. Revealing E-Mail’s Secrets in Information Week’s August 1 issue describes a new visual analytic tool for e-mail from Spotfire. “Spotfire Inc. this week will introduce a tool for uncovering patterns and relationships in information extracted from E-mail messages that will be as useful for anti-terrorism efforts as it will be for analyzing business data.” The money quote for e-discovery, citing a senior manager of CIA-funded In-Q-Tel:

‘We identified Spotfire as a leader in the visualization.’ market, he says. That market is important because it’s a place where visualization hasn’t yet been used. ‘E-mail has become an increasing part of electronic discovery.’

At least as of the time of this post, I could not find reference to e-discovery on the Spotfire web site. Those of us with a history in litigation support know that many interesting discovery tools emerged from the intelligence world, so this bears watching.

In other news….

Online Legal Services. Spotted at Amazing Firms, Amazing Practices: Why going online is better for everyone by Richard Susskind, describing a new online service by Eversheds that sets “the pace for the next generation of employment lawyers” by generating a range of documents. (The service is based on DealBuilder by Business Integrity - I was very impressed by this tool when I saw a demonstration of it recently. To call it just document assembly is probably too limiting.)

KM. Spotted at excited utterances: From laggard to leader? How the legal sector is finally embracing KM has interesting insights about KM at Baker McKenzie, Blake Cassels, and Freshfields.

Blogging. Spotted at Adam Smith, Esq.: Order From Chaos Via RSS (also from Information Week, August 1), a good overview of how corporations are using Really Simple Syndication (and a good follow-on to my recent Follow-up on Enterprise Blogging and RSS post).

The e-discovery explosion creates a risk that firms will lose sight of the larger litigation support picture. There may not be a single right way to “do lit supp” but firms should analyze their processes and strive to develop consistent guidelines. 

The table here presents a framework for thinking about litigation support. Columns represent phases of a case and rows represent player roles, software involved, and issues to consider. (Apologies, but putting the table in the blog does not work for tech reasons.)

The text in each cell is merely suggestive. Each firm, indeed each case, may have different answers. Even the rows and columns can differ. The point though is twofold:

1. Make explicit and informed litigation support decisions.
2. Analyze processes at both a micro and macro level and strive to develop a consistent approach based on adherence to a set of guidelines that covers typical scenarios.

Decision trees are a formal method to assess risk in litigation and estimate the expected value of a case. My former colleague and now law professor David Post and I explained the basics of risk analysis and decision trees in a 1990 article (the concepts have not changed since then). I have now come across new software for doing decision trees. 

The June/July issue of Law Office Computing, in Distinguished Decisions reviews two decision tree software options. One is Data, by TreeAge, referenced in our article. New to me, and the one favored by the reviewer, Grace Suarez, is DecisionPro by Vanguard Software.

Most litigators are not familiar with decision tree analysis, which I think is too bad. Granted, learning to use risk analysis is not a snap. But it strikes me as the only and best way systematically to evaluate a case. Especially with many corporate counsel now focusing on early case assessment, I have not figured out why decision trees have not gained traction. Comments anyone - am I missing something or are litigators missing something?

I recently learned about another e-discovery tool called Engenium, “a next-generation conceptual search technology created using patented methods of automatic learning and semantic analysis." 

Organizations adopting it for search recently include CaseShare (a cross-enterprise collaboration tool, including litigation support and deal rooms) and Foley Lardner for its internally developed electronic discovery system. (Engenium press releases: Caseshare, Foley).

Attenex, an e-discovery vendor with an interesting visualization interface, has raised $5 million in VC money according to the Seattle Post Intelligencer. Another sign of the rapid growth in the e-discovery market.

The jury verdict against Morgan Stanley of $604 in the Perelman-Sunbeam case - based to a large extent on doing e-discovery wrong - raises the question of just what is happening in e-discovery practices. Good data on actual practices are scarce, a problem my friends at EDDix are working to rectify. 

In Perelman Beats Morgan Stanley, the Wall Street Journal reports (5/17/05):

“The verdict shows how costly it can be for corporations who mishandle the document production in legal proceedings, a process known as discovery. Lawsuits often require companies to comb through electronic archives, and these cases are sometimes won or lost based on how the litigants perform these tasks… In the Perelman case, Morgan Stanley kept uncovering new backup tapes, couldn’t perform full searches because of technology glitches and gave material to the other side that was sometimes incomplete or late” (emphasis added)

So, just how do litigants perform these tasks? Litigation support has always been as much art as science and practices tend to vary widely, both within and across firms. So just how are law firms staffed to handle e-discovery? How much do they do internally and how much do they outsource? Who is responsible for making sure a client follows preservation policy? What drives vendor selection?

EDDix, an independent market research company focusing on e-discovery, is now running Law Firm Perspectives on EDD, which uses the Web to interview AmLaw 250 firms to determine current EDD practices. I am working with EDDix on an affiliated basis to help with this primary research, which we expect will produce deep insight into important e-discovery issues and practices. We are seeking online participants: litigators, CIOs, lit supp managers, and transactional lawyers doing due diligence. Participants will receive a free summary report of the research and access to EDDix’s already completed EDD Supplier Landscape study (my blog comments on this study and access to a couple of chapters are here).

To participate in the online interviews, click here: on the left are links that allow you to preview the interview; on the right are links to complete the interview.

The lead feature story of the Wall Street Journal today is Age of Discovery - How Morgan Stanley Botched A Big Case by Fumbling Emails (subscription req’d). If litigants have not already been put on notice about e-discovery, this article is the wake-up call. 

Both the underlying dispute between financier Ronald Perelman and investment banker Morgan Stanley and the e-discovery problems have been previously reported. This article is noteworthy because it elevates to front page status the issues of managing e-mail in the discovery process. (The court found that Morgan Stanley “deliberately” violated production orders and, as a sanction, instructed the jury to assume the company helped defraud Perelman.)

Corporate America now stands warned that they have to get discovery right or face serious court sanctions. And the article does not make quite as clear as does the court’s decision the dangers of doing “home grown” e-discovery management.

FLASH UPDATE: As of about 3pm today, newswires report the jury in this trial awarded Perelman damages of $604 million.

The volume of documents to review in discovery continues to explode. Can lawyers rely on full-text and concept-searching instead of lawyers to review documents? Craig Ball provides a good analysis of this question in Unclear on the Concept in Law Technology News. 

Ball’s answer is that “automated search systems must be periodically tested against an evolving sample of evidence scrutinized by human intelligence.” I like his empirical approach because that’s what I have argued in prior posts such as Evidence-Based Law and Thoughts on Full-Text Retrieval.

Potential limitations notwithstanding, Ball suggests concept searching has a role in overcoming OCR limits, uncovering hidden relationships, and prioritizing the order of review. He makes the useful suggestion that those who do want to rely on this approach should invite “the requesting party to contribute keywords and concepts for searching is an effective strategy to forestall finger pointing about non-production.”

As lawyers consider the reliability of different approaches, they should keep in mind that objective (bibliographic) coding is typically no more than 98% accurate. As for subjective coding, I heard one anecdote of a large law firm that used a concept search tool after a lawyer review and the tool found many documents the lawyers missed. An assumption that lawyer review yields the best result is not necessarily correct - it’s just what the profession happens to find comfortable.

The explosive growth of electronic discovery may be leading to a new position: large firm lawyers who are discovery specialists. 

Law Technology News reports in the April 2005 issue that Sullivan & Cromwell has recently hired an experienced lawyer, Thomas Barnett, as special counsel focusing exclusively on electronic data discovery: Barnett

“advises the firm’s attorneys on EDD issues, working in tandem with the firm’s litigation support staff. The firm will also offer his services to clients, both as a consultant and as an expert witness, he explains. He expects to bill at a partner rate, and reports to the executive committee. ”

I know one other AmLaw 100 firm has a partner who serves as the in-house EDD expert, though that is not his only role. That Barnett reports directly to the executive committee rather than the head of litigation suggests the position is “staff,” though he is listed as special counsel on the firm’s web site.

It will be interesting to see how this and similar positions evolve and whether other large firms will create functionally similar roles. In my view, this is a good trend. Firms would be well-served if the lawyers in these roles had not only EDD expertise, but also project management expertise. It’s not enough to know the rules of discovery and the technology - succeeding in large EDD matters also requires project management discipline.

As for reporting lines, that is a tough call. Litigation support can fall under IT, its own group, or occasionally the litigation practice. In-house lawyer-EDD experts will similarly have to cut across staff and lawyer lines, so it will take time to determine the optimal reporting relationship. But reporting to the executive committee certainly sends a strong positive signal that a firm is taking EDD and this role seriously.

A prior post reported on EDDix’s finding that the e-discovery vendor market would consolidate. Yesterday another acquisition occurred. 

FTI Consulting has acquired Ringtail, according to an FTI press release.

In my view, law firms need not yet worry about the reduction in number of vendors. FTI already worked with Ringtail, so this acquisition does not seem to change any market fundamentals. In any event, there are still numerous vendors from which to choose.

Lawyers use evidence to prove a case. Why not apply that same discipline to law practice itself? Medicine is adopting an evidence-based approach, relying on well-controlled studies to determine which diagnostic tools and treatments work best. An article about contract attorneys makes me think the same approach is necessary in the legal market. 

Calling Up Reinforcements in The Legal Intelligencer describes the extensive use of contract attorneys to review documents in litigation. It reports that at least in some instances, clients ask law firms to use contract lawyers to lower costs.

Determining the most cost-effective way to review documents is an empirical question. I have previously suggested that offshore lawyers might be able to review documents. Another option would be to rely on advanced full-text engines to perform a first-pass cut, reducing the quantity of documents humans must review. (For more on full-text, see here and here.)

To be sure, the offshore and technology options raise issues. In addressing these issues, lawyers too often rely on uninformed reactions - “we have not done that before” or “there may be ethical issues” - rather than on solid, empirical data. Perhaps a consortium of clients (as suggested recently by Cisco) could muster the resources to collect and analyze data. It would not be that hard. One would need a set of documents reviewed by the “gold standard” of associates. Then take that set and run it through contract attorneys, offshore attorneys, and various full-text engines. Have a statistician determine the comparison criteria prospectively and then compare outcomes. Be prepared to run the test more than once. Use the outcome to establish a best practice and periodically re-validate that process.

Lawyers may never conduct such tests but if e-discovery costs keep rising, a time may come when corporate CFOs step in, commissioning an empirical study and and imposing the findings on their law departments. And if the CFOs are in charge, lawyers may have less influence in how various concerns are weighed in reaching a business decision concerning the best way to review documents.

When your only tool is a hammer, everything looks like a nail. Arguably, litigation support has been a one-tool affair to date. What I saw at Legal Tech this past week suggests the possibility - if not necessity - of expanding the toolkit. 

To be sure, litigators have long been able to choose from multiple tools. But the differences among them have not been that great. More importantly, both law firms and litigation support processing vendors typically use a single tool to do their work.

At Legal Tech this past week in NYC, I saw several interesting and potentially very useful new ways of processing or analyzing the now ubiquitous vast volumes of digital data. Various full-text and semantic engines work in different ways and may produce different results. My guess is that there will be a market shake-out, but that several software developers and consultants that offer distinct approaches will prosper.

Consequently, litigation support professionals may need to change their mindset about how they process and analyze documents. The single tool approach may not longer suffice. The nature of a matter, the types of data and documents, and the goals of the analysis may all require a case-by-case determination of which tool is best for the circumstances. The hammer may be supplemented with screw drivers, wrenches, awls, etc.

Of course, firms should not lightly buy (or even rent) one of everything. But the differences in technologies and processes warrant staying aware of the options and preparing for the possibility that a single solution may not suffice in the not-too-distant future.

Newly developed software may prove promising to address the problems of duplicate and near-duplicate documents in discovery. 

Why are duplicates a problem? First, lawyers may inconsistently tag duplicates or near dups, meaning some versions are designated responsive and some non-responsive, some privileged and some not-privileged. This can lead to confusion, credibility issues, and possibly even sanctions, especially in portfolio litigation where multiple cases turn on the same set of documents. And second, numerous duplicates can significantly increase the time - and therefore cost - for reviewing documents.

Existing approaches to detecting duplicates have limitations. One approach is to use a “hash,” a mathematical technique. This approach determines only if documents are completely identical; a single difference in one character or the file path makes two documents different. Another approach is to use meta-data to detect possible duplicates.

Software start-up Equivio has software that, upon first evaluation, allows litigators to identify near duplicates and adjust what is meant by “near.” For example, drafts of the same document prepared by different authors on different days with different file names could be identified as potential duplicates. (Hashes and meta-data cannot do this.) Such differences may be relevant to the case, but often they are not. Clustering near duplicates and reviewing them simultaneously can be a great advantage in helping to insure consistent responsiveness and privilege designations and in saving review time.

This is yet another example of software that can help address the new challenges created by e-discovery. It seems increasingly clear that over the next few years, automation using highly sophisticated semantic techniques will ease the burdens of reviewing and managing digital discovery documents.

A recent article suggested that law firms should make electronic evidence discovery an in-house profit center. Blogger Adam Smith, Esq. beat me to the punch in explaining why this is an ill-considered idea. 

His post, EDD: Look Who’s Not Coming to Dinner, explains three reasons why law firms should not bring EDD in-house: highly variable demand and capacity utilization, technology and processes in a constant state of flux, and disequilibrium in the state of the industry and its profit margins.

To this, I would add another caution. It is much easier to explain and justify a third-party disbursement than a law firm’s own time or line item charges (e.g, copying). Clients realize that the EDD space is rapidly changing and can reasonably expect a law firm to seek competitive bids. This does not mean that the lowest price wins; rather, it helps assure a reasonable price for the right services.

As I indicated in my prior post, I expect that law firms will develop the expertise to manage EDD. It’s one thing to have that expertise, it’s another to own all the tools of productions and manage the process internally.

Litigation support has long been a fragmented market: many suppliers and no single one with high market share. A new study suggests this will change. It also finds - and I believe this is related - that most large law firms lack electronic evidence expertise. What’s the study and what’s the connection between findings? 

Electronic Discovery in Litigation - EDD Supplier Landscape was released this fall by EDDix, LLC , which provides independent research, analysis and reporting on the Electronic Data Discovery (EDD) marketplace (think Gartner or Meta Group).

EDDix conducted the study via extensive primary and secondary research, including interviewing 15 EDD vendors in-depth. EDDix is making the study available via selected blogs. For background, a free summary, and purchase information, click here.

Two of the study’s findings stike me as particularly interesting:
1. EDD suppliers believe that only 25% of AmLaw 200 firms have the requisite knowledge and experience to handle complex EDD matters professionally.
2. Vendors believe that the market for litigation support services will consolidate. They expect the number of specialty firms to almost double in three years but the number of full-service firms to drop by 40%. Half of the 15 vendors interviewed think there is a 50% or better chance that their company will acquire or be acquired by another EDD player in the next 3 years.
Click here for the EDDIX study findings (note this opens a PDF file in a new window). In my view, these two seemingly unrelated findings are connected.

In the age of paper, lawyers could recover from bad document management decisions by marshaling an army of lawyers or legal assistants to review hundreds of boxes of documents at the 11th hour. That just does not work in the digital age. The volume of documents, the number of file types, the issues around meta-data, and the difficulty of processing data mean that special expertise is required. Moreover, an army of people cannot do much with a bunch of disk drives at the last minute. As the mechanics have changed, so too has awareness of the issues. Lawyers are wising up and realizing they need special expertise.

The finding that 75% of law firms lack expertise is consistent with anecdotes I have heard. I believe that firms will develop this expertise (after all, they pretty much have to) and that this expertise will be centralized. Whether the expertise resides with lawyers or litigation support departments remains to be seen (I have seen examples of both). But either way, the field has become complicated enough that each large firm will need its own experts.

Once lawyers realize specialists are needed and once firms create expert groups, the ground is set to shift: purchasing will move from an individual partner basis to an institutional basis. And once purchasing is institutionalized, there is a tendency to use fewer vendors and give each one more volume. So, in my view, the current lack of know-how will contribute to vendor consolidation.

Stay tuned for additional commentary on these findings.

The article Federal Courts Propose Rules for E-Discovery in the New Jersey Law Reporter provides an interesting update on potential changes in the rules of discovery, especially with respect to digital data and electronic evidence discovery. 

Readers who follow the e-discovery world know that the on-going transition from paper to digital discovery raises a host of difficult issues, technical and legal. The article is a good read on current issues; I will focus on one: preserving digital data once a law suit begins is a concern, especially given that many large corporations have systems that are not designed to do so.

It is no surprise to read that “[t]he most controversial of the proposed rules may be an amendment to Rule 37 that would create a narrow “safe harbor,” protecting a party from sanctions for failing to provide electronically stored information in some circumstances. ” The article goes on to report that the draft rule would provide a safe harbor from the inadvertent destruction that “resulted because of the routine operation of the party’s electronic system.”

It is a surprise, however, to read that U.S. Magistrate Judge Ronald Hedges opposes this provision and “says he does not see why electronic data should be singled out for special treatment. ‘If we’re going to have a safe harbor provision, it seems to me we ought to protect information in any format, not just an electronic medium.’ ”

Excuse me? I hope this does not reflect a dominant view among the judiciary. I take no position on what rule changes would be best, but one thing is obvious: digital data and paper are different. Lawyers, including the judiciary, need to understand that. Now, if there were a type of paper that, when you opened the file drawer, it disappeared or forever changed, I might agree with Magistrate Hedges’ view. Or, if by walking in my office and turning on my light, the dates on every piece of paper in my office automatically changed to the current date, I could understand his point.

Whatever direction the rules head, they need to reflect the reality of corporate information systems, which were designed to manage data for operational purposes, not for preservation in litigation. It’s an old saw now that just turning on a computer changes some data. In a similar vein, corporations that run on SAP or other enterprise systems cannot simply turn off their systems or easily freeze all the data.

It will be hard, but I believe enough smart people are working on new rules that the right balance can be struck between the legitimate needs of business/technical operations and of lawyers in litigation. But in my view, the starting place is to recognize operational business requirements and the difference between digital files and paper. It would be a shame if our already very expensive system of litigation became even more so because discovery rule changes that end up creating unreasonable burdens to “protect” data.

Regular readers of my blog know that I have a long-standing interest in the potential of full-text search technology for both litigation support and knowledge management applications in large law firms. With the seeming explosion of new companies offering advanced full-text approaches, I have been trying to sort out what is really new and what really works, so I have asked an expert. 

A couple of years ago I met Sharon Flank of DataStrategy Consulting. Sharon has a PhD from Harvard in computational linguistics, meaning she is an expert on full-text search technology (her bio is at the web site). The company offers technology due diligence and product strategy and technology planning consulting, especially in information retrieval, natural language processing (NLP), and visualization.

It occurred to me that she was the perfect person to ask about the underlying developments. Last week, I sent her an e-mail message asking the following:

“Has there been any conceptual break-through - at an algorithm level - in full-text and semantic analysis in the last 10 years? From 1990 to 1995 I looked at many products: PLS, Verity, Excalibur, Conquest, Fulcrum, and others I can’t now remember. It seems to me that those products did much of what current products currently do, except perhaps the extensive auto-classification (though that was less of a requirement back then). Clearly, the ability to process large volumes has gone up and user interfaces have improved. I’m not close to the computer science but am curious if the underlying advances have been significant, perhaps even quantum, or merely incremental. Thanks in advance for any thoughts.”

Sharon was kind enough to send back the following reply about the current state of natural language processing and full-text search:

“There are several underlying important developments over the last decade or so:

Incorporating user feedback to refine search results, usually indirectly rather than explicitly, making results better through machine learning. [Amazon.com is the most-often cited example of this with it’s “if you like A, you’ll also like B.”]

Assessments based on usage or referral. This is what makes Google so useful and popular. This approach gives higher rankings if other web sites point to a target or if that target gets a lot of hits.

Various approaches to using taxonomies. The better applications use taxonomies as a navigation guide but don’t force it or require administrators to implement it. Vivisimo.com is an example of interesting, automated clustering approach.

Better handling of phrases. Google automatically parses phrases and deals with search terms as phrases. This now seems natural but in the AltaVista days, you couldn’t tell a Venetian blind from a blind Venetian [example courtesy of Prof. George Miller, Princeton Univ. - too good not to cite].

Context-sensitive search is now an emerging trend. Systems track what users have previously searched for and infer interest in the same domain to refine search result. So if you look for “line” and a system knows you’ve just looked for “tacklebox,” then it infers you mean “fishing line.” Or if you search for bagels and the system knows you are in 20009, it tells you that you can buy them at Comet Liquors (which happens to sell bagels).

“More generally in natural language processing, the statistical and linguistic approaches are converging in a new way: use massive amounts of data (i.e. the Web) to get statistical answers to deep linguistic questions, like “How do we figure out what the most likely referent is for the pronoun ‘they’?” Or “How do we determine the correct sense for ambiguous words?” These things aren’t in search engines yet, but you can expect to see more “intelligent” features coming out of this approach.

“Looking at this list, you can see that the conceptual changes (breakthroughs?), with the exception of better phrase handling, are primarily focused around Web searches. When dealing with one-of-a-kind document collections behind the corporate firewall, many of these developments turn out not to add much to older approaches. So, at least for enterprise search, I too remain partial to some of the older products you mention, though I am disappointed that most of the old-time vendors have not updated their approaches beyond adding taxonomy support.”

I appreciate Sharon taking the time to provide this insight. The bottom line for litigators and litigation support professionals: you need to keep your eye on emerging technologies and not necessarily take a “one-size-fits-all” approach to managing large volumes of documents.

The Wall Street Journal reports today on the growing use of technology in trials. This raises an interesting point, an extension of my prior post about lawyer resistance to change.  

“Courtroom-Technology Firms Give Evidence State-of-the-Art Look” (WSJ, 5/26/04) reports that growing use of technology in court has “spawned an industry of tech firms revolutionizing the traditional courtroom with plasma screens, flat-panel monitors, and other sophisticated display devices.” Spurred by the Internet boom, more and more courts are wired to support computer use and display. The article reviews some of the pros and cons of computers in court.

I have always been a big believer in the visual display of information. I even wrote an article called “Practicing Law with Pictures” (published in an early edition of Law Technology News), but that was so long ago, I no longer have a digital version. It examined how even “still displays” could achieve great effect, both in and out of the court. Examples, all from actual cases at Wilmer Cutler, included a transaction flow-chart that finally allowed everyone to understand all the steps in a complicated sequence; a diagram of regulatory jurisdiction over banks that was widely credited with winning an argument on appeal; and a series of about 10 “stop sequence” schematic views of the final thirty seconds prior to a crash that was essential to letting a jury understood exactly what went wrong. The fact that I remember all these 10+ years later without anything to look at is a testament to the power of visuals over words.

PowerPoint may be overused and many displays may be (in the words of Edward Tufte) “chart junk,” but that does not diminish the value of visuals in or out of court. A dozen years ago Wilmer lawyers regularly used poster boards in court. Fancier display systems have been available for almost a decade. Why has it taken the profession so long to adopt?

I suspect that the main reason is another dimension of resistance to change (see my prior post, A New Approach to Control Outside Counsel Costs). Just as managing a case requires a different way of thinking than does arguing a case, visualizing it requires a different way of thinking than does writing about it. Lawyers who are not comfortable with visualizing should at least recognize this and be sure (whether for a trial or transaction) to have access to someone skilled in visual presentations. Technology here is merely a means to an end - the content is what counts.

Marsh & McLennan announced on May 18th that it would acquire Kroll, provider of electronic evidence services via KrollOntrack. It will be interesting to see what impact, if any, this has on the litigation support and electronic evidence market. 

Identical press releases (Marsh and Kroll) provide detail, as do articles in today’s issues of the New York Times and Wall Street Journal. According the press release, “[t]he transaction will broaden significantly the range of MMC’s risk and insurance services businesses and enhance its leadership position in risk management services. ”

Last summer, Lexis acquired Applied Discovery (see my prior blog post for details). That acquisition was presumably motivated by Lexis’ interest in expanding the services it offers in the legal market. In contrast, I would guess that the discovery and electronic evidence services that Kroll offers were, at best, a minor factor in Marsh’s acquisition.

Kroll, with 2003 revenue just shy of $500 million, was already a fairly large company for a legal vendor. It’s not obvious the added heft of Marsh adds value in its own right. So it will be interesting to see how KrollOntrak fares under new ownership. One concern - and I am no expert here - is conflicts. It’s possible that Marsh’s far-flung business dealings will create more conflicts for KrollOntrak than it would otherwise have had.

Since I first studied the litigation support market in 1989, it has been highly fragmented. A 2003 survey (PDF) of the electronic evidence discovery market by George Socha confirms that this is still true. In the 1990s, before the age of digital discovery, my recollection is that Uniscribe and Merrill attempted “roll ups” of litigation support vendors. My impression is that these roll ups were not smashing successes.

With two large corporations standing behind electronic evidence vendors now, it will be interesting to see if the supply side of the market changes or consolidates.

With the explosion of digital data in litigation, lawyers may need to re-consider the application of full-text technology to manage discovery. New technology and new thinking may converge to help tame discovery.  

In previous blog postings, I have mentioned Attenex, H5 Technologies, and Cataphora as potentially interesting approaches to help identify and manage documents in discovery. More recently, I have come across InfoTame and Northrop Grumman software offered by Driven, Inc.. I believe these and other advanced full text approaches are promising, but only time will tell if they are superior to ordinary Boolean searching (for more on this, see my post Thoughts on Full Text Retrieval). [Ordinary Boolean search uses logical connectors such as AND, OR, and NOT and proximity operators such as “in the same sentence” or “within so many words.” These advanced tools usually have these features but layer on top of them various sophisticated techniques to extract more meaning automatically.]

Some serious commentators have suggested that the problem of huge volumes of data in electronic discovery could be addressed if opposing parties were to agree to use full-text searches to narrow the universe of document. I first saw this approach advocated by Adam Bendell of Strategic Discovery, Inc. in his article “Discovering a New Approach” in the The Future of Litigation, a Fall 2003 supplement to the American Lawyer magazine. More recently, Robert Brownstone of Fenwick & West writes in Collaborative Navigation of the Stormy e-Discovery Seas (10 RICH. J.L. & TECH. 53 (2004)) that “[t]o be efficient and effective, [the electronic discovery] process must mandate and enforce cooperation among the litigants as to search terms and other selection criteria needed to narrow down huge data sets into manageable subsets.”

As I understand Bendell and Brownstone, it would suffice just to use simple Boolean search software to narrow the field. That sounds right to me. But forward-thinking litigators and law firms need to consider whether more advanced search technology can provide tactical advantages and lower costs.

Perhaps the “hottest” area in legal technology right now is discovery, which is increasingly shifting from paper to digital. There seems to be activity on many fronts.  

At the Legal Tech show last week, there were many vendors offering a range of solutions. Almost every legal publication I read regularly has a range of articles on electronic discovery. And law firm technology managers are increasingly focusing on how best to serve clients in discovery and on their own records retention policies.

Those following electronic evidence discovery may be interested in a recent article appearing in the ABA Journal eReport. I recently spotted on LawTechGuru a reference to UNLOCKING ELECTRONIC EVIDENCE - ABA Task Force Offers Draft E-Discovery Standards, which is an article about proposed ABA E-Discovery Standards (note that this is a PDF file on the Federal Judicial Center web site). The article summarizes the key proposals. Separately, another set of potentially influential e-discovery principles was published in January by The Sedona Conference.

It will be interesting to see how court rules and decisions evolve over the next couple of years. Law firms that want to serve their clients effectively should be sure to have a good grasp of the law concerning and the technology options to manage electronic discovery.

Last month I met with a technical team at a US government organization that has investigative and enforcement authority. The topic of our meeting was to discuss current developments in electronic discovery. This team is guiding the organization’s strategy and tactics for dealing with electronic discovery. What I learned was very impressive.  

The organization has clearly thought through the issues of dealing with large volumes of diverse digital data. They have considered how to integrate paper and how to deal with many different file types. While they do not have all the answers - after all, no one doing electronic discovery does yet - they have a vision and the toolset to back it up. I was impressed to learn that they are using products such as Autonomy and ClearForest, both of which offer sophisticated search, categorization, and meta-tagging features.

Lawyers in the private sector often assume that the government lacks technical skill and sophistication. Clearly, the DOJ’s performance in the Microsoft antitrust case should have dispelled that notion. And this particular organization impressed me as being further ahead in both its thinking and its execution than most large law firms.

While there is no substitute for good lawyering, to some extent, modern litigation and government investigations are becoming an arms race. This race is about who can get a better handle on electronic evidence. After this meeting, I’d have to say that at least some parts of the government may be winning the race.

At some point, perhaps soon, it may be malpractice if litigators do not ask for and obtain digital data from opposing counsel. An article by Patrick F. Dorrian, Jurists Offer Perspective, TIps on Electronic Discovery in the November 2003 issue of Metropolitan Corporate Counsel examines this and other questions about electronic discovery.

Dorrian reports on a recent conference where Judge Loretta Preska spoke. She wrote the opinion in an influential case on discovery (Metropolitan Opera Ass’n v. Local 100, Hotel Employees and Rest. Employees Int’l Union, 212 F.R.D. 178 (SDNY 2003)). Dorrian writes:

“Responding to a question, Judge Preska explained that it is ‘hard to say’ whether an attorney’s failure to seek electronic discovery in a case could support a finding of legal malpractice. ‘The rules talk about the production of relevant information,” she said, “so we seem to create the burden to seek e-data.” While noting that the increased costs associated with electronic discovery ‘have changed the game,’ she added that she ‘can’t imagine how counsel who is responsible cannot seek relevant electronic information.’”

Twenty years ago, not all lawyers performed online legal research. Today, many lawyers and judges would consider that omission malpractice. If history and common sense is a guide, a similar rule is likely to apply to electronic evidence discovery.

In New Rule on Discovery of Computerized Data the New Jersey Law Journal (October 23, 2003) reports on what may be an important new rule concerning discovery of digital data in federal cases brought in New Jersey. This rule does not yet seem to appear on the Court’s own web site.

According to the article “Local Rule 26.1(d), requires lawyers, at the very start of a case, to review their clients’ computer and information management systems ‘to understand how information is stored and how it can be retrieved.’” The article reports that parties need to try to reach agreement on the preservation, scope, types of media, and who bears the cost of discovering digital data.

I spent two days and spoke at Legal Tech NYC September. A couple of my legal technology friends observed that there was nothing new and asked if I agreed or not. Drawing on my training as a lawyer, I said “yes and no.”

My first observation is that in the economy at large, the pace of technology innovation has slowed. To be sure, there are interesting developments such as Wi-Fi (wireless net connections) and grid computing (tapping the processing power of multiple PCs to solve complex problems). In reading e-Week, Information Week, and other technology trade publications, however, I have been struck over the last two years how the focus of editorial content has shifted to infrastructure upgrades and efficiencies rather than totally new systems. The “zeitgeist” is to do more and better with what’s in place today rather than buy new systems.

I think it’s safe to say the same is true in the legal market. The emphasis now is to integrate and adopt the systems that are in place. Most law firms have decent infrastructure. The challenge now is primarily to adopt new business mindsets, processes, and culture that take advantage of what’s in place; the challenge is no longer primarily one of technology acquisition.

My second observation was that there were some interesting developments at Legal Tech. E-discovery is a hot bed of activity. An increasing number of vendors offer software and services to harvest and process digital data in the discovery process. Though not necessarily visible to that many lawyers, there is a lot interesting happening behind the scenes.

More visible - and very impressive upon first viewing - is an e-discovery digital document harvesting and review systems from Attenex (a company affiliated with law firm Preston Gates). The most striking aspect of the Attenex offering is a new way to visualize large document collections. I have always been a fan of visual displays in law practice and the works of Edward Tufte on creating powerful graphic representations. Attenex has taken Tufte principles to heart in creating a compact visualization of digital documents that looks like a very promising way to identify and review large volumes of documents. I was impressed by the demo I saw.

Attenex also has an interesting product they call Knowledge Assembly Software, better known to many of us as document assembly software. The company’s literature shows a clean and intuitive interface to manage documents at a clause level. I ran out of time to see a live demo of this one.

Also new is LexisNexis Total Search, which is an automated knowldge management tool that integrates with a law firm’s document management system. The software integrates searches of LexisNexis with searches of the firm’s own work product. The video presentation and white papers explaining Total Search suggest that it is quite powerful and potentially very useful for law firms interested in better and faster access to and re-use of their work product.

Very few documents these days begin as paper; most are created on computers. In the discovery process of litigation therefore, dealing with digital data is of growing importance. As litigants increasingly seek computer files, issues concerning who bears the cost of the production have arisen. The rules governing discovery and cost-shifting have not changed, but the move from paper to digital files has changed the factual circumstances. The days where one side could point the other to a warehouse of boxes and allow the other side to look through them may be coming to a close. While one might be able to point an opposing party to your server farm and other digital storage sites and allow “looking through them,” doing so would be a very bad strategy. Especially when it comes to restoring back-up media or other relatively inaccessible data, some litigants have tried to shift the cost of recovery to the requesting party (typically the responding party pays).

Last week, an important case in the Southern District of New York further clarified the rules of cost-shifting in e-discovery. While this is a district court opinion only, lawyers with whom I have spoken tell me that the opinion is likely to be very influential. In Opinion and order re: Zubulake v. UBS Warburg, the court lays out a set of factors to consider in cost shifting. This decision, along with a prior ruling in the same matter lay out clearly the legal reasoning for when cost shifting is appropriate. The two decisions also include a useful discussion of different types of computer storage and point out that the media/systems issues can often be thought of in terms of what is accessible and what is (relatively) inaccessible. It is the accessibility that is important, not so much the media.

For those interested in e-discovery, these decisions are worth reading.

Most knowledge managers say that KM is 80% process/culture and 20% technology. I agree and usually focus on the 80%. One of the interesting 20% issues is the appropriate role for and expectations of full-text retrieval systems in KM. (This issue also applies to managing documents in discovery.)

Broadly speaking, search tools fall into two classes: simple and advanced. By simple, I mean software that allows Boolean and proximity searches, which means using “connectors” such as AND, OR, NOT, NEAR, WITHIN, etc. By advanced, I mean software that finds related words (and therefore documents that do not contain the search terms), distinguishes among related meanings of individual words, and applies advanced methods to rank how relevant documents are. The latter use pattern-matching techniques, neural networks, state space vector analysis, and other approaches.

Although I have worked with full-text software for over a dozen years, I have two lingering question:
1. What is the incremental value of sophisticated search over simple search and
2. How much upfront investment is required to get the sophisticated search to provide that incremental benefit. The “upfront investment” includes cost of software, set-up/integration, user training, and perhaps most important, the need (in some systems) to build taxonomies or provide training documents that are already categorized.

Answering this question, which in my opinion is an empirical, not theoretical matter, is expensive. Ideally, one would create test data sets containing large collections of documents, each of which was well known to a few individuals. Then you would run different search engines against each, letting the knowledgeable people “drive.†Ideally, a statistician would help set up the test and measure the results.

Some law firms have tested some advanced engines and they tell me that they have been under whelmed. And at a recent trade show, the rep for a fancy search said that his product usually does not work that much better than plain Boolean and his company no longer pushes the search feature and instead focuses on other features. Sobering.

All this having been said, I do believe that there is probably value in using sophisticated search tools. It depends on the nature of the collection and the level of training of the folks doing searches. Over a decade ago at Wilmer Cutler & Pickering, we developed one of the first integrated scan-OCR-full/text-structured/db systems. We found that, in the right hands, using a sophisticated search tool was better than a simple one. “In the right hands” was key though without knowledge and/or training, the advanced engine was not that useful. Given some recent reports show that most users of most search engines don’t do more than one or two word searches, it may be the power of advanced engines needs more support than we think.

What we need as a profession is a mechanism to perform real-world tests, both on how the search tools perform under the most favorable conditions and how they work when actual users operate them. Unfortunately, this is costly and the incentives and structures to do so just do not exist.

LexisNexis announced in a July 11, 2003 press release that it has acquired Applied Discovery, an e-discovery company.

I have always been struck by the seeming fragmentation of the e-discovery and litigation support market. My recollection is that a couple of bigger companies tried a roll-up (consolidation) in the mid- to late-1990s but that the market nonetheless remained fragmented. In the age of paper, litigation support was primarily about process and labor. As such, it is not clear that scale mattered all that much.

Now, with digital data, litigation support is becoming more focused around process and technology. Arguably, with technology playing a bigger role, scale matters more. A bigger company can invest more to develop software. And with this merger, there is the potential for synergies between both LexisNexis technology and content and e-discovery enhancements. That said, there are plenty of smaller players developing interesting technologies that are useful in e-discovery. Many an entrepreneur would say that scale works against technological innovation.

It will be interesting to see how the market evolves with a much bigger company now involved. From the law firm or law department as customer perspective, there seems to be little apparent downside. There is no lack of choice of vendors and this merger may create new offerings or spawn other transactions with the potential to create new competition on features or price.

Reading the June 23, 2003 issue of Information Week magazine was a trip down memory lane for me. An article titled Imaging Gets a Second Look reports on renewed interest in imaging. In the early 1990s when I worked at Wilmer, Cutler & Pickering, I was deeply involved in developing one of the first litigation support systems to integrate scanning, images, optical character recognition (OCR), full-text, and structured data. At the time, imaging was technically challenging but clearly the superior means of dealing with paper-based discovery in litigation. By the mid 90s, with the advent of the Web, interest in imaging among business-at-large waned.

Now, according to Information Week, the pressure to operate more efficiently is causing many businesses to take “a fresh look at some more mature, established technologies such as document capture and imaging software.”

Law firms too are looking again at scanning and image management. Ideally, this should not be necessary - everything should start and remain digital. But that is not realistic. So firms must still manage paper. In an effort to improve operations, some firms use scanning to share and distribute incoming documents (such as pleadings) as images or as part of a coherent records retention strategy.

In my opinion, the “file copy,” that is, the final retained version of client and client-related documents would be only digital - either files in native format or scanned images. Of course, this means having human processes to scan documents and to serve as gatekeepers that prevent each lawyer from sending redundant paper files for long-term storage (which not only costs money to store, but makes retrieving files more difficult).

A recently posted article in the Automated Lawyer area of law.com titled The Litigation Arms Race
describes the ever increasing costs of litigation.

The focus is on product liabilities contingency fee work, but the lessons may well apply across a wide range of commercial litigation. The article explains that modern practice requires hiring more and higher priced experts to establish the case. It goes on to explain that “[n]ew technology has also made discovery more costly.” Since 90% of documents are digital and only 30% are ever printed, litigators need tech experts, particularly data forensics experts, to help with e-discovery, especially with identifying “meta data” and dealing with backup tapes.

What the article does not make entirely clear, however, is that forensics expertise is not always required. Nonetheless, the costs of e-discovery can be very high because of the huge volume of digital data. The techniques to screen gigabytes, terabytes, or petabytes of computer files located across multiple computers are not cheap. And that says nothing about the cost of lawyer time to review the materials identified. Even where forensic expertise is not required and even where a law firm does apply appropriate technology and planning, digital discovery and the document review process is expensive. Fortunately, appropriate use of full-text retrieval and semantic analysis software can help control the costs. (See my posts from June 20th and 25th for more on e-discovery issues.)

The remainder of the article explains that “high-tech evidentiary displays” used at trial also add significantly to costs.

Picking up on the theme of my last posting concerning compliance and Sarbanes-Oxley, I’ve been struck over the last few weeks by the number of vendors offering “compliance solutions.” More interestingly, I wonder if this suggests a trend toward “preventive litigation support.”

One announcement that strikes me particularly is the launch of Aungate by Autonomy. Autonomy came to the attention of many law firm knowledge management, litigation support, and technology professionals for its sophisticated full-text searching capabilities. A June 16th press release by Autonomy announces the launch of Aungate, “a division of Autonomy offering the world’s first automated, solution for enterprise compliance and litigation support.”

Aungate offers real-time monitoring of employee communications, including e-mail, instant messaging, voice, and even SMS (cell phone short messages - bigger in overseas than in the US). One of the web pages shows some very interesting visual displays to help analyze communications.

The release also states that “Aungate provides an integrated solution to automate the analysis of these unstructured information sources in real time, enabling executives to spot trends in communications and easily pinpoint potential information misuse.” It is interesting that vendors - and perhaps their customers - are moving in the direction of “preventive litigation support,” that is systems that analyze vast volumes of data prior to litigation in an effort to prevent litigation in the first instance.

Yesterday both the New York Times and the Wall Street Journal reported that NASD will now require its members to save instant messages for at least three years (just as they must e-mail messages). The same articles mention that NYSE had imposed the same requirements on its members earlier in the week (which, if it was reported, I missed).

If saved IM records ever become subject to discovery, they will just add to the already large volume of digital data that lawyers must review. I suspect that e-discovery vendors are happy about this! I’m not sure that IM e-discovery raises unique issues. But if Wall Street IM usage is typical, abbreviations and shorthand is rampant. Moreover, since IM interactions may follow a phone call, e-mail exchange, or hallway encounter, they may reference events or things that, standing alone, are not all that clear. I suspect that therefore applying automatic analysis and screening techniques in e-discovery to IM will be even more challenging than it is to other digital data. There are specialized vendors that say they can help. For example, Cataphora says it can help put this type of information into context (meaning relating it to other documents and records) and H5 says it can help give meaning by pre-filtering based on issue definitions.

I was also surprised that both papers did not give this more press. The NYT devoted four column-inches in a corner tucked away on page C6; the WSJ about eight on page C4. I suppose that reflects my limited perspective on what’s important!