Strategic Legal Technology

Since not everyone is a Twitter fan, I reproduce here a selection of my recent Tweets. 

RT @jordan_law21 New at Law21: measuring lawyer productivity: http://tr.im/qjpD || also consider actual outcome v. expected / hoped for

RT @glambert ‘Alternative Fees - “How To” Tech’ - http://bit.ly/NFYQK - there’s Redwood, then … not much else? || good blog by @gnawledge

Altman Weil survey: signs of real change in legal market. Press release re new GC survey with link to summary PDF. http://bit.ly/nOCd4

Maybe GC don’t exercise market power b/c their law firms are job landing pads. Rees Morrison suggests a policy to fix at http://bit.ly/HjZLf

‘Pinsents – first firm to offshore work of qualified UK lawyers’ The Lawyer. http://bit.ly/5IXPE

RT @jordan_law21 New blog: the UK crucible: http://tr.im/p4lG || Great update on UK legal reform; links to recent reports

RT @PosseList Womble and Alt Fee Arrangements – Focus Report for ACC http://is.gd/16EYn || Wow - first time I see Monte Carlo sim in legal

More Sprint Nextel - RIM Blackberry 8830 issues: I’m techie and cannot get sync to work. What do non-tech users do?

@VMaryAbraham great KM blog re mixologists v bartenders. Applies to IT too? http://bit.ly/FTXrH

@RossMark RT Managing Partners on Outsourcing, Cost Control, Indian Liberalization, and Law Firms of the Future: http://tinyurl.com/ndtzjc

Alt fees = avoidance. To lower lawyering cost: accept some risk; decision tress, budgets, processes, metrics, best practices. What else?

Am I the only one who finds that the ABA website is chronically among the slowest sites to load?

Can we tell if BigLaw revenue decline is bigger than total drop in legal demand? That would be best sign of structural buying change by GC.

Jones Day tells staff to work harder http://bit.ly/TY3T0. imo: tell lawyers to get over their caste system mentality and learn team work

Anyone have experience w http://www.paperchace.com/ for litigation risk analysis w decision trees? Esp. curious to hear from TreeAge users

How is KM doing in the economic crisis? Are law firms taking advantage of other law firm departments to support KM ("baking KM into processes")? I am the author of a just-released ILTA survey that answers these and other questions. 

In June 2006, Chris Boyd, Senior Director of Professional Services at Wilson Sonsini, and I explored the idea of tapping into existing law firm processes in the June 2006 ILTA KM white paper. Three years and one economic crisis later, we thought it was timely to re-visit this topic. Rather than rely on anecdotal evidence of how law firms “power their KM windmills,” I thought that a survey would provide a better way to answer our fundamental question: Are law firms baking KM into their processes?

Today, ILTA released the survey results: Baking KM into Everyday Workflow: An Analysis of Knowledge Management Survey Data. The bad news is that law firms are not baking KM into processes as much as they could. The good news is that KM does not seem to be suffering unduly in these tough times.

As the author, I’m obviously not disinterested, but I think that anyone interested in legal KM will find the survey interesting reading. And I personally look forward to reading to the ILTA white paper released today, Knowledge Management: More than the Sum of Its Parts.

A recently released report by ValueNotes, a respected analyst firm, sheds light on what US and UK law firms think about legal outsourcing. 

Over at my employer’s blog, I co-authored a post, What Law Firms Think about Legal Outsourcing. It summarizes key findings of ValueNotes’ Legal Services Outsourcing: What Do Law Firms Think? and comments on them. Here is a summary of that post.

Many VN findings match the market assessment of Integreon as an LPO provider; there are some differences however from the supplier perspective…

LPO Penetration is Low. VN found that offshoring still has fairly low penetration among law firms; less than 3% of firms in a random sample had tried offshoring. Prior surveys and Integreon experience suggest it is much higher. VN surveyed lawyers, not firms, which may account for the lower finding.

Onshore Outsourcing is More Common. The total volume of outsourcing is higher if you take into account onshore outsourcing, which is more common than offshoring (especially in document review). In Integreon’s experience, only a small portion of the market is dogmatic about location; the vast majority let business requirements drive the location decision.

Cost Savings is the Main Driver. VN found that cost savings is the main driver but Integreon also sees that many customers, both law firms and law departments, also focus on satisfying client pressure and improving turnaround time. I know that other outsourcing providers share this view.

What Customers Seek in an LPO. VN found that customers of offshore services seek a provider with deep management and domain expertise, good references, end-to-end services, the ability to scale, and onshore/ global delivery capability. This is consistent with Integreon’s own experiences and what I hear from my peers in other LPOs.

Lack of Awareness is Biggest Reason Not to Offshore. The biggest reason law firms cite for not offshoring – 85% of firms – is lack of awareness of offshoring or no perceived need to do so. I was quite surprised since LPO has been around for 5 years and there’s been plenty of hype. Regular readers of this blog may recall I’ve been reporting on legal outsourcing since 2003!

Security Concerns. Firms cite security as a reason not to offshore. Firms can easily allay these concerns by assessing a provider’s facilities, security, and procedures. VN notes that firms with more extensive offshoring experience say that “client confidentiality and client conflict are not major concerns.” My own view is that the security I’ve seen at Integreon’s facilities exceeds that of any law firm’s I’ve seen.

Quality Concerns. Some firms that tried offshoring were not satisfied with the quality. These instances were likely ad hoc projects that were not properly planned or executed. A reputable LPO should be able to demonstrate understanding of the components of quality and a customer due diligence should reveal whether it’s real.

Document Review Dominates Offshore Work. For firms that do offshore, VN found that document review is the most popular function to send offshore. This is consistent with LPO industry experience.

Conclusions. It is certainly true that many lawyers are skeptical about both offshoring and outsourcing. We think this will change. My colleagues and I know that in the legal market, any new way of working takes years to penetrate. Firms take time to gain comfort with new ways of working; indeed, it takes time to use those ways effectively and achieve the desired quality. The legal market is slow to “tip” to a new way of working. But when it does, it tips quickly. We think in the current environment, the tipping point is upon us.

[Integreon has a free offer for the report - details at that blog post, What Law Firms Think about Legal Outsourcing. See where they agree and disagree with above observations.]

Update 26 June 2009 @ 830. The PosseList blog commented on above post. PosseList is a thoughtful and insightful blog focusing on contract lawyers in US and often comments on broader legal market trends.]

A new e-discovery study by Equivio statistically compares human versus software performance in designating responsive documents. The results are worth reading. 

I’ve argued that litigators and judges should rely on statistical analysis to determine the the most reliable and accurate approach to reviewing documents. In my March 2007 blog post, The Gold Standard for E-Discovery Document Review, I argued that lawyers’ belief in the accuracy of human review is likely misplaced. In that post, I described an empirical study H5 conducted.

The Equivio>Relevance ™ study is available by registration. It compares human review results to Equivio>Relevance ™ results and found that

“Out of the 4,107 documents in which the review analyses differed, the Oracle ["Topic Authority” in TREC lingo, aka “subject matter expert"] reviewed a statistically drawn sample of 190 documents, or slightly less than 5% of the documents in dispute. Of those 190 documents, the Oracle determined that Equivio>Relevance was correct in 147 of the cases. The human review team, by comparison, prevailed in only 43 of the disputed sample documents.”

In my view, that means the computer performed better than humans. Equivio is, however, rightly more cautious and draws this conclusion:

“computer-assisted review can dramatically increase the efficiency and accuracy of a document review team’s work”.

Instead, the study suggest litigators can use the tool for earlier and case understanding and more consistency, among other benefits. The study also notes that “Equivio>Relevance ultimately identified more than 1,000 additional responsive documents that had been overlooked or mis-categorized by the human review team.” (The H5 study written up in my ‘Gold Standard’ post also reported that software found more relevant docs than the humans.)

I think about the Equivio analysis by way of analogy. Let’s say the corpus of documents were a bunch of cells in your body. And let’s say the responsive docs represented cancerous cells. Further, the more cancerous cells you can find and segregate, the higher your chances of survival. If you, as the patient, had to choose the diagnostic test you wanted, which one would you chose? I’d go with the computer because it’s finding more cancer cells. And if the FDA were evaluating whether to approve human search or computer search, the statistics would strongly favor the latter.

Of course, we are lawyers and not doctors. And lawyers must appear before judges and defend their discovery methods. And we know judges don’t like “black box” approaches to discovery. That however, may have to change. We can learn from medical clinical trials and how the FDA approves new drugs or devices. It’s all about statistical outcomes (technically, the “mechanism of action” for a treatment does not have to be understood to obtain approval).

Anyone care to do the stats to compare smart keyword searching against concept searching?

Legal outsourcing has made mainstream and legal media news for years. After blogging on legal process outsourcing (LPO) articles regularly I stopped because there were so many. Two articles in the Times Online today, however, caught my eye. A major multi-national has gone public about legal process outsourcing to slash its legal spend by 20%. 

Rio Tinto’s legal switch puts pressure on London by Alex Spence reports on the fact. Rio Tinto deal heralds huge changes by well-known commentator and author Richard Susskind discusses the ramifications.

Spence reports that “Rio Tinto has hired a team of lawyers in India to try to reduce its annual £60 million legal bill by 20 per cent.” It is working with an LPO to recruit a team of 12 lawyers in India to “work for it on tasks such as reviewing documents and drafting contracts.” Rio expects to have 24 Indian lawyers within one year.

Susskind writes “the Rio Tinto deal suggests that imaginative pricing may not fully fix the more-for-less dilemma. Lawyers will need to go further and source their work differently, often by using less costly labour to do routine legal work…. People often assume that outsourcing and the options are applicable only to high-volume, low-value legal work. The Rio Tinto deal confirms this is wrong.”

For me, the Rio deal just adds to the list - literally - of corporations that offshore or outsource work. The list Outsourced Legal Services on this site shows 15 companies that offshore work to outsourcers or to their own offshore operations. So Rio is just one more company offshoring. Or is it? Specifically, this deal is announced with the intent of significant cost cutting and that feels new.

More generally, markets tip where the new and exotic become accepted and common. Geoffrey Moore’s Crossing the Chasm provides one framework for thinking about new approaches. Well before his work, however, you could see and model adoption of new high tech products (e.g., Polaroid cameras or Xerox-brand copiers) with S-shaped curves: slow ramp up followed by sudden acceleration.

The continuous (smooth) S-curve may not be a good fit for legal. Instead, a discontinuous step-shaped function may apply. Consider adoption in the legal market of e-mail, document management, marketing, lateral moves, or mergers. For each, there seemed to be only a few firms doing it and then, quite suddenly, many or all were. The “step function” reflects lawyer decision making: the first few adopters change slowly, gingerly, and quietly. Everyone wants to follow so once you have a dozen adopters, “the coast is clear” and the rest rush in.

Unfortunately, like calling the bottom of a recession (or the top of the market), it’s much easier to recognize the tipping point after the fact. Working at an LPO my view may be distorted, but it feels like legal outsourcing is at a tipping point. Of course, we won’t know for some time. Whether it is this year or beyond though, I am confident that, as with e-mail, marketing, etc., we will look at outsourcing and offshoring and try to remember what all the fuss was about.

I came across an interesting technology use survey sponsored by LexisNexis. It has a lot of data on tech usage and attitudes and compares legal and other professionals and three generation of users. It’s worth a look though it’s a bit overwhelming in scope and I am not sure it is actionable for IT managers and CIOs. 

The LexisNexis Technology Survey Gap [PDF]. LN conducted the survey (via a third party) in summer 2008. My take-away from the copious data and analysis:

• Almost all professionals now rely on technology. [What’s up with the few who don’t?]
• Gen Y [born 1980+] legal professionals spend more time on applications than Gen X or Boomers but may be less productive as a result
• I found page 23, hours per day spent using various devices, the most fascinating. All respondents report spending time using a fax machine, typewriter, and pager, for an average total of 2+ hours / day. Excuse me? I don’t see how this can be reliable. Even when fax was important, legal professionals did not spend much time using a fax machine (maybe reading faxes). I can’t think of anyone I know who owns a pager. I don’t think there are enough typewriters available to support the implied usage.
• CIOs who want to control non-work applications may find some good ammo for their arguments

As I read the survey, it does not address a long-standing question I have: are Gen Y legal professionals more tech savvy than older generations? Many lawyers and tech managers assert this is true. And this survey shows they use more applications. But that still leaves open the question of whether, when it comes to practicing law more effectively with technology, newly minted lawyers are better than older ones.

Have lawyers forgotten the goal of e-discovery? EDD articles and professionals seem exclusively focused on avoiding disaster. The real EDD goal is winning the case. 

In 1997, I characterized the goals of discovery as: [see note below for source]
A. Achieve the best outcome for the client
B. Minimize cost
C. Learn the facts of the case
D. Prepare offense or defense
E. Fulfill obligations to produce documents

I don’t think that has changed. Producing documents is a requirement of discovery; the goal, however, is learning case facts, telling the best story and winning. We hear countless stories of EDD gone awry. Where are the stories about how a team found the best documents, put together the best chronology and witness binders, and did the best job marshaling the facts?

Winning a case by “doing great EDD” should be newsworthy – but can you cite a single article in the last five years on this topic? Are litigation teams so paranoid about blowing e-discovery that case prep has suffered? If yes, that’s bad. If no, let’s hear the success stories, not just the disasters.

In an earlier discovery era, some were more concerned about winning. I remember the transition from paper + objectively coded databases to scanning + OCR + full-text + coded data.

Sure, we were careful to avoid pitfalls. For example, in serial litigation, inconsistent privilege designations across cases was a big problem. Corporate defendants sometimes did not turn over relevant documents in one state when they had in another. Why? Reviewers did not properly identify duplicates so a document could be in the database multiple times with different privilege designations. We mitigated this problem by using “document footprints” to de-dup.

The real excitement, however, was better case prep and winning. For example, we asked the lead partner to write his “dream smoking gun document”. The paragraph he drafted was literally copied and pasted into the search field of Personal Library Software (PLS was one of the earliest concept search tools). The search surfaced many useful documents – many that we never would have found using the then-standard coded database search and paper pulls or even Boolean searches on a full-text database. The partner estimated that finding the documents so quickly added $250,000 to the settlement value.

Then there was a time a partner was eager to find a document he felt sure must be in the collection. Many a search failed to find it. Then he had an idea: search for the fax number of the person who likely wrote it. Sure enough, even though the fax number was in small type and fuzzy, it had OCR’d well and we found a cover sheet, which quickly led to the document he sought.

Lawyers might do EDD better if they saw as much upside as downside. If anyone has good stories about how they used modern EDD tools to find the best documents and tell the best stories, please comment or send me e-mail to ron at prismlegal dot com.

[You can read more about my experience with this earlier generation of technology in an outline I wrote for a PLI presentation available on this website at Lessons Learned in Litigation Support.]

Update (23 June 2009): EDD consultant Tom O’Connor responds to this post in What Is the Goal of E-Discovery?. He writes:

“I disagree. I believe that there are two major additions to the goals of discovery since the amendments to the FRCP and they are both significant new obligations. The first is establishing the preservation of electronic documents at a point before the litigation actually commenced, that is that a proper ESI litigation hold strategy was employed and the second is the duty to cooperate in order to better facilitate FRCP 1.”

I’ve long argued that lawyers need to develop and follow best practices. Two recent cases show what happens when lawyers allegedly skip steps. 

Failure to File Routine Financial Statements Can Mean Trouble for Law Firms, AmLaw Daily Blog 9 June 2009, describes how two AmLaw 100 firms face challenges for allegedly not taking simple steps concerning UCC filings.

As I read the blog post, the alleged oversights are not about some esoteric and seldom-seen point of law. Rather, they seem to be the types of steps one would always take in similar matters. A bit like the pilot lowering the landing gear before landing? I’ve argued previously that lawyers, like pilots, should have checklists. Perhaps if the lawyers here had operated from checklists, they would not face these challenges.

I’ve also argued in the past that lawyers resist checklists or other systematic approaches because they think of themselves as artists. That view is wrong. Good artists are VERY systematic. I remember a Chuck Close exhibit at the Hirschorn that illustrates how he creates his work - he does not leave things to chance. Indeed, creativity and being systematic are not at odds. Artists may break rules but they know it when they do. And that may be more than one can say about lawyers. So, what explains lawyers’ resistance to systematizing what they do?

Since not everyone is a Twitter fan, I reproduce here a selection of my recent Tweets.  

BT Group pilots eBay-like online auction for roster of law firms. Corp Counsel Mag http://bit.ly/Nzd×9. Auction resurgence or blip? 31 May

LegalWeek launches Linkedin group for in-house lawyers http://bit.ly/3Zn3eM. Contrast to Legal OnRamp. Who’ll ‘own’ law dept online space? 24 May

Firm Management Jobs for Lawyers http://bit.ly/VT5vi Maureen Reid law.com article. Also Clark Cordner and my views at http://bit.ly/3QdXL. 24 May

Grim prognosis for BigLaw / #amlaw100 by consulting guru Peter Zeughauser, interviewed by WSJ Law Blog. http://bit.ly/Rpg6v 17 May

Geek Tip: Use mouse scroll wheel, with cursor over Firefox tabs, to scroll through all open tabs. Great for when you have many open. 17 May

BigLaw pyramids become diamonds. law.com http://bit.ly/19J3Oh. Ward Bower: more contract lawyers, outsourcing, and offshoring. 12 May

Suit Claims Firm Overcharged for Online Legal Research. NLJ http://bit.ly/8ab5f. Can firms make Wexis profit center? With disclosure ok? 10 May

Geek Tip: Use OneNote for task management. Open separate instance for ready access to task list. Hyperlink to frequently used pages. 7 May

Law firm wants text/link update on a 2004 listing at prismlegal.com. No: date is clear and list was accurate then. Preserve historic record. 3 May

Rees Morrison: ACC Covenant with Counsel is not all that game-changing http://bit.ly/WrB0U. I was reluctant to be first to say that. 3 May

In my Fall 2003 article, A Marketplace Trial, published in the litigation supplement to the American Lawyer, I argued that one could apply financial techniques to value and hedge lawsuits. The market may have taken a big step in that direction. 

Investing in Lawsuits, for a Share of the Awards (New York Times, 3 June 2009) is a front page business section article about investment companies such as Juridica Investments, which invest in lawsuits.

According to Juridica’s website, it is

“a limited liability, closed-ended investment company registered in Guernsey*, that has raised £80 million and commenced trading on AIM, a market operated by the London Stock Exchange, on 21 December 2007…. The investment objective of the Company is to build a diversified portfolio of investments in claims and to provide Shareholders with an attractive level of dividends and capital growth through investing directly and indirectly in litigation and arbitration cases, claims and disputes.”

The NYT article notes that

“Companies often jump at the chance to have an investor help pay for litigation, and lawyers usually appreciate having money set aside to pay them…. The investing companies say that because they do not take control of the lawsuit from the company and lawyers waging it, their most important task is identifying cases likely to produce a substantial return.”

An investor in a lawsuit necessarily must have a way to value the underlying claim. According to its annual report, Juridica “made substantial progress in developing scalable underwriting systems and processes for the evaluation of claims and continue to refine these.” This is tantalizingly vague - it would be very interesting to know more about the techniques (litigation risk analysis with decision trees perhaps?)

Assuming this investment model continues to work, how far can we be from the idea I proposed of rating lawsuits? It seems a small leap from an assessment for purposes of investing to an assessment for a fee.

I wonder whether these investors look at law firm efficiency and the use of legal technology in its assessment.

Whether Juridica and other investors represent a niche or the beginning of structural change in the litigation market is hard to say. Certainly the introduction of third-party forces in health care caused huge shifts. Might litigation face the same fate with the involvement of smart investors?

My prior post asked Is Cost Control Really a General Counsel Priority? My answer was “no", based generally on what I read and specifically an analysis of the Best Legal Department articles in the current issue of Corporate Counsel magazine. Since not all readers click through to comments, I am sharing here comments by Pamela Woldow of Altman Weil and Steven Levy of Lexician and formerly of Microsoft. 

Pam Woldow wrote:

Ron: As a consultant who spends her time in small, medium and large corporate legal departments, I must say that I am inclined to agree that the cost savings tend to be the imperatives of the CEO and the CFO. In general GCs, with some notable exceptions, are still stuck in the 1992 mode of seeking discounted hourly rates and asking for budgets for litigation, which did not work then and still do not – if cost control is the goal. While GCs are talking about needing better cost control, many seem not to know how best to really accomplish the goal. Instead, there is a perception that beating up firms for greater discounts will deliver the goods or, in contrast, they are living on the hope and prayer that because “law is different” they can justify the legal spend at the end of the budget year. From my perspective, they really have been placed in an awkward position. If they take serious steps to achieve cost control, executive management could second guess them.

It is a tough issue that requires GCs to lay out new and different steps to control costs and build consensus with executive management to accomplish the goals and have the requisite support. Without that support, not many GCs are going to seek innovative ways to cut costs as it will be safer to stick to the tried and true.

Steven Levy wrote:

Ron, I echo what Pamela says. I’d add one thing – it’s unclear, at least to me, how much of the lack of cost-control focus stems from the way the articles are written. If the law departments haven’t figured out how to make fiscal issues “sexy” to the reporters and editors, then the story won’t be printed, whether or not it’s there in fact.

To some extent, there is also the Geoff Moore core-and-context argument made so effectively in the Legal space by Cisco GC Mark Chandler. Is cost control “core” to the law deptarment?

Both commentators have extensive experience working in or with corporate law departments. So, are there any voices who disagree? Who will stand up and not only say but present the evidence that cost control is really a top GC priority?

The gap between what general counsels say and do about controlling outside counsel cost continues. At least that’s my take reading the Corporate Counsel magazine’s Best Legal Department 2009 issue. 

Best Legal Department – 2009 has three feature-length stories about the 2009 winner (The Hartford) and two runners-up (Exelon and IBM).

In the Editor’s Note, Anthony Paonita writes

“All of our winning legal departments are under even more pressure than usual to cut costs, both internally and in their outside spending. Our finalists all said that they’re doing more in-house, and that they’re starting to make firms think differently when it comes to fees and staffing. We may be at just the start of a fundamental transformation in legal services.”

I don’t see how the articles support that conclusion. In fact, cost control discussion seems buried. Allow me to quantify: of approximately 7,200 words in the three feature articles, only 409 relate to cost control - less than 6%. (To count, I copied text to a word processor and used a permissive standard for text about cost control.)

If cost control were really tops on the agenda, I would think it would warrant a larger share of the discussion. Moreover, the cost measures described by and large are not inspiring.

The Hartford gets discounts “on all the work one law firm does for it by agreeing to allow star associates to handle some appeals.” Other measures: “In-house lawyers actively direct all facets of cases… and are responsible for ensuring that the company receives ‘the best possible representation for the least possible cost.” The Hartford also insists “that a firm use contract lawyers on document reviews.” Useful? Yes. Transformational? No.

The most intriguing cost control measure gets one sentence in the article on IBM, which “is in the process of creating a ‘back-office hub’ of recent law graduates who do lower-level work.”

I am in no way commenting on the winning departments. Rather, my comments illustrate only another example of a public proclamation of the importance of cost control that is not backed up by clearly demonstrated and quantified action.

Raise your hand if you’ve read the TREC 2008 Legal Track study released in March. TREC is a great project to help legal professionals understand the accuracy and process of document review. To improve doc review and gain the value of this on-going study, however, TREC organizers and the legal profession need more dialogue. 

Background and Articles about TREC Study. The TREC study “focuses on evaluation of search technology for discovery of electronically stored information in litigation and regulatory settings.” Two recent articles help explain the TREC study and its importance. TREC 2008 Stresses Human Element in EDD by Jason Krause at law.com (1 May 2009) does an excellent job explain the TREC study. Krause also authored another article about TREC, In Search of the Perfect Search in the April 2009 ABA Journal.

The Study is a Hard Read. After the release of Overview of the TREC 2008 Legal Track, it generated little immediate discussion in blogs and at Twitter. The limited coverage is not surprising; I found reading it a tough slog. At risk the risk of being immodest, I have a good background to understand it: lawyer, 20+ years experience with full-text retrieval, three years of college majors-sequence math, and four years of hands-on econometrics experience. So if I have trouble understanding the report, that does not augur well for other legal professionals. It’s a tough read for three reasons: (1) long, (2) academic, and (3) too much passive voice, which makes understanding who did what hard. So I was glad to see Kruse’s articles.

TREC “Interactive Task” Focused on Replicating Decisions of Experienced Litigator. The part of the study I found most interesting focuses on how best to automate or replicate responsiveness decisions an experienced litigator would make. This was the TREC Interactive Task in contrast to the Ad Hoc and Relevance Feedback tasks. The study “gold standard” was the document designation a “Topic Authority” would make. “Gold standard” is a term of art that means the best-established or best-accepted solution. “Topic Authority” means an experienced litigator who best understands the facts and issues of the case and who, time permitting, would personally review all the documents. Topic Authority seems a variant on the more generic “domain expert” or “subject matter expert”.

Potential Issues with “Gold Standards”. The TREC study assumed that the Topic Authority’s judgment is the gold standard. More on that in a minute. Contrast that to the apparent assumption many litigators make. In a March 2007 blog post, The Gold Standard for E-Discovery Document Review , I wrote

“Many lawyers appears honestly to believe that human review is accurate, the “gold standard” for document review. “Honestly held” and “right” can diverge. I, for one, have never seen data to support the commonly accepted “gold standard.”

My concern then was that too many lawyers seem ready to dismiss software designations of documents in favor of lawyer designations, even if the lawyer has no case-specific know-how. I might accept the Topic Authority’s judgment as the gold standard but I don’t accept any old lawyer’s judgment as such (especially not a contract lawyer new to case who might have had only an hour or two of training about the matter).

Even the Topic Authority as Gold Standard raises issues for me. The accuracy and reproducibility of Topic Authority review has not been established. That said, it is not clear what standard is better. On the one hand, this standard acknowledges we must rely on the subjective opinion of an expert. On the other hand, reasonable experts can and do disagree. In contrast to doctors conducting clinical trials, lawyers cannot establish an objective and reliably reproducible standard. This conundrum is perhaps best summed up by the TREC Topic Authority, Maura Grossman, quoted in Krause’s law.com article:

“The decision process that determines what is responsive, and therefore what must be produced, is inherently a task involving human judgment. The concept of responsiveness in e-discovery is not an objective standard, it is ultimately a judgment found in the senior attorney’s head.”

TREC Compared How Closely Different Approaches Came to Replicating Human Judgment. The study examined how accurately different teams – where a team could be human or software or both – came to replicating the Topic Authority’s judgment. Each team had an opportunity to interact with the Topic Authority to refine or tune its approach. After all teams finished the review, researchers compared doc designations and, where results differed, set up an “appeal process” to decide on the correct designation.

I think it would be helpful to consider explicitly how Topic Authority judgment variability might influence thinking about e-discovery issues. We have no data on whether similarly situated Topic Authorities would agree on most document designations. (My doctor friends tell me that in clinical trials, this is the inter-rater variability problem). For example, suppose a future study employs two Topic Authorities per topic and finds a 20% variance in doc designation between them? We need parameters around the variance among Topic Authorities otherwise comparison to software approaches seem suspect. More specifically, if software approach A compares well to Topic Authority A and software approach B compares well to Topic Authority B but A and B vary by 20%, what does that mean? If the variance were under 5%, I think everyone would agree “so what.” If the variance exceeded 25%, I think many would find that troubling.

Conclusion: Need More Dialogue to Make TREC Useful to Legal Professionals. The potential variation among Topic Authorities notwithstanding, TREC offers what may be the best possible approach to evaluating the reliability of alternate approaches to document designation. The study as it stands, meaning a rather academic work, may not provide enough guidance to real-world lawyers, e-discovery professionals, and perhaps most importantly, judges. That said, I think that TREC could be instrumental to the future of EDD. Having a disinterested but highly knowledgeable third party evaluate alternative approaches based on scientific principles (meaning based on valid statistics and reproducible) is the only way to establish reliable standards.

To achieve traction in the real world of EDD, the profession needs to discuss the TREC theory and findings more widely. Organizations such as EDRM and the Sedona Group, which are already deep into e-discovery, have the potential to help digest, explain, and disseminate the findings in a way that make the work more relevant to everyday practitioners. A dialogue with practitioners would not only inform litigators and EDD professionals how best to use study results but might also influence study design to make it more useful to the legal profession.

[Note: I want to express my appreciation to Nicolas Economou, Sandra Song, and Dan Brassil of H5, who helped me parse out some of the finer points of the study.]

Signs of Innovative Life in the Practice of Law in the April issue of Law Practice Management magazine offers multiple authors’ ideas on innovative ideas for law firms. Here, in slightly edited forms, are the two two items I contributed, one on legal outsourcing, the other on working virtually.  

RETHINK ON-SITE STAFFING

Think about the vast “Middle Office” of BigLaw – all the work required to run a law firm that is neither law practice nor entirely routine back-office support (e.g., copy center, payroll, or plant watering). This includes secretarial support, IT, marketing, finance and accounting, HR support, and business research.

I’ve seldom heard of firms consciously deciding (1) the level of support they should provide and (2) how best to provide that support. Headcounts by function seem artifacts of history and management idiosyncrasy. For example, I know of firms where the lawyer to secretary ratio ranges from about 2:1 to 6:1 and IT spending as a percent of revenue from 3.5% to 7%. It’s hard to explain these big variations, especially in otherwise similar firms.

I fear that the tough economic times will compound the irrationality. Your firm may have laid-off lawyers and staff. Has it used the crisis as an opportunity to re-think and rationalize support? Probably not! Why not improve long-term performance while you reduce costs short term?

Consider what a few innovative firms did even before this crisis. These leaders consciously decided to re-tool lawyer support. Orrick opened a global operations center in low-cost West Virginia in 2003. Some British firms moved back-office operations to India around 2005. Top 30 UK firm Osborne Clarke outsourced much of its Middle Office to legal outsourcing company Integreon early in 2009 (See the OC press release).

Now that we are in a crisis, your firm should think hard about the support your lawyers need and how best to provide it. If your firm operates multiple offices, you cannot win the argument that support staff must be in the same building as lawyers. Once free of the “same building” shackles, think creatively about support. Perhaps it makes sense to centralize some functions in one office. Or perhaps you can rid yourself of the headache of owning and operating large teams and let an outsourcing company do it for you.

If you not yet analyzed what support to provide lawyers and how best to do so, now is the time to act. You may find, centralizing, offshoring, or outsourcing provide the support you need at lower cost.

OPTIMIZING VIRTUAL COLLABORATION

Five years ago I wrote an article in Law Practice Management on “working virtually.” Some firms, typically smaller ones, are finally beginning to adopt this model. Will their BigLaw brethren be smart enough to learn the same lessons?

Most lawyers believe the myth that they must work in close proximity to collaborate and sustain their culture. If you tell clients that your firm assembles the best team across all your offices, how can you argue that lawyers must show up in downtown offices? Their colleagues may well be in other cities. As for culture, notwithstanding Woody Allen’s remark that 70% of life is just showing up, simply being in the same place is neither necessary nor sufficient.

I’m not saying do away with downtown, central offices. Instead, firms can offer the option to work at home or in a low-cost suburban satellite office part of the week. This reduces both lost commuting time and the carbon load. Have lawyers come downtown when they actually need collaborate in person. Wow, what a concept: scheduling time for real collaborate instead of pretending it just happens every day. Firms that do so likely will find that sharing offices downtown becomes viable, which can dramatically lower occupancy cost.

Of course, in the current crisis, firms are shrinking, not growing. They are retaining too many lawyers, not losing enough. So the need for space or to accommodate demands for work-life balance may seem remote. Yet now is exactly the right time to make difficult changes. A firm that set out now to optimize how and where its lawyer work and says what it is doing publicly signals clients and recruits that it’s in business for the long term. Most importantly, it would be better-positioned for the inevitable economic turnaround.

Think you have nothing to worry about by ignoring this? Some large firms do allow this flexibility already. And some start-ups are built on the idea of working virtually, for example, Virtual Law Partner, LLP in the US and Optim Legal in Australia.

A recent law suit contends that a law firm’s online research client charge-backs are unlawful. This hints at the bigger issue is what should be overhead and what should be a charged to clients. 

Carolyn Elefant writes in Law Firm Markup of Research Costs: Annoying or Unlawful? at Legal Blog Watch about a plaintiff who “claims that that Chadbourne [wrongfully] charged him $20,000 for computerized legal research services that actually cost the firm only $5,000.” She cites an NLJ article and links to and discusses several blog posts commenting on the article.

I think this suit and her post raise two questions:
(1) What should law firms bill back to clients and
(2) If a service is billed back, how should the rate be set.

My answer to the first: firms charge for whatever they can easily meter. Legal technology - online research, fax, phone, copiers - all come with built-in metering. Books, the library itself, space to store client documents, and circulating periodicals do not. Paralegal work is easily metered; secretarial work is not. (Firms may have missed charging for secretarial time working on documents, which the DMS can meter.)

The second question is harder to answer because it turns on how one interprets the Canons of Ethics. In the past, clients voiced discontent about fax and copying as profit centers though I don’t know if that ever translated into a lawsuit.

Three interpretations of this situation come to mind. Law firms
(1) work hard to allocate costs fairly to clients
(2) are randomly managed and lack a coherent vision of their core business, or
(3) are opportunistic and mercenary.
Am I missing potential explanations?

ILTA’s knowledge management peer group hosts a webinar on June 8th called “Where Is KM Now and Where Is It Going?" 

The topic will cover the current state of KM and its possible future direction in the context of the global economic crisis. This is a basic to moderate webinar for Knowledge Managers, Practice Support Attorneys, and IT Directors. Where Is KM Now and Where Is It Going? registration page.

Speakers:
- Stuart Kay, Director, Global Business Systems, for Baker & McKenzie
- John Gillies, Director of Practice Support at Cassels Brock & Blackwell
- Wendy Small, Head of Knowledge Management, Eversheds LLP.

Are large law firms reeling with the rest of the economy? Reading legal headlines since January 1, you would think so. But maybe not. 

In Survey: Law Firms Don’t Expect to Make Radical Changes the AmLaw Daily reports on a new Altman Weil law firm survey (conducted March and April). [The preceding link is to a fee-based webinar about the survey.] The surprise finding, quoting AmLaw Daily:

“Law firms are not doing anything dramatic and are not planning to do anything dramatic [in response to the economic downturn],” says Eric Seeger, an Altman Weil consultant and co-author of the survey, Law Firms in Transition.

First a quibble, in law firm land, I would say the dissolutions of 2008 and lay-offs of 2009 are not just dramatic but earth-shattering. OK, so much for semantics. Is it possible nothing will change, that the economy is not so bad, that BigLaw can continue its bad ways? A recently released BTI survey finds that legal spend in 2009 will drop only 1.4% for all of 2009. Hmmm.

Here’s one take: BigLaw is playing musical chairs. Cut costs short term but hope for the best long term. “My firm will be ok, others will hurt.” If, in fact, BTI is right and legal spending is only down modestly, then it seems likely with a bit of market- and wallet-share shifts, many firms will do just fine. The question is, however, which firms will be left without a chair when the music stops.

Here’s another take: Law firm management is in total denial. A year from now, we’ll know.

Many legal publications and consultants offer advice for large law firms on how to stabilize and recover. Their suggestions focus on lawyer compensation, practice group strategy, alternative billing, and business development. I remain surprised at how little they focus on reducing law firm overhead. 

Last week, the Zeughauser Group (ZG), a leading law firm management consultancy, offered its take. Ashby Jones, in the Wall Street Journal Law Blog, summarizes a recent ZG alert (5 May 09). The ZG alert, Forward-Looking Strategies: Tough Medicine and Elixirs for Success (link to WSJ - I could not find the PDF at ZG site), has excellent advice for law firms. It suggests:
- Cut lawyer costs
- Own particular niche practices
- Take advantage of the upcoming re-structuring of financial institutions
- Expand in selected rapidly developing countries
- Offer alternative fees
- Enhance profits through matter management (law firm business intelligence in my words)

Like many similar pieces, the ZG discussion does not shine a light on BigLaw overhead. Am I mis-guided to focus on overhead? Is it big enough that management should worry? To answer that, we have to quantify overhead. Here’s a simple way to determine it using AmLaw 200 data:

• Start with gross revenue.
• Subtract partner profits (all partners, not just equity).
• Subtract associate compensation (I assume a blended $225,000/year and multiplied by associate headcount).
• The remainder is all other overhead - everything from IT and marketing to occupancy and secretaries.
• Lawyers require far more support than staff; I assume twice as much. So I multiplied the total overhead by 66.6% to determine overhead supporting lawyers.
• To compare across firms, I divided this overhead by total lawyers.

In 2007, 0verhead per lawyer ranged from over $500,000 per lawyer to just over $50,000 per lawyer. The median is about $170,000 per lawyer and the average is $185,000. Even before the crash, I would have said that’s real money.

I can explain only some of the variance. For example, NYC bulge bracket firms by and large top the list. But many variances are hard to explain. If firms with overhead per lawyer above the median reduced their overhead to the median, profits per equity partner could, in some firms, increase by more than 30%.

While the math is simple, cutting overhead is not. Adopting a rational and analytic approach to supporting lawyers could, however, for many firms, reduce overhead. Perhaps quite significantly. Ideas I’ve offered include doing more with technology, forming secretarial teams, and outsourcing middle office functions.

As firms implement the excellent strategy suggestions of ZG and others, they must not forget to shrink their often bloated overhead. The old adage that it’s easier to grow revenue than shrink costs is still true. So too is the one about not leaving money on the table.

[Side-note: I was pleased to see ZG reference outsourcing: “Non-equity partnership … are less flexible than outsourcing."]

Through the 1980s, marketing was a four-letter word in most large law firms. By the 1990s, BigLaw had build marketing teams. One survey a couple of years indicated firms were spending 2% of revenue on marketing. Where is marketing today and how has it fared in the downturn? 

I found a great answer to these questions in an e-mail I received from my friend Steve Nelson of The McCormick Group, an executive search firm with a big legal practice. Steve is an astute observer of the legal market. He was previously a practicing lawyer and editor of the Legal Times. With his permission, I reproduce here an e-mail report that he sent in April titled “TMG’s Take…On CMO Vacancies”.

“Large law firms don’t appear to be in a hurry to fill vacant Chief Marketing Officer positions. Our unofficial scorecard has 17 CMO positions at AmLaw 200 firms that have remain unfilled since the first of the year or even longer. Some searches have been put on hold, while other firms have made the ultimate decision (at least for now) to go without a CMO at all.

Given that it’s a truism that business development should be the last area for budget-slashing in bad times, this says a lot about the state of marketing and business development in law firms today. About five years ago, some firms began to hire honest-to-goodness “sales” people to spearhead their business development efforts. But that trend never really took flight. Sure, law firms hired scores of so-called business development professionals, but in reality, they turned into business development support professionals, behind-the-scenes operatives who increasingly ended up handling RFPs, developing seminar ideas, and the like. At the same time, firms began to focus their CMO searches towards those candidates who had a proven record of building a marketing infrastructure.

As a result, in today’s economic climate, it’s quite easy for firms to see a quick and easy way to cut a half million dollars or so in personnel costs by either firing their CMOs, or failing to replace those who had left for greener pastures. In many cases, those professionals accomplished the goal of building the team and the process, but never got to use any of their strategic or visionary skills. Moreover, given the fact that law firms rarely want to have “nonlawyers” or even “non-practicing lawyers” on the front lines of client team and new business development initiatives, the nexus between these C-level executives and revenue generation became tenuous at best.

At this point, most law firms’ reaction to the economic crisis has been to focus on cutting costs, rather than building revenues. Normally, such a cataclysmic change might lead some to take bold steps to increase market share, but we’re not seeing it yet.”

Comments form any law firm marketing professionals?

Incisive Legal Intelligence (formerly ALM Research) recently released a legal outsourcing study. I thought The Law Deparment Legal Outsourcing Study, 2008 would quantify the rapid growth of legal process outsourcing (LPO). 

In fact, it focuses exclusively on law departments outsourcing legal work to law firms. Incisive found that general counsels (GC) outsourced 40% of legal spend in 2008, down from 46% in 2007. The study does not quantify other law department outsourcing such as e-discovery.

With GCs outsourcing almost half of legal work, why do some seem nervous about legal process outsourcing? Many lawyers conflate “outsource” with “offshore”. Offshore is a fine option but several reputable LPOs offer onshore service as well. Perhaps GC misgivings are more about the type of resource than the location. Other than inertia, it’s hard to understand GC reluctance to expand the already broad scope of law department outsourcing by deploying more LPO services.

Law departments can reduce costs by explicitly acting as general contractors to solve company legal problems. Like any GC (general contractor that is, not general counsel), a law department should consider what resources it employs full time and what it sub-contracts. With this mindset, law departments would select a broader range of resources that better meet corporate objectives. As I explained in my post The Right Resources to Solve Legal Problems (May 2008), law departments should think about the complexity and volume of work in a 2-by-2 grid and choose the most appropriate resource.

Delegating resource selection decisions to law firms means missing out on cost savings. Aside from the usual economic conflicts inherent in an hourly billing and cost pass-through system, GCs likely have more bargaining power than their firms. Law department management consultant Rees Morrison notes in Three reasons why legal departments are better positioned to negotiate arrangements with suppliers (15 April 2009) that
(1) GC have more bargaining power than law firms,
(2) GC are more motivated than firms to save money, and
(3) GC can provide better direction to sub-contractors than can firms.
Morrison’s blog post further supports the idea of GC = “general contractor” as well as “general counsel.”

A general contractor approach to meeting corporate legal needs is one of the easiest and least painful way to get more bang for the buck.

[This post adapted from a similar entry I wrote for the Integreon KPO and LPO blog.]