Legal Technology News - E-Discovery and Compliance Blog

« Discovery Approved for Computer in Divorce Proceedings | Main | Choose your words carefully »

August 08, 2008

The Socha-Gelbmann Electronic Discovery Survey: Time for a Change

Sochag_2 The time has come for us, the creators of the Socha-Gelbmann Electronic Discovery Survey provider rankings, to kill our rankings. We intend to replace those rankings with an improved way to assess providers and their capabilities.

Why are we killing the rankings? We believe our survey rankings have reached the point where they no longer serve their original purpose. When they are announced, we are told, they can affect the share prices of publically held companies. They can have an impact on the ability of providers to obtain financing. They can be a key factor, sometimes the most important factor, in determining which provider is selected to take on a project or deliver a software program.

No set of generalized rankings should have this type of influence. Not ours. Not anyone’s.

A Bit of History

When we issued our first set of rankings six years ago, we felt they helped guide consumers entering an inchoate market. We offered the rankings as indicators of which providers appeared to be rising to the top. In the first year, as some may recall, we published a single set of rankings — a list of the top five electronic discovery providers.

Over time, we expanded the scope of the rankings. With the third survey, we extended the overall rankings from the top five to the top 20, divided into groupings of 1-5, 6-10 and 11-20.

The following year, we expanded the rankings even more. We drew distinctions between services and software providers. We offered service provider rankings based on assessments of their experience, their capacity, and the views of law firms and the corporations from whom we gathered data. We added software provider rankings based on law firm and corporate responses as well. We also added rankings for services and software providers by the Electronic Discovery Research Model stage.

This year we added a set of software provider rankings that looked at the usage of those providers’ software.

We always have sought to portray the service and software providers as objectively and as fairly as possible. This year, for example, we examined more than 350 separate categories of information. For each category, we gathered together all that data we had on that topic. We quantified the data, converting it from text to numbers where necessary. We developed something akin to a distribution curve and from that prepared a rating scale.

We recognize that perfection is never achievable, which is why we have always presented the rankings with a cautionary warning. Last year, we cautioned that consumers “should not choose a provider based solely on these lists, or solely on the basis of any other top-X list.”  This year, we stressed that “anyone who makes buying decisions primarily on these rankings is a fool.”

Time for a Change

As noted above, we are told that the rankings have become so influential that now they affect share prices and the ability to get funding. We are hearing, as well, that consumers are using them as a substitute for the work they should be doing themselves — analyzing consumer needs and assessing what and whose services and software might meet those needs.

As we stated above, this is wrong. So much for the effectiveness of warnings.

Where Do We Go From Here?

For the next round of our survey, we intend to discontinue the rankings in their current form. We hope to replace them with a system that will more effectively allow potential consumers of electronic discovery services and software match up what they perceive to be their needs with what the providers have to offer.

As we have done with previous surveys, we will work on improving our survey to deliver the most value to our subscribers. We will retool how we present providers and their capabilities. We will continue to publish a high level view of key industry trends, market size and provider and their capabilities in Law Technology News.

We welcome your suggestions — let us know what would be more useful than the rankings. You can send your thoughts to us at [email protected].

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d8345280a669e200e553f2a0d98834

Listed below are links to weblogs that reference The Socha-Gelbmann Electronic Discovery Survey: Time for a Change:

Comments

Oh sure. George and Tom wait until I'm teetering on the very precipice of being named to all ten top spots on their rankings and then pull the rug out from under me. Thanks a lot, guys.

On a more serious note, they've decided and announced the rankings would be no more, but we might as well recognize that there will always be people who makes lists and those who strive to be on them. The vacuum they leave will be filled, and perhaps by those less independent, careful, honest and clear-eyed as George Socha and Tom Gelbmann. It takes enormous character to create something so successful and then leave it behind rather than see it abused. Thanks, gentlemen, you are an inspiration.

I agree. I think george and Tom should
recognized for their hard work over the past 5 years in developing and delivering a work that has received so much acclaim and attention.

They really paved the way for people to think about E-Discovery as an industry, rather than as a mixed collection of start-ups, splinter groups, and mavericks.

I'm not sure what their next product will look like, but given their work on the prior surveys, the EDRM model, and their contacts in the industry, I'm sure it will be attention grabbing and thought provoking.

The comments to this entry are closed.

Sign Up for the E-Discovery and Compliance Newsletter

An Affiliate of the Law.com Network

From the Law.com Newswire

Sign up to receive Legal Blog Watch by email
View a Sample



Contact EDD Update


Subscribe to this blog's feed



RSS Feed: LTN Podcast

Monica Bay's Law Technology Now Podcasts are also available as an RSS feed.

Go to RSS Subscribe page




March 2013

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31            

Blog Directory - Blogged