The Truth About Processing Speed

Vendors like to brag about being "fast" at processing in electronic discovery.  However, when you look behind the speed claims, it is clear that the devil is in the details.  

Most vendors tell you how fast they can ingest/import documents for processing, but fail to mention such things as how long it takes to export documents for analysis and review and all the meta data that may be left behind. Here are some questions to ask when the need for speed is critical to your production.

1. Does the vendor extract and preserve [all] meta data during import?  Some leave important meta data behind in order not to sacrifice ingestion speed.

When a text layer (Images, vector graphic files, etc.) is not found during import, is the file OCR’d during import to extract text that would otherwise be left behind?

Does the vendor de-dupe via a distributed process or do they use a single machine and require the entire data set to be loaded into one window for processing?  

Does the vendor parallel process the data across several processors and machines?  Just because a vendor says that the application is multi-threaded, does not mean it’s scalable. 

Often a vendor completes processing during export to save time during import. For example, linking parents/children/families, tiffing and de-dupping is often done during export to save time and possibly money on import. 

Does speed really matter in your case or can you wait an extra day?

Why does it matter whether or not you tiff, DeNIST, de-dup, etc., during import?  The answer depends on what you intend to do with the data. For example, you want to de-dupe on import if you intend to make the data immediately available for early case assessment. Additionally, vendors typically don’t tiff until the very end when the data has been culled down to a subset deemed relevant.  However, most ECA tools that tiff during export, will move the data from processing, to ECA, and then back into a processing tool to get decent tiffs on extract. This adds time and cost to processing.  Personally, I like a vendor (e.g.idiscover Global, Inc. ) that does a thorough job importing ESI for processing and ECA.  

Clearly, processing is a highly technical dance with legal implications. For that reason, it is always a good idea to use a consultant who can navigate these important questions and make the right decision depending on the case.