TREC Legal Track 2010/2011 Updates
New data from the TREC Legal Track e-discovery experiment is coming soon, director Gordon Cormack said.
"The overview for TREC 2010 will be published shortly. (It is late.) The overview for TREC 2011 will be published in Feb. 2012," he wrote in an email response to Law Technology News' questions today.
However, "There won't really be a direct comparison between 2010 and 2011 because the task and evaluation methods are slightly different," added Cormack, who is also a University of Waterlook computer science professor.
| Topic | Best |
Median |
Worst |
| 401 |
58.8% |
28.0% |
8.8% |
| 402 |
58.8% |
13.1% |
2.3% |
| 403 |
72.0% |
14.2% |
3.1% |
"The measure reported there is 'hypothetical F1' which is the F1 measure that the groups would've achieved had they set their cutoff just right," Cormack added. "The best/median/worst are among the 60 or so submissions, interim and final, in all categories."
For more information, readers should stay tuned to this blog and the TLT's mailing list (the latter requiring registration.)
Comments