Information retrieval (IR) research on text collections has concentrated on improving the effectiveness of the indexing and retrieval operations. For the most part, the evaluation of IR systems has been carried out on relatively small collections of documents, queries and relevance assessments. In 1992, the first of a series of evaluation exercises called TREC (Text REtrieval Conference) was launched in the USA and these exercises have continued annually since then. What makes TREC notable in IR research is that the document collections used are huge, compared to previous ones, and the groups participating in this collaborative evaluation represent a ‘who is who’ of IR research across the world. In this paper, we present an overview of TREC, the way it operates and the specialist ‘tracks’ it supports. We then concentrate on European involvement in TREC, examining the participants and the emergence of European TREC-like exercises.
Get full access to this article
View all access options for this article.
References
1.
[1] Institute for Scientific and Technical Information , Project Amaryllis ( INIST-CNRS, Meudon, France , 1996).
2.
[2] W.B. Croft , What do people want from information retrieval? (The top 10 research issues for companies that use and sell IR systems), D-lib Magazine (The Magazine of Digital Library Research) (November 1995). Available from: http://ukoln.bath.ac.uk.dlib
3.
[3] European Commission DG XIII , The Information Engineering Sector (1996). Available from: http://www.echo.lu/ie/en/iehome.html
4.
[4] W. Kraaj and R. Pohlmann , Viewing stemming as recall enhancement . In: Proceedings of the 19th Annual ACMSIGIR Conference on Research and Development in Information Retrieval (SIGIR 96), Zurich, 1996, pp. 40-48 .
5.
[5] D.K. Harman (ed.), The Third Text Retrieval Conference: TREC-3 (NIST Special Publication 500-225, 1995). Available at: http://potomac.ncsl.nist.gov:80/TREC/
6.
[6] D.K. Harman (ed.), The Fourth Text Retrieval Conference: TREC-4 (NIST Special Publication 500-236, 1996). Available at: http://potomac.ncsl.nist.gov:80/TREC/
7.
[7] K. Sparck Jones , Reflection on TREC , Information Processing and Management31(3) (1995) 291-314 .
8.
[8] C.J. van Rijsbergen , Information Retrieval2nd ed. ( Butterworths , 1979).
