Theory and explanation in information retrieval research

Abstract

The relationship between theory and explanation in informa tion retrieval research is analysed. Problems in the development of a generalisable information retrieval theory from informa tion retrieval experiment and the associated mathematical mod elling is examined. The source of these problems is traced back to assumptions underlying information retrieval research, and to the nature of the relationship between the theoretical frame work of information retrieval research and the nature of the problems on which this framework is brought to bear. Particu lar attention is directed to problems in measurement, in estab lishing a significant relationship between test results and results from operational evaluations, and in establishing a connection between the situational and behavioural assumptions employed in information retrieval research and real situations and be haviour.

The attempt to develop information retrieval research as an experimental science akin to natural science or engineering, around the notion of the quantification of relevance is miscon ceived. In order for information retrieval research to break out of the limitations of its present theoretical framework, and of the backwater to which that framework has led, a reorientation is required. The argument of this paper is that information retrieval research needs to be reoriented from a focus on the information retrieval system and on supposed 'systemic' factors to an orientation to the user and the user's interaction with information sources.

Get full access to this article

View all access options for this article.

References

N.J. Belkin , Progress in documentation—information concepts for information science, Journal of Documentation 34 (1978) 55-85.

N.J. Belkin , Ineffable concepts in information retrieval, in: K. Sparck Jones , Ed., Information Retrieval Experiment (Butterworth, London, 1981) 44-58.

B.C. Brookes , Information technology and the science of information , in: R.N. Oddy , S.E. Robertson , C.J. van Rijsbergen and P.W. Williams , Eds., Information Retrieval Research (Butterworth, London, 1981) 1-8.

C.W. Cleverdon , Design and evaluation of information systems , Annual Review of information Science and Technology 6 (1971) 41-73.

C.W. Cleverdon , On the inverse relationship of recall and precision , Journal of Documentation 28 (1978) 195-201.

C.W. Cleverdon , J. Mills and E.M. Keen , Factors determining the performance of indexing systems, College of Aeronautics, Cranfield (1966).

C.A. Cuadra , On the utility of the relevance concept, Report No. SP-1595, Systems Development Corporation, Santa Monica, CA (1964).

C.A. Cuadra , R.V. Katter , E.H. Holmes and E.M. Wallace , Experimental studies of relevance judgments: final report, volume 1, Report no. TM-3520/001/00, Systems Development Corporation, Santa Monica, CA (1967 ).

P. Diesing , Patterns of Discovery in the Social Sciences (Aldine, Chicago, 1971).

10.

L.B. Doyle , Is relevance an adequate criterion in retrieval system evaluation?, Proc. American Documentation Institute 26th Annual Meeting, Chicago (1963) 199-200.

11.

D. Ellis , The effectiveness of information retrieval systems: the need for improved explanatory frameworks, Social Science Information Studies, to appear.

12.

T.R.G. Green , S.J. Payne and G.C. van der Veer , Eds., The Psychology of Computer Use (Academic Press , London, 1983).

13.

M.D. Heine , Simulation and simulation experiments, in: K. Sparck Jones , Ed., Information Retrieval Experiment (Butterworth, London, 1981) 179-198.

14.

A. Kent , Textbook on Mechanized Information Retrieval (Wiley, London, 2nd ed., 1966).

15.

F.W. Lancaster , Evaluation of the Medlars demand search service , National Library of Medicine, Bethesda, MD (1968).

16.

M.E. Lesk and G. Salton , Relevance assessments and retrieval system evaluation, Information Storage and Retrieval 4 (1968) 343-359.

17.

S. Maas , Why systems transparency, in: T.R.G. Green , S.J. Payne and G.C. van der Veer , Eds., The Psychology of Computer Use (Academic Press, London, 1983).

18.

R.N. Oddy , Information retrieval through man-machine dialogue, Journal of Documentation 33 (1977) 1-14.

19.

R.N. Oddy , Laboratory tests: automatic systems, in: K. Sparck Jones , Ed., Information Retrieval Experiment (Butterworth, London, 1981) 156-178.

20.

H. Putnam, Meaning and the Moral Sciences (RKP, London, 1978).

21.

S.E. Robertson , Theories and models in information retrieval , Journal of Documentation 33 (1977) 126-148.

22.

S.E. Robertson , Between aboutness and meaning, in: M. MacCafferty and K. Gray , Eds., The Analysis of Meaning: Informatics 5, Proceedings of a Conference held by Aslib Information Group and the BCS/IRS Group, 26-28 March 1979 (Aslib , London, 1979) 202-205.

23.

S.E. Robertson and N.J. Belkin , Ranking in principle, Journal of Documentation 34 (1978 ) 93-100.

24.

J.R. Sharp , Review of: K. Sparck Jones, Ed., Information Retrieval Experiment (Butterworth, London, 1981), Journal of Documentation 38 (1982) 238-240.

25.

C.J. van Rijsbergen , Retrieval effectiveness, in: K. Sparck Jones , Ed., Information Retrieval Experiment (Butterworth, London , 1981) 32-43.

26.

T.D. Wilson , On user studies and information needs, Journal of Documentation 37 (1981 ) 3-15.