This paper examines whether or not there is a problem for information systems evaluation in healthcare, given the contrast between the quantitative and qualitative research approaches. Experience gained on two systematic reviews on health information systems conducted under the auspices of EPOC (Cochrane Effective Practice and Organization of Care Group) demonstrates that searching strategies need to take account of the way technology is adapted, and accepted, with accompanying changes in terminology. One review concerned nursing record systems, the other telemedicine. Various frameworks for information systems evaluation emphasize the importance of a holistic view of evaluation, an apparent contrast to the evidence-based medicine model based on the randomized controlled trial (RCT). While RCTs are possible and desirable in health-care information systems evaluation, the equivocal results obtained from the reviews suggest that a pluralistic approach is highly desirable.
References
1.
Kitson A, Ahmed L B, Harvey G, Seers K.From research to publication: one organizational model for promoting research-based practice. Journal of Advanced Nursing1996; 23: 430-440.
2.
Audit Commission. Caring systems: effective implementation of ward nursing management systems. A handbook for managers of nursing and project managers. London: HMSO, 1992.
3.
Department of Health, NHS Executive. Information for Health: an information strategy for the modern NHS 1998-2005. London: Stationery Office, 1998.
4.
Balas E A, Jaffray F, Kuperman G, Boren S, Brown G, Pinciroli F, Mitchell J.Electronic communication with patients. Evaluation of distance medicine technology. JAMA1997; 278: 152-159.
5.
Balas E A, Iakovidis I.Distance technologies for patient monitoring. BMJ1999; 319: 1309.
6.
Taylor P.A survey of research in telemedicine. 1: Telemedicine systems. Journal of Telemedicine and Telecare1998; 4: 1-17.
7.
Taylor P.A survey of research in telemedicine. 2. Telemedicine services. Journal of Telemedicine and Telecare1998; 4: 63-71.
8.
Loane M A, Bloomer S E, Corbett R, Eedy D J, Gore H E, Hicks N, Mathews C, Paisley J, Steele K, Wootton R.Patient cost-benefit analysis of teledermatology measured in a randomized control trial. Journal of Telemedicine and Telecare1999; 5Suppl 1: S1-S3.
9.
Matthews E J, Edwards A G K, Barker Jet al. Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care. Health Libraries Review1999; 16: 112-120.
10.
Davies R, Urquhart C J, Smith, J.et al. Establishing the value of information to nursing continuing education: report of the EVINCE project. British Library Research and Innovation Centre Report 44. Boston Spa: BLDSC, 1997.
11.
Hicks C.The shortfall in published research: a study of nurses’ research and publication activities. Journal of Advanced Nursing1995; 21: 594-604.
12.
Pinch T J, Bijker W E.The social construction of facts and artefacts: or how the sociology of science and the sociology of technology might benefit each other. In: W E Bijker, T P Hughes and T J Pinch (eds.) The Social Construction of Technological Systems. London: MIT Press, 1987, 17-50.
13.
Brennan P F, Ripich S.Use of a home-care computer network by persons with AIDS. International Journal of Technology Assessment in Health Care1994; 10: 258-272.
14.
Brennan P F, Moore S M, Smyth K A.The effects of a special computer network on caregivers of persons with Alzheimer’s disease. Nursing Research1995; 44: 166-172.
15.
Elton M C J.The use of field trials in evaluating telemedicine applications. In R L Bashshur, P A Armstrong, Z I Youssef (eds.) Telemedicine: explorations in the use of telecommunications in health care. Springfield: Charles C Thomas, 1975, 253-299.
16.
Burkle T, Kuch R, Prokosch H U, Dudeck J.Stepwise evaluation of information systems in an university hospital. Methods of Information in Medicine. 1999; 38: 9-15.
17.
Marasovic C, Kenney C, Elliott D, Sindusake D.A comparison of nursing activities associated with manual and automated documentation in an Australian intensive care unit. Computers in Nursing1997; 15: 205-211.
18.
Smithson S, Hirschheim R.Analysing information systems evaluation: another look at an old problem. European Journal of Information Systems1998; 7: 158-174.
19.
Willcocks L P, Lester S.Beyond the IT productivity paradox: assessment issues. Chichester: Wiley, 1998.
20.
Fitzgerald G.Evaluating information systems projects: a multi-dimensional approach. Journal of Information Technology1998; 13: 15-27.
21.
Orlikowski W J, Baroudi J J.Studying information technology in organizations: research approaches and assumptions. Information Systems Research1991; 2: 1-28.
22.
Orlikowski W J.Improvising organizational transformation over time: a situated change perspective. Information Systems Research1996; 7: 63-92.
23.
Heathfield H, Pitty D, Hanka R.Evaluating information technology in health care: barriers and challenges. BMJ1998; 316: 1959-1961.
24.
Friedman C P, Wyatt J C.Evaluation methods in medical informatics. New York: Springer, 1997.
25.
Evans G E.Information systems as interventions: the case for outcomes based evaluation. Journal of Computer Information Systems1998; 39: 69-74.
26.
Kaplan B.Organizational evaluations of medical information resources. In Friedman C P, Wyatt J C (eds.) Evaluation methods in medical informatics. New York: Springer, 1997, pp. 255-280.
