Search filters are increasingly used to search medical databases to identify specific topics or study designs. In particular, search filters have been designed to help health-care professionals identify systematic reviews of the effectiveness of health interventions. Identifying systematic reviews in databases such as MEDLINE is problematic and research has previously been undertaken into methods to design search filters that retrieve systematic reviews effectively. The aim of this study was to improve previously developed methods to derive a more objective search strategy to identify systematic reviews in MEDLINE. A ‘quasi-gold standard’ collection of known systematic reviews was identified. A frequency analysis of words within a subset of the ‘quasi-gold standard’ was undertaken followed by a statistical analysis of the most frequently occurring words. This analysis determined which terms would best distinguish between systematic reviews, non-systematic reviews and non-reviews. The performance of the best models was tested on the remaining subset of ‘quasi-gold standard’ records and then using the OVID interface to MEDLINE. The best model had a sensitivity of 73.4% for systematic reviews in the test set and 84.2% when used with the validation set. The best model had a specificity of 98.3% in the test set and 93% in the validation set. When tested on the same ‘quasi-gold standard’ using OVID MED-LINE the model showed 100% sensitivity and 4.4% precision. The number of times a term occurs in a record adds discriminatory power to the search strategy. Apparently highly relevant terms chosen subjectively do not perform as well as those derived by a statistical approach. Some search terms may not immediately seem useful in identifying systematic reviews, but when used in combination with other terms they prove to be highly discriminating. The best performing filters were tested on the OVID interface, but without frequency and term weightings. Their performance was also compared to previously published filters. One of the strategies was found to perform better with respect to sensitivity than previously published filters, although with lower precision.
Get full access to this article
View all access options for this article.
References
1.
[1] R. Haynes , N. Wilczynski , K. McKibbon , C. Walker , J. Sinclair , Developing optimal search strategies for detecting clinically sound studies in MEDLINE , Journal of the American Medical Informatics Association1(6) (1994) 447-458 .
2.
[2] Pubmed Clinical Queries [database] ( National Center for Biotechnology Information, Bethesda, MD , 2001). Available at: www.ncbi.nlm.nih.gov/entrez/query/static/clinical.html (last accessed 8 November 2001).
3.
[3] C. Mulrow , Rationale for systematic reviews. In: I. Chalmers and D. Altman (eds) , Systematic Reviews ( BMJ Publishing, London , 1995).
4.
[4] F.M. Wolf , C. Lefebvre and W.C. McGaghie , A trend analysis and search strategies for the identification of meta-analyses in MEDLINE. (Abstract.) In: Fourth International Cochrane Colloquium, Adelaide, 1996. Available at: www.cochrane.org/cochrane/abpos22.htm (last accessed 8 November 2001).
5.
[5] J. Boynton , J. Glanville , D. McDaid , C. Lefebvre , Identifying systematic reviews in MEDLINE: developing an objective approach to search strategy design , Journal of Information Science24 (1998) 137-157 .
6.
[6] D.L. Hunt and K.A. McKibbon , Locating and appraising systematic reviews , Annals of Internal Medicine126 (1997) 532-538 .
7.
[7] NHS Centre for Reviews and Dissemination , Database of Abstracts of Reviews of Effectiveness (DARE) [database online] ( University of York, NHS Centre for Reviews and Dissemination, York , 2001). Available at: www.york.ac.uk/inst/crd (last accessed 8 November 2001).
8.
[8] A. Oxman , D.J. Cook and G.H. Guyatt , Users’ guides to the medical literature. VI. How to use an overview , JAMA272(17) (1994) 1367-1371 .
9.
[9] T. Greenhalgh , How to read a paper: papers that summarise other papers (systematic reviews and meta-analyses) , BMJ315 (1997) 672-675 .
10.
[10] Provalis Research , Simstat for Windows version 1.31[computer program] ( Provalis Research, Montreal , 1999). Available from: www.simstat.com/ (last accessed 8 November 2001).
11.
[11] Dialog Corporation , Dialog Classic Web [database host] ( Dialog Corporation, Cary, NC , 2001). Available at: www.dialogclassic.com/ (last accessed 8 November 2001).
12.
[12] Dialog Corporation , Successful Searching on Dialog: RANK[web page] ( Dialog Corporation, Cary, NC , 2001). Available at: http://library.dialog.com/success/rank.html (last accessed 8 November 2001).
[14] STN , STN - Mastering STN commands, Focus etc. [web page] ( Fiz-Karlsruhe, Karslruhe , 2001). Available at: www.fiz-karlsruhe.de/stn/messenger/training/focus.html (last accessed 8 November 2001).
15.
[15] International Committee of Medical Journal Editors , Uniform requirements for manuscripts submitted to biomedical journals . Annals of Internal Medicine126 (1997) 36-47 .
16.
[16] D. Moher , D. J. Cook , S. Eastwood , I. Okin , D. Rennie , D. F. Stroup , for the Quorum Group , Improving the quality of reports of meta-analyses of randomised controlled trials: the QUORUM statement . Lancet354 (1999) 1896-1900 .
