In contrast to traditional narrative reviews, systematic reviews are true hypothesis-driven research. Meta-analysis is a form of systematic review in which studies are selected and combined by use of a predefined protocol to reduce bias and subjectivity. A sensitivity analysis shows how results vary through the use of different assumptions, tests, and criteria. The most valid synthesis of information occurs when published and unpublished materials are subjected to the same rigorous evaluation and when results are calculated with and without unpublished sources of data. A good systematic review captures the reader's attention through a skillful blend of numbers and narrative and qualifies for publication as original research in a peer-reviewed journal. Otolaryngologists have published systematic reviews of varying quality since 1990. This article should help improve the quality and validity Of future efforts.
Get full access to this article
View all access options for this article.
References
1.
RosenfeldRMClinical research in otolaryngology journals. Arch Otolaryngol Head Neck Surg1991;117:164–70.
2.
LightRJPillemerDBSumming up: the science of reviewing research. Cambridge, Mass.: Harvard University Press, 1984:3–6.
3.
CookDJSackettDLSpitzerWOMethodologic guidelines for systematic reviews of randomized control trials in health care from the potsdam consultation on meta-analysis. J Clin Epidemiol1995;48:167–71.
4.
SacksHSBerrierJReitmanDPaganoDChalmersTCMeta-analyses of randomized control trials: An update of the quality and methodology. In: BailarJCMostellerF, Eds. Medical uses of statistics. 2nd ed.Boston: NEJM Books, 1992;427–42.
5.
NeelyJGLiterature review articles as a research form. Otolaryngol Head Neck Surg1993;108:743–8.
6.
ShaughnessyAFSlawsonDCBennettJHBecoming an information master: A guidebook to the medical information jungle. J Fam Pract1994;39:489–99.
7.
ChamblissMLPersonal computer access to MEDLINE: An introduction. J Fam Pract1991;32:414–9.
8.
WrightLCSutherlandHJJacksonJITillJEComparison of search strategies on CD Plus/MEDLINE. Can Med Assoc J1991;145:457–64.
9.
LoweHJBarnettGOUnderstanding and using the medical subject headings (MeSH) vocabulary to perform literature searches. JAMA1994;271:1103–8.
10.
Medical subject headings: annotated alphabetic list 1994. Bethesda, Md.: National Library of Medicine, 1995.
11.
DickersinKSchererRLefebvreC.Identifying relevant studies for systematic reviews. BMJ1994;309:1286–91.
12.
FremerE.Understanding MeSH for literature searches [Letter], JAMA1995;273:184.
13.
HuthEJSearching the literature. In: HuthEJHow to write and publish papers in the medical sciences. 2nd ed.Baltimore: Williams & Wilkins, 1990:12–42.
14.
SchererRDickersinKLangenbergP.Full publication of results initially presented as abstracts: A meta-analysis. JAMA1994;272:158–62.
15.
CookDJGuyattGHTyanGShould unpublished data be included in meta-analyses? Current convictions and controversies. JAMA1993;269:2749–53.
16.
ChalmersTCLevinHSacksHSReitmanDBerrierJNagalingamR.Meta-analysis of clinical trials as a scientific discipline. Control of bias in comparison with large co-operative trials. Stat Med1987;6:315–25.
17.
RosenfeldRMVertreesJECarrJClinical efficacy of antimicrobial drugs for acute otitis media: Metaanalysis of 5400 children from thirty-three randomized trials. J Pediatr1994;124:355–67.
18.
StellPMRawsonNSB.Adjuvant chemotherapy in head and neck cancer. Br J Cancer1990;61:779–87.
19.
StellPMPloidy in head and neck cancer: A review and metaanalysis. Clin Otolaryngol1991;16:510–6.
20.
RosenfeldRMMandellEMBluestoneCDSystemic steroids for otitis media with effusion in children. Arch Otolaryngol Head Neck Surg1991;117:984–9.
21.
RosenfeldRMPostJCMeta-analysis of antibiotics for the treatment of otitis media with effusion. Otolaryngol Head Neck Surg1992;106:378–86.
22.
WoolleyALHogikyanNDGatesGAHaugheyBHSchechtmanKBGoldenbergJLEffect of blood transfusion on recurrence of head and neck carcinoma: Retrospective review and meta-analysis. Ann Otol Rhinol Laryngol1992;101:724–30.
23.
SnydermanCHD'AmicoF.Outcome of carotid artery resection for neoplastic disease: A meta-analysis. Am J Otolaryngol1992;13:373–80.
24.
HaugheyBHGatesGAArfkenCLHarveyJ.Meta-analysis of second malignant tumors in head and neck cancer: The case for an endoscopic screening protocol. Ann Otol Rhinol Laryngol1992;101:105–12.
25.
GarciaPGatesGASchechtmanKBDoes topical antibiotic prophylaxis reduce post-tympanostomy tube otorrhea? A metaanalysis. Ann Otol Rhinol Laryngol1994;103:54–8.
26.
PrasadSJaneckaIPEfficacy of surgical treatments for squamous cell carcinoma of the temporal bone: A literature review. Otolaryngol Head Neck Surg1994;110:270–80.
27.
ShatkinJSDelsupeheKGThistedRACoreyJPMucosal allergy in the absence of systemic allergy in nasal polyposis and rhinitis: A meta-analysis. Otolaryngol Head Neck Surg1994;111:553–6.
28.
StoolSEBergAOBermanSOtitis media with effusion in young children. Clinical Practice Guideline, Number 12. AHCPR Publication no. 94-0622. Rockville, Md.: Agency for Health Care Policy and Research, Public Health Service, U.S. Department of Health and Human Services, July1994.
29.
OxmanADCookDJGuyattGHUsers' guides to the medical literature: Vi. How to use an overview. JAMA1994;272:1367–71.
BoisselJPBlanchardJPanakEPeyrieuxJCSacksH.Considerations for the meta-analysis of randomized clinical trials: Summary of a panel discussion. Control Clin Trials1989;10:254–81.
32.
SlavenREBest evidence synthesis: An intelligent alternative to meta-analysis. J Clin Epidemiol1995;48:9–18.
33.
WilsonAHenryDAMeta-analysis part 2: Assessing the quality of published meta-analyses. Med J Aust1992;156:173–87.
34.
FeinsteinARMeta-analysis: Statistical alchemy for the 21st century. J Clin Epidemiol1995;48:71–9.
35.
SpectorTDThompsonSGThe potential and limitations of meta-analysis. J Epidemiol Community Health1991:45:89–92.
36.
MoherDJadadARNicholGPenmanMTugwellPWalshS.Assessing the quality of randomized controlled trials: An annotated bibliography of scales and checklists. Control Clin Trials1995;16:62–73.
37.
SchulzKFChalmersIHayesRJAltmanDGEmpirical evidence of bias: Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA1995;273:408–12.
JonesDRMeta-analysis: Weighing the evidence. Stat Med1995;14:137–49.
40.
LaupacisASackettDLRobertsRSAn assessment of clinically useful measures of the consequences of treatment. N Engl J Med1988;318:1728–33.
41.
SackettDLApplying overviews and meta-analyses at the bedside. J Clin Epidemiol1995;48:61–6.
42.
BorensteinM.The case for confidence intervals in controlled clinical trials. Control Clin Trials1994;15:411–8.
43.
DersimonianRLairdN.Meta-analysis in clinical trials. Control Clin Trials1986;7:177–88.
44.
HalstedCLapMLBalassanianNEmmerichJWolinskyE.Otitis media: Clinical observations, microbiology, and evaluation of therapy. Am J Dis Child1968;10:631–9.
45.
LaxdalOEMeridaJJonesRHTreatment of acute otitis media: A controlled study of 142 children. Can Med Assoc J1970;102:263–8.
46.
MygindNMeistrup-LarsenK-IThomsenJThomsenVFJosefssonKSorensenH.Penicillin in acute otitis media: A double-blind placebo-controlled trial. Clin Otolaryngol1981;6:5–13.
47.
BurkePBainJRobinsonDDunleaveyJ.Acute red ear in children: Controlled trial of nonantibiotic treatment in general practice. BMJ1991;303:558–62.
48.
EmersonJDBurdickEHoaglinDCMostellerFChalmersTCAn empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials. Control Clin Trials1990;11:339–52.
49.
DickersinKMinYIMeinertCLFactors influencing publication of research results: Follow-up of applications submitted to two institutional review boards. JAMA1992;267:374–8.
50.
RosenthalR.The “file drawer problem” and tolerance for null results. Psychol Bull1979;86:638–41.
51.
SackettDLBias in analytic research. Journal of Chronic Diseases1979;32:51–63.
52.
TurnerJADeyoRALoeserJDVon KorffMFordyceWEThe importance of placebo effects in pain treatment and research. JAMA1994;271:1609–14.
