OBJECTIVE: To evaluate the quality of meta-analyses written on otolaryngological topics and define areas that can be improved upon in future studies.
DATA SOURCES: MEDLINE (PubMed) and EMBASE databases were searched. The Cochrane database of systematic reviews was excluded, because these meta-analyses have already been critically evaluated and found to be of high quality.
REVIEW METHODS: A systematic review of otolaryngological meta-analyses published between 1997 and 2006 (10 years) was performed in duplicate and independently by two authors. The search included 16 common otolaryngological terms. Inclusion criteria were meta-analytic methodology, otolaryngological topic, and at least one author from a department of otolaryngology. Fifty-one articles fulfilled eligibility criteria. In duplicate and independently, two reviewers assessed the quality of eligible metaanalyses using a validated 10-item index called the Overview Quality Assessment Questionnaire. Using the methods of Spearman, correlation coefficients are reported for associations examined between covariates and the Overall Score Quality.
RESULTS: The majority of studies had methodologic flaws (mean score 3.9, scale of 1-7). Variables predicting higher-quality meta-analyses were publication in journals with higher impact factors (P = 0.0007) and authors who previously published metaanalyses (P = 0.0001). Using and reporting about a validity assessment tool needs to be improved upon in future studies.
CONCLUSION: The quality of meta-analyses on otolaryngological topics is moderate. Future meta-analyses can be improved upon by following evidence-based guidelines for the reporting of metaanalyses, which include the use of a validity assessment tool, and consulting with an author familiar with meta-analysis methodology.
Get full access to this article
View all access options for this article.
References
1.
1. MoherDCookDJEastwoodS, et al. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. QUOROM Group. Br J Surg2000;87: 1448–554.
2.
2. OxmanADGuyattGHValidation of an index of the quality of review articles. J Clin Epidemiol1991;44:1271–8.
3.
3. MoherDJadadARTugwellP.Assessing the quality of randomized controlled trials. Current issues and future directions. Int J Technol Assess Health Care1996;12: 195–208.
4.
4. JadadARCookDJJonesA, et al. Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA1998;280:278–80.
5.
5. BrandweinMSRothsteinALawsonW, et al. Sinonasal melanoma. A clinicopathologic study of 25 cases and literature meta-analysis. Arch Otolaryngol Head Neck Surg1997;123: 290–6.
6.
6. KassirRRRassekhCHKinsellaJB, et al. Osteosarcoma of the head and neck: meta-analysis of nonrandomized studies. Laryngoscope1997;107:56–61.
7.
7. MerrittRMWilliamsMFJamesTH, et al. Detection of cervical metastasis. A meta-analysis comparing computed tomography with physical examination. Arch Otolaryngol Head Neck Surg1997;123: 149–52.
8.
8. WietRJKubekDCLembergP, et al. A meta-analysis review of revision stapes surgery with argon laser: effectiveness and safety. Am J Otol1997;18:166–71.
9.
9. HebertRL2ndBentJP, 3rd. Meta-analysis of outcomes of pediatric functional endoscopic sinus surgery. Laryngoscope1998;108:796–9.
10.
10. SelesnickSHJohnsonG.Radiologic surveillance of acoustic neuromas. Am J Otol1998;19:846–9.
11.
11. ValentinoJBoggessMAEllisJL, et al. Expected neurologic outcomes for surgical treatment of cervical neurilemomas. Laryngoscope1998;108:1009–113.
12.
12. CarrauRLSegasJNussDW, et al. Squamous cell carcinoma of the sinonasal tract invading the orbit. Laryngoscope1999;109:230–5.
13.
13. LeeDYoukAGoldsteinNAA meta-analysis of swimming and water precautions. Laryngoscope1999;109:536–40.
14.
14. OlsenSOThe relationship between the uncomfortable loudness level and the acoustic reflex threshold for pure tones in normally-hearing and impaired listeners—a meta-analysis. Audiology1999;38:61–8.
15.
15. SchilderAGAssessment of complications of the condition and of the treatment of otitis media with effusion. Int J Pediatr Otorhinolaryngol1999;49Suppl 1:S247–51.
16.
16. VrabecJTDeskinRWGradyJJMeta-analysis of pediatric tympanoplasty. Arch Otolaryngol Head Neck Surg1999;125:530–4.
17.
17. GoldmanACGovindarajSRosenfeldRMA meta-analysis of dexa-methasone use with tonsillectomy. Otolaryngol Head Neck Surg2000;123:682–6.
18.
18. GoodwinWJJr.Salvage surgery for patients with recurrent squamous cell carcinoma of the upper aerodigestive tract: when do the ends justify the means? Laryngoscope2000;110:1–18.
19.
19. HegazyHMCarrauRLSnydermanCH, et al. Transnasal endoscopic repair of cerebrospinal fluid rhinorrhea: a meta-analysis. Laryngoscope2000;110:1166–72.
20.
20. RosenfeldRMSurgical prevention of otitis media. Vaccine2000;19Suppl 1:S134–9.
21.
21. DulguerovPAllalASCalcaterraTCEsthesioneuroblastoma: a meta-analysis and review. Lancet Oncol2001;2:683–90.
22.
22. KayDJNelsonMRosenfeldRMMeta-analysis of tympanostomy tube sequelae. Otolaryngol Head Neck Surg2001;124:374–80.
23.
23. KiernerACZelenkaIGstoettnerW.The sternocleidomastoid flap—its indications and limitations. Laryngoscope2001;111:2201–4.
24.
24. BriggerMTHartnickCJSurgery for pediatric vocal cord paralysis: a meta-analysis. Otolaryngol Head Neck Surg2002;126:349–55.
25.
25. CarbonellRRuiz-GarciaV.Ventilation tubes after surgery for otitis media with effusion or acute otitis media and swimming. Systematic review and meta-analysis. Int J Pediatr Otorhinolaryngol2002;66: 281–9.
26.
26. GoldsmithDBWestTMMortonR.HLA associations with nasopharyngeal carcinoma in Southern Chinese: a meta-analysis. Clin Otolaryngol Allied Sci2002;27:61–7.
27.
27. RosenfeldRMKayD.Natural history of untreated otitis media. Laryngoscope2003;113:1645–7.
28.
28. SatarBYetiserSOzkaptanY.Impact of tumor size on hearing outcome and facial function with the middle fossa approach for acoustic neuroma: a meta-analytic study. Acta Otolaryngol2003;123:499–505.
29.
29. ChiaSHGamstACAndersonJP, et al. Intratympanic gentamicin therapy for Meniere's disease: a meta-analysis. Otol Neurotol2004;25:544–52.
30.
30. Cohen-KeremRKisilevskyVEinarsonTR, et al. Intratympanic gentamicin for Meniere's disease: a meta-analysis. Laryngoscope2004;114:2085–91.
31.
31. RobertsJERosenfeldRMZeiselSAOtitis media and speech and language: a meta-analysis of prospective studies. Pediatrics2004;113:e238–48.
32.
32. SelesnickSHLiuJCJenA, et al. Management options for cerebro-spinal fluid leak after vestibular schwannoma surgery and introduction of an innovative treatment. Otol Neurotol2004;25:580–6.
33.
33. WoodworthBAGillespieMBLambertPRThe canalith repositioning procedure for benign positional vertigo: a meta-analysis. Laryngoscope2004;114:1143–6.
34.
34. BennettAMClarkABBathAP, et al. Meta-analysis of the timing of haemorrhage after tonsillectomy: an important factor in determining the safety of performing tonsillectomy as a day case procedure. Clin Otolaryngol2005;30:418–23.
35.
35. DubinMGEbertCSCoffeyCS, et al. Concordance of middle meatal swab and maxillary sinus aspirate in acute and chronic sinusitis: a meta-analysis. Am J Rhinol2005;19:462–70.
36.
36. MeratiALLimHJUlualpSO, et al. Meta-analysis of upper probe measurements in normal subjects and patients with laryngopharyngeal reflux. Ann Otol Rhinol Laryngol2005;114:177–82.
37.
37. PaleriVReesGArullendranP, et al. Sentinel node biopsy in squamous cell cancer of the oral cavity and oral pharynx: a diagnostic meta-analysis. Head Neck2005;27:739–47.
38.
38. RoversMMBlackNBrowningGG, et al. Grommets in otitis media with effusion: an individual patient data meta-analysis. Arch Dis Child2005;90:480–5.
39.
39. SdanoMTFalcigliaMWelgeJA, et al. Efficacy of thyroid hormone suppression for benign thyroid nodules: meta-analysis of randomized trials. Otolaryngol Head Neck Surg2005;133:391–6.
40.
40. StewartMGSmithTLObjective versus subjective outcomes assessment in rhinology. Am J Rhinol2005;19:529–35.
41.
41. WhiteJSavvidesPCherianN, et al. Canalith repositioning for benign paroxysmal positional vertigo. Otol Neurotol2005;26:704–10.
42.
42. WiegandSDunneAAMullerHH, et al. Metaanalysis of the significance of matrix metalloproteinases for lymph node disease in patients with head and neck squamous cell carcinoma. Cancer2005;104:94–100.
43.
43. BenningerMSPayneSCFergusonBJ, et al. Endoscopically directed middle meatal cultures versus maxillary sinus taps in acute bacterial maxillary rhinosinusitis: a meta-analysis. Otolaryngol Head Neck Surg2006;134:3–9.
44.
44. BrietzkeSEGallagherD.The effectiveness of tonsillectomy and adenoidectomy in the treatment of pediatric obstructive sleep apnea/hypopnea syndrome: a meta-analysis. Otolaryngol Head Neck Surg2006;134:979–84.
45.
45. BusquetsJMHwangPHEndoscopic resection of sinonasal inverted papilloma: a meta-analysis. Otolaryngol Head Neck Surg2006;134:476–82.
46.
46. CohenSMDupontWDCoureyMSQuality-of-life impact of non-neoplastic voice disorders: a meta-analysis. Ann Otol Rhinol Laryngol2006;115:128–34.
47.
47. CohenSMGarrettCGDupontWD, et al. Voice-related quality of life in T1 glottic cancer: irradiation versus endoscopic excision. Ann Otol Rhinol Laryngol2006;115:581–6.
48.
48. DunneAAMullerHHEiseleDW, et al. Meta-analysis of the prognostic significance of perinodal spread in head and neck squamous cell carcinomas (HNSCC) patients. Eur J Cancer2006;42:1863–8.
49.
49. HochmanJBlakleyBAbdohA, et al. Post-tympanostomy tube otorrhea: a meta-analysis. Otolaryngol Head Neck Surg2006;135:8–11.
50.
50. PaydarfarJABirkmeyerNJComplications in head and neck surgery: a meta-analysis of postlaryngectomy pharyngocutaneous fistula. Arch Otolaryngol Head Neck Surg2006;132:67–72.
51.
51. QadeerMAPhillipsCOLopezAR, et al. Proton pump inhibitor therapy for suspected GERD-related chronic laryngitis: a meta-analysis of randomized controlled trials. Am J Gastroenterol2006;101:2646–54.
52.
52. RosenfeldRMSingerMWassermanJM, et al. Systematic review of topical antimicrobial therapy for acute otitis externa. Otolaryngol Head Neck Surg2006;134:S24–48.
53.
53. RoversMMGlasziouPAppelmanCL, et al. Antibiotics for acute otitis media: a meta-analysis with individual patient data. Lancet2006;368:1429–35.
54.
54. SadriMMcMahonJParkerA.Management of laryngeal dysplasia: a review. Eur Arch Otorhinolaryngol2006;263:843–52.
55.
55. SinghAPatelNKenyonG, et al. Is there objective evidence that septal surgery improves nasal airflow? J Laryngol Otol2006;120:916–20.
56.
56. OxmanADGuyattGHSingerJ, et al. Agreement among reviewers of review articles. J Clin Epidemiol1991;44:91–8.
57.
57. CuzickJ.A Wilcoxon-type test for trend. Stat Med1985;4:87–90.
58.
58. GuyattGHSackettDLSinclairJC, et al. Users' guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group. JAMA1995;274: 1800–4.
59.
59. DixonEHameedMSutherlandF, et al. Evaluating meta-analyses in the general surgical literature: a critical appraisal. Ann Surg2005;241: 450–9.
60.
60. DelaneyABagshawSMFerlandA, et al. A systematic evaluation of the quality of meta-analyses in the critical care literature. Crit Care2005;9:R575–82.
