Methodological Quality of Systematic Reviews and Meta-analyses Published in High-Impact Otolaryngology Journals

Abstract

Objective

To assess the methodological quality of intervention-focused systematic reviews (SRs) and meta-analyses (MAs) published in high-impact otolaryngology journals.

Data Sources

Ovid Medline, Embase, and Cochrane Library.

Review Methods

A comprehensive search was performed for SR and MA citations from 2012 to 2017 in the 10 highest impact factor otolaryngology journals. Abstracts were screened to identify published manuscripts in which the authors indicated clearly that they were performing an SR or MA. Applying a modified typology of reviews, 4 reviewers characterized the review type as SR, MA, or another review type. A simplified version of the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews 2) tool was used to assess the reporting and methodological quality of the SRs and MAs that were focused on interventions.

Results

Search and abstract screening generated 499 manuscripts that identified themselves as performing an SR or MA. A substantial number (85/499, 17%) were review types other than SRs or MAs, including 34 (7%) that were literature reviews. In total, 236 SRs and MAs focused on interventions. Over 50% of these SRs and MAs had weaknesses in at least 3 of the 16 items in the AMSTAR 2, and over 40% had weaknesses in at least 2 of the 7 critical domains. Ninety-nine percent of SRs and MAs provided critically low confidence in the results of the reviews.

Conclusion

Intervention-focused SRs and MAs published in high-impact otolaryngology journals have important methodological limitations that diminish confidence in the results of these reviews.