Use and reporting of risk of bias tools in 825 systematic reviews of acupuncture: a cross-sectional study

Abstract

Objective:

To assess the use and reporting of risk of bias (RoB) tools in systematic reviews (SRs) of acupuncture.

Study design and setting:

We extracted and analyzed information relating to RoB in acupuncture SRs via Medline, Embase and the Chinese CNKI (Chinese National Knowledge Infrastructure), WanFang and VIP databases from their inception to 24 November 2017. Three subgroup analyses were used to check the influence of language, journal type and impact factor, following which we used descriptive analysis.

Results:

We included 825 acupuncture SRs, of which 48% used the Cochrane RoB tool. Only 36% used the latest version of the Cochrane Handbook (version 5.1.0 at time of writing) with higher proportions among Cochrane SRs (65%) versus non-Cochrane SRs (34%), and high impact factor journals (58%) versus low or no impact factor journals (28% and 38%, respectively). In the last decade, there were notable increases in the use of the Cochrane RoB tool and Cochrane Handbook version 5.1.0, of 43% and 19%, respectively. Chinese-language SRs demonstrated proportionally higher tendencies to report an incorrect Cochrane Handbook version, increasing by 14% in the last 5 years. Additionally, 7% SRs did not report any results, and only 10% reported relatively complete and adequate RoB assessment. Cochrane SRs reported more complete assessments than Chinese-language or non-Cochrane English-language SRs.

Conclusion:

Use and reporting of RoB tools were suboptimal. Proportionally, use of the Cochrane RoB tool and Cochrane Handbook version 5.1.0 was low but rising. Our results highlight the prevalence and concerns of using unsuitable tools and the issue of incomplete RoB reporting. RoB tool application requires further improvement.