Quality of Reporting in Randomized Trials

A great deal of emphasis has been placed recently on the importance of high-quality evidence in medical literature. Placebo-controlled, randomized, prospective studies are valued especially highly. However, the fact that a study is placebo-controlled, randomized, and prospective does not necessarily mean that it was well-designed and well-reported, or that the conclusions are valid. In a recent article, Sinha et al highlighted this important issue in a review of randomized trials published in high-quality surgical journals. ¹ The quality of the journal was determined by its impact factor.

In analyzing the literature, Sinha et al used the Consolidated Standards of Reporting Trials (CONSORT) statement published in JAMA in 1996. ² The CONSORT statement resulted from international collaboration among biomedical editors, clinicians, and statisticians. In order to correct shortcomings in reporting of trial methodology, they promulgated a standardized framework to assist authors in reporting findings of clinical trials completely and transparently. The CONSORT statement was revised in 2001, ³ and it can be accessed at www.consort-statement.org. Although following CONSORT guidelines improves reporting of clinical trials, and although the statement is accepted by many biomedical journals, it has not been used as widely as one might wish.

In their study, Sinha et al analyzed 42 randomized, controlled trials (RCTs), calculating their Jadad scores. The Jadad score is determined through assessment of questions addressing randomization, blinding, and reporting of withdrawals and dropouts from studies. ⁴ The Jadad score has been validated as a scale for assessing trial reporting quality and is used to distinguish low-quality from high-quality studies. Sinha et al used a Jadad score of ≥3 for their threshold (a conservative choice, since some authors require a score of ≥4). In the three high-quality journals they evaluated, only 40% of 42 RCTs achieved a Jadad score of >3. Only 32.5% of the RCTs reported methodology adequately, and only 17% provided adequate reporting of adverse events.

Such self-analysis in the general surgical literature is commendable. In otolaryngology, we rely upon published literature to guide our patient care decisions (among other things). When we read reports that are given the highest evidence ratings, and when we see that they are prospective, randomized trials, many of us are inclined to believe the results. However, it seems appropriate for us to critique these publications as rigorously as possible, both in terms of design and in terms of quality and completeness of reporting.

All of us should be aware of the CONSORT statement. Although I have not followed Sinha's example and performed a similar analysis on otolaryngology literature yet, I believe this should be done. Only through constant, rigorous assessment of the quality of our published data can we hope to arrive at true information that can be applied ethically, and with confidence, to the care of our patients. Journals in our field should consider adopting the CONSORT statement and educating otolaryngologic reviewers to consider it (and the Jadad score) when evaluating randomized trials submitted for publication in our literature.