Sage Journals: Discover world-class research

Abstract

Background:

The American Academy of Orthopaedic Surgeons (AAOS) publishes clinical practice guidelines (CPGs) for various pathologies. An extension to the Consolidated Standards for Reporting (CONSORT) checklist provides authors with a framework for reporting harms in randomized controlled trials (RCTs).

Objectives:

The purpose of this study was to measure harms reporting among RCTs cited as supporting evidence for the AAOS CPG on the treatment of clavicle fractures. This study also sought to determine whether these reporting rates changed over time.

Methods:

Adhering to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we used the AAOS CPG for clavicle fractures and conducted a linear regression analysis to model the relationship between the year of publication and the total CONSORT checklist percentage adherence over time.

Results:

There were 28 RCTs cited as supporting evidence for the clavicle fractures CPG included in the final sample. The average number of the 18 CONSORT Extension for Harms items appropriately reported across all included RCTs was 9.32 (51.8%). Nine checklist items had more than 50% compliance (50%), 3 items had between 20% and 50% compliance (16.7%), and 6 items had less than 20% compliance (33.3%). The linear regression model demonstrated no significant improvement in CONSORT Harms reporting over time.

Conclusions:

Adverse events are inadequately reported in RCTs cited as supporting evidence for the AAOS Treatment of Clavicle Fractures CPGs. We recommend the utilization of standardized adverse event reporting tools specific to orthopedic trauma literature to facilitate ease in adverse event reporting among RCTs.

Keywords

harms reporting CONSORT clavicle fracture clinical practice guidelines

Introduction

Clinical practice guidelines (CPGs) are a set of rigorously developed recommendations on the management of a disease or condition that can promote beneficial patient outcomes if applied properly [11,28]. The American Academy of Orthopaedic Surgeons (AAOS) has published a series of guidelines for various orthopedic conditions based on a comprehensive systematic search with formal peer review and public commentary [27]. The AAOS has published 14 recommendations for the treatment of clavicle fractures (Supplemental Table 1).

The clavicle is one of the most frequently fractured bones in the human body, accounting for 3% to 5% of all fractures [21]. Historically, clavicle fractures have been treated nonoperatively with good functional outcomes, but recent research highlighting an increased risk of nonunion has prompted a rise in operative treatment [20,22,26]. Although several studies have demonstrated positive results after surgical intervention [9,23,29], there is an inherent risk of complications such as hardware irritation, infection, and neurovascular injury [16,24]. The high incidence of this injury coupled with the variability in management between providers makes clavicle fractures an ideal candidate for a CPG. For each CPG topic, experts in the field evaluate the literature on the topic to assign a strength of each recommendation labeled as inconclusive, consensus, limited, moderate, or strong.

Many of the studies underpinning the CPGs are randomized controlled trials (RCT) [10]. Because RCTs are considered the gold standard for assessing the efficacy of a treatment, it is important to analyze the extent to which they adhere to appropriate adverse event (AE) (harms) reporting [14]. The Consolidated Standards for Reporting (CONSORT) checklist was developed with the goal of improving the quality of reporting in RCTs. The original checklist was developed in 2001, a CONSORT extension has since been developed in 2004 and later updated in 2010 with the specific aim to improve harms reporting in RCTs. The CONSORT Extension for Harms is an 18-item checklist that offers a standardized method for authors to report AEs in RCTs to facilitate transparent reporting [12]. Although the checklist has existed for many years, several studies have demonstrated poor reporting of harms among RCTs in the fields of internal medicine [3,15], psychology [13], and orthopedics [1]. In light of this trend, the purpose of this study was to measure the harms-related reporting among RCTs cited as supporting evidence for the AAOS CPG regarding Treatment of Clavicle Fractures.

Methods

In this study, no institutional review board oversight was required as it did not involve human subjects. Throughout the investigation, we followed the guidelines for reporting meta-research [19] and adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [7].

To identify relevant CPGs, we utilized the AAOS CPG for the treatment of clavicle fractures available on the AAOS website. Specifically, we referred to the list of studies included in the AAOS CPG titled “Quality Evaluation-Intervention-Randomized” [10]. Two authors (A.T. and A.I.) independently reviewed the studies listed in this table to identify all RCTs cited as supporting evidence for at least 1 CPG recommendation. The following inclusion criteria were applied to each RCT: (1) the trial met the definition of an RCT according to the International Committee of Medical Journal Editors; (2) the trial was available in English (for feasibility); and (3) the trial was cited as evidence supporting one or more CPG recommendations. After the screening process, the results from the 2 data extractors were compared, and any discrepancies were resolved through group discussion.

Two authors (A.T. and A.I.) independently conducted the data collection process. The following information for each study was collected: title; PMID; year of publication; journal of publication; CPG for which RCT was cited as supporting evidence; RCT funding source (if applicable); country RCT was conducted in; sample size; blinding; and number of participating centers. Additionally, each trial was evaluated based on the CONSORT Harms checklist, which consists of 10 recommendations for reporting harms that occur over the course of an RCT. Each item was scored as “yes” (1 point) or “no” (0 points) based on adherence to the predefined requirements. The overall percentage adherence was calculated out of a possible 18 points, considering that some CONSORT Harms Extension items have multiple subcategories. Scoring was discussed between the 2 authors, and any discrepancies were resolved through group discussion.

Statistical Analysis

Descriptive statistics were used to characterize the overall compliance with the CONSORT Extension for Harms. To demonstrate the relationship between the year of publication and the total CONSORT percent adherence over time, a linear regression analysis was performed.

Results

The AAOS guidelines for clavicle fracture treatment included 63 references cited as supporting evidence for at least 1 CPG. We compared this list with the “Quality Intervention—Randomized” table in the AAOS CPG, which presented a comprehensive list of all RCTs included in the guidelines. This process resulted in 28 RCT studies being included in the final sample (Fig. 1).

Fig. 1.

Flowchart for study selection.

Each of the included studies contained a sample size of over 50 patients, with most containing a range of 101 to 250 patients (14/28, 50%). Studies were predominantly conducted at multiple centers (15/28, 53.6%). A total of 19 studies provided a funding statement (19/28, 67.9%), of which private, nonindustry funding was the most common source (9/19, 47.4%). Only 5 studies mentioned adherence to CONSORT guidelines (5/28, 17.9%). Each study examined surgical interventions except for 1 (27/28, 96.4%). Finally, the included studies were conducted in 13 different countries, with the Netherlands being the most common location (6/28, 21.4%). Additional RCT characteristics are included in Supplemental Table 2.

The average number of appropriately reported CONSORT Extension for Harms items across the included studies was 9.32 (9.32/18, 51.8%) (range = 4-14). None of the included trials sufficiently reported all 18 items. The item most successfully adhered to was item 7a (“Provides denominators for [adverse events (AEs)]”) (27/28, 96.4%), followed by items 4a (“Includes harm-associated mode of data collection”), 4b (“Includes harm-associated timing of data collection”), and 5 (“Includes plans for presenting and analyzing information on harm”), each of which 26 studies successfully fulfilled (26/28, 92.9%). Zero studies adhered to checklist item 3c (“Mentions use of a validated instrument to measure AE severity”). Only 1 study fulfilled the requirements of item 3b (“Distinguishes between expected and unexpected AEs”). Nine items had compliance on >50% of CONSORT items (9/18, 50%), 3 items had between 20% and 50% compliance (3/18, 16.7%), and 6 items had <20% compliance (6/18, 33.3%) (Supplemental Table 3). The results of the linear regression model indicated that there was no significant improvement observed in CONSORT Harms reporting over time (R² = 0) (Fig. 2).

Fig. 2.

Linear regression model of mean percent adherence to the CONSORT Extension for Harms checklist. CONSORT, Consolidated Standards of Reporting Trials.

Discussion

We sought to assess the quality and comprehensiveness with which the RCTs supporting the AAOS CPG for the treatment of clavicle fractures appropriately reported AEs. We found that AEs were incompletely reported among these studies to a variable degree, as no study fulfilled all of the categories in the CONSORT Extension for Harms checklist. Item 3c (“Mentions use of a validated instrument to measure AE severity”) exhibited the lowest adherence, but given that there is currently no gold standard tool for recording AEs in orthopedic trauma, this may not be an important reflection of overall AE reporting in RCTs. Item 3b (“Distinguishes between expected and unexpected AEs”) demonstrated the second lowest percent adherence (3.6%), indicating a consistent deficit in qualifying the anticipation of certain AEs. With an overall adherence rate of 52% to CONSORT guidelines, it suggests that there may be considerable room for improvement in the way AEs are reported in RCTs concerning clavicle fractures.

Our study was not without limitations. Although the CONSORT Extension for Harms checklist clearly describes the requirements for each item, some items were more subjective than others. We attempted to mitigate this risk of bias by having multiple authors review and grade the RCTs according to a strict definition of each criterion. In addition, we excluded studies of lower-level evidence underpinning the AAOS guideline for the treatment of clavicle fractures, which may introduce a degree of selection bias. However, we chose to assess only the RCTs due to the high level of evidence and regard in which they are held in clinical practice and determined that they should be assessed independently. Finally, while assigning 1 point to each item provides a standardized approach to grading, it may not account for variations in the relative significance and impact of different checklist items on the overall assessment of AE reporting quality.

The result that AEs are incompletely reported is widely corroborated in the orthopedic literature. Upon examining the RCTs supporting the AAOS CPG for hip and knee osteoarthritis, Anderson et al found an overall compliance of 45% to CONSORT guidelines [2]. In the AAOS CPG for rotator cuff injuries, the same authors reported a 32.6% overall adherence [1]. In addition, in a systematic review of RCTs focusing on analgesics after orthopedic surgery, Montané et al found that less than one half of all CONSORT items were adequately reported [18]. Although a trend of insufficient AE reporting is apparent, it is important to recognize that it may not be reflective of the overall study quality. A lower overall CONSORT score does not necessarily imply that AEs were not reported, but it rather indicates that the transparency and comprehensiveness with which they were reported warrant further development. The findings of this study are in agreement with the previously published literature and should continue to encourage surgeons to follow CONSORT guidelines to improve AE reporting.

Low adherence to item 3b (“Distinguishes between expected and unexpected AEs”), of which only 1 RCT successfully fulfilled, may have important clinical relevance. Given the continued debate between operative and nonoperative management in the treatment of clavicle fractures, it is important for both physicians and patients to be aware of potential sequelae, so as to make an informed treatment decision. Although AEs are inevitable in the practice of orthopedic surgery, it is essential to distinguish certain AEs that might be more expected from a specific operation, as preparing patients for possible complications is often associated with increased reporting of AEs [25]. Although it is possible that all AEs were unexpected, each study should clarify that information within the manuscript.

To strengthen the quality of RCTs used to develop evidence-based guidelines, we recommend that journals require CONSORT adherence in RCTs as a prerequisite for submission. Currently, there is no consistent validated instrument or scale to measure AE severity for clavicle fracture treatment, evident in that none of the RCTs fulfilled item 3c in the CONSORT checklist and only 1 study reported unexpected versus expected AEs. One such method to report AEs is the use of electronic health record (EHR)-based triggers to detect these events after surgery [5]. Severity of outcomes can also be weighted to differentiate between outcomes of different clinical significance [4]. Similarly, the use of validated tools such as the Global Trigger Tool [6] and Orthopedic Surgical Adverse Events Severity System (OrthoSAVES) [17] should be strongly considered, as they have comprehensively reported a greater number of AEs in other orthopedic subspecialties [8]. Implementation of these strategies will serve to further standardize research criteria and strengthen the CPGs which cite RCTs as supporting evidence.

In conclusion, AEs are inadequately reported among the RCTs cited as supporting evidence for the AAOS Treatment of Clavicle Fractures CPGs. We recommend the use of standardized AE reporting tools specific to orthopedic trauma literature to facilitate ease in AE reporting among RCTs. In addition, we advise orthopedic trauma journals to encourage or require CONSORT adherence as a prerequisite for submission, as adherence to these guidelines may result in more accurate representations of the efficacy and risks of an intervention.

Supplemental Material

sj-docx-1-hss-10.1177_15563316231222484 – Supplemental material for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures

Supplemental material, sj-docx-1-hss-10.1177_15563316231222484 for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures by Ashley A. Thompson, Avinash Iyer, Jacob L. Kotlier, Cory K. Mayfield, Frank A. Petrigliano and Joseph N. Liu in HSS Journal®

Supplemental Material

sj-docx-2-hss-10.1177_15563316231222484 – Supplemental material for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures

Supplemental material, sj-docx-2-hss-10.1177_15563316231222484 for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures by Ashley A. Thompson, Avinash Iyer, Jacob L. Kotlier, Cory K. Mayfield, Frank A. Petrigliano and Joseph N. Liu in HSS Journal®

Supplemental Material

sj-docx-3-hss-10.1177_15563316231222484 – Supplemental material for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures

Supplemental material, sj-docx-3-hss-10.1177_15563316231222484 for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures by Ashley A. Thompson, Avinash Iyer, Jacob L. Kotlier, Cory K. Mayfield, Frank A. Petrigliano and Joseph N. Liu in HSS Journal®

Supplemental Material

sj-docx-4-hss-10.1177_15563316231222484 – Supplemental material for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures

Supplemental material, sj-docx-4-hss-10.1177_15563316231222484 for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures by Ashley A. Thompson, Avinash Iyer, Jacob L. Kotlier, Cory K. Mayfield, Frank A. Petrigliano and Joseph N. Liu in HSS Journal®

Supplemental Material

sj-docx-5-hss-10.1177_15563316231222484 – Supplemental material for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures

Supplemental material, sj-docx-5-hss-10.1177_15563316231222484 for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures by Ashley A. Thompson, Avinash Iyer, Jacob L. Kotlier, Cory K. Mayfield, Frank A. Petrigliano and Joseph N. Liu in HSS Journal®

Supplemental Material

sj-docx-6-hss-10.1177_15563316231222484 – Supplemental material for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures

Supplemental material, sj-docx-6-hss-10.1177_15563316231222484 for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures by Ashley A. Thompson, Avinash Iyer, Jacob L. Kotlier, Cory K. Mayfield, Frank A. Petrigliano and Joseph N. Liu in HSS Journal®

Supplemental Material

sj-docx-7-hss-10.1177_15563316231222484 – Supplemental material for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures

Supplemental material, sj-docx-7-hss-10.1177_15563316231222484 for Harms Reporting in the RCTs Underpinning the AAOS Clinical Practice Guidelines for Clavicle Fractures by Ashley A. Thompson, Avinash Iyer, Jacob L. Kotlier, Cory K. Mayfield, Frank A. Petrigliano and Joseph N. Liu in HSS Journal®

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Human/Animal Rights

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2013.

Informed Consent

Informed consent was not required for this study.

Required Author Forms

Disclosure forms provided by the authors are available with the online version of this article as supplemental material.

ORCID iD

Frank A. Petrigliano

Supplemental Material

Supplemental material for this article is available online.

References

Anderson

Howard

Hardin

, et al. Harms-related data are poorly reported among randomized controlled trials underpinning the American Academy of Orthopaedic Surgeons clinical practice guideline recommendations for rotator cuff injuries. J Shoulder Elbow Surg. 2022;31:e620–e627. https://doi.org/10.1016/j.jse.2022.06.021.

Anderson

Howard

Staggs

, et al. Inadequate reporting of complications in randomized controlled trials cited as supporting evidence underpinning AAOS CPG recommendations for hip and knee osteoarthritis: application of the CONSORT harms checklist. J Arthroplasty. 2022;37(12):2466–2472e2. https://doi.org/10.1016/j.arth.2022.06.004.

Bafeta

Koh

Riveros

Ravaud

Harms reporting in randomized controlled trials of interventions aimed at modifying microbiota: a systematic review. Ann Intern Med. 2018;169(4):240–247.

Bohl

Ahn

Lukasiewicz

, et al. Severity weighting of postoperative adverse events in orthopedic surgery. Am J Orthop (Belle Mead NJ). 2017;46(4):E235–e243.

Calem

Horan

Taylor

, et al. Use of triggers to detect adverse events after outpatient orthopedic surgery at a single ambulatory surgery center. Orthopedics. 2022;45(2):116–121. https://doi.org/10.3928/01477447-20220105-07.

Classen

Resar

Griffin

, et al. “Global trigger tool” shows that adverse events in hospitals may be ten times greater than previously measured. Health Affairs. 2011;30(4):581–589.

D’Amico

Bodin

Delpech

Noteborn

Prisma. In D’Errico

, ed. Distributed Space Missions for Earth System Monitoring. Cham: Springer; 2013:599–637.

Garland

Chen

Poitras

, et al. Capturing adverse events in elective orthopedic surgery: comparison of administrative, surgeon and reviewer reporting. Can J Surg. 2020;63(1):E35–E37.

Han

Quan

Fang

Jin

Huang

Treatment of Neer IIb distal clavicle fractures using anatomical locked plate fixation with coracoclavicular ligament augmentation. J Hand Surg Am. 2017;42(12):1036.e1–1036.e6. https://doi.org/10.1016/j.jhsa.2017.08.005.

10.

Hopper

Heusdens

CHW

Dossche

Mackay

GM.

Posterolateral corner repair with suture tape augmentation. Arthroscopy Techniques. 2018;7(12):e1299–e1303. https://doi.org/10.1016/j.eats.2018.08.018.

11.

Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Clinical Practice Guidelines We Can Trust ( Graham

Mancher

Miller Wolman

Greenfield

Steinberg

, et al., eds.). Washington, DC: National Academies Press; 2011.

12.

Ioannidis

Evans

Gøtzsche

, et al. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med. 2004;141(10):781–788. https://doi.org/10.7326/0003-4819-141-10-200411160-00009.

13.

Jonsson

Alaie

Parling

Arnberg

FK.

Reporting of harms in randomized controlled trials of psychological interventions for mental and behavioral disorders: a review of current practice. Contemporary Clinical Trials. 2014;38(1):1–8. https://doi.org/10.1016/j.cct.2014.02.005.

14.

Kabisch

Ruckes

Seibert-Grafe

Blettner

Randomized controlled trials: part 17 of a series on evaluation of scientific publications. Dtsch Arztebl Int. 2011;108(39):663–668. https://doi.org/10.3238/arztebl.2011.0663.

15.

Kamineni

Doria-Rose

Chubak

, et al. Evaluation of harms reporting in U.S. cancer screening guidelines. Ann Intern Med. 2022;175(11):1582–1590.

16.

Melean

Zuniga

Marsalli

, et al. Surgical treatment of displaced middle-third clavicular fractures: a prospective, randomized trial in a working compensation population. J Shoulder Elbow Surg. 2015;24(4):587–592. https://doi.org/10.1016/j.jse.2014.11.041.

17.

Millstone

Perruccio

Badley

Rampersaud

YR.

Factors associated with adverse events in inpatient elective spine, knee, and hip orthopaedic surgery. JBJS. 2017;99(16):1365–1372.

18.

Montané

Vallano

Vidal

Aguilera

Laporte

JR.

Reporting randomised clinical trials of analgesics after traumatic or orthopaedic surgery is inadequate: a systematic review. BMC Clin Pharmacol. 2010;10:2. https://doi.org/10.1186/1472-6904-10-2.

19.

Murad

Wang

Guidelines for reporting meta-epidemiological methodology research. BMJ Evid: Based Med. 2017;22:139–142.

20.

Kim

Lee

Shin

Gong

HS.

Treatment of distal clavicle fracture: a systematic review of treatment modalities in 425 fractures. Arch Orthop Trauma Surg. 2011;131(4):525–533. https://doi.org/10.1007/s00402-010-1196-y.

21.

Robinson

CM.

Fractures of the clavicle in the adult: epidemiology and classification. J Bone Joint Surg Br. 1998;80(3):476–484. https://doi.org/10.1302/0301-620x.80b3.8079.

22.

Robinson

Court-Brown

McQueen

Wakefield

AE.

Estimating the risk of nonunion following nonoperative treatment of a clavicular fracture. J Bone Joint Surg Am. 2004;86(7):1359–1365. https://doi.org/10.2106/00004623-200407000-00002.

23.

Singh

Schultzel

Fleming

Navarro

RA.

Complications after surgical treatment of distal clavicle fractures. Orthop Traumatol Surg Res. 2019;105(5):853–859. https://doi.org/10.1016/j.otsr.2019.03.012.

24.

Smekal

Irenberger

Attal

Oberladstaetter

Krappinger

Kralinger

Elastic stable intramedullary nailing is best for mid-shaft clavicular fractures without comminution: results in 60 patients. Injury. 2011;42(4):324–329. https://doi.org/10.1016/j.injury.2010.02.033.

25.

Stephens

MDB

Talbot

The Detection of New Adverse Drug reactions. Cham: Springer; 1985.

26.

van der Meijden

Gaskill

Millett

PJ.

Treatment of clavicle fractures: current concepts review. J Shoulder Elbow Surg. 2012;21(3):423–429. https://doi.org/10.1016/j.jse.2011.08.053.

27.

Weber

KL.

The AAOS Clinical Practice Guidelines. JAAOS: J Am Acad Orthop Surg. 2009;17(6):335–336.

28.

Woolf

Grol

Hutchinson

Eccles

Grimshaw

Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318(7182):527–530. https://doi.org/10.1136/bmj.318.7182.527.

29.

Zhang

Huang

Luo

Sun

Comparison of the efficacy of a distal clavicular locking plate versus a clavicular hook plate in the treatment of unstable distal clavicle fractures and a systematic literature review. Int Orthop. 2014;38(7):1461–1468. https://doi.org/10.1007/s00264-012340-z.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB