The Fragility Index of Statistically Significant Findings From Randomized Controlled Trials Comparing the Management Strategies of Anterior Shoulder Instability
Restricted accessResearch articleFirst published online July, 2023
The Fragility Index of Statistically Significant Findings From Randomized Controlled Trials Comparing the Management Strategies of Anterior Shoulder Instability
Debate centering on the management of anterior shoulder instability (ASI) in recent years has led to many randomized controlled trials (RCTs) being published on the topic. The fragility index (FI) has subsequently emerged as a novel method of assessing significant findings reported in RCTs, particularly those with small sample sizes.
Purpose:
To evaluate the FI of statistically significant findings in RCTs that reported the outcomes of management strategies of patients with ASI.
Study Design:
Systematic review; Level of evidence, 1.
Methods:
Using PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, 2 independent reviewers performed a systematic review of RCTs focusing on the outcomes of management strategies of patients with ASI. There were 3 main categories of RCTs included: (1) nonoperative management in internal rotation (IR) versus external rotation (ER), (2) nonoperative management versus a surgical intervention, and (3) surgical management with arthroscopic Bankart repair versus open Bankart repair. The Fisher exact test was utilized to calculate the FI for the reversal of statistical significance in all statistically significant dichotomous outcomes.
Results:
A total of 21 RCTs were included, including 1589 shoulders (mean age, 29.4 years) with a mean follow–up of 26.8 months. There were 10 RCTs (831 shoulders) that reported outcomes after the nonoperative management of ASI in IR versus ER, with a mean FI of 6.8. There were 5 RCTs (324 shoulders) that reported outcomes comparing the nonoperative and operative management of ASI, with a mean FI of 3.5. There were 6 RCTs (434 shoulders) that reported outcomes after the operative management of ASI with either arthroscopic Bankart repair or open Bankart repair, with a mean FI of 9.6.
Conclusion:
The overall FI of RCTs reporting the outcomes of management strategies for patients with ASI was high, suggesting a moderate fragility of statistically significant outcomes including recurrence, revision stabilization, and return to play.
AcunaSASue-Chue-LamCDossaF. The fragility index: P values reimagined, flaws and all. JAMA Surg. 2019;154(7):674.
2.
AguinisHVassarMWayantC. On reporting and interpreting statistical significance and p values in medical research. BMJ Evid Based Med. 2021;26(2):39-42.
3.
AhmedWFowlerRAMcCredieVA. Does sample size matter when interpreting the fragility index?Crit Care Med. 2016;44(11):e1142-e1143.
4.
AmrheinVGreenlandSMcShaneB. Scientists rise up against statistical significance. Nature. 2019;567(7748):305-307.
5.
Archetti NettoNTamaokiMJLenzaM, et al. Treatment of Bankart lesions in traumatic anterior instability of the shoulder: a randomized controlled trial comparing arthroscopy and open techniques. Arthroscopy. 2012;28(7):900-908.
6.
BottoniCRSmithELBerkowitzMJTowleRBMooreJH. Arthroscopic versus open shoulder stabilization for recurrent anterior instability: a prospective randomized clinical trial. Am J Sports Med. 2006;34(11):1730-1737.
7.
BottoniCRWilckensJHDeBerardinoTM, et al. A prospective, randomized evaluation of arthroscopic stabilization versus nonoperative treatment in patients with acute, traumatic, first–time shoulder dislocations. Am J Sports Med. 2002;30(4):576-580.
8.
ChanSKBentickKRKuiperJHKellyCP. External rotation bracing for first–time anterior dislocation of the shoulder: a discontinued randomised controlled trial comparing external rotation bracing with conventional sling. Shoulder Elbow. 2019;11(4):256-264.
9.
CheckettsJXScottJTMeyerC, et al. The robustness of trials that guide evidence–based orthopaedic surgery. J Bone Joint Surg Am. 2018;100(12):e85.
10.
DaveyMSHurleyETHurleyDJPauzenbergerLMullettH. Magnetic resonance arthrography in rugby players undergoing shoulder stabilization for glenohumeral instability: professionals have higher frequencies of more pathologies. Arthrosc Sports Med Rehabil. 2021;3(2):e543-e547.
11.
DaveyMSHurleyETO’DohertyR, et al. Open Latarjet procedure in athletes following failed prior instability surgery results in lower rates of return to play. Arthroscopy. 2021;37(8):2412-2417.
12.
DickFTevaearaiH. Significance and limitations of the p value. Eur J Vasc Endovasc Surg. 2015;50(6):815.
13.
EvaniewNFilesCSmithC, et al. The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. Spine J. 2015;15(10):2188-2197.
14.
FabbricianiCMilanoGDemontisA, et al. Arthroscopic versus open treatment of Bankart lesion of the shoulder: a prospective randomized study. Arthroscopy. 2004;20(5):456-462.
15.
FinestoneAMilgromCRadeva-PetrovaDR, et al. Bracing in external rotation for traumatic anterior dislocation of the shoulder. J Bone Joint Surg Br. 2009;91(7):918-921.
16.
HarrisJDBrandJCCoteMPFaucettSCDhawanA. Research pearls: the significance of statistics and perils of pooling. Part 1: clinical versus statistical significance. Arthroscopy. 2017;33(6):1102-1112.
17.
HeidariKAsadollahiSVafaeeR, et al. Immobilization in external rotation combined with abduction reduces the risk of recurrence after primary anterior shoulder dislocation. J Shoulder Elbow Surg. 2014;23(6):759-766.
18.
HigginsJPTAltmanDGGøtzschePC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ (Clin Res Ed). 2011;343:D5928.
19.
HurleyETFriedJWAlaiaMJ, et al. Immobilisation in external rotation after first–time traumatic anterior shoulder instability reduces recurrent instability: a meta-analysis. J ISAKOS. 2021;6(1):22-27.
20.
HurleyETJamalMSAliZS, et al. Long-term outcomes of the Latarjet procedure for anterior shoulder instability: a systematic review of studies at 10–year follow-up. J Shoulder Elbow Surg. 2019;28(2):e33-e39.
21.
HurleyETManjunathAKBloomDA, et al. Arthroscopic Bankart repair versus conservative management for first–time traumatic anterior shoulder instability: a systematic review & meta-analysis. Arthroscopy. 2020;36(9):2526-2532.
22.
HurleyETMontgomeryCJamalMS, et al. Return to play after the Latarjet procedure for anterior shoulder instability: a systematic review. Am J Sports Med. 2019;47(12):3002-3008.
23.
HurwitzS. Evidence-based medicine in orthopaedic surgery: a way to the future. Iowa Orthop J. 2003;23:61-65.
24.
ItoiEHatakeyamaYKidoT, et al. A new method of immobilization after traumatic anterior dislocation of the shoulder: a preliminary study. J Shoulder Elbow Surg. 2003;12(5):413-415.
25.
ItoiEHatakeyamaYSatoT, et al. Immobilization in external rotation after shoulder dislocation reduces the risk of recurrence: a randomized controlled trial. J Bone Joint Surg Am. 2007;89(10):2124-2131.
26.
JakobsenBWJohannsenHVSuderPSøjbjergJO. Primary repair versus conservative treatment of first–time traumatic anterior dislocation of the shoulder: a randomized study with 10–year follow-up. Arthroscopy. 2007;23(2):118-123.
27.
KawasakiTOtaCUrayamaS, et al. Incidence of and risk factors for traumatic anterior shoulder dislocation: an epidemiologic study in high–school rugby players. J Shoulder Elbow Surg. 2014;23(11): 1624-1630.
28.
KhanMEvaniewNGichuruM, et al. The fragility of statistically significant findings from randomized trials in sports surgery: a systematic survey. Am J Sports Med. 2017;45(9):2164-2170.
29.
KirkleyALitchfieldRThainLSpougeA. Agreement between magnetic resonance imaging and arthroscopic evaluation of the shoulder joint in primary anterior dislocation of the shoulder. Clin J Sport Med. 2003;13(3):148-151.
30.
KirkleyAWerstineRRatjekAGriffinS. Prospective randomized clinical trial comparing the effectiveness of immediate arthroscopic stabilization versus immobilization and rehabilitation in first traumatic anterior dislocations of the shoulder: long–term evaluation. Arthroscopy. 2005;21(1):55-63.
31.
LiavaagSBroxJIPrippAH, et al. Immobilization in external rotation after primary shoulder dislocation did not reduce the risk of recurrence: a randomized controlled trial. J Bone Joint Surg Am. 2011;93(10):897-904.
32.
MaticsTJKhanNJaniPKaneJM. The fragility index in a cohort of pediatric randomized controlled trials. J Clin Med. 2017;6(8):79.
33.
McCormickKLTedescoLJSwindellHW, et al. Statistical fragility of randomized clinical trials in shoulder arthroplasty. J Shoulder Elbow Surg. 2021;30(8):1787-1793.
34.
MemonMKayJCadetER, et al. Return to sport following arthroscopic Bankart repair: a systematic review. J Shoulder Elbow Surg. 2018;27(7):1342-1347.
35.
MinkusMKönigshausenMMaierD, et al. Immobilization in external rotation and abduction versus arthroscopic stabilization after first–time anterior shoulder dislocation: a multicenter randomized controlled trial. Am J Sports Med. 2021;49(4):857-865.
36.
MohtadiNGChanDSHollinsheadRM, et al. A randomized clinical trial comparing open and arthroscopic stabilization for recurrent traumatic anterior shoulder instability: two–year follow-up with disease–specific quality–of–life outcomes. J Bone Joint Surg Am. 2014; 96(5):353-360.
37.
MurphyAIHurleyETHurleyDJPauzenbergerLMullettH. Long-term outcomes of the arthroscopic Bankart repair: a systematic review of studies at 10–year follow-up. J Shoulder Elbow Surg. 2019;28(11):2084-2089.
38.
MurrayJCLeclercABalatriAPeletS. Immobilization in external rotation after primary shoulder dislocation reduces the risk of recurrence in young patients: a randomized controlled trial. Orthop Traumatol Surg Res. 2020;106(2):217-222.
39.
OwensBDCameronKLPeckKY, et al. Arthroscopic versus open stabilization for anterior shoulder subluxations. Orthop J Sports Med. 2015;3(1):2325967115571084.
40.
ParisienRLConstantMSaltzmanBM, et al. The fragility of statistical significance in cartilage restoration of the knee: a systematic review of randomized controlled trials. Cartilage. 2021;13(1)(suppl):147S-155S.
41.
ParisienRLEhlersCCusanoA, et al. The statistical fragility of platelet–rich plasma in rotator cuff surgery: a systematic review and meta-analysis. Am J Sports Med. 2021;49(12):3437-3442.
42.
RanganathanPPrameshCSBuyseM. Common pitfalls in statistical analysis: clinical versus statistical significance. Perspect Clin Res. 2015;6(3):169-170.
43.
RidgeonEEYoungPJBellomoR, et al. The fragility index in multicenter randomized controlled critical care trials. Crit Care Med. 2016;44(7):1278-1284.
44.
RobinsonCMJenkinsPJWhiteTOKerAWillE. Primary arthroscopic stabilization for a first–time anterior dislocation of the shoulder: a randomized, double–blind trial. J Bone Joint Surg Am. 2008;90(4):708-721.
45.
RuzbarskyJJRauckRCManziJ, et al. The fragility of findings of randomized controlled trials in shoulder and elbow surgery. J Shoulder Elbow Surg. 2019;28(12):2409-2417.
46.
ShochetLRKerrPGPolkinghorneKR. The fragility of significant results underscores the need of larger randomized controlled trials in nephrology. Kidney Int. 2017;92(6):1469-1475.
47.
SperberAHambergPKarlssonJSwärdLWredmarkT. Comparison of an arthroscopic and an open procedure for posttraumatic instability of the shoulder: a prospective, randomized multicenter study. J Shoulder Elbow Surg. 2001;10(2):105-108.
48.
TaşkoparanHKılınçoğluVTunayS, et al. Immobilization of the shoulder in external rotation for prevention of recurrence in acute anterior dislocation. Acta Orthop Traumatol Turc. 2010;44(4):278-284.
49.
TignanelliCJNapolitanoLM. The fragility index in randomized clinical trials as a means of optimizing patient care. JAMA Surg. 2019;154(1):74-79.
50.
WalshMSrinathanSKMcAuleyDF, et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index. J Clin Epidemiol. 2014;67(6):622-628.
51.
WangDYZhangZLZhengYZRuiG. Analysis on the rotation angle and the effect of two immobilization methods after reduction of shoulder dislocation in emergency treatment. New Med. 2011;42:221-224.
52.
WhelanDBLitchfieldRWamboltEDaintyKN. External rotation immobilization for primary shoulder dislocation: a randomized controlled trial. Clin Orthop Relat Res. 2014;472(8):2380-2386.
53.
WidjajaABTranABaileyMProperS. Correlation between Bankart and Hill-Sachs lesions in anterior shoulder dislocation. ANZ J Surg. 2006;76(6):436-438.
