Abstract
Background:
Anterior cruciate ligament (ACL) reconstruction (ACLR) can be successful in restoring knee stability. However, secondary ACL injury, either through graft failure or contralateral injury, is a known complication and can significantly impact the ability of a patient to return successfully to previous activities.
Purpose:
To develop and internally validate an interpretable machine learning model to quantify the risk of graft failure and contralateral ACL injury in a longitudinal cohort treated with ACLR.
Study Design:
Case-control study; Level of evidence, 3.
Methods:
An established geographic database of >600,000 patients was used to identify patients with a diagnosis of ACL rupture between 1990 and 2016 with a minimum 2-year follow-up. Medical records were reviewed for relevant patient information and 4 candidate machine learning algorithms were evaluated for prediction of graft failure and contralateral ACL injury in patients after ACLR as identified either on magnetic resonance imaging or via arthroscopy. Performance of the algorithms was assessed through discrimination, calibration, and decision curve analysis. Model interpretability was enhanced utilizing global variable importance plots and partial dependence curves.
Results:
A total of 1497 patients met inclusion criteria. Among them, 140 (9.4%) had graft failure and 128 (8.6%) had a contralateral ACL injury after index surgery at a median follow-up of 140.7 months (interquartile range, 77.2-219.2 months). The best performing models achieved an area under the receiver operating characteristics curve of 0.70 for prediction of graft failure and 0.67 for prediction of contralateral ACL injury, outperforming a logistic regression fitted on the identical feature set. Notable predictors for increased risk of graft failure included younger age at injury, body mass index (BMI) <30, return to sports <13 months, initial time to surgery >75 days, utilization of allograft, femoral/tibial fixation with suspension/expansion devices, concomitant collateral ligament injury, and active or former smoking history. Predictors of contralateral ACL injury included greater preoperative pain, younger age at initial injury, BMI <30, active smoking history, initial time to surgery >75 days, history of contralateral knee arthroscopies, and involvement in contact sports.
Conclusion:
Less than 18% of all patients who undergo ACLR should be expected to sustain either a graft failure or contralateral ACL injury. Machine learning models outperformed logistic regression and identified greater preoperative pain, younger age, BMI <30, earlier return to higher activity, and time to surgical intervention >75 days as common risk factors for both graft failure as well as contralateral ACL injury after ACLR. Surgeon-modifiable risk factors for graft failure included allograft and femoral/tibial fixation with a suspension/expansion combination.
Keywords
The anterior cruciate ligament (ACL) is among the most injured ligaments in the knee, with a reported incidence of 1 in every 3500 people in the United States. 19 An estimated 200,000 ACL reconstructions (ACLRs) are performed annually in the United States, and recovery is a significant time and financial commitment on the part of the patients and families. 19 Risks associated with initial injury are well reported and include female sex, high risk sports, and anatomic features of the joint.34,47
However, secondary ACL injury after index surgery, either through graft failure or contralateral injury is less understood. Although known risk factors include younger age at injury, return-to-play timeline, and graft choice in select scenarios,2,3,11,27,40,42,48 the effects of myriad surgeon and patient factors7,13,25 including demographics and comorbidities, anterolateral stabilization procedures, 13 and fixation methods remain areas of active research. 43 This is particularly important given that secondary injuries can significantly impact patient ability to return successfully to previous activities and increase the risk of posttraumatic osteoarthritis. 19 Furthermore, reinjury may be more prevalent than previously assumed, with more recent literature suggesting a rate ranging from 6% to 31%.34,39,49
The need to further delineate the exact causes and risk factors for graft failure and contralateral ACL injury is an appropriate opportunity for the application of machine learning, which has demonstrated improved predictive accuracy compared with traditional statistics. 29 Reliable and clinically relevant models have the potential to not only provide real-time quantifiable risk for counseling but also identify new surgeon-modifiable risk factors for timely intervention. This study aimed to (1) describe the rate of graft failure and contralateral ACL injury from a population-based cohort abstracted from an established geographical database, (2) develop an interpretable machine learning model to assess risk factors for second ACL injuries after ACLR, and (3) validate this model internally to quantify the longitudinal risk of these pathologies. We hypothesized that (1) the epidemiology of graft failure and contralateral injury from our cohort would be similar to literature values, (2) machine learning algorithms would outperform logistic regression in its predictive performance, and (3) the algorithms would identify femoral and tibial fixation methods as well as preoperative pain as risk factors for graft failure and contralateral ACL injury.
Methods
Guidelines
After data extraction and institutional review board approval, this analysis was carried out adherent to the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines and the Guidelines for Developing and Reporting Machine Learning Models in Biomedical Research.3,24
Data Source
A query of the Rochester Epidemiology Project (REP) was undertaken for patients who experienced ACL rupture between January 1, 1990, and July 31, 2016, utilizing the appropriate International Classification of Diseases (ICD)-9/10 codes. The REP has been extensively described previously and has long served as an established warehouse of longitudinal medical records for population-based epidemiological and cohort-based studies. 23 The REP contains more than half a million medical records of patients who are residents in Olmsted County, Minnesota and its neighboring counties in southern Minnesota and northern Wisconsin. The warehoused data include documentation of all health care encounters of county residents since 1966. After initial patient identification utilizing ICD-9/10 codes, manual chart review was undertaken of individual medical records to confirm the diagnosis, defined as a documented diagnosis of an ACL injury established by a treating orthopaedic surgeon via advanced imaging or arthroscopy. The inclusion criteria consisted of (1) patients with a primary partial or complete ACL rupture during the study period, (2) patients who underwent ACLR, (3) patients with a minimum of 2-year follow-up, and (4) patients who gave consent for research. Patients undergoing revision ACLR or concomitant ligamentous reconstruction/repair and/or osteotomies and patients with a previous contralateral ACL rupture were excluded (Figure 1).
Flowchart showing patient selection. ACL, anterior cruciate ligament.
Variables
Additional manual chart review was performed after patient inclusion to obtain demographic, injury, and treatment characteristics considered for feature selection. These variables included age; sex; body mass index (BMI); sport participation; a diagnosis of hypermobility or malalignment, which was confirmed only if clinically documented by the treating orthopaedic surgeon; radiographic findings; management; and the diagnosis of interest. For the graft failure model, this was defined as subsequent graft failure confirmed by arthroscopy or magnetic resonance imaging (MRI); meanwhile, this was defined as a subsequent contralateral ACL injury confirmed by arthroscopy or MRI scan for the contralateral ACL injury model.
Missing Data
Missing data are an often encountered and increasingly recognized phenomenon. 12 The missForest multiple imputation method was used to impute variables with <35% missing data,18,44 and variables were assumed to be missing-at-random based on epidemiological convention.28,30 A list of features with number of missing cases is provided in Table 1. We considered both complete-case analysis as well as imputation for features with missing data and ultimately elected imputation to reduce bias and improve statistical power 17 ; multiple imputation has been shown to be appropriate for the management of both missing-at-random and missing completely at random data. 46
Baseline Demographics and Injury Characteristics a
ACL, anterior cruciate ligament; BMI, body mass index; DM, diabetes mellitus; IQR, interquartile range; LCL, lateral collateral ligament; MCL, medial collateral ligament; PLC, posterolateral corner; PT, physical therapy; VAS, visual analog scale. Dashes indicate no missing data.
Outcome and Analysis
For construction of the supervised machine learning prediction models, the primary outcomes assessed were binary outcomes for graft failure and contralateral ACL injury, respectively. After imputation of missing data, we identified highly collinear variables for exclusion (defined as Spearman correlation coefficients >0.75 or those considered clinically confounding by the senior authors (C.L.C., K.R.O.)). Feature selection was then performed utilizing recursive feature elimination with random forest algorithms to select the collection of input features based on discrimination as measured by area under the receiver operating characteristics curve (AUROC).
Modeling
Selected features were then input into the modeling workflow to train each of the following candidate algorithms: extreme gradient boosted machine; support vector machine with radial kernel, random forest, elastic net penalized regression; and a simple generalized linear model (GLM), otherwise known as a logistic regression. If none of the candidate algorithms reached an acceptable AUROC as measured by discrimination, a nonlinear meta-ensemble of the candidate models was generated to boost performance. The GLM was considered the baseline model given that it is the most utilized method for statistical learning and inference in orthopaedic research.
As previously described, 37 models were trained through 10-fold cross validation repeated 3 times to fit the appropriate hyperparameters and validated via 0.632 bootstrapping with 1000 resampled datasets. In brief, model evaluation consists of random partitions of the complete dataset into train and test sets for 1000 repetitions, the evaluation metrics are recorded for each repetition, and overall metrics are summarized with a standard distribution of values. Compared with internal validation through train-test split, resampling through bootstrapping has been found to optimize both model bias and variance and improve overall performance. 45 The optimal model was chosen based on the AUROC. Models were compared by discrimination, calibration, and Brier score value.
Briefly, an AUROC of 0.60 to 0.70 is considered fair, 0.70 to 0.80 is considered good, and 0.80 to 0.90 is considered excellent, according to the precedent established by Hosmer et al, 16 with >0.70 deemed an acceptable discriminative performance for a machine learning model. Calibration of the model's predicted probabilities as a function of observed frequencies in the test population are summarized in a calibration plot. Ideal calibration is a straight line with intercept of 0 and slope of 1 (ie, perfect concordance of model predictions to observed frequencies in the retrospective data). Brier score is the mean squared difference between predicted probabilities of models and observed outcomes; a Brier score is calculated for each model based on the predicted probabilities during resampling. Finally, a null model Brier score is calculated by assigning a class probability equal to the sample prevalence of the outcome for every prediction. A model is considered useful if it has a lower Brier score than the null model.
We then evaluated the clinical utility of the algorithms via decision curve analysis—a tool for assessing the effectiveness of the prediction model in informing decision-making, such as whether utilizing the model provides a significant advantage over a fixed strategy. The decision curve plots net benefit against the predicted probabilities of each outcome and provides the cost-benefit ratio for every value of the predicted probability. Curves were created for the best performing model and the base model (GLM), as well as for the default strategies of changing management for no patients or all patients, and are plotted for comparison purposes. The area under the curve represents the net benefit at each cost-benefit ratio threshold to illustrate the potential gains from utilizing either model in clinical practice.
Finally, model interpretability was enhanced globally utilizing variable importance plots and partial dependence curves. The global model variable importance plot demonstrates variable importance (a unitless, model-agnostic measure that describes the contribution of each feature to model predictive performance) normalized against the input considered most contributory to the model predictive power. Partial dependence plots were generated for each input feature to illustrate the risk of graft failure/contralateral ACL injury (y-axis) as a function of perturbations in each input variable (x-axis) while other feature values are held constant. Categorical/ordinal variables were visualized using bar plots while continuous variables were visualized with scatterplots and a best-fit curve.15,38
Results
Variable Breakdown
A total of 2349 patients had diagnosed ACL injury between 1990 and 2016 with a minimum 2-year follow-up. Of these, 1497 operatively treated patients (63.7%) met inclusion criteria for analyses. The mean age at time of injury was 25 years (interquartile range [IQR], 18-34 years; Table 1) and 59% (n = 875) were male. Mean BMI of the cohort was 26.5 (IQR, 23.8-29.5). The cohort was mostly nonsmokers and recreational athletes (76.5% and 62.4%, respectively). Additional demographic details are presented in Table 1.
Most patients experienced ACL injury due to noncontact events (66.9%), which occurred in the midsubstance of the ligament (59%) and were full thickness in nature (94.4%; Table 1). The most common concomitant injury was a meniscal tear, occurring in approximately two-thirds of patients, whereas approximately 17% had concomitant MCL injury (Table 1). Most reconstructions utilized bone-patellar tendon-bone autograft (57.7%). Meniscectomy was the most common concomitant procedure (38.0%; Table 2). A total of 140 (9.4%) patients experienced graft failure while 128 (8.6%) suffered a contralateral ACL injury after primary ACLR.
Intraoperative Characteristics a
ALL, anterolateral ligament; ITB, iliotibial band; LET, lateral extra-articular tenodesis. Dashes indicate no missing data.
Direct compression devices included interference screws, screw-washer fixation, press-fit fixation, and staples
Expansion devices included Crosspin and Rigidfix devices
Suspension devices defined as Tightrope, Endobutton, Retrobutton, knotless anchors.
Model Performance
The gradient boosting ensemble models demonstrated the best performance on internal validation for prediction of graft failure and contralateral ACL injury. For the prediction of graft failure, the model achieved an AUROC of 0.702 (95% CI, 0.688-0.723), a calibration slope of 0.94 (95% CI, 0.93-0.95), a calibration intercept of 0.0074 (95% CI, 0.0061-0.0087), and a Brier score of 0.071 (95% CI, 0.061-0.141) (Figure 2, A and B, and Table 3). For prediction of contralateral ACL injury, the model achieved an AUROC of 0.666 (95% CI, 0.664-0.667), a calibration slope of 0.87 (95% CI, 0.86-0.88), a calibration intercept of 0.121 (0.1139-0.128), and a Brier score of 0.078 (0.058-0.098) (Figure 3, A and B, and Table 4). Comparisons with the null Brier score confirmed overall clinical utility of both the graft failure and contralateral ACL injury models. Decision curve analysis identified a significant benefit of utilizing the best-performing model over the baseline logistic regression model for prediction of both graft failure and contralateral ACL injury (Figures 2C and 3C).
Performance of the graft failure model as measured by (A) area under the receiver operating characteristics curve; (B) calibration, which plots the sample prevalence of positive cases at every predicted probability; (C) decision curve analysis demonstrating net benefit generated from using the model compared with a trained logistic regression model; and (D) global relative variable importance. This plot demonstrates the statistical significance of each variable in the data with respect to its effect on the generated model, as measured on a unitless scale of 0 to 100. AUC, area under the curve; LCL, lateral collateral ligament; MCL, medial collateral ligament; RTS, return to sports; VAS, visual analog scale.
Assessment for Model to Predict Graft Failures on Internal Validation (n = 1497) a
Using 0.632 bootstrapping with 1000 resampled datasets. Null model Brier score, 0.084. AUC, area under the curve; SVM, support vector machine; XGBoost, extreme gradient boosted machine. All data presented as mean (95% CI).
Performance of the contralateral ACL injury cohort model performance as measured by (A) area under the receiver operating characteristics curve; (B) calibration, which plots the sample prevalence of positive cases at every predicted probability; (C) decision curve analysis demonstrating net benefit generated from using the model compared with a trained logistic regression model; and (D) global relative variable importance. This plot demonstrates the statistical significance of each variable in the data with respect to its effect on the generated model, as measured on a unitless scale of 0 to 100. ACL, anterior cruciate ligament; AUC, area under the curve; BMI, body mass index; RTS, return to sports.
Assessment for Model to Predict Contralateral ACL Ruptures on Internal Validation (n = 1497) a
Using 0.632 bootstrapping with 1000 resampled datasets. Null model Brier score, 0.08. ACL, anterior cruciate ligament; AUC, area under the curve; SVM, support vector machine; XGBoost, extreme gradient boosted machine. All data presented as mean (95% CI).
Interpretability
Global variable importance analysis was performed to identify risk factors that predicted graft failure and contralateral ACL injury, respectively. Partial dependence curves were utilized to illustrate the corresponding change in predicted probability of the outcome of interest with changing input. Based on this analysis, the greatest risk for graft failure was conferred by the following (in order of decreasing importance): greater preoperative pain scores, return to sports (RTS)/activity <375 days, BMI <30, time to surgery >75 days, younger age at injury, tibial fixation with an expansion device, tear off the femoral insertion, participation in contact sports, femoral fixation with a suspension device, use of an allograft, a noncontact initial injury, smoking history, previous non-ACLR arthroscopic knee surgery, and concomitant medial collateral ligament/lateral collateral ligament injuries (Figure 2D and Figure 4). For contralateral ACL injury, the greatest risks were conferred by the following (in order of decreasing importance): BMI <30, increased time to surgery for the index ACLR, RTS/activity <500 days after initial injury, younger age at initial injury, participation in a contact sport, utilization of an allograft for the index ACLR, participation in a limited-contact sport, index injury in the midsubstance of the graft, current or former smoking history, noncontact index injury, a history of multiple arthroscopies on the contralateral knee, and participation in a collision sport (Figures 3D and Figure 5).
Partial dependence curves demonstrating individual predictor effects on graft failure in patients after ACLR; the dependence of graft failure (a relative, unitless quantity) on the predictor is plotted on the y-axis while the values of each predictor variable are plotted on the x-axis. For example, with respect to preoperative pain, the dependence of graft failure increases as pain score increases. Tear location: 1, distal; 2, femoral avulsion; 3, midsubstance; 4, proximal; 5, tibial avulsion. Graft type: 1, allograft; 2, hamstring tendon autograft; 3, patellar tendon autograft; 4, quadriceps tendon autograft. ACLR, anterior cruciate ligament reconstruction; LCL, lateral collateral ligament; MCL, medial collateral ligament; RTS, return to sports; VAS, visual analog scale.
Partial dependence curves demonstrating individual predictor effects on contralateral ACL injury in patients after ACLR. The dependence of contralateral ACL injury (a relative, unitless quantity) on the predictor is plotted on the y-axis while the values of each predictor variable are plotted on the x-axis. For example, with respect to BMI, the dependence of contralateral ACL injury is increased at BMI < 30, then decreases as BMI increases to >30, and finally increasing again at BMI >35. This may suggest that patients with a lower BMI may be more active and prone to contralateral injury, patients with extremely high BMIs can sustain low velocity ligamentous injuries, whereas the risk is the smallest among patients who are sedentary and overweight but not obese. Tear location: 1, distal; 2, femoral avulsion; 3, midsubstance; 4, proximal; 5, tibial avulsion. Graft type: 1, allograft; 2, hamstring tendon autograft; 3, patellar tendon autograft; 4, quadriceps tendon autograft. ACL, anterior cruciate ligament; ACLR, anterior cruciate ligament reconstruction; BMI, body mass index; RTS, return to sports.
Discussion
Failure of ACL graft or contralateral ACL injury after index surgery remains a concern for patients undergoing ACLR. In our cohort, 9.4% experienced graft rupture and 8.6% had a contralateral ACL injury; models implemented in this study had an AUROC of 0.70 for prediction of graft failure and 0.67 for prediction of contralateral injury, outperforming traditional logistic regression modeling. Machine learning also allowed for the identification of predictors for causes of subsequent injury in a longitudinal cohort. Potential predictors of graft failure included greater preoperative pain, younger age at injury, tobacco usage, BMI <30, earlier RTS, delayed time to surgery, concomitant collateral ligament injury, usage of an allograft and suspension or expansion fixation devices. Potential predictors of contralateral ACL injury included younger age at initial injury, BMI <30, tobacco usage, delayed time to surgery, and involvement in contact sports.
The overall rate for secondary ACL rupture after index surgery is variable, with estimates in the current literature ranging from 2% to 20% in adults and 3% to 21% in adolescents.2,9,47 Our observation is consistent with the median value provided by the extant literature. Several of the identified risk factors are similarly corroborated by the current evidence. Failure has been associated with younger age, contact mechanism of initial ACL injury, decreased BMI, and earlier RTS.13,14,25,40,47,48 Webster et al 47 found returning to cutting/pivoting sports to confer 3.9 times the risk of graft failure in their under 20 cohort. In their meta-analysis, Wiggins et al 48 showed a graft-failure rate between 7% and 14% in patients under 25 years of age. However, a systematic review by Everhart et al in 2021 found mixed associations between graft failure and younger age. 9 Despite continued controversy on the subject matter, our data demonstrated a significant association between increased risk of ACL graft rupture with both younger age at initial injury, return to contact sports, decreased BMI, and delayed ACLR. While the protective effect of increased BMI against graft failure and contralateral injury may seem initially counterintuitive, Maletis et al 25 reported similar findings in the Kaiser Permanente ACLR Registry and suggested that patients at higher BMIs may have less demanding activity/sports participation that would put them at risk of increased cutting/pivoting after reconstruction. In addition, delayed ACLR is associated with an increase in subsequent meniscal injury, 23 and it is possible that disruption or delayed repair of secondary stabilizers can increase the risk of future graft failure.
The increased risk of graft failure associated with allograft reconstruction has been frequently cited as well. The Multicenter Orthopaedic Outcomes Network (MOON) study group identified an almost 5-fold increase in the risk of graft rupture after allograft reconstruction compared with autograft reconstruction; the authors also identified younger age at injury as well as increased baseline activity as measured by the Marx activity rating scale as risk factors for graft failure. 19 The authors fitted 2 multivariate logistic regression models, which achieved AUROCs of 0.81 and 0.76 in predicting graft failure and contralateral ACL rupture, respectively. However, they did not perform internal validation of these models and reported training AUROCs only.
In addition, we did observe a difference in graft failure risk associated with femoral fixation device type, with direct compression devices such as interference screws conferring the least risk followed by expansion and suspension type devices in that order, confirming the findings of both the Kaiser and the Danish registries.10,43 Comparison of tibial fixation methods observed the greatest failure risk with use of expansion devices, while direct compression and suspension devices were similar in risk. The literature likewise reports equivocal results among tibial fixation devices.1,42,43 Several reasons may underlie the observed influence of femoral and tibial fixation device on graft failure. With respect to femoral fixation, the advantage of direct compression devices with bone-tendon-bone grafts cited most often is the reliability of bone-to-bone healing afforded by the aperture fixation. 50 However, in recent years, the emergence of adjustable-length cortical devices has led some surgeons to move toward suspensory femoral fixation of bone-tendon-bone grafts, 21 with the theoretical advantage of preserved opportunity to retension the graft from the femoral side after tibial fixation in the setting of and graft length mismatch. Limited data from clinical and biomechanical evaluations comparing cortical fixation versus interference screw have been inconclusive. Specifically, 2 animal models found increased graft-bone healing with cortical loops, whereas a retrospective review of outcomes from the New Zealand ACLR registry found higher revision risk with interference fixation of hamstring tendon autograft22,36,41; conversely, a retrospective review of patients undergoing allograft reconstruction found increased graft incorporation with interference fixation. 51 With respect to tibial fixation, several studies have highlighted reduced revision risk and superior biomechanical properties using interference screws,1,31 although 1 series did note increased tunnel widening in the first 2 postoperative years when comparing these with cortical fixation. 8 Finally, a large retrospective cohort review using the Norwegian Knee Ligament Registry identified the combination of femoral suspensory fixation with tibial bioabsorbable compression fixation to incur the greatest revision risk. 32
In addition, we must point out that historical association between fixation methods and graft type may subject these findings to confounding. Indeed, aperture fixation is most frequently performed with bone-tendon-bone grafts, which has repeatedly demonstrated superior failure rates compared with hamstring tendon autografts or allografts.18,27,50
We also observed an association between preoperative pain score with graft failure. Although there is ample evidence highlighting a relationship between preoperative patient-reported outcome scores with subjective failure after ACLR, there has been minimal evidence pointing to a potential relationship between patient-reported outcome measures and clinical outcomes. Interestingly, Pullen et al 35 observed a greater risk of revision ACLR with perioperative use of nonsteroidal anti-inflammatory drugs and COX-2 inhibitors and cited several animal models highlighting a negative effect of nonsteroidal anti-inflammatory drugs use on bone-tendon healing. 6 In addition, a previous model using the Norwegian registry similarly identified preoperative Knee injury and Osteoarthritis Outcome Score quality of life subscale as a significant predictor of revision ACL. 26 These findings may point to the extent of patient-modifiable psychosocial influences or, more likely, the potential influence of increased perioperative pain and subsequent medication use on graft healing.
Rates of contralateral ACL injury after initial ACLR has been reported anywhere from 2% to 21%.4,47,49 Pooled data from a multicenter analysis demonstrated a fairly low rate of contralateral injury, with only 3% of 235 patients suffering rupture of contralateral ACL at 2-year follow-up. 49 Webster et al 47 demonstrated an overall risk of 8% for contralateral ACL rupture after index surgery. We observed an 8.6% rate of injury to the contralateral ACL, consistent with recent estimates. Similarly, most predictors of contralateral ACL injury identified in the present study are corroborated by the literature, with an unsurprising degree of overlap with factors contributing to graft failure. These include younger age at initial injury, lower BMI, earlier return to higher activity level, history of contralateral ACL injury, and allograft utilization.5,19,25,33
Machine learning represents a novel methodology for analyses of complex, large datasets. 29 Implementation of these techniques for the analysis of clinical and subjective outcomes after ACLR has previously demonstrated promising results.20,23,26 In addition, our definitions of graft failure and contralateral ACL disruption were confirmed by either repeat MRI or arthroscopy, which is more rigorous compared with those found in national registries. Nevertheless, we were only able to achieve acceptable concordance results for the graft failure and contralateral ACL injury models. Interestingly, Martin et al 26 demonstrated similar bottleneck on discriminative performance at 0.70 despite a sample size of nearly 25,000 patients. This highlights the presence of additional patient or surgeon risk factors that continue to elude our understanding. Finally, future studies may require cost-sensitive learning or anomaly detection techniques to improve performance given significant class imbalance between the groups.
Limitations
This study does have several limitations, including its retrospective nature. While the models produced demonstrated significant improvements in clinical practice compared with traditional logistic regression based on Brier score and decision curve analysis, overall concordance was fair to below the ideal threshold—AUROC of 0.70 and 0.67 for graft failure and contralateral injury, respectively. In terms of treatment, techniques of reconstruction as well as postoperative rehabilitation was not strictly uniform, especially given the length of the study period, with multiple surgeons performing the index surgery and variations in graft and fixation types utilized. Similarly, given the longevity of the study, some patients probably did not participate in optimizing interventions such as preoperative physical therapy. Furthermore, this study did not include radiographic analysis for risk factors such as malalignment or posterior tibial slope; indeed, collection of such parameters as well as psychosocial readiness and neuromuscular control could provide the performance boost necessary to make the present models more suitable for clinical application.
Conclusion
Less than 18% of all patients who undergo ACLR should be expected to sustain either a graft failure or contralateral ACL injury. Machine learning models outperformed logistic regression and identified greater preoperative pain, younger age, BMI <30, earlier return to greater activity, and time to surgical intervention >75 days as common risk factors for both graft failure as well as contralateral ACL injury after ACLR. Surgeon-modifiable risk factors for graft failure included allograft and femoral/tibial fixation with a suspension/expansion combination.
Footnotes
Final revision submitted March 2, 2024; accepted April 2, 2024.
One or more of the authors has declared the following potential conflict of interest or source of funding: J.R.L. has received hospitality payments from Medical Device Business Services and Stryker. M.H. has received education payments from Arthrex, Foundation Medical, Medwest Associates and Smith & Nephew; honoraria from Encore Medical; hospitality payments from Stryker and Medical Device Business Services; and consulting fees from Vericel. A.J.K. has received nonconsulting fees from Arthrex; consulting fees from Arthrex, JRF Ortho, and Responsive Arthroscopy; royalties from Arthrex and Responsive Arthroscopy; a grant from DJO; and honoraria from JRF Ortho and MTF Biologics. C.L.C. has received consulting fees from Arthrex; nonconsulting fees from Arthrex; and education payments from Arthrex. K.R.O. has received a grant from Arthrex; education payments from Arthrex, Endo Pharmaceuticals, Pinnacle, Medwest Associates, Smith & Nephew, Foundation Medical, and Gemini Medical; hospitality payments from Arthrex, Medical Device Business Services, Stryker, Wright Medical Technology, and Zimmer Biomet Holdings; nonconsulting fees from Smith & Nephew and Arthrex; and consulting fees from Endo Pharmaceuticals and Smith & Nephew. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
Ethical approval for this study was obtained from Mayo Clinic (PR14-005089-02).