A Novel Machine Learning Model to Predict Revision ACL Reconstruction Failure in the MARS Cohort

Abstract

Background:

As machine learning becomes increasingly utilized in orthopaedic clinical research, the application of machine learning methodology to cohort data from the Multicenter ACL Revision Study (MARS) presents a valuable opportunity to translate data into patient-specific insights.

Purpose:

To apply novel machine learning methodology to MARS cohort data to determine a predictive model of revision anterior cruciate ligament reconstruction (rACLR) graft failure and features most predictive of failure.

Study Design:

Cohort study; Level of evidence, 3.

Methods:

The authors prospectively recruited patients undergoing rACLR from the MARS cohort and obtained preoperative radiographs, surgeon-reported intraoperative findings, and 2- and 6-year follow-up data on patient-reported outcomes, additional surgeries, and graft failure. Machine learning models including logistic regression (LR), XGBoost, gradient boosting (GB), random forest (RF), and a validated ensemble algorithm (AutoPrognosis) were built to predict graft failure by 6 years postoperatively. Validated performance metrics and feature importance measures were used to evaluate model performance.

Results:

The cohort included 960 patients who completed 6-year follow-up, with 5.7% (n = 55) experiencing graft failure. AutoPrognosis demonstrated the highest discriminative power (model area under the receiver operating characteristic curve: AutoPrognosis, 0.703; RF, 0.618; GB, 0.660; XGBoost, 0.680; LR, 0.592), with well-calibrated scores (model Brier score: AutoPrognosis, 0.053; RF, 0.054; GB, 0.057; XGBoost, 0.058; LR, 0.111). The most important features for AutoPrognosis model performance were prior compromised femoral and tibial tunnels (placement and size) and allograft graft type used in current rACLR.

Conclusion:

The present study demonstrated the ability of the novel AutoPrognosis machine learning model to best predict the risk of graft failure in patients undergoing rACLR at 6 years postoperatively with moderate predictive ability. Femoral and tibial tunnel size and position in prior ACLR and allograft use in current rACLR were all risk factors for rACLR failure in the context of the AutoPrognosis model. This study describes a unique model that can be externally validated with larger data sets and contribute toward the creation of a robust rACLR bedside risk calculator in future studies.

Registration:

NCT00625885 (ClinicalTrials.gov identifier).

Keywords

ACL revision graft failure femoral tunnel tibial tunnel machine learning

Revision anterior cruciate ligament reconstruction (rACLR) continues to remain a challenge.⁴² In patients undergoing rACLR, the rate of objective graft failure has been reported to be as high as 13.7%.^7,50 Compared with primary ACLR, rACLR is a more technically demanding procedure with a 3- to 4-fold increased risk of graft failure and poorer functional outcomes.^49-51

The Multicenter ACL Revision Study (MARS) Group, created in 2005,²⁹ as well as other studies, has implicated several predictive factors for outcomes after rACLR. These include technical factors such as graft type, as well as patient factors like preoperative knee hyperextension, younger age, and higher activity level.⁹ Other investigations have implicated technical errors in graft preparation, placement, tensioning, and fixation, as well as anatomic factors like posterior tibial slope, anterolateral rotary instability, and meniscal deficiency.^14,41 Although these findings have guided clinical decision-making, rACLR outcomes appear to be dependent on multiple factors.²⁹ More accurate predictors of graft failure may improve preoperative counseling, operative management decisions, and cost of care.

Machine learning is increasingly utilized in health care to harness insights from clinical data sets (big data) to provide patient-specific predictions of an outcome of interest.³ In the orthopaedic literature, a number of studies on machine learning have built risk calculators with the potential for use in a variety of settings from preoperative counseling to advocacy for appropriate reimbursement based on patient risk.^{22,23,25,32-34} Yet the lack of standardization in machine learning methodology, awareness of machine learning techniques in the orthopaedic community, and data quality limit the clinical utility of these results.^21,24,27

AutoPrognosis is a novel, validated ensemble algorithm that combines the strongest features of traditional statistical approaches and popular algorithms like random forest (RF) and XGBoost to create a single, well-calibrated predictive “supermodel.” AutoPrognosis has been validated in various orthopaedic subspecialty studies, predicting postoperative complications after total hip arthroplasty, total knee arthroplasty, and posterior cervical spinal fusion.^10,39,40 The application of validated machine learning methodology to MARS cohort data presents a valuable opportunity to translate data into patient-specific insights.

In this study, we sought to apply novel machine learning methodology to the MARS cohort data to determine (1) an optimal predictive model of rACLR graft failure and (2) features that are important for the model to be able to accurately predict rACLR graft failure. We hypothesized that the AutoPrognosis model would have the most robust predictive and discriminative ability for the outcome of interest, and important features would span surgical, clinical, radiographic, and patient-reported outcome (PRO) variables.

Methods

Study Participants

MARS Cohort

The MARS Group consists of 83 sports medicine fellowship–trained surgeons at 52 academic and private practice sites (ClinicalTrials.gov: NCT00625885). Institutional review board approval was obtained at all investigating sites. Between 2006 and 2011, 1233 patients undergoing rACLR were prospectively enrolled. Eligible patients underwent rACLR, including a second or greater rACLR; multiligament reconstructions were excluded. Detailed descriptions of the MARS cohort have been previously published.²⁹

All patients within the MARS cohort completed an informed consent form and 13-page questionnaire of baseline patient characteristics, injury and sports participation history, knee surgical history, medical comorbidities, and baseline PROs including the Marx activity rating scale, Western Ontario and McMaster Universities Osteoarthritis Index, Knee injury and Osteoarthritis Outcome Score (KOOS), 36-Item Short-Form Health Survey (SF-36), and International Knee Documentation Committee Subjective Form. Patients were followed up at 2 and 6 years postoperatively regarding outcome measures, additional surgeries, and graft failure. Additionally, the following knee radiographs were obtained in 630 patients from a previous MARS Group investigation³¹: bilateral knee standing anteroposterior, full-extension lateral, bilateral 45° bent-knee posterior-anterior, long-leg alignment, and bilateral patellofemoral views. The radiographs were used to measure femoral and tibial tunnel positions, sagittal view physiological tibial plateau slope angle, leg alignment, and joint space narrowing. Tunnel position and size were deemed accurate or compromised by the attending surgeons who completed the surgeon questionnaires. These determinations were not held to specific numeric measurements.

Surgeons were free to perform and use the surgery and graft of their preferred choice. If an allograft was used, it was supplied by MTF Biologics to ensure processing consistency. Surgeons completed a 49-page questionnaire of intraoperative findings including examination under anesthesia results; surgical technique; concomitant cartilage injury based on modified Outerbridge classification; meniscal injuries classified by size, location, degree of tear, and treatment; and rehabilitation guidelines. These data were entered into a master database and underwent quality control checks before data analysis.

Patient Selection

MARS cohort patients who completed both the follow-up questionnaire and telephone follow-up for subsequent surgery at 6 years postoperatively were considered for inclusion in the current study. Patients with missing data for the outcome variable were excluded from analysis.

Outcomes

The primary outcome of the present study was graft failure after rACLR, which was ascertained by querying patients. All patients who reported experiencing graft failure also reported receiving professional confirmation with operative report, clinic report, or magnetic resonance imaging report.

Data Preprocessing and Feature Engineering

We conducted data preprocessing, model creation, evaluation, and interpretation using Python (Version 3.10; Python Software Foundation),⁴⁷ the Anaconda distribution platform,² and the Jupyterlab interactive development environment.²² Standard libraries including pandas,³⁷ NumPy,¹⁶ scikit-learn (sklearn),³⁸ and Matplotlib were used for the data analysis and visualization. Descriptive statistics and table creation were performed using R Version 4.1.0 (R Foundation for Statistical Computing).

All data collected for the MARS cohort were considered for inclusion in the analysis. Significant data preprocessing including feature engineering and imputation of missing values was conducted in order to yield a usable set of features for model creation. Nominal (or categorical) features were encoded using OneHotEncoder in sklearn, whereby features are split into their category levels where categories are unrelated. An example of this is the feature “current femoral fixation,” in which the options are “combination,”“cross pin,”“interference screw,”“suture + button/Endobutton,” and “other.” Ordinal features like PROs on a Likert scale, whose categories have a clear order but the size of differences between categories is not necessarily quantifiable, were encoded using sklearn category encoder's OrdinalEncoder, whereby categories were mapped to numeric values that preserved the relationships between categories.⁴⁶ For all features, category levels were required to meet a percentage threshold of ≥0.04 in order to be encoded and were incorporated into other categories until the threshold was met. Conditional variables were excluded from analysis because of low frequencies that could not be managed without compromising data quality.

Per validated methodology, features with >60% missing values were excluded. For the remaining features, missing values were imputed using the validated SimpleImputer in sklearn. Means were used for continuous and ordinal features, and modes used for binary and nominal features.

Model Development and Evaluation

Five machine learning models were developed to predict graft failure at 6 years postoperatively using preprocessed patient, clinical, and surgical features. These models included traditional binary classifiers (logistic regression [LR], XGBoost, gradient boosting [GB], and RF) and a validated ensemble algorithm (AutoPrognosis).

AutoPrognosis utilizes pipelines to build an ensemble “supermodel,” as depicted in Figure 1. Each pipeline contains an imputation, feature processing, and classification algorithm, as well as a calibration method. AutoPrognosis automatically conducts and optimizes hyperparameter searches for each classification method and selects and ranks top-performing pipelines based on suspected contribution to overall model performance. AutoPrognosis was implemented using code first provided by Alaa and van der Schaar¹; LR, RF, and GB were implemented using the sklearn library; and XGBoost was built using the XGBoost library.

Figure 1.

AutoPrognosis design diagram.

The study cohort was split 80:20 into a training and held-out test cohort. Fivefold stratified cross-validation with the training cohort was used to determine each model's discriminative power and calibration. The held-out test cohort was used to evaluate model performance, reported as mean and standard error.

Validated performance metrics were used to measure discrimination and calibration. The area under the receiver operating characteristic curve (AUROC) is the probability the predictive model correctly assigns a patient who experienced the outcome of interest a higher risk than a patient who did not. AUROC values <0.7 indicate low discriminative power, values from 0.7 to 0.9 indicate moderate discriminative power, and values >0.9 indicate high discriminative power.¹² The area under the precision-recall curve (AUPRC) is the probability that a model correctly predicts which patients experience the outcome of interest while minimizing false-positive results. AUPRC is useful in the setting of an imbalanced data set like MARS. The closer the AUPRC is to 1 and the higher it is compared with the baseline AUPRC (a result of random prediction), the closer the model is to an ideal classifier.

Calibration is a measure of how closely the model's predictions match the outcomes seen in the study cohort. The Brier score, a measure of both model calibration and discriminative power, is measured by taking the mean square error between values observed in the data and probabilities predicted by the model. Brier scores range between 0 and 1, and a lower score indicates superior performance.

Feature Importance

The Friedman partial dependence function and derived partial dependence plot (PDP)–based feature importance were used to determine the most important individual features based on their contribution to a model's predictive ability by assessing the average effect in predicted risk when a feature's value is altered.¹³ The caveat with PDP feature importance as a measure is that an important feature can have a large contribution to a model's predictive ability without necessarily being important in a model's discriminative ability. Therefore, feature importance was also calculated using perturbation-based feature importance, which ranks features in terms of AUROC and AUPRC loss with modification of an individual feature. Features identified using perturbation-based importance were key to model discriminative ability.

We reported the most important features that appear in the top 10 in PDPs and both perturbation-based feature importance measures, as these are key to both model predictive and discriminative ability. For each of these features, individual feature behavior in model prediction was further elucidated using PDPs. Furthermore, the use of multiple feature importance measures serves as an additional check to prevent inappropriate importance designations, because the black-box nature of algorithms such as AutoPrognosis can give importance to nonphysiological patient features that affect a small portion of patients in the cohort (eg, surgeon years of experience).

Results

Study Population

Of the complete MARS cohort of 1233 patients, 273 (22.1%) patients were excluded from the study (82 had incomplete data and 191 had inadequate follow-up). The final study cohort used for analysis was 960 (77.9%) patients.

The baseline cohort characteristics and key variables are summarized in Appendix Tables A1 to A4. Variables in the tables represent only a fraction of the total number of variables in the data set that underwent preprocessing and were eventually included in the model. Of the 960 patients, 530 (55.2%) were male. The median age of the cohort was 26 years, and the median body mass index was 25.1. Most often, patients were White, did not smoke, and lived with 2 other people. Patients had a median of 15 years of education and worked 40 h/wk. The reported mechanism of injury was most often nontraumatic with gradual onset (n = 512; 53.3%). At the time of initial injury, 706 (74.3%) patients were playing sports, 230 (24.3%) had contact with another player, 254 (26.9%) were jumping, and 728 (77.7%) felt or heard a pop. Most patients (88%) were undergoing their first rACLR, at a median of 3.7 years from their previous ACLR, and 283 (29.5%) patients underwent revision with the surgeon who had performed their previous ACLR. In rank order, autograft (n = 474; 49.4%), followed by allograft (n = 456; 47.5%) and combination graft (n = 29; 3.0%), was the most frequently used graft type for the current revision surgery.

Model Performance

Using the MARS cohort subset of 960 patients who completed 6-year follow-up, 5.7% (n = 55) of whom experienced graft failure, we built 5 algorithms predicting the risk of graft failure after rACLR. Pipelines were used to build the weighted ensemble AutoPrognosis model (Table 1).

Table 1

List of 3 Pipelines Fitted to MARS Cohort^a

Pipeline	Dimensionality Reduction; Scaler; Classifier	Hyperparameters	Weight
1	Variance threshold; MaxAbs scaler; random forest	({threshold=0.001}, None, {n_estimators=50, max_depth=4})	0.333
2	FastICA; MaxAbs scaler; Catboost	({n_components=3}, None, {n_estimators=100, depth=5, grow_policy=“SymmetricTree”})	0.417
3	None; none; Catboost	(None, None, {n_estimators=100, depth=3, grow_policy=“Depthwise”})	0.250

MARS, Multicenter ACL Revision Study.

While all models had moderate to good discriminative power, AutoPrognosis demonstrated the highest discriminative power compared with other models (AUROC, 0.703 ± 0.036) (Table 2). The AutoPrognosis model was well calibrated, with calibration scores similar to those of the other studied models (Brier score, 0.053 ± 0.001). The AutoPrognosis model's AUPRC (0.152 ± 0.043) outperformed most models and had scores similar to those of the XGBoost and GB models. A baseline AUPRC of 0.057 would be demonstrated if a classifier were to generate random predictions. A confusion matrix for AutoPrognosis that was created using the held-out test cohort, with a threshold of 0.05729, is illustrated in Table 3. The receiver operating characteristic curves and precision-recall curves for the AutoPrognosis and LR models are illustrated in Figure 2.

Table 2

Discrimination and Calibration Statistics^a

Method	AUROC	AUPRC	Brier Score
AutoPrognosis	0.703 ± 0.036	0.152 ± 0.043	0.053 ± 0.001
LR	0.592 ± 0.018	0.116 ± 0.036	0.111 ± 0.021
XGBoost	0.680 ± 0.021	0.150 ± 0.072	0.058 ± 0.006
GB	0.660 ± 0.020	0.151 ± 0.063	0.057 ± 0.006
RF	0.618 ± 0.062	0.129 ± 0.051	0.054 ± 0.001

Data are presented as mean ± SD. AUPRC, area under the precision-recall curve; AUROC, area under the receiver operating characteristic curve; GB, gradient boosting; LR, logistic regression; RF, random forest.

Table 3

AutoPrognosis Model Confusion Matrix^a

	Predicted Positive	Predicted Negative
Observed positive	6	5
Observed negative	18	163

Data are presented as No. of observations. Accuracy = 0.880; F1 score = 0.343.

Figure 2.

(A) Receiver operating characteristic curves and (B) precision-recall curves for the AutoPrognosis and logistic regression models.

Feature Importance

Features deemed important for AutoPrognosis differed from those for LR model performance. For AutoPrognosis, the 10 most important features based on PDP and perturbation-based feature importance measures are listed in Table 4. Although important features varied by ranking methodology, compromised prior ACLR femoral tunnel position and size, compromised prior ACLR tibial tunnel position and size, and graft type used in current rACLR ranked in the top 10 regardless of methodology. Therefore, these features were deemed most important for both AutoPrognosis model predictive and discriminative ability.

Table 4

AutoPrognosis Feature Ranking Based on PDP and Perturbation-Based (AUROC Loss and AUPRC Loss) Feature Importance Measures^a

Feature	PDP Importance Rank	Perturbation Importance Rank (AUROC Loss)	Perturbation Importance Rank (AUPRC Loss)
Surgeon years of experience	1	277	304
Contralateral hamstring tendon graft use in previous ACLR	2	182	160
Compromised prior ACLR femoral tunnel position and size	3 ^b	1	2
Prior ACLR femoral tunnel position in preop sagittal view radiographs (measurement technique 1)^c	4	35	303
Prior ACLR femoral tunnel position in preop sagittal view radiographs (measurement technique 2)^d	5	15	52
Baseline age	6	27	307
Compromised prior ACLR tibial tunnel position and size	7 ^b	2	3
SF-36 PCS subscore	8	118	281
KOOS-ADL subscore	9	278	26
Graft type used in current rACLR	10 ^b	3	1

Bolded features are those that rank within the top 10 in all feature importance measures for the AutoPrognosis model. ACLR, anterior cruciate ligament reconstruction; ADL, Activities of Daily Living; AUPRC, area under the precision-recall curve; AUROC, area under the receiver operating characteristic curve; KOOS, Knee injury and Osteoarthritis Outcome Score; LR, logistic regression; PCS, physical component summary; PDP, partial dependence plot; preop, preoperative; SF-36, 36-Item Short-Form Health Survey.

Features that were ranked within the top 10 for LR based on PDP feature importance measurements.

Recorded as a percentage of the distance from the femoral tunnel location to the Blumensaat line.

Measured as a percentage of the distance from the femoral tunnel position to the cortex width.

PDP-based importance calculations for AutoPrognosis demonstrated the following top 5 contributors to model predictive ability: surgeon years of experience, contralateral hamstring tendon graft use in prior ACLR, compromised prior ACLR femoral tunnel position and size, and prior ACLR femoral tunnel position measured via preoperative radiographs (with techniques 1 and 2). AutoPrognosis perturbation-based calculations showed the following top 5 contributors to AUROC performance: prior ACLR femoral tunnel position and size, prior ACLR tibial tunnel position and size, current rACLR graft type, prior ACLR graft choice, and current rACLR femoral tunnel position and size. Perturbation-based calculations showed the following top 5 contributors to AUPRC performance: current rACLR graft type, prior ACLR femoral tunnel position and size, prior ACLR tibial tunnel position and size, prior ACLR tibial fixation method, and patellar tendon autograft use in prior ACLR. Other important features included baseline PRO scores (SF-36 subscale scores for physical function and KOOS Activities of Daily Living), baseline age, sex, body mass index, and sports activity at time of injury (Table 5).

Table 5

Top 10 Important Features by Feature Importance Measure^a

Rank	PDP Importance	Perturbation Importance (AUROC Loss)	Perturbation Importance (AUPRC Loss)
1	Surgeon years of experience	Prior ACLR femoral tunnel position and size	Current rACLR graft type
2	Contralateral hamstring tendon graft use in prior ACLR	Prior ACLR tibial tunnel position and size	Prior ACLR femoral tunnel position and size
3	Compromised prior ACLR femoral tunnel position and size	Current rACLR graft type	Prior ACLR tibial tunnel position and size
4	Prior ACLR femoral tunnel position in preop sagittal view radiographs (measurement technique 1)	Prior ACLR graft choice	Prior ACLR tibial fixation method
5	Prior ACLR femoral tunnel position in preop sagittal view radiographs (measurement technique 2)	Current rACLR femoral tunnel position and size	Patellar tendon autograft use in prior ACLR
6	Baseline age	Surgeon’s opinion on cause of prior ACLR failure	Patient sex
7	Compromised prior ACLR tibial tunnel position and size	Patellar tendon autograft use in prior ACLR	Baseline work status
8	SF-36 PCS subscore	Ipsilateral patellar tendon graft use in prior ACLR	Sports activity involving the ipsilateral knee at the time of injury
9	KOOS-ADL subscore	Technical cause of prior ACLR failure (in the surgeon's opinion)	Baseline BMI
10	Graft type used in current rACLR	Prior ACLR tibial fixation method	BPTB graft source for prior ACLR graft

ACLR, anterior cruciate ligament reconstruction; ADL, Activities of Daily Living; AUPRC, area under the precision-recall curve; AUROC, area under the receiver operating characteristic curve; BMI, body mass index; BPTB, bone–patellar tendon–bone; KOOS, Knee injury and Osteoarthritis Outcome Score; PCS, physical component summary; PDP, partial dependence plot; preop, preoperative; SF-36, 36-Item Short-Form Health Survey.

For the top 3 AutoPrognosis features across all importance measures, PDPs illustrate the feature's relative contribution and behavior in model prediction (Figure 3). Within current graft type, allograft use as a category had the largest relative contribution to predicted risk of graft failure (Figure 3A). Regarding prior ACLR, compromised femoral position and tunnel size (Figure 3B) and compromised tibial position and tunnel size (Figure 3C) were the largest contributors to predicted risk of current rACLR graft failure.

Figure 3.

Partial dependence plots of predicted risk of graft failure after revision anterior cruciate ligament reconstruction (ACLR) based on (A) graft type used in current revision ACLR, (B) femoral tunnel placement in prior ACLR, and (C) tibial tunnel placement in prior ACLR.

Discussion

As graft failure continues to be a devastating complication of rACLR, it remains a challenge to predict failure on a patient-specific level. The most important finding of this study was that of all the described models in this machine learning analysis, AutoPrognosis, appeared to best predict rACLR graft failure with moderate discriminative power. The most important features for AutoPrognosis model performance and discriminative power were both the femoral and tibial tunnel placements and sizes during the prior ACLR and the graft type that was used in current rACLR. Specifically, compromised ACL femoral and tibial tunnel positions and sizes, and allograft use were associated with an increase in contribution to the model's predicted risk of graft failure. The AutoPrognosis algorithm and aforementioned findings build on previous studies that have reported on risk factors for rACLR graft failure.

Of the identified key features, several have been previously implicated as drivers of graft failure risk after rACLR. Conversely, several features previously demonstrated as important, like age and sex, were not found to be important features in the current analysis.^5,20 When considering the implications of these findings, it is important to note that important features must be seen within the context of the predictive model. Important features (or lack thereof) should not be interpreted as having causal relationships or lack thereof with the outcome of interest, and predictive models should not be interpreted as explanatory models.

The present study is the first to conduct a machine learning analysis of the MARS cohort in order to deliver patient-specific risk predictions of rACLR outcomes. Our findings present an opportunity to influence rACLR risk stratification from a subjective consideration based on surgeon knowledge and experience to an evidence-based quantitative measure. A risk calculator built with our model has the potential to guide preoperative counseling, shared decision-making, and intentional risk mitigation. Furthermore, in a health care environment where cost containment has increased in importance^21,36 and cost estimates for a single rACLR (in 2008 US dollars) were reportedly a median of $20,501.93 (range, $14,420.72-$51, 211.13), more accurate risk stratification may allow for fairer risk-adjusted reimbursement.⁶

Compromised femoral and tibial tunnel positions and sizes in prior ACLR procedures were found to be important contributors in the risk of current rACLR graft failure in the present study. In previous studies, femoral tunnel malpositioning has been extensively studied and widely accepted as a risk factor for graft failure,^{8,15,35,43-45} and it is one of the most commonly cited reasons for primary ACLR graft failure by surgeons, including those in the MARS Group. Although prior femoral tunnel position has not necessarily been implicated as a predictor for current rACLR graft failure in a previous MARS Group investigation,⁸ prior tunnel position and size are important considerations in rACLR planning.

Poorly positioned and sized prior tunnels may lead to ongoing challenges in the current rACLR with respect to tunnel preparation potentially requiring grafting and identification of osseous landmarks to establish the appropriate footprint.¹¹ While degree of femoral or tibial tunnel malpositioning and enlargement were not quantitatively measured in the MARS data set, the presence of a type 2 tunnel with slight malpositioning may compromise ideal placement of the revision tunnel, especially in a single-stage procedure where the graft is most at risk of sliding anteriorly.²⁶

Additionally, while bone graft procedure data and physiological tibial plateau slope angle measurements were included in model creation, they were not deemed important features. Of note, an ideally positioned prior tunnel had a higher contribution to risk of graft failure in the current model compared with an isolated malpositioned tunnel. It is important to note that the combination of tunnel position and size was highly predictive, but compromised tunnel position alone or size alone was less predictive than ideal tunnels. While the mechanism of this remains to be investigated, the extraction of hardware and reuse of the existing ideally placed tunnel may present unique challenges including iatrogenic loss of cortical containment or tunnel interference due to retained implants, sclerotic bone, or residual tissue.⁴⁸ Although there is a dearth of literature investigating the role of the tibial tunnel in rACLR graft failure, studies have demonstrated the importance of tibial tunnel positioning in restoring knee stability and minimizing graft impingement in extension.^18,19 While optimal femoral positioning may restore rotational control and improve coronal graft obliquity, in the setting of vertical graft position in the sagittal plane, tibial tunnel placement remains important in preventing instability.⁴ Similar to the femoral tunnel, prior tibial tunnel positioning and sizing may impact the ability to optimize tibial tunnel placement in the current rACLR, rendering it an important independent predictor of graft failure.

Allograft use is a known risk factor for rACLR graft failure compared with autograft use, as demonstrated by previous MARS investigations of graft failure by the 2- and 6-year postoperative time points.^28,30 Our results echo these findings and similarly implicate current graft type as an important feature in the present model.

Limitations

There were several limitations to the present study. First, because of the relative rarity of graft failure as an outcome, our data set was imbalanced. However, to address potential concern of overfitting and overprediction of negative outcomes, all models were evaluated with the AUPRC, wherein AutoPrognosis demonstrated superior performance. Second, we must acknowledge that any biases represented in the data may be amplified by machine learning methods, potentially furthering biases against underrepresented patient populations including women, ethnic minorities, and patients of lower socioeconomic status.¹⁷ Finally, while graft failure is used as a quantitative proxy for patient outcomes, it is only one of several determinants of patient success or failure. Further studies analyzing PROs, reoperations, risk for posttraumatic osteoarthritis, and graft survival could provide additional insight and support the development of a bedside risk calculator to aid in risk stratification and counseling of patients undergoing rACLR.

Conclusion

The present study demonstrated the ability of the novel AutoPrognosis machine learning model to best predict the risk of graft failure in patients undergoing rACLR at 6 years postoperatively with moderate predictive ability. Femoral and tibial tunnel sizes and positions in prior ACLR and allograft use in current rACLR were all risk factors for rACLR failure in the context of the AutoPrognosis model. This study describes a unique model that can be externally validated with larger data sets and contribute toward the creation of a robust rACLR bedside risk calculator in future studies.

Supplemental Material

sj-pdf-1-ojs-10.1177_23259671241291920 – Supplemental material for A Novel Machine Learning Model to Predict Revision ACL Reconstruction Failure in the MARS Cohort

Supplemental material, sj-pdf-1-ojs-10.1177_23259671241291920 for A Novel Machine Learning Model to Predict Revision ACL Reconstruction Failure in the MARS Cohort by Kinjal Vasavada, Vrinda Vasavada, Jay Moran, Sai Devana, Changhee Lee, Sharon L. Hame, Laith M. Jazrawi, Orrin H. Sherman, Laura J. Huston, Amanda K. Haas, Christina R. Allen, Daniel E. Cooper, Thomas M. DeBerardino, Kurt P. Spindler, Michael J. Stuart, Annunziato (Ned) Amendola, Christopher C. Annunziata, Robert A. Arciero, Bernard R. Bach, Champ L. Baker, Arthur R. Bartolozzi, Keith M. Baumgarten, Jeffrey H. Berg, Geoffrey A. Bernas, Stephen F. Brockmeier, Robert H. Brophy, Charles A. Bush-Joseph, J. Brad Butler V, James L. Carey, James E. Carpenter, Brian J. Cole, Jonathan M. Cooper, Charles L. Cox, R. Alexander Creighton, Tal S. David, Warren R. Dunn, David C. Flanigan, Robert W. Frederick, Theodore J. Ganley, Charles J. Gatt, Steven R. Gecha, James Robert Giffin, Jo A. Hannafin, Norman Lindsay Harris, Keith S. Hechtman, Elliott B. Hershman, Rudolf G. Hoellrich, David C. Johnson, Timothy S. Johnson, Morgan H. Jones, Christopher C. Kaeding, Ganesh V. Kamath, Thomas E. Klootwyk, Bruce A. Levy, C. Benjamin Ma, G. Peter Maiers, Robert G. Marx, Matthew J. Matava, Gregory M. Mathien, David R. McAllister, Eric C. McCarty, Robert G. McCormack, Bruce S. Miller, Carl W. Nissen, Daniel F. O’Neill, Brett D. Owens, Richard D. Parker, Mark L. Purnell, Arun J. Ramappa, Michael A. Rauh, Arthur C. Rettig, Jon K. Sekiya, Kevin G. Shea, James R. Slauterbeck, Matthew V. Smith, Jeffrey T. Spang, Steven J. Svoboda, Timothy N. Taft, Joachim J. Tenuta, Edwin M. Tingstad, Armando F. Vidal, Darius G. Viskontas, Richard A. White, James S. Williams, Michelle L. Wolcott, Brian R. Wolf, Rick W. Wright and James J. York in Orthopaedic Journal of Sports Medicine

Footnotes

Appendix

Appendix Table A4

Surgical Information at Time of MARS Study Enrollment (N = 960 Patients)^a

Variable	Value	Variable	Value
Time since last ACLR, y	3.70 (1.50-8.80)	Tibial fixation
Missing data	10 (1)	Interference screw	565 (59.0)
Surgeon years of experience	16.00 (8.00-20.00)	Combination	199 (20.8)
Missing data	0 (0)	Intrafix	90 (9.4)
No. of revisions		Suture and post or button	50 (5.2)
1	845 (88.0)	Other	54 (5.6)
2	99 (10.3)	Missing data	2 (0.2)
3	16 (1.7)	Medial meniscal pathology/treatment
Missing data	0 (0)	Normal	537 (55.9)
Surgeon’s opinion of failure		Excision	262 (27.3)
Traumatic	336 (35.3)	Repair	126 (13.1)
Combination	335 (35.2)	Other	35 (3.6)
Technical	206 (21.7)	Missing data	0 (0)
Biological	74 (7.8)	Lateral meniscal pathology/treatment
Missing data	9 (0.9)	Normal	628 (65.4)
Surgeon’s revision of their own failure		Excision	234 (24.4)
No	672 (70.4)	Other	48 (5.0)
Yes	283 (29.5)	Repair	50 (5.2)
Missing data	5 (0.5)	Missing data	0 (0)
Injury mechanism		LFC articular cartilage pathology
Nontraumatic, gradual onset	512 (53.3)	Grade 1 (normal)	692 (72.1)
Nontraumatic, sudden onset	257 (26.8)	Grade 2	141 (14.7)
Traumatic, noncontact	123 (12.8)	Grade 3	75 (7.8)
Traumatic, contact	66 (6.9)	Grade 3/4 or 4	52 (5.4)
Missing data	2 (0.2)	Missing data	0 (0)
Surgical exposure technique		MFC articular cartilage pathology
1-incision (AM portal)	430 (45.1)	Grade 1 (normal)	548 (57.1)
1-incision (transtibial)	339 (35.5)	Grade 2	223 (23.3)
2-incision	179 (18.8)	Grade 3	125 (13.0)
Arthrotomy/other	6 (0.6)	Grade 3/4 or 4	63 (6.6)
Missing data	6 (0.6)	Missing data	1 (0.1)
Graft type		LTP articular cartilage pathology
Autograft	474 (49.4)	Grade 1 (normal)	794 (82.7)
Allograft	456 (47.5)	Grade 2	128 (13.3)
Both autograft and allograft	29 (3.0)	Grade 3	33 (3.4)
Missing data	1 (0.1)	Grade 3/4 or 4	5 (0.5)
Graft source		Missing data	0 (0)
BPTB	485 (50.6)	MTP articular cartilage pathology
Soft tissue	444 (46.3)	Grade 1 (normal)	859 (89.9)
Other	29 (3.0)	Grade 2	67 (7.0)
Missing data	2 (0.2)	Grade 3	15 (1.6)
Femoral tunnel position		Grade 3/4 or 4	14 (1.5)
Entirely new tunnel	477 (49.8)	Missing data	5 (0.5)
Optimum position	259 (27.0)	Patellar articular cartilage pathology
Blended new tunnel	173 (18.1)	Grade 1 (normal)	682 (71.0)
Added second tunnel	29 (3.0)	Grade 2	176 (18.3)
Other	20 (2.1)	Grade 3	94 (9.8)
Missing data	2 (0.2)	Grade 3/4 or 4	8 (0.8)
Femoral fixation		Missing data	0 (0)
Interference screw	549 (57.4)	Trochlear articular cartilage pathology
Suture and Endobutton	204 (21.3)	Grade 1 (normal)	771 (80.3)
Cross pin	114 (11.9)	Grade 2	77 (8.0)
Combination	51 (5.3)	Grade 3	73 (7.6)
Other	39 (4.1)	Grade 3/4 or 4	39 (4.1)
Missing data	3 (0.3)	Missing data	0 (0)
Tibial tunnel position		Biological enhancement used
Optimum position	569 (59.4)	No	879 (91.9)
Blended new tunnel	180 (18.8)	Yes	77 (8.1)
Entirely new tunnel	162 (16.9)	Missing data	4 (0.4)
Added second tunnel	28 (2.9)
Same tunnel aperture but compromised position	19 (2.0)
Missing data	2 (0.2)

Data are originally from the MARS Group (2010).²⁷ Continuous variables are presented as median (interquartile range); categorical variables and missing data are presented as n (%). ACLR, anterior cruciate ligament reconstruction; AM, anteromedial; BPTB, bone–patellar tendon–bone; LFC, lateral femoral condyle; LTP, lateral tibial plateau; MARS, Multicenter ACL Revision Study; MFC, medial femoral condyle; MTP, medial tibial plateau.

Acknowledgements

The authors express their appreciation to the late Barton Mann, PhD (AOSSM, Rosemont, Illinois, USA), Allen F. Anderson, MD (Tennessee Orthopaedic Alliance, Nashville, Tennessee, USA), Jeffery R. Bechler, MD (University Orthopaedic Associates LLC, Princeton, New Jersey, USA), and Timothy M. Hosea, MD (University Orthopaedic Associates LLC, Princeton, New Jersey, USA), whose contributions to this work was of great significance. They sincerely appreciate Elizabeth Garofoli's years of dedicated work and effort on this study. The authors also extend their gratitude to John P. Albright, MD (University of Iowa Hospitals and Clinics, Iowa City, Iowa, USA), Jack T. Andrish, MD (Cleveland Clinic, Cleveland, Ohio, USA), John D. Campbell, MD (Bridger Orthopedic and Sports Medicine, Bozeman, Montana, USA), Diane L. Dahm, MD (Mayo Clinic, Rochester, Minnesota, USA), and Brett (Brick) A. Lantz, MD (Slocum Research and Education Foundation, Eugene, Oregon, USA) for their effort and leadership on this project. All are enjoying a well-deserved and happy retirement after many years of dedication to the advancement of orthopaedics.

Authors

MARS Group; Kinjal Vasavada, MD (Yale University, New Haven, Connecticut, USA); Vrinda Vasavada, MS (NYU Hospital for Joint Diseases, New York, New York, USA); Jay Moran, MD (Yale University, New Haven, Connecticut, USA); Sai Devana, MD (David Geffen School of Medicine at University of California at Los Angeles, Los Angeles, California, USA); Changhee Lee, PhD (Korea University, Seoul, Republic of Korea); Sharon L. Hame, MD (David Geffen School of Medicine at University of California at Los Angeles, Los Angeles, California, USA); Laith M. Jazrawi, MD (NYU Hospital for Joint Diseases, New York, New York, USA); Orrin H. Sherman, MD (NYU Hospital for Joint Diseases, New York, New York, USA); Laura J. Huston, MS (Vanderbilt University, Nashville, Tennessee, USA); Amanda K. Haas, MA (Washington University in St Louis, St Louis, Missouri, USA); Christina R. Allen, MD (Yale University, New Haven, Connecticut, USA); Daniel E. Cooper, MD (W.B. Carrell Memorial Clinic, Dallas, Texas, USA); Thomas M. DeBerardino, MD (UT Health, San Antonio, Texas, USA); Kurt P. Spindler, MD (Cleveland Clinic, Cleveland, Ohio, USA); Michael J. Stuart, MD (Mayo Clinic, Rochester, Minnesota, USA); Annunziato (Ned) Amendola, MD (Duke University, Durham, North Carolina, USA); Christopher C. Annunziata, MD (Commonwealth Orthopaedics & Rehabilitation, Arlington, Virginia, USA); Robert A. Arciero, MD (University of Connecticut Health Center, Farmington, Connecticut, USA); Bernard R. Bach Jr, MD (Rush University Medical Center, Chicago, Illinois, USA); Champ L. Baker III, MD (The Hughston Clinic, Columbus, Georgia, USA); Arthur R. Bartolozzi, MD (3B Orthopaedics, University of Pennsylvania Health System, Philadelphia, Pennsylvania, USA); Keith M. Baumgarten, MD (Orthopedic Institute, Sioux Falls, South Dakota, USA); Jeffrey H. Berg, MD (Town Center Orthopaedic Associates, Reston, Virginia, USA); Geoffrey A. Bernas, MD (State University of New York at Buffalo, Buffalo, New York, USA); Stephen F. Brockmeier, MD (University of Virginia, Charlottesville, Virginia, USA); Robert H. Brophy, MD (Washington University in St Louis, St Louis, Missouri, USA); Charles A. Bush-Joseph, MD (Rush University Medical Center, Chicago, Illinois, USA); J. Brad Butler V, MD (Orthopedic and Fracture Clinic, Portland, Oregon, USA); James L. Carey, MD, MPH (University of Pennsylvania, Philadelphia, Pennsylvania, USA); James E. Carpenter, MD (University of Michigan, Ann Arbor, Michigan, USA); Brian J. Cole, MD (Rush University Medical Center, Chicago, Illinois, USA); Jonathan M. Cooper, DO (HealthPartners Specialty Center, St Paul, Minnesota, USA); Charles L. Cox, MD, MPH (Vanderbilt University, Nashville, Tennessee, USA); R. Alexander Creighton, MD (University of North Carolina Medical Center, Chapel Hill, North Carolina, USA); Tal S. David, MD (Synergy Specialists Medical Group, San Diego, California, USA); Warren R. Dunn, MD, MPH (Fondren Orthopedic Group, Houston, Texas, USA); David C. Flanigan, MD (The Ohio State University, Columbus, Ohio, USA); Robert W. Frederick, MD (The Rothman Institute/Thomas Jefferson University, Philadelphia, Pennsylvania, USA); Theodore J. Ganley, MD (Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA); Charles J. Gatt Jr, MD (University Orthopaedic Associates LLC, Princeton, New Jersey, USA); Steven R. Gecha, MD (Princeton Orthopaedic Associates, Princeton, New Jersey, USA); James Robert Giffin, MD (Fowler Kennedy Sport Medicine Clinic, University of Western Ontario, London, Ontario, Canada); Jo A. Hannafin, MD, PhD (Hospital for Special Surgery, New York, New York, USA); Norman Lindsay Harris Jr, MD (Grand River Health, Rifle, Colorado, USA); Keith S. Hechtman, MD (Miami Orthopedics and Sports Medicine Institute, Coral Gables, Florida, USA); Elliott B. Hershman, MD (Lenox Hill Hospital, New York, New York, USA); Rudolf G. Hoellrich, MD (Slocum Research and Education Foundation, Eugene, Oregon, USA); David C. Johnson, MD (National Sports Medicine Institute, Leesburg, Virginia, USA); Timothy S. Johnson, MD (National Sports Medicine Institute, Leesburg, Virginia, USA); Morgan H. Jones, MD (Brigham and Women's Hospital, Boston, Massachusetts, USA); Christopher C. Kaeding, MD (The Ohio State University, Columbus, Ohio, USA); Ganesh V. Kamath, MD (University of North Carolina Medical Center, Chapel Hill, North Carolina, USA); Thomas E. Klootwyk, MD (Forte Sports Medicine and Orthopedics, Indianapolis, Indiana, USA); Bruce A. Levy, MD (Orlando Health, Orlando, Florida, USA); C. Benjamin Ma, MD (University of California at San Francisco, San Francisco, California, USA); G. Peter Maiers II, MD (Forte Sports Medicine and Orthopedics, Indianapolis, Indiana, USA); Robert G. Marx, MD (Hospital for Special Surgery, New York, New York, USA); Matthew J. Matava, MD (Washington University in St Louis, St Louis, Missouri, USA); Gregory M. Mathien, MD (Knoxville Orthopedic Clinic, Knoxville, Tennessee, USA); David R. McAllister, MD (University of California at Los Angeles, Los Angeles, California, USA); Eric C. McCarty, MD (University of Colorado–Denver School of Medicine, Denver, Colorado, USA); Robert G. McCormack, MD (University of British Columbia/Fraser Health Authority, New Westinster, British Columbia, Canada); Bruce S. Miller, MD, MS (University of Michigan, Ann Arbor, Michigan, USA); Carl W. Nissen, MD (Connecticut Children's Medical Center, Hartford, Connecticut, USA); Daniel F. O’Neill, MD, EdD (The Alpine Clinic, Plymouth, New Hampshire, USA); Brett D. Owens, MD (Warren Alpert Medical School, Brown University, Providence, Rhode Island, USA); Richard D. Parker, MD (Cleveland Clinic, Cleveland, Ohio, USA); Mark L. Purnell, MD (Valley Ortho, Aspen, Colorado, USA); Arun J. Ramappa, MD (Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA); Michael A. Rauh, MD (State University of New York at Buffalo, Buffalo, New York, USA); Arthur C. Rettig, MD (Forte Sports Medicine and Orthopedics, Indianapolis, Indiana, USA); Jon K. Sekiya, MD (University of Michigan, Ann Arbor, Michigan, USA); Kevin G. Shea, MD (Stanford University, Palo Alto, California, USA); James R. Slauterbeck, MD (UNC Health Southeastern, Lumberton, North Carolina, USA); Matthew V. Smith, MD (Washington University in St Louis, St Louis, Missouri, USA); Jeffrey T. Spang, MD (University of North Carolina Medical Center, Chapel Hill, North Carolina, USA); Steven J. Svoboda, MD, COL (Ret) (MedStar Orthopaedic and Sports Center, Washington, DC, USA); Timothy N. Taft, MD (University of North Carolina Medical Center, Chapel Hill, North Carolina, USA); Joachim J. Tenuta, MD (Albany Medical Center, Albany, New York, USA); Edwin M. Tingstad, MD (Inland Orthopaedic Surgery and Sports Medicine Clinic, Pullman, Washington, USA); Armando F. Vidal, MD (The Steadman Clinic, Vail, Colorado, USA); Darius G. Viskontas, MD (Fraser Orthopaedic Institute, New Westminster, British Columbia, Canada); Richard A. White, MD (Fitzgibbon’s Hospital, Marshall, Missouri, USA); James S. Williams Jr, MD (Cleveland Clinic, Euclid, Ohio, USA); Michelle L. Wolcott, MD (University of Colorado–Anschutz Medical Campus, Aurora, Colorado, USA); Brian R. Wolf, MD (University of Iowa Hospitals and Clinics, Iowa City, Iowa, USA); Rick W. Wright, MD (Vanderbilt University, Nashville, Tennessee, USA); and James J. York, MD (Luminis Health Orthopedics, Pasadena, Maryland, USA).

Final revision submitted March 9, 2024; accepted April 3, 2024.

One or more of the authors has declared the following potential conflict of interest or source of funding: This study was funded by the National Institutes of Health/National Institute of Arthritis and Musculoskeletal and Skin Diseases (grant 5R01-AR060846). See for individual disclosures. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.

Ethical approval for this study was obtained from Vanderbilt University (ref No. 070110).

Supplemental Material

Supplemental Material for this article is available at

References

Alaa

van der Schaar

AutoPrognosis: automated clinical prognostic modeling via Bayesian optimization with structured kernel learning. Presented at: Proceedings of the 35th International Conference on Machine Learning; July 10-15, 2018; Stockholm, Sweden. Accessed October 13, 2024. https://proceedings.mlr.press/v80/alaa18b.html

Anaconda. Anaconda software distribution. Accessed October 13, 2024. https://anaconda.com

Beam

Kohane

IS.

Big data and machine learning in health care. JAMA. 2018;319(13):1317-1318.

Bedi

Maak

Musahl

, et al. Effect of tibial tunnel position on stability of the knee after anterior cruciate ligament reconstruction: is the tibial tunnel position most important? Am J Sports Med. 2011;39(2):366-373.

Branche

Bradsell

Lencioni

Frank

. Sex-Based Differences in Adult ACL Reconstruction Outcomes. Curr Rev Musculoskelet Med. 2022;15(6):645-650

Brophy

Wright

Matava

MJ.

Cost analysis of converting from single-bundle to double-bundle anterior cruciate ligament reconstruction. Am J Sports Med. 2009;37(4):683-687.

Brown

Jr Carson

EW.

Revision anterior cruciate ligament surgery. Clin Sports Med. 1999;18(1):109-171.

Chen

Allen

Stephens

, et al. Differences in mechanisms of failure, intraoperative findings, and surgical characteristics between single- and multiple-revision ACL reconstructions: a MARS cohort study. Am J Sports Med. 2013;41(7):1571-1578.

Cooper

Dunn

Huston

, et al. Physiologic preoperative knee hyperextension is a predictor of failure in an anterior cruciate ligament revision cohort: a report from the MARS Group. Am J Sports Med. 2018;46(12):2836-2841.

10.

Devana

Shah

Lee

, et al. A novel, potentially universal machine learning algorithm to predict complications in total knee arthroplasty. Arthroplasty Today. 2021;10:135-143.

11.

Ferretti

Ekdahl

Shen

FH.

Osseous landmarks of the femoral attachment of the anterior cruciate ligament: an anatomic study. Arthroscopy. 2007;23(11):1218-1225.

12.

Fischer

Bachmann

Jaeschke

A readers’ guide to the interpretation of diagnostic test properties: clinical example of sepsis. Intensive Care Med. 2003;29(7):1043-1051.

13.

Friedman

JH.

Greedy function approximation: a gradient boosting machine. Ann Stat. 2001;29(5):1189-1232.

14.

George

Dunn

Spindler

KP.

Current concepts review: revision anterior cruciate ligament reconstruction. Am J Sports Med. 2006;34(12):2026-2037.

15.

Harner

Giffin

Dunteman

Annunziata

Friedman

MJ.

Evaluation and treatment of recurrent instability after anterior cruciate ligament reconstruction. Instr Course Lect. 2001;50:463-474.

16.

Harris

Millman

Van Der Walt

, et al. Array programming with NumPy. Nature. 2020;585(7825):357-362.

17.

Hashimoto

Rosman

Rus

Meireles

OR.

Artificial intelligence in surgery: promises and perils. Ann Surg. 2018;268(1):70-76.

18.

Howell

Clark

JA.

Tibial tunnel placement in anterior cruciate ligament reconstructions and graft impingement. Clin Orthop Relat Res. 1992;283:187-195.

19.

Howell

Clark

Farley

TE.

A rationale for predicting anterior cruciate graft impingement by the intercondylar roof. A magnetic resonance imaging study. Am J Sports Med. 1991;19(3):276-282.

20.

Kaeding

Pedroza

Reinke

Huston

; MOON Consortium; Spindler

. Risk Factors and Predictors of Subsequent ACL Injury in Either Knee After ACL Reconstruction: Prospective Analysis of 2488 Primary ACL Reconstructions From the MOON Cohort. Am J Sports Med. 2015;43(7):1583-90.

21.

Kao

Giangarra

Singer

Martin

A comparison of outpatient and inpatient anterior cruciate ligament reconstruction surgery. Arthroscopy. 1995;11(2):151-156.

22.

Kluyver

Ragan-Kelley

Pérez

, et al. Jupyter notebooks—a publishing format for reproducible computational workflows. In: Loizides

Schmidt

, eds. Positioning and Power in Academic Publishing: Players, Agents and Agendas. IOS Press; 2016:87-90.

23.

Kunze

Krivicich

Clapp

, et al. Machine learning algorithms predict achievement of clinically significant outcomes after orthopaedic surgery: a systematic review. Arthroscopy. 2022;38(6):2090-2105.

24.

Kunze

Polce

Ranawat

, et al. Application of machine learning algorithms to predict clinically meaningful improvement after arthroscopic anterior cruciate ligament reconstruction. Orthop J Sports Med. 2021;9(10):23259671211046575.

25.

Forlenza

Cohn

, et al. Machine learning can reliably identify patients at risk of overnight hospital admission following anterior cruciate ligament reconstruction. Knee Surg Sports Traumatol Arthrosc. 2021;29(9):2958-2966.

26.

Magnussen

Debieux

Benjamin

, et al. A CT-based classification of prior ACL femoral tunnel location for planning revision ACL surgery. Knee Surg Sports Traumatol Arthrosc. 2012;20(7):1298-1306.

27.

Makhni

Ramkumar

PN.

Artificial intelligence for the orthopaedic surgeon: an overview of potential benefits, limitations, and clinical applications. J Am Acad Orthop Surg. 2021;29(6):235-243.

28.

MARS Group. Association between graft choice and 6-year outcomes of revision anterior cruciate ligament reconstruction in the MARS cohort. Am J Sports Med. 2021;49(10):2589-2598.

29.

MARS Group. Descriptive epidemiology of the Multicenter ACL Revision Study (MARS) cohort. Am J Sports Med. 2010;38(10):1979-1986.

30.

MARS Group. Effect of graft choice on the outcome of revision anterior cruciate ligament reconstruction in the Multicenter ACL Revision Study (MARS) cohort. Am J Sports Med. 2014;42(10):2301-2310.

31.

MARS Group. Radiographic findings in revision anterior cruciate ligament reconstructions from the MARS cohort. J Knee Surg. 2013;26(4):239-247.

32.

Martin

Wastvedt

Pareek

, et al. Machine learning algorithm to predict anterior cruciate ligament revision demonstrates external validity. Knee Surg Sports Traumatol Arthrosc. 2022;30(2):368-375.

33.

Martin

Wastvedt

Pareek

, et al. Predicting anterior cruciate ligament reconstruction revision: a machine learning analysis utilizing the Norwegian Knee Ligament Register. J Bone Joint Surg Am. 2022;104(2):145-153.

34.

Martin

Wastvedt

Pareek

, et al. Predicting subjective failure of ACL reconstruction: a machine learning analysis of the Norwegian Knee Ligament Register and patient reported outcomes. J ISAKOS. 2022;7(3):1-9.

35.

Morgan

Dahm

Levy

Stuart

Femoral tunnel malposition in ACL revision reconstruction. J Knee Surg. 2012;25(5):361-368.

36.

Novak

Bach

Bush-Joseph

Badrinath

Cost containment: a charge comparison of anterior cruciate ligament reconstruction. Arthroscopy. 1996;12(2):160-164.

37.

Pandas 1.0.0. Accessed October 13, 2024. https://github.com/pandas-dev/pandas

38.

Pedregosa

Varoquaux

Gramfort

, et al. Scikit-learn: machine learning in Python. J Mach Learn Res. 2011;12:2825-2830.

39.

Shah

Devana

Lee

Kianian

van der Schaar

SooHoo

NF.

Development of a novel, potentially universal machine learning algorithm for prediction of complications after total hip arthroplasty. J Arthroplasty. 2021;36(5):1655-1662.

40.

Shah

Devana

Lee

, et al. Machine learning-driven identification of novel patient factors for prediction of major complications after posterior cervical spinal fusion. Eur Spine J. 2022;31(8):1952-1959.

41.

Southam

Colosimo

Grawe

Underappreciated factors to consider in revision anterior cruciate ligament reconstruction: a current concepts review. Orthop J Sports Med. 2018;6(1):2325967117751689.

42.

Spindler

Wright

RW.

Anterior cruciate ligament tear. N Engl J Med. 2008;359(20):2135-2142.

43.

Stevenson

III Johnson

DL.

“Vertical grafts”: a common reason for functional failure after ACL reconstruction. Orthopedics. 2007;30(3):206-209.

44.

Taggart

Kumar

Bickerstaff

Revision anterior cruciate ligament reconstruction: a midterm patient assessment. Knee. 2004;11(1):29-36.

45.

Trojani

Sbihi

Djian

, et al. Causes for failure of ACL reconstruction and influence of meniscectomies after revision. Knee Surg Sports Traumatol Arthrosc. 2011;19:196-201.

46.

UCLA Statistical Consulting Group. Ordinal variables definition: what is the difference between categorical, ordinal and interval variables?Accessed October 13, 2024. https://stats.oarc.ucla.edu/other/mult-pkg/whatstat/what-is-the-difference-between-categorical-ordinal-and-interval-variables/

47.

van Rossum

de Boer

. Interactively testing remote servers using the Python programming language. CWI Q. 1991;4(4):283-303.

48.

Wolfson

Mannino

Owens

Waterman

Alaia

MJ.

Tunnel management in revision anterior cruciate ligament reconstruction: current concepts. Am J Sports Med. 2023;51(2):545-556.

49.

Wright

Dunn

Amendola

, et al. Risk of tearing the intact anterior cruciate ligament in the contralateral knee and rupturing the anterior cruciate ligament graft during the first 2 years after anterior cruciate ligament reconstruction: a prospective MOON cohort study. Am J Sports Med. 2007;35(7):1131-1134.

50.

Wright

Gill

Chen

, et al. Outcome of revision anterior cruciate ligament reconstruction: a systematic review. J Bone Joint Surg Am. 2012;94(6):531-536.

51.

Wright

Magnussen

Dunn

Spindler

KP.

Ipsilateral graft and contralateral ACL rupture at five years or more following ACL reconstruction: a systematic review. J Bone Joint Surg Am. 2011;93(12):1159-1165.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.17 MB