Abstract
Background:
Anterior cruciate ligament (ACL) reconstruction (ACLR) aims to restore knee stability and function; however, recovery outcomes vary widely, highlighting the need for predictive tools to guide rehabilitation and patient readiness.
Purpose:
To identify the most effective machine learning models for predicting the successful recovery of Patient Acceptable Symptom State (PASS) in terms of subjective function, symptoms, and psychological readiness 12 months after ACLR using physical performance measures obtained 3 months after ACLR.
Study Design:
Cohort study; Level of evidence, 3.
Methods:
The authors retrospectively analyzed the data of 113 patients who underwent single-bundle anatomic ACLR. Physical performance measures at 3 months after ACLR included the Y-balance and isokinetic muscle strength tests. The successful recovery of PASS outcomes at 12 months were assessed using the International Knee Documentation Committee (IKDC) and the ACL–Return to Sport after Injury (ACL-RSI) scale. Five machine learning algorithms were assessed: logistic regression, decision tree, random forest, gradient boosting, and support vector machines.
Results:
The gradient boosting model demonstrated the highest area under the curve (AUC) scores for predicting SRPAS of the IKDC (AUC, 0.844; F1, 0.889), and the random forest model demonstrated the highest AUC scores for predicting the successful recovery of PASS of the ACL-RSI (AUC, 0.835; F1, 0.732) during test models. Key predictors of the successful recovery of PASS outcomes included young age and low deficits in the 60 deg/s flexor and extensor peak torque for the IKDC, low 180 deg/s extensor and flexor mean power deficit, and low 60 deg/s flexor peak torque deficits for the ACL-RSI.
Conclusion:
Machine learning showed that younger age and greater 3-month isokinetic strength at 60 deg/s predicted attainment of the successful recovery of PASS of the IKDC at 1 year after ACL. Greater 3-month isokinetic strength at 180 deg/s was most predictive of attaining the successful recovery of PASS of the ACL-RSI at 12 months.
Anterior cruciate ligament (ACL) tears are a widespread and challenging issue for patients, leaving uncertain their ability to resume previous levels of activity.1,25,34 The primary objective of ACL reconstruction (ACLR) and subsequent rehabilitation is to stabilize the knee, enabling a safe return to the preinjury level of sports.2,4,25 Despite advancements in surgical techniques and rehabilitation protocols, as many as 30% of patients encounter knee reinjury after ACLR, increasing the risk of long-term complications.1,6,45 This underscores the need for a comprehensive approach that addresses both the physical and the psychological aspects of recovery.
After ACLR, individuals often grapple with weakened knee muscles due to graft harvesting and quadriceps inhibition, along with compromised postural stability stemming from ACL mechanoreceptor injury.11,19 These factors lead to reduced physical function, presenting substantial hurdles in recovery. Hence, restoring muscle strength, balance, and overall functional performance within the initial 3 months after ACLR is deemed critical for attaining a successful recovery of patient acceptable symptom state (SRPAS).7,15,35 The 3-month mark represents a pivotal phase in rehabilitation, marking the initiation of impactful drills like double-limb jumping tasks and running.15,21 Despite the emphasis on SRPAS, there is a significant gap in understanding early rehabilitation factors predicting SRPAS performance, particularly amid a changing health care landscape with fewer ACLR patient visits. 21
Patient demographics, including age at the time of injury and sex, can play a role in the SRPAS capacity. Additionally, both functional impairments and psychological factors, as reported by patients, significantly affect the SRPAS process.16,46 These functional impairments often manifest as knee pain, joint swelling, instability, and diminished strength.26,27 Additionally, psychological hurdles are also important, including the fear of reinjury, kinesiophobia, and fear of movement extending to kinesiophobia in extreme cases.24,26 Recently, the integration of machine learning into clinical settings has emerged as an advanced methodology for predicting the outcomes of ACLR.23,31,32,48 Most studies have focused on nonmodifiable factors related to surgery or joint conditions, such as age, graft diameter, knee laxity, and posterior tibial slope.36,48 In addition, machine learning analysis of a national knee ligament register predicted the risk of ACLR revision with moderate accuracy based on the creation of an in-clinic calculator for point-of-care risk stratification using the input of 5 variables (age, Knee injury and Osteoarthritis Outcome Score Quality of Life at primary surgery, graft choice, femoral fixation device, and time between injury and primary surgery). 32 However, during the early rehabilitation stage after ACLR, muscle weakness and reduced balance commonly occur for various reasons.13,38,41 Physical performance measures, such as strength, balance, and biomechanical patterns, are modifiable through rehabilitation. Machine learning, a subset of artificial intelligence, employs historical data to predict clinical outcomes and offers insights into factors affecting prediction accuracy.17,18,48 This study aimed to establish targeted rehabilitation strategies for subjective function, symptoms, and psychological readiness after ACLR by predicting SRPAS using early physical performance data.
Despite the acknowledged variability in recovery outcomes after ACLR, there remains a significant gap in our understanding of how early rehabilitation strategies can be optimized to address both physical and psychological factors influencing recovery. While recent studies have begun to explore the potential of machine learning in predicting rehabilitation outcomes, this approach has not been fully leveraged to understand the complex interplay of factors during the critical early post-ACLR period. Thus, this study aimed (1) to determine the most effective machine learning models for predicting SRPAS related to subjective symptoms and psychological readiness 12 months after ACLR, using physical performance variables assessed 3 months after ACLR, and (2) to confirm the key predictors influencing these outcomes. This study offers valuable insights into the rehabilitation strategies for patients recovering from ACLR. It aims to help health care professionals optimize these strategies and improve patient outcomes. This study hopes to provide reliable predictions of the patient's subjective function, symptoms, and psychological readiness at 12 months after surgery by utilizing machine learning models.
Methods
Patients
This study was approved by the ethics committee of Inje University Seoul Paik Hospital (institutional review board No. PAIK 2023-02-009). Retrospective approval was obtained to review the medical records and collect demographic and clinical data from 113 patients who underwent single-bundle anatomic ACLR between June 2016 and April 2022. The inclusion criteria were 18- to 45-year-olds who underwent single-bundle ACLR and completed tests at 3 and 12 months after surgery. Baseline characteristics at 3 months after ACLR were measured using data collected during outpatient visits between 3 and 4 months after surgery. SRPAS outcomes at 12 months after ACLR were assessed using data obtained during outpatient visits between 11 and 13 months after surgery. The exclusion criteria were multiple ligament injuries, bilateral injuries, fractures, meniscus root and radial repair, cartilage repair, mechanical alignment osteotomy, subtotal or total meniscectomy, revision ACLR, and previous knee surgery on the involved and uninvolved sides. Table 1 presents the patient characteristics, and Figure 1 depicts the flowchart. We aimed for a minimum of 90 participants to adhere to the guideline of 1 variable for every 10 events, ensuring comprehensive statistical analysis of the SRPAS with ≤9 variables.
Baseline Characteristics at 3 Months After ACLR and of Outcomes at 3 and 12 Months After ACLR a
Data presented as n or mean ± SD. ACL-RSI, Anterior Cruciate Ligament–Return to Sport after Injury; ACLR, anterior cruciate ligament reconstruction; BMI, body mass index; IKDC, International Knee Documentation Committee form; SRPAS, successful recovery of patient acceptable symptom; YBT, Y-balance test.
IKDC and ACL-RSI scores were dichotomized, with scores ≥75.9 for IKDC and ≥56 for ACL-RSI classified as 1, indicating SRPAS, while scores <75.9 for IKDC and <56 for ACL-RSI were classified as 0, indicating unsuccessful recovery.
Flowchart of patient selection and machine learning algorithm development. ACL, anterior cruciate ligament; ACL-RSI, ACL–Return to Sport after Injury; ACLR, ACL reconstruction; AUC, area under the curve; BMI, body mass index; IKDC, International Knee Documentation Committee; SRPAS, successful recovery of patient acceptable symptom state; YBT, Y-balance test.
Postoperative Rehabilitation
All the patients followed a standardized rehabilitation program involving home-based exercises. Range of motion (ROM) exercises began 3 to 5 days after ACLR, followed by the introduction of weightbearing 1 to 2 days later. Patients were advised to wear a functional knee brace in full extension for the initial 3 weeks, with full weightbearing permitted at 3 weeks after ACLR, determined based on the patient's assessment of knee stability, typically occurring between 4 and 6 weeks after ACLR. In patients who underwent meniscal repairs, partial weightbearing with crutches was mandated for 6 weeks. Open kinetic chain exercises targeting the quadriceps muscle group were initiated between 2 and 6 weeks postoperatively. These exercises were performed in a seated position, without additional resistance, and through a ROM of 90° to 0° of knee flexion. At the 6-week postoperative mark, progressive resistance training using a leg extension machine was introduced. However, to minimize anterior shear forces on the healing graft, the ROM was limited to 90° to 60° of knee flexion. Concurrently, at 6 weeks after surgery, open kinetic chain hamstring curl exercises were commenced, with active curl exercises performed from weeks 6 to 8, followed by progressive resistance exercises from week 8 onward. Following the 12-week postoperative time point, progressive resistance training was advanced, and the ROM restrictions were lifted, allowing for open kinetic chain exercises to be performed from 90° to 0° of knee flexion. Perturbation training was initiated 6 weeks after ACLR. Between 2 and 3 months after ACLR, open kinetic chain quadriceps and hamstring load-progression exercises were begun, involving isometric or isotonic contractions with light-to-moderate external resistance. Running was initiated at 3 months after ACLR, while cutting and pivoting exercises were introduced at 5 months. Sports-specific training began at 6 months, and return to sports was allowed from 9 months after ACLR.
Procedure
All assessments at 3 and 12 months after ACLR were conducted in a random order by one of the authors (J-S.K.). Physical performance variables, serving as features or independent variables, were assessed at 3 months, including the Y-balance test (YBT) and isokinetic muscle strength tests. SRPAS variables, serving as target or dependent variables, were assessed 12 months after ACLR and included the International Knee Documentation Committee (IKDC) and ACL–Return to Sport after Injury (ACL-RSI) forms. The YBT and isokinetic muscle strength tests were conducted randomly at 3 months. The order of the tests was randomized using a randomization website (www.randomization.com).
Patient-Reported Outcomes
To guide return to activity decisions and monitor patient progress, patient-reported outcomes were considered. 22 The IKDC is an 18-item questionnaire designed to assess knee-related symptoms, functional capabilities, and sports participation levels in individuals experiencing a range of knee pathologies, including ligament and meniscal injuries, patellofemoral pain, and articular cartilage lesions. 35 An IKDC score of ≥75.9 was considered indicative of a return to participation for knee-specific measures of function, symptoms, and sport-specific activity. 37 The ACL-RSI, consisting of 12 items examining psychological constructs related to SRPAS, generated a score ranging from 0 to 100. A score of ≥56 indicated psychological readiness to return to functional activities, reflecting patient confidence and risk appraisal. 7
Y-Balance Test
Dynamic balance was assessed using the YBT (Move2Perform).20,30 The YBT device features a central plastic plate with 3 tubes in anterior, posteromedial, and posterolateral positions, each marked at 0.5-cm intervals. Patients, standing barefoot on 1 leg in the center of the YBT with hands on the iliac crest, were instructed to move the pointer as far as possible with the nonsupport limb in the anterior, posteromedial, and posterolateral directions. Patients standing on their weightbearing injured leg in a box were instructed to softly push the side of the box using the unsupported and uninjured leg, reaching as far as possible in 3 directions: anterior, posteromedial, and posterolateral. 30 Each attempt was only recorded when the patient could return to the starting position with full control. Patients underwent 6 practice trials followed by 3 measurement trials of the YBT and the mean of 3 measurements was used for data analysis (unit of measurement: cm). 20
Isokinetic Muscle Strength Test
Isokinetic muscle strength assessments were conducted using the HUMAC-NORM isokinetic extremity system (Computer Sports Medicine Inc).11,20 Measurements were taken with patients seated, employing angular velocities of 60 deg/s and 180 deg/s for the knee extensors’ and flexors' peak torque and mean power, respectively. Peak torque assessment involved patients’ performing 4 repetitions of concentric quadriceps and hamstring contractions at 60 deg/s after 2 practice sessions within 90° to 0° (knee flexion).11,20 Mean power measurements consisted of 10 repetitions at 180 deg/s after 2 practice sessions within the same knee flexion range.11,20 The system automatically logged the highest recorded data from the 4 repetitions of peak torque and the mean power measurements calculated from the 10 repetitions of mean power for both the uninjured and the injured legs in ACLR. Quadriceps and hamstring deficits were determined as percentages using the following formula:
Machine Learning Modeling
Machine learning analysis was performed using the Orange data mining software (Version 3.3.0) and Python (Version 3.6.15; Python Software Foundation). A flowchart of the machine learning algorithm development is shown in Figure 1.
Preprocessing
Nine numerical predictors (age, body mass index [BMI], YBT anterior direction, YBT posteromedial direction, YBT posterolateral direction, 60 deg/s knee extensor deficit, 60 deg/s knee flexor deficit, 180 deg/s knee extensor deficit, and 180 deg/s knee flexor deficit) were included in the present study. The IKDC and ACL-RSI scores were dichotomized, with scores ≥75.9 for IKDC and ≥56 for ACL-RSI classified as 1, indicating SRPAS, while scores <75.9 for IKDC and <56 for ACL-RSI were classified as 0, indicating unsuccessful recovery (Table 1).7,37 For discovering missing values in the data set, we performed exploratory data analysis. However, upon further investigation, we found that there were no actual missing data points that required imputation or removal of instances with unknown values. By using these visualization methods (box plot, scatter plot, and linear projections), we were able to confirm the distribution and properties of each variable, ensuring that the data set was suitable for the subsequent machine learning analysis. In addition, the Kolmogorov-Smirnov normality test was conducted to confirm the assumption of a normal distribution.
Machine Learning Algorithm
We split the entire data set (N = 113) into a training set (81.0%; n = 91) for model development and a test set (19.0%; n = 22) for external validation to predict the performance. Five machine learning algorithms—logistic regression, decision tree, random forest, gradient boosting, and support vector machine—were trained via 5-fold cross-validation. In the selection of machine learning algorithms for this study, we aimed to incorporate a diverse set of models that offered a broad spectrum of strengths in data analysis and prediction capabilities, thereby enhancing the robustness and reliability of our findings. The 5 algorithms chosen were selected based on their complementary characteristics and proven efficacy in various predictive modeling. 18 Logistic regression was chosen for its simplicity and interpretability, providing a solid baseline for performance comparison. It is particularly effective for binary classification problems and offers straightforward insights into the relationship between independent variables and the outcome.17,29 Decision trees were included for their ability to handle nonlinear relationships and their ease of interpretation through a hierarchical structure of decisions. 44 This model can reveal important variable interactions and is intuitive to understand, making it valuable for identifying key predictors in the early stages of rehabilitation. 44 Random forest, an ensemble method that builds multiple decision trees, was selected for its superior accuracy and robustness against overfitting. 9 By aggregating the predictions of numerous trees, it improves prediction accuracy and is effective in handling complex interactions and high-dimensional data. 9 Gradient boosting, another ensemble technique, was chosen for its ability to sequentially correct errors of weak learners, leading to improved model performance. 3 This algorithm is particularly useful for its high predictive accuracy and flexibility in optimizing for various loss functions. 3 Support vector machines were included for their effectiveness in high-dimensional spaces and their capability to model complex nonlinear boundaries. This algorithm is beneficial for its robustness and efficiency, especially in cases where the number of dimensions exceeds the number of samples. 48 Together, these algorithms encompass a range of modeling approaches from simple linear models to complex ensemble and nonlinear methods allowing us to comprehensively explore the predictive potential of early rehabilitation metrics on recovery outcomes.
Model Validation
The primary metric for assessing model performance involved calculating the area under the curve (AUC) for both training and test data sets. Secondary performance metrics such as classification accuracy, recall, precision, and F1 score were also computed for both the training and the test data sets. Classification accuracy represents the proportion of correct predictions (both true positives and true negatives) among the total number of instances evaluated, providing an overall measure of how well the model classifies the data into the correct categories. Recall, also known as sensitivity, is the ratio of true positives to the sum of true positives and false negatives, measuring the model's ability to correctly identify positive instances, in this case, SRPAS. A high recall indicates that the model effectively captures the majority of positive instances. Precision, on the other hand, is the ratio of true positives to the sum of true positives and false positives, quantifying the model's accuracy in predicting positive instances. A high precision suggests that when the model predicts an instance as positive, it is highly likely to be correct. Last, the F1 score is the harmonic mean of precision and recall, providing a balanced measure of the model's performance, particularly useful when the data set has an uneven class distribution. A high F1 score indicates that the model achieves a good balance between precision and recall. The predictive performance of the model was categorized as excellent (≥0.9), good (≥ 0.8 to <0.9), fair (≥0.7 to <0.8), or poor (<0.7), based on the AUC values. 48
To identify critical factors, the feature permutation importance for each predictive variable was determined using the training data.17,48 The analysis quantified the contribution of each feature by measuring the prediction error increase. In feature importance, black bars presented the standard error of the importance measure for each feature, and the blue bars represented the importance scores of each feature. A Shapley Additive Explanation summary plot was created to demonstrate variable importance and direction.18,48 The y-axis displayed predictive variables arranged by importance, with the most critical factors at the top. The analysis focused on arranging predictive variables by their relative importance and visualizing individual contributions to the overall Shapley Addictive Explanation value.
Results
Patient Characteristics
Table 1 shows the mean values and variability of all the variables. The Kolmogorov-Smirnov normality test presented normally distributed data of all the independent variables (P > .05). Our study evaluated 113 patients who underwent ACLR and were analyzed using machine learning models. The mean age of the participants was 29.9 ± 9.5 years, with a male predominance (67.3%). The distribution of graft types included hamstring (43.4%), quadriceps tendon (24.8%), hybrid (16.8%), and allografts (15.0%). The mean IKDC score at 3 months after ACLR was 66.4 ± 13.9. Twelve months after ACLR, 77.9% of patients (88 out of 113) exceeded the IKDC threshold of 75.9, indicating SRPAS for subjective function. For the ACL-RSI, 47.8% of patients (54 out of 113) surpassed the threshold of 56, suggesting satisfactory psychological readiness for return to sports.
Predictive Models of Machine Learning
The performances of the 5 machine learning models in predicting the SRPAS for IKDC and ACL-RSI during model training and testing are summarized in Tables 2 and 3. The importance of feature permutation, Shapley additive explanation, and the receive operating characteristic curve are shown in Figures 2 and 3.
For SRPAS of IKDC, Performance Metrics of 5 Machine Learning Algorithms in the Training and Test Set a
AUC, area under the curve; Acc, Accuracy; IKDC, International Knee Documentation Committee; SRPAS, successful recovery of patient acceptable symptom state.
For SRPAS of ACL-RSI, Performance Metrics of 5 Machine Learning Algorithms in the Training and Test Set a
ACL-RSI, Anterior Cruciate Ligament–Return to Sport after Injury; AUC, area under curve; Acc, Accuracy; SRPAS, successful recovery of patient acceptable symptom state.
(A) Feature permutation importance of gradient boosting model in the training set for predicting SRPAS of the IKDC. (B) Shapley additive explanation analyses of gradient boosting model in the training set for predicting SRPAS of the IKDC. (C) Receiver operating characteristic curves of 5 machine learning algorithms in the training set for predicting SRPAS of the IKDC. AP, average power; AUC, area under the curve; BMI, body mass index; FP, false positive; IKDC, International Knee Documentation Committee; PT, peak torque; SRPAS, successful recovery of patient acceptable symptom state; TP, true positive; YBT, Y-balance test.
(A) Feature permutation importance of gradient boosting model in the training set for predicting SRPAS of the ACL-RSI. (B) Shapley additive explanation analyses of gradient boosting model in the training set for predicting SRPAS of the ACL-RSI. (C) Receiver operating characteristic curves of 5 machine learning algorithms in the training set for predicting SRPAS of the ACL-RSI. ACL-RSI, Anterior Cruciate Ligament–Return to Sport after Injury; AP, average power; AUC, area under the curve; BMI, body mass index; FP, false positive; PT, peak torque; SRPAS, successful recovery of patient acceptable symptom state; TP, true positive; YBT, Y-balance test.
During the training, gradient boosting algorithm models predicting SRPAS for the IKDC (AUC, 0.814 [good]; F1, 0.802) and ACL-RSI (AUC, 0.930 [excellent]; F1, 0.824) performed with the highest AUC. Gradient boosting algorithm models predicting SRPAS for the IKDC performed in the highest AUC during the test model (AUC, 0.844 [good]; F1, 0.889). Random forest algorithm models predicting SRPAS for the ACL-RSI performed in the highest AUC during the test model (AUC, 0.835 [good]; F1, 0.732).
For feature permutation importance, age, 60 deg/s extensor peak torque deficit, and 60 deg/s flexor peak torque deficit were the top 3 predictors of SRPAS for IKDC 12 months after ACLR in the gradient boosting model. The age, 60 deg/s extensor peak torque deficit, and 60 deg/s flexor peak torque deficit were notable contributors, with AUC decreases of 0.045, 0.017, and 0.015, respectively. In addition, 180 deg/s extensor mean power deficit, 180 deg/s flexor mean power deficit, and age were the top 3 predictors of SRPAS for ACL-RSI 12 months after ACLR in the gradient boosting model. The 180 deg/s extensor mean power deficit, 180 deg/s flexor mean power deficit, and age were notable contributors, with AUC decreases of 0.022, 0.020, and 0.012, respectively. In Shapley Additive Explanation summary plot, the y-axis displayed predictive variables arranged by importance, with the most critical factors at the top. Each red point on the x-axis represents an individual's contribution to the overall Shapley Additive Explanation value, with positive contributions extending further to the right. In the gradient boosting model, the top 3 predictors of SRPAS for IKDC at 12 months after ACLR, based on Shapley additive explanations, were young age and low deficits in the 60 deg/s flexor and extensor peak torque. Additionally, low 180 deg/s extensor and flexor mean power deficit and low 60 deg/s flexor peak torque deficit were the top 3 predictors of SRPAS for ACL-RSI 12 months after ACLR in the gradient boosting model.
Discussion
Whenever feasible, interpretable machine learning models should be developed and applied for musculoskeletal disorder prevention by training and evaluating multiple models for comparison.10,18,28 The present study selected 9 predictive variables (age, BMI, 60 deg/s extensor and flexor peak torque deficit, 180 deg/s extensor and flexor mean power deficit, and YBT in the anterior, posteromedial, and posterolateral directions) 3 months after ACLR and 2 outcome variables of SRPAS (IKDC and ACL-RSI) 12 months after ACLR to validate a total of 10 machine learning models, with 5 machine learning algorithms for each clinical outcome. Our study identified the best-performing gradient boosting models during the test model predicting SRPAS for IKDC (AUC, 0.844 [good]; F1, 0.889) and ACL-RSI (AUC, 0.818 [good]; F1, 0.732). Age and 60 deg/s extensor and flexor peak torque deficits were the top predictors of SRPAS for IKDC in the gradient-boosting model of feature permutation importance. The 180 deg/s extensor and flexor mean power deficits and age were the top predictors of SRPAS for ACL-RSI in the gradient-boosting model of feature permutation importance. The variables from our study can serve as guidelines for therapy during the early rehabilitation stage for SRPAS, focusing on subjective symptoms and psychological readiness at 12 months after ACLR. The superiority of the gradient boosting model in our study not only underscores the potential of machine learning in rehabilitation strategies but also invites comparison with traditional approaches documented in the existing literature.
The threshold for the IKDC in SRPAS, indicative of an overall health state where patients consider themselves to be “feeling well,” was determined to be a score of 75.9, with a sensitivity of 0.83 and a specificity of 0.96. 35 Attaining an SRPAS for the IKDC suggests enhanced function, increased sports participation, and improved knee-related quality of life. Menzer et al 33 reported that factors such as age, single-leg hop for distance, triple hop for distance, and mean knee extensor power at 180 deg/s exhibit significant associations with IKDC scores, with correlation coefficients of −0.243 (P = .002), 0.346 (P = .002), 0.282 (P = .011), and 0.222 (P = .038), respectively, in patients with ACLR around 6 months after ACLR. 33 When these variables were analyzed through a stepwise linear regression model, age and single-leg hop symmetry emerged as significant predictors, accounting for 17.7% of the variance in IKDC scores (P = .02). Our study introduces a significant methodological advancement by leveraging a gradient-boosting machine learning model. The gradient boosting model likely outperformed other machine learning models in our study due to its ability to effectively capture complex, nonlinear relationships between the predictor variables and the SRPAS outcomes. Gradient boosting is an ensemble technique that combines multiple weak learners (decision trees) to create a strong predictive model. It iteratively trains each decision tree to focus on the instances that were misclassified by the previous trees, allowing the model to progressively improve its predictions by learning from its mistakes. In contrast to traditional regression models, this model does not assume linear relationships among variables. Rather, it progressively improves its predictions by learning from traditional regression models,33,49 enabling it to effectively manage diverse and complex data related to ACLR recovery. This capability is important, considering the intricate combination of factors, such as physical rehabilitation progress, that might not be completely captured by linear models. The superior performance of our gradient boosting model, as indicated by the AUC values of 0.814 during training and 0.844 during external validation, can be attributed to several key factors. First, gradient boosting effectively harnesses the power of ensemble learning, where multiple weak predictive models combine to form a strong predictor.3,47 This approach inherently improves the model's robustness and accuracy. Second, the model's flexibility allows for the incorporation of a wide range of variables, including dynamic balance measures and detailed isokinetic muscle strength assessments 3 months after ACLR. While previous research has laid the groundwork for understanding ACLR recovery,33,49 our application of gradient boosting models represents a significant methodological leap, offering a more nuanced analysis of the complex interplay between variables affecting SRPAS.
Regarding psychological readiness for SRPAS, previous studies have focused on demographic and clinical variables, often overlooking the critical early rehabilitation physical performance measures, which typically resulted in modest predictive accuracies with AUC values around 0.75.24,46 Previous studies have highlighted the importance of psychological readiness in understanding the multifaceted recovery process after ACLR.7,39,43 Our study diverges by embracing a more comprehensive approach that integrates both physical performance metrics and patient-reported outcomes at 3 months after ACLR. In our research, the random forest algorithm models for predicting SRPAS for ACL-RSI displayed superior performance, achieving an AUC of 0.835 and an F1 score of 0.732 during the test model. These results could be attributed to the random forest algorithm's robustness in handling complex data sets, its ability to perform effective feature selection, 9 and its capacity to capture the intricate interactions between variables influencing psychological readiness for SRPAS. The ensemble approach of the algorithm, which aggregates predictions from multiple decision trees, 9 significantly reduces the risk of overfitting, thereby enhancing the model's accuracy and generalizability in predicting SRPAS based on ACL-RSI scores. Combined, these methodological advancements not only refine our understanding of ACLR but also have direct implications for rehabilitation practice, suggesting more personalized and targeted approaches based on the predictive insights provided by our models.
The predictive outcomes for IKDC scores were substantially better than those for ACL-RSI in our study. This difference in predictive performance could be attributed to the selection of independent variables, which may have been more closely aligned with the functional aspects assessed by the IKDC. The IKDC primarily focuses on knee-related symptoms, function, and sports activity, which are more directly influenced by the physical performance measures included in our study, such as isokinetic strength and power deficits. On the other hand, the ACL-RSI encompasses psychological factors, such as emotions, confidence, and risk appraisal, which may not be as strongly linked to the physical performance measures assessed in our study. This discrepancy highlights the importance of incorporating a broader range of psychological and functional measures in future studies to improve the predictive accuracy of models for ACL-RSI outcomes.
For feature permutation importance and Shapley additive explanation, we identified differences in the key predictors of the SRPAS of IKDC and ACL-RSI within the machine learning model. The identification of 60 deg/s extensor and flexor peak torque deficit as key predictors for IKDC and 180 deg/s extensor and flexor mean power deficit for ACL-RSI could be interpreted as the differential effect of muscle strength and power at varying speeds on functional recovery and psychological readiness for return to sports. Knee flexor and extensor peak torque deficits are primary predictors of IKDC, highlighting the essential role of muscle strength in functional recovery.5,12,14 At a slower speed of 60 deg/s, extensor, and flexor peak torque measurements provide a direct assessment of maximal muscle strength, reflecting the patient's ability to generate force during controlled movements. This could be relevant in the early stages of rehabilitation, where restoring muscle strength is paramount for achieving basic functional goals and ensuring the structural stability of the knee. The peak torque deficit at this speed highlights the importance of addressing significant strength deficits to improve overall knee function, which is directly related to higher IKDC scores, indicating successful functional recovery. Conversely, the 180 deg/s flexor and extensor mean power deficit role as a key predictor for ACL-RSI highlights the importance of muscle power and the ability to perform rapid, dynamic movements in determining psychological readiness for return to sports. The ACL-RSI, which evaluates psychological factors such as emotions, confidence, and risk appraisal related to returning to the sport, is influenced by the individual’s perception of his or her knee's capacity to handle high-speed, sport-specific tasks.40,42,49 At 180 deg/s, flexor, and extensor mean power measurements can assess the muscle's ability to quickly generate force, a critical component of sports performance that involves running, jumping, and changing directions. 8 The focus on 180 deg/s flexor and extensor mean power deficits at this higher speed suggests that improvements in muscle power, particularly in the ability to execute quick and powerful movements, could be important for enhancing an individual’s confidence and reducing fear of reinjury, thereby positively influencing ACL-RSI. The differential emphasis on peak torque deficit at 60 deg/s for IKDC and mean power deficit at 180 deg/s for ACL-RSI reflects the multifaceted nature of ACLR recovery, which encompasses both physical and psychological dimensions. Although IKDC scores are more directly influenced by the foundational strength of the knee, which is essential for basic functional activities and stability, ACL-RSI scores are affected by an individual’s confidence in the ability to perform high-speed, dynamic tasks without fear of reinjury. This distinction underscores the need for a comprehensive rehabilitation approach that addresses both muscle strength at lower speeds for functional recovery and muscle power at higher speeds for psychological readiness, facilitating a holistic recovery process that optimizes both objective functional outcomes and subjective readiness for return to sports. However, it is important to note that the approach of focusing on muscle strengthening at lower speeds, followed by muscle power training at higher speeds once dynamic control has been achieved, is a fundamental principle in ACL rehabilitation.
YBT measures did not significantly contribute to the prediction of SRPAS outcomes in our study. Although the YBT is a widely used tool for assessing dynamic balance and neuromuscular control, its predictive value for long-term outcomes after ACLR may be limited. This finding is intriguing and warrants further discussion. It is possible that the YBT, while useful for evaluating postural control and identifying asymmetries, may not capture the specific functional demands required for successful return to sports. Future studies could explore alternative tests for coordination, proprioception, and balance, such as the single-leg hop tests or dynamic postural control assessments, which may provide better predictive value for long-term outcomes.
These findings highlight the importance of targeted early rehabilitation interventions focusing on improving muscle strength and power at different velocities. While our models offer clinicians the ability to accurately forecast SRPAS as early as 3 months after ACLR, it is important to note that the use of an isokinetic dynamometer for strength testing may limit widespread clinical application. Future research should focus on incorporating more clinically feasible measures to enhance the utility of these predictive models in guiding targeted rehabilitation interventions for patients at risk of suboptimal recovery after ACLR.
Limitations
Although our findings offer promising directions for enhancing ACLR rehabilitation, it is crucial to acknowledge the limitations inherent in our study design, which may affect the generalizability and applicability of our predictive models. The sex-specific imbalances identified in the present study represent significant limitations that could affect the generalizability and accuracy of our findings. The overrepresentation of male participants in our study could restrict the relevance of our findings across a more varied and inclusive demographic group. This discrepancy highlights the importance of incorporating a more equitable gender representation in future studies to ensure that outcomes are representative of the entire population after ACLR. Second, the disproportion between the number of SRPAS for IKDC within our data set introduces a potential for bias, which could tilt the model's accuracy in favor of the predominant group. Employing strategies to correct this disproportion could improve the effectiveness of the model in accurately distinguishing between individuals from diverse categories. Third, the retrospective design and reliance on data from a single institution may have limited the generalizability of our findings. Future studies could use a multicenter approach to validate our model across diverse patient populations. Fourth, our study did not involve graft type as a potential predictor variable. Graft type has been shown to influence outcomes after ACLR, with differences in healing properties, donor-site morbidity, and postoperative rehabilitation protocols. However, due to the relatively small sample size in our study, including graft type as a predictor variable may have led to overfitting and reduced the generalizability of our models. Future studies with larger sample sizes should consider incorporating graft type as a potential predictor to provide a more comprehensive understanding of the factors influencing SRPAS outcomes. Fifth, our study focused exclusively on patients undergoing single-bundle anatomic ACLR, excluding those with different surgical techniques or concomitant injuries, which may have affected the applicability of our results to all individuals recovering from ACLR. Additionally, the exclusion of psychological measures beyond the ACL-RSI meant that other relevant psychological factors influencing recovery were not considered. Sixth, machine learning models perform best when trained on very large data sets, such as those from national registries. Our study's sample size of 81.0% (n = 91) for modeling and 19.0% (n = 22) for testing may limit the definitive conclusions that can be drawn from our model and its usefulness in a clinical setting. Future studies should aim to include larger sample sizes to improve the robustness and generalizability of the predictive models. Seventh, our study did not incorporate measures of physical examination, such as knee effusion, laxity, pivot shift, and ROM, which could potentially influence SRPAS outcomes. Future studies should consider including these clinical factors to provide a more comprehensive assessment of the patient's recovery status. Eighth, we did not account for the type and level of sports activity in our patients, which may have had an effect on their rehabilitation progress and SRPAS outcomes. Incorporating information about the patient's specific sport and competitive level could provide valuable insights into the factors influencing their recovery. Ninth, our study did not consider the time from injury to surgery, which may have affected the patient's preoperative status and subsequent rehabilitation outcomes. Future studies should include this variable to assess its potential effect on SRPAS outcomes. Finally, our study did not assess other tests that have been used to predict return to sports, such as hop tests. Incorporating these additional measures could provide a more comprehensive evaluation of the patient's functional recovery and readiness to return to sports. While our machine learning models demonstrated good predictive performance, the complexity of these models may pose challenges for implementation in clinical settings without specialized expertise. Future research should aim to simplify these models without compromising their predictive accuracy or enhancing their utility for clinicians.
Conclusion
The present study demonstrates the effectiveness of machine learning models in predicting the achievement of SRPAS at 12 months after ACLR based on early rehabilitation outcomes at 3 months. Machine learning analysis revealed that younger age and greater isokinetic strength at 60 deg/s at 3 months after ACLR were the most significant predictors for attaining SRPAS of IKDC at 1 year. Additionally, greater isokinetic strength at 180 deg/s at 3 months after ACLR was found to be the most predictive factor for achieving SRPAS of ACL-RSI at 1 year.
Footnotes
Acknowledgements
We thank Sports Medical Center team in Seoul Paik Hospital for their active participation and cooperation.
Final revision submitted August 6, 2024; accepted September 6, 2024.
The authors declared that there are no conflicts of interest in the authorship and publication of this contribution. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
Ethical approval for this study was obtained from Inje University Seoul Paik Hospital (reference No. PAIK 2023-02-009).
Data Accessibility Statement
The data sets analyzed during the current study are available from the corresponding author upon reasonable request.
