Determining the Substantial Clinical Benefit Values for Patient-Reported Outcome Scores After Primary ACL Reconstruction

Abstract

Background:

Little is known about the threshold of patient-reported outcomes (PROs) at which patients perceive to have attained a substantial clinical benefit (SCB) after anterior cruciate ligament reconstruction (ACLR).

Purpose:

To determine the SCB value of PROs 1 year after ACLR in the general population and to determine factors that predict SCB attainment.

Study Design:

Case series; Level of evidence, 4.

Methods:

The Lysholm, International Knee Documentation Committee Subjective Knee Evaluation Form (IKDC-SKF), Tegner, and Anterior Cruciate Ligament–Return to Sport after Injury (ACL-RSI) scores were examined postoperatively in 88 patients who underwent ACLR. At the 12-month follow-up visit, the patients answered 2 independent anchor questions about daily discomfort and functional recovery. Receiver operating characteristic (ROC) curve analysis was used to differentiate PRO scores between patients who responded as having no difficulty versus some difficulty with daily knee discomfort (anchor question 1) and between patients with substantial recovery versus nonsubstantial recovery of knee function (anchor question 2). The area under the ROC curve (AUC) was calculated to assess reliability, and the differences between the AUC values were compared. Multivariate logistic regression analyses were performed to determine predictors affecting SCB attainment.

Results:

There were 76 patients enrolled in this study. The 12-month Lysholm, IKDC-SKF, Tegner, and ACL-RSI scores that corresponded to the SCB were 88.0, 85.1, 6.5, and 64.2 for anchor question 1 and 84.5, 77.7, 5.5, and 57.1 for anchor question 2, respectively. The AUC values obtained from the ROC curve analyses showed acceptable to excellent reliability (anchor question 1: Lysholm, 0.90; IKDC-SKF, 0.86; Tegner, 0.71; ACL-RSI, 0.92; anchor question 2: Lysholm, 0.80; IKDC-SKF, 0.90; Tegner, 0.82; ACL-RSI, 0.82) and were all statistically significant (P < .001 to P = .028). For all PROs, younger age (Odds Ratio (OR), 0.88-0.94; P < .001 to P = .027) and greater muscle strength (OR, 1.03-1.07; P < .001 to P = .023) were predictive factors of SCB attainment.

Conclusion:

The postoperative Lysholm, IKDC-SKF, Tegner, and ACL-RSI scores showed acceptable to excellent reliability in predicting the SCB after ACLR in the general population. Age at surgery and thigh muscle strength influenced SCB attainment for all PROs.

Keywords

anterior cruciate ligament reconstruction patient-reported outcome significant clinical benefit general population

In recent decades, patient-reported outcomes (PROs) have been used to help assess the postoperative status of patients who have undergone anterior cruciate ligament (ACL) reconstruction (ACLR).^3,4,12,28,33 Even if these PROs indicate a statistically significant improvement postoperatively, the data may not be clinically relevant or meaningful to patients. Consequently, the minimal clinically important difference (MCID) was proposed for the evaluation of meaningful outcome improvement, defined as the least difference in outcome perceived as clinically important by the patient.¹⁴ However, the MCID value only represents the minimum requirement for clinical improvement, and most patients who have undergone ACLR show an improvement more than the MCID postoperatively.²³

An alternative benchmark for deciding whether there is a clinically significant improvement in patients who have undergone ACLR is the substantial clinical benefit (SCB). Originally proposed by Glassman et al,⁷ the SCB is defined as the clinical improvement that represents an SCB. The most critical difference between the SCB and MCID is that the SCB is inherently more of a goal in terms of defining clinical success than is the MCID, which is considered as a floor value.⁷ No previous reports, however, have specifically defined the SCB for patients after ACLR and the factors affecting SCB attainment.

Additionally, to predict ACLR outcomes, most previous studies have focused on whether athletes were able to return to sports (RTS) postoperatively.^15,23 In the general population, patients tend to also be concerned with whether they will experience problems in their daily lives, including performing daily tasks after surgery.¹⁹ To our knowledge, no previous study has evaluated the predictive value of PROs on daily activity after ACLR in individuals in the general population who participate in recreational sporting activities.

The purpose of the present study was to determine the SCB value of PROs 1 year after ACLR in the general population and to define the influencing factors that can be used to predict attaining an SCB. We hypothesized that the PROs commonly used in ACLR would have acceptable reliability in predicting SCB attainment and that previously identified factors, such as characteristic factors and isokinetic tests, would be able to predict attaining an SCB for each PRO.

Methods

This study was designed as a retrospective analysis of data obtained via a prospective longitudinal observational study of primary ACLR surgery, conducted between June 2016 and December 2019. All operations were performed by a senior surgeon (J.H.B.). All patients were given written information about the study, and informed consent was obtained. Ethics approval was obtained from the institutional review board of our hospital.

Participants

Of the patients who had undergone primary ACLR at our institution, those who underwent all evaluations at the 1-year postoperative follow-up were eligible for inclusion. Patients consisted of a general population who participated in recreational sporting activity. Those with >6 months between the injury and surgery were excluded because of differences in postoperative emotional responses and activity levels compared with those who underwent surgery closer to the time of injury.¹ Patients with other concomitant ligament injuries requiring surgical treatment, previous knee surgeries, secondary ACL injuries (contralateral ACL rupture, ipsilateral ACL graft failure), and postoperative infection were excluded. Characteristic data were collected from electronic medical records.

All patients underwent anatomic single-bundle ACLR, and the drilling of the femoral tunnel was performed using the outside-in technique. The graft types used for ACLR consisted of 4-strand gracilis and semitendinosus tendon (hamstring) autografts (n = 60; 78.9%), bone–patellar tendon–bone autografts (n = 8; 10.5%), and fresh-frozen Achilles tendon allografts (n = 8; 10.5%). To select the ACL graft type, the patient was asked to select either an autograft or allograft after the risks and benefits of both alternatives were elucidated. After the operation, the same rehabilitation protocol was applied to all patients.

Partial weightbearing was permitted immediately after surgery, and this was progressed to full weightbearing depending on patient tolerance. Range of motion and isometric quadriceps exercises were started 1 to 2 days after surgery. Gait training, balance exercises, proprioceptive exercises, and closed kinetic chain strengthening exercises were started 3 to 4 weeks postoperatively, and open kinetic chain exercises were started 6 to 8 weeks postoperatively. Patients were allowed to run on a treadmill 3 to 4 months after surgery and then progressed to advanced exercises.

Patient Assessment

Physical examination of instability, including the Lachman test, anterior drawer test, and pivot-shift test, was performed at each postoperative visit. Concurrently, independent staff (experienced athletic trainers and physical therapists) from our sports rehabilitation center conducted all isokinetic tests and obtained the PROs.

Three types of PROs were used as primary outcomes for this study: the Lysholm, International Knee Documentation Committee Subjective Knee Evaluation Form (IKDC-SKF), and Tegner activity scores. These 3 measures have been validated for various knee conditions, including chondral disorders, meniscal injuries, and ACL injury.^4,12,34 Additionally, we used the Anterior Cruciate Ligament–Return to Sport after Injury (ACL-RSI) scale, consisting of a specific 12-item questionnaire, to evaluate the psychological effect of returning to sports and recreational activities after ACLR.^21,33

Isokinetic tests were performed on both quadriceps and hamstring muscles, and values were calculated as limb symmetry indices (LSIs) to objectively determine the discrepancies between the legs. Strength was tested at 60 deg/s, which is a common angular velocity for assessing strength before and after ACLR.^13,32 Numerical values were expressed as a percentage. In addition, the Genourob (GNRB) arthrometer (Genourob) test was performed to measure the knee joint laxity at 9 to 12 months (mean, 10.6 ± 3.0 months) after ACLR. All patients were examined on both the injured and uninjured knees, and the measurements were repeated 3 times with a force of 134 N. Afterward, 3 measurements for the injured and uninjured sides were averaged, respectively, and the results were expressed as a mean side-to-side difference (STSD; in millimeters). The previous study has demonstrated the better diagnostic value of GNRB test in ACL injuries, including the testing force of 134 N.²⁹

Anchor-Based SCB Calculations

An anchor-based method was used to calculate the SCB for each PRO. At the 12-month follow-up, patients were asked 2 independent anchor questions to evaluate their overall daily activity level and functional recovery. The first question was derived from question 4 of the Knee Injury and Osteoarthritis Outcome Score (KOOS)-Quality of Life (QOL) subscale: “In general, how much difficulty do you have with your knee?” Previous studies have recommended the use of the KOOS-Sport/Recreation and KOOS-QOL subscales as primary outcomes in patients with ACL injury because these subscales had the highest content validity and greatest room for improvement and showed the largest effect sizes after ACLR.6,28 Question 4 of the KOOS-QOL asks about the discomfort that patients experienced in their daily life during the last week (“none,” “mild,” “moderate,” “severe,” and “extreme”). Despite the lack of surgical intervention specificity, the question is easy to answer. It is also clinically meaningful, as it quantifies the patient’s daily discomfort. The difference between “none” and “mild” was used to estimate the SCB.

The second question was a global assessment rating anchor about function: “How much knee function do you have compared to before the injury?” Possible answers ranged from 1 (indicating no recovery compared with preinjury status) to 10 (indicating the same function as during preinjury sports activities), with a higher score indicating that the patient perceived knee function as improved. The psychometric nature of this scale is unknown because it is part of our self-designed questionnaire regarding sports abilities, but it has the advantage of focusing more specifically on subjective functional recovery after the injury. We considered a score ≥8 to indicate SCB. This cutoff value was set as a scale corresponding to the top 20% to 33% in other precedent studies with anchor questions using a scale ≥10.^10,27

To determine which value was more reliable, we analyzed 2 sets of scores for each PRO: the net score change from preoperatively to 12 months postoperatively and the raw score at 12 months postoperatively. Receiver operating characteristic (ROC) curve analysis was performed to define SCB values that best discriminated between the no-difficulty group (“none”) and the some-difficulties group (“mild,” “moderate,” “severe,” “extreme’’) for the first anchor question and the substantial recovery group (≥8 points) and nonsubstantial recovery group (<8 points) for the second anchor question. The cutoff values were calculated using the Youden index. The area under the ROC curve (AUC) was calculated to assess reliability. An AUC value of 0.7 to 0.8 was regarded as acceptable, and an AUC value of 0.8 to 0.9 was regarded as excellent. Additionally, the differences between the AUC values were compared, and the ROC power test was performed to verify that the sample size was sufficient by substituting the number of positive and negative groups of each anchor question and the AUC values of each PRO. A power value >0.8 was considered acceptable.

Statistical Analysis

Continuous variables are presented as mean ± standard deviation, and categorical variables are presented as frequency and percentage. The normality of data distribution was tested using the Kolmogorov-Smirnov test for all evaluation periods.

Repeated-measures analysis of variance (ANOVA) was used to assess the significance of changes in PROs over time. The sphericity assumption was evaluated using the Mauchly test, and, if not met, then Greenhouse-Geisser or Huynh-Feldt correction was used according to the ε value. When comparing the mean values of isokinetic tests according to the types of ACL graft, we used the Welch robust ANOVA because of the difference in sample size between groups. Additionally, the Bonferroni method was used to adjust for multiple comparisons of means. The threshold for significant group differences was set at P < .05.

Multivariate logistic regression analyses were performed to determine and model the relationship between each PRO and other predictors of SCB attainment. Factors with P < .05 were included stepwise in the multivariate logistic regression analysis to evaluate interactions between predictive factors. The SPSS software (version 21.0; SPSS, Chicago, IL, USA) was used for the statistical analyses.

Results

A total of 88 patients met all inclusion criteria. Of these, 76 patients who underwent all evaluations at 12 months postoperatively were ultimately enrolled in this study (Figure 1).

Figure 1.

Study flowchart. ACL, anterior cruciate ligament; ACLR, anterior cruciate ligament reconstruction; postop, postoperative; preop, preoperative; PROs, patient-reported outcomes.

Table 1 summarizes the patient characteristics and clinical variables. In the isokinetic tests, the postoperative 12-month LSI values of quadriceps (P < .001) and hamstring muscle (P = .002) were significantly improved compared with the postoperative 6-month LSI values. There was no significant difference between the ACL graft types in quadriceps and hamstring isokinetic strength at the preoperative (quadriceps and hamstring, P = .332 and P = .447, respectively), 6-month postoperative (P = .216 and P = .257), or 12-month postoperative (P = .161 and P = .704) points. On the GNRB test performed after the surgery, the difference from the uninjured side was measured to be 0.5 mm on average.

Table 1

Characteristics and Clinical Variables of the Study Cohort (n = 76) ^a

Variable	Value
Characteristic and surgical variables
Age at surgery, y	28.6 ± 12.0
Male sex	65 (85.5)
BMI	24.6 ± 3.3
Right side involved	41 (53.9)
Time from injury to ACLR, wk	6.7 ± 5.3
Graft type
Hamstring autograft	60 (78.9)
BPTB autograft	8 (10.5)
Achilles tendon allograft	8 (10.5)
Concomitant injury
Yes	47 (61.8)
No	29 (38.2)
Preinjury Tegner score	8.0 ± 1.5
Returned to preinjury level	16 (21.1)
Preoperative physical examination
Lachman test 3+	51 (67.1)
Pivot-shift test 3+	40 (52.6)
Anterior drawer test 3+, STSD	24 (31.6)
Isokinetic tests, LSI, %
Preoperative
Quadriceps	62.3 ± 23.4
Hamstring	70.5 ± 36.1
6 mo postop
Quadriceps	62.9 ± 21.0
Hamstring	78.7 ± 20.1
12 mo postop
Quadriceps	73.5 ± 23.0
Hamstring	87.4 ± 15.5
Postoperative GNRB test, STSD, mm ^b	0.5 ± 1.7

^a Data are presented as mean ± SD or n (%). ACLR, anterior cruciate ligament reconstruction; BMI, body mass index; BPTB, bone–patellar tendon–bone; GNRB, Genourob; LSI, limb symmetry index; postop, postoperative; STSD, side-to-side difference.

^b n = 70.

PRO Scores and Anchor Questions

The average values over time of the 4 primary outcome variables are shown in Table 2. All PROs had significant linear improvements over time (P < .001), except for the difference in the ACL-RSI scale from 6 to 12 months postoperatively (P = .177). For the other 3 PROs, the postoperative 12-month value was significantly higher than the postoperative 6-month value (Lysholm, P < .001; IKDC-SKF, P < .001; Tegner, P = .047).

Table 2

Overall Patient-Reported Outcome Scores ^a

	Preoperative	6 Mo Postop	12 Mo Postop	P	95% CI
Lysholm	52.8 ± 28.9	76.1 ± 17.3	81.8 ± 15.7	<.001 ^b	22.0 to 36.0 ^b
IKDC-SKF	47.7 ± 16.2	67.8 ± 16.7	75.5 ± 15.6	<.001 ^b	23.6 to 32.2 ^b
Tegner	2.7 ± 1.8	5.0 ± 1.7	5.6 ± 1.8	<.001 ^b	2.3 to 3.3 ^b
ACL-RSI	—	46.1 ± 22.1	49.0 ± 24.2	.177 ^c	–7.0 to 1.3 ^c

^a Data are presented as mean ± SD. Bolded P values indicate a statistically significant difference between groups as specified (P < .05). ACL-RSI, Anterior Cruciate Ligament–Return to Sport after Injury; IKDC-SKF, International Knee Documentation Committee Subjective Knee Evaluation Form; Postop, postoperative. A dash indicates that the score was not investigated.

^bP value and 95% CI for the difference between the preoperative and 12-month postoperative groups.

^cP value and 95% CI for the difference between the 6- and 12-month postoperative groups.

For the first anchor question regarding daily discomfort, most patients responded “mild” (n = 44; 57.9%), while 11 patients (14.5%) answered “none.” No patient had extreme discomfort in daily life. Therefore, 11 and 65 patients were assigned to the no-difficulty and some-difficulties groups, respectively, for the SCB calculation. For the second anchor question regarding postoperative knee function, 23 patients (30.3%) had a score ≥8, while 53 patients had a score <8 (69.7%). Hence, 23 and 53 patients were assigned to the substantial recovery and nonsubstantial recovery groups, respectively, for SCB calculation (Table 3).

Table 3

Responses to Anchor Questions

Anchor Question	n (%)	Study Group
1. In general, how much difficulty do you have with your knee?
None	11 (14.5)	No difficulty
Mild	44 (57.9)	Some difficulty
Moderate	16 (21.1)	Some difficulty
Severe	5 (6.6)	Some difficulty
Extreme	0 (0)	Some difficulty
2. How much knee function do you have compared to before the injury?
1	4 (5.3)	Nonsubstantial recovery
2	3 (3.9)	Nonsubstantial recovery
3	6 (7.9)	Nonsubstantial recovery
4	5 (6.6)	Nonsubstantial recovery
5	14 (18.4)	Nonsubstantial recovery
6	4 (5.3)	Nonsubstantial recovery
7	17 (22.4)	Nonsubstantial recovery
8	12 (15.8)	Substantial recovery
9	8 (10.5)	Substantial recovery
10	3 (3.9)	Substantial recovery

Regarding the first anchor question, the no-difficulty group tended to have a higher mean value for PROs of raw postoperative and net change compared with the some-difficulties group (Table 4). A similar trend was identified in the subgroup analysis of the second anchor question (Table 5). Additionally, in contrast to the results of the analysis conducted on all patients, the ACL-RSI scale of the patients who attained the SCB for both anchor questions was significantly improved at 12 months postoperatively compared with 6 months postoperatively (first anchor question, P = .003; second anchor question, P = .008). Patients who did not attain the SCB had a rather decreased ACL-RSI scale at 12 months postoperatively compared with 6 months postoperatively (first anchor question, P = .930; second anchor question, P = .592).

Table 4

Pre- and Postoperative PRO Scores in the No-Difficulty and Some-Difficulties Groups (Anchor Question 1) ^a

	No-Difficulty Group, n = 11	Some-Difficulties Group, n = 65	P	95% CI ^b
Lysholm
Preoperative	54.2 ± 32.6	52.5 ± 28.5	.862	–17.2 to 20.6
1 y postoperative	95.7 ± 3.8	79.4 ± 15.8	<.001	11.8 to 20.9
Net change	41.5 ± 32.1	26.9 ± 30.3	.145	–5.2 to 34.5
IKDC-SKF
Preoperative	49.4 ± 11.8	47.3 ± 16.8	.691	–8.4 to 12.7
1 y postoperative	90.0 ± 9.7	73.1 ± 15.1	.001	7.5 to 26.3
Net change	40.5 ± 14.4	25.8 ± 18.6	.014	3.0 to 26.5
Tegner
Preoperative	2.7 ± 2.0	2.7 ± 1.8	.973	–1.1 to 1.2
1 y postoperative	6.9 ± 2.6	5.3 ± 1.6	.077	–0.2 to 3.4
Net change	4.2 ± 3.2	2.6 ± 1.9	.141	–0.6 to 3.7
ACL-RSI
6 mo postoperative	63.8 ± 26.8	43.2 ± 19.9	.004	7.0 to 34.2
1 y postoperative	84.5 ± 19.5	43.0 ± 19.3	<.001	29.0 to 54.1
Net change	20.8 ± 21.2	–0.2 ± 16.0	<.001	10.0 to 31.8

^a Data are presented as mean ± SD. Bolded P values indicate a statistically significant difference between groups (P < .05). ACL-RSI, Anterior Cruciate Ligament–Return to Sport after Injury; IKDC-SKF, International Knee Documentation Committee Subjective Knee Evaluation Form; PRO, patient-reported outcome.

^b 95% CI for the difference between the no-difficulty and some-difficulties groups.

Table 5

Pre- and Postoperative PRO Scores in the Substantial Recovery and Nonsubstantial Recovery Groups (Anchor Question 2) ^a

	Substantial Recovery Group, n = 23	Nonsubstantial Recovery Group, n = 53	P	95% CI ^b
Lysholm
Preoperative	62.5 ± 24.2	48.7 ± 30.2	.058	–0.5 to 28.0
1 y postoperative	92.1 ± 5.6	77.5 ± 16.7	<.001	9.5 to 19.8
Net change	29.7 ± 26.4	28.8 ± 33.0	.908	–14.7 to 16.5
IKDC-SKF
Preoperative	49.3 ± 15.2	46.8 ± 16.8	.547	–5.7 to 10.6
1 y postoperative	88.7 ± 7.0	69.8 ± 14.9	<.001	13.8 to 23.9
Net change	39.4 ± 14.8	23.0 ± 18.3	<.001	7.8 to 25.1
Tegner
Preoperative	3.2 ± 1.9	2.5 ± 1.7	.101	–0.1 to 1.6
1 y postoperative	7.0 ± 1.8	4.9 ± 1.4	<.001	1.4 to 2.9
Net change	3.8 ± 2.3	2.4 ± 2.0	.009	0.4 to 2.4
ACL-RSI
6 mo postoperative	56.0 ± 22.7	42.0 ± 20.7	.01	3.4 to 24.7
1 y postoperative	68.3 ± 22.0	40.6 ± 20.3	<.001	17.3 to 38.1
Net change	12.2 ± 19.2	–1.4 ± 16.5	.003	4.9 to 22.3

^b 95% CI for the difference between the substantial recovery and nonsubstantial recovery groups.

SCB by Anchor Question 1

The 12-month postoperative PRO scores that corresponded to the SCB for anchor question 1 are shown in Table 6. The AUC values were all acceptable and statistically significant, especially for the Lysholm, IKDC-SKF, and ACL-RSI, which showed excellent reliability. In the analysis of net score changes, the AUC was within the acceptable range and was statistically significant only for the IKDC-SKF (AUC, 0.73; P = .016). When comparing the 12-month postoperative PRO scores to the net score changes, only the Lysholm score had a significantly greater AUC for the raw score versus the net score change (P = .005). At 12 months postoperatively, the AUCs for the Lysholm (P = .041), IKDC-SKF (P = .002), and ACL-RSI (P = .013) were significantly greater than the AUC for the Tegner score.

Table 6

SCB Values for the PRO Scores by Anchor Question ^a

	SCB	AUC ^b	P	Sensitivity, %	Specificity, %	Achieved SCB, n (%)
Anchor question 1
12-mo postop score
Lysholm	88.0	0.90	<.001	100.0	64.6	34 (44.7)
IKDC-SKF	85.1	0.86	<.001	81.8	81.5	19 (25.0)
Tegner	6.5	0.71	.028	54.5	78.5	20 (26.3)
ACL-RSI	64.2	0.92	<.001	81.8	92.3	14 (18.4)
Net score change
Lysholm	24.0	0.63	.174	72.7	53.8	38 (50.0)
IKDC-SKF	32.3	0.73	.016	81.8	60.0	35 (46.1)
Tegner	4.5	0.67	.067	45.5	86.2	14 (18.4)
Anchor question 2
12-mo postop score
Lysholm	84.5	0.80	<.001	91.3	67.3	38 (50.0)
IKDC-SKF	77.7	0.90	<.001	95.7	73.1	36 (47.4)
Tegner	5.5	0.82	<.001	78.3	73.1	32 (42.1)
ACL-RSI	57.1	0.82	<.001	69.6	84.6	24 (31.6)
Net score change
Lysholm	4.5	0.52	.739	91.3	30.8	49 (64.4)
IKDC-SKF	29.9	0.74	.001	82.6	59.6	34 (44.7)
Tegner	2.5	0.67	.019	78.3	48.1	38 (50.0)

^a Bolded P values indicate a statistically significant AUC value (P < .05). ACL-RSI, Anterior Cruciate Ligament–Return to Sport after Injury; AUC, area under the curve; IKDC-SKF, International Knee Documentation Committee Subjective Knee Evaluation Form; postop, postoperative; PRO, patient-reported outcome; SCB, substantial clinical benefit.

^b AUC values: 0.7-0.8 = acceptable, 0.8-0.9 = excellent.

In the power analysis, excellent power values (>0.99) were calculated for PRO scores at 12 months postoperatively, except for the Tegner score (0.63). In contrast, all the ROC analyses conducted using the net score changes showed unacceptable power values (Lysholm, 0.29; IKDC-SKF, 0.71; Tegner, 0.45).

SCB by Anchor Question 2

The 12-month postoperative PRO scores that corresponded to the SCB for anchor question 2 are listed in Table 6. The AUC values all indicated excellent reliability. In the analyses of net score change values, the IKDC-SKF (AUC, 0.74; P = .001) showed acceptable reliability. All 3 PROs available for comparison had significantly greater AUCs for the 12-month postoperative scores than for the net score changes (Lysholm, P = .001; IKDC-SKF, P = .038; Tegner, P = .020). For AUC values analyzed by the 12-month postoperative PRO scores, there was no significant difference among the 4 PROs except for between the Lysholm and IKDC-SKF (P = .039).

Excellent power values (>0.99) were calculated from ROC analyses based on 12-month postoperative PRO scores. However, the ROC analyses performed using the net score changes showed power values <0.80 (Lysholm, 0.05; Tegner, 0.68), except for the IKDC-SKF (0.94).

Multivariate Regression Analysis

Four separate multivariate analyses were performed to identify the predictive factors for SCB attainment according to 12-month postoperative PRO score (which was more reliable than net score changes according to AUC analysis) (Table 7). The only factor that remained in the regression models of main outcomes was age at surgery. Younger age was predictive of achieving SCB with various PROs. Additionally, higher pre- and postoperative LSI values promoted achievement of SCB in most PROs.

Table 7

Results of Multivariate Regression Analysis of Factors Affecting SCB Attainment by 12-Month Postoperative PRO Score ^a

	SCB Attainment, OR (95% CI)	P
Anchor question 1
Lysholm
Age at surgery	0.9 (0.85-0.96)	.001
6-mo postop quadriceps LSI	1.06 (1.02-1.10)	.002
12-mo postop hamstring LSI	1.06 (1.01-1.12)	.023
IKDC-SKF
Age at surgery	0.93 (0.87-0.99)	.027
12-mo postop quadriceps LSI	1.04 (1.01-1.07)	.016
Tegner
Age at surgery	0.88 (0.81-0.96)	.003
Preop hamstring LSI	1.03 (1.00-1.05)	.020
6-mo postop quadriceps LSI	1.04 (1.00-1.07)	.063
ACL-RSI
12-mo postop quadriceps LSI	1.03 (1.00-1.06)	.035
Anchor question 2
Lysholm
Age at surgery	0.91 (0.86-0.96)	.001
6-mo postop quadriceps LSI	1.07 (1.03-1.10)	<.001
IKDC-SKF
Age at surgery	0.94 (0.90-0.99)	.013
12-mo postop quadriceps LSI	1.04 (1.01-1.07)	.002
Tegner
Age at surgery	0.89 (0.83-0.95)	<.001
Preop hamstring LSI	1.03 (1.01-1.05)	.005
ACL-RSI
6-mo postop hamstring LSI	1.05 (1.01-1.09)	.012

^a ACL-RSI, Anterior Cruciate Ligament–Return to Sport after Injury; IKDC-SKF, International Knee Documentation Committee Subjective Knee Evaluation Form; LSI, limb symmetry index; postop, postoperative; OR, odds ratio; Preop, preoperative; PRO, patient-reported outcome; SCB, substantial clinical benefit.

Discussion

We found that all the 12-month postoperative PRO score thresholds calculated for each anchor question had acceptable to excellent predictive reliability for attaining the SCB. This was particularly the case regarding the ACL-RSI score for anchor question 1 (AUC, 0.92; P < .001) and the IKDC-SKF score for anchor question 2 (AUC, 0.90; P < .001). Conversely, thresholds of the net PRO score changes had unacceptable reliability (AUC, 0.52-0.74; P = .001 to P = .739). Younger age at the time of surgery (OR, 0.88-0.94; P < .001 to P = .027) and stronger pre- and postoperative thigh muscle (OR, 1.03-1.07; P < .001 to P = .023) were positive predictive factors for attaining SCB.

Since Glassman et al⁷ introduced the concept of SCB after lumbar spine arthrodesis, SCB after surgical treatment for various orthopaedic diseases has been increasingly reported. For ACLR, SCB has not been reported; rather, the MCID and patient-acceptable symptom state (PASS) have been used to define clinical success.^9,11,20 However, the MCID cannot capture changes perceived as substantial by the patient, and the PASS does not reflect a meaningful recovery of daily function postoperatively.^20,23 Thus, our findings are meaningful, as they show the probability of obtaining SCB in terms of daily activities via surgical management and facilitate treatment choices for both the patient and physician.⁷

We attempted to define the SCB value for the raw postoperative score and the net outcome change for each PRO. The SCB derived from a raw postoperative score allows independent assessment of postoperative success in other studies that did not report preoperative scores.²² The SCB values of the first anchor question calculated from the raw postoperative PROs can be compared with the PASS threshold values in previous studies of patients who underwent ACLR. Muller et al²⁰ defined the PASS threshold for the IKDC-SKF as 75.9 at an average of 3.4 years after ACLR. This corresponds to our study threshold of 85.1 for the IKDC-SKF 1 year postoperatively. Considering the longer follow-up length and higher preinjury activity level of this study, the threshold defined in our analysis is a higher criterion. The SCB of the first anchor question calculated using the net score change could be compared with previously reported MCID values of patients who underwent ACLR. Nwachukwu et al²³ reported the MCID values 2 years postoperatively for the IKDC-SKF and Lysholm as 9.0 and 10.0, respectively, which were relatively smaller than our SCB values, despite their longer follow-up (IKDC-SKF, 32.3; Lysholm, 24.0). However, the results of the ROC analyses of the net postoperative change in the Lysholm score were not statistically significant, and the AUC values were unacceptable. Thus, results must be interpreted cautiously. Certainly, the SCB values obtained by all 4 PROs 12 months postoperatively had acceptable reliability, which is meaningful for explaining the postoperative prognosis for patients with ACL injury.

For each anchor question, the proportion of patients who achieved SCBs differed depending on the PRO type, but it was less than half when the reliable cutoff values were implemented. Nevertheless, it should be noted that the intensity of daily activity and preinjury knee function differ between the elite athletes and the general population. In the general population, Minzlaff et al¹⁹ reported the average time of returning to work after ACLR as 7 weeks in patients who worked on a regular basis. Additionally, in previous studies of the general population who underwent ACLR, the average IKDC-SKF, Lysholm, and Tegner scores at 12 months postoperatively were 87.0,¹⁷ 87.1, and 5,¹⁹ respectively. Compared with the SCB values obtained in this study, these mean Lysholm and Tegner scores did not reach the SCB values of the first anchor question. Our study population consisted of a general population participating in recreational activities who might have had less motivation and time for intensive rehabilitation than professional athletes have. This might have affected postoperative PRO scores and functional performance.¹⁸ Our results are clinically significant because they provide a reference for predicting the postoperative prognosis in the general population with ACL injury and interpreting the results of other studies in future.

Unlike for the other 3 PROs, we found no significant difference in the mean ACL-RSI score at 6 and 12 months postoperatively. Sadeqi et al³⁰ reported that the mean ACL-RSI score increased by 6.3 points between 6 months and 1 year after ACLR, which was 3.4 points higher than our finding. When interpreting our results, it should be noted that the tendency is opposite, depending on whether the SCB was attained at 12 months postoperatively. The ACL-RSI score of the patients who attained the SCB for both anchor questions was significantly improved at 12 months postoperatively compared with 6 months postoperatively, while it decreased in the patients who did not attain the SCB. This suggests that the positive psychological responses correlate with the attainment of SCB at 1 year postoperatively. Another noteworthy point is that only 21.1% (n = 16) of our patients returned to the same activity level after ACLR. Previous studies have demonstrated a strong correlation between returning to the same preinjury level of sports and ACL-RSI scores.^2,16,30 Taken together, our results provide supporting evidence that psychological readiness is important for the general population with ACL injury to attain the SCB after ACLR and return to their preinjury level of sports.

We used 2 anchor questions with different clinical significance. The first question considered the discomfort experienced in daily life over the previous week. Among the 5 answers, we defined “none” as the cutoff for clinical success. For determining whether SCB has been obtained, this definition can be considered similar to the cutoff point used in other studies to obtain SCB, such as “much improved” or “much better.”^7,22,24,25 Conversely, the second anchor question focused on subjective knee function, compared with preinjury status, and scored it from 1 to 10. We defined substantial recovery as a score ≥8, which is in the top 30%. In 1 study of patients who underwent shoulder rotator cuff repair surgery, a question about shoulder function was used as an anchor question.¹⁰ The answer to the question comprised up to 15 steps, and the top 3 answers (reflecting the top 20%) were considered to indicate substantial improvement. In another study of patients with shoulder biceps tenodesis, a question about postoperative pain improvement was used as an anchor question to calculate the SCB.²⁷ The degree of improvement was scored from –1 to 7, and scores from 2 to 4 were classified as indicating significant improvement. Hence, our use of the top 30% as the cutoff point is similar to the cutoff points used in these previous studies, but direct comparison is inappropriate because the content and answers to the question differed across the studies. Since the SCB values and reliability of analyses are affected by how the cutoff point is determined, future research should be conducted with reference to our results.

In the multivariate analysis results of 12-month postoperative PROs, we identified younger age and better pre- and postoperative isokinetic tests as independent factors influencing SCB attainment. Magnitskaya et al¹⁷ reported a positive correlation of age at surgery with the IKDC-SKF and KOOS scores within the first year after ACLR, consistent with our results. In terms of pre- and postoperative isokinetic tests, the Lysholm, IKDC-SKF, and Tegner scores of the first anchor question correlated positively with the LSI of the quadriceps muscle 6 months postoperatively. The importance of quadriceps muscle activation in patients who underwent ACLR has been demonstrated, and the predictive value of the quadriceps muscle LSI for achieving better Lysholm and IKDC-SKF scores has been reported previously.^26,31 Additionally, the Tegner score of both anchor questions and the ACL-RSI score of the second anchor question correlated positively with the hamstring muscle LSI. Previous studies have emphasized the importance of hamstring muscle function recovery in patients who underwent ACLR, as it is associated with the occurrence of knee osteoarthritis and reinjury upon returning to sports.⁵ Our results emphasize the role of quadriceps and hamstring muscle strength in achieving better subjective outcomes in patients undergoing ACLR.

Limitations

Our study has several limitations. First, the sample size was small. In the ROC power analyses for assessing the reliability of ROC analyses, excellent power values (>0.99) were calculated for most 12-month postoperative PROs. However, the power calculations of net score changes were mostly in the unacceptable range (<0.80), except for the IKDC-SKF score regarding the second anchor question. That means the sample size was insufficient to say that the ROC analyses performed to find out the SCB values of net score change of PROs were reliable. Second, the follow-up period was 1 year, which may be a short period for analyzing correlations. Since the time period for evaluating the function after ACLR and deciding to RTS is 12 months postoperatively, our results provide clinically meaningful information in this respect.^2,21

A third limitation is that our results can be affected by the cutoff value of the anchor questions for SCB. Although previous studies have considered significant clinical improvement as reaching 66% to 80% of the preinjury condition, it is not clear if the same criterion can be applied to patients who underwent ACLR.^10,27 Fourth, an allograft in 8 patients (10.5%) could have affected the results. Previous studies have described that using autografts in ACLR achieves better clinical outcomes than using irradiated allografts.^8,35 Further studies that unify the graft types should be conducted to support our findings. Finally, we excluded the patients who had experienced subsequent injuries or infection after ACLR. If so, the results of our study could be more widely applied to patients who underwent ACLR regardless of the postoperative situation. However, in this study, we wanted to find to what extent the score indicates clinical success when examining the PROs in the patients who underwent primary ACLR who had not experienced the above postoperative situations.

Conclusion

The postoperative Lysholm, IKDC-SKF, and ACL-RSI scores showed acceptable to excellent reliability in predicting the SCB after ACLR in the general population. The age at the time of surgery and the thigh muscle strength were significant factors influencing these outcomes. The SCBs can be used as indicators to evaluate the treatment effect of clinical trials and predict clinical outcomes in patients with ACL injury.

Footnotes

Final revision submitted December 22, 2021; accepted February 8, 2022.

The authors declared that they have no conflicts of interest in the authorship and publication of this contribution. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.

Ethical approval for this study was obtained from Korea University Guro Hospital (reference No. 2016GR0161).

References

Ardern

Taylor

Feller

Webster

. Fear of re-injury in people who have returned to sport following anterior cruciate ligament reconstruction surgery. J Sci Med Sport. 2012;15(6):488–495.

Ardern

Taylor

Feller

Whitehead

Webster

. Psychological responses matter in returning to preinjury level of sport after anterior cruciate ligament reconstruction surgery. Am J Sports Med. 2013;41(7):1549–1558.

Ariel de Lima

de Lima

de Souza

NGR

, et al. Clinical outcomes of combined anterior cruciate ligament and anterolateral ligament reconstruction: a systematic review and meta-analysis. Knee Surg Relat Res. 2021;33(1):33.

Briggs

Lysholm

Tegner

, et al. The reliability, validity, and responsiveness of the Lysholm score and Tegner activity scale for anterior cruciate ligament injuries of the knee: 25 years later. Am J Sports Med. 2009;37(5):890–897.

Buckthorpe

Danelon

La Rosa

, et al. Recommendations for hamstring function recovery after ACL reconstruction. Sports Med. 2021;51(4):607–624.

Collins

Prinsen

Christensen

, et al. Knee Injury and Osteoarthritis Outcome Score (KOOS): systematic review and meta-analysis of measurement properties. Osteoarthritis Cartilage. 2016;24(8):1317–1329.

Glassman

Copay

Berven

, et al. Defining substantial clinical benefit following lumbar spine arthrodesis. J Bone Joint Surg Am. 2008;90(9):1839–1847.

Gupta

Singh

Kapoor

, et al. Graft tunnel integration occurs early in the tibial tunnel compared with the femoral tunnel after anterior cruciate ligament reconstruction with preserved insertion hamstring tendon graft. Knee Surg Relat Res. 2021;33(1):37.

Hamrin Senorski

Svantesson

Beischer

, et al. Factors affecting the achievement of a patient-acceptable symptom state 1 year after anterior cruciate ligament reconstruction: a cohort study of 343 patients from 2 registries. Orthop J Sports Med. 2018;6(4):2325967118764317.

10.

Haunschild

Gilat

, et al. Establishing the minimal clinically important difference, patient acceptable symptomatic state, and substantial clinical benefit of the PROMIS Upper Extremity Questionnaire after rotator cuff repair. Am J Sports Med. 2020;48(14):3439–3446.

11.

Ingelsrud

Terwee

Terluin

, et al. Meaningful change scores in the Knee injury and Osteoarthritis Outcome Score in patients undergoing anterior cruciate ligament reconstruction. Am J Sports Med. 2018;46(5):1120–1128.

12.

Irrgang

Harner

. Use of the International Knee Documentation Committee guidelines to assess outcome following anterior cruciate ligament reconstruction. Knee Surg Sports Traumatol Arthrosc. 1998;6(2):107–114.

13.

Karanikas

Arampatzis

Bruggemann

. Motor task and muscle strength followed different adaptation patterns after anterior cruciate ligament reconstruction. Eur J Phys Rehabil Med. 2009;45(1):37–45.

14.

Katz

Paillard

Ekman

. Determining the clinical importance of treatment benefits for interventions for painful orthopedic conditions. J Orthop Surg Res. 2015;10:24.

15.

Lai

CCH

Ardern

Feller

Webster

. Eighty-three per cent of elite athletes return to preinjury sport after anterior cruciate ligament reconstruction: a systematic review with meta-analysis of return to sport rates, graft rupture rates and performance outcomes. Br J Sports Med. 2018;52(2):128–138.

16.

Langford

Webster

Feller

. A prospective longitudinal study to assess psychological changes following anterior cruciate ligament reconstruction surgery. Br J Sports Med. 2009;43(5):377–381.

17.

Magnitskaya

Mouton

Gokeler

, et al. Younger age and hamstring tendon graft are associated with higher IKDC 2000 and KOOS scores during the first year after ACL reconstruction. Knee Surg Sports Traumatol Arthrosc. 2020;28(3):823–832.

18.

McPherson

Feller

Hewett

Webster

. Psychological readiness to return to sport is associated with second anterior cruciate ligament injuries. Am J Sports Med. 2019;47(4):857–862.

19.

Minzlaff

Heidt

Feucht

, et al. Patient satisfaction with health is substantially improved following ACL reconstruction. Knee Surg Sports Traumatol Arthrosc. 2018;26(2):582–588.

20.

Muller

Yabroudi

Lynch

, et al. Defining thresholds for the patient acceptable symptom state for the IKDC Subjective Knee Form and KOOS for patients who underwent ACL reconstruction. Am J Sports Med. 2016;44(11):2820–2826.

21.

Muller

Kruger-Franke

Schmidt

Rosemeyer

. Predictive parameters for return to pre-injury level of sport 6 months following anterior cruciate ligament reconstruction surgery. Knee Surg Sports Traumatol Arthrosc. 2015;23(12):3623–3631.

22.

Nwachukwu

Chang

Fields

, et al. Defining the “substantial clinical benefit” after arthroscopic treatment of femoroacetabular impingement. Am J Sports Med. 2017;45(6):1297–1303.

23.

Nwachukwu

Chang

Voleti

, et al. Preoperative Short Form Health Survey score is predictive of return to play and minimal clinically important difference at a minimum 2-year follow-up after anterior cruciate ligament reconstruction. Am J Sports Med. 2017;45(12):2784–2790.

24.

Ogura

Ackermann

Barbieri Mestriner

Merkely

Gomoll

. Minimal clinically important differences and substantial clinical benefit in patient-reported outcome measures after autologous chondrocyte implantation. Cartilage. 2020;11(4):412–422.

25.

Ogura

Ackermann

Mestriner

Merkely

Gomoll

. The minimal clinically important difference and substantial clinical benefit in the patient-reported outcome measures of patients undergoing osteochondral allograft transplantation in the knee. Cartilage. 2021;12(1):42–50.

26.

Pua

Chan

Khoo

Chong

. Associations of isokinetic and isotonic knee strength with knee function and activity level after anterior cruciate ligament reconstruction: a prospective cohort study. Knee. 2017;24(5):1067–1074.

27.

Puzzitiello

Gowd

Liu

, et al. Establishing minimal clinically important difference, substantial clinical benefit, and patient acceptable symptomatic state after biceps tenodesis. J Shoulder Elbow Surg. 2019;28(4):639–647.

28.

Roos

Lohmander

Ekdahl

Beynnon

. Knee injury and Osteoarthritis Outcome Score (KOOS)—development of a self-administered outcome measure. J Orthop Sports Phys Ther. 1998;28(2):88–96.

29.

Ryu

Shon

. Diagnostic tools for acute anterior cruciate ligament injury: GNRB, Lachman test, and Telos. Knee Surg Relat Res. 2018;30(2):121–127.

30.

Sadeqi

Klouche

Bohu

, et al. Progression of the psychological ACL-RSI score and return to sport after anterior cruciate ligament reconstruction: a prospective 2-year follow-up study from the French Prospective Anterior Cruciate Ligament Reconstruction Cohort Study (FAST). Orthop J Sports Med. 2018;6(12):2325967118812819.

31.

Sherman

Birchmeier

Kuenze

, et al. Thigh-muscle and patient-reported function early after anterior cruciate ligament reconstruction: clinical cutoffs unique to graft type and age. J Athl Train. 2020;55(8):826–833.

32.

Undheim

Cosgrave

King

, et al. Isokinetic muscle strength and readiness to return to sport following anterior cruciate ligament reconstruction: is there an association? A systematic review and a protocol recommendation. Br J Sports Med. 2015;49(20):1305–1310.

33.

Webster

Feller

Lambros

. Development and preliminary validation of a scale to measure the psychological impact of returning to sport following anterior cruciate ligament reconstruction surgery. Phys Ther Sport. 2008;9(1):9–15.

34.

Williams

Burley

Evans

, et al. The structural validity of the IKDC and its relationship with quality of life following ACL reconstruction. Scand J Med Sci Sports. 2020;30(9):1748–1757.

35.

Zeng

Gao

, et al. Autograft versus allograft in anterior cruciate ligament reconstruction: a meta-analysis of randomized controlled trials and systematic review of overlapping systematic reviews. Arthroscopy. 2016;32(1):153–163.e118.