Optimal cutoff score for patient-reported outcome measures after anterior cruciate ligament reconstruction using load–displacement curve analysis

Abstract

To assess psychological components after anterior cruciate ligament (ACL) injury, we developed a new patient-reported outcome measure, the Japanese anterior cruciate ligament injury questionnaire 25 (JACL-25), and demonstrated that the JACL-25 is valid, reliable, and sufficiently responsive to evaluate psychological factors that are associated with outcomes in individuals with ACL injuries. The purpose of this study was to identify the optimum cutoff score for the JACL-25 that indicates successful clinical outcomes, combining with the graphical data obtained using the KT-2000. We studied 30 patients who underwent primary ACL reconstruction with a patella tendon or hamstring tendon autograft. On the same day of the JACL-25 evaluation, anterior knee laxity tests were performed using the KT-2000 arthrometer. ACL stiffness was calculated on the graphs of the hysteresis, which can be used to measure side-to-side differences in anterior translation. Akaike’s information criterion (AIC) was used to determine the most appropriate cutoff level for the JACL-25 score for the stiffness reduction ratio and anterior–posterior (AP) translation. The average JACL-25 score was 23.8 (standard deviation (SD) 18.4). The average side-to-side displacement difference in AP translation was 2.4 (SD 4.2) mm. The average stiffness reduction ratio was 24.4% (SD 31.8). The smallest AIC values for the cutoff level for the JACL-25 score, the stiffness reduction ratio, and side-to-side differences of the anterior translation were 9, 27%, and 1.1 mm, respectively. We determined the optimal cutoff score for the JACL-25 that identifies successful clinical outcomes.

Introduction

An improved understanding of the anatomy and function of the anterior cruciate ligament (ACL) as well as technological evolution has made ACL reconstruction a successful surgery with good clinical results. ^{1 –3} Nevertheless, it is still a challenge to reconstruct a normal ACL. The persistence of instability, after surgery, likely contributes to anxiety and fear. Physical examinations, such as the Lachman test and the pivot shift test, the knee ligament arthrometers, magnetic resonance imaging, and the knee scoring system, are often used to assess outcomes objectively after ACL reconstruction. Yet, patients often complain of symptoms, such as anxiety and fear of recurrent injury, and these psychological effects may influence clinical results after the surgery. ^{4 –7} To assess psychological components after ACL injury, Nagao et al. developed a new patient-reported outcome measure, the Japanese anterior cruciate ligament injury questionnaire 25 (JACL-25; Table 1), which helps to subjectively measure fear caused by the instability. They further demonstrated that the JACL-25 is valid, reliable, and sufficiently responsive to evaluate psychological factors that are associated with outcomes in individuals with ACL injuries. ⁸ However, to use the JACL-25 effectively, it is necessary to know the cutoff score that indicates successful clinical outcomes after ACL reconstruction.

Table 1.

Contents of the JACL-25.^a

This questionnaire asks about the condition of your injured knee during the past month. For each question, please check □ the box that best describes your condition.

1. How much were you bothered by the injured leg when standing on a train or bus?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

2. Have you hesitated to walk briskly on a gravel or rough road?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

3. How much have you been afraid of walking on a slippery floor or road?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

4. How much have you been afraid of getting out of a car firstly with your injured leg?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

5. How much have you been afraid of jumping down from a dozen inches of difference in level such as the steps of a bus?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

6. How much did you have a lack of knee balance when performing a knee bend with the injured leg only?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

7. How much have you worried about running backward?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

8. How much have you been afraid of running down a slope?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

9. How much have you worried about running up the stairs two at a time?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

10. How much have you worried about running down the stairs?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

11. How much were you bothered by the injured leg at sprinting?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

12. How much have you been afraid of jumping with the injured leg for takeoff?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

13. How much have you been afraid of jumping and landing on the injured leg?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

14. How much have you been afraid of landing on both feet at one time?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

15. How much have you been afraid of performing a standing broad jump with the injured leg only?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

16. How much have you been afraid of a sudden change in direction (cutting) with the injured leg?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

17. How much have you been afraid of hopping on the injured leg?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

18. Did you protect the injured leg when stopping suddenly after sprinting?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

19. How much were you bothered by the injured leg in figure-of-eight running?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

20. How much have you worried about kicking a ball with the injured leg?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

21. How much have you worried about kicking a ball using the injured leg as a pivot?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

22. How much have you been afraid of someone bumping into your injured leg from the side?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

23. How much have you been afraid of holding his/her ground for a sideways jump with the injured leg?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

24. How much have you been afraid of taking a side step?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

25. How much have you hesitated to slide with the injured leg?

□ Not at all □ Slightly □ Moderately □ Considerably □ Extremely

Please check you have answered every question.

Thank you for your cooperation.

JACL-25: Japanese anterior cruciate ligament injury questionnaire 25.

The KT-2000 (MEDmetric^® Corporation, San Diego, California, USA) has been widely accepted to provide an objective measurement of ACL-related instability. ^{9 –11} Furthermore, the hysteresis curve that is generated using the KT-2000 indicates knee stability and stiffness. ⁹ Knee instability may influence the psychological response after ACL reconstruction. ⁷ In combination, the JACL-25 and the graphical data obtained using the KT-2000 can help to assess the clinical outcomes after ACL reconstruction more precisely. The purpose of this study was to identify the optimum cutoff score for the JACL-25 that indicates successful clinical outcomes.

Materials and methods

This study received the approval of the ethics committee of Juntendo Nerima Hospital (no. 12-26). Patients included in this study underwent primary ACL reconstruction with a patella tendon or hamstring tendon autograft. ^12,13 We studied 30 patients who had resumed sports after surgery, including 2, who experienced a reinjury. Study participants included 13 men and 17 women, aged 16–53 (mean 27.7) years, who had a mean follow-up period of 13 months. We excluded (1) patients with Kellgren–Lawrence grade 2 or higher, (2) patients with multiple ligament injuries, (3) or any patient who had had contralateral knee ligament injury.

The patients were evaluated using the JACL-25. The JACL-25, which was originally written in Japanese, was developed to measure psychological factors associated with outcomes after ACL injury. The items were scored on a 5-point scale, with 0 indicating no impairment and 4 indicating severe impairment. The scores were arithmetically added to produce a total score (minimum 0 and maximum 100). A higher score was associated with a worse condition. On the same day of the JACL-25 evaluation, anterior knee laxity tests were performed using the KT-2000 arthrometer at 134 N. The X–Y plotter connecting the KT-2000 arthrometer recorded the graphs of the hysteresis (Figure 1), which can be used to measure side-to-side differences in anterior translation. To calculate the ACL stiffness, in N/mm, which is defined as the ascending slope of the hysteresis curve, the information from the graphs was processed in the following way. The graphs were digitized using an image scanner (EPSON scan^® 300 dpi, Seiko Epson Corporation, Nagano, Japan) and then converted into numbers using the digitizing software (ImageToGraph 32^®, Tokyo, Japan). Savitzky-Golay filters (OriginPro^®, LightStone Corporation, Tokyo, Japan) were then used to account for noisy data and the numbers were differentiated. The mean differential values of the last 10 points obtained closest to 134 N in the numbers converted from the graphs were calculated as the ACL stiffness. The side-to-side differences in the stiffness after surgery, called the stiffness reduction ratio, were calculated as follows:

$\text{Stiffness reduction ratio}\hspace{0.17em}\left(\text{\%}\right)\hspace{0.17em}\text{=}\hspace{0.17em}\left(\text{uninjured stiffness}-\text{injured stiffness}\right)\times 100/\text{uninjured stiffness}$

Figure 1.

The hysteresis curve recorded by the X–Y plotter connecting the KT-2000 arthrometer. The ascending slope was used to calculate the ACL stiffness. ACL: anterior cruciate ligament.

Statistical analyses were performed using SPSS software version 21 (IBM Company, New York, USA). Akaike’s information criterion (AIC) ¹⁴ was used to determine the most appropriate cutoff level for the JACL-25 score for the stiffness reduction ratio and anterior–posterior (AP) translation. The AIC was proposed by Akaike as an information criterion for model selection, and it is defined as follows:

$\text{AIC}=-2\log L+2k$

where L is the maximum likelihood and k is the number of parameters in the model. The AIC statistic has been widely used to compare different models with the same data. A smaller AIC value indicated a more reliable model for predicting the outcome. Categorical variables were compared using either Pearson’s χ ² test or Fisher’s exact test. The p values <0.05 were considered to indicate statistical significance.

Results

The average JACL-25 score was 23.8 (standard deviation (SD) 18.4). The average side-to-side displacement difference in AP translation was 2.4 (SD 4.2) mm. The average stiffness was 153.9 N/mm (SD 57.8) in uninjured knees and 105.0 N/mm (SD, 32.5) in operated knees. The average stiffness reduction ratio was 24.4% (SD, 31.8).

The smallest AIC values for the cutoff level for the JACL-25 score, the stiffness reduction ratio, and side-to-side differences of the anterior translation were 9, 27%, and 1.1 mm, respectively (Figure 2). In addition, the AIC values were calculated for combinations of the JACL-25 score and the stiffness reduction ratio. The smallest AIC value indicated the most appropriate cutoff level, where the JACL-25 score was 9 and the stiffness reduction ratio was 27% (Table 2). Based on the results of the AIC statistics, the patients were divided into four groups using a JACL-25 score of 9 and a stiffness reduction ratio of 27% as cutoff values. There were significant differences among the groups, as indicated by the χ ² test (p = 0.003; Table 3). The same investigation was performed using a combination of the JACL-25 score and the AP translation difference. A JACL-25 score of 9 and a translation difference of 1.1 mm indicated the most appropriate cutoff level. There was a significant difference between the groups that were divided using the AIC statistics (p = 0.018).

Figure 2.

The smallest AIC values for the cutoff level for the JACL-25 score, the stiffness reduction ratio, and side-to-side differences of the anterior translation were 9, 27%, and 1.1 mm, respectively. (a) The JACL-25 score, (b) the stiffness reduction ratio, and (c) side-to-side differences of the anterior translation. AIC: Akaike’s information criterion; JACL-25: Japanese anterior cruciate ligament injury questionnaire 25.

Table 2.

The smallest 10 AIC values for cutoff score combinations of the JACL-25 and stiffness reduction ratio.

JACL-25 score	Stiffness reduction ratio (%)	AIC
9	27	−6.88
10	27	−6.88
9	28	−5.75
10	28	−5.75
9	19	−5.05
9	20	−5.05
9	21	−5.05
9	22	−5.05
9	23	−5.05
9	24	−5.05

AIC: Akaike’s information criterion; JACL-25: Japanese anterior cruciate ligament injury questionnaire 25.

Table 3.

Number of cases divided cutoff score.^a

		JACL-25
		<9	≥9
Stiffness reduction ratio (%)	≥27	0	15
	<27	7	8

JACL-25: Japanese anterior cruciate ligament injury questionnaire 25.

^a There was a significant difference among the groups in the χ ² test (p = 0.003).

Discussion

The JACL-25 was developed to evaluate psychological aspects after ACL injury. There are some subjective scales focusing on psychological factors, ^4,15,16 but no studies have identified a cutoff score for the evaluation scales using continuous variables after ACL reconstruction. In this study, the optimal cutoff score to identify successful clinical outcomes was calculated using the JACL-25 score and objective data were obtained from the KT-2000. To our knowledge, this is the first report to identify a cutoff score that indicates clinical result after ACL reconstruction using a patient-reported outcome measurement and objective measurements of ACL-related instability. In this study, the differential values of the last 10 points toward the top of the hysteresis curve were used to calculate knee stiffness to identify the cutoff value since they are important for high performance activities, such as cutting and pivoting. The current study demonstrated that good clinical results produce a JACL-25 score of ≤9, a stiffness reduction ratio of ≤27%, and a side-to-side difference in anterior translation of ≤1.1 mm. Interestingly, previous convention has been to consider a ≤3 mm difference in anterior translation as an indicator of good clinical results. ^17,18 Recently, outcomes of surgery have improved because of the development of new surgical techniques. ¹⁹ Our results imply that the cutoff value of 1.1 mm may divide excellent and good clinical outcomes. Furthermore, a stiffness reduction ratio of ≤27% and a JACL-25 score of ≤9 also suggest excellent clinical results.

The objective parameters examined in this study included the stiffness reduction ratio and the differences of the anterior translation. Although the differences in anterior translation have been used for clinical evaluations after ACL reconstruction, the stiffness, which is defined as the slope of the load–elongation curve, is commonly applied to the biomechanical measurement of the tensile and mechanical properties of ligaments. ^20,21 As the hysteresis curves obtained from the KT-2000 arthrometer were generated as analog data in our study, the method of calculating stiffness is complex. To calculate the stiffness more precisely, it is necessary to develop a ligament measurement device from which fine, digital data can be obtained.

In this study (Table 3), no cases exhibited poor stiffness (≥27 in stiffness reduction ratio) and a good JACL-25 score (<9). However, eight cases exhibited good stiffness (<27) and a poor JACL-25 score (≥9). Patients in these cases still indicated feelings of anxiety and fear after surgery, despite the knees being biomechanically stabilized. This psychological response was presumed to be the result of other factors, such as muscle power and awareness of the number of knees that give way before surgery.

There are some limitations to this study. The sample size was small and, therefore, we could not investigate the effects of various factors, such as age, gender, preoperative period after injury, muscle power, or sports level. Furthermore, the follow-up periods were short. The aim of this study was, however, not to investigate clinical results after the surgery but to study the method of evaluation. To investigate clinical outcomes after surgery, it is necessary to evaluate the long-term results of ACL stabilization and the use of the JACL-25 score.

Conclusion

We determined the optimal cutoff score for the JACL-25 that identifies successful clinical outcomes. In doing so, we demonstrated that a JACL-25 score of 9, a stiffness reduction ratio of 27%, and a translation difference of 1.1 mm represent the most appropriate cutoff level.