Performance of PROMIS Computer Adaptive Testing As Compared With Established Instruments for Multiple-Ligament Knee Injuries

Abstract

Background:

The Patient-Reported Outcomes Measurement Information System (PROMIS) computer adaptive testing (CAT) is a patient-reported outcome metric that has been validated for anterior cruciate ligament (ACL) injuries, ACL reconstructions, and meniscal injuries. Thus far, the system has not been validated for multiligament knee injuries. The Multiligament Quality of Life (MLQOL) questionnaire is a validated, disease-specific patient-reported outcome instrument for this population that can serve as a gold standard for validation of the newer PROMIS.

Purpose/Hypothesis:

The purpose of this study was to further validate the PROMIS CAT for multiligament knee reconstruction. We hypothesized that the PROMIS CAT modules would correlate with the Lysholm knee score (Lysholm), Tegner activity scale (Tegner), and MLQOL for postoperative multiligamentous knee injury patients and that the PROMIS CAT would use fewer question items than the Lysholm, Tegner, and MLQOL while still avoiding floor and ceiling effects.

Study Design:

Cohort study; Level of evidence, 2.

Methods:

A total of 30 patients meeting the inclusion criteria were retrospectively identified and agreed to participate (87% male; mean ± SD age, 36.3 ± 13.5 years). Consenting patients were administered the Lysholm, Tegner, MLQOL, and PROMIS CAT for Physical Function, Mobility, and Pain Interference. Spearman correlations were used to test agreement across survey instruments. Floor and ceiling effects were assessed for all instruments.

Results:

PROMIS Pain Interference had excellent correlation with MLQOL Activity Limitations (r = 0.71, P < .0001). Excellent-good correlations were detected between PROMIS Physical Function and MLQOL Activity Limitations (r = –0.63, P = .0002) and PROMIS Mobility and MLQOL Activity Limitations (r = –0.62, P < .0002). Good correlations were found between the Lysholm and the PROMIS Physical Function and PROMIS Mobility (for each, r = 0.50, P = .005). Additional correlations were present with other subsections. There were no floor or ceiling effects for the PROMIS CAT instrument in any category.

Conclusion:

The PROMIS CAT correlates well with existing outcome measures for multiligament knee injury patients without floor or ceiling effects. The PROMIS CAT is a concise adjunct to the validated injury-specific outcome tool for multiligament knee injury. Surgeons should consider implementing the PROMIS CAT because of its broad validity, including ACL injuries, meniscal tears, and now multiligament knee injuries.

Knee dislocations with resultant multiple-ligament knee injuries (MLKIs) are an uncommon but devastating spectrum of injuries. The extent of knee damage can vary from lower-energy bicruciate injuries occurring during sports to higher-energy open fracture dislocations with nerve and arterial damage from motor vehicle collisions. The variable injury patterns and infrequency of these injuries make them inherently difficult to study in prospective well-controlled trials. ⁸ Consequently, there is a paucity of outcomes research relative to more common injuries, such as anterior cruciate ligament (ACL) tears. In addition, there was previously a lack of dedicated patient-recorded outcome instruments to study the effects of these injuries and their treatment on quality of life.

In 2014, the Multiligament Quality of Life (MLQOL) instrument was introduced and validated for MLKIs. ⁴ The validation of this instrument demonstrated correlations with the Tegner activity scale (Tegner) and 36-Item Short Form Health Survey form but with the goal of being more responsive to unique elements of MLKIs, such as neurologic and vascular injuries. ^13,15 This instrument has 4 domains with a total of 52 questions, making it detailed but lengthy. It has subsequently been used in MLKI outcomes research, demonstrating more responsiveness to outcome differences than the Lysholm knee score (Lysholm). ⁷ Recently, there has been an emphasis on validating a more universal instrument, the Patient-Reported Outcomes Measurement Information System (PROMIS) computer adaptive testing (CAT), for use in knee injuries. ^3,6,9,10,12 This instrument has been shown to correlate well with existing measures for ACL injuries as well as for meniscal injuries. However, to our knowledge, it has not been tested in the context of MLKIs. This instrument has the potential to be universally applicable while staying responsive to the spectrum of disability associated with MLKIs. A universally applicable instrument gives researchers and clinicians the ability to obtain a prediagnosis baseline for patient-reported outcomes while leaving open the option for supplemental disease-specific instruments to be used during the treatment course.

The purpose of this study was to assess the correlation of the PROMIS score with the MLQOL instrument, ⁴ Tegner, ¹³ and Lysholm. ¹³ We hypothesized that the PROMIS CAT Physical Function and Pain Interference domains would have a strong correlation with these existing measures while consisting of fewer questions.

Methods

We conducted a retrospective review of MLKI surgical cases performed at our institution between 2010 and 2017. Patients were included in the study if they sustained an MLKI, had a minimum 1-year follow-up, were treated surgically with at least 1 ligament reconstruction, had valid contact information in the medical record, were older than 18 years at the time of contact, and were willing to participate in the study. Patients were excluded if they had nonoperative treatment, were lost to follow-up, could not be contacted, were not English speaking, or were not willing to consent to inclusion in the study. This study was approved by the University of Southern California Institutional Review Board.

A total 89 potential study patients met initial inclusion criteria. Of these, 30 had valid contact information on file, spoke English as a primary language, and were willing to participate in the extensive phone surveys required for data collection (87% male; mean ± SD age, 36.3 ± 13.5 years; range, 17-69 years; body mass index, 30.1 ± 8.4). Patients were contacted via telephone by 1 of 2 authors (S.K., A.E.S.). After a scripted study description was given and informed consent was obtained, 4 outcome measures were administered—the 8-item Lysholm, the 33-item Tegner, the 52-item MLQOL, and the variable-length PROMIS CAT Pain Interference, Physical Function, and Mobility instruments. The number of questions answered for each CAT survey was recorded. Demographic information, including sex, age, and body mass index, were collected by chart review. Injury specifics, Schenck classification, and treatment method were also recorded.

Statistics were performed with Stata (v 13.0; StataCorp). Spearman correlations were calculated among the following: PROMIS Pain Interference, Physical Function, and Mobility; MLQOL Physical Impairments, Emotional Impairments, Activity Limitations, and Societal Involvement; and the Lysholm and Tegner scores. Correlations were classified per Anthony et al ¹ as poor (<0.4), good (0.4 ≤ r ≤ 0.6), excellent-good (0.6 < r ≤ 0.7), or excellent (r > 0.7). A P value of .05 was used for statistical significance. Floor and ceiling effects were determined to be present if >15% of responders reached the minimum or maximum possible score, respectively. ¹⁴ A post hoc power analysis was performed with a 1-sided Fisher Z test for each correlation.

Results

For the 30 study participants, the mean time from surgery to survey was 2.93 years (range, 1.04-9.72 years). Injury patterns and surgical treatments are detailed in Table 1.

TABLE 1

Demographic, Injury, and Treatment Information for Survey Participants ^a

	Demographics				Injury Information
No.	Follow-up, y	Age at Surgery, y	BMI	Sex	Schenck Type 11	Surgical Treatments
1	2.14	38	43	Male	KDI	ACL-R
2	1.40	36	25	Male	KDI	LCL-R, ALL-R, PCL-R
3	1.42	34	21	Female	KDI	ACL-R, MCL-R
4	2.19	29	26	Male	KDIII-L	PCL-R, ACL-R, LCL-R, ALL-R, POP-R
5	2.34	28	31	Male	KDIII-L	PCL-R, ALL-R
6	2.51	50	25	Female	KDIII-L	ACL-R, PCL-R, PLC-R
7	2.71	60	29	Male	KDIII-M	ACL-R, MCL-R, PMC-R
8	3.62	45	27	Male	KDI	PCL-R
9	4.51	23	27	Male	KDIII-L	PCL-R, ACL-R, LCL-R, POP-R
10	4.76	55	52	Male	KDIII-L	ACL-R, PCL-R, LCL-R, POP-R
11	5.02	27	31	Male	KDII	PCL-R, ACL-R
12	2.87	48	27	Male	KDI	ACL-R, LCL-R, POP-R
13	1.40	18	27	Male	KDIV	PCL-R
14	1.67	21	22	Male	KDIII-L	ACL-R, PLC-R, ALL-R
15	1.90	26	33	Male	KDI	PCL-R, MCL-R
16	2.07	29	32	Male	KDI	PCL-R, MCL-R
17	2.09	20	37	Male	KDIII-L	PCL-R, ACL-R, LCL-R, POP-R, ALL-R
18	2.09	49	30	Male	KDIII-M	ACL-R, PCL-R, MCL-R
19	1.04	54	27	Male	KDV	ACL-R
20	2.72	33	25	Male	KDIV	ACL-R, LCL-R, POP-R, MCL-R, ALL-R
21	2.78	17	31	Male	KDIII-L	PCL-R, ACL-R, LCL-R, POP-R
22	2.52	19	22	Male	KDI	ACL-R, LCL-R, POP-R + PLC-R, LCL-R, ALL-R
23	2.32	37	38	Female	KDIV	PCL-R, ACL-R, LCL-R, POP-R, ALL-R
24	8.53	30	NA	Male	KDIII-M	PCL-R, ACL-R, MCL-R, PMC-R
25	3.08	39	29	Male	KDI	LCL-R, PLC-R
26	2.65	45	33	Female	KDII	ACL-R, ALL-R
27	9.72	40	30	Male	KDI	ACL-R, MCL-R, LCL-R
28	1.73	26	31	Male	KDIV	ACL-R, LCL-R, ALL-R, MCL-R
29	3.10	69	35	Male	KDI	LCL-R, POP-R
30	1.13	48	29	Male	KDIII-L	ACL-R, LCL-R, ALL-R,

^a There were 11 KDI injuries, 2 KDII injuries, 12 KDIII injuries (3 medial, 9 lateral), 4 KDIV injuries, and 1 KDV injury (KD, knee dislocation; L, lateral; M, medial). ACL-R, anterior cruciate ligament reconstruction; ALL-R, anterolateral ligament reconstruction; BMI, body mass index; LCL-R, lateral (fibular) collateral ligament reconstruction; MCL-R, medial collateral ligament reconstruction; NA, not available; PCL-R, posterior cruciate ligament reconstruction; PLC-R, complete posterolateral corner reconstruction; PMC-R, posteromedial reconstruction; POP-R, popliteal reconstruction.

All correlations are illustrated in Table 2. An excellent correlation was detected between the PROMIS Pain Interference and MLQOL Activity Limitations (r = 0.71, P < .0001). Excellent-good correlations were detected between PROMIS Physical Function and MLQOL Activity Limitations (r = –0.63, P = .0002) and PROMIS Mobility and MLQOL Activity Limitations (r = –0.62, P < .0002). Good correlations were found between PROMIS Pain Interference and MLQOL Societal Involvement (r = 0.55, P = .002), PROMIS Physical Function and Lysholm (r = 0.50, P = .005), and PROMIS Mobility and Lysholm (r = 0.50, P = .005). Good correlations were also detected between PROMIS Pain and MLQOL Emotional Impairments (r = 0.61, P = .004), PROMIS Pain Interference and MLQOL Physical Impairments (r = 0.47, P = .008), PROMIS Physical Function and MLQOL Physical Impairments (r = –0.48, P = .007), and PROMIS Mobility and MLQOL Physical Impairments (r = –0.45, P = .014). The Tegner score showed good correlation with PROMIS Physical Function (r = 0.49, P = .006) and PROMIS Mobility (r = 0.42, P = .021). All measured correlations were statistically significant (P < .05). There were no floor or ceiling effects present in any of our survey items with this population group (ie, <15% of responders hit the maximum or minimum score for each section). Post hoc power analysis examining each comparison demonstrated at least 80% power for all correlations except for the Tegner score versus PROMIS Mobility, for which the power was 75.2%.

TABLE 2

Correlations Between PROMIS Measures and Established Instruments ^a

PROMIS-CAT: Comparator	Q	Spearman Rho	P
Pain Interference	5.8
MLQOL EI	15	0.61	.0004
MLQOL SI	6	0.55	.0018
MLQOL PI	19	0.47	.0083
MLQOL AL	12	0.71	<.0001
Physical Function	4.5
MLQOL PI	19	–0.48	.0067
MLQOL AL	12	–0.63	.0002
Lysholm	8	0.50	.0049
Tegner	11	0.49	.0063
Mobility	5.5
MLQOL PI	19	–0.45	.0136
MLQOL AL	12	–0.62	.0002
Lysholm	8	0.50	.0045
Tegner	11	0.42	.0208

^a The negative correlations in the Physical Function and Mobility sections highlight differences in directionality of the scoring system, with high scores being a good result for Physical Function and Mobility but a worse quality of life for MLQOL. Pain Interference scores were concordant with MLQOL because higher scores indicate more pain. AL, activity limitations; CAT, computer adaptive testing; EI, emotional impairments; MLQOL, Multiligament Quality of Life; PI, physical impairments; PROMIS, Patient-Reported Outcomes Measurement Information System; Q, number of question items in the survey instrument; SI, societal involvement.

PROMIS CAT categories offered fewer questions than the MLQOL domains and the Lysholm score. PROMIS Pain Interference had a mean 5.83 questions; PROMIS Physical Function, 4.53 questions; and PROMIS Mobility, 5.53 questions. All 3 PROMIS sections individually offered a range of 4 to 12 questions. For the MLQOL, the Emotional Impairments section was 15 questions; Societal Involvement, 6 questions; Physical Impairments, 19 questions; and Activity Limitations, 12 questions. The Lysholm score is 8 questions, and the Tegner score is 33 questions (11 items, with 3 time references each). Table 2 also compares the number of PROMIS items with their associated best correlates.

Discussion

Patient-reported outcome measures are becoming increasingly important in patient care and reimbursements. It is important to understand the validity of each test instrument for specific injury patterns. In this study, we demonstrated that in postoperative MLKI, the PROMIS CAT correlates well with an existing generalized knee instrument (Lysholm/Tegner) and with a disease-specific instrument (MLQOL). We also found no floor or ceiling effects in our population, indicating that the PROMIS CAT is able to characterize these patients appropriately despite their unique severity of injury. To our knowledge, this is the first study to demonstrate the utility of the PROMIS CAT for MLKI.

The PROMIS CAT was developed to be a universal instrument that was concise and patient friendly. ² It can be implemented through a tablet computer interface with easily understandable prompts, and it provides full data exports to the supervising physician. The computer adaptive portion is designed to make questions more efficient based on a patient’s initial responses. In contrast, existing measures are static. This means that some of the questions may be inapplicable or superfluous for a given patient.

The PROMIS CAT has been validated for a number of knee conditions. ^5,6,12 Hancock et al ^5,6 demonstrated the utility of the PROMIS CAT for ACL injuries, finding the mean number of questions to be 4.2, which is less than that of our study. This likely is a sequela of the increased severity of MLKI. In addition, that study found excellent correlations (Spearman r > 0.7) between PROMIS and the Knee injury and Osteoarthritis Outcome Score (KOOS), Marx knee activity rating scale, and the EuroQol 5-Dimensions questionnaire. The results of Hancock et al ⁵ for ACL injuries are corroborated by work from Scott and colleagues, ¹² which demonstrated similar correlations for postoperative ACL reconstruction. Similar validation studies have been performed for meniscal injuries as well.

In our study, we chose a different population and compared the PROMIS CAT with a different disease-specific instrument (MLQOL) as well as generalized instruments unique from the Hancock et al ⁶ study (Lysholm/Tegner). Our correlation values were similar (>0.7) for the comparison of PROMIS Pain Interference with the MLQOL Activity Limitations. However, many of our other correlations were slightly weaker. This could be the result of comparing a general instrument with a disease-specific instrument, or it could be secondary to our smaller sample size for this less common injury.

Our study does have limitations. First, as with all studies of MLKI, there is inherent variability in injury severity and injury patterns (see Table 1). Despite this variation, we were still able to achieve strong correlations with other outcome measures. Additionally, the rarity of these injuries makes it difficult to achieve a large sample size. With 30 patients, we were able to reach statistical significance of all measures, with adequate power to detect correlations between all measures, except the comparison between postoperative Tegner score and PROMIS Mobility outcome. The statistics in this study also limit the effect of injury variation. Comparisons between outcome scores were made for each patient. This limits the effect of different injury patterns as compared with studies of pooled data or population means. Another potential limitation is that the mean age of our patients was 36 years, as opposed to 26 years for the initial MLQOL validation study. ⁴ Our study was not sufficiently powered to detect differences in treatment outcomes or injury severity, as the effect sizes were small at late follow-up and the surgical treatments were heterogeneous. In future studies of the PROMIS and MLQOL for MLKIs, larger sample sizes may be needed to determine which outcome measures are more sensitive to differences in injury severity or treatment outcomes.

Conclusion

The PROMIS CAT instrument correlates with a validated injury-specific outcome tool (MLQOL) for patients with MLKIs at a cross-sectional time point postoperatively. The generalized PROMIS instrument correlated with MLQOL across all categories and was free of any floor or ceiling effects in this population. The PROMIS CAT can be used as a prediagnosis tool and can supplement the MLQOL for postdiagnosis monitoring of patient-reported outcomes in patients with MLKIs. Clinicians should consider using the PROMIS tool as an adjunct to the MLQOL instrument in patient-reported outcome studies of MLKIs.