Abstract
Background:
The Patient-Reported Outcomes Measurement Information System (PROMIS) has not been fully evaluated for outcomes assessment after hip arthroscopy to correct femoroacetabular impingement syndrome (FAIS).
Purpose/Hypothesis:
The purpose of this study was to compare the accuracy of the PROMIS Physical Function (PF) and Pain Interference (PI) subscales with the 12-Item International Hip Outcome Tool (iHOT-12) to define patients with 3 unique substantial clinical benefit (SCB) scores—patients who reported ≥80%, ≥90%, and 100% satisfaction at 1 year after hip arthroscopy for FAIS. We hypothesized that the iHOT-12 would be more accurate than the PROMIS-PF and PROMIS-PI subscales in identifying these 3 patient groups.
Study Design:
Cohort study (diagnosis); Level of evidence, 2.
Methods:
We reviewed the records of patients who underwent hip arthroscopy for symptomatic FAIS at 3 centers from January 2019 through June 2021 and had 1-year clinical and radiographic follow-up data. Patients completed the iHOT-12, PROMIS-PF, and PROMIS-PI on initial assessment and at 1 year (±30 days) postoperatively. Postoperative satisfaction was reported on an 11-category scale with anchors defined as “0% satisfied” and “100% satisfied.” Receiver operator characteristic analysis was performed to determine the absolute SCB values for the iHOT-12 and PROMIS subscales that would most accurately identify those patients who reported ≥80%, ≥90%, and 100% satisfaction. Area under the curve (AUC) values and 95% CIs for the 3 instruments were compared.
Results:
Included were 163 patients (111 [68%] women and 52 [32%] men), with a mean age of 26.1 years. Corresponding absolute SCB scores for patients who reported ≥80%, ≥90%, and 100% satisfaction were as follows: iHOT-12, 68.4, 72.1, 74.7; PROMIS-PF, 45, 47.7, 49.9; and PROMIS-PI, 55.9, 52.4, 51.9. The AUC ranged between 0.67 and 0.82, with overlapping 95% CIs indicating a minimal difference in accuracy between the 3 instruments. Sensitivity and specificity values ranged between 0.61 and 0.82.
Conclusion:
The PROMIS-PF and PROMIS-PI subscales were as accurate as the iHOT-12 in defining absolute SCB scores for patients reporting ≥80%, ≥90%, and 100% satisfaction at 1-year follow-up after hip arthroscopy for FAIS.
Patient outcome assessment continues to be an area of interest with over 15 different patient-reported outcome measures (PROMs) being used to assess hip arthroscopy outcomes for femoroacetabular impingement syndrome (FAIS). 34 These PROMs include well-established or legacy instruments, such as the International Hip Outcome Tool (iHOT; 12-item and 33-item versions), Hip Outcome Score, and modified Harris Hip Score. 34 Clinicians and researchers often use multiple PROMs simultaneously for various reasons, including large numbers of available PROMs, no clear consensus as to which instrument is best, and the desire to compare outcomes across multiple centers. This poses problems with patient compliance and efficiency in data collection. To help overcome the problem associated with large numbers of PROMs, the National Institutes of Health developed the Patient-Reported Outcomes Measurement Information System (PROMIS). The goal of the PROMIS is to provide a singular, standardized PROM that can be applied to a wide variety of conditions. 5
The PROMIS has potential advantages over legacy instruments. These include the use of item response theory and computerized adaptive testing, which reduces questionnaire fatigue by reducing the number of responses required to produce an accurate outcome score.2,5,13 The PROMIS can assess multiple health-related domains, including physical function (PF) and pain interference (PI). The PROMIS-PF subscale is commonly used to assess self-reported function in those with extremity conditions, while the PI subscale assesses the impact of pain on social, cognitive, emotional, physical, and recreational activities. 29 In patients undergoing hip arthroscopy, Browning et al 6 found that PROMIS scores could be obtained in <1 minute with only 4 to 5 questions. Also, since the definition of a standard normal score has been determined to be 50 with an SD of 10, the obtained scores can be interpreted according to these normative values and compared across a wide spectrum of orthopaedic and nonorthopaedic conditions.13,29 The potential disadvantage of the PROMIS is that it does not have hip-specific questions across the spectrum of ability, and therefore it may not be able to define patient status as accurately as legacy PROMs.
The PROMIS does have psychometric evidence to support its use for hip arthroscopy patients.3,4,10,19,20,24,27,32 An essential psychometric property in outcome assessment is the ability to define a patient's current status. This can be done using clinically important outcome values (CIOVs), such as minimal clinically important difference, Patient Acceptable Symptom State (PASS), substantial clinical benefit (SCB), and maximal outcome improvement.7-9 The benefit of CIOVs is that they allow patients, clinicians, and researchers to interpret a patient's single score or change in score. Therefore, CIOVs may be valuable when making individual treatment decisions. In addition to other hip-specific legacy instruments, including the iHOT-12, CIOVs have also been defined for the PROMIS-PF and PROMIS-PI subscales.4,20 However, the accuracy of these CIOVs for the PROMIS and legacy instruments in defining the patient's current status has not been directly compared.
When interpreting scores from multiple PROMs, determining which CIOVs are most accurate in identifying the patient's current status is of high importance. Absolute PASS values are typically determined by the question: “Are you satisfied?” with a “yes” or “no” response. Therefore, an absolute single PASS score only represents whether the patient is satisfied or not with one's outcome and does not give information about the level of satisfaction. Absolute SCB values reflect upper thresholds of status and may best represent various levels of satisfaction.
The purpose of the present study was to determine and directly compare the accuracy of the PROMIS-PF and PROMIS-PI subscales with the iHOT-12 in determining 3 unique SCB values for those reporting ≥80%, ≥90%, and 100% satisfaction at 1-year after hip arthroscopy for FAIS. We hypothesized that the iHOT-12 would be more accurate than the PROMIS-PF and PROMIS-PI subscales in identifying these 3 groups.
Methods
This was a retrospective review of a research registry containing prospectively collected data on consecutive patients undergoing hip arthroscopy from 3 independent orthopaedic surgeons (T.E., J.D.H., R.K.) at 3 centers from January 2019 through June 2021. Patients were excluded from the registry if they could not read or understand English. The inclusion criteria for this study were patients with 1-year follow-up clinical and radiographic examination data after hip arthroscopy for symptomatic FAIS. The exclusion criteria were as follows: hip arthroscopy for an indication other than FAIS; concomitant procedures, including periacetabular osteotomy, femoral derotational osteotomy, or extra-articular surgical procedures; signs of osteoarthritis (Tönnis grade >1); hip dysplasia (lateral center-edge angle ≤25°); a history of slipped capital femoral epiphysis or Legg-Calf-Perthes disease; avascular necrosis; Ehlers-Danlos; revision hip arthroscopy; and rheumatoid arthritis. Patients with incomplete data and bilateral surgical procedures or surgical procedures during the follow-up period were also excluded. All patients had standardized radiographs, computed tomography scans, and magnetic resonance imaging (MRI) scans with all imaging measures taken by the respective orthopaedic surgeon. MRI arthrograms and anesthetic and/or corticosteroid injections were performed at the physician's discretion. Additionally, all patients had unsuccessful relief of symptoms with at least 6 to 12 weeks of conservative treatments that consisted of physical therapy, nonsteroidal anti-inflammatory drugs, and activity modification, as indicated. Institutional review board approval was obtained for the study protocol, and all included patients provided written informed consent.
Data Collection
Patient age, body mass index (BMI), sex, radiographic measurements, and surgical procedures were recorded from the registry. The patients completed the iHOT-12, PROMIS-PF, and PROMIS-PI within 2 weeks before surgery. Postoperative iHOT-12, PROMIS-PF, and PROMIS-PI scores, as well as a postoperative satisfaction question, were completed via an online survey at the 1-year postoperative time point. The satisfaction question (“What is your overall satisfaction with your surgery?”) was scored using an 11-category scale, with the 2 anchors defined as “0% satisfied” and “100% satisfied.”
Surgical Procedures
All surgical procedures were performed by 1 of 3 high-volume, fellowship-trained hip arthroscopy surgeons (T.E., J.D.H., R.K.), with experience of over 1000 hip arthroscopy procedures per surgeon. Surgery was conducted utilizing the modified supine position with a high friction pad, allowing for post-free distraction, in all patients as described by Kollmorgen et al. 18 Briefly, the patients were placed on the hip distractor with Trendelenburg positioning used according to the orthopaedic surgeon's discretion. Air arthrogram was utilized before applying distraction. Standard anterolateral, midanterior, and distal anterolateral accessory portals were utilized as indicated. The labrum was repaired, reconstructed, or augmented depending on the quality of the tissue at the time of index operation. Bone morphology was corrected based on the preoperative radiographic and intraoperative evaluation. After surgery, all patients utilized crutches and a hip brace for 3 to 4 weeks. Physical therapy protocols were standardized between the 3 centers, and patients underwent formal therapy that lasted for a minimum of 12 weeks.
Statistical Analysis
Receiver operator characteristic (ROC) analysis was performed to define the sensitivity and specificity of absolute iHOT-12, PROMIS-PF, and PROMIS-PI SCB scores that would most accurately identify those that reported ≥80%, ≥90%, and 100% satisfaction at 1-year follow-up from those not reporting ≥80%, ≥90%, and 100% satisfaction. The accuracy of the PROMIS-PF, PROMIS-PI, and iHOT-12 in distinguishing between these groups was determined by calculating the area under the curve (AUC) at a 95% CI.1,21 An AUC of >0.7 and a 95% CI that does not contain 0.5 are considered acceptable levels of responsiveness.11,21,28 The AUCs would be considered significantly different if there were no overlaps in the 95% CIs. The Youden index was used to optimize sensitivity and specificity values to identify PROMIS and iHOT-12 scores that are likely to identify those reporting ≥80%, ≥90%, and 100% satisfaction at 1-year follow-up. 30 Descriptive statistics were calculated for age, sex, radiographic measurements, and BMI. One-way analysis of variance was performed to assess for a difference in pre- and 1-year postoperative PROMIS-PF, PROMIS-PI, and iHOT0-12 scores. Statistical analysis was performed using the SPSS software package (Version 26; SPSS Inc).
Results
There were 826 patients identified with 1-year postprimary hip arthroscopy for FAIS and Tönnis grade 0/1, with 163 (20%) patients meeting the inclusion criteria. The mean follow-up time for the 163 patients was 12 months (±30 days). A patient flow diagram is presented in Figure 1. Demographic information regarding age, sex, BMI, radiographic measurements, and surgical procedures is presented in Table 1. All patients had >1 procedure performed during hip arthroscopy, with femoral osteochondroplasty, capsular repair, and labral repair being the most common combination.
Patient flow diagram. FAIS, femoroacetabular impingement syndrome; FDO, femoral derotational osteotomy; PAO, periacetabular osteotomy.
Patient and Surgery Characteristics a
BMI, body mass index; LCEA, lateral center-edge angle.
Missing data for 1 patient (<1%).
Psychometric Results
The mean preoperative and 1-year postoperative PROMIS-PF, PROMIS-PI, and iHOT-12 scores of patients, as well as patients reporting <80%, 80%, ≥90%, and 100% satisfaction at 1-year follow-up, are presented in Table 2. The associated AUCs with 95% CIs, sensitivity values, and specificity values for those reporting ≥80%, ≥90%, and 100% satisfaction with their surgery are presented in Table 3, and results of the ROC analyses are presented in Figure 2. All AUC values ranged between 0.67 and 0.82. Absolute SCB scores for the iHOT-12, PROMIS-PF, and PROMIS-PI scores were accurate in identifying those ≥80%, ≥90%, and 100% satisfaction at 1-year follow-up, as the AUCs were >0.70, with 95% CIs not containing 0.5. The only exception to this was the PROMIS-PI in identifying those 100% satisfied, with an AUC value of 0.67. There was minimal difference in accuracy between the 3 instruments, as all AUC 95% CIs considerably overlapped.
Preop and Postop iHOT-12, PROMIS-PF, PROMIS-PI, and Postop Satisfaction a
Data are presented as mean ± SD or n (%). Dashes indicate areas not applicable. iHOT, International Hip Outcome Tool; PF, Physical Function; PI, Pain Interference; Postop, postoperative; Preop, preoperative; PROMIS, Patient-Reported Outcomes Measurement Information System.
SCB Values for Patients ≥80%, ≥90%, and 100% Satisfied at 1-Year Follow-up a
AUC, area under the receiver operating curve; iHOT, International Hip Outcome Tool; PF, Physical Function; PI, Pain Interference; PROMIS, Patient-Reported Outcomes Measurement Information System; SCB, substantial clinical benefit; SN, sensitivity; SP, specificity.
ROCs for identifying patients with different degrees of satisfaction at 1-year follow-up after hip arthroscopy for FAIS: (A) ≥80% satisfied; (B) ≥90% satisfied; (C) 100% satisfied. iHOT, International Hip Outcome Tool; FAIS, femoroacetabular impingement syndrome; PF, Physical Function; PI, Pain Interference; PROMIS, Patient-Reported Outcomes Measurement Information System; ROC, receiver operating curve.
Discussion
The most important finding of this study was that the PROMIS-PF and PROMIS-PI subscales were as accurate as the iHOT-12 in identifying those reporting
Clinicians and researchers often use multiple PROMs simultaneously because there is no standardization in outcome reporting. This causes inefficient data collection, with an increasing burden on patients, clinicians, and researchers, potentially leading to noncompliance, fatigue, and incomplete information. The PROMIS has gained popularity in an effort to streamline data collection, encourage uniform outcome reporting, allow for easier comparison across centers, and allow for a comparison with standardized normative population values.5,13,29 Although the PROMIS-PF and PROMIS-PI subscale scores were found to correlate with legacy hip PROMs,3,19,24,27 the lack of hip-specific questions may affect their ability to accurately define outcome status. At 6-month and 1-year follow-up using group-level assessments, the iHOT-12 was found to be more responsive than PROMIS-PF and PROMIS-PI subscale scores based on relative efficiency scores.3,27 However, relative efficiency compares 2 PROMs to find which measure has the highest power for a fixed sample size. 22 Group-level differences between PROM scores do not guarantee a perceptible or clinically meaningful difference for individual patients.7-9
As an alternative, CIOVs translate PROM scores into clinically relevant terms and therefore may provide better benchmarks for individual patient success.7-9 Bodendorfer et al 4 reported PASS scores for the question “Do you consider that your current state is satisfactory?” in patients at 1 year after hip arthroscopy for FAIS. The corresponding PASS scores for the iHOT-12, PROMIS-PF, and PROMIS-PI were found to be 64.1, 47, and 53.7, respectively. Others have reported iHOT-12 PASS scores of 63 and 69.1 in similar patients at 1-year follow-up.25,26 These reported PASS values were consistent with the SCB scores for being ≥80% satisfied in the present study. Beleckas et al 2 also reported SCB scores for being “a great deal better” or “a good deal better” since surgery with corresponding SCB scores for the iHOT-12, PROMIS-PF, and PROMIS-PI to be 73.6, 49.9, and 51.9, respectively. Others have reported iHOT-12 SCB scores of 86 and 72.6 at 1 year after hip arthroscopy.23,25 Kuhns et al 20 found PROMIS-PF and PROMIS-PI 1-year PASS scores to be 51.8 and 51.9, respectively, for those reporting an “excellent” or “good” rating of function. These reported PROMIS and iHOT-12 SCB scores were similar to the SCB scores found for being 100% satisfied in the present study.
The present study provided absolute SCB scores for those with ≥80%, ≥90%, and 100% satisfaction after surgery and compared the accuracy of the PROMIS-PF and PROMIS-PI subscales with the iHOT-12 in defining these outcomes. The AUC from ROC analysis defines the strength of association and the accuracy of the score in distinguishing between groups, with values ranging from 0.7 to 0.8 being acceptable and >0.8 to 0.9 being excellent.16,21 In the present study, AUC values for the PROMIS-PF and iHOT-12 were similar and ranged between 0.73 and 0.82. The PROMIS-PI subscale had the lowest AUC values, ranging between 0.67 and 0.75. Sensitivity refers to the ability of a score to identify patients who reported being ≥80%, ≥90%, and 100% satisfied, with specificity being the ability of a score to identify patients who do not reach those thresholds. All 3 instruments had similar sensitivity, and specificity values ranged between 0.60 and 0.82. The PROMIS-PF was most accurate in identifying those being ≥80% satisfied at 1-year follow-up with values for AUC, sensitivity, and specificity being 0.82, 0.82, and 0.72, respectively. This means that those who score ≥45 on the PROMIS-PF will be correctly identified as being ≥80% satisfied 82% of the time, and as a score decreases, it becomes less likely they would report being >80% satisfied. While the PROMIS-PF was as accurate as the iHOT-12 in distinguishing between groups, it is unclear whether another existing legacy instrument or a new more developed instrument could be more accurate.
There is evidence to support the PROMIS for various orthopaedic conditions in joints other than the hip, including the spine, 35 shoulders,31,33 knees,12,17 foot, and ankle.14,15 Similar to the hip, PROMIS scores generally correlate with legacy instruments.31,33,35 In patients with foot and ankle pathology 15 and those after knee arthroscopy,12,17 the PROMIS-PF and PROMIS-PI subscales demonstrated similar responsiveness to legacy PROMs with group-level statistics. However, further studies are needed to investigate the accuracy of CIOVs for the PROMIS in identifying, grading, and classifying changes in scores at the individual level. Patients with FAIS in this study had a mean preoperative PROMIS-PF score of 40.8, which improved to 50.8, which is a change in score at the 10-point SD for PROMIS scores. In contrast, the mean initial iHOT-12 score was 33.8, improving to 68.1 at the 1-year follow-up. This large change in score indicates a significant improvement. The fact that the PROMIS is a general measure of function, while the iHOT-12 is disease-specific may explain why the PROMIS scores were higher at baseline, with the iHOT-12 scores having a greater change. The change in score on all 3 instruments was significantly different when comparing pre- and 1-year postoperative scores. Further study is needed to determine whether this greater change in score on the iHOT-12 allows for more accurate differentiation between various changes in improvement.
Limitations
While these results represent a multicenter study, there are a number of limitations that need to be acknowledged. This study is limited in the use of the absolute SCB values to English-speaking patients at the 12 months ± 30 days follow-up for hip arthroscopy for FAIS. This study is also limited to the anchor-based question and responses for the level of postoperative satisfaction. Other methods to determine SCB values—such as looking at subgroups of those with FAIS—could produce different results. Although patients were given instructions to complete the PROMIS-PF and PROMIS-PI subscales in regard to their surgical hip, the lack of hip-specific questions may have posed problems for those with other issues that limited their function. The nonrandomized, retrospective study design, limited sample size with 16% incomplete surveys, and only a 1-year follow-up are other limitations.
Conclusion
The PROMIS-PF and PROMIS-PI subscales were as accurate as the iHOT-12 in defining absolute SCB scores for those with at ≥80%, ≥90%, and 100% satisfaction at 1-year follow-up after hip arthroscopy to correct FAIS.
Footnotes
Final revision submitted January 21, 2023; accepted January 30, 2023.
One or more of the authors has declared the following potential conflict of interest or source of funding: J.D.H. has received research support from DePuy and Smith & Nephew; education payments from MedInc and Smith & Nephew; consulting fees from Smith & Nephew; nonconsulting fees from Smith & Nephew and Xodus Medical; royalties from SLACK and Thieme Medical Publishers; and hospitality payments from Stryker; and had stock/stock options in PatientPop. T.E. has received education payments from Arthrex; consulting fees and honoraria from Graymont Professional Products and Medacta; and hospitality payments from Stryker. R.K. has received hospitality payments from Stryker. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto. Ethical approval for this study was obtained from the University of California, San Francisco (reference No. 18-26040).
