Abstract
Background:
Research has identified factors that influence achievement rates of threshold scores on patient-reported outcome measures (PROMs) after hip arthroscopy. However, little data exist on whether failure to achieve a threshold score (minimal clinically important difference [MCID] or substantial clinical benefit [SCB]) in the short term after hip arthroscopy predicts the risk of future hip surgery.
Purpose/Hypothesis:
The purpose of this study was to determine if failure to achieve the MCID or SCB on PROMs within 1 year of hip arthroscopy can be considered a risk factor for repeat surgery within 5 years of primary hip arthroscopy. It was hypothesized that failure to achieve threshold scores would increase the risk of subsequent hip surgery.
Study Design:
Case-control study; Level of evidence, 3.
Methods:
A retrospective review of prospectively collected data was completed. Four PROMs were collected preoperatively and within 1 year of hip arthroscopy: modified Harris Hip Score, Hip Outcome Score–Activities of Daily Living, Hip Outcome Score–Sports, and the 33-Item International Hip Outcome Tool.
Results:
Two cohorts were formed: (1) a study cohort (n = 88) composed of patients who underwent repeat hip surgery within 5 years of hip arthroscopy and (2) a control cohort (n = 288) composed of patients who did not require repeat hip surgery. The study cohort had significantly (P < .001) lower scores on all postoperative PROMs, and a significantly (P < .001) smaller percentage of the study cohort met the MCID and SCB. Multivariable regression analysis demonstrated that not achieving the MCID or SCB on each of the PROMs is an independent risk factor for repeat hip surgery. For every PROM in which a patient failed to achieve the MCID, the odds of subsequent surgery increased by 1.68 (95% CI, 1.42-1.98; P < .001). For every PROM in which a patient failed to achieve the SCB, the odds of subsequent surgery increased by 1.63 (95% CI, 1.35-1.97; P < .001).
Conclusion:
Failure to meet threshold scores on PROMs after hip arthroscopy was an independent risk factor for subsequent hip surgery. This study establishes a novel utility of PROMs and confirms the importance of these metrics in the orthopaedic literature.
The number of patients undergoing hip arthroscopy for symptomatic femoroacetabular impingement (FAI) with or without a concomitant acetabular labral tear continues to increase annually. 17,23,37 This can be attributed to the documented success of the operation in randomized control trials, prospective clinical studies, and retrospective cohort studies 22 combined with lower rates of complications compared with open surgical approaches. 21
The metrics used to define the outcome of hip arthroscopy as a success or failure are numerous and include revision surgery, physical examination, and return to sport. 3,4,11,35,39 With the ongoing shift in medical care toward a patient-centered approach, patient-reported outcome measures (PROMs) have become a focal point of hip arthroscopy literature. 1,15,19,24 These outcome measures are completed by the patient, instead of the clinician, and therefore represent an accurate patient-centered perspective on the benefits of medical care. However, a clinical interpretation of PROMs can be a unique challenge because statistically significant changes in PROMs are sometimes associated with clinically insignificant differences. 16,31 This can make it difficult for a practicing physician to extrapolate data on PROMs to everyday practice. However, the challenge is somewhat overcome via utilization of the minimal clinically important difference (MCID) and substantial clinical benefit (SCB). While the value of the MCID and SCB can be dependent on the particular technique utilized for calculation, these are numeric changes in PROMs that indicate a significant improvement in clinical function. 10,14,27
The MCID was initially introduced to determine the significance of any change in PROMs 10 and is now defined as the smallest change in outcome score a patient considers meaningful. 27 The SCB is a similar value that ascertains the upper threshold of change in outcome score and is defined as improvements perceived as clinically considerable. 14 Achievement of these threshold scores after hip arthroscopy allows a clinician to state that the operation was a success. As a result, a central component of hip-preservation research has focused on the identification of preoperative factors that increase or decrease the likelihood of achieving the MCID or SCB. However, little data exist on the clinical progression of FAI for patients that fail to meet a threshold outcome score such as the MCID and/or SCB after hip arthroscopy. This concept has been investigated in the field of total hip arthroplasty (THA), with the results demonstrating that failure to meet a threshold score is predictive of revision risk status post–primary THA. 8,33,38 Based on these findings, it is reasonable to infer a similar trend in hip-preservation surgery.
The purpose of this study was to determine if failure to meet the MCID and/or SCB within 1 year of hip arthroscopy can be considered an independent risk factor for subsequent hip surgery within 5 years of primary hip arthroscopy. Our principal hypothesis was that failure to achieve the MCID and/or SCB within 1 year of hip arthroscopy would significantly increase the risk of subsequent hip surgery within 5 years of the primary procedure. In addition, we hypothesized that the risk of subsequent surgery would increase for each additional outcome measure in which a patient failed to achieve the MCID and/or SCB.
Methods
This study was approved by our organization’s institutional review board. PROM data utilized in the present study were collected from 2006 to 2018 via a separate institutional review board–approved hip preservation registry. The registry is a standard-of-care, multisurgeon, hospital-wide registry that tracks the clinical course and PROMs of patients diagnosed with symptomatic FAI that is managed medically and/or surgically. PROMs are collected prospectively for all patients preoperatively and at set time intervals after hip arthroscopy. In the present study, patients were included from four high-volume, fellowship-trained hip arthroscopy surgeons. A retrospective chart review was then completed to identify patients who could be included in the analyses.
All patients were assessed before and after hip arthroscopy with 4 hip specific PROMs: the modified Harris Hip Score (mHHS), 2 Hip Outcome Score–Activities of Daily Living (HOS-ADL), 18 Hip Outcome Score–Sports (HOS-Sports), 20 and the 33-Item International Hip Outcome Tool (iHOT-33). 25 All postoperative PROMs were collected between 5 and 11 months after primary hip arthroscopy. Previously established values for the MCID and SCB using outcomes for patients 1 year post–hip arthroscopy 12,19,25,26 were applicable for the present study and therefore utilized for statistical analyses. MCID values of 8.2, 8.3, 14.5, and 12.1 27 and SCB values of 19.8, 10.0, 29.9, and 24.5 26 were used on mHHS, HOS-ADL, HOS-Sports, and iHOT-33, respectively.
Exclusion criteria consisted of (1) prior ipsilateral hip surgery, (2) history of traumatic ipsilateral hip injury, or (3) radiographically confirmed hip osteoarthritis (Tönnis grade >1). Further patients were excluded from the study if they had not completed at least one of the outcome measures before surgery and the same outcome measure within one year of their primary procedure.
After application of the aforementioned criteria, 1,509 patients who met the criteria were included in initial chart review. Chart review was completed to identify patients (1) who underwent subsequent ipsilateral hip surgery within 5 years of their primary procedure and (2) who were greater than 5 years removed from hip arthroscopy and did not require additional hip surgery. We initially identified 101 patients who underwent subsequent surgery; however, 13 of these patients underwent surgery longer than 5 years after their initial hip arthroscopy and were therefore excluded from the study. Of the 1408 patients who did not undergo additional surgery after their primary procedure, 1120 of these patients either were less than 5 years removed from primary surgery (n = 585) or had not returned for an office visit greater than 5 years after their primary procedure (n = 535) and were thus excluded from the study.
After application of the above parameters, the study cohort consisted of 88 patients, whereas the control cohort consisted of 288 patients. The flowchart documenting patient selection can be seen in Figure 1. Of the 88 patients in the study cohort, 86 completed mHHS and HOS-ADL both pre- and postoperatively, 84 completed HOS-Sports both pre- and postoperatively, and 64 completed iHOT-33 both pre- and postoperatively. Of the 288 patients in the control cohort, 284 completed HOS-ADL both pre- and postoperatively, 278 completed mHHS both pre- and postoperatively, 274 completed HOS-Sports both pre- and postoperatively, and 222 completed iHOT-33 both pre- and postoperatively.
Flowchart of patient inclusion and exclusion process for this study.
Statistical Analysis
In each of the cohorts, patient characteristics and outcome measures were described. Continuous variables were reported as means with ranges, while categorical variables were reported as counts and percentages. Chi-square and t tests were used to compare categorical and continuous variables between the 2 cohorts, respectively. Multivariable mixed-effects logistic regression was used to determine if failure to meet the MCID or SCB for each PROM within 1 year of surgery is associated with undergoing a subsequent surgery. A final multivariable model was run to assess the impact of failing to meet the MCID or SCB on more than one outcome measure. Age, sex, and baseline PROM score were controlled for in each multivariable analysis. Differences with a P <0.05 were considered statistically significant in the present study.
The sensitivity, specificity, positive likelihood ratio (LR+), and negative likelihood ratio (LR–) were calculated, accompanied by 95% confidence intervals (CIs), for each outcome measure. Of note, because MCID and SCB values are calculated as a change from baseline, a small number of patients with a high enough baseline score were unable to achieve the MCID/SCB. All statistical analyses were performed using RStudio Version 1.2.1335.
Results
Patient characteristics, baseline PROMs, surgical indications, and surgical procedures for both cohorts are reported in Table 1. No significant differences were apparent between cohorts concerning age, sex, baseline PROMs, and surgical indications. When comparing procedures performed during arthroscopy, the control cohort did have a significantly greater percentage of patients undergo labral repair (83.7% vs 68.2%; P = .005) and cam decompression (92% vs 77.3%, P = .003). There were no significant differences between cohorts concerning the remaining surgical procedures.
Patient Characteristics, Baseline PROMs, Surgical Indications, and Surgical Procedures a
a Data are reported as n (%) or mean (range). Bolded P values indicate a statistically significant difference between groups (P < .05). HOS-ADL, Hip Outcome Score–Activities of Daily Living; HOS-Sports, Hip Outcome Score–Sports; iHOT-33, 33-Item International Hip Outcome Tool; mHHS, modified Harris Hip Score; PROMs, patient-reported outcome measures.
The average time to subsequent surgery in the study cohort was 22.3 months (range, 5.5-59.3 months). Twenty-four patients (27.3%) in the study cohort underwent a subsequent THA, with the average time to THA being 26.4 months (range, 5.5-59.3 months). The remaining 64 patients (72.7%) underwent subsequent surgery at an average of 20.8 months (range, 6.0-51.7 months). Of these 64 patients, the most common subsequent surgery was a revision hip arthroscopy (50 patients). The remaining procedures included surgical hip dislocation (6 patients), periacetabular osteotomy (5 patients), open excision of heterotopic ossification (2 patients), and 1 patient underwent repair of the gluteus medius. This information is presented in Table 2.
Type and Time to Subsequent Surgery for the Study Cohort
a Other subsequent surgeries included hip arthroscopy (n = 50), surgical hip dislocation (n = 6), periacetabular osteotomy (n = 5), excision of heterotopic ossification (n = 2), and gluteus medius repair (n = 1).
Postoperative PROMs and the percentage of each cohort to achieve the MCID and SCB are presented in Table 3. The study cohort reported significantly lower results across all PROMs (P < .001). Additionally, on all PROMs, a significantly smaller percentage of patients in the study cohort achieved the MCID and SCB when compared with those in the control cohort (P < .001).
Postoperative Score and Percentage of Cohort to Achieve the MCID and SCB a
a Data are reported as mean (range) or % (n/total). All differences between groups were statistically significant (P < .05). iHOT-33, 33-Item International Hip Outcome Tool; HOS-ADL, Hip Outcome Score–Activities of Daily Living; HOS-Sports, Hip Outcome Score–Sports; MCID, minimal clinically important difference; mHHS, modified Harris Hip Score; PROM, patient-reported outcome measure; SCB, substantial clinical benefit.
b Because MCID and SCB values are calculated as a change from baseline, a small number of patients with a high enough baseline score were unable to achieve the MCID/SCB.
Multivariable logistic regression demonstrated that failure to achieve the MCID or SCB on any PROM significantly increased the risk of a subsequent surgery when controlling for age, sex, and baseline PROMs. Failure to achieve the MCID on any single PROM significantly increased the odds of subsequent surgery, with odds ratios of 5.39 for mHHS, 5.20 for HOS-ADL, 4.78 for iHOT-33, and 4.02 for the HOS-Sports (P < .001 for all). Failure to meet the SCB on any single PROM also significantly increased the odds of revision surgery (odds ratios, 5.2 for iHOT-33, 4.69 for HOS-ADL, 4.57 for HOS-Sports, and 4.11 for mHHS; P < .001 for all). This information is presented in Table 4.
Multivariable Logistic Regression for Subsequent Surgery Risk Controlling for Age, Sex, and Baseline PROMs a
a Part A shows the risk of revision surgery associated with failure to achieve the MCID or SCB on any PROM. Part B shows the risk of revision surgery associated with the cumulative effect of failure to achieve the MCID or SCB on multiple PROMs. All results were statistically significant (P < .05). iHOT-33, 33-Item International Hip Outcome Tool; HOS-ADL, Hip Outcome Score–Activities of Daily Living; HOS-Sports, Hip Outcome Score–Sports; MCID, minimal clinically important difference; mHHS, modified Harris Hip Score; PROM, patient-reported outcome measure; SCB, substantial clinical benefit.
A final multivariable model showed that each additional PROM in which a patient did not achieve a threshold score significantly increased the risk of subsequent surgery when adjusting for age, sex, and baseline scores. For each additional PROM, failure to meet the MCID increased the odds of subsequent surgery by 1.68 (P < .001), and failure to meet the SCB increased the odds of subsequent surgery by 1.63 (P < .001). This model is presented in Table 4.
Sensitivity, specificity, and likelihood ratios are reported in Table 5. The mHHS had the highest sensitivity (0.41; 95% CI, 0.33-0.49) and LR+ (3.06; 95% CI, 2.12-4.67) for the MCID. The mHHS (0.87; 95% CI, 0.82-0.91) and HOS-Sports (0.87; 95% CI, 0.81-0.92) had the highest specificity for the MCID. When the SCB was examined, iHOT-33 had the highest sensitivity (0.34; 95% CI, 0.27-0.42) and LR+ (3.55; 95% CI, 2.13-6.86). The highest specificity for the SCB was seen on HOS-Sports (0.91; 95% CI, 0.83-0.95). The LR– was <1.0 for all outcome measures for both the MCID and SCB.
Sensitivity, Specificity, Positive Likelihood Ratio, and Negative Likelihood Ratio for the PROMs Evaluated a
a Data in parentheses are 95% CIs. Sensitivity, specificity, LR+, and LR– were calculated as follows:
HOS-ADL, Hip Outcome Score-Activities of Daily Living; HOS-Sports, Hip Outcome Score-Sports; iHOT-33, 33-Item International Hip Outcome Tool; MCID, minimal clinically important difference; mHHS, modified Harris Hip Score; LR+, positive likelihood ratio; LR–, negative likelihood ratio; PROM, patient-reported outcome measure; SCB, substantial clinical benefit.
Discussion
The results of this study support our initial hypothesis that failure to meet the MCID and/or SCB on the mHHS, HOS-ADL, HOS-Sports, or iHOT-33 within 1 year of hip arthroscopy significantly increases the risk of a subsequent hip surgery within 5 years of the primary procedure. Patients who underwent subsequent surgery had significantly lower scores on all postoperative PROMs compared with those who did not undergo a subsequent surgery. Additionally, a significantly smaller percentage of patients who underwent subsequent ipsilateral hip surgery reported a change in outcome score that met the MCID and/or SCB compared with those who did not undergo a subsequent hip surgery. Our principal hypothesis was then substantiated by multivariable logistic regression controlling for age and sex. The results demonstrated that failure to meet the MCID or SCB on any of the 4 included PROMs is an independent risk factor for subsequent hip surgery. A final multivariable logistic regression also examined the effect of failing to meet the MCID or SCB on more than one PROM. This analysis of the cumulative effect of failure to meet a threshold score revealed that for each additional PROM in which a patient failed to achieve a threshold score, their risk of subsequent surgery significantly increased. Overall, failure to meet the threshold score for the SCB on iHOT-33 was associated with the largest increase in risk for subsequent surgery. When examining only the MCID, failure to meet the threshold score for iHOT-33 was again associated with the greatest risk of subsequent surgery.
The outcome measure with the highest reported sensitivity for subsequent surgery was mHHS (0.41) and iHOT-33 (0.34) for the MCID and SCB, respectively. Regarding specificity, the highest values were reported on both mHHS and HOS-Sports (0.87) for the MCID and on HOS-Sports (0.91) for the SCB. It is important to recall that sensitivity, defined as the true-positive rate, and specificity, defined as the true-negative rate, function in an inverse relationship, and as one value increases, the other will decrease. 36 Therefore, our data, upon initial review, suggest that the utilization of threshold values after hip arthroscopy can act as a highly specific measure of the risk of subsequent surgery. Despite this initial assessment, further analysis of the data reveals that the LR+ and LR– capture the utility of these measures with greater accuracy.
Likelihood ratios function as a means to understand how probable it is that a patient with a condition is expected to have a result compared with patients without the condition. 6,29 In regard to the present study, it is the ratio of the probability of subsequent hip surgery in people who met the MCID and/or SCB compared with people who did not. A likelihood ratio increases in strength as it gets farther away from a value of 1.0, and in this analysis, all LR+ values were >1.0 and all LR– values were <1.0. This finding is in line with our initial hypothesis, as it suggests that failure to achieve threshold scores for the MCID or SCB increases the risk of undergoing a subsequent surgery and that patients who do achieve these threshold scores conversely have a smaller risk of subsequent surgery.
However, for diagnostic and prognostic purposes, an LR+ value >10 is desired for a test to be considered highly specific, and an LR– value <0.1 is desired for a test to be considered highly sensitive. 28,29 While all LR+ values in this study were >1.0, the highest LR+ value was 3.55. Similarly, while all LR– values were <1.0, the smallest LR– was 0.69. One can therefore conclude that failing to achieve a threshold score after hip arthroscopy is indeed associated with increased risk of subsequent surgery. However, when reviewing our results and the information from this discussion on the utility of likelihood ratios, it is apparent that the actual prognostic value of threshold scores from a clinical perspective may be limited. Therefore, we recommend these measures be used for prognostic purposes only when combined with other objective criteria. A threshold score can provide a comprehensive risk assessment for future hip surgery when combined with certain criteria such as physical examination findings, radiographic measures, and the patient’s desired level of physical activity. This can serve as valuable information to the hip-preservation surgeon for patient counseling throughout the postoperative course.
Several studies 7,9,30 in the field of hip-preservation surgery have demonstrated that preoperative PROMs carry prognostic value for postoperative outcome because of an established correlation with postoperative outcome scores as well as subsequent surgeries, including conversion to THA. Despite this, there is a paucity of data on the predictive risk of subsequent hip surgery with regard to achievement of the MCID and/or SCB after arthroscopic treatment of FAI. A recent retrospective study by Rosinsky et al, 32 to our knowledge, was the first to publish data supporting the novel concept that a patient’s failure or achievement of a threshold score correlates with rates of subsequent hip surgery after hip arthroscopy. This, however, was a single-surgeon study that investigated 2 PROMs (mHHS and HOS-ADL) and only established a correlation between threshold scores and subsequent hip surgery. A later study by Shapira et al 34 did investigate this relationship in a causal manner and found that the risk for secondary hip surgery may be evaluated with mHHS scores at 1 year after hip arthroscopy.
The results of our study substantiate the findings of the aforementioned studies. Further, the present study has expanded the discussion to include additional PROMs and included a regression analysis with parameters that controlled for confounding variables. Our findings indicate that failure to reach a threshold score can serve as an independent risk factor for subsequent hip surgery. As such, close monitoring should be implemented for patients who fail to achieve a threshold score in the short-term after hip arthroscopy as they are at a statistically significantly increased risk of subsequent hip surgery. Our results underscore the importance of the continued utilization of PROMs and threshold scores in orthopaedic literature. The inclusion of PROMs, in the form of both absolute score and comparisons to threshold values, allows the reader to gain a more robust understanding of the generalizability of study results and therefore provides the clinician the capacity to extrapolate the results to everyday clinical practice.
The strength of this study includes the large sample size, the use of established metrics to define meaningful clinical improvement, and the robust statistical analysis of our data on short-term PROMs. Further, the follow-up window requiring patients to have a clinical visit at a minimum of 5 years after surgery provides insight into how short-term achievement of threshold scores predicts mid- to long-term results after hip arthroscopy.
The present study has several limitations. This was a single-center retrospective study based on hospital-wide registry data collected in a prospective manner, which may limit the study’s generalizability. Future research that distinguishes between the causes of secondary surgery may be useful to determine where threshold scores have the greatest predictive quality. Additionally, this study did not differentiate patients based off the type of subsequent hip surgery or by specific procedures performed during arthroscopy. However, a significantly greater percentage of the control cohort underwent a cam decompression and a labral repair, which presents a possible confounding variable. A study wherein patients were divided on the basis of procedures during hip arthroscopy and the type of subsequent hip surgery they underwent would be of further use in investigating this topic. Certain objective preoperative clinical information, such as preoperative body mass index, joint parameters, and residual impingement, was not available for all patients and therefore not included in this analysis. Our study may underestimate the benefit of hip arthroscopy in the small number of patients who were unable to achieve the MCID/SCB owing to a high baseline PROM. Lastly, this study used previously established values for the MCID and SCB from other studies as reference values. While substantial heterogeneity does exist in MCID quantification methods, the MCID can be calculated as a score specific to a study population, 5,13 which reduces inconsistency. Thus, our decision to use reference values inherently limits the validity of the study.
Conclusion
The results of the current study show that the MCID and SCB thresholds for the mHHS, HOS-Sports, HOS-ADL, and iHOT-33 are all predictive measures of subsequent surgery risk for patients after primary hip arthroscopy. This study establishes a novel utility of PROMs and underscores the importance of these metrics in the orthopaedic literature.
Footnotes
Final revision submitted July 3, 2021; accepted August 4, 2021.
One or more of the authors has declared the following potential conflict of interest or source of funding: B.U.N. has received grant support from Arthrex, education payments from Arthrex and Smith & Nephew, and hospitality payments from Stryker and Wright Medical, and is the founder of and has an ownership interest in BICMID. A.S.R. has received consulting fees from Arthrex, Anika, Bodycad, Enhatch, Marrow Cellution, NewClip, Ranfac, Smith & Nephew, and Xiros; and speaking fees from Pfizer; and has an ownership interest in Conformis and Enhatch. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
Ethical approval for this study was obtained from the institutional review board of Hospital for Special Surgery (study No. 2020-1840).