Abstract
Background:
Minimal clinically important differences (MCIDs) increase patient-reported outcome measure (PROM) utility by defining benchmarks of meaningful improvement. The threshold for appreciable improvement may be unique to specific patient characteristics. To help establish benchmarks and improve evaluation of patient progress, this study investigates whether surgery type and demographic characteristics impact MCID.
Methods:
All foot and ankle patients within a single institution’s PRO registry were screened. Patients without preoperative or ≥ 180-day follow-up in PROMIS physical function (PF) Computer Adaptive Test (CAT) (v2.0) and pain interference (PI) CAT (v1.1) were excluded. Foot and Ankle Single Assessment Numeric Evaluation (FA SANE) was used as an anchor for MCID calculations. MCID was compared across 11 foot and ankle surgical categories, reported sex, body mass index (BMI), and social deprivation index (SDI).
Results:
We included 1243 patients with an average follow-up of 456 ± 176 days. Patients significantly improved in PF by 8.2 ± 10.9 (P < .001) and PI by −8.1 ± 9.5 (P < .001). Global MCID estimates for PF and PI were 5.5 (95% CI 4.9, 6.0) and −5.8 (95% CI −5.3, −6.3), respectively, when anchored against FA SANE. The MCIDs using a ½ SD distribution-based method were 5.5 and −4.8, respectively. Surgery type greatly affected MCID, ranging from 2.3 and −3.8 for hardware removals to 15.7 and −9.8 for small soft tissue surgeries. MCID differences across reported sex, BMI, and SDI never exceeded 1.9 points.
Conclusion:
A 6-month postoperative improvement of ≥ 5.5 and ≤ –4.8 in PROMIS PF and PI suggest meaningful clinical differences from baseline. However, MCID values for PROMIS scales vary across foot and ankle surgeries and patient characteristics, suggesting a global MCID for patient populations may not reflect the true threshold of meaningful change for each individual patient. Anchoring to FA SANE allows physicians to more objectively gauge appreciable change in patient-specific contexts.
Level of Evidence:
Level III, retrospective cohort study.
This is a visual representation of the abstract.
Keywords
Introduction
Patient-reported outcome measures (PROMs) serve as a valuable adjunct to interpreting changes in clinical measures and as a tool for assessing treatment outcomes from the patient’s perspective. Often, they are more cost-efficient, practical, and precise than the clinical measures they supplement. 4 Determining the minimal clinically important difference (MCID) of PROMs increases their utility in clinical practice, research initiatives, and regulatory programs by providing benchmarks of appreciable improvement for populations of interest. 16 Furthermore, the MCID may better define meaningful treatment effects that cannot be otherwise captured by clinical assessments and significance tests.8,20,32
There are 2 recommended methods for calculating the MCID. Jaeschke et al 13 first described an anchor-based method which uses an independent anchor to define meaningful improvement. These approaches are the most faithful to the intended purpose of the MCID by being linked to patient perception. Alternatively, distribution-based methods utilize a sample’s SD alone, most commonly ½ of the SD, to derive MCIDs.8,25
Because of substantial variability in PROMs collected throughout orthopaedic subspecialties, MCID values tend to be multifarious.7,16,18,31,32 In foot and ankle, MCID values have been described for PROMs including the Foot and Ankle Outcome Score, 26 visual analog scale, 22 36-Item Short Form Health Survey, 26 and the Foot and Ankle Ability Measure.3,12,22 Anchor-based MCID estimates for measures within the Patient-Reported Outcomes Measurement Information System (PROMIS) are limited. 9
Despite being well described, there are still uncertainties within MCID estimates and limitations in the methodologies. First, MCID values reported in literature may be an overgeneralization of entire patient populations. The MCID may be affected by disease type and severity. 16 Calculating the MCID for surgical interventions related to specific conditions and injuries is needed to confirm this suggestion. Sociodemographic factors have also been explored as variables affecting the ability to meet meaningful changes in PROMIS scores.6,33 However, these influences require further research to draw conclusions. Without a better understanding of which variables affect the MCID, and to what degree, clinical teams cannot confidently apply a single benchmark measure to all patients within an orthopaedic subspecialty.
Regarding the methodologies, anchor-based approaches can produce unreliable MCID estimates if the anchor is not adequately correlated to the PROM of interest or is irrelevant to the patient. 34 Furthermore, this approach typically utilizes categorical responses, like a global ratings of change scale, to determine the MCID. 30 This requires the clinical team to subjectively determine which response represents the degree of improvement considered meaningful. In addition to dismissing patient perception by removing an external anchor, the distribution-based methodology is scrutinized for providing the smallest detectable, rather than the smallest meaningful change. Establishing an approach that pulls from both methods by applying the ½ SD of an objective and numerically continuous PROM as an anchor could address the limitations of both methods, but no such studies exist.
The purpose of this study is to establish benchmarks of meaningful change in PROMIS physical function and pain interference within foot and ankle orthopaedic surgery. We aim to define global MCID using a novel anchor-based method. Furthermore, we aim to identify differences in MCID values based on 11 foot and ankle surgical categories, reported sex, body mass index, and social deprivation. Calculating MCIDs for specific patient populations will more precisely establish benchmarks of meaningful change and improve the ability to evaluate patient progress.
Materials and Methods
Patient Identification
All patients at a single institution were enrolled into an automated patient-reported outcome (PRO) database once scheduled for foot and ankle surgery. PROMs were voluntarily collected at preoperative (< 30 days prior to operation) and 3-, 6-, 12-, and 24-month intervals following a patient’s associated surgery and stored in a HIPAA-compliant data capture system (Patient IQ, Chicago, IL). Institutional review board approval was granted for secondary analysis of all outcomes collected through this PRO system.
Patients with completed preoperative measures of Foot and Ankle Single Assessment Numeric Evaluation (FA SANE), PROMIS physical function (PF) Computer Adaptive Test (CAT) (PROMIS Bank v2.0) and PROMIS pain interference (PI) CAT (PROMIS Bank v1.1) were exported from the PRO database. Exported data spanned August 2019 to September 2023. An internal data integration service was used to match demographic information from health records to PROM data. Patients aged ≥ 18 years were included for analysis if they had matching demographic data and PROMIS PF, PROMIS PI, and FA SANE measures at baseline and ≥ 180 days follow-up. Patients with multiple surgeries (n = 192) were included for analysis once, using their earliest surgery unless a subsequent surgery with greater significance was performed (ie, ankle arthroscopy followed by ankle arthroplasty).
Statistical Analysis
Demographic data were used to create specific patient cohorts, including surgery category, reported sex, body mass index (BMI) classification, and social deprivation. Subgroup data were nearly complete for the whole sample, only excluding 6 patients for BMI analysis and 2 patients for social deprivation index (SDI) analysis because of missing data. The author team used Current Procedural Terminology (CPT) codes to organize all patients into 11 surgical categories including Achilles repair, amputation, ankle arthritis/degeneration, ankle fracture, excision, hardware removal, infection, midfoot/forefoot arthritis/degeneration, midfoot/forefoot deformity, midfoot/forefoot fracture, and small soft tissue procedures (eg, fasciotomy, gastrocnemius recession, tarsal tunnel release). BMI was categorized as underweight (≤ 18.50), normal weight (18.51-25), overweight (25.1-30.0), and obese (> 30.0). Social deprivation was defined using the 2019 ZCTA level Robert Graham Center Social Deprivation Index (SDI) data, a 7–demographic characteristic composite measure of socioeconomic standing that has been used in the orthopaedic literature. 19 Popular indices such as the Area Deprivation Index use 9-digit zip codes; however, our data set was limited to 5-digit zip codes, making the SDI the most accurate index compatible with our patient data. The SDI was categorized by quartiles, with the first quartile including patients within neighborhoods of the lowest calculated SDI. Improvements in PROMIS PF and PROMIS PI were assessed using paired t tests. A P value of <.05 was considered statistically significant.
Calculating the MCID
Calculations of the MCID were performed in R Studio (R Version 4.3.2). We calculated the MCID using a novel anchor-based method, applying the FA SANE, a validated outcome measure for foot and ankle pathologies, 5 as an anchor for improvement. On a sliding scale of 0-100, the FA SANE asks patients, “If 100% is perfectly normal, what percent of normal is your foot/ankle today?” We aimed to define an objective benchmark of appreciable improvement in FA SANE to serve as the anchor so as to avoid subjective assignment of meaningful change. To establish this benchmark, we calculated ½ the SD for the sample’s pre- to postoperative difference in FA SANE, similar to traditional distribution-based methods. We then constructed a simple linear regression, with the pre- to postoperative change in FA SANE as the primary predictor and the change in the PROM of interest (PF or PI) as the outcome. The global MCID was predicted by evaluating the regression model at the defined ½ SD anchor benchmark. For subsequent subgroup analyses, separate linear regressions were used to calculate an anchor-based MCID for each patient cohort, including FA SANE and cohort as main effects, along with their interaction. A 95% CI was calculated for each predicted MCID value. As a control to our novel method, we also calculated a distribution-based MCID using the ½ SD of PROMIS PF and PI improvements as the MCID threshold, similar to prior studies. 12
Results
Study Sample
Our PRO registry contained 7520 unique surgeries with 2933 and 2863 having preoperative PF and PI scores, respectively. Following exclusion, 1243 unique patients were included for analysis with both preoperative and ≥ 180-day follow-up PF and PI (Figure 1).
Screening for inclusion of patients with PROMIS PF and PROMIS PI scores for MCID analysis.
The average preoperative PROMIS PF and PI scores were 39.6 ± 8.5 and 61.5 ± 7.0, respectively. Preoperative FA SANE scores were an average of 41.7 ± 25.3. Patients reported a statistically significant improvement in PF of 8.2 ± 10.9 (P < .001) and PI of −8.1 ± 9.5 (P < .001) at longest follow-up. FA SANE observed a significant average improvement of 31.1 ± 32.8 points (P < .001). The average length of follow-up was 456 ± 176 days (Table 1).
Patient Demographics.
BMI, body mass index.
Global MCID
At a minimum of 180 days follow-up, improvement in FA SANE demonstrated a significant association but a modest linear relationship with PF (R2 = 0.31, P < .001) and PI (R2 = 0.30, P < .001) (Figure 2).
Linear relationships between the difference in FA SANE (anchoring metric) and (
The anchor-based MCID in PF, referred to as the global MCID, was 5.5 (95% CI 4.9, 6.0). The global MCID for PI was −5.8 (95% CI −5.3, −6.3). The distribution-based method yielded a comparable MCID in PF at 5.5, whereas the MCID in PI was notably lower at −4.8 (Tables S1 and S2).
MCIDs for Unique Surgery Categories
Consideration of specific surgery category introduced variability to the calculated MCIDs, particularly in PF (Figure 3). Patients undergoing small soft tissue surgeries had the highest MCID, with values of 15.7 (95% CI 11.4, 20.0) and −9.8 (95% CI −5.7, −13.8) for PF and PI, respectively. The smallest MCID was hardware removal, with improvements in PF and PI of 2.3 (95% CI 0.3, 4.2) and −3.8 (95% CI −2.0, −5.7). In addition to small soft tissue surgeries, the MCID tended to be high for trauma-related surgeries of the ankle and Achilles tendon, but not the midfoot and forefoot. The MCID for midfoot and forefoot arthritis and degeneration was comparable to ankle arthritis and degeneration. The distribution-based method was less variable in MCID, and commonly resulted in lower values (Supplementary Tables S1 and S2).
MCID values with 95% CIs for (
Reported Sex MCID
The MCID in PF for male patients narrowly exceeded the global MCID at 5.7 (95% CI 4.8, 6.6), whereas female MCID was slightly lower than the global MCID at 5.2 (95% CI 4.6, 6.0) (Figure 4). The MCID values for PI were nearly identical for males, −5.8 (95% CI −6.6, −5.0), and females, −5.8 (95% CI −6.4, −5.2). Compared with the distribution-based method, the MCID for PF was similar, although PI was roughly 1 point lower (Supplementary Tables S1 and S2).
MCID values with 95% CIs for (
Body Mass Index MCID
Normal-weight patients reported the highest MCID values for PF at 5.9 (95% CI 5.0, 6.9), and PI at −6.2 (95% CI −5.3, −7.0) (Figure 5). The highest and lowest BMI classifications were associated with the lowest MCID values. Apart from a higher MCID for underweight classification, the distribution-based values were comparable for PF and roughly one point lower for PI (Tables S1 and S2).
MCID values with 95% CIs for (
Social Deprivation Index MCID
Patients below the 50th percentile SDI reported MCIDs narrowly below the global MCID for PF, and those above the 50th percentile reported values above the global MCID (Figure 6). Little variation existed for the MCID in PI. Interestingly, patients in the fourth quartile had an MCID above the global value for PF, but below the global value for PI. Again, distribution-based results were similar for PF but consistently lower for PI (Supplementary Tables S1 and S2).
MCID values with 95% CIs for (
Discussion
This study used a novel method of calculating the MCID by using the FA SANE as an anchor for appreciable clinical change, which was modestly correlated with improvements in PF and PI at an ≥ 180-day follow-up. Applying this method to a sample of 1243 patients, we calculated the MCID in PF to be 5.5 and the MCID in PI to be −5.8. The distribution-based MCID values for our sample were 5.5 and -4.8, respectively. The observed MCID differences between methodologies are consistent with the literature. A systematic review from Gilat et al found that across orthopaedic subspecialties, anchor-based MCID values were greater than distribution-based values by 1.28 points in PROMIS PI but only by 0.08 points in PROMIS PF. 9
Our distribution-based values are similar to those in other foot and ankle studies.9,11,12 Many anchor-based studies use a global ratings of change scale to determine the MCID, 30 requiring subjective determination of where the meaningful threshold exists on a Likert scale of improvement. An MCID analysis of foot and ankle surgeries using the global change anchor reported values for PF of 7.8 (SD 7.7) and PI of 8.0 (SD 6.7). 12 Additionally, Stephan et al 29 demonstrated that at 6 months postoperatively, improvements in PF and PI vary by as much as 6 points when the global change anchor moved from “helped a little” to “helped.” Our anchor-based MCID values are notably lower, and more consistent, falling in between the distribution-based and anchor-based values previously reported.
A similar assessment to anchor-based MCID calculations is the Patient Acceptable Symptom State (PASS) determination. PASS also uses an anchor to determine single postoperative PROM scores that demonstrate a patient’s satisfaction with their postoperative state. 17 In the foot and ankle literature, PASS thresholds have been defined for PROMIS PF at scores ranging from >44.7 to >50.0 and PI at scores ranging from < 56.0 to < 46.9.2,27 PASS is advantageous because of its ability to obviate the need for baseline scores while still defining a threshold to achieve patient-perceived meaningful improvement. However, it does adequately appreciate patients who may experience meaningful postoperative improvement, but due to low baseline scores, do not achieve PASS. In this setting, counseling patients with poor preoperative PROMs using PASS thresholds may be discouraging, rendering accurate anchor-based MCID values, like those in this study, more useful for guiding patient expectations and progress.
Ultimately, our methodology avoids recall bias, subjective threshold assignment, and potential for post hoc revisions that often distort the reliability of many anchors-based methodologies. 15 Taking the ½ SD of a PROM has been historically used as an objective MCID approximation, lending credence to its use as an FA SANE benchmark to anchor other PROMs. To then build a predictive model between FA SANE and PROMIS measures that uses the ½ SD in FA SANE, we introduce an anchor that is numerically objective and based on patient perception.
Evaluating MCID by Surgery Category
Stratification of MCID calculations by surgery category revealed considerable variation. Small sample sizes in some categories led to very wide CIs. The impact of surgery type is not well understood, largely due to heterogeneous patient populations, PROMs, and MCID methodologies.9,30 Karhade et al 14 compared 7 common orthopaedic surgeries using an anchor-based MCID for PROMIS PF, demonstrating a moderate range of values from 3.5 to 8.1. However, these surgeries did not include any surgeries represented in this study. The work of Karhade et al valuably demonstrates cross-subspecialty differences but did not capture the MCID range that may exist within different surgeries of a single subspecialty. For example, our ankle arthritis cohort, containing all ankle arthroplasties, reported a PF MCID of 4.9. Meanwhile, the ankle fracture cohort was twice as large at 10.6. In addition to small soft tissue surgeries, our anchor-based method demonstrated higher MCID values among trauma-related operations. This trend could be explained by the acuity of these injuries, and an expectation of returning to the level of function and pain prior to injury. Conversely, in the setting of prolonged pain and limited function associated with chronic conditions, small improvements in PI or PF could relieve daily disability, explaining a smaller MCID. Ultimately, the variability observed in MCID values across surgical categories should be considered by clinical teams to more precisely set expectations for distinct surgical populations.
Reported Sex
Reported sex had the smallest impact on MCID across all variables. For PI, both sexes had MCID values nearly identical to the global value. Males and females had values on either side of the global MCID for PF, but only differed by 0.5 points. Some evidence suggests that PROMIS PF and PI are lower in females than males, 24 but upper extremity cohorts have observed increased odds of obtaining the MCID among female patients. 1 Furthermore, among lumbar fusion patients, the MCID attainment rate for PROMIS PF differed by an insignificant rate of 6% between reported sex. 21 Given the contradicting evidence in different orthopaedic specialties, and the lack of investigation within foot and ankle surgery, further work is needed to validate any conclusions based on reported sex.
Body Mass Index
In our sample, patients at either end of the BMI classification required the smallest improvements in PF to achieve the MCID. For PI, underweight individuals had an MCID nearly half of the global value, while all other classifications did not differ from the global value by more than 0.4 points. A limited sample size in the underweight classification created large CIs that limit the strength of any conclusion regarding the influence of underweight BMI on MCID. Shaikh et al 28 reported that for rotator cuff repair, BMI had no influence on odds of achieving the MCID in PROMIS PF or PI. PROMIS scores have also lacked significant differences between BMI classifications among lumbar fusions after 6 months. 23 No MCID values reported in our foot and ankle sample showed substantial differences when considering the CIs, but some variation suggests consideration should be taken in future investigations.
Social Deprivation Index
Patients living in the lowest SDI quartile neighborhoods had MCID values for PI and PF similar to the global value. Higher MCID from the global value was observed within the highest SDI quartiles. Large differences between area deprivation index quartiles in PROMIS PF have been previously reported across orthopaedic subspecialities, 33 but the effect on magnitude of change following treatment was not explored. Bernstein et al 6 found that the anchor-based MCID for PROMIS PF did not differ by income, but PROMIS PI did. By not using a multidimensional tool for deprivation, direct comparisons to our analysis are limited. While we observed higher MCID values for the higher SDI quartiles in PF and unique differences in PI, the overlap in CIs indicates that SDI did not significantly impact the MCID. Ultimately, the influence of sociodemographic factors on individual health is complex and best assessed with multidimensional indices. 10 Further investigation into how the SDI affects the perception of meaningful clinical change in PROMs is necessary to make MCID interpretations more holistic.
Considering Recursion in the MCID Methodology
The MCID was met by 657 patients (53%) for PF and 691 (56%) for PI in our sample. On consideration of our MCID methodology, we recognize that attainment rate may be affected by recursion in the calculation. The regression model for the MCID is built using the linear relationship between the change in PROM of interest and the change in FA SANE. The methodology becomes recursive by using the ½ SD of FA SANE as the predictor variable for the MCID output.
Ultimately, our methodology includes benefits of both distribution-based and anchor-based approaches by incorporating statistical parameters and an objective external outcome anchor. This value can be used to compare individual postoperative improvement to the aggregate. However, the recursive nature of the predictive model makes it difficult to evaluate patients at a population level and calculate the proportion of a sample meeting the MCID.
Limitations
The racial identity of our region’s population, and therefore study sample, is not necessarily representative of the United States as a whole, being predominantly White or Caucasian. The results we obtained may not be generalizable to more diverse samples, and further research should be done to explore factors impacting the MCID within more representative cohorts. Furthermore, although our sample size is a strength of our study, preoperative noncompliance and patients not yet reaching adequate follow-up excluded nearly 75% of the surgeries from our registry. The entire registry and the analysis cohort were comparable in age (49.8 ± 17.3 vs 51.3 ± 15.7) and reported sex (56.8% female vs 61.1% female), but our registry does not collect other subgroup data, such as BMI or SDI that was collected via the data integration service. Additionally, well-matched demographics do not confirm if compliant patients report similar PROMs as those that are noncompliant. Additional limitations exist in our PROM dissemination. Our registry only administers PROMs in English, and follow-up PROMs are often administered digitally if patients are not seen for a clinic visit after surgery. These restrictions may cause non–English-speaking patients and those with limited access to technology to be less represented in our analysis.
Despite the numerous categories of foot and ankle surgeries, considerable heterogeneity still exists within each group. For example, Achilles repair includes both insertional and midsubstance ruptures and ankle arthritis/degeneration includes both arthroplasty and arthrodesis. Each of these patients have distinct presentations, operations, and rehabilitations. However, an analysis of each unique surgery would substantially increase CIs, as seen in the infection group. This would greatly limit the reliability of MCID estimates and reduce the ability to compare estimates across groups. Nonetheless, even with heterogeneity within groups, there is appreciable differences between groups, suggesting that the MCID should be addressed in surgery-specific contexts. We encourage future multiinstitutional studies to achieve sample sizes that would support the necessary degree of surgical specificity.
Conclusions
A 6-month postoperative improvement of ≥ 5.5 and ≤ –4.8 in PROMIS physical function and pain interference, respectively, suggest a meaningful clinical difference from baseline in foot and ankle surgery. However, the MCID for PROMIS scores are highly affected by surgery type, and a global MCID estimate may inaccurately guide patient expectations. Clinical teams should set expectations and appreciate improvement in the precise context of a patient’s surgical profile. More variation can be expected in PROMIS physical function than pain interference. Little variation exists in MCID values across patient sex, BMI, and SDI but may be used to supplement projections in postoperative improvement primarily set by a patient’s surgery. Using an objective numerical anchor, such as the FA SANE, may more accurately calculate the variability in MCIDs for large orthopaedic surgical populations.
Supplemental Material
sj-pdf-1-fao-10.1177_24730114251334055 – Supplemental material for PROMIS Minimal Clinically Important Differences Across Foot and Ankle Surgeries
Supplemental material, sj-pdf-1-fao-10.1177_24730114251334055 for PROMIS Minimal Clinically Important Differences Across Foot and Ankle Surgeries by Nathaniel E. Zona, Michael A. Hewitt, Carson Keeter, Sara E. Buckley and Kenneth J. Hunt in Foot & Ankle Orthopaedics
Footnotes
Appendix
MCID Values for PROMIS Pain Interference Using FA SANE Anchor-Based and Distribution-Based Methods.
| Subgroup | FA SANE Anchor | ½ SD |
|---|---|---|
| Global | −5.81 | −4.76 |
| Procedure category | ||
| Achilles repair | −6.83 | −4.56 |
| Amputation | −5.90 | −4.87 |
| Ankle arthritis/degeneration | −5.38 | −4.53 |
| Ankle fracture | −6.97 | −5.01 |
| Excision | −6.67 | −5.36 |
| Hardware removal | −3.82 | −3.82 |
| Infection | −5.65 | −5.49 |
| Midfoot/forefoot arthritis/degeneration | −5.49 | −4.41 |
| Midfoot/forefoot deformity | −5.31 | −4.35 |
| Midfoot/forefoot fracture | −5.14 | −5.03 |
| Small soft tissue | −9.79 | −4.75 |
| Biological sex | ||
| Male | −5.76 | −4.84 |
| Female | −5.79 | −4.70 |
| BMI classification | ||
| Underweight | −2.9 | −4.71 |
| Normal weight | −6.15 | −4.73 |
| Overweight | −5.75 | −4.82 |
| Obese | −5.66 | −4.72 |
| Social Deprivation Index | ||
| First quartile | −5.74 | −4.89 |
| Second quartile | −5.86 | −4.36 |
| Third quartile | −6.17 | −5.03 |
| Fourth quartile | −5.44 | −4.66 |
Abbreviations: FA, foot and ankle; MCID, minimal clinically important difference; PROMIS, Patient-Reported Outcomes Measurement Information System; SANE, Single Assessment Numeric Evaluation
Ethical Approval
The project was approved by the University of Colorado Institutional Review Board, no. 22-1281.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Disclosure forms for all authors are available online.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
