Abstract
Background:
The Victorian Institute of Sport Assessment–Achilles (VISA-A) questionnaire is validated and widely used in Achilles tendinopathy. How well it can evaluate treatment outcomes is not well understood.
Purpose:
To evaluate the responsiveness of the VISA-A in midportion Achilles tendinopathy and compare it with other patient-reported outcome measures.
Study Design:
Cohort study (diagnosis); Level of evidence, 2.
Methods:
Enrolled were 97 participants with clinically diagnosed Achilles tendinopathy (median age, 50 years [interquartile range, 18 years]; symptom duration, 10 months [interquartile range, 28.7 months). The participants underwent a baseline evaluation and completed between 1 and 6 follow-up evaluations at 8, 16, 24, 32, 40, and/or 48 weeks. Participants completed the VISA-A, the Patient Reported Outcomes Measurement Information System short form Version 2.0 (PROMIS) Physical Function and Pain Interference subscales, and the Tampa Scale for Kinesiophobia (TSK). Three thresholds were evaluated with a receiver operating characteristic analysis (minimal clinically important difference [MCID], substantial benefit [SB], and complete recovery [CR]) using an 11-point global rating of change scale as an anchor. Thresholds were evaluated on raw scores as well as changes from baseline.
Results:
The VISA-A was able to detect all 3 thresholds for changes over time, with raw scores >70.5, >77.5, and >89.5 representing the MCID, SB, and CR, respectively; thresholds for changes from baseline on the VISA-A were increases of 23.5, 19.5, and 37.5 points from baseline, respectively. The PROMIS subscale raw scores had identical thresholds for SB and CR (52.45 for Physical Function and 45.6 for Pain Interference). A score <34.5 on the TSK was the threshold for SB.
Conclusion:
The VISA-A was the most responsive outcome measure evaluated. Raw scores had increasingly higher thresholds for the MCID, SB, and CR, which were therefore logically consistent.
Keywords
The author Frank Herbert wrote, “To know a thing well, know its limits” 14 —a relevant quotation for the Victorian Institute of Sport Assessment–Achilles (VISA-A) questionnaire, a tool to measure the severity of symptoms in Achilles tendinopathy. 31 Some recently exposed limits of the VISA-A include ceiling effects on many items, 4 as well as low comprehensiveness and comprehensibility. 19 The VISA-A score is also influenced by physical prowess: the ability to hop, do heel raises, and walk for 30 minutes. 31
It is worthwhile to know the VISA-A well. It is commonly used as an outcome measure in Achilles tendinopathy, 22 which can affect patients of any demographic. 1,6 Outcome measures stand or topple on a single limit: when and for whom does it work? A systematic review identified 3 studies that evaluated the responsiveness of the VISA-A. 15,16,25 Of the 3 studies, 2 merely evaluated if people with Achilles tendinopathy scored differently from healthy controls. 15,16 As clinical researchers, we put more stock in the ability to measure improvements over time, an ability only evaluated by the third study, 25 with just 15 participants. The study concluded that 6.5 points was the minimal clinically important difference (MCID). 25 That is a very small change, considering that the minimal detectable difference is at least 7 points. 20
The MCID is a low bar to clear. Most of our patients have higher aspirations, and some even want to fully recover from their tendinopathy. Yet, no one has evaluated if the VISA-A can measure anything beyond a MCID. The aim of this study was to evaluate how responsive the VISA-A is to changes over time as well as full recovery in people with midportion Achilles tendinopathy, and to compare the VISA-A to other outcome measures.
Methods
Study Overview
This retrospective cohort study was a secondary analysis using all currently available data from an ongoing prospective treatment study in midportion Achilles tendinopathy (ClinicalTrials.gov study identifier NCT03523325). 27 Some of the same participants have been used in previously published articles. 10,33 The study protocol was approved by the institutional review board at the University of Delaware, and all participants provided informed consent.
We wanted to include as many people as possible for the study. To that end, inclusion criteria for the study were broad: age between 18 and 65 years and a clinical diagnosis of midportion Achilles tendinopathy based on a clinical examination conducted by the study clinicians. 21 Participants with symptoms at the Achilles tendon insertion were included, but only if the midportion was the primary complaint. Exclusion criteria were having a history of Achilles tendon ruptures or surgery to the Achilles tendon. Participants who answered "yes" to any of the questions on the Physical Activity Readiness Questionnaire 37 were asked to seek physician clearance to exercise and if such clearance was not granted the participants were excluded. Recruitment sources for participants were primarily social media advertisements, mailing lists of various organizations, and referrals from local clinics.
The study participants were evaluated at baseline, and every 8 weeks for up to a year. Treatment was administered by study physical therapists who did not perform any of the evaluations. The treatment protocol consisted of progressively loaded heel-rises 34 with any adjustments the physical therapists deemed appropriate on a case-by-case basis.
Questionnaires
At each time point, participants answered the VISA-A, 31 as well as the Patient Reported Outcomes Measurement Information System short form version 2.0 (PROMIS-29) 13 and the Tampa Scale for Kinesiophobia 17-item questionnaire (TSK). 26 The PROMIS-29 was developed as a general-purpose outcome measure that could be used for many different patient populations. 13 TSK scores have relevance as a potential prognostic indicator. 23
Participants answered all subscales of the PROMIS-29, but only the Physical Function and the Pain Interference subscales were used for this study, as together they measure the same variables as the VISA-A. 35 Both PROMIS subscale scores were converted to a t metric, in which 50 is the mean of a reference population (general population) and the standard deviation is 10. For the Physical Function subscale, a higher score indicates more physical function (a better score), whereas for the Pain Interference subscale, a higher score indicates more pain interference (a worse score).
Study data were collected and managed using the REDCap (Vanderbilt University) tools hosted at the University of Delaware. 11,12 After completing the questionnaires, the participants additionally underwent a series of clinical tests, an ultrasound examination, and a lower extremity functional evaluation, but these were not used for this analysis and are therefore not described.
Defining Improvement Categories
We used the anchor-based method, a common approach to calculate the MCID. 5 The method relies on a reference standard (anchor) to determine which participants have shown improvement. Unfortunately, there is no objective way to measure tendinopathy yet. Instead, we settled for the global rating of change (GRC) scale as the reference standard. The GRC scale was administered at all follow-up visits (not at baseline) and preceded the other questionnaires. We posed the question, “With respect to your Achilles tendon injury, how would you describe yourself now compared to when you began the study?” Responses were on an 11-point scale, in which –5 was considered “very much worse,” 0 was considered “unchanged,” and 5 was considered “completely recovered.”
We chose values of 2 to 3 on the GRC scale to represent the MCID. This choice was arbitrary but is not unprecedented. In the landmark anchor-based study on MCIDs, values of 1 to 3 of 7 were used as the definition. 17 As the reliability of an 11-point GRC scale could be as low as 0.8, we were not confident that a score of 1 would reliably indicate a clinically meaningful change; thus, 2 other categories of improvements were also defined: substantial benefit (SB), defined as a 4 on the GRC scale, similar to the original definition 8 ; and complete recovery (CR), which was explicitly defined as the highest score on the GRC scale, as it is the ideal outcome in Achilles tendinopathy.
Statistical Analysis
Incomplete questionnaires were excluded from the analysis. To determine the questionnaire thresholds for each improvement category, we used a receiver operating characteristic (ROC) analysis. Each threshold was defined as the highest combined sensitivity and specificity to differentiate between those that were improved and a lower category. The MCID was the highest combined sensitivity and specificity to identify a score of 2 or 3 versus a score ≤1 on the GRC scale. The SB differentiated between a GRC score of 4 versus a score of 1 or 2, and CR differentiated between a GRC score of 5 versus a score of 3 or 4. The area under the ROC curve (AUC) was used as the effect size. The AUC is a measure of how well the outcome measure functions to differentiate between improvement categories and is the probability that a randomly selected improved patient scores higher than a randomly selected unimproved patient.
A clinically important difference could be an improvement from baseline or reaching an important raw score threshold. As there is no way to determine which is more useful, we calculated both. This yielded a total of 6 ROC analyses per questionnaire (3 categories of improvement, 2 ways to express change). There is no method to adjust ROC analyses for multiple comparisons. However, a significance test of an AUC calculation is equivalent to the Wilcoxon signed-rank test, 24 which was therefore used to test for significance. The false discovery rate could then be controlled using the procedure of Benjamini and Hochberg 2 for the 6 independent analyses within each questionnaire, and alpha was set at the traditional .05. All data processing and statistical analysis were performed using R. 28 The ROC analysis and plotting were done using the package pROC. 30
Results
Participants
A total of 181 potential participants were screened for eligibility between July-2018 and April-2021. The majority of participants were recruited through online advertisements on social media or mailing lists from local sports organizations. The number of people screened, reasons for exclusion, and the number of participants at each follow-up are presented in a flow diagram in Figure 1. The final analysis included all 97 participants (48 female) from the ongoing trial who had completed at least 1 follow-up visit.
Flowchart of participants screened and excluded. The number of participants at each follow-up sums to >97 because most participants completed >1 follow-up appointment. Email advertisements were from advertisements sent to mailing lists of various local sports organizations, as well as the internal University of Delaware emails.
The median age of the included participants was 50 years (interquartile range [IQR], 18 years), and the duration of symptoms was 10 months (IQR, 28.7 months). The mean (± SD) height was 173 ± 8.7 cm and weight was 81.8 ± 19.9 kg. The median current physical activity evaluated on a 6-point scale was 5 (IQR, 2). Twenty participants reported previous Achilles tendon symptoms, and 4 reported a history of traumatic injury (but not ruptures) to the Achilles tendon. Detailed baseline characteristics, including medical diagnoses and medication use, are presented in accordance with a recent consensus statement 29 in Appendix Table A1. The mean follow-up was 32 weeks, and 56 participants had completed the 1-year follow-up evaluation. All available data were used for the analysis; therefore, each participant had between 1 and 6 follow-up time points. No participant was excluded because of missing data. At baseline, the median scores were as follows: VISA-A, 50 (IQR, 22); PROMIS–Physical Function, 45.3 (IQR, 7.6); PROMIS–Pain Interference, 53.9 (IQR, 8.9); and TSK, 38 (IQR, 7.5). The number of times each GRC score was reported is shown in Table 1.
GRC Scale and the Number of Times Each Value Was Reported a
a GRC, global rating of change.
b Number of times a participant had given the rating. Note: Each participant answered the GRC scale 1 to 6 times depending on the length of follow-up at the time of the analysis.
ROC Analysis
VISA-A
Both the VISA-A raw score and change from baseline had thresholds for MCID, SB, and CR (Table 2) but with overall larger AUCs for raw scores. The raw score thresholds increased systematically with each category (70.5, 77.5, and 89.5, respectively) (Table 2 and Figure 2A), while the change from the baseline threshold was lower for an SB than for an MCID (Table 2, Figure 2B).
Results of the ROC Analysis a
a MCID, SB, and CR values were determined based on global rating of change scores: MCID = 2-3, SB = 4, and CR = 5. Boldface P values indicate statistical significance (P < .05). AUC, area under the curve; CR, complete recovery; MCID, minimal clinically important difference; PROMIS, Patient Reported Outcomes Measurement Information System short form Version 2.0; ROC, receiver operating characteristic; SB, substantial benefit; VISA-A, Victorian Institute of Sport Assessment–Achilles; ↑, increased; ↓, decreased.
b Adjusted based on the method of Benjamini and Hochberg 2 for multiple independent hypotheses.
Receiver operating characteristic curves for the VISA-A for (A) raw scores and (B) changes from baseline. Black dots mark the position of the highest combined sensitivity and specificity. CR, complete recovery; MCID, minimal clinically important difference; SB, substantial benefit.
PROMIS–Physical Function
The raw t score for the PROMIS–Physical Function subscale had the same threshold for SB and CR (Table 2, Figure 3A) and no threshold for MCID (Table 2). When using change from baseline, there were thresholds for MCID and SB but not for CR (Table 2, Figure 3B).
Receiver operating characteristic curves for the PROMIS–Physical Function for (A) raw scores and (B) changes from baseline. Black dots mark the position of the highest combined sensitivity and specificity. CR, complete recovery; MCID, minimal clinically important difference; SB, substantial benefit.
PROMIS–Pain Interference
The raw t score for the PROMIS–Pain Interference subscale had the same threshold for SB and CR (Table 2, Figure 4A) and no threshold for MCID. The change from baseline had the same threshold for MCID and SB but no threshold for CR (Table 2, Figure 4B).
Receiver operating characteristic curves for the PROMIS–Pain Interference for (A) raw scores and (B) changes from baseline. Black dots mark the position of the highest combined sensitivity and specificity. CR, complete recovery; MCID, minimal clinically important difference; SB, substantial benefit.
TSK
The TSK had thresholds for SB in both raw scores (Table 2, Figure 5A) and changes from baseline (Table 2, Figure 5B) but no thresholds for MCID or CR.
Receiver operating characteristic curves for the Tampa Scale for Kinesiophobia for (A) raw scores and (B) changes from baseline. Black dots mark the position of the highest combined sensitivity and specificity. CR, complete recovery; MCID, minimal clinically important difference; SB, substantial benefit.
Discussion
The main finding of the study was that the VISA-A is still seaworthy. It can detect changes over time in people with midportion Achilles tendinopathy, and it does better than the PROMIS-29 subscales we evaluated, as well as the TSK. Unfortunately, none of the outcome measures had large AUCs, not even the VISA-A. Ideally, a responsive outcome measure would have a pattern with the lowest threshold for the MCID, followed by the SB and finally CR. This pattern was only observed for the VISA-A raw scores.
VISA-A
One study had previously identified an MCID on the VISA-A using the anchor-based method. 25 That study reported a change of 6.5 points on the VISA-A as the MCID, much lower than the 23.5-point change in the current study. Important differences between the studies explain the discrepancy. The previous study had a sample size of 15 individuals, and only 3 of those were not classified as improved. 25 The authors also pooled together minimally improved individuals and those who received greater benefits. 25
VISA-A expressed as the change from baseline required a large improvement to show a clinically important change, ∼20 points. Unlike the raw scores, changes from baseline thresholds did not increase systematically across the thresholds. Our results demonstrate that using raw scores is better than using changes from baseline because doing so yields increasing thresholds across categories of improvement.
Studies that report VISA-A scores for healthy participants generally see scores above 90 points, 16,35 which is consistent with our threshold score of 89.5 for CR. Yet slightly <80% of our completely recovered participants achieved this score on the VISA-A. The VISA-A is really a composite of 2 factors: symptoms and physical activities. 35 Those who do not regularly exercise cannot achieve a score >90, but they can self-report CR.
PROMIS-29
The results from the PROMIS-29 subscales were harder to interpret, as neither raw t scores nor changes from baseline consistently increased/decreased across improvement categories. A Physical Function score above 52.45 classified both the SB and the CR. No threshold classified MCID. Achieving either SB or CR required the highest possible score. The MCID from baseline for Physical Function was 1.65,the lowest possible increase. The Pain Interference subscale had a similar pattern; the threshold for CR was the best possible score (least pain interference) and the threshold for SB was the second best score.
No studies have evaluated the reliability or smallest detectable changes for the PROMIS-29 subscales in Achilles tendinopathy. In patients undergoing hip arthroplasty, the smallest detectable changes in Physical Function and Pain Interference are 6.6 and 8.8, respectively. 36 The SB threshold for the Physical Function subscale exceeded the smallest detectable change, but the Pain Interference threshold did not.
A big advantage of the PROMIS is the t-score conversion. 13 Essentially, t scores enable direct comparisons between vastly different populations, such as patients with heart disease compared with those with tendinopathy. Recent editorials and reviews have begun to suggest adopting the PROMIS as a replacement for disease-specific outcome measures. 3,32 However, a counterargument is the potential trade-off in responsiveness compared with disease-specific measures. 18 Our results support the latter viewpoint; the PROMIS-29 subscales are not responsive enough to evaluate changes in Achilles tendinopathy.
TSK
The TSK has 1 threshold: a score below 34.5 indicates an SB. The TSK is different from the other questionnaires in our study in that it evaluates a very small and specific component of the bigger picture. We included it because a systematic review on psychological outcome measures in tendinopathy found an association between kinesiophobia and worse outcomes in Achilles tendinopathy. 23 Reduced willingness to perform painful tendon loading exercises likely explains this effect. Our results are consistent with how the TSK is likely to be used in clinical practice. A single important threshold score close to the frequently used cutoff (38 points) classifies high levels of kinesiophobia. 7,38 A change in TSK scores in which a participant scores below 34.5 points is therefore clinically meaningful.
Limitations
Our study has several important limitations to consider. This is a secondary analysis of data originally collected for the purpose of comparing outcomes between men and women. The analysis is therefore inherently exploratory. To decrease the risk of false-positive results, we have reported all analyses performed and adjusted our P values. The study recruited individuals aged between 18 and 65 years regardless of activity level. There was no additional sampling from the parent study, as all data that had been collected in the parent study were used for this analysis. However, the study sample had a median age of 50 years, and an activity level median of 5 of a possible 6 on the physical activity questionnaire. The findings should therefore only be generalized to that population.
GRC scores are subject to a recall bias in which the current health status, as well as real changes, affects scores. As the time from baseline increases, more changes are required for a given GRC score, which in turn increases the thresholds for each improvement category. 9 This likely adds uncertainty and may contribute to the fairly low AUC values observed.
We used the PROMIS-29 Version 2.0 short form. 13 This static version of the PROMIS subscales can alternatively be administered dynamically as a computer-administered test. Although the static short forms used in this study were not demonstrated to be as useful as outcome measures in Achilles tendinopathy, the computer-administered tests might be.
CONCLUSION AND CLINICAL RELEVANCE
Our study evaluated the responsiveness of patient-reported outcome measures in patients with midportion Achilles tendinopathy. The VISA-A is sufficiently responsive to be useful as an outcome measure in Achilles tendinopathy. Achieving a score >70.5 can be considered the MCID; >77.5, an SB; and >89.5, CR.
The PROMIS short form Version 2.0 subscales of Physical Function and Pain Interference are not responsive enough to be used as outcome measures in Achilles tendinopathy. The TSK shows a single useful threshold in which achieving a score <34.5 on the TSK is a meaningful improvement.
In the introduction, we touched on shortcomings of the VISA-A when viewed in light of modern methods. 4,19 Despite these shortcomings, the VISA-A was the only outcome measure evaluated in our study that demonstrated sufficient responsiveness to be used as an outcome measure. However, through this study we have highlighted a limit of VISA-A. Since nonathletes may not be able to achieve a score >89.5 on the VISA-A, for use in a mixed pool of athletes and nonathletes, the VISA-A may be in need of maintenance.
Footnotes
Acknowledgment
The authors acknowledge the following people for collecting the data used in this paper: Andrew Sprague, Patrick Corrigan, Shawn Hanlon, Brian Honick, Kayla Seymour, and Hayley Powell. They also thank the Tendon Research Group at the University of Delaware for help with data collections.
Final revision submitted March 10, 2022; accepted April 8, 2022.
The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
One or more of the authors has declared the following potential conflict of interest or source of funding: Research reported in this publication was supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health (award No. R01AR072034). AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
Ethical approval for this study was obtained from the University of Delaware Institutional Review Board.