Abstract
Dear Editor,
Thank you for the opportunity to address the comments recently submitted in response to our recently published analysis: “OASIS 1: Retrospective analysis of four different microprocessor knee types.” 1 The letter clearly articulates five points of discussion to which we would address our response.
First, regarding the assertion that we refer to the retrospective nature of our study as the most important aspect of the study design, this is incorrect. It is specifically the cross-sectional nature of the study design that is most salient. The letter notes “cross-sectional study designs assess outcomes at only one point in time and, thus, do not allow for any conclusions on causal relationships to explain the results.” We would submit that we are very clear to not note causality, and indeed pose many statements in hypothetical context. For example, regarding findings with mobility, we have deliberately noted that the data would ‘suggest that when improved mobility is considered a primary aim for MPK prescription’ clinicians could consider all models of knees. The suggestive language is preferred over stronger, more absolute language.
The authors of the letter later comment on the potential for clinically meaningful differences with relation to the findings for injurious falls despite lack of statistical significance. They suggest we “neglected” these differences, and that we “stopped short of drawing the conclusion.” We agree with the emphasis on clinically meaningful differences. However, the two statements made in the letter are contradictory as initially the letter places emphasis on posing discussion points more as hypotheses needing further testing. The language throughout the manuscript is consistent, the data would ‘suggest that when stability and falls reduction is considered a primary aim for MPK prescription’, clinicians may consider the C-Leg or Orion.
Regarding “absence of bias,” we are misquoted in the letter. The manuscript specifically states “absent of potential bias due to manufacturer funding or publication.” Nowhere within the manuscript do we state that the study is without bias. As mentioned within the manuscript, the study design carries inherent biases such as selection bias, and our falls questionnaire is subject to recall bias. There is also potential for observation bias with patients performing outcomes for their clinicians. We contend, while manufacturers certainly publish studies with attempts at minimizing bias, there will always be a need for additional studies that would not carry the same potential for confirmation bias, or pressure of publication bias.
In the second point of the letter, it is noted that the “study was unable to control for a potential clinician bias in the selection of the different MPK.” We acknowledged selection bias in the manuscript, but there is great value in the increased ecological validity when examining the outcomes of more than 600 MPK users.
The third point raised in the letter revolves around the term “parity.” The correspondents contend it to be a legal term and scientifically inadequate. We are unable to find where the term parity is noted as a legal term. The first definition of Merriam-Webster for parity is “the quality or state of being equal or equivalent,” and indeed nowhere within Merriam-Webster’s definition does the term “legal” appear. Searches of other resources show similar definitions. However, while the term “equivalent” appears within the definition, the letter is correct in that we have carefully, and deliberately, not used the term equivalent, or non-inferiority. As noted by the letter, such usage of terms in a scientific context implies certain statistical tests, and accompanying hypotheses, which were not the focus of our study.
Regarding the suggestion for the need for formal tests of equivalence, we thank the correspondents for this recommendation and take this opportunity to expand on our original work. A common two one-sided test (TOST) was applied to the dataset of outcomes for mobility, quality of life (QoL) and satisfaction with amputation status (SAT) used in the OASIS 1 study to assess equivalency. The initial publication refrained from such analysis as any evaluation of equivalency relies on a margin of equivalency, or defining upper and lower equivalence bounds. 2 These definitions are absent in the prosthetic outcomes literature. However, our mobility outcome measure, the Prosthetic Limb Users’ Survey of Mobility (PLUS-M) has a defined minimal detectable change (MDC) of 4.5 points. 3 As such, we use this as a minimum potential margin of equivalency. Note that a larger margin of equivalency will result in increased likelihood of two interventions being found equivalent. With regards to QoL and SAT from the Prosthesis Evaluation Questionnaire, there is no published MDC. However, the assessment was implemented as a discrete 10-point scale, and subsequently the minimum response difference is only a single point, representing a 10% difference in scale. We apply one point as the minimum potential margin of equivalency again accepting the limitation that this may be too small, and if this is too small any potential findings of non-equivalence may be a false positive that would subsequently not be found with a larger margin of equivalency (i.e. greater than 1 point).
For mobility, with equal variance assumed (Levene’s test F3,583 = 0.33, p = 0.81), there were no comparisons between knees that demonstrated non-equivalency (Figure 1(a) and Table 1).
Mean differences with 95% confidence interval for comparisons of mobility (a), quality of life (b), and satisfaction (c). Vertical gray dash lines representing lower and upper equivalency bounds. Confidence intervals that span these bounds signify non-equivalence. * = non-equivalence at 0.05.
Two one-side tests equivalency results (shaded = non-equivalence).
For QoL, with equal variance assumed (Levene’s test F3,591 = 0.46, p = 0.71), the comparison between C-Leg and Plié found these devices were not equivalent, with C-Leg on average having higher QoL (Figure 1(b)). 3 For SAT, with equal variance assumed (Levene’s test F3,589 = 2.05, p = 0.11), the comparison between Plié and Rheo found these devices were not equivalent, with Rheo on average having higher SAT (Figure 1(c)). 4 As noted in the primary manuscript, this may be the result of the limited sample size of the Rheo. Ultimately, while the OASIS study originally noted points of parity, the current analysis demonstrates there is a degree of equivalence among the four MPK models in the areas of mobility, SAT, and QoL.
Lastly, the letter also raises issue with the use of the term “decline”, which is purported to signify time series data which is not what is captured through the cross-sectional analysis. We believe that there may be confusion with the term “reduction,” or “reduced,” which is synonymous with measuring changes over time. A quick search will note many studies using the term decline in cross sectional analyses. There are studies in areas of medicine outside of prosthetic rehabilitation, but also studies in prosthetic rehabilitation. For example, consider the recent publication by Hahn et al., “Analysis of clinically important factors on the performance of advanced hydraulic, microprocessor-controlled exo-prosthetic knee joints based on 899 trial fittings.” This cross-sectional analysis of MPK fittings led the study authors to conclude “Responsiveness and compound functional benefit decline in those above 76 years.” 4 We would agree the use of the term “reduced” would have been inappropriate given the lack of longitudinal measurements as patients age, and would agree that there is possibility that the age-related declines in mobility may be a result of selection bias.
We believe the publication of the OASIS study highlights parities and differences among four primary MPK models. We see this as the first step to understanding clinical determinants of success to better inform clinical decisions versus partialities impressed upon clinicians through marketing materials. We thank the correspondents for their interest in our work and the opportunity to respond to their comments.
Sincerely, James H Campbell, Phillip M Stevens and Shane R Wurdeman
