Comment Regarding Article “Quantitative T2 MRI Mapping and 12-Month Follow-up in a Randomized,Blinded,Placebo Controlled Trial of Bone Marrow Aspiration and Concentration for Osteoarthritis of the Knees”

Dear Editor,

Recently, Shapiro et al. published in Cartilage their article titled “Quantitative T2 MRI Mapping and 12-Month Follow-up in a Randomized, Blinded, Placebo Controlled Trial of Bone Marrow Aspiration and Concentration for Osteoarthritis of the Knees.” ¹ We would like to acknowledge the authors for performing a randomized controlled trial (RCT) including both clinical and radiographic evaluations. However, there are some methodological issues that we would like to share. First, the authors state that their study is a pilot RCT to assess the feasibility of BMAC (bone marrow aspiration and concentration) injective approach, but we have our main concern in this regard. In fact, they previously reported the 6-month results of the same cohort of patients in 2016, ² and it is unusual to present updated results of a pilot trial, especially considering that the 12-month results are comparable to the 6-month results, without significant intragroup and intergroup difference. Furthermore, as the patients were treated between 2013 and 2015, it would have been better to report the 3-year data, which would represent a follow-up long enough to properly assess the overall duration of the beneficial effect, any eventual difference in terms of retreatment rate, and eventual MRI (magnetic resonance imaging) changes over time between the 2 groups. Moreover, the authors were more focused on discussing the reasons for the absence of any clinical and radiologic differences between BMAC and placebo than considering some relevant aspects of the study design that might have had an impact. The main goal of a pilot trial is not to demonstrate the efficacy of a treatment but, as clearly expressed in the CONSORT 2010 statement, the “rationale of a pilot trial is to investigate areas of uncertainty about the future definitive RCT. The primary aims and objectives of a pilot trial are therefore about feasibility, and this should guide the methodology used.” ³ Based on this premise, we would like to underline some aspects that could play a relevant role in the interpretation of the results of the article: in the BMAC group there were more patients who had received previous knee surgery (9) compared to the saline group (3), and this could have had a significant influence, especially if those patients received meniscectomy (the most common arthroscopic knee procedure) with the inherent consequences on the joint environment. Furthermore, in the BMAC group there were more Kellgren-Lawrence 3 patients (12) compared to the saline group (7) and, on the other hand, more Kellgren-Lawrence 2 patients were included in the saline group: treating patients with more advanced knee degeneration might underestimate the real potential of any injective treatment, including biologic ones. The most relevant issue to consider is that, given the low number of patients included (25 in total), it is very likely that statistics would yield no significant difference in the baseline features of the 2 treatment groups, but this does not mean that such discrepancies should be neglected: we all know that clinical experience sometimes is more helpful than any statistical test. Therefore, the “underpowered nature” of pilot studies is not only responsible for nonsignificant results (as largely expected in this kind of trial and as documented in the present study) but could also lead to risk of relevant bias that could affect interpretation of results. In this specific case, the potential lack of homogeneity between groups should warrant an accurate reevaluation of the inclusion criteria and the randomization methods in order to have 2 groups with comparable features at baseline, which is a mandatory condition for any robust RCT. The real utility of a pilot trial lies in the possibility of critically reviewing and challenging research methods to identify and correct potential weak points. In this regard, there are other aspects worthy of attention in our opinion: (1) The lack of use of validated questionnaires such as IKDC (International Knee Documentation Committee) subjective or WOMAC (Western Ontario and McMaster Universities Osteoarthritis Index) score for evaluating the patients. Among the aims of a pilot trial, there is the collection of data to perform a power analysis to calculate the sample size of a larger RCT, and the aforementioned clinical scores are the most commonly used to this purpose, so their use seems recommendable in the setting of a pilot trial. (2) The addition of platelet-poor plasma (PPP) to the BMAC before intraarticular injection, which could represent a confounding factor since PPP may exert its own biologic actions^4,5 and therefore limit the understanding of the contribution of BMAC itself. Last, the choice of treating patients affected by bilateral knee osteoarthritis seems also questionable since, as shown in previous studies, ⁶ clinical questionnaires cannot independently score both knees of one patient as they were 2 knees of 2 different patients: in bilateral osteoarthritis, one knee might affect the contralateral one and therefore this effect should be taken into account when performing the statistical evaluation, and a test with “correction for patient” (e.g., generalized linear model with the patient as a random effect) should be used. Anyway, the authors did not specify this particular aspect in their methods and perhaps a clarification would be helpful. In general, we believe that studies including bilateral patients receiving simultaneous treatment are not the ideal ones to assess both the safety and the efficacy of a treatment, due to the many biases associated to this particular study design. ⁷

We truly believe that there is a substantial need for randomized controlled trials to elucidate the real clinical effectiveness of mesenchymal stem cells’ concentrate, and the preliminary reports by Shapiro and colleagues represent a useful step in the attempt of promoting high-quality research in the field of biologic treatments for osteoarthritis. Pilot trials are fundamental to this purpose, and we think that, beyond reporting of results, methodological considerations are also necessary to address any doubt prior to starting a bigger and more demanding RCT.