Abstract
Clinical trials involving a placebo enable researchers to determine the effectiveness of a product; however, ensuring a placebo matches an active treatment takes great consideration, time and costs. We aimed to assess the reporting quality of blinding descriptions for placebo medication treatments and consider this in relation to funding support (commercial or non-commercial). The Journal of the American Medical Association (JAMA); the Lancet and the New England Journal of Medicine (NEJM) were searched for randomised clinical trials, and 117 papers involving a placebo medication, published between 1 April 2016 and 31 March 2017, were extracted. The data were analysed for the number of publications reporting characteristics of placebo treatments, frequency of the characteristics and source of funding. Three quarters of the articles reported at least one characteristic of the placebo. The Lancet and JAMA consistently had this information present; however, this was observed less in the NEJM. The most common characteristic was ‘matching placebo’, followed by contents of the placebo, packaging and appearance. Texture, taste and smell were least reported. Within those supported by commercial funding, two-thirds reported at least one characteristic of the placebo treatment, whilst almost all of the articles without commercial funding reported at least one characteristic. Efforts are being made to include descriptions of blinded medication; however, inconsistencies suggest that guidelines are not always being followed, and more can be done to improve reporting. Future research should focus on the reasons for inadequate recording and aim to reduce the inconsistencies observed.
Keywords
Introduction
Clinical trials are planned experiments that enable researchers to investigate the effect of an intervention on an outcome, based on the responses from a group of participants’ representative of the target population. Blinded randomised controlled trials are considered to be the gold standard of research. 1 In a double-blind trial, neither the investigators, outcome assessors nor the participants are aware of the treatment being received, and therefore this represents a strong design to minimise participant and investigator bias. 2 Wider members of the research team such as healthcare providers and statisticians may also be blinded to the designated allocation. Placebos are inert products which enable researchers to compare treatments while considering the psychological effect associated with the perception of a drug to provide a cure. 3
Blinding the medication used for clinical trials to ensure both the active and placebo drug match can present many practical challenges, which unfortunately can be underestimated and potentially impact on the successful delivery of the study. 4 When considering the specifications of a placebo, the various characteristics of the product must be identical to the medication under investigation, and difficulties in sourcing adequate placebos without collaboration of pharmaceutical companies have been reported previously. 5 Identical packaging and labelling are also necessary and present their own challenges in relation to the time required to repackage a medication, associated costs and the stability of the product. 4 As highlighted by Wan et al., 4 the reporting of blinding procedures is often incomplete or missing despite being included in the Consolidated Standards of Reporting Trials (CONSORT) statement; and while the impact of inadequate blinding may vary between studies, adequate blinding cannot be assumed when such details are absent from published materials.
Descriptions of blinding in published articles can vary significantly, and Schulz and Grimes 6 discussed how many articles do not contain adequate reporting, and therefore the reduction of bias often associated with blinding should be judged by the individual reader. Previously published literature appears to suggest that there are inconsistencies when reporting the descriptions of blinded medication, and Fergusson et al. 7 stress the need for trialists and journals to routinely report the methods of blinding. It is therefore important to understand the prevalence and scope of placebo medication descriptors being included. Our review aimed to assess the reporting quality of blinding descriptions for randomised clinical trials involving a placebo medication treatment, and review how these differ across three high impact medical journals, to determine if authors and publishers are including this level of detail in recently published manuscripts.
Methods
Three high impact medical journals (the Journal of the American Medical Association (JAMA); the Lancet and the New England Journal of Medicine (NEJM)) 8 were searched for randomised clinical trials, published between April 2016 and March 2017, and 117 papers involving a placebo medication treatment were extracted. The authors consider the journals included in the review to be of the highest impact and therefore setting the standard for following reporting guidelines. While the journals selected may not be an entirely representative sample, it would not be possible to include all journals that report randomised trials within the confines of this review. Characteristics used to describe the medication blinding were classified into the following categories: ‘matching placebo’ or similar phrase, appearance, packaging, labelling, smell, colour, taste, texture, contents of placebo, volume and none described.
At least two people were independently involved in the data extraction and checking process, with any discrepancies resolved through discussion with a third party. All data were recorded in an electronic spreadsheet (Microsoft Excel 2016).
Results
One hundred and seventeen articles regarding clinical trials involving a placebo medication treatment were published during the review period. Of these, 56 were from the NEJM, 35 were from the Lancet and 26 were from the JAMA.
Publications reporting characteristics of placebo treatments
Approximately three quarters of the articles published during the review period reported at least one characteristic of the placebo treatment, and therefore one quarter did not provide this level of detail (Table 1). The Lancet and the JAMA consistently had this information present within the articles they accepted (89% and 85%, respectively); however, articles published within the NEJM only included this in approximately half of their clinical trials involving a placebo medication treatment (57%).
Number and percentage of publications reporting characteristics of placebo treatments.
NEJM: New England Journal of Medicine; JAMA: Journal of the American Medical Association.
Reported characteristics of the placebo treatment
Within the 117 articles found during the review, there were 134 descriptions of characteristics for the placebo medication treatments, indicating that some articles included more than one descriptor. As shown in Table 2, the most common characteristic was the use of the phrase ‘matching placebo’ or similar (such as ‘identical placebo’) (38%), followed by contents of the placebo (17%), packaging (16%) and appearance (12%). Texture, taste and smell were the least reported characteristics (1%, 2% and 3%, respectively).
Number and percentage of reported characteristics of placebo treatments.
NEJM: New England Journal of Medicine; JAMA: Journal of the American Medical Association.
Articles published within JAMA most frequently reported on the contents of the placebo (31%) and appearance (23%), while articles within the Lancet focussed on descriptions of the packaging (29%), and articles with the NEJM described the contents of placebo and packaging most commonly (11% for each). The individual characteristics of the placebo medication treatment varied across the journals, although remained with similar proportions for labelling, colour, smell, taste and texture; the least frequently described features.
Publications reporting characteristics of placebo treatments in relation to commercial or non-commercial funding support
As shown in Table 3, 77 out of 116 articles were supported by commercial funding (66%). Across all journals, 48 (62%) articles reported at least one characteristic of the placebo treatment and 29 (38%) did not describe any. Most articles published within the Lancet and the JAMA that were supported by commercial funding reported at least one characteristic of the placebo treatment (85% and 75%, respectively), compared to only half of the articles within the NEJM (44%).
Number and percentage of publications reporting characteristics of placebo treatments in relation to commercial or non-commercial funding support.
aThe publication with an unknown funding source has not been included.
Of the 39 studies not supported by commercial funding (34%), most articles across all of the journals reported at least one characteristic of the placebo treatment (95%). Only one article did not declare its funding source, and in this instance no characteristics of the placebo treatment were reported.
Discussion
Main findings
Three quarters of the articles published during the review period reported at least one characteristic of the placebo treatment, suggesting that authors and/or journals recognise the benefits of including this level of detail within publications. However, with one quarter of articles not including this information, this indicates that there are still efforts to be made to ensure authors and journals are aware of the importance of including a detailed description of the placebo treatment, and that adequate blinding cannot be assumed. While articles within the Lancet and the JAMA consistently included at least one characteristic of the placebo treatment, only half of those in the NEJM did so. While the three journals with articles involved in this review all request that authors conform to the CONSORT guidelines, the Lancet specifically requests that all reports of randomised trials include a section entitled ‘Randomisation and masking’ within their Methods. The Lancet had the highest number of articles reporting at least one characteristic of the placebo treatment, and the journal guidelines described may contribute to this detailed level of reporting. Other journals may also benefit from revising their author guidelines to advise researchers to include this level of detail in their articles.
Use of ‘matching placebo’ or a similar phrase was most popular within the reviewed articles across the NEJM, the Lancet and the JAMA; however, with such encompassing terminology, it is difficult to understand if those involved in the research have considered all of the individual elements that must be examined to ensure successful blinding. Work by Desbiens 9 describes how researchers usually trust that the placebo obtained from drug companies is identical, and therefore successful blinding has been achieved. However, Boutron et al. 10 describe how a study investigating treatment with zinc for the common cold failed due to its distinctive taste. Researchers would therefore be wise to consider seeking confirmation from those involved in the manufacturing process, that the placebo characteristics match those of the active treatment, and therefore the investigational products are truly blinded. Within our review, descriptions involving the contents of a placebo, its packaging and appearance were most commonly used after ‘matching placebo’ or a similar phrase; however, their use was inconsistent across the journals reviewed, suggesting that author guidelines across journals do not consistently request this level of detail. Those features not involving appearance, such as smell, taste and texture appeared least within the reviewed articles, indicating that the matching of these to the active treatment is rarely recorded within published materials. This level of information provides absolute clarity if included in a publication and therefore may avoid the inadequate reporting of blinding, which can result in a negative perception of the integrity of the research findings.
Fergusson et al. 7 assessed the reporting of placebo controlled trials published in five top medical journals (including the BMJ, the JAMA, the Lancet, the Annals of Internal Medicine and the NEJM) and four top psychiatry journals. They found that 53% of the general medical journals reported matching characteristics of the placebo and intervention, with appearance and taste most commonly reported. Interestingly, one of the trials reported the dissimilarity between the placebo and intervention. In comparison, only 32% of the psychiatry journals reviewed reported the matching characteristics between intervention and placebo. Again, appearance was most often reported. Fergusson et al. 7 also highlight that several of the trials across both journal categories did not report the type of placebo used. In comparison, our review found that 73% of articles reported at least one placebo characteristic, with use of ‘matching placebo’ or a similar phrase being most prevalent (38%). While Fergusson et al. 7 found appearance and taste to be most commonly described, we found that these characteristics featured less than others (12% and 2%, respectively), such as the contents of the placebo (17%) and its packaging (16%). Our review has revealed inconsistencies that researchers should aim to minimise in order to improve reporting quality.
Our review included 117 articles, two-thirds of which were supported by commercial funding. Sixty-two percent of those with commercial funding reported at least one characteristic of the placebo treatment, in comparison to 95% of articles without commercial support. Within the context of this review, the findings suggest that non-commercial researchers appear to be more diligent in ensuring the characteristics of the placebo are included within their publications; however, it must be considered that there were fewer non-commercial articles published during the review period. The findings may be due to commercially funded studies utilising Interactive Response Technology systems and central dispensing to address their blinding requirements, compared to non-commercial studies which may rely on academics familiar with the written requirements necessary to record the methodological aspects of blinding and reporting.
Blinding is included in the ‘CONSORT 2010 checklist of information to include when reporting a randomised trial’ recommending that authors specify ‘If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how’ and ‘If relevant, description of the similarity of interventions’. The CONSORT statement suggests that such characteristics include the appearance, taste, smell and method of administration. 11 In a systematic review of 819 articles published in high impact-factor journals, Boutron et al. 10 found insufficient reporting of the methods of blinding and therefore suggested a need to increase the requirement related to blinding issues in the CONSORT statements. Journals and authors would benefit from ensuring closer compliance with the guidelines to enable reporting to be standardised. Previously, CONSORT guidelines included an additional level of detail, requesting that authors include ‘how the success of blinding (masking) was assessed’; however, this has since been removed due to a lack of empirical evidence and concerns regarding the validity of assessing this aspect. 12
Limitations
The review involved searching the main articles published by researchers and did not include the review of any supplementary materials, such as protocols or supplementary appendices, as these were not consistently available. It is possible that the detailed blinding descriptions were within these documents; however, it must be considered how the main publication is often the key source of the data for readers, and therefore the most commonly utilised.
The review did not consider all possible blinding techniques that researchers may employ, such as achieving blinding by using two separate teams (such as a blinded and unblinded nurse) for treatment administration and follow-up. We also did not evaluate the differences in the intervention type (e.g. tablet, liquid) or administration route (e.g. oral, intravenous) in relation to level of blinding detailed, and we anticipate there may be some differences both in relation to level of blinding achieved and reported.
Further research
Future research should focus on investigating the reasons for inadequate recording of blinding descriptions, with the aim of reducing the inconsistencies observed and improving reporting. Should reporting guidelines be amended in relation to blinding at any point, it would be beneficial to evaluate the impact of this.
Conclusions
The findings of our review highlight that efforts are being made to include descriptions of blinded medication within publications; however, there are inconsistencies in reporting which suggests that guidelines are not consistently being followed by the authors and journals involved in the review process. In the context of this review, it is not possible to determine the impact of adherence to the CONSORT guidelines. The inconsistencies observed suggest that more can be done to improve the reporting of medication blinding in randomised trials, particularly with frequent use of ‘matching placebo’ or a similar phrase, which does not provide definitive proof that all elements of blinding have been considered and successfully executed. Whilst the number of words allocated for journal articles is limited, authors should be keen to abide by the CONSORT guidelines and include a succinct description of blinding activity to avoid uncertainty in their methods and potential misinterpretation of research findings.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
