Abstract
Dear Editor,
In his recent article, 1 Peter Gøtzsche summarises the history of citation bias research, starting with his own publication from 1987. 2 He remarks that our systematic review on citation bias had failed to include this publication (and also Schmidt and Gøtzsche from 2005). We discussed both these publications when working on our review, and we agree that they were important for the development of citation bias research. Still, we decided not to include them in our review.
Why is this the case? Only one of the articles4 did appear in the output of our Web of Science – Core Collection search; back then this database did not include any publications before 1988. But the Medline search and reference checking indeed identified both articles. Nevertheless, neither of them fitted our inclusion criteria.
Our review protocol 3 stated: ‘The study of citation bias is in fact about the association between study outcome and the number of citations, or citation count’ (p. 4). ‘In order to be included, a paper needs to contain primary data on the relation between study outcome and citation count’ (p. 7). Note the difference with Gøtzsche's approach: ‘(…). I noted whether the proportion of references with a positive outcome for the new drug was the same, lower, or higher than the proportion among all articles assumed to have been available to the authors.’ 2 In other words: whereas Gøtzsche looked at the proportions of (positive and negative) references in the reference lists of citing publications, we had limited our review to studies that looked at the actual number of citations that (positive and negative) publications accumulated over the years. Even though his approach seemed valid, we felt compelled to stick to our protocol and therefore excluded his work. Of note, these were the only two articles that we excluded because of this specific reason, so its impact on the conclusions of our systematic review remains limited.
Perhaps, in hindsight, we should have been less strict and called for a protocol deviation, especially as we wanted to include all the primary studies on citation bias. For sure, we realise now that it would have been appropriate to mention Gøtzsche's pioneering work in our introduction or discussion section. We thank Peter Gøtzsche for bringing this omission to our attention.
