Abstract
Peer review, once considered a sacred cow, is running scared. Three damning incidents have raised more questions about the value of a process that determines how grants are awarded, research published, careers made, and riches earned. The outcomes of peer review sometimes achieve their intended purpose, although the peer-review process employed by the National Institute for Health and Clinical Excellence, for example, will be more directly beneficial or harmful than the peer-review process of any medical journal.
Journal editors might wish that their publications had great influence on the behaviour of clinicians and patients but they rarely do. And when they do, the effects are likely to be regrettable. A former editor of the BMJ would remind his editorial team that in his many years of editing he had never lost sleep over a single paper that the BMJ had rejected. It is the papers that editors publish that can cause problems, such as the ‘MMR’ paper published by The Lancet in 1998 and fully retracted last month following a General Medical Council (GMC) ruling against some of the researchers.
A retraction is an admission of a mistake, however unintentional. No contrition has been forthcoming from leading climate change researchers accused of running a scientific mafia to distort the debate in journals and beyond. Their agenda was to promote the ‘hockey stick’ graph of climate change, which purports to prove that after centuries of decline global temperatures have recently shot up to unprecedented levels. Supporters of the hockey stick analysis, also unflatteringly known as the hockey team, are said to have boycotted journals opposed to their ideas and favourably peer-reviewed research papers from fellow hockey team members.
Meanwhile, in a separate incident, the editor of an orthopaedic journal was not on a hockey team but played on the team of a major device manufacturer. He is accused of earning millions of dollars during his editorship, while regularly publishing papers either from the device manufacturer or about its products. With such incredible revelations, has peer review – the sacred cow of scientific publishing – ever had it so bad? Should journal editors be judged by the enormity of their mistakes or the rapidity of their contrition? Or can we hide behind the thin but protective cloak of peer review?
The Lancet undoubtedly published its MMR article in good faith, an attempt to ‘raise new ideas’. But those new ideas were amplified unreasonably by the media's support of the lead author, Andrew Wakefield, and his campaign against the vaccine. The result was a drop in vaccination rates and a rise in measles cases, despite mounting evidence against the suggestion of a link between the MMR vaccine and autism.
In 2004, The Lancet issued a partial retraction of the paper from 10 of the authors, limited to the interpretation of the main findings but maintained that the link between MMR and autism deserved further investigation. This year's GMC verdict finds that the research was flawed because of a lack of ethical approval, patients were not consecutively recruited as claimed in the paper, blood samples were collected at a child's birthday party, competing financial interests were not declared, and Wakefield, said the GMC, had shown a ‘callous disregard’ for children participating in his study. The investigation took two and a half years.
The Lancet's full retraction is welcome and will help restore confidence in the MMR vaccine, although critics of Wakefield and The Lancet will still argue that the paper should never have been published and – at the very least – fully retracted six years ago. Equally, critics of the triple vaccine will not be silenced by a mere verdict from a medical regulator. The forces against the MMR vaccine would have always created a controversy but would it have carried the same credence if the MMR paper had not been published in a journal as prestigious as The Lancet? Probably not but it is easy to be wise with hindsight – all editors make mistakes, especially when peer review is their closest ally.
Indeed, how can editors know when authors are being deceitful about the conduct of a study? Given the sheer number of research papers that major scientific journals receive each year and the amount of work required to investigate suspicions about even one research paper, little wonder that studies confirm that peer review is hopeless at detecting fraud and research misconduct. The research and publication process relies on a relationship of trust between funders, authors, reviewers, editors and readers. When editors take risks, as the editors of the most interesting scientific journals tend to do, that circle of trust is even more crucial. Most communications are taken at face value, deceit is easy. But, as Andrew Wakefield should have concluded and The Lancet knows, there is also a great deal to lose – not least your reputation.