Are We Conditioning EBM Researchers to be Innovative or Narrow?

Abstract

This short essay considers preferential publication and impact factor as stimuli, instrumentally conditioning medical researchers. The author postulates that publication houses emphasising publication of the highest levels of evidence (ie, meta-analyses) at the detriment of other levels of evidence, is inadvertently guiding researchers to overlook necessary research for more individualised care. The author recommends preferential publication and impact factor should be openly discussed by medical educators to ensure we are training researchers to conduct meaningful, high quality, innovative research.

Evidence-based medicine (EBM) emerged to increase certainty in clinical decision making; however, we are now plagued by a residual narrowness which we must acknowledge and address. EBM undoubtedly has foibles which we are still working on Timmermans and Mauck ¹ but there are a number of issues which have also manifested through this relatively new medical tradition. For example, no one could have anticipated that the hierarchy of evidence (Figure 1) would guide publication houses and researchers alike, to select and publish specific methods. These adaptations to EBM are not peripheral factors and cannot be overlooked, they are central to the development of modern medicine. Therefore, a systems-based intervention is required to manage preferential publication and the interpretation of impact factor (IF), if we want to avoid what I describe as publication gaming.

Figure 1.

An hierarchy of evidence

While we have seen a decline in publication bias, we are also witnessing a rise in preferential publication for the highest levels of evidence, for example, meta-analyses. Previous researchers have described a ‘preference’ to publish research conducted by associated board members. ² Although few have considered publishing house preferences as causal in meta-analytical research myopia. Undoubtedly, meta-analyses are very useful but without a critical eye or a qualitative understanding of lived experiences, they provide potentially unwarranted scaffolding for practitioner decision-making. Few publication houses balance their outputs, and knowledge clustering is increasingly based upon specific topics using increasingly sophisticated statistical methods. ³ This means, while publication houses may no longer be displaying traditional bias for findings which reject the null hypothesis, they continue to provide an imbalanced perspective by favouring certain research methods. Preferential publication, as described here, is stimulating researchers to design projects based on meta-analytical techniques rather than on questions which may require interdisciplinary or mixed-methods approaches.

Similarly, IF which emerged at around the same time as EBM ⁴ may be reinforcing project choices and designs. IF is actually not related to EBM, nor is it a measure of quality. IF is a measure of citation frequency and is generally calculated by dividing the number of citations by the total number of citable articles within a journal. Similar to the aforementioned problems with preferential publication, the inappropriate use of IF might be reducing the sophistication and strength embedded across (and within) a systematically sourced evidence base. But, there is a game afoot, because all high IF research is also based broadly upon novelty. Therefore, identifying unanswered questions within high impact journals increases the likelihood of publication therein, especially if one adopts a sophisticated meta-analytical technique. This means, IF and preferential publication are the stimuli and reinforcement in conditioning EBM researchers, but these may not be eliciting the most appropriate behaviours.

The conditioning process described may be creating a culture of publication gaming. This is where researchers design projects around IF and publication preferences rather than around meaningfulness or the demands of modern medicine, such as individualised care. Anyone who knows EBM can see that we have entered an age of mass production of redundant, perhaps misleading, and often conflicting meta-analyses. ⁵ Therefore, conditioning researchers with preferential publication and IF is not enabling clinicians to tailor interventions for their patients. Moreover, researchers maybe looking to increase the likelihood of publication in high impact journals before projects even begin. High impact publications are also often coupled with incentives such as career development and opportunities for increased funding. So, researchers may again be consciously narrowing the number of methodological options in their respective tool belts, knowing that case reports, qualitative studies and even systematic reviews are less generalisable and more difficult to publish.

We, as researchers, should not be trapped in this cycle, perpetually looking for the idiosyncratic questions and adopting statistical methods to be rewarded by publishing houses. The questions we look for should not be as methodologically narrow, as they perhaps are. Likewise, publishing houses should understand their purpose is not to methodically narrow their journal but rather to cluster evidence and provide more sophisticated, reliable knowledge bases for practitioners within specific fields. This narrowing of publications does not represent the future of medicine which is multifaceted and so publication houses should not be perpetuating this problem. If EBM is to once more emerge as a guiding light, we must ensure we are conducting overlapping, interdisciplinary research which is more necessary today than it ever has been.

No matter how elegant a study may be, there is always missing data and insights which might have been overlooked or are unattainable using statistical methods. Therefore, studies which link methodologies have to be the next logical step, but publishers must encourage this and take responsibility because the knowledge they impart is central to medical practice. Their actions inadvertently condition researchers which influences real-world decisions and provides knowledge to patients, relatives and friends in increasingly open, educated societies. Editors may not have considered the influence of their choices on individual researchers or research cultures but these choices constrict a researchers methodological options. So, while we become increasingly aware of the need for individualised care, we are inadvertently limiting our scope through essentially meaningless metrics and preferential publication.

We must remember, our efforts have real-world implications and the organisations where we work are inextricably linked. Publication houses are not on the periphery or unaccountable, having a public function they are central in terms of responsibilities and in driving innovation. If we do not learn to critically consider preferential publication, IF and what has been described here as publication gaming, we will not actualise individualised care. Adopting combinations and mixed-methods more accurately reflects illness and human existence therefore while journals may focus on one field, they have a duty to publish a plethora of research methods. This is not a call to cease meta-analytical studies, quite the opposite. But, we as researchers and medical educators, need to engage in debate about publishing houses as authorities, and their influence in the apparent myopic focus on meta-analytical studies.