Abstract
Recognizing and understanding research bias is crucial for determining the utility of study results and an essential aspect of evidence-based decision-making in the health professions. Research proposals and manuscripts that do not provide satisfactory detail on the mechanisms employed to minimize bias are unlikely to be viewed favorably. But what are the rules for qualitative research studies? Whenever I am reviewing a thesis, manuscript, or research proposal involving qualitative research and I come across attempts to manage “bias,” it always gives me cause for concern. Here, I outline the reasons for my concern and reflect on whether the growing tendency of qualitative researchers trying to manage “bias” in their work is due to the increasing pressure to demonstrate research outputs lead to quantifiable impact.
What Constitutes Bias in Qualitative Research?
Bias—commonly understood to be any influence that provides a distortion in the results of a study (Polit & Beck, 2014)—is a term drawn from the quantitative research paradigm. Most (though perhaps not all) of us would recognize the concept as being incompatible with the philosophical underpinnings of qualitative inquiry (Thorne, Stephens, & Truant, 2016). Instead, qualitative researchers generally agree that considering concepts such as rigor and trustworthiness are more pertinent to the reflexive, subjective nature of qualitative research. A host of strategies for upholding these concepts during the qualitative research process have been developed and written about extensively, and engaging with this literature is a rite of passage for most doctoral students and novice researchers who are new to qualitative methodology. That Morse, Barrett, Mayan, Olson, and Spiers’s (2002) paper on verification strategies for establishing reliability and validity in qualitative research remains the most read and cited paper published in International Journal of Qualitative Methods is testament to this.
Yet I have found the issue of bias is raising its head with increasing regularity. Stories of research funding bodies and journal peer reviewers rejecting proposed qualitative methods or study findings due to “bias” are not uncommon. Usually, I find this relates to a perception by peer reviewers that the way data have/will be collected or analyzed is too closely aligned with the personal agenda of the researcher(s). Reflective of this, one of the most frequent questions I get asked when teaching graduate students about approaches to qualitative data analysis is whether directed or probing questions from an interviewer is evidence of bias, that is, that they are mining for data that will affirm their own preconceptions. I understand their confusion. In nursing, we teach the principles of evidence-based practice, aiming to give practitioners the knowledge and skills to use tools and checklists to critically appraise the trustworthiness and relevance of research evidence to inform their professional practice and decision-making. The most commonly used tool in this regard, the Critical Appraisal Skills Programme (CASP, 2017) qualitative checklist, makes specific reference to bias in Question 6, asking us to consider: if the researcher critically examined their own role, the rigor of qualitative research is particularly vulnerable when it lacks some of the devices that have been employed in quantitative research to ensure that what is produced is not just well-composed rhetoric of a well-meaning, but
Bias, Funding, and Impact
My sense is that the root of the matter is partly in the increasing prominence we place on being able to demonstrate the “impact” of our research. In the United Kingdom, the key driver of this is the research excellence framework, a research impact assessment for establishing reputational benchmarks for higher education institutions and determining what size slice of the £1 billion “block grant” funding pie they receive (quality-related research funding). Research, they say, is all about impact (Higher Education Funding Council for England, 2017). Here, the impact of research outputs is not solely evaluated using academic measures (e.g., number of citations) but on its “wider impact” beyond academia, such as on the economy, society, culture, public policy or services, health, or the environment.
The desire to ensure qualitative research is impactful is laudable and necessary. Another “red flag” for me is when the product of a qualitative study is claimed to be not transferable beyond the sample that was studied, but that is a topic for another editorial! Applying an academically rigorous approach is a key aspect of this as Morse, a nursing academic, reminds us poorly conducted qualitative research is “worthless, becomes fiction, and loses its utility” (Morse et al., 2002, p. 14). But I wonder whether the ever increasing pressure to demonstrate impact is leading some qualitative scholars to draw on what Thirsk and Clark describe as devices that have been employed in quantitative research to control for bias.
Although the recent #BMJnoQual debate demonstrates that enthusiastic skepticism still exists around the value and utility of qualitative research for informing health service delivery (see https://storify.com/shereebekker/bmjnoqual), it continues to make increasingly important contributions in the field. More applied qualitative health research than ever is being funded, often as an adjunct to quantitative studies, with the aim of better understanding factors that influence the implementation of interventions. This is a welcome trend. However, in many countries, stand-alone qualitative projects are seldom supported by health research funding bodies. The reason? I think it comes down to difficulty in being able to demonstrate measurable impact, usually in the form of quantifiable patient benefit.
Where Do We Go From Here?
Qualitative research is perhaps often viewed as being at the bottom of the hierarchy of evidence for informing (and thus having impact on) health policy and practice, a hierarchy predicated on level of bias. Seeing “bias” as a problem to be managed during the process and reporting of qualitative research may be a way of trying to establish a firmer footing on this hierarchy, but I have concerns that it may have the opposite effect and further weaken the standing of qualitative research as an impactful enterprise.
Thorne (2009) has written eloquently on the challenges and complexities of the evidence-based movement for understanding the potential contributions of qualitative research and offers some sage advice that can help us identify a way forward here. Principally, that our challenge is not to try and convince that qualitative work reflects objective, opinion-free neutrality. Rather, it is to better articulate the unique value that qualitatively derived knowledge can play within a system that measures impact through an evidence-based decision-making lens. Although it may be more difficult to quantify the impact of qualitative research, we should resist the temptation to reach for a positivist tape measure to solve this problem. To do so will lead us to become apologists for the subjectivity that is the very strength of interpretive work.