Abstract
By prioritising certain kinds of evidence and devaluing or excluding other kinds of evidence, orthodox evidence-based evaluation practice perpetuates epistemic injustice, we argue. In contrast, by advocating a more inclusive approach to evidence, Evidential Pluralism can avoid these epistemic injustices. This motivates the adoption of Evidential Pluralism as a new methodology for evaluation. These arguments are supported by case studies concerning Covid-19 public face-mask mandates and global value chains for development. Consideration of epistemic injustice introduces a moral dimension to the debate concerning the epistemic virtues and limitations of the orthodox approach and Evidential Pluralism.
Keywords
Introduction
The aim of this paper is to demonstrate how Evidential Pluralism can mitigate epistemic injustices in evidence-based evaluation.
Epistemic injustice involves wronging someone ‘specifically in their capacity as a knower’ (Fricker 2007, 1). The concept of epistemic injustice has recently been used to criticise orthodox evidence-based evaluation practice. Orthodox evidence-based practice is characterised by evaluation methods and evidence hierarchies that focus almost exclusively on comparative studies, especially randomized controlled trials (RCTs). This is the dominant approach to evidence-based evaluation practice that has been widely adopted around the world, including by powerful and influential institutions. For example, the Cochrane Collaboration, an international organisation that focuses on evaluating healthcare interventions, and the Campbell Collaboration, an international organisation that focuses on evaluating economic and social policy interventions, endorse this approach. In devaluing or excluding certain kinds of evidence, it is argued, orthodox evidence-based practice commits epistemic injustice (Michaels 2021; Mormina 2022). In this paper, we build on this line of argument against orthodox evidence-based practice.
Firstly, we argue that the exclusionary nature of orthodox evidence-based practice is insufficient, on its own, to ground a charge of epistemic injustice. To substantiate a charge of epistemic injustice, the practices of exclusion and downgrading must be motivated by prejudice. We argue, however, that continued practices of exclusion and downgrading by proponents of orthodox evidence-based practice are indicative of prejudice. Despite longstanding criticism of its adequacy, the orthodox approach remains dominant and the shared concept of ‘good evidence’ that it has created, especially the stereotype of RCTs as the ‘gold standard’ of evidence, is often applied inflexibly and dogmatically. This is illustrated in the fourth section through the example of Covid-19 face-mask mandates. We argue that such practice reflects an underlying prejudice and therefore commits epistemic injustice.
We then introduce Evidential Pluralism as an alternative approach to evaluation and argue that it avoids epistemic injustices committed by orthodox evidence-based practice. By advocating a more inclusive approach to evidence and focusing on objects of evaluation rather than hierarchies of evidence, Evidential Pluralism avoids prejudicially dismissing or downgrading evidence. Evidential Pluralism therefore avoids epistemic injustices committed by orthodox evidence-based practice. This is supported through consideration of a case study concerning global value chains for development.
Epistemic injustice is a moral wrong. Although orthodox evidence-based practice has previously been criticised on epistemic grounds, the charge of epistemic injustice introduces a moral objection to orthodox evidence-based practice. Similarly, Evidential Pluralism has previously been defended on purely epistemic grounds. Our argument that Evidential Pluralism is able to avoid the epistemic injustices committed by the orthodox approach introduces an additional moral dimension to the case for Evidential Pluralism.
We begin, in the second section, by introducing the concept of epistemic injustice. In the third section we highlight the exclusionary nature of orthodox evidence-based practice and emphasise the difficulty in grounding a charge of prejudice, and thereby epistemic injustice. In the fourth section we substantiate a charge of epistemic injustice by arguing that continued practices of exclusion and downgrading reflect a dogmatic, closed-minded, and insensitive attitude that constitutes prejudice. In the fifth section, we introduce Evidential Pluralism and argue that it can mitigate epistemic injustices that result from orthodox evidence-based practice. In the sixth section, we provide a case study to illustrate Evidential Pluralism’s more inclusive approach and how it avoids epistemic injustice in evaluation.
Epistemic Injustice
Epistemic Injustice, according to Fricker, is a ‘distinctively epistemic kind of injustice’ that involves wronging someone ‘specifically in their capacity as a knower’ (Fricker 2007, 1). In elucidating the concept of epistemic injustice, Fricker distinguishes two distinct kinds: testimonial injustice and hermeneutical injustice (Fricker 2007). 1
Testimonial injustice involves attributing too little credibility to a speaker on the basis of prejudice.
2
Importantly, rejections of someone’s testimony are not necessarily unjust. We can have good reason to reject, or at least refrain from fully accepting, someone’s testimony. Furthermore, even when a person’s testimony is mistakenly given less credibility than it deserves, this does not necessarily constitute epistemic injustice. As Fricker makes clear, it is important to distinguish between cases of credibility deficit that result from ethically innocent mistakes and those that are motivated by prejudice. Only the latter, according to Fricker, constitute epistemic injustice.
3
It seems that the ethical poison of testimonial injustice must derive from some ethical poison in the judgement of the hearer, and there is none such wherever the hearer’s error is ethically non-culpable […] the ethical poison in question is that of prejudice. (Fricker 2007, 22; italics in the original)
Testimonial injustice thus occurs when someone’s testimony is rejected, or not given the weight it deserves, on the basis of prejudice.
According to Fricker, stereotypes are a key mechanism by which prejudice corrupts credibility judgements. Stereotypes, according to Fricker, play a crucial role in our ability to make spontaneous credibility judgements. In accepting medical advice from a doctor, for example, we often rely on the stereotype of doctors being dependable medical experts (Fricker 2007, 32). However, when a stereotype embodies prejudice, this can lead to unjust credibility deficits.
Fricker understands prejudices as judgements that exhibit a ‘resistance to counter-evidence owing to some affective investment on the part of the subject’ (Fricker 2007, 35). Prejudices lead to unjust credibility deficits by distorting ‘the hearer’s perception of the speaker’ (Fricker 2007, 36).
Prejudices that relate to social identities, such as race, gender, or socio-economic status, track people through different dimensions of social activity and make them susceptible to a range of different social injustices. When such a social identity prejudice results in an unjust credibility deficit, the testimonial injustice is ‘systematic’ in the sense that it is systematically connected to other kinds of social injustice (Fricker 2007, 27, 35). Systematic testimonial injustice constitutes Fricker’s ‘central case’ of epistemic injustice (Fricker 2007, 27).
Although discussions of epistemic injustice typically focus on the systematic kind resulting from negative social identity prejudices, epistemic injustice can result from other kinds of prejudice. For example, Fricker asks us to imagine: a panel of referees on a science journal who have a dogmatic prejudice against a certain research method. It might reasonably be complained by a would-be contributor that authors who present hypotheses on the basis of the disfavoured method receive a prejudicially reduced level of credibility from the panel. Thus the prejudice is such as to generate a genuine testimonial injustice. (Fricker 2007, 27)
When an unjust credibility deficit is the result of a prejudice that is not a social identity prejudice, such as a prejudice against a scientific method, the testimonial injustice is ‘incidental’ in the sense that the underlying prejudice does not make the person susceptible to other kinds of injustice and the impact is, therefore, localised (Fricker 2007, 27, 35). Incidental testimonial injustice nevertheless constitutes an inherent wrong and it can have detrimental consequences for the victim. As Fricker emphasizes, To categorize a testimonial injustice as incidental is not to belittle it ethically. Localized prejudices and the injustices they produce may be utterly disastrous for the subject, especially if they are repeated frequently so that the injustice is persistent. (2007, 40)
Prejudice against a scientific method, for example, could result in a person repeatedly receiving less credibility than they deserve in respect of their scientific research, which in turn could negatively affect their chances of securing publications or funding, which in turn negatively impacts their career.
Testimonial injustice, whether systematic or incidental, wrongs a speaker in their capacity as a knower. As a result of prejudice, their standing as a knower, and their ability to communicate their knowledge, is negatively impacted.
Hermeneutical injustice involves a lack of conceptual resources to make sense of and communicate one’s experiences. Fricker gives the example of sexual harassment (Fricker 2007, 6). Prior to the development of the concept of sexual harassment, women who suffered sexual harassment lacked the conceptual resources to properly understand their experience and to intelligibly communicate their experience to others. For a lack of conceptual resources to constitute hermeneutical injustice, it must result from the prejudicial marginalisation of a social group from generating social meanings. In the case of sexual harassment, for example, the gap in conceptual resources can be understood as resulting from the prejudicial exclusion, or at least unequal participation, of women in the practices through which social meanings are generated.
Testimonial injustice and hermeneutical injustice are ‘discriminatory’; they involve prejudicially downgrading or marginalising epistemic agents, which in turn limits their ability to communicate their knowledge and to make sense of their experience (Fricker 2017, 53). Importantly, both testimonial injustice and hermeneutical injustice are unwitting, or unintentional. In the case of testimonial injustice, prejudice distorts the hearer’s judgement of the speaker’s credibility. This is distinct from cases in which a person deliberately misrepresents another’s testimony as false or irrational. While such deliberate misrepresentation is wrong and is intended to cause others to doubt or downgrade the epistemic status of the speaker, it does not involve the perpetrator misjudging the epistemic status of the speaker. Similarly, cases of hermeneutical injustice are distinct from cases in which a hearer deliberately fails to understand a speaker or pretends not to understand a speaker. In cases of hermeneutical injustice, a hearer may be sincerely trying to understand but be unable to do so due to a lack of shared concepts, which itself is the result of prejudicial marginalisation. Although unintentional, the discriminatory nature of epistemic injustice means that it constitutes a significant moral wrong (Fricker 2007, 2017). 4
The concept of epistemic injustice has recently been applied to orthodox evidence-based practice. In devaluing or excluding certain kinds of evidence, it is argued, orthodox evidence-based practice commits epistemic injustice (Michaels 2021; Mormina 2022). 5 In this paper, we develop this line of argument. Importantly, we argue that a claim of epistemic injustice can be substantiated by showing that continued practices of devaluing and excluding evidence result from prejudice against certain types of scientific methodology, namely non-comparative study designs. 6 This leads to incidental testimonial injustice and hermeneutical injustice.
The Exclusionary Nature of Orthodox Evidence-Based Practice and the Importance of Prejudice
Orthodox evidence-based practice is here being used to refer to evaluation methods and evidence hierarchies that focus on comparative studies, especially randomised controlled trials (RCTs), and systematic reviews and meta-analyses of such studies.
Orthodox evidence-based practice began in the 1990s with the evidence-based medicine movement (Sackett et al. 1996). This movement aimed to provide rigorous, systematic evaluation to ensure the safety and efficacy of medical interventions. The approach has since been widely adopted, including in public policy, to improve decision-making and outcomes. In prioritising certain kinds of evidence and downgrading or excluding other kinds of evidence, orthodox evidence-based practice devalues or excludes certain voices. We shall argue in this section, however, that this devaluing or exclusion is insufficient on its own to substantiate a charge of epistemic injustice.
Orthodox evidence-based practice includes the following two commitments: (i) that RCTs are, typically, the gold standard of evidence, and (ii) that causal claims ought to be evaluated solely on the basis of comparative studies—studies that measure a putative cause and effect to assess whether the cause is associated with a difference in the effect. This is most clearly captured in the evidence hierarchies of evidence-based medicine, which place systematic reviews and meta-analyses of RCTs at the top of the hierarchy, followed by individual RCTs and then observational studies (SUNY 2004 [2026]). All other kinds of studies are relegated to a lower position on the hierarchy and, in practice, excluded from consideration. Some orthodox approaches to evidence-based medicine allow for a little more flexibility. For example, the popular GRADE approach allows, in some circumstances, for high-quality observational studies to be considered as superior to very low quality RCTs (Guyatt et al. 2002 [2015]). However, the basic idea that RCTs are the gold standard of evidence and that causal claims ought to be evaluated solely on the basis of comparative studies is the same. In evidence-based policy, association studies, especially RCTs, are similarly prioritised. For example, the UK Government’s Magenta Book, which provides evaluation guidance for all Government policy and regulatory interventions, prioritises the use of experimental or quasi-experimental studies and advocates the use of other kinds of study only when these are not available. 7
In prioritising comparative studies and devaluing or excluding other kinds of evidence, orthodox evidence-based practice devalues or excludes the voices of the researchers who carry out these other studies, as well as the voices of the participants in these studies. Furthermore, RCTs, the preferred methodology of the orthodox approach, themselves typically exclude certain groups, such as the elderly or those with co-morbidities. Also, in their focus on average treatment effects, RCTs can fail to take account of first-hand experience and perspectives of participants, despite participants being in a privileged position to know how an intervention has affected them (Khosrowi and Reiss 2019). 8 A failure to take such insights into account can result in adverse effects going undetected as well as complacency in the findings of RCTs that yield positive results (Khosrowi and Reiss 2019). Although inclusion of participant and public values and experience is increasingly recognised as important (Frith 2023), they are still rarely incorporated (Dusin et al. 2023) and, even when they are, they are typically not considered to provide ‘good evidence’ (Greenhalgh et al. 2024). 9
It is indisputable that orthodox evidence-based practice devalues or excludes certain kinds of evidence and thereby certain voices. Such devaluing and exclusion have recently been connected to epistemic injustice. Michaels (2021) discusses how the orthodox focus on RCTs excludes people from contributing to the generation of evidence, including those systematically excluded from being participants in RCTs, as well as researchers who provide non-RCT evidence. Michaels notes that the privileging of RCTs can have a ‘self-perpetuating’ effect, where RCTs become a measure not only of scientific rigour but also of the status of the researcher, leading to editorial boards, research funding boards, and priority-setting committees being dominated by researchers with a vested interest in RCT research. To be denied credibility as a knower and excluded from contributing to the generation of evidence and knowledge is, Michaels claims, ‘a clear example of testimonial injustice’ (2021, 418).
Although we agree that the exclusionary practices of the orthodox approach can result in epistemic injustice, we think more needs to be done to substantiate a charge of epistemic injustice. As explained in the second section, Fricker makes it clear that it is important to distinguish between cases of credibility deficit that result from ethically innocent mistakes and those that are motivated by prejudice. Only those motivated by prejudice, according to Fricker, constitute epistemic injustice. To substantiate a claim that orthodox evidence-based practice commits epistemic injustice in devaluing or excluding certain kinds of evidence, it is thus necessary to connect such devaluing and exclusion to prejudice.
The importance of prejudice to substantiating a charge of epistemic injustice is recognised by Mormina (2022), who argues that ‘in group favouritism’ can lead to orthodox ‘gold standard’ evidence being privileged and other kinds of evidence being prejudicially dismissed. In considering policy responses to Covid-19, Mormina argues that expert disagreement alone cannot explain the reluctance to introduce precautionary measures, such as face-mask mandates, airport testing, and bans on mass gatherings, when there was at least some non-gold standard evidence of their effectiveness. Instead, such reluctance is explained by in group favouritism that privileged the knowledge and evidence of likeminded proponents of the orthodox approach. As a result of this in-group favouritism, government decision-makers committed testimonial injustice against those pursuing less rigid approaches to evidence (Mormina 2022).
While we do not deny that in-group favouritism might be an affective motivation underlying at least some instances of evidence being prejudicially excluded or devalued, we think more needs to be done to substantiate a charge of prejudice. The exclusion and devaluing are based on the kind of evidence being offered and proponents of the orthodox approach defend such exclusion and devaluing by appealing to the methodological standards of their approach, which in turn can be defended on epistemic and moral grounds. It is therefore not obvious that the orthodox practice of excluding and devaluing evidence can be attributed to prejudice.
The prioritisation of RCTs in orthodox evidence-based practice is grounded on epistemic values, including objectivity, precision, the reduction of bias, and the ability to make causal inferences (Khosrowi 2019; Worrall 2007). In defence of the prioritisation of comparative studies, proponents of orthodox evidence-based practice argue that science should conduct strictly controlled studies to produce evidence of the efficacy and effectiveness of interventions that is untainted by moral, social, cultural, or political values (Catwell and Sheikh 2009). Furthermore, the exclusion of some individuals as participants in RCTs is motivated by ethical concerns to protect vulnerable individuals from harm. Defenders of orthodox evidence-based practice can thus attempt to resist a charge of prejudice, and thereby a charge of epistemic injustice, by appealing to these reasonable epistemic and moral reasons to justify their practices of devaluing and excluding. We can, of course, disagree that these epistemic and moral reasons are sufficient to justify the prioritisation of certain kinds of evidence and the resultant practices of downgrading and exclusion. We can also provide good reasons for adopting a more inclusive approach to evidence. However, as long as we accept that there are reasonable epistemic and moral grounds underlying the orthodox approach, then a claim of prejudice, and thereby epistemic injustice, is not warranted.
Substantiating a Charge of Epistemic Injustice
To substantiate a charge of epistemic injustice, it is necessary to show that practices of downgrading and exclusion by proponents of orthodox evidence-based practice stem from prejudice. Recall that, following Fricker, prejudice is here being understood as a judgement that is resistant to counterevidence due to some underlying affective motivation. In this section, we argue that the orthodoxy’s continued practices of devaluing or excluding evidence can indeed be attributed to prejudice because: (i) There is long-standing evidence of the limitations of the orthodox approach. (ii) Proponents of the orthodox approach are unreasonably dogmatic and inflexible in their adherence to their approach, reflecting insufficient sensitivity to this evidence.
Let us begin with condition (i). RCTs are designed to answer simple, focused questions in a stable context (Greenhalgh 2020; Greenhalgh et al. 2022; Greenhalgh and Papoutsi 2018). Well-designed RCTs can provide very good evidence of, for example, the efficacy of drugs and vaccines. Given its origins in the context of evidence-based medicine, the primacy of RCTs in orthodox evidence-based practice is understandable. In reality, however, many actual RCTs suffer from flaws related to, for example, bias, sample size, representation, or blinding (Greenhalgh et al. 2022). When we turn to complex interventions, the limitations of RCTs are even more significant. Not only are there increased difficulties with randomising and blinding, but controlled study environments typically fail to capture all the important features of the real-world environment. Similarly, with complex interventions, it can be extremely difficult to eliminate bias and confounding in observational studies. These limitations reduce the internal validity and external validity of results and, in turn, the usefulness of such studies for decision-makers. Appreciation of this complexity and the associated need to extend the evidence base to include more than comparative studies have long been the basis of critiques of orthodox evidence-based practice (Bronzini and Polini 2025; Cartwright and Hardie 2012; Greenhalgh 2020; Greenhalgh et al. 2022; Greenhalgh and Papoutsi 2018; Grossman and Mackenzie 2005; Head 2010; Plsek and Greenhalgh 2001). Although proponents of orthodox evidence-based practice sometimes acknowledge the limitations of RCTs and the need for flexibility in the application of their approach, they rarely honour this in practice (Greenhalgh 2020; Greenhalgh et al. 2022). Instead, as we argue in condition (ii), they stick dogmatically and inflexibly to their approach of prioritising comparative studies, especially RCTs, and excluding other kinds of evidence.
Now let us now consider condition (ii). Despite longstanding criticism of the adequacy of orthodox evidence-based practice, especially in relation to complex issues, proponents of orthodox evidence-based practice adhere dogmatically and inflexibly to their approach.
To see this, consider the example of Covid-19 public face-mask mandates. During the pandemic, robust evidence from RCTs was sought to ensure effective intervention measures. RCTs played a crucial role in the development of vaccines. The narrow focus on RCTs, however, resulted in uncertainty and controversy surrounding other public health measures, including public face-mask mandates.
At the start of the pandemic, observational data indicated that countries that introduced public masking very early in the pandemic had dramatically lower mortality rates compared to countries that did not introduce public masking mandates (Leffler et al. 2020). An influential RCT, however, found that being advised to wear a face mask outside of the home made no statistically significant difference to SARS-CoV-2 infection rates (Bundgaard et al. 2021). This finding was hailed by advocates of orthodox evidence-based practice as demonstrating that face masks do not work. ‘[N]ow that we have properly rigorous scientific research we can rely on, the evidence shows that wearing masks in the community does not significantly reduce the rates of infection’ (Heneghan and Jefferson 2020). The study, however, had numerous design flaws, including focusing on whether masks protect the wearer rather than whether they reduce community transmission (Greenhalgh et al. 2022), which is the public-health outcome of interest.
Similarly, a highly influential Cochrane Review that included only RCTs found the evidence to be inconclusive in relation to the effectiveness of face masks (Jefferson et al. 2006 [2023]). As the authors of the review acknowledge, firm conclusions could not be drawn due to various design flaws of the RCTs included. Nevertheless, many mistakenly interpreted the review to mean ‘mask mandates did nothing’ (Stephens 2023). Cochrane’s editor in chief released a statement to clarify that, in Cochrane’s view, the review does not support such claims (Soares-Weiser 2023). However, a continued commitment to RCT evidence is reflected in the review’s conclusion that ‘there is a need for large, well-designed RCTs addressing the effectiveness of [….] these interventions in multiple settings and populations, as well as the impact of adherence on effectiveness’ (Jefferson et al. 2006 [2023], 3).
In addition to exaggerated claims based on inadequate and inconclusive RCT evidence and repeated calls for further RCT evidence, there was also a dismissal of non-RCT evidence. For example, an interdisciplinary review that included a range of non-RCT evidence found widespread use of basic face masks, including homemade cloth masks, can contribute to reducing onward transmission by asymptomatic and pre-symptomatic wearers (DELVE Initiative 2020). The review was criticised by proponents of the orthodox approach for being ‘a non-systematic review of anecdotal and non-clinical studies’ and for recommending public face-mask policies in the absence of RCT evidence (Science and Media Centre 2020). 10
This fixation on RCT evidence and the dismissal of other kinds of evidence resulted in delays in the implementation of crucial public health measures, including public face-mask mandates. This, in turn, likely had significant detrimental effects. As Greenhalgh et al. put it: Thousands of lives were likely lost as a result of what was incorrectly claimed to be an “evidence-based” approach—dismissing or downgrading mechanistic evidence, overvaluing findings from poorly designed or irrelevant RCTs, and advocating for inaction where RCT evidence was lacking. (Greenhalgh et al. 2022, 253).
The Covid-19 face-mask mandate example provides a clear illustration of a dogmatic adherence to orthodox evidence-based practice. The overstated claims regarding the conclusions that can be drawn from RCT evidence, the repeated calls for further RCT evidence and the dismissal of non-RCT evidence indicate a dogmatic and closed-minded commitment to orthodox evidence-based practice that is insufficiently sensitive to the limitations of RCTs and to the merits of and need for other kinds of evidence. Ironically, these considerations reflect an insensitivity to the evidence—evidence that orthodox evidence-based practice is sub-optimal, at least in the context of complex interventions. Reasonable sensitivity to the longstanding criticism of the adequacy of the orthodox approach and open mindedness to the need for and merits of other kinds of evidence would have led to a less dogmatic fixation on RCT evidence. 11
Thus, the extent to which comparative studies, especially RCTs, were prioritised and other kinds of studies were dismissed goes beyond reasonable scientific disagreement about what constitutes good evidence and is instead indicative of prejudice. This prejudice may stem from a sense of loyalty to the status quo, from feeling threatened by intellectual innovation, from finding the task of changing the status quo too daunting, or Mormina’s (2022) in-group favouritism. Whatever the underlying motivation, such dogmatic adherence to the orthodox approach reflects unreasonable insensitivity to the evidence and is therefore indicative of prejudice.
Identifying underlying prejudice substantiates a charge of epistemic injustice. When evidence is prejudicially downgraded or excluded, testimonial injustice is committed against the researchers providing the evidence and any participants in the studies. In prejudicially devaluing or dismissing evidence, proponents of orthodox evidence-based practice prejudicially attribute too little credibility to both researchers and participants. As a result, the ability of researchers and participants to communicate their knowledge is limited and they are wronged in their capacity as knowers.
Since the underlying prejudice is a prejudice against certain kinds of evidence, the testimonial injustice is incidental. The fact that the injustice is localised does not, however, mean that it is insignificant. As explained in the second section, to be prejudicially downgraded as an epistemic agent is a significant moral wrong. Furthermore, given the significant influence of orthodox evidence-based practice, the testimonial injustice is likely to be persistent in the sense that researchers or participants are repeatedly subject to such testimonial injustices. This not only means that they are persistently wronged in their capacity as a knower, but the injustice is also likely to have detrimental practical effects, such as difficulty publishing work or securing funding, which in turn impacts their career opportunities.
In addition to committing testimonial injustice, orthodox evidence-based practice also results in hermeneutical injustice. The principles and standards of orthodox evidence-based practice have created a shared concept or stereotype of evidence, or ‘good evidence,’ that is highly influential. Given the dominance and influence of orthodox evidence-based practice, its proponents occupy a privileged position of power from which they can perpetuate the idea that the only causal evidence worthy of being considered is comprised of comparative studies, especially RCTs. In limiting the concept of good evidence, orthodox evidence-based practice prevents or limits the ability to make sense of and understand other kinds of studies as good evidence, at least in relation to the question of whether an intervention works. 12 This limits the ability of researchers conducting other kinds of studies from communicating their knowledge because their research is not recognised or acknowledged by others as providing good evidence. It also limits the researchers’ ability to recognise their own research as providing good evidence: for example, qualitative researchers often do not see their research as providing good evidence of the effectiveness of an intervention, instead preferring to focus on its role in helping to interpret or understand social activity (Alasuutari 2010). Even where authors of such studies do understand their research as providing good evidence, to maintain such a perspective in the face of persistent refusals from others to acknowledge their research as good evidence takes a significant degree of resilience. Similarly, participants in these kinds of studies are limited in their ability to understand their experiences as providing good evidence and to have others understand their experiences as providing good evidence.
There is, therefore, a mutually re-enforcing relationship between this hermeneutical injustice and instances of testimonial injustice (Fricker 2007, 159–60). The influential stereotype of ‘good evidence’ created and perpetuated by orthodox evidence-based practice prevents studies other than comparative studies from being understood as good evidence of whether an intervention works. This, in turn, leads to the credibility of researchers and participants in other studies being unduly deflated. As a result of this deflated credibility, the researchers and participants are marginalised and prevented from participating equally in the practices by which the shared meaning of evidence is generated and developed. Hence, they continue to be subject to persistent testimonial injustice.
It is crucial to address the epistemic injustices that result from a dogmatic commitment to orthodox evidence-based practice. Epistemic injustice is a significant moral wrong that, as a dysfunction in the epistemic system, can have significant practical consequences. As the Covid-19 face-mask example illustrates, it can lead to inadequate policy responses that cost lives.
As we argue in the following section, the more inclusive approach to evidence advocated by Evidential Pluralism can mitigate the epistemic injustices that result from a dogmatic and inflexible commitment to the orthodox approach.
Evidential Pluralism
In this section, we introduce Evidential Pluralism as an alternative to orthodox evidence-based practice. 13 We argue that, in advocating a more inclusive approach to evidence, Evidential Pluralism mitigates the epistemic injustices committed by the orthodox approach.
Evidential Pluralism is a philosophical account of causal enquiry. According to Evidential Pluralism, establishing a causal claim typically requires establishing the existence of both a correlation and a mechanism. This is captured in the following principle (Shan and Williamson 2023): Object Pluralism: To establish a causal claim, it is normally necessary to establish (i) that the putative cause and effect are appropriately correlated, and (ii) that there is some mechanism complex connecting the putative cause and effect and which can account for the extent of the correlation.
Object pluralism leads to a second principle: Study pluralism: Evaluating a causal claim normally requires assessing relevant association studies and mechanistic studies, where available.
The evidential relationships posited by Evidential Pluralism are depicted in Figure 1. The evidential relationships posited by Evidential Pluralism (Williamson 2021).
Comparative studies carried out on the population of interest are classified as ‘association studies’ by Evidential Pluralism. Association studies test primarily for the presence of a correlation (channel α1 of Figure 1). Establishing a correlation between A and B normally involves carrying out studies that repeatedly measure instances of A and B to determine whether A and B are probabilistically dependent, conditional on potential confounders. Experimental studies, including RCTs, are particularly well suited to this, although observational studies can also be helpful. Experimental studies such as RCTs can also provide indirect evidence that the correlation is attributable to some underlying causal mechanism, rather than confounding, say (channel α2 of Figure 1).
A more direct investigation of the mechanisms in question begins by hypothesizing specific features of the mechanism complex linking A and B (channel μ2 of Figure 1). Such features may include mediating variables, entities, or activities involved in a mechanism, or ways in which the mechanism is organised, such as its spatial organisation. Evidential Pluralism classifies studies that test for the presence of these features as ‘mechanistic studies’ (Channel μ1 of Figure 1).
The requirement to assess mechanistic studies alongside association studies motivates a more inclusive approach to evidence, when compared with orthodox evidence-based evaluation. Although Evidential Pluralism acknowledges that experimental and observational studies are well suited to providing evidence of correlation, it does not preclude the possibility of other types of evidence contributing to establishing correlation (via channel μ3 of Figure 1). Furthermore, according to Evidential Pluralism, any kind of study that assesses the hypothesized features of the mechanism complex counts as a mechanistic study.
Unlike orthodox evidence-based evaluation, Evidential Pluralism does not commit to rigid rankings of studies based on the methods employed by those studies. Instead, it focuses on the objects of enquiry, namely correlation and mechanisms, and takes each item of evidence on its own merits.
The principles of Evidential Pluralism have enabled the development of a new approach to evidence evaluation, including in medicine (Parkkinen et al. 2018), 14 social science and public policy (Shan and Williamson 2023) and law (Trofimov and Williamson 2025).
The requirement to assess evidence of mechanisms alongside evidence of correlation not only enables more robust evaluations but can also mitigate the epistemic injustice committed as a result of a dogmatic commitment to orthodox evidence-based practice.
By not committing to rigid rankings of studies based on the methods employed by those studies, Evidential Pluralism avoids creating a simple stereotype of ‘good evidence’ and thereby avoids certain types of evidence being dismissed or devalued as a result of prejudice against specific study designs.
Furthermore, requiring evidence of mechanisms alongside evidence of correlation will typically require a broad range of evidence types to be considered. As we shall see in the next section, this helps to ensure that different experiences, expertise, and knowledge can be included in an evaluation.
It is important to note that although Evidential Pluralism motivates a more inclusive approach to evidence, it does not advocate an ‘anything goes’ approach to evidence. Firstly, only evidence that is relevant to establishing either correlation or mechanisms will be considered in an Evidential Pluralism evaluation. Secondly, Evidential Pluralism does not deny that studies of all kinds, including RCTs, can be of either good or poor quality. The quality of individual studies will affect their evidential strength in an Evidential Pluralism evaluation. However, Evidential Pluralism does not preclude certain kinds of studies from providing high quality evidence simply in virtue of their study design or method.
Evidential Pluralism offers a framework for systematically including a broad range of evidence and thereby avoids the epistemic injustice committed by orthodox evidence-based evaluation. The more inclusive approach to evidence advocated by Evidential Pluralism ensures that different expertise, experiences, and knowledge are included in an evaluation. By focusing on correlation and mechanisms as objects of evaluation, Evidential Pluralism avoids prejudicially dismissing or devaluing certain types of evidence. The fact that Evidential Pluralism provides an alternative, more inclusive approach to evidence-based evaluation that avoids epistemic injustice supports the adoption of Evidential Pluralism as a new methodology for evidence-based evaluation.
Case Study: Global Value Chains for Development
In this section, we provide a case study to illustrate the more inclusive approach of Evidential Pluralism. The example demonstrates how Evidential Pluralism values different kinds of experience, expertise, and knowledge and thereby avoids epistemic injustice in evidence evaluation. We show how Evidential Pluralism can incorporate qualitative evidence that provides crucial insights into the impact of global value chains and thereby avoids the epistemic injustice that would result from a prejudicial refusal to take account of such evidence.
As we shall explain, international economic institutions promote participation in global value chains (GVCs) as key to development. The link between participation in GVCs and development is supported by quantitative data which shows that employment and real wage levels increase when firms integrate into GVCs, especially in developing countries. This reliance on quantitative data to support an association aligns with orthodox evidence-based approaches. However, taking account of a broader evidence base, including insights from those who are supposed to be benefitting from employment and wage increases, reveals shortcomings in the claimed association between participation in GVCs and development. The failure to take account of such evidence risks making unsubstantiated claims and, we shall argue, a prejudicial refusal to take account of such evidence commits epistemic injustice.
Global economic institutions, such as the World Bank and the World Trade Organisation, advocate participation in GVCs to achieve development (World Bank 2017, 2020; World Trade Organization 2019). GVCs involve the international exchange of parts, components, and tasks. GVCs contrast with classical international trade, which involves the international exchange of complete products and goods manufactured within national borders. GVCs account for an increasing share of international trade and offer developing countries new opportunities to integrate into the global economy.
Global economic institutions claim that participation in global value chains can positively contribute to development by increasing growth, employment, and real wages, and thereby reduce poverty and improve people’s material and social conditions.
To enable their firms to integrate into GVCs, developing countries are urged to negotiate and adopt deep preferential trade agreements that include WTO Plus and Extra Provisions. These go beyond the tariff reductions included in traditional trade agreements and include provisions such as protection of intellectual property rights and other intangibles, the liberalisation of services and investment and the free movement of capital (Alessandrini 2022). Adoption of these provisions, it is claimed, will provide the regulatory environment necessary to secure investor confidence and to ensure harmonious and efficient cross-border trade (Ruta 2017). It is further argued that these regulatory provisions will provide the resources for labour, social and environmental protections.
The claim that adoption of WTO Plus and WTO extra provisions will facilitate integration into GVCs is supported, primarily, by theoretical arguments. 15 However, there are also quantitative studies that indicate that the adoption of such regulatory provisions is positively correlated with integration into GVCs (Ruta 2017). The claim that participation in GVCs facilitates development is similarly supported by quantitative data that find a correlation between integration into GVCs and increases in employment and real wage levels, particularly in developing countries (World Bank 2017; World Trade Organization 2019).
As highlighted by Alessandrini (2022), these quantitative data are problematic. Firstly, alternative quantitative analyses reveal a weaker link between integration into GVCs and development. A comprehensive quantitative analysis by Bernhardt and Pollak (2016), for example, found that participation in GVCs resulted in economic upgrading in just over a quarter of the countries in their sample and that a combination of economic and social upgrading is ‘rather scarce’ (Bernhardt and Pollak 2016, 1238–39).
Secondly, consideration of qualitative evidence further challenges the claimed link between participation in GVCs and development. Case study analyses involving in-depth exploration of the real-world context and lived experience through, for example, interviews and direct observations reveal that increases in employment and real wages can be accompanied by, for example, deteriorating working conditions, increased precarity and reductions of non-wage benefits and over-time pay (Mezzadri 2017; Selwyn 2019). 16 Such factors counteract the ability of increased employment and higher real wages to reduce poverty and improve working and living conditions.
The insights from qualitative case study analyses reveal the limitations of employment and real wage levels as indicators of development (Alessandrini 2022; Bernhardt and Pollak 2016). Not only do these indicators reveal little about the quality of working and living conditions, they also do not capture informal workers, who account for a significant portion of the labour force in developing countries.
The reliance on quantitative data analysis conforms to the orthodox evidence-based approach of privileging observational studies and devaluing or excluding more qualitative case study analyses. 17 Although the reports from the World Bank and World Trade Organization acknowledge limitations in the data, including the limitation that the data does not capture informal workers, they nevertheless promote participation in GVCs for development on the basis of this data. As we have seen, however, a more inclusive approach to evidence reveals the weakness of relying on quantitative data analysis and devaluing or excluding other kinds of evidence.
Given the global authority of the international institutions, their reliance on quantitative data analysis is particularly problematic. Firstly, it reinforces the authority of orthodox evidence-based practice. Secondly, their reports produce authoritative ‘knowledge’ on which others rely. As Alessandrini puts it, the reports construct ‘an authoritative discourse about the nexus between GVCs and development that gets repeated, picked up and embedded in subsequent research, advice and policy-making, thereby shaping institutions’ and states’ mindset and conduct’ (Alessandrini 2022, 6–7).
Evidential Pluralism, in contrast, takes the full range of relevant evidence into account. An Evidential Pluralism evaluation begins by specifying the causal claim of interest. In this case, the causal claim of interest is that adoption of deep trade agreements, including WTO Extra and WTO Plus regulatory provisions, facilitates development. According to Evidential Pluralism, establishing this causal claim requires evidence of both correlation and mechanism.
The basic mechanism hypothesis is that adoption of WTO Plus and WTO Extra provisions facilitates integration into GVCs which leads to development, understood in terms of increased economic growth, better jobs, and reductions in poverty.
The quantitative studies on which global economic institutions rely provide evidence of mechanisms that provide some support for the link between adoption of regulatory provisions and insertion into GVCs and the link between insertion into GVCs and development.
The qualitative case study analyses discussed above provide evidence of the mechanism linking participation in GVCs to development. As we saw above, they reveal that increases in employment and real wages can be accompanied by deteriorating working conditions, increased precarity and reductions of non-wage benefits and over-time pay. The case study analyses therefore provide evidence of counteracting mechanisms that undermine the link between participation in GVCs and development. Taking account of such evidence decreases confidence in the claim that participation in GVCs facilitates development.
This example reveals the value of qualitative study insights and how they are incorporated within an evaluation based on the principles of Evidential Pluralism. If participation in GVCs continues to be promoted on the basis of quantitative data from observational studies without taking account of the broader evidence base, including insights from qualitative case study analyses, then this would indicate prejudice against these scientific methodologies. To prejudicially refuse to take account of a broader range of evidence, including qualitative analyses that reveal valuable insights, perpetuates epistemic injustice against the stakeholders whose views those studies consider, as well as the researchers who conduct those studies. By acknowledging the value of such insights and incorporating them into an evaluation, Evidential Pluralism avoids such epistemic injustices.
Conclusion
The aim of this paper has been to demonstrate that Evidential Pluralism can mitigate epistemic injustice in evidence-based evaluation.
We have argued that continued practices of exclusion and downgrading by proponents of orthodox evidence-based practice reflect prejudice and therefore constitute epistemic injustice. To support this, we have argued that, despite (i) long-standing evidence of the limitations of the orthodox approach, especially in the context of complex interventions, (ii) orthodox evidence-based practice’s conception of ‘good evidence,’ especially the stereotype of RCTs as the gold standard of evidence, is often applied inflexibly and dogmatically. This was illustrated through the example of Covid-19 face-mask mandates. To refuse to take account of other evidence when RCT evidence is lacking, inadequate or inappropriate, or when other kinds of evidence give us reason to question RCT evidence, reflects a dogmatic, closed-minded attitude that is unreasonably insensitive to the limitations of RCTs and the need for and merits of other kinds of evidence. Such dogmatism, closed-mindedness and insensitivity reflect prejudice. In devaluing or excluding evidence on the basis of prejudice against non-comparative study designs, orthodox evidence-based practice leads to incidental testimonial injustice and hermeneutical injustice.
Addressing the epistemic injustices that result from a dogmatic commitment to orthodox evidence-based practice is crucial. Epistemic injustice is a significant moral wrong that can have significant practical consequences. Evidential Pluralism, we have argued, offers an alternative approach that can mitigate the epistemic injustices that result from orthodox evidence-based practice. Evidential Pluralism advocates an inclusive approach to evidence and focuses on correlation and mechanisms as objects of evaluation rather than on pre-determined hierarchies of evidence. As a result, Evidential Pluralism avoids evidence being prejudicially dismissed or devalued. Evidential Pluralism, therefore, helps to avoid epistemic injustice in evidence-based evaluation. This was illustrated by the case of global value chains for development.
The fact that Evidential Pluralism can avoid epistemic injustice supports the adoption of Evidential Pluralism as a new methodology for evidence-based evaluation. 18
Footnotes
Acknowledgements
We are very grateful to the anonymous referees for many helpful comments. Versions of this work were presented at the Manchester Online Seminar on Evidential Pluralism, the Causation in Higher Education Conference at the London School of Economics (London, 2025), and the European Network for Philosophy of Social Sciences (Venice, 2025). We thank audiences for their helpful questions and discussions.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Leverhulme Trust (grant no. RPG-2022-336). Jon Williamson’s research is also supported by UKRI (CRCRM grant no. 25130).
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
