Abstract
Racial bias in research impacts a study’s relevancy, validity and reliability, though presently this aspect is not addressed in critical appraisal tools, and consequently appraisers may not take racial bias into account when assessing a paper’s quality.
Drawing on critical race theory (CRT) tenets that racism is ubiquitous and race a social construct, this paper discusses concerns regarding racism in research which have been broadly divided into two categories for critical appraisers to consider: the underrepresentation of minoritised ethnic groups in health studies, especially where minoritised populations see higher rates of disease occurrence and; the utilisation of racial/ethnicity data to interpret disparities in outcomes, including speculation of biological race, the misinterpretation of genetic ancestry as race, and the lack of investigation into social determinants of health, including systemic, institutional and interpersonal forms of racism. The injustices exposed in this paper impact the health of minoritised ethnic groups and are therefore a Black Lives Matter issue. They risk resurrecting dangerous theories regarding biological inferiority among minoritised ethnic populations, as well as hindering study findings.
The application of CRT frameworks in health science research quality appraisal is discussed in relation to the above themes – using largely UK-based contexts with supporting examples from the US – followed by recommendations for critically appraising for antiracism. Further information to support critically appraising for antiracism can be found via
Introduction
Structural inequity is ubiquitous (Tsai, 2021), and the institute of research is no exception, where racist practices, whether intentional or unintentional, influence research methodologies and outcomes. Despite this, it is not expected that research quality appraisers asses racial bias when evaluating a study.
Utilising critical race theory (CRT), this paper has divided concerns regarding racism in research into two categories for critical appraisers to assess: the underrepresentation of minoritised ethnic groups in studies, and the use of racial/ethnicity data to understand disparities in outcomes. These themes and their consequences in relation to health disparities as well as quality appraisal will be discussed, and recommendations for future appraisal provided. The health sciences literature reviews in this article are interdisciplinary in topic and chosen as they are particularly relevant to minoritised ethnic populations, by way of discussing issues prevalent within these groups.
CRT is an appropriate theoretical framework for discussing underrepresentation and interpretation issues, applicable to both appraisers and authors of research. Using CRT praxis, appraisers must examine, recognise and challenge the conduct that reinforce or reproduce racist practices, holding researchers accountable. While researchers conducting heath science trials have an obligation to the populations they serve to not only ensure their representation, being race conscious in understanding why underrepresentation occurs and acknowledging that racial bias exists in healthcare, but also guarantee that disparities in research outcomes are interpreted as fully as possible, and not capitulated to speculation or theories of biological race.
Racial bias in research
Critical race theory (CRT) identifies that racism is embedded into the everyday institutions and structures of society (Bell, 1992) including health services and research practices. Racial bias in research impacts a study’s relevancy, validity and reliability, though presently this aspect is not addressed in critical appraisal practice, and consequently consumers may not take racial bias into account when assessing a paper’s quality. However, critically appraising for antiracism not only alerts consumers to limitations in published research, giving them an idea of a study’s strength and methodological robustness, but this process also underlines the importance of diversity and application of CRT from the beginning of the research cycle, and in doing so supports researchers of the future in overcoming these limitations. Those who participate in, consume, and appraise research have a role to play in its recalibration; to reflect on previous practices with critical race perspectives, acknowledge complicity in replicating social inequalities, and actively engage to produce more inclusive research (Rai, 2022).
Critical appraisal
When a research trial is undertaken, there are three possible explanations for its results: 1) the findings are true 2) the findings represent random chance 3) the findings are influenced by bias (Mhaskar, 2009). Random error (chance) can be prevented in ways such as increasing sample or effect sizes, whilst systematic errors (bias) can also be influenced by shortcomings in the design, conduct or analysis of a trial.
Consumers of research, including practitioners, researchers and librarians, must determine if a study’s results are true and not subject to chance or bias, as well as if the results are applicable in order to consider its interventions for practice (Greenhalgh, 2019). Critical appraisal involves the careful and systematic assessment of a piece of research’s trustworthiness and methodological accuracy (Tod, 2021) and aims to answer three broad questions: 1) To determine the reliability of a study – If researchers employed accurate measures and comprehensive use of data. 2) To determine study validity – were the design and methodology strong? What are the results, and should they be believed based on potential threats to internal validity? 3) To determine if a piece of research is relevant to the appraiser’s practice, or the population under study (Burls, 2014).
To assist with the critical appraisal process, assessors of research frequently use standardised critical appraisal tools, such as those from the Critical Appraisal Skills Programme (
It is imperative that appraisers and tool developers recognise and address the impact of racial bias in this process, as the incessant underrepresentation of minority populations, and the unacceptable misinterpretations of health disparities between populations, have consequences not just for a paper’s methodological robustness, but also for the health and wellbeing of these groups.
Underrepresentation of minoritised ethnic populations in research
In the United Kingdom, the number of Black, Asian and minority ethnic (BAME) populations have been rising, and in 2018 about 13.8% of the UK population was from a minoritised ethnic background (ONS, 2021). Despite this, BAME populations continue to be underrepresented in clinical trials. Minoritised ethnics have often had lower participation rates in research than should be expected from population estimates, but with a higher healthcare burden that is not matched by the volume of research designed for these groups (NIHR, 2020). This has been evidenced recently with the COVID-19 vaccine trials. To date in the UK over 536,000 volunteers have been enrolled into COVID-19 vaccine trials (NHS Digital, 2021) though only 7.5% of these participants are minoritised ethnic people, including 0.7% from Black backgrounds and 4.1% from Asian backgrounds. However, BAME populations comprise 17.5% of the population in England and Wales alone, including 7.5% from Asian backgrounds, and 3.4% from Black backgrounds (Gov.UK, 2020). Additionally, due to the endemic ubiquity of racism (Bell, 1992), the human toll of COVID-19 has not been equal, and it became evident early in the pandemic that people from minority groups in the UK and in the USA were disproportionately impacted by COVID-19 complications and deaths (Treweek, 2020). Unfortunately, these findings and outcomes are just the latest in a difficult history of minoritised ethnics in the health system.
The importance of diversity in healthcare research
Recent racial justice and equality movements, in combination with the impact of the COVID-19 pandemic, have highlighted the longstanding underrepresentation of minoritised ethnic populations in clinical research, and calls for greater diversity have been amplified (Etti, 2021).
Clinical trial researchers have a duty to recruit a sample of volunteers that closely represent the population under study to ensure its results are generalisable (Oh, 2015). However, the integration of racial injustice in the health research system results in most clinicians being informed by research extrapolated from a largely homogenous population, usually White and male (Oh, 2015), producing biased results, and leaving questions about how clinicians can apply the research findings to their own patients (NIHR, 2020). If there is a lack of evidence on the impact of an intervention for a certain group, then clinicians may be reluctant to offer that intervention to individuals of that group (NIHR, 2020) and in such cases, clinical opinion may replace evidence. The UK government’s vision of ‘no decision about me, without me’ regarding shared understanding and decision-making (Coulter, 2011) further indicates that researchers have a responsibility and obligation to make certain all populations are included in research. Recently, the UK’s National Institute for Health Research (NIHR), recognising the importance of research participant diversity, sought to address the lack of representation of under-served groups with the introduction of the INCLUDE Framework and Guidance (NIHR, 2020).
Reasons for minority underrepresentation
At its core, critical race theory for health sciences begs the question, “How has racialisation contributed to the issue?” (Ford, 2010) and reasons for the underrepresentation of minority ethnic and racial groups in trial participation are many and complex. Through race critical consciousness, CRT can be applied to understand the influence of racism on the underrepresentation of these populations in clinical trials, by examining their experiences of institutional and structural racism, discrimination from health providers, historical factors which have led to fear and scepticism of clinical trials (Sneed, 2021), or feelings of inadequacy in taking part (NIHR, 2020).
Racial bias leads to the discouragement of and fewer opportunities for minoritised ethnics. For example, research has shown that physicians are less likely to offer a clinical trial participation to minority patients than to white patients, as they may believe minority groups are unsuitable study candidates (Hamel, 2016). Reasons for this include clinician belief that minority patients are less likely to comply with recommended treatment, and the influence of racial stereotyping on physicians’ diagnoses and treatments (Chapman, 2013).
Information regarding clinical trials can be made more accessible to all communities if researchers accept the pervasiveness of racism, practice race critical consciousness, evaluate information barriers such as presentation and dissemination of participation opportunities (Wright, 2020) and understand cultural considerations (NIHR, 2020). The location in which a trial is undertaken undoubtedly has implications for recruitment, and a recent study by Rai and colleagues (2021) highlighted this factor, finding that areas with higher levels of deprivation and higher populations of minoritised ethnics, are often under-supported to conduct research. Data collection deadlines push recruitment in areas already set up to support research; those areas with more educated and middle-class populations (Rai, 2022).
In countries where healthcare is not free such as the USA, minorities are more likely to be underinsured (Artiga, 2021) and may have concerns about the cost of trial participation due to usually earning less than their White counterparts (United States Census Bureau, 2021), problems which further stem from systemic racism.
Mistrust due to racist historical research events have certainly had an impact on study recruitment today. The Tuskegee syphilis trial is widely recognised as a reason for ongoing mistrust due to the extent and duration of dishonesty and abuse suffered by the all-African American participant group of nearly 400 (Saini, 2020). In this prospective study conducted between 1932 and 1972, participants were not told of their syphilis diagnosis, remaining uninformed of the study’s true purpose; to observe the natural history of untreated syphilis. Studies like this, as well as the medical experimentation conducted by large pharmaceutical companies in Africa between the 1970s and 1990s (Washington, 2006) have led minorities to believe they bear the risks of medical research. These historical events are for some reinforced by racism in the health system and personal discriminatory events in healthcare settings that continue to this day.
Racism in research and healthcare systems impacts the service provider as well as the user. Though the British NHS has one of the most diverse workforces in the public sector, 2020 research by the King’s Fund reported a lack of ethnic minority representation at senior levels, with ethnic minority staff reporting higher incidences of discrimination at work, and fewer ethnic minority staff reporting their institution offers equal opportunities for career progression (King’s Fund, 2020). Lack of diversity within the NHS may be particularly discernible for its librarians and information professionals as, in the UK overall, just 3% of the library and information workforce are non-White (Hall, 2016). These facts will undoubtedly also have a negative impact on the diversity of research teams, yet it is known that diverse groups are better equipped to address health disparities, and adapt to needs of diverse populations (Swartz, 2019). If researchers are to recruit for diversity in their trials, their own teams must reflect the population they aim to serve.
As there are many barriers to recruiting for diversity, it may not always be possible for researchers to recruit a true sample. When this occurs, researchers should practice disciplinary self-critique and address inadvertencies within the study’s limitations because, if study participants are different from the study population as a whole, these differences need to be justified and explained (NIHR, 2020). Using race critical consciousness to address issues of underrepresentation within published limitations can drive the research discipline toward more equitable awareness of its mission (Ford, 2010).
Minoritised ethnic populations and health conditions
Systemic racism perpetuates discrimination and disadvantage, and this is especially clear in healthcare where evidence has continuously demonstrated that minoritised ethnic people are more likely to suffer and die from various conditions. For example, in the UK, patients from BAME backgrounds have greater risk of heart and circulatory diseases (British Heart Foundation, 2020); stillbirths and infant mortality are highest among babies from Black and Pakistani groups; and, as previously mentioned, the COVID-19 pandemic impacted minority groups disproportionately, with Black, Asian and most other ethnic minority groups more likely to be diagnosed with, get severely ill and die from COVID-19, compared to the White population (Raleigh, 2021).
Unfortunately, even for conditions that disproportionately impact ethnic minority populations, they remain underrepresented in the trials that address these conditions. One well-reported area is mental health disorders where, in the US, African American patients have shown greater levels of impairment and severity of disorder, with poorer treatment outcomes relative to their White counterparts (Murphy, 2016). The U.S. Department of Health and Human Services have reported that African Americans aged 18 and over are 40% more likely to report serious psychological distress compared to the White population (Tchouankam et al., 2021). Yet, in a systematic review looking at the African American representation in family and twin studies of mood and anxiety disorders, of 209 studies at least 80 did not include any African Americans at all. “At least”, because 88 of the studies either did not report any data on ethnicity, or only reported ethnicity data for its White participants (Murphy, 2016). When the data for only one racial/ethnic group is provided, usually that of White participants, it places this group as a priority, or the ‘norm’, and further ‘others’ the remaining populations. This type of reporting is a necessity when the number of remaining participants is so few that publishing racial or ethnicity data could risk breaking anonymity of these participants (Flanagin et al., 2021) and though it is vital that anonymity is maintained, this restriction is further confirmation that an insufficient number of non-White participants are being recruited. If samples of non-Whites are too few, then statistically significant results cannot be produced, and results may represent random chance (Faber, 2014). This is a key limitation; these studies that could offer vast insight into cause and effect of health disparities, are then not relevant for African American populations.
In another systematic review by Cwalina and colleagues (2021) looking at racial and ethnic diversity in orthopaedic trials of the last 20 years, researchers found that just 8.5% (89 of 1043) of studies reported any data on race and ethnicity. The review further found that trials located in Europe, North America, Australia, New Zealand or Asia were less likely to report race or ethnicity data. In those trials which did report racial/ethnicity data, unsurprisingly, White populations were over-represented whilst minority groups were underrepresented (Cwalina, 2021).
As most trials suffer from underrepresentation of minoritised ethnics, when successful interventions are reviewed for consideration into the health system, the lack of minority representation is of course not a deterrent to decision-making. A review by Loree (2019) looking at the disparity of race reporting and representation in clinical trials leading to US cancer drug approvals from 2008 to 2018, found that despite about 13% of cancer patients in the US being Black, (and 15% of cancer mortality patients being Black) Black participation in trials which led to FDA drug approvals was less than 5%. Again, White populations saw appropriate representation in these trials (Loree, 2019), further confirming that the ubiquitous nature of racism in research ensures that healthcare interventions primarily benefit dominant groups.
Critically appraising for underrepresentation
Many consumers of research will make use of critical appraisal tools for guidance throughout the appraisal process, and it is true that the majority of critical appraisal checklists do address the issue of representation. For example, the widely utilised Critical Appraisal Skills Programme (CASP) checklist for randomised controlled trials (RCTs) asks its users to consider if the results can be ‘applied to your local population/in your context?’, prompting with the question ‘Would any differences between your population and the study participants alter the outcomes reported in the study?’ (CASP, 2021, p. 4). However, these questions implore its users to analyse the way the results can be applied to the
In their cohort study tool, the Scottish Intercollegiate Guidelines Network (SIGN) reminds appraisers that ‘For valid study results, it is essential that the study participants are truly representative of the source population’ (SIGN, 2012, p. 3). Though this note implies the inclusion of all groups who make up the population of interest, there is no explicit mention of the under-served groups who are
If minoritised ethnic populations are disproportionately more likely to suffer and die from conditions, illnesses, coronaviruses, then they must be represented in the trials and studies that aim to save people’s lives – it is a Black Lives Matter issue.
It can be no coincidence that groups who are racially marginalised and discriminated against are more likely to suffer from certain ailments, and it is no coincidence that they are not represented in research and trials – these “truths” are shaped and influenced by structural inequalities (Hamel, 2016; NIHR, 2020; Oh, 2015; Wright, 2020). Research studies are pivotal to the introduction of interventions into our health systems, and if the methodologies are not sound at a foundational level, it can become a matter of life and death (Rai, 2022). Appraisers of research should apply a CRT lens to evaluate whether minoritised ethnics were appropriately represented in research and, if underrepresentation occurred, whether authors practiced disciplinary self-critique and provided explanations for this shortcoming. Praxis matters in critical race theory, and applying a CRT framework to critical appraisal teaching allows consumers of research to not only name racism, but challenge it in the pursuit for racial equity.
Interpretation of race and ethnicity data
Health inequalities among minoritised ethnic populations
People from racially/ethnically minoritised populations often experience poorer health outcomes than their White counterparts, and CRT can be used to take apart the structures that drive these racial inequalities by helping researchers and appraisers understand that ‘race’ is a vague, socially constructed and flexible way in which people have been grouped by society, primarily based on appearance, yet assumed to suggest deeper biological links. However, race is not biology. Race is a social category with biological consequences (Graves, 2021; Roberts, 2011; Hoberman, 2012; Fuentes et al., 2019; Saini, 2020).
Due to the societal nexus of racism – the structural, institutional, and interpersonal forms of racial bias – minoritised ethnic populations are more likely to be socially and economically disadvantaged, which consequently impacts their health and wellbeing. In the UK, about 50% of all Bangladeshis and 46% of all Pakistanis are among the most deprived fifth of the population, compared to 20% of White British people (Institute of Health Equity, 2020). 46% of Black African and Caribbean people and 32% of those with a mixed ethnic heritage are living in poverty, compared with 19% of White British people (Social Metrics Commission, 2020). In the US in 2020, 19.5% of Black people and 17% of Hispanic people were living below the poverty line, compared to 8.2% of White people (United Stated Census Bureau, 2021). Studies have also explored the differences in harm to minoritised ethnics, from air pollution due to living in deprived areas nearer to major sources of pollution (American Lung Association, 2001; US EPA, 2019), from working in jobs with the highest risk of injury (Seabury, 2017) and from racial discrimination significantly impacting their mental health (Royal College of Psychiatrists, 2018).
The above demonstrates what CRT informs, that differences in health outcomes cannot be explained by racial/ethnicity data alone; structural, institutional and interpersonal forms of racism must be investigated within research, and voices of the oppressed need to be heard, to determine relationships with health. However, for millennia, scientists and doctors justified racial discrimination through the belief that there is empirical evidence which supports racial inferiority (Saini, 2020). This scientific racism was practiced heavily during the slave trade and World War II, to legitimate White Supremacy (Collins, 2008). During the era of slavery, physiological and psychological grounds were justification for the hot, damp and suffocating working conditions, which were seen as conditions Africans could withstand, ascribing a racial ‘hardiness’ to Black people which still endures today (Hoberman, 2012). For example, a 2016 study by Hoffman and colleagues demonstrated that beliefs about biological differences between Black and White people which date back to slavery continue to be associated with the idea that Black people feel less pain, often resulting in inadequate treatment recommendations for Black patients (Hoffman, 2016).
The myth of biological race
In 2003, researchers working on the Human Genome Project confirmed to the world that race has no scientific and genetic basis, that it is a fluid concept and the vast majority of genetic variation exists within racial groups and not between them (Bonham, 2005). These findings inform researchers that there is no biological foundation for race, that race and ethnicity should be used to represent social experiences, not biological facts, when explaining a phenomenon under study (Ross, 2020). However, despite the repeated debunking of the existence of biological race, and the growing evidence regarding the health consequences of racism (Paradies, 2015; Priest, 2013) researchers continue to speculate that health disparities may be due to biological differences among “races”; attributing effects to race rather than to racialisation or racism.
A (now amended) 2020 article in
Another area in research where race is often concluded as the cause of differences in outcome is metabolic rates, specifically for drugs such as antidepressants. Studies have reported that antidepressants are less effective in non-White populations (Gibiino et al., 2014), that antidepressants are more effective for African American and Hispanic patients (Chen et al., 2009) that Asian patients often respond to lower doses of antidepressants (Chen et al., 2002). It is important to understand that these trends, if they exist, are linked to genetic ancestry. Genetic ancestry is the genetic origin of an individual’s population, which is different from race (Borrell, 2021). It is true that race and ethnicity may well capture information about the likely presence of genetic variants, but ancestry is a much clearer predictor of why some mutations, genetic or metabolic conditions are more prevalent within certain groups than others (Borrell, 2021), and using terms such as ‘genetic ancestry’ prevents researchers resorting to notions of racial inferiority. However, despite concluding possible genetic factors as reasons for health inequalities, it is concerning that genomic testing is rarely involved (Cwalina, 2021) especially since evidence has shown that individuals are frequently genetically more similar to members of other groups than to members of their own group (Witherspoon, 2007; Yu, 2002). Moreover, if researchers are stratifying data by genetic ancestry, they must be aware that the genetic admixture of populations, due to ever-increasing transcontinental migration, undermines the use of even self-identified ethnicity in assuming genetic ancestry (Shah, 2018). Therefore, genetic testing
In addition to genetic or biological claims, researchers may hypothesise ‘cultural differences’ to explain health inequalities, routinely privileging the voices of majority perspectives for everything from diet to sexual practices (Rai, 2022) without naming specific practices and evidencing their impact. Blaming culture, as with blaming race, blames the patient and assumes that there is little that the health or political system can do to alleviate these inequalities. Whiteness theory problematises the standardisation of Whiteness (Green, 2007), and the reporting of cultural differences typically does just that; places White culture as default and centric, ‘othering’ non-White groups whose voices are seldom heard, while White culture is rarely collectively analysed to explain White people’s health outcomes.
Stratifying data by race
Though recruiting for trial diversity should be seen as essential in the planning of research, stratifying results data by race may not be. The purpose for including race or ethnicity as a variable should be clarified and justified, and the use of these variables should be limited to topics in which they can directly contribute to the production of new knowledge about the studied phenomenon (Cwalina, 2021). The research question should support the use of race and ethnicity variable, ideally to explore social experiences that impact outcomes. This way, research studies can contribute to CRT by moving beyond simply reporting health disparities, but also understanding and challenging the societal structural forces by which they are influenced. Although race may be considered a proxy for racism, race proxies do not clarify the mechanisms by which racism effects occur (Ford, 2010). Appraisers of health literature must consider how race and ethnicity are relevant to the study, how the use of these variables can improve understanding of the study topic, and if these variables were used to support understanding of social experiences that impact the phenomenon under study (Rai, 2022). Researchers should justify the reporting of race and ethnicity data in their research protocol before beginning a trial, to prevent speculation of genetic/biological factors if disparities in outcomes occur, when such investigations were never intended to be scientifically explored. Findings should instead advance knowledge on a topic in ways that promote racial equity. Claims of an independent association between race and health outcomes should be interpreted with scepticism – correlation is not causation.
From evidence to practice
As racial and ethnicity categories are socially constructed, social covariates must be reported alongside race and ethnicity data. Health differences between racial and ethnic groups can only be understood when race is framed as a social construct – how political and historical contexts shape the lives of people through, for example, access to and experience of education, housing, and employment (Stafford, 2020). Intersectionality theory, as developed by Kimberlé Crenshaw (Crenshaw, 2017) can also support the understanding of health inequalities and improve research robustness. The incorporation of intersectionality, by recognising whether wider social determinants of health are relevant to outcomes and accordingly avoiding the use of proxy variables, can enhance the validity of research by reducing measurement bias and improving construct validity (Bauer, 2014).
Numerous publications illustrate the consequences of disregarding these wider determinants of health, by outlining various racial algorithms which have the potential to perpetuate or even amplify race-based health inequities (Benjamin, 2019; Noble, 2018; Vyas et al., 2020). One example is the US Kidney Donor Risk Index (KDRI), which predicts the likelihood of kidney graft failure from potential donors (Vyas et al., 2020). In its algorithm, being of African American race signals a risk of kidney graft failure, thereupon reducing the number of kidney donations from African American patients. As African American patients are more likely to receive kidneys from African American donors (Harding, 2017), this algorithm reduces African American patients’ odds of receiving a kidney donation. African Americans
As seen previously, there are numerous examples of research which conclude that different races may process or metabolise antidepressant drugs at different rates, suggesting biological differences between different racial groups (Chen, 2002; Chen, 2009; Gibiino, 2014) and evidence suggests that antidepressant treatment for minoritised populations is negatively impacted due to race-based conclusions such as these. Studies have found that nearly all minority groups in the US are at risk of inadequate antidepressant use (Quinones, 2014), that African Americans are less likely to receive a depression diagnosis, and even when they do, are less likely to be treated (Akincigil, 2012), and that Black and Hispanic people are less likely to receive a prescription for antidepressants (Lagomasino, 2011). In 2002,
It is important to understand that doctors and researchers are practicing race medicine when they consider the colour of a patient. The KDRI tool described above was not created by racist clinicians, just as Dr. Satel did not believe she was practicing race medicine – their aim is to provide the best care possible based on the available evidence, including the interpretation of that evidence. However, if the evidence has been given unsuitable interpretations, that biological differences between races are the cause of health disparities, then tools and clinicians will of course take that into account. By not naming racism within published research, solutions to racism as a targeted intervention cannot be produced.
Critically appraising for inappropriate interpretations of disparities in outcome
As described above, issues around the interpretation of research results includes speculation of biological race-based differences despite lack of investigations, the misinterpretation of genetic ancestry as race, and the lack of investigation into social determinants of health, including systemic, institutional and interpersonal forms of racism. These issues can impact the reliability and validity of study findings, as well as resurrect dangerous theories regarding biological inferiority among minoritised ethnic populations.
Current critical appraisal checklists do not apply critical race theory frameworks to consider racial bias in results interpretation. Appraisers are no doubt aware that bias is a common issue in research, and critical appraisal tools do always address this, but the variations of bias that researchers attempt to avoid (and appraisers look out for) are generally from a standard group of commonly taught biases such as allocation bias; detection bias; attrition bias; reporting bias etc. Though racial bias may contribute to these more commonly-taught biases, for example selection bias, it is generally not an accepted or expected issue, and therefore resources which teach critical appraisal do not address it as a factor to be mindful of.
The
The assessment of confounding factors is likely the best way appraisers can address the issue of racial bias when using critical appraisal tools, as most checklists will prompt users to investigate if researchers have taken confounders into account. A confounding factor is an additional unmeasured variable that influences both the supposed cause and the supposed effect, resulting in a distortion of the true relationship between the two (Skelly, 2012). Most commonly confounding factors will falsely demonstrate an association between cause and effect when in reality no such association exists. In critical appraisal practice, when ‘race’ is used in association with a ‘biological’ outcome, it can be determined that confounding bias has occurred, due to unmeasured factors such as socio-economic status, access to healthcare and other forms of structural and institutional racism. Further, if researchers have conducted genetic investigations to conclude race-based differences, then genetic ancestry can confound these associations.
Race may serve as a convenient proxy for some genetic differences however, it is genetic diversity that is the cause of disease risk differences among populations, not the inequalities defined by socially determined ‘race’ (Sirugo, 2021). Here, CRT can be applied as a powerful tool for change, helping researchers and appraisers understand that the use of race in research outcomes should be used to reflect the different social experiences, and that the reporting of race alone, with no investigation into income, housing, employment, literacy, and other factors influenced by racism, is counterproductive in alleviating health disparities (Cwalina, 2021).
Recommendation for critically appraising for antiracism
The following questions can support the appraisal for racial bias in research and may be used in conjunction with any other standardised critical appraisal tool. Further information to support critically appraising for antiracism can be found via
1. Have minoritised ethnic populations been recruited onto the study?
To generate accurate interpretation of research findings, researchers must recruit for diversity. Minoritised ethnics exist in all populations and should be included in all relevant research. Given the ubiquitous nature of racism and the large amount of barriers to recruitment, this may not always be possible, in which case researchers should provide an explanation in the limitations.
2. Is the sample of minoritised ethnic participants representative of the population under study?
Researchers should also consider that some minority groups are disproportionately impacted by certain conditions. If reporting on health disparities between ethnic groups, the ethnicity sub-samples should be large enough to provide statistically significant results.
3. Is the data on race and ethnicity full and accurate?
This is required to maximize data richness. Researchers should where possible avoid only reporting the ethnicity of White participants and ‘othering’ the rest of the participants – This places White populations as a priority.
4. If racial/ethnicity results data was stratified by race, was this justified?
Although there is a clear need for racial and ethnic representation in clinical trials, the usefulness stratifying results data by race or ethnicity likely depends on the specific topic, prior evidence of disparities, and individualised study hypotheses.
5. Have the differences in outcome between racial and ethnic groups been reported, appropriately interpreted and addressed?
As ‘race’ is a social construct, differences in outcomes may only be understood when researchers discuss the wider social determinants of health.
Question genetic interpretations of race because these are not grounded in science and promote an idea of racial inferiority.
6. Did researchers avoid assigning race as a variable, a risk factor or a proxy for genetic ancestry?
When race as a variable is assigned as a risk factor, the idea of biological differences may be cemented into tools and guidelines. Researchers must consider confounders such as systemic, institutional and interpersonal forms of racism. Race should not be used as a proxy for genetic ancestry as these are not the same.
Conclusion
Applying a CRT framework, this paper has outlined issues around racial bias in healthcare research methodologies and analysis, their influence on research conduct and outcomes, and how this negatively impacts minoritised ethnic populations. These concerns have been divided into two themes: underrepresentation and interpretation, with examples highlighting their empirical importance, and determining that these issues must be addressed in critical appraisal practice.
Racism’s contribution to the underrepresentation of minoritised ethnic populations in research and the disparities in health outcomes cannot be understood without acknowledging the role of racialisation – applying the critical race theory tenets that racialisation is integral to society, and that race is not biological. Racial bias influences a research trial’s validity, relevance and reliability, which can cause further consequences for minoritised ethnic people in the health system – racism in research is a Black Lives Matter issue.
Researchers must be held accountable and practice disciplinary self-critique, while appraisers must recognise and contest racial bias to diminish the creation and perpetuation of racism in research, allowing investigation into appropriate interventions for racism. Critical race theory must be applied to critical appraisal to take apart the structures that drive racial inequalities, and Library and Information, Health Sciences and Education disciplines must collaborate in addressing racial injustice through CRT-informed praxis.