Abstract
Objectives:
Implicit bias can affect clinical decisions that influence the care received by patients whose ancestors had been subjected to unfair medical and social practices. However, literature describing the effects of implicit bias training as part of continuing medical and nursing education is scarce. We conducted a longitudinal evaluation of a training for maternal health care clinical and nonclinical staff.
Methods:
A total of 80 staff members at 2 clinical sites in Cleveland, Ohio, participated in the training and evaluation in 2020 and 2021. We used a mixed-methods evaluation to capture changes in knowledge, awareness of bias, and application of strategies to reduce biased behavior by conducting pre- and posttraining surveys immediately after training and interviews at 3 and 6 months posttraining. We conducted univariate and bivariate analyses of the surveys and recorded, transcribed, and analyzed interviews for themes.
Results:
Using a threshold of answering 3 of 5 knowledge questions correctly, 50 of 80 (62.5%) trainees who engaged in the evaluation passed the pretraining knowledge questions and 67 (83.8%) passed the posttraining knowledge questions. Of the 80 participants, 75 (93.8%) were women. Interviewees (n = 11) said that low staff-to-patient ratios, lack of racial and ethnic diversity in leadership, inadequate training on implicit bias, and lack of institutional consequences for poor behavior exacerbated bias in maternity care. Interviewees reported having heightened awareness of bias and feeling more empowered after the training to advocate for themselves and patients to prevent and mitigate bias in the hospital.
Conclusion:
Additional study describing the effect of implicit bias training as part of continuing medical education should be conducted, and administrative and management changes should also be made to prevent bias and improve quality of care.
Despite high spending on health care costs, the maternal mortality rate in the United States is nearly 3 times greater than in other high-income countries. 1 Racial and ethnic disparities contribute to poor maternal and infant health outcomes.2,3 Implicit bias can affect clinical decisions that influence the care received by historically marginalized patients (ie, patients whose ancestors had been subjected to unfair medical and social practices).4,5
Because of the need to incorporate discussion about social drivers of health and address biased care, some medical education institutions have developed implicit bias trainings.6,7 One study showed that a 1-hour implicit bias training enabled first-year medical students to identify a strategy to use when they recognize their own implicit bias. 8 Another study described the effectiveness of using role-play to teach first-year medical students how to identify and address bias. 9 Other research showed that trainings for faculty are useful for improving faculty confidence in recognizing and managing implicit bias for themselves, teaching learners, and presenting role-modeling behaviors.10,11
Anti-bias training is incorporated into medical and nursing schools, 4 and an opportunity exists for offering continuing education for medical providers and professionals who create hospital policies. While implicit bias trainings are commonly proposed as a solution to implicit bias, literature describing the effect of implicit bias training on practitioners, particularly those working in maternal and infant health, is scarce. It is difficult to measure the effectiveness of implicit bias trainings.12-15 The ultimate goal of anti–implicit bias trainings is to improve the quality of care. Many evaluations of trainings stop short of measuring changes in the behavior and attitudes of health care providers over time, both of which are critical steps toward improved quality of care.12-15
Research on the framework of implicit bias training among medical students and practitioners has found gaps in trainings that create difficulties in measuring their effectiveness in standard practice. Many programs focus on increasing health care provider knowledge and awareness as an outcome measure to show effectiveness. 16 Social psychology research suggests that knowledge and awareness are not enough to cause someone to reduce their bias; health care providers must also feel motivated to address their implicit bias and feel that they have strategies that are effective against default bias behavior. 17 Therefore, evaluations that focus solely on knowledge or awareness as the primary outcome fail to describe the effect of training on health care providers’ attitudes and beliefs in the workplace and do not adequately reflect opportunities for improvement through the use of educational tools to adjust behavior.
Purpose
In 2019, the March of Dimes initiated the Breaking Through Bias training to provide health care professionals with the skills to recognize and remedy implicit bias in maternity care settings. The objective of this article was to add to the understanding of implicit bias trainings through a longitudinal evaluation that describes the change in knowledge and awareness of bias and the application of strategies to reduce bias. To our knowledge, no studies have examined the long-term effects of implicit bias training on self-reported awareness of bias and changes in behavior by health care providers and administrative staff.
Methods
Breaking Through Bias training is available in 2 formats: a 1-hour self-paced e-learning and a 3-hour live-facilitated group interactive format. Both formats have a curriculum that provides an introduction to implicit bias, a historical overview of structural racism in the United States, strategies to mitigate racial bias in maternity care, and strategies to build a culture of equity. The 3-hour live-facilitated group interactive format provides educational material and also engages participants in interactive conversation with peers about the course topics. Participants are eligible to earn continuing nursing education and continuing medical education credits upon completion of both formats.
In 2020 and 2021, hospital leaders, clinicians, and staff participants at 2 clinical sites in Cleveland, Ohio, were invited to participate in the 3-hour live-facilitated group interactive format Breaking Through Bias training program. To comply with COVID-19 safety rules, the training was facilitated via Zoom.
To evaluate the training, the study team used a mixed-methods evaluation method from December 2020 through November 2021 to identify the short- and long-term effectiveness of the training to (1) improve knowledge and raise awareness about implicit bias in health care and poor health outcomes and (2) teach strategies to address implicit bias in practice. The Solutions Institutional Review Board reviewed this study and determined it was exempt because it was a program evaluation and not human subjects research according to 45 CFR 46.104(d)(4).
Data Collection
Surveys
We captured quantitative data through pre- and posttraining surveys immediately before and after the training. The March of Dimes evaluation team developed the surveys with support from the organization’s professional education team. The tools collected information on sociodemographic characteristics, knowledge about racial bias, attitudes related to bias, and participants’ recommendations and opinions. The pretraining survey was administered via an online March of Dimes registration system, and the posttraining survey was administered via an online March of Dimes survey. Participants were informed that survey responses were for evaluation purposes and that participation was voluntary.
We captured data on sex (male, female), age (20-29, 30-39, 40-49, ≥50 y), ethnicity (Hispanic/Latinx, non-Hispanic/Latinx), race (White, Black, other), and job function (administrative or other staff, doctor, manager or director, nurse, support staff). After reviewing responses, we recategorized race into White, Black or African American, and other. We included 5 multiple-choice questions on the pre- and posttraining surveys to assess participants’ knowledge about bias: (1) According to maternal mortality and morbidity data, the United States is the ___ among developed nations at protecting the lives of new mothers; (2) The most recent data show that Black women are ____ to die from a pregnancy-related death compared with White women in the United States; (3) What is internalized racism? (4) What is not healthy equity? and (5) How can individual providers reduce bias when talking with patients? These questions were part of the training’s standard knowledge check, which was routinely used by the professional education team to understand participant absorption of the training material. The questions were designed by the professional education team and were informed by the literature about implicit bias and the training material. Informed by the professional education team, using a threshold of >50% as passing, a passing score was achieved when 3 of 5 questions were answered correctly. Two statements to capture discriminatory attitudes adapted from the Quick Discrimination Index, 18 a validated scale, were included on surveys to gauge participants’ attitude toward racial diversity: (1) Overall, I think racial minorities in America complain too much about racial discrimination, and (2) I feel (or would feel) very comfortable having a person of color as my primary physician. Participants were asked to select the response that reflected their perception about the statement using a 5-point Likert scale (from 1 = strongly disagree to 5 = strongly agree). Using the Quick Discrimination Index scoring guidelines, we assigned scores into 3 categories: unfavorable attitude toward racial diversity, moderately favorable attitude toward racial diversity, and highly favorable attitude toward racial diversity.
Interviews
We collected qualitative data on perspectives of the training, changes in behavior, and recommendations for continuing to address bias through semistructured interviews with participants. The March of Dimes Evaluation Team developed moderator guides and conducted interviews via Zoom with each participant at 3 and 6 months posttraining. Interviewees indicated their willingness to participate in interviews on their posttraining survey and included relevant contact information. Before the interviews, participants were informed of their right to decline to participate and that their responses would be stored securely and anonymously. Both interview guides captured the same information, and responses were compared across the 2 periods of data collection. On the posttraining survey, 31 of 80 evaluation participants (38.8%) volunteered to be interviewed, and 11 (13.8% of 80) completed interviews. Of the 11 participants who were interviewed, 7 completed both 3- and 6-month interviews and 4 completed only the 3-month interview.
Statistical Analysis
We analyzed survey results for participants who completed both pre- and posttraining surveys. We conducted univariate analyses to describe participants who had engaged in the training and to present overall knowledge and attitude scores. We conducted bivariate analyses to identify associations between sociodemographic characteristics and knowledge and attitude scores using the Pearson χ2 test and t test. We set significance levels at α ≤ .05. We used SAS version 9.3 (SAS Institute, Inc) to analyze data.
We sent interview recordings to a third-party transcription company for verbatim transcription. Following an inductive coding approach, initial themes were identified by each author individually through a preliminary review of findings. A codebook was developed based on review of each author’s codes for similarity. In areas of disagreement, the authors finalized codes by mutual agreement following conversation. Using the codebook, thematic analysis was performed for all transcripts using the qualitative software MaxQDA 2020 (VERBI Software).
Results
Evaluation Participants
Two hundred 3-hour live-format training seats were made available, and 173 clinicians and staff members engaged in the training. Among participants, 93 (53.8%) completed the pretraining survey and 80 (46.2%) completed both surveys. Across both hospitals, most (n = 57; 71.2%) reported that they had received some form of implicit bias training in the past, most were female (n = 75; 93.8%), aged 30-39 years (n = 38; 48.1%), White (n = 56; 70.0%), non-Hispanic/Latinx (n = 77; 96.3%), and employed as a nurse (n = 55; 66.8%) (Table 1).
Characteristics of respondents to the March of Dimes Breaking Through Bias training evaluation, Cleveland, Ohio, 2020-2021 (N = 80) a
The Breaking Through Bias training was developed in 2019 to provide health care professionals with the skills to recognize and remedy implicit bias in maternity care settings.
Other was not defined.
Knowledge of Racial Bias and Attitudes Toward Health Equity
Of the 80 participants who completed both surveys, nearly two-thirds (n = 50; 62.5%) of respondents passed the pretraining knowledge survey questions and 83.8% (n = 67) passed the posttraining knowledge survey questions. On the pretraining survey, scores ranged from 0 to 5 correct answers, with the average number of correct answers being 3.7 of 5. On the posttraining survey, scores ranged from 3 to 5, and the average number of correct answers was 4.4 of 5. The mean difference was 0.6, and the difference in average scores was significant (P < .001) in a paired t test.
Of participants who improved, most were aged 30-39 years (47.5%; n = 38), were nurses (68.8%; n = 55), and had prior implicit bias training (71.3%; n = 57). Most participants (88.8%; n = 71) also showed highly favorable attitudes toward racial diversity, and no significant change was identified from pre- to posttraining. We found no significant associations between sociodemographic characteristics and knowledge gain.
Reported Changes in Perception of Bias as a Result of the Training
While some participants stated they were aware of bias in health care before the training, many reported changes to their perception of bias after the training related to feeling aware of personal bias and feeling empowered to address bias.
[It] has helped empower me a little bit to feel more comfortable in certain situations. I did feel a little more like, “It is important to speak up in these situations and when I’m able to, that will be a goal of mine. I will be more comfortable.” It helped to reinforce some of the things that I had been questioning. . . . seeing some bias with my clients and I feel more free to talk to her [client] about it to see if my perceptions are correct. [I feel] more conscious for myself of things that I might do or ways that I might think about someone else or activities that are happening. I began to classify things as bias that I would not have classified before as bias.
Reported Changes in Behavior as a Result of the Training
Participants self-reported changes in biased behavior. Of note, 3 categories emerged upon analysis: changes in participants’ interactions with patients and hospital staff; changes in their advocacy for themselves, other staff, and patients; and changes in how they hold other people accountable for biased behavior (Table 2).
Self-reported changes in behavior as a result of participation in the March of Dimes Breaking Through Bias training at 2 clinical sites in Cleveland, Ohio, 2020-2021 (n = 11 participants) a
The Breaking Through Bias training was developed in 2019 to provide health care professionals with the skills to recognize and remedy implicit bias in maternity care settings.
Perceptions of Other Staff Behavior as a Result of the Training
One question asked participants to identify the ways in which they may have noticed a change in their colleagues as a result of the training (Table 3). Participants reported that they perceived changes in other staff members’ behavior. Some expressed that staff members were more empowered, and others expressed that staff were more aware of their potential for bias.
Health care provider and administrative staff perceptions of other staff behavior as a result of the March of Dimes Breaking Through Bias training at 2 clinical sites in Cleveland, Ohio (n = 11 participants) a
The Breaking Through Bias training was developed in 2019 to provide health care professionals with the skills to recognize and remedy implicit bias in maternity care settings.
Reactions to Identified Bias Behavior After the Training
Participants reported how they felt when they found themselves engaging in biased behavior and how they changed their behavior. Upon identification of their biased behaviors, participants felt remorseful or unhappy. Most reported that they tried to change their behavior by acknowledging their own bias and approaching situations differently. Others reported that they tried to change their behavior by educating themselves on bias.
Try to first analyze why I have that bias . . . I try to overcome that bias and try to do my homework, so to say, on that bias so that I may still have the bias, but it won’t be a negative connotation to that. . . . if it’s an offense that could’ve been received by the other individual, then acknowledging . . . and then letting that other person know that I just caught myself and apologize for the misstep and try not to do it again. Hitting the pause button to say, “Wait, I need to stop.” This is a bias . . . catching myself and saying, “Wow, I really got to watch the next time when I encounter a similar situation.” I stopped myself and I was like, “It’s not right.” It’s important to look at groups to see trends so you can help, but it is dangerous to look at groups and just automatically put people in those groups. I would feel pretty crappy, speaking completely honestly, because that’s never my intention to judge anyone. I’m very open to being educated on whatever I have made a stupid mistake about. “Please tell me what I did wrong so I don’t do it moving forward.”
Factors That Facilitate Continuation of Bias in Health Care
Participants provided examples of factors that allowed bias to exist in health care. Themes included having busy schedules, poor staff-to-patient ratios, inadequate hiring processes that do not identify issues of bias in staffing, lack of diversity in leadership, inadequate training on bias in medical and nursing schools, lack of ongoing education about bias, no institutional consequences for biased behavior, and hospital leadership not addressing bias.
Lessons Learned
Many participants, especially physicians, had adequate knowledge about bias before completing the training. Responses to questions about knowledge suggest a fairly limited impact on improvements in knowledge about bias; however, these findings are consistent with the findings of other studies.11-15
Our study used qualitative findings to describe self-reported changes in health care provider perceptions and behaviors over time. Of the few prior studies that measured changes in implicit bias scores or behaviors, no studies evaluated the long-term results of implicit bias trainings. 13
Participants provided numerous and varied examples of the ways in which they changed their perceptions of bias and addressed their own or others’ biased behaviors. Participants also shared examples of feeling empowered to make changes in their behavior. These findings are important for continuing to address the implicit bias among medical providers and administrators to improve care delivered. 5 While these findings related to behavior change and awareness of others’ behavior are self-reported and cannot be used as absolute proof of any changed behaviors or awareness, it is encouraging that participants recalled examples of the ways in which they applied newly learned strategies. Our results suggest that participants who engaged in the evaluation understood and were applying lessons learned to implement strategies to identify bias and address it, depending on their capacity to do so. 19 Furthermore, responses related to perceptions of changed behaviors by staff colleagues also support that participants were digesting new information and using it.
Our results are important for offering antibias trainings, as organizations and workplaces are increasingly requiring employees to complete trainings to address bias. Findings from a 2019 meta-analysis suggest that improvement in implicit bias score is not always associated with a reduction in self-reported biased behaviors. 13 This finding suggests that the relationship between implicit bias scores and behavior is complicated and nonlinear. The Breaking Through Bias training incorporated skill training related to mitigating racial bias in maternity care and building a culture of equity. We recommend additional research to identify how improvement in cognitive awareness of implicit bias may be linked to subsequent behavior change and how skill training may promote behavior change.
We found that no sociodemographic characteristics were associated with improvements in knowledge about bias. This finding may be impacted by a low response rate. As with other evaluations about implicit bias training, 11 this evaluation was affected by fewer participants engaging in the evaluation than in the actual training. To assess bias in participation, we compared those who completed only the pretraining survey (n = 93) with those who completed both surveys (n = 80). We found no significant differences between those who completed both surveys and those who did not. Despite this finding, the low response rate affected our ability to fully assess the effects of the training because only 46.2% (n = 80) of those who completed the evaluation completed the surveys, and only 13.8% of all evaluation participants participated in interviews. While the low response rate in our study is comparable with the response rate in other studies, it would be useful to conduct additional exploration into the participants’ rationale for participating or not participating in the evaluation to identify whether systemic barriers to participation (eg, lack of time) may also be contributing to unintentional biased decision-making when interacting in the workplace. 11
Because implicit bias has been found to result in poor health outcomes and experiences for patients whose ancestors faced discrimination and were historically socially and medically underserved,4,5 the effectiveness of continuing education in the form of implicit bias training should continue to be studied to support policy and practice decisions. In addition, the quality of new training should be carefully evaluated to ensure that the trainings do not reinforce stereotypes and contribute to continued bias. Finally, while evaluation of implicit bias trainings has identified some effect on implicit bias in health care settings, a continued need exists for refined instruments to effectively measure changes in impact. 16
This study had several limitations. First, we conducted this longitudinal evaluation at 2 clinical sites in Ohio; as such, findings are not generalizable to other populations in other settings. Second, fewer participants engaged in the evaluation than in the actual training, which affected our ability to fully describe the effect of the training on all participants. Importantly, the self-reported findings related to behavior change and change in bias are derived from a convenience sample of just 14% of the 80 evaluation participants, and these findings cannot be considered to be representative beyond the evaluation sample. In addition, 75 of 80 participants who engaged in the evaluation were women, which limits our ability to describe the training’s impact on men. Third, because of evaluation setup constraints, the evaluation captured data only on self-reported behavior change and self-reported awareness of bias (including both bias by others and self). These self-reported behaviors and awareness may be affected by social desirability bias. Future evaluations should identify strategies to capture documented examples of behavior change. Finally, the evaluation did not capture patients’ perspective on how the training affected their experience of receiving care. Additional research incorporating this viewpoint should be conducted to fully understand the effect of these trainings.
Implicit bias training should be incorporated into continuing education for health care providers and administrators. Noting the limitations of this evaluation, findings from this evaluation suggest that the March of Dimes Breaking Through Bias training contributes to improvements in knowledge about racial and ethnic equity in health care and health care settings and contributes to improved awareness of bias. To inhibit bias in health care, factors such as balancing staff-to-patient ratios, improving hiring processes to reduce bias in staffing, increasing racial and ethnic diversity in leadership, creating and enforcing policies to prevent and mitigate bias, and improving access to education about bias should be addressed.
Footnotes
Acknowledgements
The authors thank Elena Schmidt, PhD, Jennifer Sedlmeyer, MSN, Rahni Jenkins, MEd, Lisa Holloway, MBA, and Sherenne Simon, MPH, from the March of Dimes for their support in the implementation of the Breaking Through Bias training and the associated longitudinal evaluation.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This longitudinal evaluation was funded through sources from Case Western Reserve University. The funder had no role in the design of the evaluation, data collection, analysis, or interpretation of the data.
