Engaging the Entire Learner: Pathway Program Administrators’ Experiences of Providing Students with Research Experiences in Academic Medicine

Abstract

Objective

Pathway programs designed to recruit and retain students from groups historically excluded from science and medicine have focused on providing academic and social support through programs that provide mentored experiences. However, for students in science, technology, engineering, math, and medicine (STEMM) majors, students from underrepresented groups tend to leave science-oriented programs at higher rates than students who are not underrepresented. As such, they are also underrepresented in medical fields, including academic medicine. Insight into how pathway programs contribute to addressing this issue is critical.

Methods

This study took a qualitative approach to investigating the experiences of pathway program administrators in academic medicine. Interviews were conducted with 12 program administrators working on 8 different programs throughout the country. Interviews were analyzed using directed content analysis while also allowing for the development of new themes based on the data.

Results

The codes were organized into 6 overarching themes: mentorship, student engagement, determining program success, administrative time and program logistics, diversity and inclusion, and transition to virtual learning (due to COVID-19). Within each of these themes, program administrators described challenges along with some strategies programs employed to overcome these challenges.

Conclusions

The greatest overall challenge described was finding and sustaining relationships with faculty and nonfaculty mentors. To address this issue, many programs have worked within their institutions to incentivize this work. For student engagement, program administrators reported issues with tailoring to skill sets and interests of multiple students while still fostering community. Program administrators have also expanded definitions for determining program success. Program administration is a challenge, and more support staff or time to devote to these programs is often needed. Diversity challenges encompass recruiting faculty and students from groups underrepresented in STEMM and the logistics of getting all necessary accommodations for students. Finally, transition to virtual learning, due to the COVID-19 pandemic, brought about challenges and opportunities.

Introduction

Diversity in the medical workforce can improve both the experiences of patients and their care.^1,2 Pathway programs designed to recruit and retain students from groups historically excluded from science and medicine have focused on providing academic and social support through programs that provide mentored experiences. ³ Recent work has found that mentored research experiences for underrepresented undergraduate students have a significant impact on diversifying the scientific workforce. ⁴ Research experience, mentorship influences, and community involvement have been found to impact science self-efficacy and identity as a future scientist, which in turn may impact commitment to a science-related career. ⁵ However, for students in science, technology, engineering, math, and medicine (STEMM) majors, students from underrepresented groups tend to leave science-oriented programs at higher rates than students who are not underrepresented.^6,7 As such, they are also underrepresented in medical fields, including academic medicine. ⁸ Insight into how pathway programs contribute to addressing this issue is critical to mitigating the loss of intelligent and talented individuals in STEM and clinical research fields.

Pathway programs are typically designed to recruit and retain students from underrepresented groups, such as Black, Indigenous, and people of color students, low socioeconomic status students, and first-generation college students, into STEMM fields. These programs become particularly prevalent in high school and early college and typically focus on socialization into the fields of science and medicine, research and clinical skill development, mentoring relationships, peer group development, and development of academic skills.^3,9 Numerous studies have investigated and found evidence for the positive impact of these programs on students’ intent to pursue science and medicine, and their confidence and self-efficacy for pursing these fields. ¹⁰ Additionally, these programs have been found to improve persistence in training pathways into STEMM fields.^4,11 Further research has found evidence for particularly positive impacts on STEMM interest and persistence on students from varied demographic backgrounds historically excluded from and therefore underrepresented in STEMM. ¹²

To understand the impacts of these programs on students, we can consider both prior work on the effectiveness of pathway programs as well as a conceptual model of the type of learning environment in which many of these experiences take place. DuBois et al ¹³ conducted a literature review of pathway programs, their success, and the elements that contributed to their effectiveness. They found that these programs can be helpful for cognitive, socioemotional, and identity development, but that these effects are moderated by program practices and mentoring relationships. Thus, effective program design and implementation is critical to success. Additionally, Gruppen et al’s ¹⁴ framework on medical and academic learning environments, the context in which many pathway programs take place, can also help us to understand how the elements of these programs interact. According to this framework, personal learner factors, social relationships and interactions, physical and virtual spaces, and organizational structures all interact to impact the learning experience. Therefore, we can use this framework to reflect on the elements that facilitate learning in pathway programs and elements that may present challenges for learning.

To allow these pathway programs to function and serve students, program directors and coordinators are essential to oversee the continued development and administration of these programs. Despite this, there is little work addressing challenges experienced by these individuals and how they are mitigated, or not, as programs are run. This is particularly true for programs serving K-12 students, for which funding, especially at the federal level, can be more difficult to obtain. Further, there is little discussion about what is needed to mitigate these issues at a systems level. Therefore, the goal of this work is to explore perceived impacts of the 8 programs in the areas described above from the perspectives of the program directors and coordinators. The work addresses the question: What are the challenges in program implementation and how have those challenges been addressed by program administrators of programs serving high school students? This will allow for a better understanding of shared challenges, possible solutions, and directions for future work.

Figure 1.

Diagram for Interview Enrollment and Completion. *Diagram for enrollment and completion based on COREQ guidelines.

Methods

Data collection

This project utilized the Centers for Disease Control framework for program evaluation in public health. ¹⁵ This includes gathering information from program directors and coordinators on the needs that the program is meeting, the activities involved in the programs, the resources that are a part of these activities, and the challenges and opportunities in implementation. This work was part of a larger program evaluation project that engaged in an in-depth process evaluation to illuminate the internal dynamics of the programs’ operations. ¹⁶ The consolidated criteria for reporting qualitative research checklist was completed for this study and can be found in Supplemental material.

Nature of study

As part of evaluating the processes and outcomes of the 8 programs, we conducted a qualitative study that included semistructured interviews with program directors and coordinators of the Clinical Research Continuum: High School to College program. Survey data were collected from alumni of 8 high school-to-college pathway programs throughout the United States, all of which received funding from the Doris Duke Charitable Foundation. ¹⁷ This study was reviewed by an institutional review board (IRB) and approved as exempt from further review (ID: 2020-0985).

Place

The programs were located in cities in the West Coast, Midwest, and East Coast areas. Eligible students included high school students from populations meeting the National Institutes of Health's definition of underrepresented in the extramural scientific workforce. ¹⁸ Most programs were conducted as summer research programs between students’ junior and senior years of high school. All programs provided students with a mentored experience in clinical research. This study was reviewed and deemed exempt by the UW-Madison IRB Office.

Duration

We reached out via email to the directors and coordinators of all 8 programs that were part of the foundations’ endeavors to provide students with engaging, valuable experiences in the sciences that cultivate their interests toward careers in that area. The PI of the study (SJ) sent recruitment emails inviting the directors and coordinators to participate in a 45 min semistructured interview over Zoom discussing the factors that influenced the success of their program. Six programs participated in the interviews. Interviews were conducted between January and July of 2021. Authors obtained verbal informed consent from the participants before proceeding with the recorded interview, as was approved by the IRB.

Participants

We interviewed a total of 7 directors and 5 coordinators, out of a potential 9 directors and 8 coordinators. One program had 2 directors (see Figure 1). The program director interviews mainly discussed program goals, strengths and challenges, as well as the benchmarks for success. Program coordinator interviews provided a deeper dive into the components of program curriculum and mentor recruitment and training activities. Interview questions were based upon the initial assumptions about the underlying mechanisms at work to facilitate programs’ success. Interviews were semistructured, with questions asked of all program directors and coordinators, but still allowed the freedom for the interviewer to follow up on programmatic differences that came up during the interviews. The interview guides can be found in the Appendix. Exclusion criteria were not being program directors or coordinators of Doris Duke Charitable Foundation funded programs.

Data analysis

All interviews were audio-recorded, transcribed, de-identified, and uploaded to NVivo Software for data management. We analyzed the data using directed content analysis ¹⁹ while capturing emergent themes during the process. The analysis team consisted of a professor in education research and development (SJ) with a PhD, a health services researcher (SR) with a BS, and a qualitative scientist (EA) with a PhD. We developed an initial codebook based on the meta-analysis of constructs demonstrated to influence the mentoring programs for youth in DuBois et al. ¹³ We characterized the emergent constructs as any topics within the data that were not captured by DuBois' framework and incorporated into our codebook as new codes. All members of the research team (SJ, SR, EA) consensus coded 5 transcripts (10% of the total) independently to test the initial set of codes. After each transcript was coded independently, the group convened to discuss themes, adjudicate differences, and determine code definitions. Coders’ memos and annotations were reviewed to facilitate group discussions. Once the codebook was refined and finalized using the technique of constant comparison, ²⁰ each coder coded approximately one-third of the remaining transcripts independently. We met regularly as a group to discuss segments for which codes could not easily be applied. Through discussion, we achieved coding consensus. Due to the small number of potential interviewees, data collection was based on the number of program directors and coordinators who agreed to be interviewed and was not based on saturation.

To reach higher-level concepts, all team members created narrative summaries organized by the main codes and the subcodes. Each code summary was reviewed by the team and discussed during regular team meetings. For this analysis, we examined all data associated with codes relating to program Challenges, Mentoring, Program Structure, Program Adjustments, and Measuring Program Success.

Results

The codes were organized into 6 overarching themes: mentorship, student engagement, determining program success, administrative time and program logistics, diversity and inclusion, and transition to virtual learning (due to COVID-19). Within each of these themes, we describe challenges along with the strategies programs employed to overcome these challenges.

Mentorship

Mentorship encompassed challenges related to finding and retaining effective mentors for high school students in the programs. A challenge common to programs is recruiting good mentors with time and availability. A program director said, “So, I think that's probably our biggest struggle now is getting enough mentors to have rich experiences for the students.” Programs have attempted to address this challenge with strategies such as engaging nonclinical mentors, in fields such as biology, chemistry, sociology, and engineering, as well as by limiting the “asks” of mentors. According to one program coordinator, “Since most mentors are volunteering, we try to give them as little to do as possible.”

Another challenge identified by program administrators is training for mentors. This issue is exemplified in the statements that, “The least satisfying part, really accurately identifying who can potentially be a good new mentor to a 17-year-old,” commented a program director. “Faculty is gonna say they’re busy. They’re ‘too busy’ you know to always do training,” according to a program coordinator. Programs have addressed this issue by offering structured programs of mentor and mentee training programs utilizing feedback from mentors on the kinds of skills they feel they need to develop as well as those they would like to see in their mentees. One program coordinator talked about how their mentor relations office would assist them in using this information to build the training programs needed by their mentors and mentees. “…they're going to be helping us in developing the curriculum specifically focused on the young learner and how to engage a high school student in research and the typical things that we run into about finding the balance with your young students.”

A third challenge for programs around mentorship is appropriately engaging nonfaculty mentors. All programs discussed engaging nonfaculty mentors, often trainees themselves or alumni of the programs, in effective ways. A major challenge to this was making sure that these trainee mentors had a valuable experience mentoring the students but were not over-burdened by their role. Once program experienced issues with incorporating medical student mentors. “And while some medical students are great mentors, some of them aren’t prepared to mentor a high school student, so the degree of autonomy and engagement was wildly different,” offered one program coordinator. Programs have worked on better supporting their nonfaculty mentors by putting more structure around mentoring process with clear expectations for the roles of mentors and students. Many have also started selecting alumni for which they think the experience will be most valuable for them and the students, as described by a program director, “And then, of course, we hand pick the ones who are staying in science and college, so they can get that perspective and what it's like.”

A final notable challenge is incentivizing mentors for sustained participation. Major issues around this have included lack of funding or formal benefits by mentors and lack of formal recognition from the institutions that faculty are taking the time to mentor in these programs. This is illustrated well by a comment from one of the program directors, “So, I would say that the funding aspect of it is hard and, and then just continuing to encourage mentors to participate and give them some, some perks along the way, you know, training and allowing them to see the trajectory of the students and so forth.” Programs have worked to provide nonmonetary types of incentives for mentors, including helping them to see the impact that they make on students.

Student engagement

Student engagement in this context refers to keeping students involved and active with the elements of the programs, including didactic sessions, research, and other opportunities. One challenge within this is developing programming that represents the interests and skills sets of students. For example, one program had to make a change in the way they do their cardiopulmonary resuscitation (CPR) training due to the lack of prior knowledge of the learners. A program coordinator commented that, “We used to use internal CPR services, but those are really aimed at faculty nurses who’ve already done it before and the students were a little lost. So we brought in somebody who's used to teaching people from you’ve never done this before.” It can also be difficult to engage varied students’ interests when new mentors are difficult to acquire, as noted by one program director, “We go back to the same mentors every year. The good news is they know exactly who we are, exactly why we’re placing students with them, exactly what those students need. But when you’re relying on the same group of people every year that can be limiting.” As such, even though program administrators acknowledge that it is beneficial to have dependable mentors each year who are familiar with the program and types of students served, recruiting new mentors doing different types of research from the regular pool of mentors can be a challenge.

A second challenge is supporting and developing multiple skills of learners. This includes acknowledging the needs of younger learners when learning about and engaging in the research process, which encompasses not only learning about how to do research but also the skills to engage professionally with research. For example, learning how to function as part of a collaborative research team, communicating research findings, and building relationships with mentors are important components of academic medicine. Programs have made this a specific focus, as quoted by one program director, “We focus a lot more on professional development, which it really used to be just a pure science program. And now we've talked about how that's just one aspect. And if we don't engage the entire learner that they're not going to get as much out of the program.” Many programs feel this is important for helping to fight imposter syndrome. “One of the biggest things is imposter syndrome. Right? And even, how am I here? How do I belong here? And then you’re looking at yourself, right? And measuring yourself against the other students who just seem kind of so on it,” stated a program coordinator. Combating this imposter syndrome tendency to question one's abilities, particularly in comparison to others, is one of the reasons developing professional skills in research is seen as essential in these programs.

One way that program administrators have worked to address the challenges of student engagement is by developing program community. Developing community is seen as a way to help students stay actively involved in STEMM during the program and as program alumni. As such, a sentiment shared by programs is captured by one program coordinator's statement, “And so we’ve tried to bridge that gap because one of the most important things for our program, also is networking, right? Is having those peers because it's hard to be persistent in something when you’re the only one doing it, right? And everyone else is doing other things. And so just trying to create that sense of community within that cohort, that helps, that creates friends. That creates a peer group. That creates colleagues that will help them along their journey like to keep pushing forward in addition to their mentorship and in addition to our program's continued support.”

Determining program success

Determining program success relates to understanding the outcomes of the students who participate in the program. Within this, collecting evidence of program impact is a major hurdle for programs. Some programs have enlisted external evaluators to help them better understand their programs. All program administrators discussed the need to expand definitions of program success beyond whether students end up going into STEM or medical fields. One program director captured this sentiment well, “I think our success, do we get them into college, and if they go into STEM fields, great. That's awesome. But I think number one, we should expand our definition of what relates to STEM or healthcare even to include a lot of the social sciences. I mean, we have some kids have gone into social work, that are psychology, nutrition, things that sort of fall a little outside of our, like, traditional clinical definition that are so important to the healthcare system. And I think that deserves full credit.” Other measures of program success are often collected by programs as they have developed better understandings of what students are getting. This includes things like STEM identity as noted by a program director, “Wow! We didn’t even think about what STEM identity is. So we have now started to make more of an effort. We’ve actually started a baseline survey with the incoming sophomores, asking them several questions about STEM identity.”

Program administration

Program administration refers to challenges with the logistics involved running the program. This includes administering the program with limited time and resources. Program administrators typically have multiple responsibilities in addition to their work with these programs, so they often are “challenged in terms of what they can accomplish through their role,” according to a program director.

Diversity and inclusion

Diversity and inclusion in this context refers to issues around the diversity of students and mentors in the program. One of the challenges discussed is making appropriate accommodations for students. As one program coordinator said, “Getting all of the accommodations from the University of getting an interpreter [for a deaf student] or getting the special knocker on the door in their dorm. So any time, the more we need from the University, the harder the job becomes. And especially the more something is unusual.”

A second challenge for programs includes lack of diverse faculty and worries about placing undue burden on faculty from underrepresented groups. As one program director said, “When I’ve approached faculty of color… They tend to be more junior, right? They tend to have less funded projects for my students to work on.” Program directors often adopt an approach of diversity in terms of the fields mentors are in, “We try to do it more off of the students’ interests, and match them up with somebody that's actually doing what they think they're interested in.”

Another challenge is reaching diverse students to enroll in the programs. One program administrator discussed their program's particular difficulty with reaching African American male students. “A lot of African American males, either have other passions, you know. So they're interested in, in finding success in in sports, or that they are intimidated by the, by what they see our program represents–is what our focus groups are telling us,” according to a program director.

Transition to virtual learning

Finally, transition to virtual learning encompassed the swift transitions many programs made to virtual learning due to the challenges of COVID-19. A first challenge experienced by programs is creatively supporting students in a virtual space. In particular, program administrators discussed ways that they tried to develop community virtually. One idea brought forth by a program coordinator was the following, “So, for example, this year we tried to do some events for the virtual program. So, like watching, you know like streaming some sort of movie night or something like that.” A second challenge is maintaining program quality. Program administrators discussed standardizing the program by providing more guidance for mentors, for example a program coordinator commented, “…we’ve realized that they really felt like they lacked guidance in the virtual setting…” Many administrators also acknowledged the affordances for expanding the programs and opportunities for engaging with students created by shifting to a virtual space. One program director expressed this idea particularly well in their interview, “And this, in the virtual world I think is where we have the opportunity to do this easier than we normally would. And we start to see this, right. I’m still working with the student I worked in the summer. And she's taking care of her little siblings while we’re working together on research so trying to engage the younger kids who are sitting behind them or you know talking or communicating more with the guardians is an opportunity that we haven’t seized yet.”

Discussion

Our analysis of the interviews with program directors and coordinators allowed us to organize our findings into 6 overarching themes: mentorship, student engagement, determining program success, administrative time and program logistics, diversity and inclusion, and transition to virtual learning. In regard to mentorship, the greatest overall challenge described was finding and sustaining relationships with faculty and nonfaculty mentors. To address this issue, many programs have worked within their institutions to incentivize this work and identify nonfaculty and faculty mentors for whom this work may be particularly meaningful. For student engagement, program administrators reported issues with tailoring to skill sets and interests of multiple students while still fostering community. This challenge has been a focus of program improvement in the form of making program changes to program content and social structures based upon feedback from students who have completed the programs. Program administrators have also worked to expanding definitions for determining program success, which now include the development of academic skills and identities, in addition to STEMM degree completion. Program administration is often a challenge, given that most program directors and coordinators have multiple other roles and obligations they are juggling. More support staff or time to devote to these programs may be needed, particularly with the regulations involved in working with minors and having them on academic campuses or at health institutions. Diversity challenges encompass recruiting faculty and students from groups underrepresented in STEMM as well as the logistics of getting all necessary accommodations for students. Finally, transition to virtual learning, due to the COVID-19 pandemic, brought about challenges with student and mentor engagement as well as issues with ensuring programming quality, but program directors and coordinators felt that this change also created space for innovation and engaging students who may otherwise struggle to have the time to be part of the programs. These ideas about virtual learning have been supported by recent work looking at the impact of the switch to virtual mentoring during the COVID-19 pandemic and how this created not only challenges but room for realigning and reimaging these relationships. ²¹ These were the major challenges and solutions highlighted by our program administrators.

We can understand these findings in relation to the elements essential for effective learning environments. In Gruppen et al’s work, the personal, social, physical and virtual spaces, and organizational structure are highlighted as key components of academic spaces in which learning often takes place. ¹⁴ For instance, engaging all students and measuring their individual successes highlights the personal component, social elements are captured in the importance of mentoring relationships and facilitating community, physical and virtual spaces are highlighted in discussions of transitioning to virtual learning and the affordances and constraints that come with learning in that space. Lastly, many of the challenges highlighted by program administrators refer to elements of organizational structure, namely ensuring support for participation of diverse students and mentors and allocating time and funds for program administration. Understanding and reflecting on pathway programs, including their challenges and associated solutions, within this framework allows those working on those programs to identify components of the learning environment that may be particularly strong, or alternatively, may need attention. Further, understanding not only that these issues occur but perspectives on why they occur has been discussed as critical to improving students’ persistence in STEMM fields. ²²

Given that ours is the only study of which we are aware that highlights the perspectives of pathway program coordinators and directors for a program focused on high school students, our study adds this perspective to the literature, as work examining these programs is typically from the perspective of the participants. ²³ We were able to explore the perspectives of program administrators from 8 programs with similar goals that serve different populations of students. Common challenges raised were maintaining engagement for all students in program experiences and activities, developing community and engaging enough mentors, and supporting learners and mentors during virtual learning. Addressing these issues is important given that previous studies have found that student engagement in scientific research and activities can help foster individual interest and participation in science outside of pathway programs and facilitate persistence in STEMM.^24,25 Some successful strategies for addressing these issues included changes the ways in which program successes were defined and assessed, facilitating community within the program, and providing incentives for mentors, including nonmonetary incentives. Other work has found that trainees in programs for more advanced undergraduate and graduate students have discussed additional potential concrete actions for improving programs, including greater peer mentoring and support, supplemental mentorship, and required mentor training, ²⁶ all of which are strategies considered and at times utilized to the best of their ability by the administrators of these programs. In addition to trainees’ voices, understanding the perspectives of these program administrators is essential given that coordinated plans across programs designed to support students across their educational pathway are critical and have been shown to be successful in expanding biomedical workforce diversity. ²⁷

It is clear from the conversations with program administrators that they are designing their programs to help their students understand the reasons for the formal learning they are doing in the programs, to grant students opportunities and access to the scientific community, and engage students in STEMM content, all of which have been found to support an interest in STEMM. ²⁸ When thinking about how best to support program administrators in their enactment of pathway programs, it is essential to consider all aspects of learning environment. One area in which those not directly responsible for the administration of these programs can lend support is in the instructional culture. We need to make sure that programs such as this, which support diversity and inclusion in STEMM are well-funded and that faculty are encouraged and incentivized to participate. Indeed, some institutions now place a greater emphasis on equity, diversity, and inclusion work as part of promotion. However, transformational leadership will be required to make this the norm and ensure programs such as these are top of mind for institutions. ²⁹ Furthermore, robust infrastructures and training resources have been built to expand equitable access to research mentoring, which could be utilized in this work at both the micro program level and the institutional macro level. ³⁰

Limitations

This is a small study with a specific population of program directors and coordinators who, although their programs different somewhat in size and focus, are all funded by the Doris Duke Charitable Foundation, and therefore may not be representative of directors and coordinators of programs with other funding mechanisms. Further, although our interview guide was reviewed by a qualitative research group, which included directors of other types of programs, given the already small pool of program directors and coordinators, it was not pilot tested with any of the target group.

Conclusion

This work raises further questions and areas for future work. Some additional questions include: What did alumni perceive as challenges? and How do organizations react to these challenges and how might some be solved through organizational changes? Currently, these programs are continuing their great work while learning from each other and addressing challenges and implementing changes based on what has been learned. This research adds to an understanding of the complex learning environments in STEMM research pathway programs from the perspective of the administrators responsible for developing and facilitating these experiences.

Supplemental Material

sj-docx-1-mde-10.1177_23821205231189981 - Supplemental material for Engaging the Entire Learner: Pathway Program Administrators’ Experiences of Providing Students with Research Experiences in Academic Medicine

Supplemental material, sj-docx-1-mde-10.1177_23821205231189981 for Engaging the Entire Learner: Pathway Program Administrators’ Experiences of Providing Students with Research Experiences in Academic Medicine by Sarah Jung, Alexandra A Rosser and Esra Alagoz in Journal of Medical Education and Curricular Development