Abstract
Problem
Health systems science (HSS) curricula in medical schools facilitate an understanding of social determinants of health (SDOH) and their impact on health outcomes. After implementation of an experiential, patient-centered program based around SDOH screening, however, our medical college noted poor student receptivity and engagement. In order to improve the program, we chose a design thinking approach based on the perceived value of actively engaging learners in the design of education. The role of design thinking in curricular quality improvement, however, remains unclear.
Intervention
We sought to determine if a current educational model for SDOH screening could be improved by reforming the curriculum using a design thinking workshop involving student and faculty stakeholders.
Context
The current study is a retrospective analysis of first-year medical student, end-of-year evaluations of the Clinical Experience (CE) program at the Sidney Kimmel Medical College before (2018-19) and after (2019-20) implementation of the design thinking workshop and subsequent curriculum changes
Impact
Overall positive results significantly increased across all survey questions after the curricular intervention (p < 0.01), indicating increased student satisfaction with the revised curriculum.
Lessons Learned
Few studies assess outcomes of design thinking-driven curricular changes. The current study of an SDOH screening program details the implementation of initiatives that originated from a design thinking sprint and assesses program evaluations following these curricular changes. Most of the well-received curricular changes concerned improvements in student training, patient screening and follow-up, and the leveraging of existing technology. The study reinforces the importance of co-creation among stakeholders when redesigning medical curricula.
Introduction
Social determinants of health (SDOH), defined by the World Health Organization as “the conditions in which people are born, grow, work, live, and age,” are known to have a disproportionate impact on patient health outcomes, yet are only recently being formally taught in medical education.1,2 Using data from the 2015 County Health Rankings, Hood et al 3 estimated that 80% of overall health outcomes may be explained by the relative contributions of socioeconomic factors, health behaviors, and the physical environment. Despite this recognition, patients are infrequently screened for SDOH in clinical settings. In a recent cross-sectional survey analysis of responses by physician practices and hospitals, Fraze et al 4 assessed screening for SDOH in five domains: food insecurity, housing instability, utility needs, transportation needs, and experience with interpersonal violence. These domains were chosen based on their inclusion in the Centers for Medicare & Medicaid Services’ Accountable Health Communities model. The authors found that only 24.4% of hospitals and 15.6% of physician practices reported screening for all five domains. Eight percent of hospitals and 33.3% of physician practices reported no SDOH screening at all.
SDOH screening may be improved by formalized training in undergraduate and graduate medical education.5,6 The Association of American Medical Colleges (AAMC) has consistently advocated for inclusion of SDOH learning into the undergraduate medical curriculum, and to do so, has suggested utilization of a Health Systems Science (HSS) curriculum.7,8 While the HSS curriculum at our medical college is generally composed of didactic lectures and patient panels, undergraduate medical education (UME) leadership sought a more experiential format to teach students about SDOH. However, there is limited literature regarding the training of medical students in the application of these particular HSS principles in a clinical environment.9–11 The goal was to design a program that would present students with opportunities to engage (and assist) patients with unmet social needs in a clinical environment, while simultaneously providing value to the hospital. The program sought to offer a “value-added medical education,” described by Gonzalo et al (2021) as “experiential roles for students in practice environments that have the potential to positively impact individual patient and population health outcomes.” 12 In order to provide students with an experiential, patient-centered application of the SDOH foundational domains, the Sidney Kimmel Medical College (SKMC) created the Clinical Experience (CE) program. The program, designed and implemented by UME leadership, involves pre-clerkship students screening patients for SDOH using a modified Health Leads © (Health Leads, Boston, MA) screening tool (Figure 1). With the assistance of community health workers (CHW), the students determine the appropriate community resources to address patients’ unmet social needs. Despite a seemingly sound theoretical framework, in its first two years the CE program was rated poorly by students and it was felt that the course needed to be reassessed. In order to better understand the students’ concerns and identify opportunities for improvement, it was determined that curricular change would require the diverse input of the main program stakeholders. Based on the perceived value of engaging learners in the design of education (co-creation) 13 , a design thinking methodology was chosen to drive the curricular changes.
Modified health leads© questionnaire used for patient screening.
Design thinking is a well described method for process improvement in industry and has been increasingly utilized in healthcare and medical school settings. It helps to elucidate inefficiencies and discover opportunities for change through engagement of stakeholders. The methodology has been employed across a variety of fields, including education and curricular reform, to address complex problems.14,15 The current study set out to determine if application of a design thinking approach to the CE curriculum would result in improved program ratings within the student course surveys. Our hypothesis was that the design thinking-driven changes to the program would result in significantly improved student satisfaction, as measured by end-of-year survey data.
Methods
Study Design, Setting, and Population
The current study is a retrospective analysis of first-year medical student evaluations of the CE program at SKMC following implementation of various design thinking-driven curricular changes. The setting was an allopathic medical school located in an urban, academic medical center. Of note, all 274 first-year medical students participate in the CE program, during which they are deployed to various clinical environments across the health system (eg, outpatient clinics, emergency departments, and short-stay inpatient units) in order to perform the SDOH screenings. In advance of their CE sessions, students take part in didactics that introduce concepts regarding the structural barriers and hurdles faced by largely minoritized and socially oppressed patient populations. These didactics include – but are not limited to – lectures on structural racism, patient panels addressing SDOH, small group discussions, and scenario-based questions on exams. At all times during their CE sessions, students work in conjunction with, and under the direct supervision of, trained CHWs who are there to educate and advise students who have questions regarding specific resources or patient circumstances. This study was reviewed by the institutional review board and was determined to be exempt.
Study Protocol
Design thinking is a human-centered process, described by Tim Brown as a “methodology that imbues the full spectrum of innovation activities with a human-centered design ethos.” 16 At its core, the methodology promotes empathy by encouraging open communication with relevant stakeholders to understand their experiences and define issues based on their experiences. Once issues are identified, the focus shifts to the problem solution space, in which ideation sessions lead to solution brainstorming. 17 Design thinking emphasizes a bias toward action, in which suggested solutions evolve into rapid prototyping and cyclical redesign based on the stakeholder feedback. Design thinking can be summarized into three main phases of creative problem-solving: Observe (ie, looking, listening, and gathering insights), Imagine (ie, generation of ideas, sorting, and analogies), and Make (ie, rapid prototyping, storyboarding, and presenting to end-users; Figure 2). 18
Health design thinking methodology. [Ku B, Lupton, E. Health Design Thinking: Creating Products and Services for Better Health. first ed. MIT Press; 2020].
In August 2019, a “design thinking sprint” (three-hour design thinking workshop) was conducted with the main program stakeholders: the CE director (Social Worker), the education programs administrator, the 5 CHWs for the program, a second-year medical student who had taken part in the CE program the year prior, and 2 third-year medical students (for a total of 10 sprint participants plus the sprint facilitator). The design sprint was organized to follow the three main phases of design thinking as described above: observe, imagine, and make (Figure 2). The session was composed of the following activities over three hours:
Welcome and introduction lecture (30 minutes)
Ice-breaker (your name, what you do, and describe something awesome that you’ve done [or has happened to you] in the last month) Design thinking didactic
Brief history of design thinking The decision thinking ideology and process (with acknowledgement of the work of IDEO and the Stanford d.school) Discussion of “design” in healthcare
Examples of poor design in the clinical environment Design challenge assigned – “Redesign the Clinical Experience program” (10 minutes)
Empathy question burst – instructions:
“Think about the CE program - consider EVERYTHING from start to finish” “Write down as many questions as you can related to these thoughts” “You will share your insights with your team” Gathering insights – 3 rounds of one-to-one interviews (50 minutes)
Round 1 focuses on the CHW experience Round 2 focuses on the medical student experience Round 3 focuses on the administration of the program Interview debriefs (15 minutes)
Teams sort and categorize the ideas and insights generated from the interviews Theme sorting using Post-It notes Point-of-view (POV) statement generation (13 minutes)
Brief didactic on problem definition Teams create multiple POV statements based on the interviews “How Might We” (HMW) generation (7 minutes)
Brief didactic on creating a HMW question Teams create a single HMW question Ideation (17 minutes)
Brief didactic on effective brainstorming/ideation Teams ideate solutions based on the defined problem(s) Storyboarding (13 minutes)
Brief didactic on the role of storyboarding as a form of prototyping Teams develop a storyboard to explain their solutions Bodystorming (18 minutes)
Brief didactic on the use of scenes, props, and roles to “bodystorm” (act out through role-play and simulation) their solutions Teams create a bodystorm to explain their solutions Final presentation – teams present their solutions to the larger group (7 minutes)
Figure 3 demonstrates a storyboard generated from the design sprint. The slides and session timeline/resources used for the design sprint may be found in
Example of a storyboard from the design thinking sprint.
Following the design sprint, all proposed changes were presented during sequential feedback sessions (60-70 students per session) with the entire second-year student body (n = 274); all of whom were students who had recently completed the CE requirement. During the feedback sessions, if the group consensus regarding the proposed change was positive based on a thumbs-up/thumbs-down vote (>75%), the idea was adopted. If the vote was mixed or negative, then the idea was discarded. A final list of proposed changes was generated based on student votes. These changes were then categorized into themes based on teams analyzing the central focus of each proposed change and grouping those proposed changes with common central foci into a singular theme. These changes were implemented for the upcoming CE year (2019-2020).
End-of-year, anonymous survey responses by first-year medical students were compared in the academic years immediately before (2018-2019) and after (2019-2020) the design-driven curricular changes (Table 2). Of note, the surveys were optional for students. They prompted students to respond to various statements regarding the CE program based on a 5-point Likert Scale (1 = Strongly Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, 5 = Strongly Agree). Survey questions were phrased so that a response of “Strongly Disagree” or “Disagree” represented a negative response and a response of “Agree” and “Strongly Agree” represented a positive response. The three items common to academic years 2018–2019 (pre-intervention) and 2019–2020 (post-intervention) that we compared in our analysis were:
“The Clinical Experience course has provided a valuable opportunity for me to interact with patients.” “The Clinical Experience course has helped me better understand how social determinants of health impact patients' health and wellbeing.” “Working with a Community Health Worker has helped me learn about working with interprofessionals in a healthcare setting.”
Statistical Analysis
Standard descriptive statistics were performed on all survey questions across both years using IBM SPSS Statistics for Macintosh, Version 27.0. Comparison of results for the three survey questions that were held constant between years was then performed using chi-square tests. P-values <.05 were considered statistically significant.
Results
Over 30 proposed changes and initiatives were generated from the design sprints. These changes were presented during sequential feedback sessions with medical students who had recently completed the CE program and voted upon. Ultimately, 19 of the proposed changes and initiatives were incorporated into the CE program for the 2019–2020 academic year. The 19 changes fell into one of four categories: student education, student clinical experience, community health worker experience, and SDOH screening workflow (Table 1).
Design-driven changes (19) implemented in the 2019-2020 CE program.
After incorporation of the proposed changes, CE students were surveyed, and results were compared to the survey from the prior year. CE program surveys were completed by 142 students in 2018–2019 and 171 students in 2019–2020, corresponding to survey response rates of 51% (142/274) and 63% (171/273), respectively. After the intervention, a significant trend was noted in students rating the course more positively across all questions. For example, after the intervention, 25% of students stated they strongly agreed that “The Clinical Experience course has provided a valuable opportunity for me to interact with patients,” compared to only 6% in the preintervention cohort (Table 2). Analysis of the three questions common in both surveys demonstrated a similar trend of positive improvement (Figure 4; p < 0.01).
Distribution of Likert responses to survey questions that were common to academic years 2018-2019 and 2019-2020 (before vs after the intervention).
Survey data by academic year.
Discussion
In the current study, we found that student evaluations of the CE program significantly improved after implementation of the ideas and initiatives that originated from a design thinking sprint. While these findings are best reflected by the survey items that were consistent across the academic years, the overall trend in the composite evaluations suggests improvement in student perceptions of the program, as well. Notably, most of the implemented changes concerned student training/preparation and improved screening and follow-up of patients. In addition, a number of these changes leveraged existing technology (eg, the electronic health record) and web-based resources (www.jeffce.com website).
The choice of design thinking to reimagine the CE program was based on its core principle of “co-design.” This follows a growing appreciation for active learner involvement, or co-creation, in the design and development of medical education. This concept is expanded upon by Englander et al 19 , who describe a “coproduction model” for health professions education in which they suggest that teachers and learners transition to “more fully shared processes of goal setting, curricular design, and learning assessment.” Design thinking embraces many of the principles of co-creation and co-production, and a growing body of literature has highlighted the value of design thinking methodology in driving curricular change.15,20 For example, McLaughlin et al (2019) 21 published a qualitative review of the literature addressing the role of design thinking in healthcare, in which they report a number of efforts to teach design thinking within the health professions. Sandars & Goh (2020) describe the potential of the design thinking process to address the complex problems of healthcare 15 , while Gottlieb et al (2017) speak to its application in medical education curricular development. 20 At Harvard Medical School, a student-centered design process with co-creation teaching sessions between medical school faculty and students led to pedagogical changes in medical education. 22 The current study, however, is unique in that it measures a specific impact (course evaluations) of design thinking-driven changes to a medical education program.
The design sprint identified major domains that are important to stakeholders at our institution for the CE curriculum. While these insights are specific to a single center, themes identified may be applicable to other institutions. We recommend program directors build on the theoretical framework of the HSS curriculum by considering the student educational and clinical experience, the educational experience for CHWs, and ensuring the workflow for SDOH screening allows for EHR compatibility, tracking, and follow-up. A full framework for optimal course design should be further considered in future studies.
The study has a number of limitations. First, the survey items from the student evaluations were not entirely consistent across the academic years. Specifically, in 2018–19 there were 4 total questions, and in 2019–20 there were 5 total questions. Three questions were common to both survey years. While the current (2019-2020) survey items are thought to best gauge the student experience during CE, it is possible that the updated questions may have simply produced more favorable results. Next, the improved evaluations may not have solely been due to the curricular changes. Student evaluations have steadily improved since the program inception, which suggests there may be other confounders that explain the improvement in survey responses. For example, the recent favorable trend may have been due to improved messaging to students (eg, including them in the program design process) and more clarity regarding student roles and expectations. In addition, Likert scales may have been affected by a central tendency bias (respondents may avoid extreme response categories) or by a social desirability bias (respondents may have sought to portray the program in a positive light). Additionally, the surveys had low response rates (51% and 63% per year, respectively). Future analyses should work to ensure higher student participation in end-of-year evaluations to evaluate student opinion more accurately. We also did not collect demographic data from the student survey respondents, although the sociodemographic makeup of medical school cohorts was relatively similar between years. A limitation of the design sprint was the omission of any members of the community who could represent the patient perspective with regards to SDOH screening. Any future design sprints should certainly include these important stakeholders. Finally, it is unclear whether the entire design thinking process or only specific steps are required to effectively revamp an educational program. It is possible that a well-run focus group with inclusion of relevant stakeholders would produce similar results.
Of note, this study does not fulfill the five criteria to be considered design-based research (DBR) as enumerated by Dolmans & Tigelaar in their Association for Medical Education in Europe (AMEE) Guide No. 60. 23 Instead, it was conducted as a quality improvement project to enhance the student experience in an UME program. To satisfy the AMEE criteria, we would have benefitted from continuous cycles of design, evaluation, and redesign, a mixed-methods approach, and the involvement of a wider team of designers, researchers, and practitioners. Ultimately, a future DBR study is needed to assess the impact of design thinking-driven curricular changes to other UME programs, with a focus on defined outcome measures relevant to students (eg, evaluations, competencies, etc), instructors, and patients.
When we first introduced CE at SKMC as an experiential, patient-centered program that would educate medical students about SDOH, we were surprised by the low student receptivity and engagement despite its seemingly sound theoretical rationale. With the goal of improving the program through the input and involvement of various stakeholders, we sought an approach that would incorporate the principles of co-creation and co-production. While design thinking embraces these principles, it was not clear how the ideas and initiatives that arose from a design thinking sprint would be perceived by students participating in the program. Based on our data, the CE program does appear to have benefitted from the co-design efforts of students, CHWs, and program leadership. Design thinking and other forms of co-creation and co-production present exciting opportunities in UME curricular development.
Footnotes
Acknowledgements
The authors would like to thank all of the Sidney Kimmel Medical College medical students, faculty members, and CHWs for their participation in the design sprint and CE program. Publication made possible in part by support from the Thomas Jefferson University Open Access Fund.
Disclosure Statement
There are no potential conflicts of interest to report.
Ethical Approval
Not applicable, because this article does not contain any studies with human or animal subjects.
Informed Consent
Not applicable, because this article does not contain any studies with human or animal subjects.
Trial Registration
Not applicable, because this article does not contain any clinical trials.