Abstract
Objectives:
Public health education must respond to 21st-century public health challenges in an ever-evolving landscape. We describe implementation and educational outcomes of the Columbia University Master of Public Health (MPH) Core (hereinafter, Core) curriculum since its inception.
Methods
This retrospective evaluation combined 6 years (2013-2018) of student survey data collected from students (N = 1902) on the structure and delivery of the Core curriculum to quantify implementation, student experience, and learning outcomes, both during study (Core Evaluation Survey [CES]) and after graduation (Graduate Exit Survey [GES]). We used χ2 tests and analysis of variance to compare outcomes across years, and we used McNemar tests to compare differences in outcomes from the same students at different time points.
Results
Of 1902 respondents to the CES, 1795 (94.4%) completed the Core curriculum. During the study period, 81.7% of students were able to integrate concepts across Core curriculum modules with ease; postgraduation, a similar proportion of respondents were able to apply Core curriculum content to departmental and certificate coursework and applied field experiences. On-time graduation rates were high (range, 85%-93%).
Conclusions:
The high percentage of students who reported their ability to integrate concepts and who completed the Core during the study period likely reflected changes to teaching team structures, training, attention to inclusion and equity, and collaboration to implement active learning strategies. The Core curriculum meets its intended goals by providing critical learning abilities to support ongoing interdisciplinary work.
Public health leaders must be prepared to meet increasingly complex and rapidly changing public health challenges during long careers and to lead in a new era of a lifecourse approach to prevention. The Columbia University Mailman School of Public Health (CUMSPH) undertook a redesign of its master of public health (MPH) curriculum in 20101 -4 with 2 goals: (1) to offer a vision for next-generation public health education that aligns with the school’s strategic vision on societal needs from public health and (2) to educate leaders in domestic and global public health issues, research, and practice, with advanced technical and critical thinking skills.
The schoolwide curriculum was designed to ensure excellence of departmental and disciplinary training, balancing departmental strengths and an integrated core curriculum.1,2 The Columbia MPH Core (hereinafter, the Core) replaced the traditional 5 foundational public health courses taught in isolation with a unified, comprehensive curriculum led by interdisciplinary faculty teams from all 6 CUMSPH departments (Biostatistics, Environmental Sciences, Epidemiology, Health Policy & Management, Population & Family Health, and Sociomedical Sciences).1 -4 The Core was designed to give students a broad introduction to public health, spanning the historical, ethical, biological, social, environmental, and quantitative foundations of population health. All incoming MPH students complete the same coursework in their first semester, addressing the same competencies, in cohorts intentionally comprising students from multiple CUMSPH departments. The Core comprises minicourses (“modules”) nested within courses (“studios”). The current Core has 6 studios, each with 2 or 3 modules lasting 312 weeks: Foundations of Public Health, Research Methods and Applications, Determinants of Health, Public Health Interventions, Global and Development Perspectives, and Health Systems (Figure 1). The Core has a case-based course, “Integrating Science & Practice” (ISP), to foster critical thinking and decision-making skills, and a Leadership course to develop individual and team-based leadership skills.
Structure of the Columbia University Mailman School of Public Health MPH Core curriculum for MPH students entering in fall 2018. The course-module boxes are not drawn to scale. Each module ranged in duration from 4 to 30 sessions, with total sessions in the studio reflected by the points allocated to it (in parentheses). Abbreviation: MPH, master of public health.
In 2016, the Council on Education for Public Health issued revised competencies to support interdisciplinarity and integration of cross-cutting capabilities in public health education.5,6 Other public health institutions have undergone similar revisions to ensure that public health education continues to respond to 21st-century challenges, 7 with several designing a core curriculum that integrates content across departments and disciplines.8,9
Although core competencies in public health are well explained, and some case studies of the initial implementation of core curricula have been published,1 -4,10,11 little has been published on the adaptation and sustainability of such curricula. To fill this gap, we present a process and outcome evaluation during 6 years to demonstrate how an integrated, interdisciplinary core public health curriculum has adapted and how it has sustained and expanded on successes documented in an earlier short-term evaluation. 1 We describe how the curriculum is implemented, outline processes for continuous monitoring, and highlight key student experiences and learning outcomes after graduation.
History of the Core
Core Elements
The Core has 7 core elements: interdisciplinarity, critical thinking, technical skills, global perspective, lifecourse approach, teamwork and collaboration, and leadership. 3 Our evaluation focused on assessing the Core’s interdisciplinary nature, whether and how active learning and assessments engage critical thinking and skill building, and student integration of concepts.
Interdisciplinarity
Reflecting the nature of public health, the Core is interdisciplinary in its content, faculty and teaching structure, and student groupings. It brings together faculty from all CUMSPH departments, representing diverse disciplines such as law, education, history, public health, and medicine.
Integrating Science & Practice
The ISP course organizes students into small (ie, 20-person) groups to engage in case-based learning explicitly integrating Core concepts. Case studies include implications of Zika virus on disability rights and reproductive health and the ethical implications of the US Food and Drug Administration’s blood donor policy. Beyond ISP, Core teaching and activities deliberately draw connections between the Core curriculum and current events. In 2017, the Core established semimonthly integrative panels (ie, Core Conversations) to bring diverse perspectives, stimulate dialogue, and delve into emerging public health issues identified by students and faculty, such as mass incarceration, global mental health, gun violence, and the opioid crisis.
Implementation of the Core and Adaptations
Adaptations to Core structure
The content and structure of the studios and modules are reviewed annually by the Core Advisory Committee (CAC). In fall 2012, 18 modules were grouped within 5 studios. 1 Since the Core’s initial design, 2 all but 2 studios (Foundations of Public Health and Health Systems) have been restructured: 5 modules shifted to different studios, and 1 module was dropped. Adaptations to the Core structure are guided by the CAC to reflect shifts in core competencies in the field and in response to course and Core evaluations. During the study period, the Core shifted from lecture time to interactive laboratories and discussion sections.
Adaptations to Core faculty and teaching teams
Faculty receive salary support commensurate with the number of sessions taught; new faculty are identified collaboratively by studio leads, Office of Education staff members, deans, and department chairs.
Integration
Having faculty teach across multiple Core components achieves efficiency in faculty hiring, training, team formation, and orientation and facilitates integration of content. To support student engagement and the grading of assessments to support critical thinking, the teaching team structure changed in 2016 to have fewer teaching assistants (from 141 in 2014 to 100 in 2017), dedicating more hours and receiving more training, including a summer teaching assistant training institute.
Adaptations to Course Assessments
Course assessments include essays, tests and quizzes, problem sets, and group projects and presentations. All assessments are developed with pedagogical support from the Center for Teaching and Learning. The senior director of educational initiatives, a CAC member, consults with Core faculty to create, update, and refine assessments.
Guided by quantitative and qualitative feedback, the CAC, with the Office of Education and Center for Teaching and Learning, systematically examines the frequency and type of assessments, moving away from learning checks of disciplinary knowledge toward more complex assignments and integrated examinations that test students’ ability to synthesize information. In response to feedback requesting more complex applied assignments, the Core has adapted the frequency, timing, and types of assessments.
Active Learning
Developing learning activities and exercises that stimulate critical thinking for large cohorts of students, compounded by a lack of flexible physical space to support small-group learning, is a challenge. To support active learning, in 2017, Core faculty attended a required multiday training workshop on active learning strategies and meaningful assessments. The Center for Teaching and Learning reviewed all formative and summative Core assessments and provided one-on-one technical assistance on classroom assessment techniques, 12 ways to promote critical thinking, and grading rubrics.
Addressing Issues of Diversity, Equity, and Inclusion
Ensuring a meaningful Core experience centers on ensuring that classroom discussions are inclusive and deal directly and openly with challenging issues such as race, power, identity, and intersectionality. The CUMSPH developed robust curricula for students and faculty to address issues of diversity, equity, and inclusion to help them deepen their understanding of the historical roots of health inequities and engage in critical conversations about social justice. Since 2012, during orientation, students have participated in Self, Social, and Global Awareness (SSGA), a day-long cultural competency program focused on issues of power, privilege, culture, and social justice. 13 Since 2017, all Core faculty have been required to participate in a day-long Inclusive Teaching Institute, which adapts and extends the SSGA model for faculty, focusing on promoting diversity and inclusion in the classroom through pedagogical techniques. Syllabi for all studios are reviewed by the Office of Diversity, Culture, and Inclusion using the School Reform Initiative’s Tuning for Equity protocol (www.schoolreforminitiative.org), which identifies opportunities for incorporating diversity and inclusion into course content.
Core Evaluation Methods
The Office of Education at CUMSPH engages in ongoing monitoring and evaluation of its educational programs, including the MPH Core (Figure 2). At the heart of the monitoring and evaluation process is the CAC, which comprises lead faculty from each studio (studio leads, n = 6), the vice dean for education, the dean of students, and staff members from the Office of Education. The CAC meets biweekly throughout the year to review evaluation outcomes data; discuss Core issues from students, teaching assistants, and faculty; consider pedagogical improvements; and decide on improvements to Core structure and content. The CAC guides short-term improvements and identifies longer-term adaptations needed to ensure the Core evolves to meet changing student and public health needs. Course evaluations for Core modules and studios are reviewed by the CAC, studio leads, and Core faculty at annual retreats. Evaluations of the Core include (1) the Core Evaluation Survey (CES), which is administered the month after the fall semester of the Core and provides data on the Core beyond studio and module evaluations, and (2) the Graduate Exit Survey (GES), which is administered after graduation on graduated students’ view of the Core in the context of the entire MPH program.
Annual Columbia University Mailman School of Public Health Master of Public Health (MPH) Core curriculum evaluation process. Data sources are used in the Core Evaluation Survey (CES) and Graduate Exit Survey (GES). Abbreviation: ISP, Integrating Science & Practice.
Methods
Data Sources and Participants
This evaluation used 6 years of sequential cross-sectional educational evaluation data compiled by the Office of Education: administrative data on student, faculty, and educational program characteristics; CES data (including qualitative feedback); and GES data. Office of Education administrative data provided summary information on course structure, enrollment, faculty, assessments, and aggregate 2-year graduation rates for MPH graduates from 2009 through 2018, not linked to CES or GES data. The Office of Education emailed all students who completed the Core during 2013-2018 (n = 2537) via a link to the CES in Qualtrics 1 month after Core completion, with up to 10 reminders. Response rates to the CES varied from 83.1% in 2014 to 64.6% in 2017. An incentive for survey completion (eg, a raffle for an iPad) was introduced in 2018, and the response rate was 74.3%.
Students received a link to the GES via email in December after graduation. The Office of Education linked the CES and GES data to the student’s identification number, data on sociodemographic characteristics (age, sex, race/ethnicity, and citizenship status), and educational information (program, department, cohort) for respondents and nonrespondents and subsequently deidentified the data set. Of 1902 respondents to the CES, 1101 (57.9%) respondents who completed the Core during 2013-2017 responded to the GES. GES analyses excluded data on students who completed the Core in 2018 because analyses were completed before the planned GES date (December 2020). The Columbia University Irving Medical Center Institutional Review Board deemed this study not human subjects research and, therefore, exempt from review.
Measures
Student characteristics
The CES collected data on the following demographic characteristics: age (as a continuous variable), sex (male, female), race/ethnicity (Black, White, Hispanic, Asian, and “other” [American Indian/Alaska Native, multiracial]), and US residency. Data on race/ethnicity were not collected for non–US residents. Office of Education student data included academic program (2-year MPH, accelerated MPH [a 1-year program for advanced professionals], and dual degree) and department or concentration.
Student experience
Measures in the CES and GES included satisfaction with courses and teaching and likelihood of recommending the program to a friend with similar interests. We combined responses of “somewhat satisfied” and “very satisfied” into “very satisfied/satisfied” (yes/no) and “definitely recommend” and “likely to recommend” into “very likely/likely to recommend” (yes/no).
Learning outcomes
Learning outcomes in the CES included self-reported ability to integrate concepts learned in the Core across modules (with ease, with difficulty, or unable). Learning outcomes in the GES self-reported ability to apply concepts learned in the Core to subsequent coursework and field practice (with ease, with difficulty, or unable). Data on graduation rates for 2009 through 2018 came from the Office of Education records. A student who received a grade lower than B–, unofficial withdrawal, or incomplete in any studio was classified as not completing the Core. Graduation rates are the aggregate proportions of MPH students who graduated within 2 years, the timeline for >99% of MPH students (full-time, accelerated, and dual degree).
Analysis
Descriptive analyses summarized data on Core structure, faculty and student characteristics, and student experiences and learning outcomes. We conducted bivariate analyses, with α = .05 considered significant, to compare characteristics and outcomes by year using the Pearson χ2 test, the Fisher exact test, and analysis of variance. For the respondent subset with both the CES and GES data (n = 1101), we assessed the stability of outcomes over time for respondents’ answers to 3 similar questions (satisfaction with courses, satisfaction with teaching, and likelihood of recommending) by comparing responses at the 2 time points using McNemar tests. Two independent coders (S.G., S.R.B.) extracted and synthesized qualitative data from the CES. We used IBM-SPSS Statistics version 25.0 (IBM Corp) for all analyses.
Results
During 2013-2018, the size of the class, cohort, and ISP section increased; the number of studios remained the same (n = 6); and Core lecture seat time (in hours) decreased (Table 1). The total lecture time decreased from 224 hours to 208 hours (7.1% reduction), and smaller-group section time increased from 25.5 hours to 28.5 hours (11.8% increase). Each studio brought together, on average, faculty from 2 or 3 departments, and most (59%-63%) modules were taught by teams with multiple faculty. The proportion of faculty who taught in >1 of the 3 components of the Core (module, ISP, and Leadership) increased: in 2018, 10.8% (4 of 37) of Core faculty taught in all 3 Core components (vs 1 in 2013), and 16.2% (6 of 37) of Core faculty taught both ISP and Leadership (vs none in 2013).
Columbia University Mailman School of Public Health Master of Public Health Core curriculum structure, 2013-2018
Abbreviation: ISP, Integrating Science & Practice.
Includes laboratory and discussion sections and scheduled project work time.
No faculty taught in Leadership and modules but not ISP.
Terminal degree in biostatistics, environmental health sciences, epidemiology, public health, or population and family health.
Terminal degree in the arts (fine arts, music, performing arts, literature), philosophy, religious studies, social science (anthropology, geography, history, jurisprudence, linguistics, political science, psychology, sociology), or mathematics.
Terminal degree in law or education.
Includes the first year of teaching as 1 year, not 0 years.
During 2013-2018, 2537 students completed the Core, of whom 1902 (74.9%) responded to the CES (Table 2). Of the 1902 respondents, most (83.6%, n = 1591) were in the 2-year MPH program, and 10.9% (n = 208) were in the 1-year accelerated program. Most respondents were female (83.0%) and racially and ethnically diverse, representing 62 countries.
Characteristics and educational outcomes of respondents to the Columbia University Mailman School of Public Health Masters of Public Health Core curriculum Core Evaluation Survey, 2013-2018 (N = 1902) a
Abbreviation: MPH, master of public health.
Survey administered to students in January after completion of the Core curriculum. All values are number (percentage) unless otherwise indicated. Not all percentages are based on the number indicated in the column head, because not all respondents answered all questions. Percentages may not total to 100 because of rounding.
Difference by year using independent t test for age and χ2 test for all other variables, with P < .05 considered significant.
Includes American Indian/Alaska Native and multiracial.
Part-time MPH students total 18 (<1%).
Biostatistics had only an accelerated MPH program before 2017.
All general public health students are accelerated MPH students.
Student Experience
Overall, student satisfaction with the Core was high: in 2018, 82.9% of CAS respondents were very satisfied/satisfied with Core modules and content and 89.6% were very satisfied/satisfied with teaching in the Core (Table 2), although these measures declined since inception of the Core. A stable proportion (ranging from a low of 81.3% in 2016 to a high of 87.7% in 2014) of respondents were very likely/likely to recommend the Core to a friend with similar interests.
Among respondents who completed the Core during 2013-2017 and responded to the CES and GES (n = 1101), the student experience was similar and not significantly different during the students’ time at Columbia University and after graduation: students’ satisfaction with courses was similar in 2018 immediately post-Core (84.8%) and after graduation (84.2%) (McNemar P = .78), and students’ satisfaction with teaching was similar immediately post-Core (91.1%) and after graduation (90.3%) (McNemar P = .56) (Tables 2 and 3). The likelihood that students would recommend the program was significantly lower after graduation (75.3%) than immediately post-Core (85.9%) (McNemar P = .03).
Learning outcomes and student experience of respondents to the Columbia University Mailman School of Public Health (CUMSPH) Masters of Public Health (MPH) Core curriculum Graduate Exit Survey, 2013-2017 (N = 1101) a
Abbreviation: ISP, Integrating Science & Practice.
This group of respondents is a subset of the group that also completed the Core Evaluation Survey (CES), which was administered 1 month after Core completion. The Graduate Exit Survey (GES) was administered in December after graduation. All values are number (percentage) unless otherwise indicated. Not all percentages are based on the number indicated in the column head, because not all respondents answered all questions. Percentages may not total to 100 because of rounding.
Students who completed the Core in 2018 had not graduated at the time of data analysis.
Difference by year using independent t test for age, χ2 test for all other variables, with P < .05 considered significant.
Accelerated students do not complete a certificate.
Graduation rates are only available in the aggregate (not linked to CES or GES data); includes all nonexecutive MPH students, including accelerated and dual-degree students, who are anticipated to graduate within 2 years after completing the Core.
Learning Outcomes
Academic performance
During the study period, 94.4% (1795 of 1902) of students completed the Core, a proportion that increased from 89.6% in 2013 to 95.0% in 2018 (Table 2).
Integration of concepts
Of 1902 respondents to the CES, 1464 (81.7%) reported being able to easily integrate concepts across the Core and 1447 (80.8%) reported being able to easily integrate concepts between the Core and ISP (Table 2).
Qualitative data indicate that students appreciated integration. One student noted, “Covering multiple areas of public health was a great idea to give MPH students a well-rounded experience.” Another student indicated, “The integration and diversity of courses is wonderful. It’s only been a semester, but I already feel like I’m thinking differently and approaching problems with a more interdisciplinary, public health approach.” The primary challenge to integration, and a source of student dissatisfaction, was frustration with assessments. Students suggested more opportunities to demonstrate in-depth integrated knowledge (eg, “questions about the bigger picture”) and more applied assignments. One student said, “I wish the Core pushed me to . . . provide innovative approaches to different issues.” Other students grappled with complex assignments, indicating they were “too complicated and too stressful.”
A Core assessment designed to foster integration was an op-ed requiring students to support a public health action, explicitly drawing on content from 3 or more studios. The assessment prompt encouraged students to cross disciplinary boundaries and advocate for integrated solutions. Students heralded the op-ed as a prime opportunity to demonstrate learning. One student noted, “I really enjoyed the iterative process of writing the op-ed . . . I came out of the process with a finished product that I was proud of, something I felt was evidence of a real-world skill I had required, instead of just an assessment for class.”
Respondents to the GES indicated that learning outcomes were sustained, although students reported some difficulty in applying concepts to field practice. Of 1101 respondents to the GES postgraduation, 989 (92.3%) reported being able to apply content from the Core to departmental coursework with ease (76.3%) or with difficulty (16.0%), 803 (87.2%) reported being able to apply their certificate coursework to their practicum with ease (69.3%) or with difficulty (17.9%), and 872 (81.3%) reported being able to apply their field practice requirement to their practicum with ease (61.2%) or with difficulty (20.1%).
Graduation rates
The proportion of MPH students graduating in 2 years was 93% in 2013 and 91% in 2018 (Table 3).
Discussion
A central strategy of the Core, when it was designed, was integrative, interdisciplinary training. 3 The Core’s short-term evaluation identified the primary goal of implementing an MPH curriculum with rigorous interdisciplinary content. 1 Results from this study indicate that the Core sustained, during the 6-year study period, a high level of student integration of content and concepts from multiple disciplines. Results of the current evaluation build on results of a previous evaluation 1 by assessing students’ ability to translate knowledge into action through applied field experience, following later outcomes through graduation, and providing richer detail on the implementation of the Core (its structure, faculty, and assessments) to explain observed learning outcomes.
Several findings support the Core’s success in integrative, interdisciplinary training. Most students endorsed being able to easily integrate concepts across modules within the Core and between the Core and ISP (>80% for both). These percentages were markedly higher than the percentage of students who reported being able to integrate content in all courses taken at CUMSPH (52.5%) postgraduation. The results of our evaluation indicate that the Core provides a solid foundation for subsequent coursework and field practice: most students were able to apply content learned in the Core to departmental coursework (92.3%), certificate coursework (87.2%), and their practicum (81.3%).
Although this evaluation sought to link the implementation of the Core with student experience and learning outcomes, external factors may have affected evaluation findings. Although satisfaction with the Core was high throughout the study period, satisfaction dipped from 2015 to 2016 both overall (6.8 percentage-point reduction) and with teaching (6.9 percentage-point reduction). Notably, in the GES, 78.1% of Core students reported being very satisfied/satisfied with courses in 2016, a 14.9 percentage-point decrease from 2015. The cross-sectional nature of the data limited the exploration of reasons for the decrease, but qualitative data suggest that the school climate was negatively affected by the US presidential election in 2016. Core students in 2016 explicitly mentioned the effect of the political landscape on their work and expressed a need for “skills in navigating this current political climate” and training on “how to effect change in [the] political environment we are currently in.”
This evaluation of 6 years of Core implementation data builds on data from a previous evaluation by examining learning outcomes beyond the students’ time in the Core. Although this evaluation identified sustained student satisfaction and learning outcomes, it also identified improvements. Ongoing and systematic monitoring and evaluation guided regular revisions, including shifting lecture time to opportunities for hands-on work (to support critical thinking), reorganizing of studio structures (integration), increasing the proportion of faculty teaching across Core components (integration and efficiency), and increasing faculty and teaching assistant training (satisfaction). Most students completed the Core during the study period, a proportion that increased from 2013 to 2018 and reflected intentional changes to teaching team structures, faculty and teaching assistant training, increased attention to inclusion and equity, and active involvement of the Center for Teaching and Learning to integrate active learning strategies. One unanticipated positive effect of the curriculum renewal was the increase in the percentage of 2-year graduates from the years before the Core (76%-77% during 2009-2011) to the year the Core was launched (93% in 2013) and beyond (91% in 2018). Having a unified Core in which all students complete schoolwide requirements during their first semester removed barriers to timely completion of departmental and certificate coursework.
Some challenges remain in developing assessments that empirically measure students’ ability to integrate concepts. Connecting the intended interdisciplinary framework to the teaching of public health skills as they are practiced—and the ability to adequately measure that application—requires vigilance and, likely, new methods. Although an op-ed assignment was designed to provide an opportunity for students to integrate concepts, developing such assessments for large cohorts of students and grading them meaningfully is challenging.
Limitations
This evaluation had several limitations. First, we focused on implementation, student experience, and learning outcomes until only 6 months postgraduation but not on effects during a longer follow-up period. Although longer-term effects (eg, employment) are tracked, they were not linked to these data. Second, public health programs serve a changing student body with heterogeneous needs and baseline skills and knowledge bases.7,14,15 Although the current evaluation made use of robust data during a 6-year period, further questions merit examination, including longer-term effects on all students and student subgroups, such as students in underrepresented racial, ethnic, and socioeconomic groups.
Conclusions
This evaluation of the Core, spanning 6 years of data from thousands of MPH students through the year postgraduation, describes the Core in detail to serve as a model for other schools and programs of public health. The MPH Core curriculum meets its intended goal of supporting integrated, interdisciplinary training and skills to adapt to changing realities. 3 The Core will continue to evolve, with a monitoring and evaluation process to foster improvement and innovation.
Footnotes
Acknowledgements
The authors acknowledge the Office of Education, the Center for Teaching and Learning, Core faculty and studio leads, Core teaching assistants, and Core students who contribute each year to a vital learning experience and a rigorous evaluation process.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
