Exposure to ACGME Core Competencies through Mentored Basic Science Research: A Pilot Analysis

Abstract

Since 1998, the Short-Term Training Program (STTP) at the David Geffen School of Medicine at the University of California, Los Angeles has nearly tripled (from 30 to 89) in the number of first-year undergraduate medical students participants. STTP supports mentored research projects in the areas of basic sciences, clinical sciences, medical education, and public health (local and international). Although projects can be very specific in scope, the overall experience in STTP exposes students to some, if not all, of the Accreditation Council for Graduate Medical Education (ACGME) six core competencies–-Patient Care, Medical Knowledge, Practice-based Learning and Improvement, Interpersonal and Communication Skills, Professionalism, and Systems-based Practice. Thus, STTP has been an important aspect of medical education to prepare our students for residency programs. We describe and analyze the STTP as a model system to introduce the ACGME core competencies at an early point in undergraduate medical education. We conclude with a call to provide more mentored anatomical sciences basic and clinically applied research opportunities.

Introduction

In 1999, the Accreditation Council for Graduate Medical Education (ACGME) established six core competencies to unify graduate medical education–-patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice. These competencies help serve as public assurances that resident and attending physicians will be compassionate in their service, knowledgeable of and continually learning best scientific and clinical practices, diligent in communicating information with other physicians and with patients, professional, ethical, and resourceful. Undergraduate medical students begin learning these competencies in pre-clinical and clinical settings.

The Liaison Committee on Medical Education recognizes the importance of vertical and lateral learning opportunities for undergraduate medical students based on accreditation standards for the Academic Environment. ¹

Since 1998, the Short-Term Training Program (STTP) has provided scientific learning opportunities through mentored research at the David Geffen School of Medicine (DGSOM) at the University of California, Los Angeles (UCLA) that expose each student to many, if not all, of the six ACGME core competencies in the span of a 7–1/2 week period during the summer intercession between the first and second pre-clinical years. STTP has nearly tripled (from 30 to 89) in the number of participants from 1998 to 2013 and has been an important aspect of medical education for preparing students for residency programs. We describe the program here and demonstrate educational themes that relate to the ACGME core competencies.

Description

Ucla Sttp

Although research is an optional activity, approximately 94% of DGSOM at UCLA students were involved in a summer research activity in 2013; of these students, 79% were involved in STTP. Originally funded by the National Institutes of Health, the Dean's Office now completely supports the 7–1/2 week elective summer STTP. Students engage in mentored research in areas of their choice from the basic sciences, clinical sciences, medical education, and public health (local and international). Students must find a UCLA faculty principal investigator (PI) sponsor who directs and supports the research. The PI must be a faculty member with a permanent position at UCLA or one of its affiliate sites. Typically, students choose established research projects, as the time frame is too short for novel pursuits. However, some students are able, with the guidance of their mentor, to initiate, develop, and execute new projects. Students learn to be resourceful and professional by searching for and contacting PI's whose research fits their interests. Almost all students have reported engaging in an employment process of submitting a curriculum vitae with cover letter to 10–20 PI's, interviewing with several PI's and receiving even fewer offers.

Upon acceptance, a student and PI work together to craft a 1–3 page proposal that describes the specific aims of the proposed research, hypothesis being tested in general terms, the methods used, and how the data will be analyzed. The student and PI describe the techniques that will be learned, and potential difficulties that may be encountered in the acquisition and analysis of data. If an Institutional Review Board (IRB) or any type of training is required, the student must take the initiative to complete the necessary application and training to meet compliance. Mentors agree to spend the necessary time to provide guidance, instruction, and laboratory support. At the end of the summer, the student is required to submit a first author abstract and present a poster at the DGSOM Josiah Brown Poster Fair. The mentor is invited to this fair to support his/her STTP student and other students presenting. It is not uncommon for a student to also present his/her work at conferences for a society to which the PI belongs. Both student and mentor sign a contract agreeing to these terms of the program.

Medical students are supported in their research endeavors through stipends, funding for conference travel, and administrative support of research fellowships and grants. The funding for these research activities come from a variety of sources. STTP is a line item in the Dean's Office budget. In contrast, other School of Medicine programs are funded by grants, endowments, the UCLA Hospital System, and other philanthropic sources. Finally, national research fellowships are funded through the national awarding agency.

Student satisfaction with the STTP experience

In 2012, Thakur et al ² reported results from a survey ascertaining satisfaction with their STTP experience to the same population of students included in this study. We re-analyzed the qualitative data from that study and report representative positive and negative student responses pertaining specifically to the six ACGME core competencies (Table 1).

Table 1.

Representative student responses regarding their STTP experience within the context of the six ACGME core competencies–- Patient Care, Medical Knowledge, Practice-based Learning and improvement, Interpersonal and Communication Skills, Professionalism, Systems-based Practice.

PATIENT CARE
Positive
“My favorite part about my research experience was actually talking with the patients that I interviewed. It was invaluable to hear their perspectives on why they were in the hospital, and what we could have done as their medical team to perhaps prevent them from having to come back.”
“I enjoyed seeing patients with my doctor.”
Negative
None to report.

MEDICAL KNOWLEDGE
Positive
“I enjoyed exploring a medical field that wasn't covered as in depth during the first year curriculum.”
“The applicability of what I was doing to us as residence in the future made the work feel very worth it.”
“Flexibility with my project, getting to do more hands on research since I am a medical student (vs being an undergraduate).”
“My mentor was amazing and very supportive of my interests both research and with opportunities to shadow in clinic.”
Negative
None to report.

PRACTICE-BASED LEARNING AND IMPROVEMENT
Positive
“I liked the variety of experiences I was exposed to.”
“Background research to understand the applications of my research project was very interesting.”
“Novel, self-guided learning.”
Negative
None to report.

INTERPERSONAL AND COMMUNICATION SKILLS
Positive
“I enjoyed being able to work side-by-side with my research PI. He was very hands-on and taught me about ways to navigate a career in academic medicine.”
“The best part was my patient interaction and relationship with mentor.”
Negative
“I wanted more time to talk with my peers about their research [at the poster fair], but felt pressured to visit more posters.”
“My mentor's lack of interaction/interest with me was hard. I think a lot of it was just his personality. At least I had a very helpful fellow and research assistant to mentor me.”

PROFESSIONALISM
Positive
“I enjoyed the opportunity to create my own research project.”
Negative
“Sharp learning curve.”
“I felt unproductive at the beginning when I didn't have very good direction.”
“The loose guidance from my research mentor. There were several times where I would spend too much time feeling lost.”

SYSTEMS-BASED PRACTICE
Positive
“I loved having the opportunity to engage with the larger LA medical community and be part of an exciting project that aims to increase quality of care for low-income communities.”
Negative
None to Report.

Themes of STTP research

We performed a two-step qualitative analysis of student exposure to basic science and clinical science research, and educational scholarship. We first performed a word frequency analysis of the titles and abstracts that students submit as their capstone presentation, followed by a thematic analysis.

We created tag clouds (http://www.wordle.net) to visually represent our analysis of presentations from 2011 to 2012. Words used with greater frequency are represented in a larger font size relative to less-frequent words represented in smaller font sizes. Common English language articles, prepositions and conjunctions, and abstract section headings (and derivatives) were excluded from display in the tag clouds. In our analysis, we included ancillary programs that sponsor STTP work, but are funded independently. The artistic representation of word frequencies led to the qualitative thematic analysis.

During these two summers, 229 students participated in summer research, of which 181 completed projects were funded by STTP (79%). Research encompassed many disciplines related to basic sciences, clinical sciences, medical education, and public health (local and international). We categorized the projects into these disciplines using the following operational definitions, and then used the titles and abstracts text to create categorical tag clouds.

Basic sciences projects included those involving non-human and animal studies, or studies that examine methodologies or mechanisms.

Clinical sciences projects included human studies directly related to patient care.

Medical education projects included studies that examined the improvement of patient education or self-efficacy and of physician training.

Public health projects included studies that examined clinical treatment to specific medical and/or ethnic populations in a community.

Overall projects

Figure 1 (top) depicts word frequencies from titles of 181 presented abstracts (total word count = 2,341). The most frequent word, patients, appears 37 times. Words that appear approximately 25–50% as frequent as patients include development, effects, treatment, health, cancer, cell, evaluation, and surgery.

Figure 1.

Word frequencies in titles and abstracts of all 2011 and 2012 STTP presentations. There were 181 projects in total.

Figure 1 (bottom) depicts word frequencies from abstract texts (total word count = 56,589). The most frequent word, patients, appears 487 times. Words that appear approximately 25–50% as frequent as patients include study, treatment, compared, health, and data.

Basic science projects

Figure 2 (top) depicts word frequencies from titles of 48 presented abstracts (total word count = 574). The most frequent word, cells, appears 19 times. Words that appear approximately 25–50% as frequent as cells include effects, stem, cancer, model, human, mechanical, inflammatory, signaling, carcinoma, evaluation, and formation.

Figure 2.

Word frequencies in titles and abstracts of basic sciences of 2011 and 2012 STTP presentations. Basic sciences projects included those involving non-human and animal studies, or studies that examine methodologies or mechanisms. There were 48 basic sciences projects.

Figure 2 (bottom) depicts word frequencies from abstract texts (total word count = 15,656). The most frequent word, cells, appears 102 times. Words that appear approximately 25–50% as frequent as cells include expression, patients, compared, scaffolds, mice, mutation, protein, levels, control, and gene.

Clinical science projects

Figure 3 (top) depicts word frequencies from titles of 104 presented abstracts (total word count = 1,318). The most frequent word, patients, appears 27 times. Words that appear approximately 25–50% as frequent as patients include treatment, surgery, development, cancer, outcomes, novel, effects, and health.

Figure 3.

Word frequencies in titles and abstracts of clinical sciences of 2011 and 2012 STTP presentations. Clinical sciences projects included human studies directly related to patient care. there were 104 clinical sciences projects.

Figure 3 (bottom) depicts word frequencies from abstract texts (total word count = 31,941). The most frequent word, patients, appears 485 times. Words that appear approximately 25–50% as frequent as patients include study, compared, data, cancer, treatment, care, used, risk, and analysis.

Medical education projects

Figure 4 (top) depicts word frequencies from titles of four presented abstracts (total word count = 57). The most frequent word, quality, appears two times. Words that appear approximately 25–50% as frequent as quality include all other words in the titles.

Figure 4.

Word frequencies in titles and abstracts of medical education of 2011 and 2012 STTP presentations. Medical education projects included studies that examined the improvement of patient education or self-efficacy and of physician training. There were four medical education projects.

Figure 4 (bottom) depicts word frequencies from abstract texts (total word count = 1,028). The most frequent word, residents, appears 11 times. Words that appear almost as frequent as residents include patient, students, quality, and care. Words that appear approximately 25–50% as frequent as residents include safety, medical, skills, arthroscopy, training, and themes.

Public health projects

Figure 5 (top) depicts word frequencies from titles of 25 presented abstracts (total word count = 392). The most frequent word, Ghana, appears seven times. Words that appear approximately 25–50% as frequent as Ghana include health, rural, prevalence, services, Western, survey, and rural.

Figure 5.

Word frequencies in titles and abstracts of public health (local and international) of 2011 and 2012 STTP presentations. public health projects included studies that examined clinical treatment to specific medical and/or ethnic populations in a community. There were 25 public health projects.

Figure 5 (bottom) depicts word frequencies from abstract texts (total word count = 7,964). The most frequent word, health, appears 79 times. The word patients appears 60 times. The word children appears 44 times. Words that appear approximately 25–50% as frequent as health include population, treatment, care, HIV, diarrhea, and services.

Thematic analysis

Across basic science, clinical science, medical education, and public health research, students were focused on some aspect of patient care, regardless of the arena or research topic. This may not come as a surprise when considering the focus of clinical science, medical education, or public health research, but it is noteworthy that students engaging in basic science research would make direct connections to patient care, especially since the vast majority of projects dealt with cells.

Interested in the basic science–patient care connection, we followed-up our study with a cursory analysis of students who we were able to mentor in the basic science field of anatomy. In particular, we wanted to know if anatomy research exposed students to something seemingly unrelated–-the ACGME core competencies. Additional analysis needs to be done for students who participated in cell biology research.

STTP experience specific to anatomical sciences research

From 2006 to 2012, four students participated in STTP through research projects in the anatomical sciences. As a pilot study, we contacted these students and asked them to complete an IRB-approved survey through SurveyMonkey (www.surveymonkey.com) ascertaining their experience in the program and what exposure they had to the six ACGME core competencies. Three students participated in STTP in 2008 and are now residents. One student participated in 2010 and is currently a 4th year medical student. Three of the four students returned the survey: one medical student and two residents.

The survey asked the following questions and, where appropriate, required a 5-point Likert response followed by an optional free-response section for elaboration. Questions are listed in detail in Table 2, and in summary, with answers in Table 3. We recognize that our pilot study is limited to the responses of these three individuals and plan to validate and use this instrument for future studies of STTP participants.

Table 2.

Survey questions given to medical students who completed STTP research in the anatomical sciences.

1. What year did you begin medical school?

2. What is your current year in training?

3. Participation in STTP was important to me for exposure to research opportunities.

4. Participation in STTP was important to me for exposure to educational opportunities.

5. Participation in STTP was important to me for financial reasons.

6. Participation in STTP provided exposure, either directly or indirectly, in whole or in part, to the ACGME Core Competency of Patient Care.

a. Please use the following description of Patient Care upon which to base your rating: Provide patient care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health.

7. Participation in STTP provided exposure, either directly or indirectly, in whole or in part, to the ACGME Core Competency of Medical Knowledge.

a. Please use the following description of Medical Knowledge upon which to base your rating: Demonstrate knowledge of established and evolving biomedical, clinical, epidemiological and social-behavioral sciences, as well as the application of this knowledge to patient care.

8. Participation in STTP provided exposure, either directly or indirectly, in whole or in part, to the ACGME Core Competency of Practice-based Learning and Improvement.

a. Please use the following description of Practice-based Learning and Improvement upon which to base your rating: Demonstrate the ability to investigate and evaluate care of patients, to appraise and assimilate scientific evidence, and to continuously improve patient care based on constant self-evaluation and life-long learning. Develop skills and habits to be able to meet the following goals:

i. identify strengths, deficiencies, and limits in one's knowledge and expertise;

ii. set learning and improvement goals;

iii. identify and perform appropriate learning activities;

iv. systematically analyze practice using quality improvement methods, and implement changes with the goal of practice improvement;

v. incorporate formative evaluation feedback into daily practice;

vi. locate, appraise, and assimilate evidence from scientific studies related to patients' health problems;

vii. use information technology to optimize learning; and,

viii. participate in the education of patients, families, students, residents and other health professionals.

9. participation in STTP provided exposure, either directly or indirectly, in whole or in part, to the ACGME Core Competency of interpersonal and Communication Skills.

a. Please use the following description of Interpersonal and Communication Skills upon which to base your rating: Demonstrate interpersonal and communication skills that result in the effective exchange of information and collaboration with patients, their families, and health professionals. Interns, Residents, and Fellows are expected to:

i. communicate effectively with patients, families, and the public, as appropriate, across a broad range of socioeconomic and cultural backgrounds;

ii. communicate effectively with physicians, other health professionals, and health related agencies;

iii. work effectively as a member or leader of a health care team or other professional group;

iv. act in a consultative role to other physicians and health professionals; and,

v. maintain comprehensive, timely, and legible medical records, if applicable.

10. Participation in STTP provided exposure, either directly or indirectly, in whole or in part, to the ACGME Core Competency of Professionalism.

a. Please use the following description of Professionalism upon which to base your rating: Demonstrate a commitment to carrying out professional responsibilities and an adherence to ethical principles. Interns, Residents, and Fellows are expected to demonstrate:

i. compassion, integrity, and respect for others;

ii. responsiveness to patient needs that supersedes self-interest;

iii. respect for patient privacy and autonomy;

iv. accountability to patients, society and the profession; and,

v. sensitivity and responsiveness to a diverse patient population, including but not limited to diversity in gender, age, culture, race, religion, disabilities, and sexual orientation.

11. Participation in STTP provided exposure, either directly or indirectly, in whole or in part, to the ACGME Core Competency of Systems-based Practice.

a. Please use the following description of Systems-based Practice upon which to base your rating: Demonstrate an awareness of and responsiveness to the larger context and system of health care, as well as the ability to call effectively on other resources in the system to provide optimal health care. Interns, Residents, and Fellows are expected to:

i. work effectively in various health care delivery settings and systems relevant to their clinical specialty;

ii. coordinate patient care within the health care system relevant to their clinical specialty;

iii. incorporate considerations of cost awareness and risk-benefit analysis in patient and/or population-based care as appropriate;

iv. advocate for quality patient care and optimal patient care systems;

v. work in interprofessional teams to enhance patient safety and improve patient care quality; and

vi. participate in identifying system errors and implementing potential systems solutions.

12. Choosing the right mentor was important to me for ensuring that i had a good STTP experience.

13. My STTP experience influenced my medical specialty choice for residency.

14. I am practicing medicine and/or engaged in medical research in the same field of study as my STTP project.

15. What are the strengths of STTP in exposing medical students to the ACGME Core Competencies?

16. What are the weaknesses of STTP in exposing medical students to the ACGME Core Competencies?

17. What would you do to improve the ability of STTP to exposing medical students to the ACGME Core Competencies?

Table 3.

Responses to the survey from 3 (of 4) students (one medical student and two residents) who participated in STTP through anatomical sciences research.

QUESTION SUMMARY	MEDICAL STUDENT RESPONSES		RESIDENT 1 RESPONSES		RESIDENT 2 RESPONSES
	LIKERT SCORE	COMMENTS	LIKERT SCORE	COMMENTS	LIKERT SCORE	COMMENTS
Important for research opportunities	4	Good program that didn't have a limit to the number of applicants as long as we had a project	5		5	In addition to strengthening my background in science, this program opened up a wide avenue of opportunities to engage in research projects across the different fields of medicine
Important for educational opportunities	4	i did research in an area i wasn't considering but learned a lot about it	4		5
Important for financial reasons	5		4		5
Exposure, either directly or indirectly, in whole or in part, to the ACGME Core Competency of:
Patient Care	4		2	I did not have direct patient care	5
Medical Knowledge	5		4		5
Practice-based Learning and Improvement	4		4		4
Interpersonal and Communication Skills	3		5		5
Professionalism	4		4		5
Systems-based Practice	4		4		4
Choosing the right mentor was important	5		5		5
Influenced my medical specialty choice for residency	2		4		4
Practicing medicine and/or engaged in medical research in the same field of study as STTP project	no		no		no
Strengths of STTP in exposing medical students to the ACGME Core Competencies	Ability to work with a mentor of our own choosing		I had a positive STTP experience which helped me better understand the benefits and challenges of clinical research, improved my confidence in interacted with attending physicians, and led to participation in national conferences		This program introduced key concepts regarding analyzing scientific data, designing and executing a research project, and engaging in the scientific community. the ACGME asks for residents to learn how to understand, evaluate, and critique research data, all of which this program incorporates
Weaknesses of STTP in exposing medical students to the ACGME Core Competencies	Limited access to medical faculty if not doing a clinically based project		I believe the STTP experience is very mentor dependent		The only weakness of the STTP program is that mentorship does not begin until the summer. I believe the program would be stronger if mentorship began earlier
Suggestions to improve the ability of STTP to exposing medical students to the ACGME Core Competencies?	Nothing differently		I'm not sure, other than perhaps maintaining a list of attendings with which students had a positive experience, or somehow incorporating prior student feedback into resources available for those seeking projects		I think students should have weekly journal club to review how to evaluate research data and how to design better research projects

Notes: The medical student is undecided. Intern 1 is practicing in ophthalmology and Intern 2 is practicing in internal medicine.

Discussion

STTP themes related to ACGME core competencies

The tag clouds demonstrate the breadth of exposure students have on aspects of all six ACGME core competencies. Altogether, the scientific aspects of STTP exposed students to Patient Care, Medical Knowledge, and Practice-based Learning and Improvement. However, based on formal and informal surveys, it is clear that the experience of working in a research team, and with a mentor, exposed students to Interpersonal and Communication Skills, Professionalism, and Systems-based Practice.

STTP research breadth

By using operational definitions to classify the different projects into categories of basic sciences, clinical sciences, medical education, and public health, we were able to ascertain the main areas of research in which students are engaged. Basic sciences projects focused on cellular biology. Clinical sciences projects focused on patient care with regard to treatments and outcomes. Of these, many projects focused on cancer. Medical education projects interestingly focused on residents, patients, and quality care. Although there were relatively much fewer medical education studies than other types of studies, this category in particular highlights the bridge between graduate and undergraduate medical education. Public health projects focused on international and rural care with regard to HIV, children, and African populations. This reflects the strong international global health program at the DGSOM. Across all topics, and interestingly including basic sciences research, students related their work back to patient care.

STTP research in the anatomical sciences

Although the number of anatomical sciences projects in the STTP is a mere fraction of the total number of projects, it is interesting to note that based on the data from our informal survey, the three students who responded felt their research experience in a traditional basic science discipline exposed them to the six Core Competencies with an overall Likert score average of 4.27. Clearly, we need to–-and we plan to–-use this survey instrument with more students from other basic sciences fields. However, if this trend holds true, it would speak very highly of the value of basic anatomical sciences research in medical education.

Students who participated in STTP presented their findings at the annual UCLA-sponsored summer research fair, 42% of the students who participated in STTP went on to present their research findings at the Western Student Medical Research Forum in their second year of medical school. The group interviewed published two papers on the clinical application of mapping the cervical sympathetic chain ganglia for improved targeted transient sympathectomies:^3,4 one award-winning conference paper on the anatomical etiology of Eagle's Syndrome ⁵ and several conference papers on the anatomical validity of Diffusion Tensor Imaging.^6,7 Although anatomy is typically considered a basic sciences discipline, our work shows that when applied to clinical problems, anatomy research may transcend stereotypical bench research. The basic sciences abstracts tag cloud from Figure 2 exhibits cells as the most frequent word and the clinical sciences abstracts tag cloud from Figure 3 exhibits patients as the most frequent word. We assert that clinical anatomy research can be a bridge between basic science and clinical care. This needs to be studied in more depth, but will require more programs across the country to encourage students to participate in clinical anatomy research.

Future directions for improvement in aligning with ACGME core competencies

We, and other investigators,^2,8,9 have shown that mentorship plays a key role in motivating students to pursue a career in research. Therefore, strong emphasis needs to be placed on the role of the mentor in STTP and coaching the PI to become a better research mentor. This was clear in student and responses to surveys. The mentor is key to the success of a student's experience, but standardizing mentors may be difficult.

Instead, intermittent workshops throughout the course of STTP to help students address common problems with research, such as working with a mentor, IRB issues, research progress, statistical analyses, may be a better option for improving student experiences. Likewise, student training on professionalism in research, including ethics of research, manuscript preparation, would be very helpful in providing more exposure to ACGME core competencies. We can also be more explicit in program goals and assessments that point directly to learning the Core Competencies through post-STTP evaluation surveys, and perhaps adding in a component of focus groups to determine how STTP is helping students to pursue research beyond the summer.

Future research could also review the medical student research titles for greater stratification of the research types, further aligning the research themes with ACGME core competencies. The purpose of this preliminary analysis was to get a sense of the general research themes; future analyses would delve into the students' chosen research subject and how many of the core ACGME core competencies they address.

Author Contributions

Conceived and designed the experiments: JJW, ST and MES. Analyzed the data: JJW, ST and MES. Wrote the first draft of the manuscript: JJW. Contributed to the writing of the manuscript: JJW, ST and MES. Agree with manuscript results and conclusions: JJW, ST, and MES. Jointly developed the structure and arguments for the paper: JJW, ST, and MES. Made critical revisions and approved final version: JJW, ST, and MES. All authors reviewed and approved of the final manuscript.