Abstract
Introduction
The authors developed and evaluated a faculty development program on clinical teaching skills to address barriers to participation and to impact teaching behaviors.
Methods
Four one-hour workshops were implemented over five months. Evaluation included participant satisfaction and pre/post self-assessment. Pre/post faculty teaching ratings by trainees were compared.
Results
A total of 82% of faculty (N = 41) attended. Participants rated workshops highly (mean, 4.43/5.00). Self-assessment of skills and comfort with teaching activities improved. A total of 59% of residents and 40% of fellows felt that teaching received from participating faculty was highly effective. The majority observed targeted teaching behaviors by the faculty. Teaching ratings improved after the workshops (P = 0.042).
Conclusion
Our series of short workshops during a standing conference time was associated with increased self-assessed skill and comfort and an increase in faculty ratings on teaching evaluations. Effective faculty development programs can be implemented in flexible formats and overcome common barriers to participation.
Introduction
Teaching in the busy clinical setting is one of the many responsibilities of clinical teaching faculty. Clinical educators frequently teach a variety of learners across multiple learning domains and within multiple different clinical contexts. Lack of formal training on how to effectively instruct learners in the clinical setting and time constraints are among reported reasons faculty may experience discomfort with incorporating teaching into their daily routine. 1
Faculty development programs that address clinical teaching skills have been implemented in a wide range of settings, time frames, and areas of practice, 2 though, to our knowledge, none have been described in subspecialty medicine practices. Barriers for attendance in such programs have been described, 3 which include lack of time and logistics required for participating. Programs have been shown to be well received and to improve knowledge.2,4 However, there are only a small number of studies reporting changes in behavior after faculty training programs on teaching skills.5–8
In our cardiology fellowship program, a needs assessment identified room for improvement in faculty comfort with teaching in the clinical setting. As has been described in other programs, 9 giving effective feedback was identified as a particular area for improvement, along with clinical teaching models and assessment. Our faculty cited barriers similar to those that have been reported in the literature, 1 both regarding incorporating teaching into daily activities and time limitations to allow participation in faculty development programs. This prompted us to investigate practical methods of training our faculty in clinical teaching skills across multiple learning domains that would limit interference with their daily activities. We sought to incorporate training activities into the routine schedule of educational conferences in a high-volume academic subspecialty medicine practice. This paper reports on the development, implementation, and evaluation of a learner-centric faculty development program focused on impacting teaching behaviors for clinical cardiologists.
Methods
Participants
All cardiology faculty whose primary duties were clinical and located at our university hospital site (N = 50) were eligible to participate in the faculty development program and were solicited via email by the faculty developing the curriculum. Participation was voluntary. The Institutional Review Board at our university declared this study exempt.
Needs assessment
A 41-item needs assessment survey (developed by the Educator Development Program Team) was sent via email to faculty who expressed an interest in participating. The anonymous survey identified their comfort with clinical teaching skills, methods, and theory. Respondents selected the main content they would like to see included in a faculty development program on clinical teaching skills. The needs assessment was used to prioritize topics to be covered in the faculty development curriculum.
Development and implementation
After securing support from division leadership, four one-hour workshops were developed and implemented between August and December 2013. Three faculty members with expertise in medical education (an internal medicine (IM) specialist with a Masters in Education and two cardiologists with formal, nondegree, medical educator development training) developed and facilitated the workshops. Workshops were held during a one-hour educational conference at noon, a time typically reserved for the cardiology division's weekly grand rounds. Based on the needs assessment data (Fig. 1), topics chosen for the workshops were as follows: (1) General Principles of Learning and Teaching in the Clinical Setting (including learning goals and objectives), (2) Clinical Teaching Methods, (3) Competencies, Milestones, and entrustable professional activities (EPAs), and (4) Giving Feedback. Learning objectives for each workshop are listed in Appendix A. Using a blended learning format, participants were asked to review material on the topics to be covered prior to the workshops. The workshops were based on principles of adult learning theory and available evidence-based best practices. They were highly interactive and included role play, modeling, and simulation. Faculty received Continuing Medical Education credit for participation under the Institution's Educator Development Program, and faculty who participated in all sessions received a certificate of completion of the series.
Needs assessment of faculty comfort with (A) teaching concepts and (B) assessment concepts.
Evaluation
We followed standard training program evaluation principles. 10 For level one and two assessments, participants completed Workshop Evaluations on each workshop they attended. Measures included overall satisfaction with the workshop, completion of objectives, intent to change behaviors, and before and after training self-assessment of skill with teaching principles. Items were ranked on a 5-point rating scale, with 5 being the highest level. In June 2014 (six months after completion of the workshop series), participants received a 16-question survey via email to assess comfort level with covered topics and implementation into practice (Faculty Self-Assessment of Comfort and Implementation). Simultaneously, a 10-question anonymous survey was sent via email to IM residents (N = 78) and cardiology fellows (N = 19) who had rotated on cardiology clinical services from December 2013 to June 2014. This Trainee Survey of Faculty Teaching Performance assessed teaching effectiveness of cardiology faculty and inquired about the residents’ and fellows’ observation of specific teaching behaviors taught in the workshops. Participating faculty standard teaching evaluation ratings (ranked on a 5-point rating scale, with 5 being the highest) from IM residents and cardiology fellows were collected from July 2012 through July 2013 (prior to the training program) and January through June 2014 (after the training program). A timeline of the curriculum development, implementation, and evaluation is shown in Figure 2.
Timeline for curriculum development, implementation, and evaluation.
Analysis
The primary outcome was change in pre/post program overall faculty teaching rating on standard teaching evaluations. Secondary outcomes included changes within individual domains on the standard teaching evaluation. Statistical differences were determined using Wilcoxon signed-rank test and a P-value for significance of <0.05. All statistical analyses were performed using IBM SPSS statistical software version 22 (IBM Corp, released 2013). Descriptive analysis using Microsoft Excel software to report percentages was performed on faculty self-assessment of skill and comfort from before and after training, incidence of transfer of behaviors, and resident and fellow evaluations of teaching effectiveness.
Results
A total of 46% of eligible faculty (N = 23) completed the needs assessment survey. Faculty were most uncomfortable with learning goals and objectives, methods of learner assessment and evaluation, and giving feedback (Fig. 1).
A total of 82% of eligible faculty (N = 41) attended at least one of the four workshops. Characteristics of participating faculty are listed in Table 1. The majority were men (88%) with an academic rank of assistant professor (75%), which is consistent with the overall distribution of the division. The majority (71%) of participating faculty attended an inpatient cardiology teaching service from January to June 2014, and all participating faculty had clinical teaching interactions with either IM residents or cardiology fellows during this time frame.
Characteristics of participating faculty.
Workshop Evaluations revealed participants rated both the workshops and the facilitators highly (mean of 4.43/5.00 and 4.72/5.00, respectively, range of 4.0–5.0). Participants’ pre/post self-assessment of all skills improved on average from 2.94/5.00 to 4.33/5.00. The largest gains were seen in the workshops on General Principles of Learning and Teaching in the Clinical Setting (pre 2.47/5.00 to post 4.11/5.00) and Competencies, Milestones, and EPAs (pre 2.69/5.00 to post 4.47/5.00). Participants felt that they were likely to use the skills learned (mean of 4.51/5.00). Features of the workshops cited as being most helpful included learning formal models of teaching, practicing giving feedback, and the interactivity of the workshops. Suggestions on how to improve the workshops included “allowing more time for application of skills” and “education on providing effective written feedback”.
Only 12% of the participating faculty (N = 6) completed the Faculty Self-Assessment of Comfort and Implementation in June 2014. For these limited faculty, self-assessed level of comfort with various teaching concepts and activities improved compared to the needs assessment, though remained lowest in the areas of learning goals, objectives, and areas of assessment. A majority of faculty who did respond reported incorporating the learned skills into their teaching practice. Giving more direct feedback and defining specific goals and objectives were cited as examples of implementation.
A total of 37% of the surveyed IM residents (N = 29) and 53% of the cardiology fellows (N = 10) completed the Trainee Survey of Faculty Teaching Performance in June 2014. Results are summarized in Table 2. The majority of learners (90% of residents and 80% of fellows) worked with one of the participating faculty between January and June 2014. A total of 59% of residents and 40% of fellows felt that the teaching received was quite or extremely effective. The majority of these learners reported observing teaching behaviors taught in the workshops from the faculty and receiving helpful feedback.
Results from trainee Survey of faculty teaching performance.
Figure 3 shows mean teaching evaluation ratings of participating faculty. Overall teaching ratings improved significantly after the workshops (P = 0.042). All individual domains of the standard teaching evaluation improved.
Standard faculty teaching evaluations completed by IM residents and cardiology fellows before and after the faculty development program.
Discussion
We designed, implemented, and evaluated a learner-centric faculty development program to enhance clinical teaching skills for clinical cardiologists. This was well received and associated with positive perceptions of faculty teaching by trainees as well as an improvement in overall teaching evaluations. To our knowledge, this is the first report of such a program in a subspecialty medicine practice.
Faculty development programs on clinical teaching skills have previously been shown to be well received and to improve knowledge of teaching concepts and strategies as well as confidence in teaching.2,4 Our program was similarly well received and showed increased self-assessed skill level with the concepts taught. The program addressed a need for improvement in clinical teaching skills 9 and answered a call for further evaluation on behavior and teaching outcomes with faculty development programs. 11 Our program assessed teaching behaviors by surveying trainees and comparing pre and post standard teaching evaluations. Both of these assessments reflected a positive perception of faculty teaching after the program. Trainees frequently identified teaching behaviors taught in the program.
In our subspecialty practice, teaching settings for faculty are broad and require teaching cognitive, psychomotor, and affective skills. Faculty teach in both procedural and nonprocedural clinical settings. Teaching interactions may occur serially, across time or in isolated encounters. They may include learners at many different levels of training, from medical student to subspecialty fellow. Our program was designed to provide faculty development on clinical teaching skills to busy clinicians required to teach in diverse settings and across multiple learning domains. It was a priority in developing the program for faculty to be able to participate without additional disruption or demands to their daily workflow and to receive education that would impact their clinical teaching practice, mirroring the efforts that are being made more broadly in continuing professional development programs. 12 Previously, 60–90-minute modules on clinical teaching given discontinuously have been shown to improve faculty knowledge and confidence 13 and ratings on feedback and overall teaching. 5 Similarly, our program was given in short segments over the course of several months, but we integrated the workshops into normal daily workflow as opposed to the end of the workday and covered teaching concepts across multiple learning domains as well as assessment.
There are several unique strengths to our program. The covered topics were based on a needs assessment from our faculty, and they addressed concepts in which our faculty reported significant lack of comfort. The workshops were given in a format, setting, and time frame that minimized common barriers to participating in faculty development programs. We had buy-in and support from the division leadership, which allowed us to secure the time slot used. The rate of participation by eligible faculty was high, showing interest in faculty development on clinical teaching skills. With support from the Institution's Educator Development Program, faculty received Continuous Medical Education credit for participation. The series of workshops was low in cost and was not resource intensive, particularly given the impact seen.
There are limitations to our study. The program was implemented in a single subspecialty academic practice and may not be generalizable to other specialties or practice environments. The fact that relatively small numbers responded to the follow-up surveys (particularly the follow-up Faculty Self-Assessment of Comfort and Implementation) is an important limitation and raises the possibility that these results may be biased by selection. Participating faculty could have had other experiences that positively impacted their teaching behaviors in addition to the program, and trainees’ perception of faculty teaching could have been positively impacted by additional factors in the learning environment. Most of the participating faculty did not attend all of the workshops, so we may not have seen the maximum possible impact from the program.
Moving forward, methods of direct assessment of faculty teaching skills and teaching outcomes, such as peer teaching assessments or Objective Structured Teaching Examinations, can be used to further evaluate the effectiveness of such a faculty development program. In future faculty development programs, we need additional focus on methods to increase faculty attendance, potentially including incentives or repeated offerings. Our findings also suggest that efforts to provide faculty development that is even more incorporated into the daily workflow of faculty may be effective and warrant further study.
Conclusion
In our division, a needs assessment-based series of short faculty development workshops given during a standing conference time was favorably received and was associated with increased self-assessed skill level with teaching concepts, a positive perception of teaching by trainees, and an increase in faculty ratings on teaching evaluations. Effective faculty development programs can be implemented in a flexible format focused on the learner in order to help overcome common barriers to participation.
Author Contributions
All the authors have contributed to study design and data analysis and interpretation, have written and/or critically revised the manuscript, and have approved the final version.
Supplementary Material
Footnotes
Acknowledgments
We would like to acknowledge and thank the Educator Development Program Medical Education consult service for consultative services and use of Residents as Teachers and Leaders (RATL™) teaching materials supported by Office of Health Sciences Education, Vanderbilt School of Medicine. We would also like to acknowledge Erin Riley for technical editing.