Abstract
Objective
This survey study assessed trainee attitudes and perceptions of the longitudinal public speaking curriculum at Mayo Clinic Pediatric and Adolescent Residency Program.
Methods
This 3-part survey of Mayo Clinic pediatric residents was conducted from 2013 to 2017 and utilized yes/no questions, Likert scales, and open-ended questions. Code book content analysis was used to analyze open-ended questions.
Results
A total of 121 participants (96.8%, n = 121/125) completed the survey. Most respondents (93.3%, n = 111/119) either agreed or strongly agreed that the public speaking requirement enabled them to be more effective in delivering information to large groups. Most respondents (89.2%, n = 107/120) agreed or strongly agreed that they incorporated feedback from previous public speaking feedback sessions into subsequent presentations. The 2 activities that respondents found most useful for preparing later presentations were setting goals for future presentations in a feedback session and discussing performance in a feedback session.
Conclusions
There was a perceived effectiveness of a longitudinal public speaking curriculum among pediatric residents at our program. Multidimensional feedback seems to be a critical aspect of a public speaking curriculum in graduate medical education.
Introduction
Physicians’ communication and interpersonal skills are included in the Accreditation Council for Graduate Medical Education (ACGME) competencies and are frequently discussed in the literature. 1 By contrast, little attention has been given to the formal public speaking skills of physicians. While physicians are commonly expected to present information through public speaking (eg, lectures, grand rounds, research presentations, and advocacy), few trainees receive any structured training.
Public speaking anxiety appears to be as common in medical students and residents as in the rest of the population. The prevalence of public speaking anxiety in the general population is reported to be between 15% and 30%. 2 One study found that ∼17% of residents and medical students experienced symptoms of anxiety during public speaking. 3 Another study found that medical students had a medium-to-high level of communication apprehension, with public speaking being the most troubling dimension of communication. 4
There are few published reports of standardized public speaking curricula for physicians and physicians-in-training, though one report describes a public speaking curriculum focused on morbidity and mortality presentations, and another report describes an apprentice-ship style program focused on academic conference presentations. However, much literature focuses on how to advice,5–9 and little-to-no existing literature reports on either the subjective or objective effectiveness of public speaking curricula. Those who have studied public speaking anxiety in GME conclude (a) public speaking anxiety may be under-recognized in GME, and (b) medical educators and program directors should incorporate curriculum to improve public speaking skills. 3
A longitudinal, 3-year public speaking skills curriculum grounded in the conceptual frameworks of deliberate practice and reflective practice has been incorporated into the training of pediatric residents at Mayo Clinic in Rochester, Minnesota.10–12 This curriculum has been previously described and includes requirements to present a formal lecture-style continuing medical education presentation to the department with the guidance of a formal faculty advisor; uses a standardized evaluation/feedback form; and offers in-person feedback sessions for participants utilizing videotaped recordings, slide review, composite evaluations, self-reflection, and concludes with goal setting. Evaluation of the participants in the curriculum has demonstrated quantifiable improvement in public speaking skills as measured through statistical analysis of the composite evaluations. 12
The goal of this qualitative study is to survey trainee attitudes, perceptions, and application of the longitudinal public speaking curriculum components to identify strengths and areas for further curricular development, generalizability, and dissemination.
Methods
This study was determined to be exempt by the Mayo Clinic Institutional Review Board (IRB) (ethics committee). An anonymous 9-question survey was developed by the curriculum director about each key curricular component and distributed using the Survey Monkey tool (https://www.surveymonkey.com/) (Table 1). A total of 3 attempts at contact were made. Participants did not receive compensation for participation. Consent for participation was implied by participants who chose to complete the survey questions as approved by IRB review. The survey tool collected data with no identifying information to protect the anonymity of participants.
Survey Tool Used to Assess Residents’ Perceptions of a Public Speaking Program.
Pediatric resident graduates (n = 125) from classes entering our institution between 2004 and 2014 were contacted by email for participation in the study at 2 points in time (2004-2010 surveyed in 2013 and 2011-2014 surveyed in 2017). Responses were included if respondents reported attending pediatric residency training at Mayo Clinic and presenting 2 or more formal lectures as part of their training. All responses from the 2 survey periods were aggregated into one data set for analysis.
The qualitative survey assessing trainee attitudes and perceptions of the longitudinal public speaking curriculum was divided into 3 parts. Part 1 (Questions 1-3, “Yes” or “No”) asked whether respondents presented multiple lectures and received feedback as part of our resident curriculum. Part 2 (Questions 3-6, Likert scale) assessed the respondents’ perception of the quality and utility of the curricular components, including the feedback they received. Likert scale options and number correlates included 1 = strong disagree; 2 = disagree; 3 = neutral; 4 = agree, and 5 = strongly agree. Part 3 (Questions 7-9, free text) invited respondents to reflect on the educational quality of the curriculum (Questions 7-8) and provide any additional comments about their overall educational experience in this program (Question 9).
Free text responses for Question 7 (most valuable part of learning experience) and Question 8 (opportunities to improve learning experience) were evaluated using codebook-based content analysis. One author (JRG) reviewed all free-text responses and inductively identified common themes among the responses. He subsequently created preliminary codebook definitions for each of the identified themes. Two coders (TJB and JHH) applied this codebook to the free text responses. Nonsubstantive responses (eg, “No suggestions for improvement” and “Nothing I can think of”) were excluded. The 2 coders and codebook author jointly reviewed 10 randomly selected responses to each question to identify ambiguities or oversights, and disagreements were resolved by consensus. Minor codebook edits and clarifications were made during this review process. The final, edited codebook was then applied to all remaining responses by the 2 coders. Discrepancies were resolved by a third coder (DJH). Interrater reliability was calculated using Cohen's kappa, and the 10 jointly reviewed responses were excluded from interrater reliability calculations. Free text responses for question 9 (additional comments) were reviewed by a single author (JRG). The reporting of this study conforms to the Defined Criteria To Report INnovations in Education (DoCTRINE) statement 13 (Supplemental Material completed checklist).
Results
A total of 121 participants (96.8%, n = 121/125) completed the survey and reported presenting 2 or more formal lectures in our resident public presentation series. Of the 121 respondents included in this study, 116 (95.9%) confirmed receiving written or verbal feedback about one or more of their presentations.
Most respondents (93.3%, n = 111/119) either agreed or strongly agreed (weighted mean Likert scale of 4.40) that a public speaking requirement equipped them to be more effective in delivering information to large groups. Most respondents (89.2%, n = 107/120) agreed or strongly agreed (mean = 4.01) that they incorporated feedback from previous public speaking feedback sessions into future presentations.
Self-reported improvement in public speaking comfort level was assessed through 5 components of the feedback sessions (Question 5, Table 1). These results are contained in Table 2.
Domain of Feedback and Its Receptivity. Likert Scales Allowed for Ranking Each Domain from 1 to 5 with 1 Being Strongly Disagree and 5 Being Strongly Agree.
Many respondents (89.3%, n = 108/121) provided free-text responses for Question 7: “What part of this experience provided the most learning and why?” Four main themes were identified among these responses: postpresentation feedback, opportunity to present, preparation, and watching other residents. Full codebook definitions are presented in Supplemental Appendix A. Responses with a theme of postpresentation feedback discussed feedback provided by an advisor or audience member following the presentation. Responses with a theme of opportunity to present discussed the value of the actual act of giving the presentation. Responses with a theme of preparation discussed work done prior to a presentation, including researching, creating, and practicing the presentation. Responses with a theme of watching other residents discussed the value of observing other presentations. Coders showed substantial-to-perfect agreement coding these themes (median k = 0.96, range 0.76-1.00). See Supplemental Material for full interrater reliability results.
Over half of respondents (61%, n = 65/108) stated that feedback provided for the most learning benefits (“The extensive feedback after the presentation was extremely helpful. Focusing on specific areas of improvement based on reviewing my slides and the video was great.”). Many residents specifically noted the utility of reviewing recorded videos of their presentations (“I hated watching the video, but it was extremely useful to do so.”). Thirty-two responses (29.6%, n = 32/108) mentioned that the opportunity to present provided the most learning benefit (“Great to have the longitudinal experience of doing one each year…”). Thirty-two responses (29.6%, n = 32/108) discussed that the process of preparation (including preparation sessions with faculty) provided the most learning benefit (“Preparing for it with the help of a mentor provided the most learning…”).
Three respondents (2.8%, n = 3/108) felt that learning from other residents’ talks provided the most learning benefit (“…seeing other residents do it”).
Sixty-nine (57.0%) former trainees provided substantive comments for Question 8: “What could be altered to make this an even more useful learning experience?” Two major themes emerged from these comments: additional guidance and timely feedback. Full codebook definitions are presented in Supplemental Appendix A. Responses with the theme of additional guidance expressed a desire for more targeted or more accessible guidance prior to delivering the presentation. Responses with the theme of timely feedback expressed a desire for more expeditious feedback (or, in some cases, for any feedback at all). Coders showed almost perfect agreement in identifying the guidance theme (k = 0.90) and substantial agreement in identifying the timely feedback theme (k = 0.72)
Almost half of all respondents (48%, n = 33/69) expressed a desire for more guidance (“More support during the planning process would be helpful”). Over a quarter of respondents (27.5%, n = 19/69) stated they would have liked more timely feedback (17.2%, n = 15/87) (“The feedback session was sometimes quite delayed from the time of the presentation. As much as possible, feedback directly after the presentation may be more memorable.”). Additional responses did not have significant overlap and discussed miscellaneous topics, including venue, technology, and time constraints.
An optional Question 9 allowed for additional comments about the public speaking curriculum. A total of 34 respondents provided further details of the experience. Twenty-one (61.8%, n = 21/34) individuals stated that the opportunity to practice in this setting improved their ability to present in subsequent public speaking engagements.
Discussion
Public speaking is a skill that can be developed and improved through deliberate practice, reflection, and feedback. However, few studies have focused on public speaking in a graduate medical education (GME) environment. Nie et al 9 wrote about their framework for teaching public speaking to medical students at Yale. They discussed their approach to teaching 3 components of public speaking: delivery, improvisation, and speaking. There is a scarcity of literature describing curricula to teach public speaking skills, along with the methods for assessing the efficacy/outcomes of public speaking programs in GME. Our study found a highly positive opinion overall of a resident public speaking program, with 93.3% of respondents either agreeing or strongly agreeing that a formalized public speaking requirement enabled them to become more effective in delivering information to large groups.
The importance of structured feedback emerged repeatedly as a key aspect of our program to optimize resident educational experiences. Survey participants felt that more guidance, mentorship, and timely feedback would have further improved their educational experience in this program. High proportions of respondents incorporated feedback and what they learned into future presentations. These findings are consistent with the adult learning principle of deliberate practice, in which participants are instructed to improve an aspect of a well-defined task, receive feedback on their performance, and are provided opportunities to improve their performance by performing a similar task repeatedly. 10 Similarly, the opportunity to reflect on one's own experience (eg, videotape review) and set goals for the future build on the educational framework of reflective practice wherein adults learn through reflection-in-action and reflection-on-actions. 11
The strengths of this study included a high response rate and high rates of respondent participation in the curriculum, which minimizes reporting bias. Anonymous results reported in this study likely reflect the true opinions of residents participating in this curriculum, as they were obtained after graduation. There is no reason to suspect a more favorable survey response out of concern that a negative response could be used against the respondent. Limitations to this study include data derived from a single residency program, the qualitative nature of the data, and the potential for recall bias since some of the graduates received the survey up to 6 years after completion of the program.
This study helps fill an important gap in the GME literature by demonstrating the perceived effectiveness and accessibility of a longitudinal public speaking curriculum from the recipient's perspective, complementing previously published information about observed (quantitative) improvement in the public speaking skills of curriculum participants. 12 This indicates both acceptability and effectiveness of this type of curriculum. Public speaking is a valuable tool for physicians interested in research, advocacy, and education, and is a skill, like any other, that can be developed and improved. A high percentage of learners experience a significant degree of anxiety with this activity and are therefore likely to avoid it in the absence of a formalized requirement for deliberate practice in a supportive training environment. Ultimately, like so many other competencies in medicine, great speakers are not born; they are made.
Given our findings, we feel a program such as ours is advantageous for medical trainees and could be easily reproduced by other residency programs. Future studies should focus on (a) additional quantitative assessments of public speaking curricula outcomes in GME and (b) a detailed description of existing public speaking curricula in GME, including resident physicians.
Conclusion
Public speaking is an important skill for health professionals. There is often no formal instruction provided to develop public speaking for physicians in training. The inclusion of a longitudinal structured public speaking curriculum that includes learning objectives, instruction strategies, opportunities to practice, and direct feedback was positively endorsed by physician trainees in this study, and they reported a perceived effectiveness. Offering this type of training to others may have similar benefits.
Supplemental Material
sj-docx-1-mde-10.1177_23821205251390313 - Supplemental material for Longitudinal Public Speaking Curriculum for Residents: Trainee Attitudes and Perceptions
Supplemental material, sj-docx-1-mde-10.1177_23821205251390313 for Longitudinal Public Speaking Curriculum for Residents: Trainee Attitudes and Perceptions by Jacob R. Greenmyer, Tyler Benning, Richmond Castillo, James Homme, Erin Knoebel, Nathaniel W. Taggart, Molly Youssef, David Hall and Jason H. Homme in Journal of Medical Education and Curricular Development
Supplemental Material
sj-docx-2-mde-10.1177_23821205251390313 - Supplemental material for Longitudinal Public Speaking Curriculum for Residents: Trainee Attitudes and Perceptions
Supplemental material, sj-docx-2-mde-10.1177_23821205251390313 for Longitudinal Public Speaking Curriculum for Residents: Trainee Attitudes and Perceptions by Jacob R. Greenmyer, Tyler Benning, Richmond Castillo, James Homme, Erin Knoebel, Nathaniel W. Taggart, Molly Youssef, David Hall and Jason H. Homme in Journal of Medical Education and Curricular Development
Footnotes
Ethical Considerations
Institutional Review Board (IRB) exempt status.
Consent to Participate
Individuals consented to participate in the study, provided as part of the agreement to participate in the survey completion.
Authors’ Contributions
Dr Jason Homme conceptualized and designed the study, drafted the initial manuscript, and reviewed and revised the manuscript. Drs Greenmyer and Benning carried out data analysis and reviewed and revised the manuscript. Dr Hall carried out data analysis and reviewed and revised the manuscript. Drs Castillo, Homme (James), Knoebel, Taggart, and Youssef assisted in data collection and reviewed and revised the manuscript.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this paper is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
