Abstract
Role modelling is important in developing professionalism with a need for reliable, evidence-based tools to assess professionalism in the learning environment (LE). The Learning Environment for Professionalism (LEP) survey is brief, anonymous and balanced assessing medical trainees' and attendings’ positive and negative professionalism behaviours that can be tracked longitudinally and identify problem areas in the LE. Seven training programs agreed to facilitate administration of the LEP survey at four hospitals in Ottawa, Canada. The survey was carried out iteratively between 2013 and 2020. A total of 3783 LE ratings of training programs and hospitals were assessed longitudinally using univariate linear regression. A Bonferroni corrected p-value of ≤.0045 was used to account for multiple comparisons. Positive professional behaviours were observed across time with some of the negative behaviors having improved. A negative signal was found, with attendings appearing to be treating patients unfairly because of their financial status, ethnic background, sexual or religious preferences. Applying LEP survey longitudinally across diverse training programs and institutions is feasible and may assist programs to identify areas requiring attention and acknowledging areas of exemplary professionalism. Continuous monitoring of LE to meet requirements of accrediting bodies can also be considered an important quality improvement metric.
Introduction
Professionalism is a core medical competency and involves commitment to integrity, honesty, empathy, collaboration, confidentiality, altruism, excellence, appropriate relationships with patients, and respect for their autonomy (Asghari et al., 2011; Nagler et al., 2014). Professionalism is one of the learning outcomes of medical education and is included in multiple competency frameworks, such as the CanMEDS and CanMEDS-FM framework in Canada and the Accreditation Council of Graduate Medical Education (ACGME) list of core competencies in the United States (Frank et al., 2015; Shaw et al., 2017; The Accreditation Council for Graduate Medical Education, 2020). The teaching and evaluation of professionalism is vital to patient care and to continuous quality improvement, and is also an accreditation standard for undergraduate and postgraduate medical training in Canada (Pauls, 2012).
Professionalism is also a key component of a physician’s professional identity that develops as they progress through their clinical training. Unfortunately, there is still no unifying theoretical or practical model for teaching professionalism in the medical curriculum (Berger et al., 2020; Birden et al., 2013). Professionalism is learned through both explicit teaching and implicit learning through observing and emulating role models’ actions (Byszewski et al., 2017). For example, a review of professionalism teaching in ACGME-accredited radiology programs found that methods included role modeling, case-based scenarios, debriefing, simulations, narrative medicine, facilitated discussions, and reflections (Kelly & Mullan, 2018). Trainees have stated that their professional behaviour is strongly influenced by the academic faculty and environment (Asghari et al., 2011). There is a benefit from experiences with positive role modeling of professionalism (Marisette et al., 2020). However, role modeling can also result in imitative behaviours that perpetuate detrimental norms rather than encouraging active learning and reflection (Benbassat, 2014). This set of influences at the organizational and cultural level that affect the development of physicians’ beliefs and behaviours has been referred to as the “hidden” curriculum (Hafferty, 1998; Lawrence et al., 2018).
In a systematic review of the topic, the “hidden” curriculum is cited to be often perceived as negative or in conflict with the formal curriculum (Lawrence et al., 2018). The hidden curriculum is also considered to be a contributor to the deterioration in trainees’ professional behaviour during clinical training (Kohn et al., 2017), such as medical students becoming emotionally detached and less empathetic toward patients (Mahood, 2011). When trainees witness attending staff behaving unprofessionally with impunity, the risk of learners having professionalism lapses increases (Nagler et al., 2014). Trainees also experience dissonance when differentiating between the professional behaviours they were taught and what is being modelled to them (Kohn et al., 2017).
Given the importance of role modelling and the hidden curriculum for professionalism, it raises the importance of the LE. For example, the Committee on Accreditation of Canadian Medical Schools specifies in its standards that medical schools are required to ensure that their LE are conducive to development of appropriate professional behaviors in their medical students, faculty, and staff. In order to assess professional behaviour in the LE, there is a need for assessment tools that produce valid results and that can identify positive and negative influences on trainees’ professional development. Over the last three decades, more than 70 assessment tools have been developed to assess professionalism in the medical LE (Li et al., 2017). As demonstrated in this review, unfortunately, a majority of these assessment tools’ parent studies lacked satisfactory methodological quality including the presence of insufficient sample sizes and missing data which could have led to bias. The one exception to these issues is the Learning Environment for Professionalism Survey (LEP) developed and validated originally at the University of Arkansas (Thrush et al., 2011) (Supplemental Figure 1).
With any assessment tool, it is important to collect validity evidence in a number of contexts to ensure results are generalizable. Studies to date using the LEP have not tested it in diverse programs nor over an extended time frame. If the LEP’s efficacy could be demonstrated in these applications, then it would support undergraduate and postgraduate medical programs using the LEP to assess professionalism as per accreditation standards. The aim of this study therefore was to expand the application of the LEP survey to more varied programs and to examine LEs longitudinally over a longer period of time and across several institutions. If the tool functions as expected, it should identify trends in positive and negative professional behaviours of residents and attending physicians (in aggregate) over time.
Methods
A major strength of the LEP survey (Supplement 1) is that it is relatively brief and balanced, consisting of eleven questions that assess both positive and negative professionalism behaviours. The LEP is free to use, is filled out anonymously at the end of each four week rotation (this being a clinical placement for a given residency program at one of the four hospitals) and focuses on the LE as a unit rather than specific individuals. An added benefit is that the LEP assesses the behaviors of both trainees and attending physicians as a group anonymously and can serve to signal an area that warrants attention or recognition.
More recently, the LEP survey has been applied to the postgraduate setting at the University of Ottawa (Byszewski et al., 2017). In this pilot study conducted in 2013, residents in four residency training programs completed 187 rotation ratings using the LEP. Most items were rated favorably, though ratings for altruism and derogatory comments were rated least favorably. Overall, the study demonstrated that the LEP survey is a feasible tool to implement and can produce results with some degree of validity.
The current project was presented at the University of Ottawa Postgraduate Training Committee to all program directors with the goal to apply the LEP longitudinally over a longer period and with more diverse residency training programs. Seven program directors agreed to have their trainees participate (anesthesiology, cardiac surgery, plastic surgery, urology, family medicine, internal medicine, and pediatrics). The project coordinator then approached these programs’ administrators to facilitate adding the LEP to the University of Ottawa One45, an online evaluation and scheduling system (One45) where end of rotation evaluations were being housed. The LEP survey was administered to residents at four hospitals in Ottawa, Ontario, Canada. These were the Ottawa Hospital General Campus, Ottawa Hospital Civic Campus, Ottawa Heart Institute, and Children’s Hospital of Eastern Ontario. An information email about the study was created first to be sent out to prospective trainee participants. This email explained that participation was voluntary, all information would be confidential and not identifiable and would not affect their training. Resident consent was then obtained electronically, when completing their end of rotation evaluations. The LEP survey was carried out iteratively, that is after each four-week rotation, over a period of seven years (2013-2020) and offered to all residents, not just a sample. The surveys were completed anonymously. The rotations remained stable, and the purpose was to evaluate the LE longitudinally at the cohort level, and not the individual trainees themselves as they were completing the survey.
Residents in postgraduate training were asked to rank their observations of fellow residents and attendings using the eleven-question survey, containing five positive and six negative items. Each item was listed on a 4-point Likert scale, where 1 = never, 2 = occasionally, 3 = frequently, and 4 = consistently. Each item response was used to assess the professionalism LE of the programs in clinical settings. The LEP ratings in each training program were assessed longitudinally using univariate linear regression. A similar analysis was used to assess ratings by hospital site. For positive items on the LEP survey, having higher ratings over time is indicative of an improvement, whereas decreases in ratings over time reflects worsening in observed behaviors. Conversely, for negative items on the LEP survey, having higher ratings over time reflects worsening, whereas decreases in ratings over time indicates an improvement. To control for the increase in Type 1 error that occurs with making multiple comparisons, a Bonferroni corrected p-value of ≤.0045 (i.e., p ≤ .05/11 items) was considered statistically significant. The data were analyzed using R version 4.0.3.
Research ethics approval was obtained from the Ottawa Hospital
Results: (Data as Figures Available in Supplement 2)
Number of Responses Received by Training Programs and Sites, (2023 Data as Proxy Representative Total Possible Responses)
Overall Findings
There were several statistically significant, though small, changes in ranking over time across items. The frequency of negative professional behaviour observed for residents and attending physicians significantly decreased for four of the five negative items, item 2 (derogatory comments about patients and their families; residents p < .001, attendings p < .001), item 6 (derogatory comments about healthcare providers; residents p < .001, attendings p < .001), item 8 (treating non-physician healthcare workers inappropriately; residents p < .001, attendings p < .001), and item 10 (inappropriately discussing confidential information; residents p < .001, attendings p = .003) (Figure 1). There was no significant reduction in negative item 4 (inappropriately withholding information or intentionally giving incorrect information to patients; residents p = .05, attendings p = .73).
However, there was a significant increase in negative professional behavior for item 9 for attending physicians (treating patients unfairly because of their financial status, ethnic or religious background, or sexual preferences; residents p = .07, attendings p = .004).
While overall ratings were positive, no significant change in positive items was observed in item 1 (positive role modeling of the doctor-patient relationship; residents p = .01, attendings p = .005), item 3 (educating patients about their illnesses; residents p = .40, attendings p = .03), item 5 (valuing human contact with patients; residents p = .25, attendings p = .07), item 7 (concern about patients’ overall wellbeing; residents p = .08, attendings p = .36), or item 11 (placing patients need ahead of their own interest; residents p = .08, attendings p = .27).
Subgroup Analysis by Program
Several programs demonstrated significant improvements over time on positive and negative items. There was no significant deterioration in any of the domains.
Anesthesiology
766 LEP surveys were received. There were significant improvements in negative item 2 (for attendings p < .001, but not for residents p = .03) and item 8 (for residents p = .002, but not for attendings p = .01). There were no significant changes in positive items 1, 3, 5, 7, 9, 11 and negative items 4, 6, 10 (Figure 2).
Cardiac Surgery
46 LEP surveys were received. There were significant improvements in all positive items: 1 (residents p < .001, attendings p < .001), item 3 (for residents p < .001, but not for attendings p = .02), item 5 (residents p < .001, attendings p < .001), item 7 (residents p < .001, attendings p = .002), and item 11 (residents p < .001, attendings p < .001). There were no significant changes in negative items 2, 4, 6, 8, 9 and 10 (Figure 3).
Family Medicine
744 LEP surveys were received. There were significant improvements in negative item 2 (residents p < .001, attendings p < .001) and item 6 (residents p < .001, attendings p < .001). There were no significant changes in positive items 1, 3, 5, 7, 11, and negative items 4, 8, 9 and 10 (Figure 4).
Internal Medicine
1278 LEP surveys were received. There were significant improvements in positive item 7 (for residents p = .003, but not for attendings p = .04), item 11 (residents p < .001, attendings p < .001) and negative item 2 (residents p < .001, attendings p < .001), item 6 (residents p < .001, attendings p < .001), and item 8 (for residents p = .003, but not for attendings p = .08). There were no significant changes in positive items 1, 3, 5 and negative items 4, 9, 10 (Figure 5).
Pediatrics
710 LEP surveys were received. There was a significant improvement for negative item 8 (residents p < .001, attendings p < .001). There were no significant changes in positive items 1, 3, 5, 7, 11 and negative items 2, 4, 6, 9, 10 (Figure 6).
Plastic Surgery
65 LEP surveys were received. There were significant improvements in positive item 7 (for residents p = .003, but not for attendings p = .02), item 11 (for residents p = .003, but not for attendings p = .01). There were no significant changes in positive items 1, 3, 5, and negative items 2, 4, 6, 8, 9, and 10 (Figure 7).
Urology
174 LEP surveys were received. There was a significant improvement in negative item 10 (residents p = .002, attendings p < .001). There were no significant changes in positive items 1, 3, 5, 7, 11, and negative items 2, 4, 6, 8, and 9 (Figure 8).
Subgroup Analysis by Hospital
CHEO
749 LEP surveys from this site were received. There was significant improvement in negative item 8 (residents p < .001, attendings p < .001). There were no significant changes in positive items 1, 3, 5, 7, 11, and negative items 2, 4, 6, 9, and 10 (Figure 9).
Civic Hospital
1725 LEP surveys from this site were received. There were significant improvements in positive item 3 (for attendings p < .001, but not for residents p = .02), item 7 (residents p = .002, attendings p = .004), item 11 (for attendings p = .002, but not for residents p = .01) and negative item 2 (residents p < .001, attendings p < .001), item 6 (residents p < .001, attendings p < .001), item 8 (residents p < .001, attendings p < .001), and item 10 (residents p < .001, attendings p < .001). There were no significant changes in positive items 1, 5 and negative items 4, 9 (Figure 10).
General Hospital
1295 LEP surveys from this site were received. There were significant improvements in negative item 2 (residents p < .001, attendings p < .001), item 6 (residents p < .001, attendings p < .001), and item 8 (for residents p < .001, but not for attendings p = .01). There were no significant changes in positive items 1, 3, 5, 7, 11 and negative items 4, 9, 10 (Figure 11).
Heart Institute Hospital
14 LEP surveys from this site were received. There were no significant changes in any of the positive or negative items (Figure 12).
Discussion
This longitudinal study demonstrated significant improvements in 4 of the 5 negative items over time, specifically item 2, 6, 8 and 10. However a concerning finding of this study is the fact that the negative item related to attendings treating patients unfairly because of their financial status, ethnic background, sexual or religious preferences worsened somewhat over the years (item 9). Collecting validity evidence for any assessment tool is an ongoing process to ensure that results will generalize to new contexts and are replicable. The goal of this study was to determine if an existing assessment tool for professionalism could be generalized across diverse programs and over time. As was found in previous studies (Ashari et al., 2011; Byszewski et al., 2017) the present study demonstrated generally positive professional behaviours in the LEP across time and programs. Perhaps this is due to increasing awareness and attention on professionalism in the LE in recent years, and the resulting efforts of training programs to continuously improve on this aspect of the LE. One concern of professionalism evaluations can be a reluctance to report colleagues’ and attendings’ negative behaviours. The hierarchical nature of medical institutions contributes to trainees’ hesitation to report their superiors, due to fear of retaliation and potential ramifications on their own evaluations and professional opportunities. Kohn and colleagues found that students reported major violations 70%–90% of the time, however, they reported minor or moderate violations only 30% of the time (Kohn et al., 2017). The LEP survey’s design mitigates these concerns by avoiding the need to identify specific incidents or names and is part of a larger and extemporaneous monitoring strategy that would not link a particular trainee to the results. A definite concerning finding of this study is the fact that the negative item related to attendings treating patients unfairly because of their financial status, ethnic background, sexual or religious preferences worsened somewhat over the years (question 9). To determine what factors could have contributed to possible similar trends, one LEP study had third-year medical students fill out the survey at the end of their clerkship and then they used focus groups to explore the types of derogatory comments observed, why the comments were made, and how they were interpreted (Tariq et al., 2016). In most cases, it was only one or two staff members who made derogatory comments, and students felt the comments were associated with cumulative stress and burnout. Similar factors may have contributed to our findings as well and it is reassuring that trainees can clearly identify when attendings unfairly treated their patients. Likewise, in this same study by Tariq and colleagues, medical students were able to recognize staff physicians making derogatory comments about patients (Tariq et al., 2016). As suggested by Benbassat, training students and residents to identify inappropriate behaviour is important for preventing the propagation of negative traits and allowing trainees to make the most of positive modeling (Benbassat, 2014).
Next Steps
As mentioned, demonstrating validity is an ongoing process and therefore next steps should extend the application to additional areas to ensure valid results are produced. For example, the LEP could possibly be applied to wider stakeholder groups in the future, including patients, nurses, allied health, and faculty/staff physicians. Further to that, future studies can include more diverse survey respondents to determine whether this leads to greater reporting of unprofessional behaviour. Alexis and colleagues found that marginalized groups (e.g., women, gender and sexual minority groups, ethnic and religious minorities, and individuals with disabilities) were more likely to report lapses in professionalism (Alexis et al., 2020). These characteristics were not included in our study.
As well, the results of the LEP survey, once implemented as part of standard LE evaluation, could be shared with the participating residency programs to initiate measures to improve the LE professionalism. Subsequently, the LEP survey could be re-administered to evaluate whether there were any significant improvements following the implementation of measures aimed at promoting positive and mitigating negative influences on professionalism.
Limitations
One limitation is that the LEP survey has only been applied to learners, neglecting input from their supervisors and patients observing interactions within the LE. As a result, this study and previous studies using the LEP survey may have overlooked the perspectives of other stakeholders in the LE. For example, Ashgari and colleagues presented a different questionnaire to residents, staff physicians, nurses and patients (Asghari et al., 2021). They found that residents scored the professionalism environment lower than nurses and faculty members, perhaps because of their increased sensitivity to its flaws. Another observation was that faculty are reluctant to report the professionalism lapses of their trainees and colleagues, as evidenced by Ziring and colleagues (Ziring et al., 2018). This was due to several barriers, including uncertainty about the reporting process, not having all the facts related to incidents, the potential effects on learners, and time constraints. Overall, these studies suggest that trainees, staff physicians, and allied health may rate professionalism in the LE differently, which has yet to be demonstrated with the LEP survey.
Not all programs agreed to participate, perhaps because they felt that the results of the survey could negatively reflect on their LE. However, since study completion, all undergraduate and postgraduate medical programs at the University of Ottawa have adopted the LEP survey as part of their longitudinal evaluation of the clinical LE, making the LEP survey mandatory. Doing so will support interventions to improve the LE and help uncover concerns or trends that if left unrecognized could propagate the hidden curriculum.
A further limitation of this study was variability in yearly response rates, which could have impacted our ability to observe changes over time. There was a gradual increase in survey responses from 2013 to 2016, a relative plateau between 2016 and 2018, followed by a fall in 2019 and 2020. Given that the survey was voluntary, some people may have decided not to participate due to the factors we previously discussed (e.g., reluctance to report) likely affecting the variable response rate. The administrative support challenges with One45 and the Covid epidemic in 2020 prevented an accurate tally of the surveys distributed. As a proxy measure (given that trainee and rotation numbers have remained mostly unchanged as verified with the postgraduate office) Table 1 includes data for possible rotation surveys numbers for 2023.
The sample size of survey responses by residents varied but these were different participants responding to the survey (as the residents progressed in their training), but the rotations remained constant. The objective of the study was to examine how the LE changed over time based on the LEP survey responses in specific rotations across time longitudinally. However, we believe that as the cohorts were stable, given selection criteria remained the same and there was no major longitudinal change in the postgraduate curriculum, that this was not a fatal error.
Conclusion
This study demonstrated that applying the LEP survey longitudinally across a diverse set of training programs is feasible. Additional benefits are that the LEP survey is an anonymous, short, and balanced tool that can be used by programs and institutions as a means of quality assurance to identify areas that need attention and to acknowledge areas of exemplary professionalism. We have demonstrated implementation of the LEP survey as a feasible strategy for continuous monitoring of the LE which can be shared with accrediting bodies and can be considered as an important quality improvement metric. Our results suggest that overall positive professional behaviours were observed across time and programs. However, one negative item relating to treating patients unfairly because of their financial status, ethnic background, sexual or religious preferences worsened which may indicate enhanced attention to these facets of medicine or represent a deterioration in compassion toward some patient groups. Possibly frustration on the part of health care professionals, due to systemic issues including lack of resources, such as access to social work, primary care and drug coverage. Such areas could be the focus of future quality improvement initiatives, as professionalism has been connected to positive patient outcomes (Nagler et al., 2014). This is particularly important given the current efforts being made to promote and improve professionalism as it relates to equity, diversity, and inclusion within healthcare and the LE.
Supplemental Material
Supplemental Material - Professionalism in Residency Training: The Learning Environment for Professionalism Survey
Supplemental Material for Professionalism in Residency Training: The Learning Environment for Professionalism Survey by Anna Byszewski, Alexander Pearson, Heather Lochnan, Donna L. Johnston, Sharon Whiting, and Timothy Wood in Evaluation & the Health Professions.
Supplemental Material
Supplemental Material - Professionalism in Residency Training: The Learning Environment for Professionalism Survey
Supplemental Material for Professionalism in Residency Training: The Learning Environment for Professionalism Survey by Anna Byszewski, Alexander Pearson, Heather Lochnan, Donna L. Johnston, Sharon Whiting, and Timothy Wood in Evaluation & the Health Professions.
Footnotes
Acknowledgments
We are grateful to all the postgraduate programs who participated in this study, as well the program directors and program assistants that made this project possible through data collection. We also acknowledge Marijana Bijelic for her assistance with data analysis and Elizabeth Lacasse with manuscript journal formatting for submission.
Author Contributions
AB, HL, TW, DJ made a substantial contribution to the conception, design and acquisition of data for of the study. AB, HL, TW, DJ, SW, AP made a substantial contribution to the analysis as well drafting the work and subsequent revisions. All authors have approved the final submitted version. All authors agree to be both personally accountable for the author’s own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Data collection supported by funds from The Department of Medicine Medical Education Research grant and University of Ottawa Postgraduate Medical Education Initiatives grant. Data analysis supported by Office of Professionals Affairs, Faculty of Medicine, University of Ottawa.
Ethical Statement
ORCID iDs
Data Availability Statement
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
