Abstract
Background:
Learning from managing patients in a real-world context is by far superior to simulation learning. A substandard educational environment is associated with poor patient care and suboptimal learning outcomes. The measurement of the educational environment provides insight into what is needed to improve the level of training.
Objective:
To measure the educational environment as perceived by trainees within the Malta Foundation Programme.
Methods:
This study used the Postgraduate Hospital Educational Environment Measure (PHEEM) to measure the educational environment at the Malta Foundation Programme. Descriptive statistics were used to describe the demographics of the study population. Nonparametric comparative statistics were used to identify statistically significant differences between groups.
Results:
Ninety-eight trainees out of 370 (26.5%) completed the online questionnaire. These consisted of 39 FY1s (31.5% of 124), 33 FY2s (24.8% of 133), and 26 extended FYs (23.0% of 113). The 40-item PHEEM showed good reliability with a Cronbach’s α value of .912. These doctors perceived their educational environment as more positive than negative. Perceptions are worst among trainees at the end of their first year of training. Those who had just finished their training have reported better perceptions. The 3 worst scoring items are related to when the trainees are on call.
Conclusion:
Among trainees within the Malta Foundation Programme, perceptions of role autonomy and social support are areas where most work is needed. Teaching seems to be moving in the right direction, but there is always room for improvement.
Introduction
For healthcare professionals, there is no substitute for learning in the clinical environment. 1 Learning from managing patients in a real-world context is by far superior to simulation learning. Over the past 3 decades, the subject of the educational environment, both at the undergraduate and postgraduate levels, has been the focus of much research and discussion among all stakeholders in medical education. Both the General Medical Council (GMC) 2 and Kilty et al 3 have underlined that a substandard educational environment is associated with poor patient care and suboptimal learning outcomes. Not only do young trainees learn from their work with patients, but patient care would suffer were these trainees to be removed from teaching organizations as their work is essential for any healthcare organization. This is further confirmed by the GMC 2 when it states, “patient safety is inseparable from a good learning environment and culture that values and supports learners and educators.” The educational environment is, therefore, considered an important measure in both undergraduate and postgraduate medical training.
Postgraduate medical trainees have a dual contrasting role. On the one hand, they are trainees transitioning to hospital life, still in need of support and still adapting to their new responsibilities. 4 On the other hand, as practicing doctors, they need to provide answers to patients’ questions, work long hours, be accountable for their actions and may find inadequate support from their more senior colleagues. Trainees may feel under constant evaluation by their patients, colleagues, and senior grades. 5 This dual role is known to increase the risk of stress, anxiety, depression, and burnout. 6 Medical trainees are known to be one of the most vulnerable categories of employees to occupational stress. 7
The MACY Foundation
8
sees the educational environment as: “the social interactions, organizational cultures and structures, and physical and virtual spaces that surround and shape participants’ experiences, perceptions, and learning.”
8
The American Medical Association defines the educational environment as: “a social system that includes the learner, the individuals with whom the learner interacts, the setting(s) and purpose(s) of the interaction, and the formal and informal rules/policies/norms governing the interaction.”
9
These definitions, among others, identify the complexity of the educational environment when compared to education in a class room.
The measurement of the educational environment provides insight into what is needed to improve the level of training. This is done by identifying the weaknesses and strengths of a training program and which areas should be prioritized in any improvement that is planned.
For many years, the educational environment was ignored, possibly because of the lack of suitable and validated instruments that could be used to assess it. This has now changed, and a number of validated instruments have been used to assess the different aspects or domains of the educational environment. One of the first to be developed was the Dundee Ready Education Environment Measure (DREEM). 10 DREEM uses 5 subscales in its evaluation of the education environment:
Students’ perception of learning
Students’ perception of teachers
Students’ academic self-perceptions
Students’ perceptions of atmosphere
Students’ social self-perceptions
Following DREEM, a number of other instruments aimed at assessing the education environment in different settings were developed and validated. List 1 provides a selection of instruments for evaluating specific learning environments in medical education.
List 1 11
PHEEM is a 40-item questionnaire scored on a Likert scale as follows:
4—Strongly agree
3—Agree
2—Uncertain
1—Disagree
0—Strongly disagree.
PHEEM is sub-divided into 3 sub-scales, each measuring perceptions of role autonomy (14 items), teaching (15 items), and social support (11 items). When interpreting the scores for each of the 3 sub-scales, Roff et al 26 recommend the following schema:
I. Perceptions of role autonomy
0 to 14—very poor
15 to 28—a negative view of one’s role
29 to 42—a more positive perception of one’s role
43 to 56—excellent perception of one’s job
II. Perceptions of teaching
0 to 15—very poor quality
16 to 30—in need of some retraining
31 to 45—moving in the right direction
46 to 60—model teachers
III. Perceptions of social support
0 to 11—non-existent
12 to 22—not a pleasant place
23 to 33—more pros than cons
34 to 44—a good supportive environment.
Roff et al 26 recommend the following interpretation of the overall score:
0 to 40—very poor
41 to 80—plenty of problems
81 to 120—more positive than negative but room for improvement
121 to 160—excellent
Individual items can be scored as follows:
>3.5—very positive point
<2—problem area
2 to 3—item can potentially be improved.
PHEEM was specifically created to measure the educational environment within a postgraduate setting. Thus, PHEEM was an ideal tool for this study.
With the use of different measures of the education environment now widespread in the literature, such instruments can be used to gain holistic views of the curriculum; understand the students’ perceptions of learning, teaching, and the atmosphere; compare the different perceptions of the various stakeholders; compare environments between schools or departments; provide organizations with indications of what needs to be changed or improved; and evaluate the results of any changes made to the education environment. 29
Objective
To measure the educational environment as perceived by trainees within the Malta Foundation Programme.
Methodology
The aim of this study was to analyze the educational environment within the Malta Foundation Programme as perceived by the trainees within the program themselves.
The Malta Foundation Programme (MFP) 30 was founded in 2009. The Malta Foundation Programme is a 2-year training program for newly graduated doctors consisting of structured teaching, hands-on training, and assessments while working in a supervised hospital or primary-care environment.
As the MFP bases its operations on the same Reference Guide while offering the same Curriculum and training opportunities as the UK Foundation Programme (UKFPO), the MFP was awarded the status of Affiliate Program by the UK Foundation Programme. This status has ensured that trainees of the MFP, upon completion of their programme, can compete on the same level as those completing the UK Foundation Programme for training posts in specialities in Malta, the UK, or elsewhere. This affiliation has since been renewed at regular intervals after the MFP repeatedly fulfilled the Quality Standards of the Malta Medical Council and the UKFPO. 30
A cross-sectional observational method was chosen for a study of the educational environment and burnout among foundation doctors in Malta. It is the author’s personal opinion that the educational environment at the Malta Foundation Programme may be related to possible burnout among the trainees. The aim of the study was to analyze both the educational environment and burnout at the Malta Foundation Programme. This paper reports on the results for the educational environment. The other results are reported elsewhere.
The instrument chosen to assess the educational environment is the PHEEM questionnaire administered online through Google Forms to all doctors within the Malta Foundation Programme. The PHEEM questionnaire has been validated in various settings and in different countries. Minor changes were made to the wording of the questionnaire to ensure that each question was relevant to the Maltese setting. Direct correspondence with the original author of the PHEEM questionnaire ensured permission to use and validity of the minor changes to the wording.
PHEEM was administered in English, as all medical tuitions in Malta are carried out in English. The questionnaires were distributed to all foundation program doctors via their year representatives in the form of a Google Form online questionnaire. As the Foundation Programme does not offer the service of forwarding emails directly, the questionnaire was initially sent to the Foundation School Secretary, who then passed it on to the foundation doctors’ representatives. A reminder was sent after 7 days to increase the response rate. All respondents were requested to submit solely a single reply, especially as this was an anonymous questionnaire. The period of data collection coincided with a period when 3 cohorts of foundation doctors could be sampled. The first years were sampled at the end of their first rotation, 3 months into practice. The second years were sampled between their first and second years, a year into practice. A third category, though technically not under the responsibility of the Foundation Programme, was the extended Foundation doctors. These were doctors who had successfully finished their foundation training and were waiting for their BST posts to be decided. These trainees were sampled at the end of their 2-year foundation training. The sampling period ran between July and August 2020 for FY2s and extended FYs and in October 2020 for FY1s.The deadline for the collection of data was 2 weeks after the reminder was sent.
The responses were analyzed using SPSS 25.0. Questions 7, 8, 11, and 13 are negatively worded statements and need to be reversed for scoring. Reliability tests of the whole instruments and the 3 factors were performed using Cronbach’s alpha. Descriptive statistics were used to describe the demographic features of the respondents. Tests for normality (Kolmogorov-Smirnov and Shapiro-Wilk) for all factors revealed that no question was normally distributed. Therefore, all comparative statistics were of the nonparametric type. The Mann-Whitney U test was used to analyze differences between sexes. The Kruskal-Wallis H test was used to test for differences between foundation years. The threshold for statistical significance was a P < .05 and 95% confidence interval. Factor analysis of all 40 items was also performed using both the scee plot and a criterion of an eigenvalue >1.
Ethics approval and consent to participate
Ethical approval was obtained from the University of Malta Faculty Research Ethics Committee, and approval to disseminate the questionnaire was obtained from the Malta Foundation Programme.
A covering letter and a participant information sheet accompanied the questionnaire. Informed consent was obtained from all participants. All methods were carried out in accordance with relevant guidelines and regulations.
Results
Demographics
Table 1 describes the response rates for the different cohorts. 60.2% were female and 94.9% were single. 87.8% were Maltese. A total of 86.7% worked in excess of the 48-hour week stipulated in the European Working Time Directive.
Response rates.
Internal consistency
The 40-item PHEEM showed good reliability with a Cronbach’s α value of .912. Cronbach’s α for each of the 3 subscales of the PHEEM was as follows:
Autonomy—.790
Teaching—.885
Social support—.683
Item analysis
The responses of each FY cohort to each of the 40 items forming up the PHEEM are listed in Table 2.
Summarizes the responses to each of the 40 questions according to year.
Table 3 illustrates the cumulative scores for the 3 themes and for the total PHEEM score for each of the 3 cohorts assessed in this study.
Scores for the 3 PHEEM themes and total score.
The authors of PHEEM recommend that any item with a mean of 2 or less should be examined in detail as it may indicate a problem area. Table 4 provides a summary of the items with a mean total score below 2 for the 3 cohorts in the study. The items have been arranged in ascending order.
Summary of the items with a mean total score below 2 for the 3 cohorts in the study. The items have been arranged in ascending order.
There was only 1 question that was rated >3 by the 3 cohorts: “I have good collaboration with other doctors in my grade.” Another question, “My clinical teachers promote an atmosphere of mutual respect,” was rated 3.00 by Foundation Year 1 respondents. The original authors of PHEEM have suggested that items with a mean score of 3.5 or over are real positive points. None were identified in this study.
Statistically significant differences between genders were identified on 7 items, as listed in Table 5. In general, females seemed to give higher rankings in almost all 7 items. The only exception is the question regarding sex discrimination in the workplace.
Items with statistical significance between genders.
Table 6 reports on the mean total PHEEM and perceptions of autonomy, teaching, and social support by gender.
Mean total PHEEM and perceptions of autonomy, teaching, and social support by gender.
Statistically significant differences between foundation-year cohorts were identified in 5 items using the Kruskal-Wallis H test. These results are illustrated in Table 7.
Items with statistical significance between years of training.
Generally speaking, mean scores seem to dip midway through the 2-year training period, only to pick up again at the end of training, but never reaching the same levels as at the early stages of training.
Discussion
The current study evaluates foundational doctors’ perceptions of the educational environment at the Malta Foundation Programme. These doctors perceived their educational environment as more positive than negative. The educational environment scores worst among trainees at the end of their first year of training. Those who had just finished their training have reported better perceptions but these do not reach the levels of those at the start of training. This dip could signify that the perception of the educational environment hits a low midway through the 2-year period, but as their career progresses, foundation doctors are quick to adapt. As a result, their perception of the educational environment improves toward the end of the training program. One needs to keep in mind that these results are the reflection of the perception of different cohorts. A longitudinal observational study can provide more concrete results to explain this dip.
Perceptions of teaching and social support ranked higher than perceptions of role autonomy. Trainees have demonstrated particularly low rankings in perceptions of role autonomy midway and at the end of training.
Interestingly, out of the 11 items with a mean score less than 2, 6 form part of the autonomy subscale, 5 form part of the social support subscale, and only 2 were part of the teaching subscale.
The 3 worst scoring items seem to be related to when the trainees are on call. Malta persists with a system of on call duties that has been scrapped in many countries. Under the current conditions, foundation doctors work for some 28 hours at a stretch. While adoption of the European Working Time Directive is part of Maltese law, foundation doctors may feel that if they choose not to exceed 48 hours of work in a week, they may find themselves at a disadvantage when applying for training posts at a later stage in their career. On call duties are made worse by the lack of proper catering facilities on site and by an inordinate number of inappropriate pages.
Statistically significant differences between genders were identified on 7 items, as illustrated in Table 4. Five of these items form part of the perceptions of role autonomy subscale, and 2 form part of the perceptions of social support subscale. Female trainees gave higher mean rankings for all 7 items except for “There is sex discrimination in this post.” It is encouraging to note that female trainees do not perceive the educational environment at the Malta Foundation Programme to be sexually discriminatory.
Foundation Year 1 trainees scored significantly higher on 5 items, as shown in Table 5 (3 items on the autonomy subscale and 2 on the social support subscale). There seems to be a trend of deterioration in perceptions across all items and subscales as training reaches its midpoint. Perceptions then improve by the end of training but never reach the same levels of ranking achieved at the start of training. Whether this could be related to the development of burnout in trainees will be analyzed in a further study. As this is an observational, not longitudinal, study conclusions are guarded in this respect.
In a similar local study, Farrugia Jones and Cacciotolo 31 assessed the postgraduate educational environment in the Department of Medicine at Mater Dei Hospital, Malta. The response rate for house officers in this study was 10%. Problem areas identified by trainees in this study were the absence of protected time, no access to an individual educational program and working hours that do not conform with the European Working Time Directive. The same problems have, unfortunately, again been highlighted by this study. Poor catering facilities, poor access to career advice and counseling opportunities, a strong blame culture, and an inappropriate workload were also replicated in the current study.
There are multiple benefits of a healthy educational environment. Studies have shown that the learning environment is associated with the quality of care provided. 32 The educational environment also influences the prescribing habits of trainees 33 and the management and use of health care services and resources. 34 In surgical training, higher rated surgical training programs were associated with lower complication rates. 35 A healthy learning environment has been associated with better residents’ outcomes including satisfaction with training, the use of knowledge, and a professional identity development.36,37
The importance of having a high-quality clinical learning environment stems from the knowledge that a high-quality learning environment will have a direct impact on workplace learning and, on the quality, and safety of the clinical care received by patients. Trainees exposed to a high-quality clinical learning environment will have been exposed to participation in the clinical care of patients in a supervised manner. They will also be exposed to coaching, assessment and feedback, deliberate practice, and peer collaboration. 3
Many trainees are meant to learn and develop their clinical skills in understaffed, underfunded, uncontrolled, and overcrowded clinical educational environments.38,39 Exceeding a certain critical level of workload will result in a decline in trainee learning. Heavy workloads were found to be linked to a number of undesirable outcomes in trainees. These outcomes include an increased likelihood of burnout and lower engagement, health, and well-being.40,41 Lower levels of patient satisfaction, poor standards of care, and higher mortality were also associated with self-reported heavy workloads.42,43
In view of the above benefits of a healthy educational environment on quality of care, learning, trainees’ mental health and safety of care, efforts at improving the role autonomy, and social support aspects of the educational environment need to be prioritized.
Limitations
This study has limitations. The response rate was relatively low despite a reminder being sent. The quantitative nature of the study precludes an in-depth exploration of the reasons why trainees have expressed perceptions. A qualitative additional study is warranted. The study was also carried out in a single center, albeit this is the only 1 on the island. The study did not take into account the effect of a pandemic on the educational environment.
Conclusion
This study aimed to evaluate the educational environment of the Malta Foundation Programme. The first 2 years of postgraduate medical education signify a transition that can be rough and demanding, as trainees are suddenly burdened with professional responsibilities. It is a period where learning has not ended with the undergraduate years but is only beginning in the postgraduate years.
The current study has identified areas in the educational environment where improvement is needed. Perceptions of role autonomy and social support are areas where most work is needed. Teaching seems to be moving in the right direction, but there is always room for improvement. A comparison with another local study identified similar problem areas.
The study was conducted at a time where the COVID-19 pandemic was having its toll on healthcare systems across the world. It may also have had an impact on the results of this study.
Footnotes
Acknowledgements
Dr. Marco Grech is a Senior General Practitioner. He is currently Assessment Lead at the Malta College of Family Doctors and has served as AKT Lead and Education Secretary for the same College in the past. He is a visiting senior lecturer within the Department of Family Medicine, Faculty of Medicine and Surgery, University of Malta. He is also a tutor in Medical Education at the University of South Wales (UK). Stefania Grech is a medical student with special interest in medical education.
Funding:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors have self-funded this study.
Declaration of conflicting interests:
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Author Contributions
Marco Grech was the main contributor to this manuscript having written and reviewed all the manuscript. Stefania Grech contributed with data collection and analysis and reviewing of the manuscript.
Ethics Approval and Consent to Participate
Ethics approval was obtained from the Faculty Research Ethics Committee at the University of Malta. Permission to conduct the study was also obtained from the Foundation Programme, Malta. Informed consent was obtained from all participants. The participants also consented to the publication of results. All methods were carried out in accordance with relevant guidelines and regulations.