Feedback Framework in Psychiatry Residency Training: A Cross-sectional Survey Amongst Psychiatry Post-graduate Residents of India

Abstract

Background:

Feedback has proven crucial in various training systems, especially medical training. However, data regarding the effectiveness of a feedback framework in psychiatry post-graduate (PG) training is limited. Hence, we undertook this survey to assess the essential components of a feedback mechanism in psychiatry PG training in India.

Methods:

This is a cross-sectional survey in which a questionnaire was circulated online to residents pursuing psychiatry training across various medical colleges, institutions, and hospitals in India. The questionnaire aimed to cover various aspects of a feedback program for psychiatry PG training. The target audience was kept as residents since they are the main benefactors of an effective feedback system.

Results:

We received responses from 262 residents from 84 departments nationwide. The majority (n = 248, 94.7%) believed that a feedback system is necessary in psychiatry residency. Most believe feedback about outpatient and inpatient clinical performance and patient management skills is highly valuable. Real-time feedback was reported to be helpful, and faculty, followed by senior residents, became the most desired sources of feedback.

Conclusions:

The survey highlights that residents strongly need a feedback mechanism during psychiatry residency. Implementing a feedback program and constantly evaluating its effectiveness could go a long way in improving psychiatry training outcomes in India.

Key Messages:

• Explored perspectives on a feedback mechanism for psychiatry PG residents

• Feedback should be regular, real-time, and include most aspects of clinical work

• Results inform possible elements of a feedback framework for psychiatry PG residents

Feedback is information given by someone (such as a teacher, coworker, or relative) about an individual’s performance and/or knowledge. ¹ Feedback plays a crucial role in medical education by facilitating learning and assuring compliance with standards. ² Regrettably, the task of clinical education that involves this component is challenging, leading clinical instructors to frequently evade this aspect of their duties. The body of research on feedback in medical education is extensive and diverse. Several models guide effective feedback, such as the feedback sandwich, Pendleton’s rules, and the ALOBA (agenda-led outcome-based analysis) principles.^3–7 However, as per a systematic review by Weallens et al., the available evidence for the effectiveness of feedback provision is limited, as only 10% of feedback models have been empirically researched, and most studies rank low on the hierarchy of evidence-based medicine. ⁸ Feedback in medical post-graduate (PG) training encompasses several elements, including supervisor evaluations, peer assessments, input from support personnel, inter-professional feedback, and patient feedback. A review by Baines et al. indicates that patient feedback can lead to measurable changes in behavior, and its efficacy is affected by several key factors: specificity, perceived credibility, alignment with physician self-perceptions, facilitation, and reflection. ⁹ Nurudeen et al. conducted 360-degree evaluations of 385 surgeons across eight academically affiliated hospitals. Participants responded that, in most cases, feedback was accurate and led to beneficial changes in practice and behavior. ¹⁰

Likewise, research across several medical and surgical specialties indicates that feedback, in its multiple forms, is beneficial during PG training.

Rationale for the Study

While extensive evidence exists for the usefulness of feedback for medical students,^11,12 pieces of literature for PG students are comparatively limited, especially regarding psychiatry PG training and the framework and effectiveness of feedback in this context, both locally and globally. Hence, this study was undertaken to fill the gaps in existing literature around a feedback framework in psychiatry PG training in India. To date, and the best of our review, no published study or survey from India has assessed the essential components of a feedback mechanism for psychiatry PG training. Further, the unique point of the current study is that the participants are psychiatry PG residents who are the most crucial stakeholders of the residency training program.

Aim

The main aim of this study was to describe the pattern of feedback received by the psychiatry residents and the perception of psychiatry residents about the feedback during the training.

Objectives

• Assessing current feedback practices in psychiatry PG training

• Eliciting users’ perspectives on essential components of a feedback framework in psychiatry PG training

Methods

After obtaining approval from our Institutional Ethics Committee, this cross-sectional survey questionnaire was sent to the following participants: any resident pursuing an MD or DNB in psychiatry at a medical college/institute and hospital in India with fluency in English and willingness to participate. We utilized the CHERRIES (Checklist for Reporting Results of Internet E-Surveys) to check for the completeness of reporting in our survey (Supplementary File 1).

As per Raju, there are around 250 psychiatry training departments in India across its states and union territories, with around 1000 candidates enrolling for PG training each year. ¹³ There are around 3000 psychiatry residents in India (pursuing MD or DNB psychiatry), from which we had access to valid contacts of around 1200 residents. With a 95% confidence level, a 5% margin of error, and an expected sample of 30% to respond to the survey, we had a sample size target of 255 responses.

The survey questionnaire was prepared through a review of the literature on existing surveys among medical professionals and a discussion between the authors. For instance, Sugumar et al. studied the development of a structured feedback schedule for the medical PG curriculum. ¹⁴ The assessment part of the questionnaire had some elements adopted for our study. Natesan et al. developed an evidence-based guide for best practices on feedback in medical education. ¹⁵ These guidelines suggest that feedback mechanisms be structured and standardized, encourage learners to take an active role in the feedback process, the feedback be given on observed behaviors, and should be clear, specific, timely, and actionable. These components were also kept in mind while designing our questionnaire. For content validity, the questionnaire was designed through the Delphi method and focused on group discussions between two undergraduate students, two psychiatric consultants, and a clinical psychologist, along with a thorough review of global literature. Consensus on item selection and content validity was reached through progressive modifications based on expert ratings and recommendations. This was followed by a pilot survey of 10 PG psychiatry residents to test the survey’s feasibility and gain feedback from the pilot survey respondents. After this, a final version was prepared and circulated (supplementary file 2). Pilot testing was done in May 2024, and data was collected from June 2024 to August 2024.

To assess the construct validity, the scale was distributed to four experts in psychiatry who were not involved in the study. They were asked to give responses based on the relevance of the questions to assess the scale’s objective. The Cronbach alpha was then calculated using the co-variance, item, and inter-item variance and a total number of items, which came out to be 0.89.

We sent the survey questionnaire via Google Forms via email and WhatsApp messaging software. It was not published on any website. The survey was open and voluntary. A statement of consent from the participant was included as part of the survey questionnaire. Personal information such as name was optional. Other personal information, such as age, residence and religion, was not collected as it was not essential to our survey aims and objectives. We mentioned question items with mandatory responses. Hence, we did not expect any attrition. Adaptive questioning was included, where specific questions were only to be answered based on responses to some initial questions. There was a total of 22 questions across a single window page. Some questions were mandatory and were marked accordingly, and where applicable, the option to enter an open response, that is, a response not already provided in the options, was provided. Intake was started on June 13, 2024, and closed on August 31, 2024, along with sending three reminders at around three weeks. At the end of our survey and after removing incomplete responses, we received 262 complete responses (Figure 1). We did not have data on participation, recruitment, or completion rate. No randomization was done. No cookies, Internet Protocol (IP) address checks, or log file analysis were used, and no incentives were offered to the survey respondents. Participants were able to review their answers. Only complete questionnaires were analyzed, and the time taken to complete responses was not considered.

Figure 1.

Flowchart Showing Stepwise Inclusion of Survey Respondents.

Statistical Analysis

Descriptive analysis via SPSS version 23 was employed to analyze the data collected from the cross-sectional survey with 262 responses. ¹⁶ No statistical correction was applied as the analysis was mainly descriptive. This analysis aimed to summarize and present the key characteristics of the dataset, providing insights into the variability of the survey responses. Key descriptive statistics, including frequencies and percentages, were calculated to identify patterns and trends within the data.

Results

A total of 262 residents from psychiatry departments across 84 medical institutions of the country gave complete responses to the survey questionnaire. Out of them, 21.8% (n = 57) were in the first year of residency, 37.4% (n = 98) and 40.8% (n = 107) were in the second and third years, respectively. There were 167 male (63.7%) and 95 female respondents (36.3%). About three-fourths of residents (72%) reported that their department had offered guidance on how they wish to assess the resident’s performance through the tenure of three years. Among the residents who received feedback from their department (n = 211), 44.5% reported receiving both formative and summative assessments, 45% reported receiving formative assessment only, and the remaining received summative assessment only.

Among those receiving feedback, around 84% of residents received direct feedback. Around 40% of the time, residents said they received the feedback quarterly, while another 31% said they received it half-yearly. The rest either received it annually or once in three years. Additionally, two-thirds of the time (66%), the feedback was backed by an objective score. Three-fourths of the residents (75.2%) reported receiving in-person feedback, while the rest were given a handover of their score.

Most (n = 248, 94.7%) believed a feedback system is necessary for a psychiatry residency. Most (87%) believed that feedback about inpatient and outpatient performance coupled with patient management skills (86.6%) is highly valuable (Figure 2). Three-fourths of the respondents reported feedback on presentation skills could be valuable (75%), while two-thirds had the same say on reading and learning material (68.3%). Around half the residents in the survey agreed that feedback on administrative and liaison skills would be valuable.

Figure 2.

Aspects of Residency Where Respondents Feel Feedback Would Be Most Valuable.

Reading and learning material; ward and OPD performance (history taking, formulation and management plans); administrative skills; liaising skills with other departments; presentation skills (case and academic activities); patient management skills (includes delivering psychotherapy and routine follow-ups); inter-personal interaction with colleagues, seniors and other staff; time management; research work; documentation.

Regarding the mode of feedback, the majority (89.7%) believed in-person feedback was the most desirable way to deliver feedback. Regarding the frequency, around half recommended quarterly feedback (45%), while one-fourth (25%) vouched for monthly or half-yearly timelines. Around half of the residents (47.7%) believed feedback from allied departments could be beneficial. Faculty (88.2%) and senior residents (82.1%) were considered the most important sources of feedback (Figure 3). Around half of the respondents believed feedback from co-residents (61.8%) and patients (50%) could be helpful. Around one-third of the respondents believed feedback from nursing staff (39.5) and hospital (ward/OPD) attendants (36.5%) could be helpful. Most respondents (86.6%) agreed that real-time feedback will be valuable, mainly in case and academic presentations. Two-thirds of the residents (63%) preferred to have their MD thesis work included in their feedback. Most of the residents (92.7%) believed there should be a system in place for them to provide feedback to the department about their residency program.

Figure 3.

Recommended Sources of Feedback As Per Respondents.

Comparing subgroups, most results retained consistency across the year the participants pursued their residency. Compared to first-year residents, more second and third-year residents believed that feedback regarding administrative and liaisoning skills was essential. While a quarterly frequency remained the most popular choice for feedback, a few first and third-year residents were also recommended annual feedback, for which there were no takers among second-year residents. Regarding the choice of including MD thesis work in the feedback mechanism, around 72% of first-year residents agreed to include it, while around 60% of second and third year opted for it.

The last question in the survey was kept open-ended to allow the participants to openly express their views regarding a feedback mechanism in psychiatry training for PGs. We received 88 responses to this question. The most common theme from the responses was that there should be a component of the feedback where the resident can provide their views about the department faculty. Some responses voiced that the feedback should be kept critical and impartial, not further creating hostility. Another common suggestion was that the feedback be anonymous. Some responses also suggested that feedback can be categorized into essential, desirable, and optional.

Discussion

The survey highlights that a feedback system is necessary for PG psychiatry training, as expressed as a need felt by the respondents. These findings are in synchrony with those of Baines et al. and Nurudeen et al., who concluded that a feedback system for medical PG training is beneficial.^9,10 Further, our findings, such as the feedback being real-time, coming from multiple sources, and being consistent, reflect the measures suggested in the guide for effective feedback by Natesan et al. ¹⁵ While the Pendelton model of providing effective feedback served as a starting point in general, feedback systems in psychiatry PG training need to be tailored and regularly evaluated. ¹⁷ Our results provide a step in that direction.

Some essential components that need to be incorporated into the feedback system are performance in clinical work in inpatient and outpatient settings and presentation skills (case and academic). Clinical work performance supplemented by sound academics remains one of the core elements of residency training for any clinical branch; hence, feedback on the same will be crucial in helping the resident gain much-needed direction at regular points during his training period. Other components that can be strongly considered are administrative and liaison skills and feedback from allied departments.

Further, the availability of a real-time feedback mechanism was highly encouraged by the respondents. This, again, is in synchrony with existing literature on feedback in medical training. ¹⁸ Feedback given in real-time, that is, immediately after a case presentation or an academic presentation, provides valuable insights into clinical performance, enabling identifying improvement areas and refining skills more efficiently. This prompt feedback fosters ongoing learning, motivating residents to modify their methods of patient care, diagnostic procedures, and communication strategies in real time. Moreover, real-time feedback cultivates a collaborative learning atmosphere, enabling residents to participate in reflective practice, which enhances their comprehension of psychiatric principles and refines clinical decision-making. It also bolsters residents’ confidence, enabling them to rectify mistakes or uncertainties with assistance promptly and diminishing the likelihood of recurring errors.

Whether to adopt an exclusive formative or summative assessment is open to discussion. A blended approach could be most beneficial, as Svensäter and Rohlin also reported. ¹⁹ Regarding feedback frequency, most respondents advocated for a quarterly assessment, while monthly remained the second most preferred option. However, quarterly or half-yearly appears more convenient to incorporate. This will ensure no delay between successive feedback and give enough time for the resident to work on the feedback given and for the department to monitor the progress after the previous feedback.

Regarding the source of the feedback, as will be suitable and reflected in our survey, faculty, and senior residents prove to be the foremost resource persons. Further, it is necessary to include other stakeholders such as co-residents, nursing staff, and patients. Wherever available, feedback from clinical psychology and psychiatric social work professionals should also be considered. This will ensure comprehensive feedback and eliminate biases, if any.

Psychiatry PG training in India is provided in exclusive Mental Institute and General Hospital Psychiatry Units. Further, various administrative systems and clinical rotations are within each setup, such as unit versus non-unit systems. Hence, feedback mechanisms will vary according to the training setup. ²⁰

Lastly, the National Medical Commission, India, provides guidelines on MD. Psychiatry training elaborates on the essential competencies required. It also provides a brief assessment framework with students’ performance rated as less than satisfactory, satisfactory, or more than satisfactory. These can also be included while designing a feedback mechanism.

The survey has strengths in that the direct benefactors of the research agenda were the target respondents, psychiatry PG residents. Residents from across the country participated, ensuring coverage from different types of training setups. There was scope for open-ended answers in the survey questionnaire, which helped in better understanding the results.

Limitations

The survey captures feedback at a point that may not accurately reflect the residents’ experiences or opinions over their entire training period. While the survey can gather quantitative data, it may not allow in-depth exploration of complex training curricula and environmental issues. Different departments across the country vary in aspects such as the number of working faculty and residents, years of establishment and liaison with other departments. This heterogeneity could not be wholly accounted for in the survey. The survey had a relatively low response rate of 22%. Lastly, access to valid contacts of a good number of residents was not available, which further limited the outreach of the survey.

Conclusion

The index survey highlights that a feedback mechanism during psychiatry residency is a need that residents strongly feel. While it is heartening to see many departments have a feedback mechanism, this may not be a true story for all departments as various departments were not represented in this survey. Inpatient and outpatient clinical performance coupled with patient management skills is the area where the respondents felt the highest area of need, followed by feedback on reading and learning material. Further steps in this direction will include various feedback frameworks being implemented by departments nationwide and their effectiveness being evaluated. National bodies such as the Indian Psychiatry Society (IPS) and Indian Teachers of Psychiatry (IToP) can consider framing a consensus framework or guidelines. Further, studies evaluating the outcomes of such systems will help arrive at a certain consensus on frameworks to provide feedback.

Supplemental Material

Supplemental material for this article available online.

Supplemental Material

Supplemental material for this article available online.