Summative Assessments in Psychiatry Postgraduation in India: A Teacher’s Perspective on Current Situation and the Way Forward

Summative assessments measure learning outcomes at the end of an instructional unit or at a specific time, comparing student knowledge or skills to established standards or benchmarks. Rather than focusing on an individual, it gauges how a group responds to an intervention. This way, it helps teachers, administrators, and parents assess the effectiveness of curriculum, materials, and instructions at various levels. ¹

Postgraduate (PG) training in India has stayed almost the same and has been difficult to modify as an exoskeleton. The training program consists of three years of rigorous clinical duties, finishing the mandatory thesis, and getting through the final theory and practical exam.

Recently, there has been an effort to switch from standard, repetitive methods to a competency-based curriculum (CBME) for PG education. ² The proposed changes aim to transform the training and evaluation of PGs. As the National Medical Commission (NMC) has already begun implementing them, we intend to draw attention to two core aspects of summative assessments in PG training: the “mandatory PG thesis” and the “Final exams.”

Given the paucity of literature on these crucial aspects of PG education in India, our perspective sheds light on the difficulties encountered in completing a thesis study and in conducting and attempting final PG exams. In addition, we propose specific modifications that could potentially tackle these challenges, as seen from the viewpoints of both faculty members and PG students. We divide the review into two parts. Each part outlines the various challenges encountered in both aspects and offers recommendations to enhance the summative assessment process.

The Mandatory Thesis

Meena Anand et al outline the objectives of thesis writing as meeting NMC requirements, enhancing research expertise and scientific curiosity, contributing to the branch’s literature, identifying and reviewing gaps in existing research, and understanding clinical relevance to translate findings into practice for improved patient care. ³ At the end of three years, a PG trainee is expected to submit a completed study to be eligible for their final exams. It begins with having to submit a synopsis within the first six months of joining the course. This is to be reviewed by the members of the institutional ethics committee and the university. The guide and the student must collaborate to plan the methodology, data collection, data management, and assessment tools, as well as how to integrate them all.

At the end of data collection, the major task at hand is to make sense of the data, analyze it using various statistical techniques, and complete writing it up. To conclude this, one needs the signature of their guide, suggesting that the PG has completed the thesis and is eligible for the exams. Two to three independent reviewers review the degree at the end of the course to ensure its validity. Both the PG and guide encounter various obstacles, such as a need for more clarity about the allocation of guides, the method of determining a thesis topic, and the absence of institutional policies. Addressing these challenges can make the thesis process less burdensome and more enjoyable.

The process begins with having to choose a topic. As the students are new to the subject, they are often left at the mercy of their guides when choosing a topic. Quite often, the interests of the PG and the guide are different. For most, research is a new experience, and it requires mentoring. Due to clinical, administrative, and personal commitments, the guides are inaccessible and, when available, spend less time than is necessary. For various logistics reasons, some guides are known to have met the students only twice or three times before submission. ⁴

When multiple guides are involved (e.g., co-guide, statistician), they may make contradictory suggestions, leaving the resident needing clarification. In certain cases, disagreements between them can result in the residents having to figure things out for themselves. To avoid being in this situation, many PGs resort to hiring others to compute data and put together the thesis for varied fees. Since publications are essential for advancing in one’s career, they sometimes take precedence over teaching a resident how to write a thesis correctly. This illustrates a different perspective, where completing the work takes precedence over the quality of the research work.

Among the several issues faced by the faculty, the need for adequate training in research methodology is a significant one. Quite often, the PG is left to figure out certain aspects by themselves or get the help of their fellow PGs or senior colleagues. This compromises the quality and results in inadequate learning, which is later passed on to their juniors. Several institutes and departments need more rigor for clinical research, and this reflects the poor scientific climate within them. The insufficient number or quality of journal clubs, research methodology workshops, seminars, and programs fail to emphasize the importance and need for research in clinical practice. Long working hours, especially in busier institutes, have an impact on the time spent doing the thesis. This paves the way for the fabrication and falsification of data. The resident’s lack of interest and punctuality can also present a challenging task for the guides. ⁵

Research requires skills in approaching patients, obtaining their consent, gathering data, and tabulating it. Only through observation, guidance, and conducting a pilot study can one acquire these skills. A result-oriented process impairs skill building. When PGs attempt to make sense of their data, they are still determining what statistics to employ, how to protect them, or what it means for their study. This results in poor representation, delays in submission, and unnecessary fines.

Many institutes need more senior faculty, specifically professors and associate professors, in their departments, leading to an uneven guide-to-resident ratio. Frequently, a guide must oversee the completion of more than one PG thesis. This increases the workload for the guide, ultimately influencing the quality of the product. In the last couple of years, newer medical institutes have opened, increasing the number of PG seats. Despite their exorbitant tuition fees, some of these medical institutes need more research infrastructure, such as a dedicated research department or the availability of databases, research assistants, and statisticians. On some occasions, when the resident has not been able to meet the deadlines, the guides could resort to taking a hands-on approach to complete the thesis. This is because sometimes faculty members fear potential repercussions that could harm both the institutions and their own reputations. Similarly, the institution requires the guide or evaluator to accept the thesis in order to maintain high pass rates. We have also heard of instances where the guide has completed the PG thesis, a mandatory requirement in some universities before they can become a PG examiner.

Recommendations^6,7

1. To improve the quality of the thesis, it is necessary to establish a committee that can facilitate the following steps:

2. The PG could discuss and choose the thesis topics based on their knowledge and interests. Once they confirm the thesis topic, they can select a co-guide based on their area of expertise.

3. After assigning the PG guide, the guide and the student must meet regularly and record this in a logbook. Discussions should be free from bureaucratic and communication barriers between the guide, co-guide, student, and reviewers.

4. Prior to submission, the synopsis must be presented in the department with the guide as the chairperson and an external chair who is an expert in the field of research.

5. The regular academic program must make thesis update presentations mandatory at regular intervals.

6. Brief workshops or courses should be conducted for PGs in practical aspects of the thesis, such as data entry, using SPSS, approaching a statistician, and writing a thesis.

7. The guide, co-guide, or IEC members must regularly check a random sample of data for any evidence of data fabrication or manipulation.

8. At the end of the thesis submission, feedback about the guide and the PGs’ experience of the thesis process must be recorded. Based on the feedback, the HOD can review the guides.

Each institute and university must have a thesis repository to avoid reduplication of the study.

Collecting data from other centers must be permissible to improve the quality of the research and provide flexibility in finishing the thesis within time constraints.

More community-based research must be encouraged targeting certain vulnerable or Indigenous populations, such as detention centers, prisons, rescue homes, and orphanages.

All PGs must be encouraged to publish their work. They can be tagged along with a PG alumnus to help in the process. At the same time, if part of the research was conducted ethically, publication of it must be strictly prohibited.

Authorship credentials must be discussed in advance, and the first authorization must be given to the PG.

Encourage medical PGs to present a portion of their thesis work at conferences, allowing them to receive feedback and suggestions from external experts and peers. This helps to consolidate and build upon the guidance already received intramurally from guides and co-guides.

The institutes can organize a faculty development program, with a primary focus on research methodology and statistics, to assist guides and co-guides in their work. In addition, attention should be paid to soft skills, mentoring, effective communication, etc.

The quality of the thesis and its timely completion could be rewarded. The thesis could receive 10% of the practical score as its reward. The departments can award the “best thesis” for each batch.

The thesis can be part of a collaborative project or submitted for awards or grants, hence giving it the attention it deserves.

Establishing clear guidelines and ethical standards is crucial to preventing coercive practices by thesis guides during thesis publication. Often, they face requests to publish in unsuitable journals to align with their promotion schedules, claim first authorship, or compel PGs to pay Article Processing Charges (APC).

The Examination Process

In most institutes affiliated with a central university, a PG student’s assessment occurs at the end of their final year. The assessment consists of four theory papers, followed by practical exams. Paper I covers basic sciences related to psychiatry. Paper II covers clinical psychiatry. Paper III covers psychiatry specialties, and Paper IV covers neurology and recent advances. The practical exam takes place on a single day, and it consists of two long cases and two short cases, followed by viva voce, which are conducted across four stations (differs across institutes and universities). The theory and practical marks are totaled, and a percentage score is given at the end. ⁵

Based on anecdotal evidence, sometimes the candidates are aware of the cases kept for the exam prematurely. This biases their approach to the case. Depending on the number of candidates appearing for the practical exam, there can be differences in the examination between the first and last candidate (time spent, number of questions asked, etc.).

Many institutes lack separate geriatric, child, and adolescent units, so there is less focus on these cases on exam day, limiting the range of evaluation. Occasionally, candidates may be required to take their exams at a different center. Its unfamiliarity is unsettling and interferes with their performance.

Suppose there are a greater number of candidates per day, which is usually the case nowadays, resulting in time constraints. In that case, examiners invariably must cut short the assessment process during the examination. This may be for adequate observation of the student’s skills or elicitation of psychopathology. The assessment of a neurology candidate prioritizes the localization of the lesion over cognitive assessment or higher mental functions, as this is more clinically relevant for future practice.

The evaluation of students is inadequate in areas such as their ability to use appropriate interviewing techniques, their sensitivity when interviewing patients and caregivers, and their use of common psychotherapeutic skills and skills related to the psychoeducation of patients and family members. Similarly, the knowledge and skills of various psychosocial interventions go unexamined. A PG could be the most hardworking and diligent, but they may need more skillsets suited for this format of exams, giving a false impression of their capabilities. A candidate’s evaluation objectives need to be clarified, and a lack of consistency among the examiners can interfere with their preparation and performance.

Many PGs who possess valuable real-world skills need to be noticed in this format of examination. Evaluating a candidate’s three years of hard work in a single day can be unfair. Given the potential impact of this day on a candidate’s future, they often experience overwhelming anxiety, which can hinder their performance. During regular clinical practice, the candidates are often told to take their time in assessing a patient and making a diagnosis. However, the provision of 45 minutes to do so and write up the case puts a tremendous amount of pressure on the candidate. In conclusion, a single day judges three years of training and practice based on the candidate’s ability to retain information, manage their nerves, and their luck.

Recommendations ⁸

1. The entire assessment process can be divided into three parts rather than a single examination at the end of three years. As directed by the NMC recently, the PGs must have a theory and practical examination every six months. This could be designed to account for periodic summative assessments alongside formative assessments.

2. Given that the majority of reputable national and international tests design MCQs instead of essay questions, MCQs may comprise a portion of the paper. The topics can be divided as per how the four theory papers are divided at the end of every year.

3. Practical examinations can be redesigned into OSCE (objective structured clinical examination) formats. At the end of each year, the PG student is expected to have certain skills (clinical, communication, and soft skills) and must be examined for them through the above-mentioned format.

4. At the end of each year, the PG students must receive feedback on their performance in the theory and OSCE. Those who have failed to attain a passing grade should be given a re-attempt at the end of two months.

5. Having a child or adolescent patient as an exam case must be a rule rather than an exception. The case needs to be a formal child psychiatry case rather than a developmentally impaired child. A formal evaluation of competency is required for interviewing children and parents during examinations, rather than simply asking a few questions about developmental anomalies to cover child psychiatry topics. This is good for geriatric patients as well as all cases.

6. OSPE (objective structured practical examination) can be adopted for drugs, electroencephalogram (EEGs), magnetic resonance imagings (MRIs), etc. The knowledge of these can be examined according to the year of PG and what the student is expected to know in that particular year.

7. PG students must also receive grades based on how they use patient scales like the Hamilton Depression Rating scale (HAM-D), Mini Mental State Examination (MMSE), and Patient Health Questionnaire-9 (PHQ-9).

8. At the end of each year, scores must be allocated for completing portfolios such as psychotherapy skills and peripheral posting marks. Weight must be given to their performance on regular departmental academics recorded in their logbooks. Students conducting awareness programs as part of their District Residency Programme (DRP) can earn marks.

9. Prospective examiners should undergo training and accreditation before they can conduct exams. The day before the exam, all examiners should receive a briefing on conducting the practical examination. The number of candidates examined in a single day should not exceed four. Transparent mechanisms must be in place to address any grievances by candidates regarding the assessment of their exam performance.

10. Uniform guidelines between universities and institutes for providing leave to PGs prior to the exams are necessary.

Conclusion

We explored the complexities and difficulties encountered by PG students and faculty members, and there is much room for improvement to ensure a more favorable and productive learning environment.

Establishing institutional frameworks that provide standardized direction and supervision is critical to addressing the thesis’s issues. Establishing regular meetings between guides and students, conducting periodic presentations to monitor progress, and implementing ethical checks to prevent data falsification are also important to improving the quality of research output. Furthermore, PGs may develop a culture of research excellence through programs like creating thesis archives, promoting teamwork, and offering incentives for publishing and conference presentations.

In a similar vein, having a comprehensive, effective, and authentic summative assessment is essential to enhance the quality of psychiatry residency. Several changes to the examination procedure are essential to providing an impartial and thorough assessment of the PG’s capabilities. A single test might cause unnecessary strain. Instead, a more segmented assessment strategy with frequent feedback and chances for improvement can help. Accepting contemporary assessment formats, such as OSCEs and OSPEs, and providing examiners with standardized training can improve the validity and reliability of assessments. We should prioritize comprehensive evaluation criteria that go beyond simple information retention, like clinical understanding and communication skills, to more accurately represent PG’ holistic competencies.

An important addition to psychiatry education in India has been the CBME structured framework for undergraduate education. This has helped bridge the gap in psychiatry education between colleges with good resources and colleges with minimal resources. Similarly, uniformity in PG CBME is still in its early stages, with gaps that create discrepancies in psychiatry PG education across the country. In the end, these proposed changes could develop a new generation of medical professionals with strong research abilities and clinical knowledge, in addition to streamlining the PG training program. We can create conditions for a more effective and fulfilling PG medical education system in India by tackling systemic issues and putting up productive reforms.

This review is limited in capturing the opinions of a few psychiatry teachers working in a private medical college affiliated to a state university (including the authors of this paper). Additionally, surveying PGs and psychiatry teachers of various institutes at various levels and creating a forum could provide more comprehensive insights and new suggestions.