Abstract
Formative assessment (FA) monitors and encourages student learning, helping students identify their academic strengths and weaknesses. The FA is often described as low-stakes and designed to inform students and teachers, a change from the more traditional summative assessment, a high-stakes evaluation process designed to measure students against a benchmark. Integrating FA is necessary in higher education for effective student learning, specifically related to the conceptual framework of self-regulated learning. 1
Contemporary evidence-based medical educational practices and the broader adaptation of competency-based medical education (CBME) across countries have revolutionized traditional evaluation processes, especially FA strategies for postgraduate medical education. In CBME, measuring and documenting competencies using high-quality assessment is as essential as summative and FA. 2 In India, the role of FA was further emphasized after the National Medical Commission (NMC) implemented the CBME-based curriculum in 2019. In the last two decades, FA has become a prominent theme in postgraduate medical education to facilitate and enhance learning throughout the training period.1,3 Nevertheless, a paradigm shift has been witnessed; the role of purely summative assessments is waning while FAs are gaining ground in postgraduate training.4,5
In a clinical setting, trainees need to be observed and assessed in real situations so that necessary corrections can be provided. A variety of FA tools have been developed, including direct observation, for example, mini clinical evaluation exercise (mini-CEX), Direct Observation of Procedural Skills (DOPS), objective structured assessment of technical skills (OSATS), and simulations.5–7
However, using FA methods in their typical forms and fairness in a clinical setting is not a context-independent practice. It is affected by a myriad of factors, such as collegial relationships, engagements of stakeholders, lack of implementation due to faculty scarcity, time constraints, attitudinal inertia, and poor acceptability.5,8–11
The current discussion pertains to formative strategies adopted in General Hospital Psychiatry Units (GHPUs) under the Institution of National Importance (INIs), which has a semester system. As a semester system, the entries of postgraduates and final summative degree examinations occur every six months; the curriculum was divided into six semesters and rotated every three years, in contrast to the annual system where postgraduates and degree examinations are held annually in medical training centers under NMC.
We share our experience of an FA strategy adopted in GHPUs in INIs, with innovation in strategies of work-based assessment (WBA), 360° assessment, making an electronic record, and its further utilization (Figure 1). The 360° evaluations are recorded to project the longitudinal progression of the postgraduate resident. The graphical projection of individual trainees is used as an objective substrate for constructive feedback for the postgraduate. The components of the formative evaluations used are as follows.
A Logbook
All newly joined postgraduates are provided a “logbook & working manual” at the beginning of their residency after orientation and an ice-breaking session with other residents and faculties. The logbook provides a road map for the whole training program, compatible with semester-wise curriculum and competency levels. The residents enter their academic activities, hours of psychotherapy, achievements, and details of internal and external clinical rotations. The Postgraduates need to get it signed by the respective supervisor of the academic or training event by the end of each semester. Submission of the completed logbook is a must before the final examination of the course.
Assessment of Academic Activities
An academic schedule is released every six months, comprising seminars, case conferences, psychosomatic discussions, and journal clubs; two academic events are held a week for 60–90 minutes. The academic schedule is devised to keep proportional equality and adequate time intervals for each trainee; care is considered for all possible leaves and absentees of the student; in case of a missed event, it is carried over to the next semester to be performed by the same resident. Hence, in a three-year rotation, each resident gets equal assignments and opportunities to work with senior residents and faculty.
The newly joined resident is exempt from active academic assignments but attends all routine academic activities, considering the first six months as a probation period. All new residents attend evening classes taken by senior residents, which introduce basic concepts and facilitate the resident’s transition as a beginner.
Two to three seminar topics were collected from each faculty member from the current semester; two residents of different seniority share the presentation in equal proportion and time. A faculty and a senior resident supervise each activity as a co-supervisor who guides the residents in their academic exercises. Each exercise has a precise format, a specific learning objective (SLO), a demonstration (elicit specific findings during the case demonstration), and timelines, with 15 minutes of discussion or a question-answer session and verbal feedback from all the faculties at the end. Finally, the supervisor and a senior faculty member present during the academic activity provide a score on a structured score sheet; these scores are entered into a Microsoft Excel sheet for the respective residents.
Work-based Assessment
To exercise “in-training assessment,” a strategy based on observational assessment (OA) has been adopted. Each postgraduate is assessed in various domains, such as clinical skills, history documentation, examination, routine notes, attitude, initiatives, interpersonal relationships with other residents, discipline, and punctuality (Table 1). Each resident was allotted patients and entrusted with the responsibility of clinical care individually; they also assigned specific observable tasks during their clinical rotations, such as administering a long-acting antipsychotic injection intramuscularly, detecting motor evoked potential (MEP) during transcranial magnetic stimulation, or performing electroconvulsive therapy each followed by feedback by a faculty. Faculty also assessed each trainee independently based on their day-to-day observations during clinical rounds, patient management, assignments, and duties by providing a score on a score sheet. Scores are collected from each faculty member; the final score is achieved by averaging the scores. Hence, each faculty member gets an equal chance to rate trainees, and the trainees also receive multiple scores from different sources numerous times, as this exercise is repeated every six months.
The Areas to be Assessed Based on Observation by the Faculty Every Six Months.
*The scores are converted into percentages; A score below 50 requires strong attention; 50–60 needs to be worked on; 60–70 is good; >70 is excellent.
Internal Assessments Exams
At the end of each semester, trainee residents undergo an internal assessment (IA) comprising written theory and practical exams. The newly joined resident also appears for their first IA; their score is considered probation marks. All residents transition to the next semester irrespective of their marks. Hence, these IAs are considered part of FA, except the last IA, which is conducted as per institution policy to determine the resident’s eligibility to appear in the final examinations, which are extensive and summative. For the theory examination, two to three faculty members contribute questions, and they also check the answer sheet for their respective questions. For practical examination, the resident is allotted a fresh case from the outpatient department (OPD); the resident must work up the case in forty-five minutes and present it to the faculty available in the OPD. Usually, a trainee undergoes six IA before appearing for the final exam. The marks allotted in theory and practical exams are also entered into the individual Microsoft®Excel sheets.
360° Evaluation/Multisource Assessment
Multiple supervisors’ assessments of the trainees in varying settings on multiple occasions contribute vital and reliable information about a trainee. Though these enlisted pedagogical FA strategies are commonly employed in postgraduate medical training in most institutions, combining it with a WBA with the element of innovation of recording individual trainees’ scores on Microsoft Excel and projecting as graphs visualizing trainees’ progression helped generate an overall evaluation of trainees’ sphere of influence (Figure 2). Further, these pictorial graphs are helpful for teachers in providing astute feedback and enabling trainees to visualize their progress.
The Components of Formative Strategies are used to Generate Progress Graphs for the Trainees.
A Dummy Depiction of the Trainee's Overall 360-Degree Assessment Report.
The assessment scores of each semester’s assignments, internal assessments (theory and practical), work-based assessments (clinical skills, documentation, communication, examination and interview skills), attitude and other initiatives are assessed semester-wise, and scores are averaged and projected as graphs. A Score Below 50 Requires Strong Attention; 50–60 needs to be worked on; 60–70 is Good; >70 is Excellent.
Feedback
In addition to contextual feedback from time to time, an assessment-based feedback session is held at the end of each semester; before that, all faculty members review the trainees’ score sheets and convert scores into a graphical form for better visualization by the trainees (Figure 1). Later, individual residents are called, and their strengths, weaknesses, and difficulties are discussed. At the end, each resident is also asked for his/her opinion.
Notable observations are that the assessments and opinions of individual supervisors, the trainee’s reflection, and the student’s 360° assessment report largely agree with and concur with the opinion of the external examinee during the final degree examination. Hence, the methods adopted provide a ground of minimal discrepancy where trainees and teachers mutually agree to pursue common training goals and potentially have predictive value.
The 360° assessment is meant to assess trainees in various training domains, and a semester-wise projection gives an idea of students’ progress. The employed method helps to gaze at the trainee’s progress, but its educational impact is hard to attribute in the current context. However, the preliminary observation revealed the improving trends of graphs of trainees’ progress. Still, we require a more systematic, methodologically robust observation to assess the outcome measures.
The challenges perceived are that maintaining meticulous records of each trainee is time-consuming and requires effort; moreover, compiling electronic data from a physical logbook results in the duplication of work. Hence, using more sophisticated technology, such as asynchronous or synchronous databases, can be considered a solution in the future. 12
In conclusion, considering the challenges of a clinical department, we used simple strategies with reasonable objectivity involving stakeholders (faculties) and multiple evaluations from multiple sources to strengthen the validity of the overall assessment, with an innovation of having graphical projections using Microsoft PowerPoint® which enabled teachers and students to visualize their progress. The preliminary observation favors the positive educational impact of pictorial presentations of students’ progress, facilitating monitoring and feedback. There is a need for technologically advanced adaptation in FA strategies to enhance feasibility and broader applicability. Moreover, systematic evaluation of students’ and teachers’ perspectives and outcome measures are required to assess the educational outcome measures.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Declaration Regarding the Use of Generative AI
None used.
Ethical Approval
The authors shared their experiences regarding the strategies adopted officially by the department. Hence, the institutional ethics committee waived off the need for consent.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.