Abstract
INTRODUCTION
The National Medical Commission's mandate to standardize medical education through implementing Competency-Based Medical Education (CBME) has necessitated the need for robust assessment strategies aligned with desired clinical competencies.
METHODOLOGY
In the context, the current study was conducted at Pramukhswami Medical College, Karamsad on 150 first-year MBBS students and investigates the efficacy of Direct Observation with Checklist as a teaching-learning and assessment method in anatomy education, specifically surface anatomy in the Abdomen and Pelvis regions of gross anatomy. The study employed qualitative and quantitative methods to evaluate student performance and perceptions.
RESULTS
Quantitative analysis revealed a significant difference in scores between Direct Observation with Checklist and traditional viva voce methods, underscoring its effectiveness. Qualitative findings from focus group discussions highlighted that this method enhanced understanding, retention, and confidence among students. It provided structured feedback, promoted reflective learning, and reduced assessor bias, thus supporting its role in assessing competencies at the “shows how” level of Miller's pyramid and aligning with CBME objectives.
CONCLUSION
The research underscores the transformative potential of Direct Observation with Checklist as a pedagogical tool in anatomy education, advocating for its wider adoption across medical disciplines as it evolves towards competency-based frameworks, such methods are crucial for ensuring that graduates are well-prepared for life-long clinical practice.
Keywords
Introduction
The National Medical Commission (NMC), erstwhile Medical Council of India (MCI), in their aim to standardize the output of graduate medical education in the form of an “Indian Medical Graduate” proposed curricular reforms through their document “Vision – 2015”. While the proposed changes emphasized on designing and implementing a curriculum that focused on desired and observable abilities in the form of competencies 1 ; Unfortunately, the alignment of assessment towards measuring these competencies was neither discussed in appropriate details nor included. 2 In 2018, MCI with the help of panel of experts drawn from across the country, laid the basic framework in the form of three volumes of curricular framework for the proposed Competency-based Medical Education (CBME). 3 The thrust in the new regulations is to make medical education more learner and patient centric, outcome oriented with interactive teaching-learning methods and skill acquisition, but once again there is minimal guidance on assessment leaving it to the discretion of individual universities and institutions. 4
This worldwide shift towards CBME ensures active participation of students in learning and clinical competence, but the restructured curriculum also mandates large-scale reforms in assessment approaches and methodologies. 5 It warrants a robust and multi-faceted assessment system with enhanced attention to formative assessment that drives learning by setting clear expectations, providing opportunities to self-monitor and receiving feedback. Such competence can be better inferred from observable workplace performance using Workplace-based Assessment tools that assess at the “does” level of Miller's pyramid. 6 On the other hand, poorly aligned assessments can mar the quality of learning.7,8
In current pedagogical trends of anatomy, the function of living and surface anatomy has increased in medical curricula with it forming a bridge between clinical practice and gross anatomy. The surface anatomy provides ample opportunity to practice physical examination at very early stage in medical education preparing for life-long clinical practice. Also living and surface anatomy are better tested as a skill, assessed at “Shows How” level of Miller's pyramid. 9 Hence the current study was designed to evaluate Direct Observation with Checklist as a teaching-learning and assessment method in anatomy.
Methodology
Setting and sample
The current observational interventional study was conducted at Pramukhswami Medical College, Bhaikaka University, Karamsad on undergraduate students of first MBBS (n = 150). The NMC Nodal center for Medical Education and technology runs a Competency-Based curriculum mandated by the National Medical Commission in 2019.
Design
The interventional study introduced Direct Observation with Checklist as a teaching-learning and assessment method for surface marking in anatomy and explored student's perceptions through focused group discussions.
Data collection
Sample checklist (Stomach).
Data analysis
The two assessment scores were computed using descriptive and relational statistics (Z-test) for analyzing significance. Z-test is a widely used statistical method to determine whether there is a significant difference between two samples when the population data is normally distributed. As for the FGDs, content analysis was performed through an inductive approach towards the data by two authors, whereby the discussions were openly coded through a thematic step-by-step analyzing method. The discussions were screened in reference to the interview guide to generate codes and categories that lead to a central organizing concept or a theme, having a certain level of pattern or meaning in relation to the questions in the data. Collaborative insights were generated among the researchers, findings were summarized, and a clear, organized final report was generated. 14 No software was used for analysis and the report was shared with the participants for their feedback on the findings.
The reporting of this study conforms to the consolidated criteria for reporting qualitative studies (COREQ) 15 (Appendix 2)
Results
Quantitative analysis
We observed a significant difference in the scores obtained by the students through both the methods. Z test computation also revealed sufficient evidence that there exists a significant difference between the scores obtained through Direct Observation with checklist and traditional viva voce methods (Table 2).
Scores obtained by the students (n = 147).
Qualitative analysis
The thematic analysis of the FGDs revealed that surface marking with direct observation as a teaching-learning and assessment method enhanced the understanding, long-term retention, visualization as well as empathy through a structured approach and point-wise instructions. The participants also praised the method for efficient revision, instilling confidence through repetition of practice as well as catering to different learning styles. As a method of assessment for learning, the participants rated it as a valuable self-assessment tool that aids in error identification and reflective learning. An immediate structured feedback through the checklist often contributed to their accuracy and continuous growth. Moreover, the analysis also underscores the importance of checklist method in reducing assessor bias, increasing objectivity, and having a fair and uniform structured evaluation approach.
The participants also critically analyzed that while it may have led to an improvement in the overall scores there were valid concerns of deep conceptual learning and taking the point-wise instructions as a mere mechanical exercise without any additional effort for exploring the depths of the subject. While opinions varied on its clinical applicability, the method's benefit in aiding creating mental frameworks and improving precision were evidently helpful for further clinical studies. The participants strongly supported the continued use of checklist method for surface marking as well as its incorporation into other areas and subjects owing to its ability to address challenges and positive effects on performance but also suggest taking a hybrid approach through incorporating diagrams and reference texts for streamlining complex structures (Table 3).
Thematic analysis of the focus group discussions.
Abbreviations: FGD, focus group discussion.
Discussion
The results revealed a significant difference in the performance and scores of the students as well as a favorable reception for Direct Observation with Checklist as a teaching-learning and assessment method for surface marking in anatomy. The method aligns with the objectives to assess surface marking at “shows how” level with a constructive feedback mechanism as well as an effective self-assessment. It also promotes deep understanding of the subject by providing ample opportunities for reflective as well as collaborative learning and caters to diverse learning styles.
Abu Bakar et 16 al in their scoping review of effective teaching strategies in surface anatomy suggested multimodal approaches with teaching-learning methods that encompass an active learning environment and interactive feedback to contextualize the learning as well as provide ample opportunities for experiential learning through hands-on-activity and by reflecting on the learned subject. This is in concordance with Direct Observation with Checklist as a teaching-learning method that uses a structured and instructive communication with the students to perform, reflect, and conceptualize specific tasks as checkpoints on a mummified cadavers / mannequin through guided hands-on-activity and provides adequate opportunities for reciprocal and reflective learning by engaging into a self-assessment and feedback as well as peer-to-peer interaction. Such strategies have often lead to improved understanding of anatomy17–19 could probably have led to an improved understanding of anatomy and performance of the students.
Conventionally, for formative purposes, such as feedback and improvement, more weightage is given to educational impact acceptability and cost as weighting criteria for conceptualizing an assessment tool. 20 The results underline the impact and appropriateness of Direct Observation with Checklist as an assessment method. Moreover, direct observation of student performance has been a reliable approach for evaluating proficiency in demonstrating a range of clinical skills and performing physical examinations 21 and a checklist in such context can be useful in assessing competence by breaking them down into specific actions done or not done. 22
While, anatomy education has often been viewed as dull, intensive and pertaining to rote-memorization students often recognize pedagogical frameworks to enhance their understanding of anatomy and its clinical relevance as well as bridge the integration gap with clinical practice.23,24 We believe, teaching and assessment methods such as Direct Observation with checklist that empower students to actively engage in learning and foster their participation can be of significant value addition during this paradigm shift in medical education in India to address the critical challenges of paucity of trained faculty members as well as standard assessment strategies25,26 and the need to deliver a substantial amount of anatomy content within limited time constraints, while also appealing to high-achieving academic candidates. 27
We acknowledge that the FGDs were conducted by the principal investigator and participants were selected by purposive sampling, which though necessary to capture a wide range of outcomes through a fruitful FGD13,14 may have affected the discussions and the notable as well as positive qualitative data analyzed may be interpreted in the context of motivated participants as well as the potential bias (such as confirmation bias, moderator bias) of the principal investigator, despite the intended approach to remain open to the student's experiences. We mitigated the selection bias by ensuring selection of participants for discussion by other faculty of the department and framing bias was avoided by a pre-validated interview guide, also analysis and interpretation was done by co-authors other than PI to avoid any over/under-interpretation of data. While sample size/power analysis was not performed, all students were included for quantitative assessment and adequate representation was ensured for qualitative analysis. And while the results may not be automatically generalizable to different educational settings, the critical question is whether if directly observing a student performance with a pre-rated checklist does simulate a workplace-based assessment? If yes, to what extent?
Conclusion
Direct Observation with Checklist is a an affective teaching-learning and assessment method for psychomotor domain of anatomy education that can not only help in a significantly improving student performance by fostering deeper understanding and promoting active engagement among learners but also be implemented across medical education disciplines as it evolves towards a competency-based framework. It aligns with the competency-based educational framework advocated by the National Medical Commission, in the context of being student-centric, outcome-based, with ample opportunities of feedback and competency assessment, empowering students for a life-long clinical practice. There is a need to explore the method, especially during teaching-learning and formative assessment of psychomotor skills through checklists, even if in hybrid mode for optimum benefits not only in other areas of anatomy but other disciplines as well.
Supplemental Material
sj-docx-1-mde-10.1177_23821205251321807 - Supplemental material for Exploring Competency-Based Anatomy Education Through Direct Observation with Checklist as Teaching-Learning and Assessment Method: Lessons Learnt Through Student Feedback
Supplemental material, sj-docx-1-mde-10.1177_23821205251321807 for Exploring Competency-Based Anatomy Education Through Direct Observation with Checklist as Teaching-Learning and Assessment Method: Lessons Learnt Through Student Feedback by Jaikumar B Contractor, Sanat Rathod and Praveen Singh in Journal of Medical Education and Curricular Development
Supplemental Material
sj-docx-2-mde-10.1177_23821205251321807 - Supplemental material for Exploring Competency-Based Anatomy Education Through Direct Observation with Checklist as Teaching-Learning and Assessment Method: Lessons Learnt Through Student Feedback
Supplemental material, sj-docx-2-mde-10.1177_23821205251321807 for Exploring Competency-Based Anatomy Education Through Direct Observation with Checklist as Teaching-Learning and Assessment Method: Lessons Learnt Through Student Feedback by Jaikumar B Contractor, Sanat Rathod and Praveen Singh in Journal of Medical Education and Curricular Development
Footnotes
Acknowledgements
Dr Dinesh Kumar, Professor, Department of Community Medicine, Pramukhswami Medical College, Bhaikaka University, Karamsad
Author contributions
PS devised and supervised the entire project. JC and PS were involved in data collection while JC and SR processed and statistically analyzed the data. JC and SR were the major contributors in writing and editing of the manuscript. All authors read and critically reviewed the final manuscript.
Data availability
Raw data can be made available upon reasonable request to the correspondence author.
DECLARATION OF CONFLICTING INTERESTS
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethics approval and consent to participate:
The research study has been approved by Institutional Ethics Committee – 2, Pramukhswami Medical College, Bhaikaka University on 14/09/2019 under reference no.: IEC/HMPCMCE/2019/Ex.12/167/19; and the permission to conduct online focus group discussions was approved under IEC/HMPCMCE/2019/Ex.12/142/20. The Institutional Ethics Committee – 2 has been registered with the National Ethics Committee Registry for Biomedical and Health Research, Department of Health Research, Ministry of Health & Family Welfare, Government of India (File No.: EC/NEW/INST/2021/592). As part of the approved protocol, the informed consent requirement by the IEC-2 was waived for quantitative data collection. A verbal informed consent from the subjects prior to online FGD (qualitative data collection) was approved and recorded on ZOOM Meetings.
FUNDING
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The article processing charge for this article was funded by Bhaikaka University, Karamsad.
Supplemental material
Supplemental material for this article is available online.
Appendix – 1
Appendix – 2
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
