Abstract
Background
The standards set by accreditation bodies for student assessment during higher education, such as those of the National Commission for Academic Accreditation & Assessment (NCAAA), are necessary in formulating educational programs. These serve as a benchmark for how colleges or universities are assessed and reflect students' learning. Following the implementation of these guidelines, the Department of Obstetrics and Gynaecology, within the Faculty of Medicine in King Abdulaziz University (KAU), established assessment strategies appropriate to their curriculum, which were valid and reliable, thus enabling students to be fairly assessed throughout their undergraduate course. Since KAU is currently preparing for accreditation by the NCAAA, this study was a necessary undertaking to ensure that the assessment strategies designed by the Department of Obstetrics and Gynaecology are aligned and conform to the NCAAA student assessment guidelines, thereby outlining the standard of expected performance and learning outcomes for students.
Objectives
This study aimed to evaluate the assessment plan of the obstetrics and gynecology clerkship for undergraduate medical students within the Faculty of Medicine, KAU, in comparison to the standard criteria for student assessment as implemented by the NCAAA.
Materials and Methods
A cross-sectional study was conducted, which employed two questionnaires containing questions based on the NCAAA guidelines. The surveys were distributed among the teaching staff and students rotating for 12 weeks within the Department of Obstetrics and Gynaecology, KAU, from September to November 2013. In total, 100/116 (86.2%) students and 26/36 (81.25%) teaching staff participated in the study.
Results
Two sets of results were obtained regarding the student assessment practices in the Department of Obstetrics and Gynaecology, KAU, one from fifth-year medical students and the second from the teaching staff. The results showed that the majority of student respondents agreed that the department conformed to the guidelines and standards as set out by the NCAAA. However, the statistics gathered from the teaching staff participants implied that although the department is adhering to the standards, further improvements are still required in terms of the implementation and design of assessment strategies.
Conclusions
Adherence to the NCAAA student assessment guidelines and its strict implementation are vital to achieving effective learning outcomes among obstetrics and gynecology students. Faculty staff need be appropriately trained in the field of exam preparation and assessment. Further improvements to the existing student assessment strategies used by the Department of Obstetrics and Gynaecology, KAU, are recommended.
Introduction
Student assessment plays a major role in the process of medical education and in the lives of medical students, 1 and it informs society of the competence of physicians as individuals who have the skills to take care of the public. Indeed the very foundation of a medical curriculum is built around assessment milestones for students. 2 The National Commission for Academic Accreditation & Assessment (NCAAA) was established to set standards, and accredit institutions and programs within post-secondary education in Saudi Arabia. 3
Provision of the best educational experience is recognized internationally as is one of the major aspirations of most colleges and universities. Teaching strategies are continuously improved and innovated in order to achieve quality learning among students. The effectiveness of the teaching methods employed on student learning should be regularly assessed, and assessment instruments have been designed to help determine how faculty members actually teach and how students really learn.
Consequently, it is important to evaluate the student assessment plan for the obstetrics and gynecology clerkship, which is studied by undergraduate medical students in the Faculty of Medicine, King Abdulaziz University (KAU). 4 This will ensure that it meets the professional standards, continues to strive toward academic excellence, strengthens its various higher learning accreditation requirements, and best serves its students.
Background
The standard of teaching in medical schools must exceed, or at least be equivalent to, that implemented in high-quality international institutions. In a study by Van Zanten et al, 5 it was shown that by applying national and international standards, institutions can improve and help promote high-quality education experiences. “Based on the National Standards Programs 6 of August 2013,” The NCAAA in Saudi Arabia was established to develop a set of standards for quality assurance and accreditation of higher education institutions and programmes, which covered eleven general areas of activity: mission goals and objectives, programme administration, management of programme quality assurance, learning and teaching, student administration and support services, learning resources, facilities and equipment, financial planning and management, employment processes, research, and relationships with the community. These standards are based on what is generally considered to be good practice in higher education globally and have been adapted to meet the particular circumstances of higher education in the Kingdom of Saudi Arabia.
According to Wilson, 7 because medical schools must constantly conform with accreditation standards, they can establish a formative accreditation process that ensures the awareness of new standards of revisions and maintains ongoing compliance. In the General Medical Council's (GMC) 2010 report, 8 Dr Katherine Boursicot wrote: “Medical schools have had to scrutinise their own assessment strategies and have endeavoured to move away from disjointed, localised, inconsistent, outmoded and often unmonitored (ie, not quality assured) assessment in various parts of their courses, and take an overview of assessment over the whole curriculum. This movement has been variably successful and requires more work to ensure that students are fairly and adequately assessed over their whole undergraduate course. The development and implementation of a coherent and consistent assessment strategy in each institution is still a major challenge.” The Guidelines for Good Assessment of Student Learning 9 states: “The primary purpose of assessment at Eastern is to improve the educational experience so that students leave us better prepared for their professional, civic, and personal lives. This does not imply criticism of the status quo, but it is a recognition that as students, society, and the workplace change, education must adapt to those changes. Every program, even the most successful, can benefit from a process of continuous inquiry, reflection, and improvement. If improvement is the point, then the most important comparison for each programme is what it accomplishes in relation to its own past record. Comparisons to other programmes on this campus, or to similar programmes on other campuses, are of secondary interest.”
Subjects and Methods
A cross-sectional study was undertaken that employed two questionnaires, which were distributed to the teaching staff within the Department of Obstetrics and Gynaecology, KAU, and the undergraduate students who were undertaking a 12-week placement within the department from September to November 2013, respectively. All the respondents were asked to complete the questionnaires regarding their perception of the current student assessment plan, which is based on the NCAAA guidelines. Out of 116 undergraduate students, 100 (86.2%) participated, and among the 32 teaching staff, 26 (81.25%) responded. Ethical approval for the study protocol was granted by the Unit of Biomedical Ethics Research Committee, King Abdulaziz University.
With reference to the NCAAA guidelines, the student assessment methods and teaching strategies include web-based student performance systems, benchmarks, key performance indicators, analysis, exams, portfolios, long and short essays, log books, analytical reports, individual and group presentations, posters, journal, case studies, laboratory manuals, video analysis, group reports, laboratory reports, debates, speeches, learning logs, peer evaluations, self-evaluations, videos, graphs, dramatic performances, tables, demonstrations, graphic organizers, discussion forums, interviews, learning contracts, anecdotal notes, artwork, and concept mapping. The teaching strategies selected and utilized are based on the curriculum being taught and on the intended learning outcomes of the students.
The obstetrics and gynecology clerkship for undergraduate medical students attending KAU spans in total 12 weeks. It fulfils 16 credit hours out of the 240 hours required for the MBBS program, and it is undertaken during fifth year of study.
The course learning outcomes aim to achieve the following:
By the end of the course, the student will utilize the basic sciences of human embryology, anatomy, histology, and physiology, to interpret female clinical and morphological changes.
The student should have acquired sufficient knowledge about common obstetrics and gynecological conditions and will be able to understand the natural history of diseases that particularly affect women.
The student should have skills in taking histories and performing clinical examinations in order to formulate the appropriate management of a patient, particularly when a woman's life is threatened.
The student will learn to function as part of a team and will respect the relevant Islamic values concerning ethics.
The 12-week clerkship is a continuous process that is divided into three phases, each with different but complementary objectives:
Phase 1 (Week 1): basic foundation in obstetrics and gynaecology,
Phase 2 (Weeks 2–4): knowledge and clinical skills development,
Phase 3 (Weeks 5–10): knowledge and clinical skills application, and
Weeks 11 and 12 are devoted to revision and the final examination.
The objectives of the clerkship is to train students to be equipped with the knowledge and skills with which to identify clinical problems, pose appropriate clinical questions, seek answers, outline a management plan for patients, and adopt an evidence-based practice. In addition, students are expected to develop their competence in defined clinical skills and also their professional attitude.
The clerkship is composed of lectures, tutorials, self-directed learning (SDL), and also problem-based learning (PBL) through virtual case analysis. In addition, case presentation sessions are held whereby students take the medical history of patients. Students also have many opportunities to receive further clinical and patient-oriented teaching through exposure to different clinical facilities and on some occasions will actively participate in settings such as out-patient clinics, in-patients wards, labor rooms, the fetal medicine unit, operating theaters, and the emergency department. In order to facilitate these opportunities, students are divided into 10 groups, each composed of 15 students, at the beginning of clerkship. Lectures are delivered to the students usually in the main auditorium of the university hospital.
The current clerkship assessment system is composed of the following modalities:
Contribution and attendance (10 marks): Students are expected to not only attend all the scheduled teaching sessions but also positively contribute during learning activities, particularly within clinical environments (wards, clinics) and also during tutorials and PBL sessions.
Assignment (10 marks): Students are required to submit a written assignment based on a particular case they have observed during their clerkship. This requires a brief description of the patient's complaint and a current discussion on a related aspect, a differential diagnosis, and a management plan for follow-up. The case should not be more than one A4 side in length and should be submitted to their tutor before Week 8 of the course.
Mid-clerkship multiple-choice examination (MCQ) (10 marks): This is based mainly on the lectures and tutorials administered during the first week of the course.
End of clerkship MCQ (30 marks): This covers all aspects of the obstetrics and gynecology syllabus.
Final in-active objective structured clinical examination (OSCE) (10 marks): This consists of a slide show examination and addresses both clinical skills and basic knowledge.
Final active OSCE (30 marks): This aims to assess the various skills acquired by a student during the course.
Formative assessment: A formative assessment with feedback exercises, including MCQs and OSCEs, is administered during Week 7 of the course. This helps the student to direct their preparation for the final assessment examination.
To date, all of these student assessment methods and strategies are facilitated and implemented by the members of the Obstetrics and Gynaecology Department within the Faculty of Medicine, KAU.
In December 2013, the Faculty of Medicine, KAU, decided to establish a student assessment unit whose primary mission is to provide comprehensive, standardized quality testing services, course assessments, and report to the responsible administrators of the faculty. In the future, this unit may bring about many changes regarding student assessments with the aim of improving the current situation.
Results
Two sets of results were obtained from this study: those from the students and those from the educators. The perceptions of the medical students undertaking the obstetrics and gynecology rotation from September to November 2013 are summarized in Table 1 and Figure 1. From the results obtained, 60 (60%) students agreed that they considered the assessment to be a component of the entire learning process. Just over half of the students (52, 52%) agreed that assessment tasks were designed to foster valued study habits, while 51 (51%) concurred that the student assessment in the study guide was clearly explained. Similarly, 58 (58.6%) students agreed that the alignment between expected learning outcomes, what was taught and learned, and the knowledge and skills assessed, was clear, and 52 (52%) students agreed that the assessment tasks assessed the capacity to analyze and synthesize knowledge, rather than simply recall information. The majority of students (61, 61.6%) agreed that a variety of assessment methods were employed, and 57 (57%) students agreed that the assessment tasks were designed to assess relevant generic skills as well as subject-specific knowledge and skills. Just under half of the students (46, 47%) thought that there was a provision for their choice in assessment tasks and weighing at certain times, while 48 (49.5%) agreed that the learning load was considered during the scheduling and design of the assessment tasks. The majority (57, 57%) concurred that excessive assessment was avoided, and similarly they agreed that early assessment was used to provide them with feedback (66, 69.5%). Approximately half (51, 52.6%) also thought that the grades were generally fair, and 47 (47.5%) agreed that they did receive explanatory and diagnostic feedbacks,. A similar proportion (50, 50%) thought that there were no inherent biases in the assessment that may disadvantage particular groups, and 48 (48%) believed that cheating was minimized through careful task design, explicit education, and appropriate monitoring of honesty.
Perception of students of the student assessment plan for obstetrics and gynecology clerkship, KAU, 2013.
The statements most and least agreed upon by students.
The second set of results is those regarding the perception of the teaching staff, and this is summarized in Table 2 and Figure 2. Based on the results, there were nine (34.6%) educators who agreed that assessments were considered as a component of the entire teaching and learning process, and the same number (9, 36%) also agreed that there was a faculty or department policy that guides assessment practices. There were 11 (44%) teaching staff who thought that there was a clear explanation regarding the student assessment in the study guide, and 7 (28%) agreed that there was a clear alignment between expected learning outcomes, what is learned and the knowledge and skills assessed. Only 7 (26.9%) educators thought that assessment tasks assessed the capacity to analyze and synthesize, rather than simply to recall information, but 15 (57.7%) concurred that a variety of assessment methods were employed. However, only seven (26.9%) teaching staff believed that the assessment tasks were designed to assess relevant generic skills, as well as subject-specific knowledge and skills. Among the teaching staff, six (23.7%) agreed that there was provision for student choice in the assessment tasks and weighing at certain times, and eight (33.3%) agreed that student and staff workloads were considered in the scheduling and design of the assessment tasks. There were 10 (40%) educators who believed that excessive assessment was avoided and that the assessment tasks were designed to sample student learning. The majority of the teaching staff 17 (65.4%) concurred that early formative assessment was used to provide students with feedback, and just under half (12, 48%) agreed that grades were calculated and reported on the basis of clearly articulated learning outcomes and criteria for levels of achievement. There were only nine (34.6%) teaching staff who thought that students received explanatory and diagnostic feedback as well as grades, and the same number (nine, 34.6%) also believed that there were no inherent biases in the assessments that may disadvantage particular student groups. Finally, 10 (38.5%) teaching staff agreed that plagiarism was minimized through careful task design, explicit education, and appropriate monitoring of academic honesty.
Perception of faculty members of the student assessment plan for obstetrics and gynecology clerkship, KAU, 2013.
The statements most and least agreed upon by teaching staff.
Discussion
This study has shown that the student assessment methods and guidelines being implemented within the Department of Obstetrics and Gynaecology, Faculty of Medicine, KAU, requires further improvement and emphasis to highlight the aim of providing a quality evaluation at the higher education level.
Although the criteria used by the department for the student assessment were closely matched to the NCAAA standards, their implementation should be clear and specific, not only among the students but also for the teaching staff. The standard student assessment must be comprehensively explained to faculty members. The study guidelines as well as the course specifications must be strictly adhered in compliance with the NCAAA standards.
Faculty members should be trained on how to write MCQ items and on how to rate e-students during the OSCEs, as this would help them to become more effective implementers of student learning and assessment. 10
Conclusions and Recommendations
The quality of learning and teaching experience are key components of higher education. According to Norcini et al, assessment within medical education drives and stimulates learning, while also providing data on educational effectiveness, and protects patients. As stated earlier, the assessment criteria should be made explicit in order to form an effective assessment improvement plan. In a study conducted by Baartman et al, 11 10 criteria were listed for evaluating the quality of a competence based assessment: (1) authenticity, (2) cognitive complexity, (3) fairness, (4) meaningfulness, (5) directness, (6) transparency, (7) educational consequence, (8) reproducibility, (9) comparability, and (10) cost and efficiency.
Improving the current student assessment methods is recommended, and this should incorporate Internet-based learning strategies, similar to an idea proposed by one of the students who participated in this study. Faculty staff should continuously participate in workshops and further their training regarding the preparation of student examinations, the setting of test questions, and their teaching methods are the core foundation of effectively implementing assessment strategies for the students. 12
While this study is helpful, future studies are recommended that should focus on specific weaknesses and strengths of the existing assessment methods currently employed within the Department of Obstetrics and Gynaecology, KAU. An investigation of how well the new student assessment unit has performed in improving the overall student assessment strategy is also proposed. In addition, psychomotor item analysis may be useful in order to acquire qualitative assessment results and to evaluate student performance, thus providing better feedback on their learning outcome.
Author Contributions
Conceived and designed the experiments: AMSA, HA. Analyzed the data: AMSA, BE, HS, MB. Wrote the first draft of the manuscript: AMSA, AA, HA, HS, MB. Contributed to the writing of the manuscript: AA. Agree with manuscript results and conclusions: BE, AMSA, AA, HA, HS, MB. Jointly developed the structure and arguments for the paper: AA, AMSA. Made critical revisions and approved final version: AMSA, AA. All authors reviewed and approved of the final manuscript.