Abstract
Coronavirus disease 19 (COVID-19) pandemic has shaken the higher education sector. Indian academic institutes are well acquainted with the traditional closed book examination; however, the pandemic has forced the institutes to resort to new methods to evaluate the students. Many academic units in India did not take the examination in the light of public health, but in the case of the University of Delhi, it decided to conduct an online open book examination (OBE). Therefore, in the present paper, we aim to evaluate the strength, weaknesses, opportunities, and challenges of conducting online OBE by referring to the vast literature available. We further extended our study by assessing the students’ performance in an OBE and closed book examination. The results unveiled that students tend to score higher marks in the case of an OBE set up compared to closed book examination.
Coronavirus was discovered in December 2019 and since then the virus has spread quickly across the globe. Coronavirus has claimed millions of deaths and has turned once the hectic metropolitan cities into phantom cities. In the light of public health, most countries have resorted to nationwide lockdown, thereby halting economic activities, closing offices, public institutes, educational institutes, etc. The pandemic has caused a drastic reduction in economic activities coupled with invisible costs, such as loss of lives, starvation, indebtedness, and extreme stress.
Coronavirus disease 19 (COVID-19) has hit the economy hard and the education sector is no exception. The virus has forced education institutes to redesign curriculum, reinvent the teaching–learning process, and restructure students’ performance evaluation system. Universities worldwide have encouraged e-learning and online examination, and Indian universities have also resorted to the same. Scholars have criticized online teaching methods for being discriminatory to the marginalized students, due to inaccessibility of the internet, computer illiteracy, etc. However, it has been praised for its flexibility, affordability, etc. During these tough times, e-learning is no longer an option, but a necessity.
Similarly, the online mode of examination is the panacea for this crisis. In India's case, many private universities were unaffected by the crises as they already had a sound digital infrastructure for conducting an online mode of examination. However, underfunded public universities found it difficult to adapt themselves to new situations. Education plays a crucial role in developing a progressive society. Therefore, the education sector should be resilient. However, this pandemic has questioned the education sector's preparedness to deal with such a crisis-like situation.
As per an estimate by the United Nations Educational, Scientific and Cultural Organization (UNESCO) (Table 1), about 34 million learners are affected by the COVID-19 pandemic at the tertiary level of education. However, at the preprimary, primary, and secondary levels about 10, 143, and 133 million learners are affected, respectively. Given the quantum of affected learners, it becomes crucial to evaluate the aftermath of COVID-19 on the higher education sector of India.
Affected Learners in India Based on School Type.
Source: The United Nations Educational, Scientific and Cultural Organization (UNESCO).
COVID-19 has dictated the universities to restructure the conventional mechanism used to evaluate the student's performance. In this study, we aim to discuss how COVID-19 has impacted the student's evaluation tools by taking the case of the open book examination (OBE) at the University of Delhi. The objectives of this study are as follows:
To conduct a strength, weakness, opportunity, and challenges (SWOC) analysis of an online OBE during the pandemic at the University of Delhi. To examine whether there is any difference in the performance of students when an online OBE is adopted in place of a closed book examination (CBE).
Literature Review
OBE as a Tool for Student Evaluation
Evaluation in education is a rational act (Suchman, 1972). Student evaluation helps to critique the teaching and learning process and guides future teaching acts (Izard, 1992; Johnson, 1977). One of the tools or methods for evaluating the students is an OBE. In an OBE, students are encouraged to use classroom notes, textbooks, etc., while answering the questions. The rationale of such an examination is “reasoning” rather than recalling the facts (Tussing, 1951).
OBE enhances the learning environment and helps students to understand and respond to questions in better way (Brightwell et al., 2004). While preparing for OBEs students consult various sources, such as textbooks, classroom notes, online blogs, etc. This helps them interrelate the concepts and encourages them to acquire knowledge in a more creative approach and avoid “rote learning” (Theophilides & Dionysiou, 1996; Theophilides & Koutselini, 2000). The studies conducted in the past have shown that while comparing students’ scores for OBE and CBE, there is no statistical difference, even though students who took CBE had a slightly higher score (Brightwell et al., 2004; Ioannidou, 1997).
Education Sector Under the Arrest of COVID-19
Perhaps, crises and disasters (man-made and natural) are some of the biggest obstructions in the road to education. Therefore, the education sector must remain resilient even during tough times such as COVID-19. In the past, education sector has dealt with crises and disasters and has devised plans to continue teaching and learning. For instance, the University of Canterbury collapsed in February 2011 due to an earthquake; however, Information and Communication Technology (ICT) tools encouraged the university to stand back on its feet (Todorova & Bjorn-Andersen, 2011). Similarly, after violent Hurricane Rita and Hurricane Katrina, Southern University at New Orleans succeeded in providing basic education to its students via ICT tools (Omar et al., 2008).
Now during the coronavirus crisis, the education sector has gone through a transformation. Universities are adopting online lectures, OBEs, online tests, etc. (Mohapatra, 2020; Strielkowski, 2020). Education institutes in India are well acquainted with a traditional face-to-face classroom setup and therefore reluctant to switch to e-learning. Prior to COVID-19, some institutes had gradually started implementing blended learning techniques, but not on a large scale. However, the sudden strike of COVID-19 has forced all the academic units across India to shift to an online teaching mode overnight. In these tough times, e-learning is a cure-all to COVID-19 (Dhawan, 2020). Initially, institutes were reluctant to resort to the online mode of teaching. However, hesitation to change will be infertile. Academic units shall work in a direction that helps them introduce changes faster and maintain the quality of education (Carey, 2020).
As macroeconomic evidence points out, education is a crucial determinant of income and gross domestic product (GDP) growth (Hanushek & Woessmann, 2007; Krueger & Lindahl, 2001). Therefore, it is apt to say that loss of education due to COVID-19 will only add to the economic losses. This skill loss can cause a loss of 1.5% to the GDP, equivalent to USD15.3 trillion, for the rest of the century (Hanushek & Woessmann, 2020).
After looking at the literature, students’ evaluation during the COVID-19 is an under-theorized topic. Although many research pieces are available on how online teaching has become a vital component of the education sector during this pandemic, how it has impacted the student evaluation mechanism remains an underexplored topic.
Methodology and Methods
To meet the dual objectives of the study, two methodologies have been proposed and implemented.
SWOC Analysis (Descriptive Study)
SWOC is done to identify various strengths, weaknesses, opportunities, and challenges of online OBE during this crisis-like situation. To conduct this SWOC analysis, a descriptive study is undertaken to critically understand how India's higher education sector has restructured the mechanism to assess learner's performance. Considering the qualitative aspects of the research study, extant literature is being examined throughly. The SWOC analysis is completely based on the secondary data collected from various journals, reports, search engines, research papers, scholarly articles, and other academic publications.
OBE versus CBE (Empirical Analysis)
We further extended our study by analyzing if there is any difference in the students’ performance when OBEs are being utilized as a technique for evaluation. In the past, some attempts have been made by scholars to compare the marks scored by the students in OBE and CBE. In a study by Brightwell et al. (2004), 196 students first completed a closed book assessment and after 1 week, students were asked to complete an open book assessment with the assistance of required textbooks. The students were asked the same set of questions under similar circumstances. Further attempts were made in this direction by Cade et al. (2018).
In the present study, the student's performance has been evaluated by comparing the marks scored by the students in OBE and CBE. In other words, we aim to test if there is any difference between the grades awarded in case where OBE is adopted in place of CBE. For this purpose, the test was administered under similar conditions for the CBE and OBE. For online OBE, Google form was employed. The test population comprised 212 fifth semester Bachelors of Commerce (Hons) students at the University of Delhi. Students were given 1 hour limit to complete a CBE of management accounting and 2 days later were given an OBE, where they were allowed to use their course textbooks and internet sources. The same set of questions was given to the students for both closed and open book sittings. The question paper consisted of 20 multiple choice questions (two marks each).
Recent literature suggests that OBE creates a better learning environment and improves students’ performance. Therefore, the following hypotheses have been formulated:
SWOC Analysis of Online OBE at the University of Delhi
Online OBE at the University of Delhi: Little Background
COVID-19 forced all the institutes to shut their doors and encouraged distance learning. The strict nationwide lockdown was imposed and met all the international standards (Ray & Subramanian, 2020). In these tough times, when traditional face-to-face setup was impossible, the University of Delhi became the first central university to restructure its curriculum. Earlier, the university typically followed the “75 plus 25” regime for most of the subjects offered under the choice-based credit system. Under this regime, semester examination was conducted for 75 marks and internal assessment (IA) was weighed for 25 marks. IA was further divided typically into three parts: class tests (10 marks), assignments (10 marks), and attendance (5 marks). For some subjects/courses, IA and semester examination also consist of lab practicals.
After the imposition of lockdown, colleges and departments are arranging online classes via Google Meet, Microsoft Teams, Zoom, etc. Many colleges have developed their own platforms for conducting online classes (Ashri et al., 2020). However, scholars believe the usage of e-learning portals varies across genders and magnifies the challenges faced by hearing-impaired students (Manzoor, 2020; Shahzad et al., 2020). These sudden changes in pedagogy also call for changes in the student's evaluation system. The previous “75 plus 25” regime could no longer be used as a reliable policy as arranging physical CBEs becomes impossible. Therefore, to cope with this crisis, the university adopted a one-time measure to promote students by grading students on the basis of IA and marks awarded in the previous year/semester/term by attaching 50% weightage to each. This one-time measure was applicable to only students of the second semester or fourth semester of undergraduate and postgraduate courses and students of second year undergraduate courses enrolled with School of Open Learning and Non-Collegiate Women’s Education Board under annual mode (University of Delhi, 2020a). However, students of the final semester/term/year for the undergraduate and postgraduate courses were supposed to appear for online OBEs for the academic year 2019–20 (University of Delhi, 2020b).
Strengths of Online OBE
It is commendable that the University of Delhi adopted the OBE strategy along with the blend of technology. Developing an e-portal whereby the students can download question papers and upload their answers to the particular questions from any part of the country is challenging, but the university succeeded in its pursuits. The “Anytime Anywhere” feature of the OBE portal is beneficial during the time of not only the corona pandemic but also any other disaster or calamity. Providing flexibility to the students to access and developing a portal that requires only basic computer literacy are two significant aspects of OBE.
Talking about the benefits of OBEs as a tool for evaluating students, the literature suggests that OBEs can reduce the students’ fear, anxiety, and stress levels (Dale et al., 2009; Michael et al., 2019; Tussing, 1951). It is to be noted that anxiety and emotional block can negatively influence the student's test performance by diminishing working memory resource availability (Grimley et al., 2008) and leading students to use inappropriate cognitive strategies (Diseth et al., 2008). OBE encourages learning and prepares students for real-world decision making (Green et al., 2016). In an OBE student also seeks answers from various internet sources apart from course textbooks, classroom notes, etc. Studies have shown that allowing students to access the internet for searching the answers boosts student performance and face validity (Dijksterhuis et al., 2013).
Weaknesses of Online OBE
OBE suffered from many weaknesses. Students tend to avoid their thinking and become reliant on their textbooks (Ioannidou, 1997). Many studies have pointed out that students spend less time preparing for OBEs (Broyles et al., 2005; Eilertsen & Valdermo, 2000) and avoid diving deeply into the books (Heijne-Penninga et al., 2010).
Indian academic institutes are certainly not ready to adapt to such a new system. The hasty development of the OBE portal at the University of Delhi has also led to technical glitches. Students often complain about how these glitches have led to the late submission of answer sheets. Apart from the technical drawbacks of the portal in terms of its working, technical issues could be observed in the result declaration. Such cumbersome and mismanaged OBE has led to a delay in the result deceleration, which negatively affected the students. The misery of the students did not end here; some students were marked absent or zero. Due to this, students could not apply for higher education in their preferred institutes timely, and offer letters from corporates got delayed (Chettri, 2020).
Many academicians and teachers believe that this sudden shift to the online teaching and evaluation mode has resulted in more academic dishonesty (Moralista & Oducado, 2020). Internet penetration in India, especially in rural India, is still a concern. Even during the online mode of teaching, students have raised concerns over internet connectivity (Agormedah et al., 2020; Agung et al., 2020; Farooq et al., 2020; Figueroa et al., 2020; Hodgson & Hagan, 2020; Paudel, 2020; Qazi et al., 2020). This problem of internet connectivity continued even during the OBE. The main factors responsible for a low rate of technology or internet penetration rates in developing countries, such as India, are income disparity, telephone density, legal quality, and human capital (Chinn & Fairlie, 2010).
Opportunities for Online OBE
The Indian education system has been criticized for its curriculum-driven teaching (Joshi, 2016; Thomas, 2017). Question papers consist of questions that often require recalling the facts and answering the questions in a “bookish” language. In other words, students are not encouraged to go beyond the literal text. OBE as a tool of evaluation allows putting higher-order questions in the question paper. Online OBE does not demand a student to copy from textbooks or internet sources merely. The learners are required to understand and identify the topics to locate and use what is appropriate. OBEs can be curated for students to solve a problem effectively and efficiently by using textbook materials. Greater performances could be observed if a student knows what to search, where to search, how to search, and how to effectively use the data. This requires specific knowledge, skills for identifying credible sources, and proper navigation with a time factor (Mekala, 2011; Zagury-Orly & Durning, 2020).
Therefore, online OBEs provide an opportunity to evaluate a student based on his skills to identify, apply, analyze, synthesize, and evaluate information instead of merely recalling the facts and delivering literal text.
Challenges to Online OBE
Implementing OBE at the University of Delhi was met by resistance from teachers and student unions. They argued that online OBE setup is discriminatory to marginalized students, who find it difficult to arrange electronic gadgets and proper internet connectivity. Furthermore, students have expressed their concerns about the nonavailability of the required study material (Ashri et al., 2020). In a nutshell, two broad challenges to online OBE are lack of digital infrastructure and nonavailability of requisite study material.
However, the biggest challenge of all is perhaps lack of familiarity to OBEs themselves. In India, not only students but also the teachers are not well acquainted with the concept of OBEs. Curriculum and teaching practices at the university have never focused on developing the skills required for a student to crack an OBE. OBE focuses on the synthesis, analysis, and application of interrelated concepts. However, traditional set up has always encouraged rote memorization. Furthermore, many academicians, who are being appointed to design question papers, may not know how to set a question that enables them to apply knowledge through analysis and critical thinking. OBE should be devised to examine the learner's interpretation and application of knowledge and necessary thinking skills.
Empirical Results: Open Book versus Close Book
As a part of the initial analysis of the data collected, that is, the marks obtained by the students in a closed book and open book set up, descriptive statistics have been presented in Table 2. The maximum marks obtained in both situations is 40. However, the minimum marks in an open book set up and a closed book set up stood at 30 and 24, respectively. The median marks for the CBE are being reported at 29, whereas the median marks in an open book set up is being reported at 35.
Descriptive Statistics.
To compare the test results of closed book and open book sittings, a paired t-test is being employed. The results of the same are being presented in Table 3. The results showed that t statistic of the test stood at 7.0874. Due to the low p value (<1%), the null hypothesis is rejected. In other words, there is a significant difference between the mean marks obtained in open book and closed book settings. The alternate hypothesis is being accepted. Therefore, it is being concluded that mean marks scored by the students in an OBE is significantly higher than the mean marks scored in a CBE.
Student Marks (Difference in Mean Marks).
*Represents significance level at 1%.
This is in line with the results of studies conducted in the past, showing that students score higher in an OBE (Agarwal et al., 2008; Francis, 1982; Gharib et al., 2012; Krarup et al., 1974). However, many researchers have also provided empirical evidence that there are no significant differences in the performance of the students (Brightwell et al., 2004; Ioannidou, 1997; Pauker, 1974). OBE equips the students with the required material for attempting questions and the learning strategies changes. They tend to replace surface learning with deep learning (Myyry & Joutsenvirta, 2015) and open book assessment encourages higher cognitive processes (Gulikers et al., 2006). Therefore, the academic performance of the students improves.
Recommendations
COVID-19 has caused significant damage to the education sector. In the near future, opening up colleges and schools seems difficult. This closure of academic institutes will result in the students’ academic loss, and mitigation of this loss seems even more difficult. It becomes imperative that the teaching and assessment strategies be molded to minimize academic loss during these tough times.
This study discussed how the University of Delhi met the challenges posed by the COVID-19 by adopting an OBE as a tool for assessing the students. However, this mechanism needs improvement. First, appropriate investments need to be made in the development of digital infrastructure. Second, universities should revamp the curriculum. The present curriculum lacks conceptual learning. Third, the evaluation system needs to be upgraded. The traditional three hour examination does not focus on the actual learning outcome. Students should be evaluated based on communication and leadership skills and extracurricular activities. Finally, educators need to be better trained. Teachers need to be more flexible in adopting new technology, teaching practices, and evaluation methods.
This pandemic has raised several questions on the education system's preparedness to deal with crisis-like situations. Perhaps, COVID-19 shall be viewed as an opportunity to revamp the education sector. Rather than relying on the conventional mechanism, the institutes should incorporate new changes to encounter the corona pandemic's real challenges.
Conclusion
The present study aims to meet the dual objective; to conduct SWOC analysis of OBE and comparison of the student's performance (marks) in an OBE with CBE by taking the University of Delhi case. To meet these objectives, many peer-reviewed studies were referred to.
OBE is a panacea for the higher education sector to the COVID-19 crisis. OBE is an antidote to the rote-learning-oriented education system of India. Looking at the literature, it appears OBE is perhaps a good tool for assessing the student's performance if the question paper is designed in a way that it tests the higher-order thinking skills of the students. OBE challenges the traditional mode of examination, which merely requires recalling the textual facts on a sheet of paper. Though, in the case of India, implementing online OBE is challenging. Proper resources and time needs to be devoted to design a well-versed curriculum and training of the students and teachers. OBE can help improve the quality of education and make the education sector resilient.
This study contributes to the literature by critically examining the impact of COVID-19 on student's evaluation system in the higher education sector.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
