Abstract
Emergency remote education (ERD) adopted by universities amidst the COVID-19 pandemic has pursued to maintain students’ satisfaction. The current research inquiries into perception of the satisfaction towards the quality of classes during ERD. The research is carried out through a mixed sequential approach. Six focus groups in its qualitative phase, and 2074 students from the business undergraduate program in Colombia, Peru, and Mexico in its quantitative phase. Measure of satisfaction towards ERD establishes three dimensions: Concerns about academic quality, teaching strategies used by professors, and perceptions of access limitations. This study identifies a moderating effect of the perceptions of access limitations on the relationship of the teaching strategies and concerns about academic quality. In high constraints, =.-016, p < 0.0001, the relationship is weaker than medium constraints, = −0.22, p < 0.0001. In turn, the medium access limitation condition shows a weaker effect than the high limitation condition, = −0.28, p < 0.0001.
Introduction
Historically, higher education in Latin America has faced serious and structural issues in regards to infrastructure, coverage, and the quality of education (Ferreyra et al., 2017). With the appearance of COVID-19, the educational conditions in the region have worsened. On the one hand, low network coverage of households in the region became an obstacle for thousands of students to continue their studies (CAF, 2020). On the other hand, the poor capacity, limited planning and the lack of resources of higher education institutions (HEIs) to assume remote education, forced face-to-face education to be displaced by emergency remote education (ERE).
According to the (World Economic Forum WEF, 2020) during the pandemic, the quality of education depended on the level and quality of internet access. For Latin American countries, the situation aggravates and delves into old issues regarding higher education. Meanwhile, in developed countries, the Digital Ecosystem Development Index is around 80% (out of 100) and in the region, it does not surpass 50% (CAF, 2020). Under these circumstances, many concerns arise towards how the change in the educational model amidst the pandemic affected the quality of the classes in emerging countries. Therefore, the present research investigates how business students from public and private universities in Colombia, Mexico, and Peru perceive satisfaction with the quality of classes amidst the pandemic.
In the region the transition towards ER as a response to the confinement, experienced barriers in relation to technology (Bokolo, 2021), the digital ecosystem development in life conditions of young people in Latin American (Bárcena, 2020; Katz et al., 2020), and the lack of preparation teachers faced when making the transition towards ERD (Whalen, 2020).
The purpose of HEIs was to maintain training without a clear concern for the effect this decision would have on quality of education (Pérez-Villalobos et al., 2021). According to Karalis (2020), the move to ERE was made with serious drawbacks. Teachers accustomed to face-to-face teaching were not clear about how to administer their classes adequately on communication platforms (Bokolo, 2021). On the other hand, Khlaif and Salha (2020) state that “ERE by its very nature, may not accommodate all socioeconomic levels that are present in a face-to-face environment” (p. 130).
Under this scenario, the impact of COVID-19 on educational systems requires input from student perception gathered using a different study modality, such as ERE, which is not comparable to virtual education (Hamdan et al., 2021).
This study was conducted through a sequential mixed methodology (Creswell and Creswell, 2017). In the qualitative phase, six focus groups were conducted with undergraduate students from private and public universities located in three major cities in Colombia. The question explored in the focus groups revolved around the perception of the quality of remote classes. Subsequently, items were created using a Likert scale and submitted for evaluation by four international peers. These items constituted an instrument that was applied online in universities in Colombia, Peru, and Mexico.
The findings reveal that student satisfaction with ERE is dependent on three dimensions: (1) concerns about academic quality; (2) didactic strategies implemented by the teacher, and (3) access limitations. Moreover, there are relationships among these dimensions; for example, access limitations can modify the relationship between didactic strategies and quality concerns. Therefore, the relationship between teaching strategies and academic quality concerns becomes stronger when there are fewer access limitations to technological resources, although it weakens as they increase. Consequently, the perception of satisfaction is limited by the access students have to sustain the teaching-learning process.
Literature review
Since the 90s, remote education has been growing in the university teaching context (Siemens, 2015), and along with it, different teaching modalities have emerged. Virtual education is understood as an educate model where learning activities are carried out online (Perez-Villalobos et al., 2021). This educate model requires multidisciplinary work for the creation of content materials, assessments, and tasks. Therefore, focused on autonomous student learning, virtual education offers resources for students to achieve the desired levels of training (Kerres, 2020; Zayapragassarazan, 2020). In contrast, blended learning is the integration between classroom teaching with online learning experiences (Garrison and Vaughan, 2008). Although these two modalities of remote education require thorough planning, Bokolo (2021) has found “Policy pressure positively influences professors to implement blended learning” (p. 714).
On the other hand, according to Barbara et al. (2014), the planning process of remote education cuts across nine dimensions: modality, pacing, student–instructor ratio, pedagogy, the instructor role online, student role online, communication, assessments, and feedback. Consequently, a high-quality remote education ecosystem requires not only vast resources but also thorough planning.
ERE evaluation variables.
HEI: higher education institutions.
Elaborated based on Hodge et al. (2020)
Recent research has identified some weaknesses of the ERE approach, gleaned from its application during the pandemic. Espino-Dıaz et al. (2020) state that HEIs were not prepared to withstand the demands of technological support. In consequences, teachers became a determining factor in the generation of quick solutions that meant educational continuity as they mitigated the limitations faced by students. Additionally, the teaching-learning environment was not ideal for building rigorous formative processes (Bokolo, 2021). Moreover, teachers' lack of knowledge of technological tools, such as limited and hurried training in online communication became an element that negatively impacted ERE (Karalis, 2020).
Specifically, the lack of rigorous planning to make the transition from face-to-face education to ERE had consequences for online learning (Krishnamurthy, 2020) and student expectations (Mehall, 2020).
Perception of student satisfaction
Literature related to quality of satisfaction (QS) in HEIs is plentiful (Bakrie et al., 2019), and its nature depends on the demographic and sociocultural context (El Alfy and Abukari, 2020; Santini et al., 2017). Quality of satisfaction is directly related to student satisfaction (Elliott et al., 2002). For Barnett (2011), student satisfaction is a potential indicator to assess QS. Therefore, satisfaction is an indicator of quality performance. According to Mukhtar et al. (2015), student satisfaction is related to the perception of experience quality and the performance of educational institutions when it comes to providing educational services.
Quality of satisfaction is a priority for HEIs because it has effect on their market positioning and competitive advantage. Gruber et al. (2010) claim that student satisfaction is a consequence from the service quality perception. Therefore, student perception becomes an important input for the HEIs reputation (Hassan et al., 2019; Wong et al., 2016), and an important criterion for QS decision-making policies (El Alfy and Abukari, 2020). For HEIs, measuring satisfaction is a means to be more competitive in the market (Ahmed, 2021). Hence, student perception plays an important role in determining the factors affecting QS (Nguyen et al., 2020; Oliva and Gómez, 2014).
Since the end of the 80s, satisfaction-measuring tools have been developed. SERVQUAL (Parasuraman et al., 1988) and SERVPERF (Cronin and Taylor, 1994), designed to augment profit for service companies, were the first tools applied to HEIs. Nonetheless, they had several limitations (Annamdevula and Bellamkonda, 2012). These tools failed to reflect the contingency of the country’s context (Weerasinghe and Fernando, 2018), and the literature has proven that QS factors vary according to student perception (Kwek et al., 2010). Consequently, these models have been adjusted (HedPERF, Abdullah, 2006; HiEdQUAL, Annamdevula and Bellamkonda, 2012), and at the same time, the need to create custom-made scales has motivated the development of new tools (Farahmandian et al., 2013; Mukhtar et al., 2015; Saif, 2014) that show student perception variability (El Alfy and Abukari, 2020).
Consequently, tools to measure satisfaction developed for the context of HEIs are not copied or adapted in this article. This is because, first, satisfaction depends on the country’s context and second because QS satisfaction in ERT requires a detailed view focused on a historical and disruptive moment as that caused by the COVID-19 pandemic.
Measuring satisfaction with ERE.
Own elaboration.
Reading the ERE satisfaction measurement scales shows that, although the cultural context of the four countries is diverse, some associated factors are common across countries. In this sense, teacher evaluation is a factor found in the Iranian and Indian; the content design factor is common in the Iranian, Indian, and Jordanian scales. In contrast, Jordan and Iran have in common the interactive learning activities.
On the other hand, each scale has its particularity which confirms that students’ experience with ERE is intersected by the cultural context. For example: the singular factor for Iranian students is the availability of the platform (Osmani, 2021); in India, the reactions to student feedback were presented (Gopal et al., 2021); in Chile, most important was the evaluation of the student’s context (Pérez-Villalobos et al., 2021); and in Jordan, internet self-efficacy and self-regulated learning were prioritized (Hamdan et al., 2021).
The strategies implemented in ERE by HEIs lead to differentiated perception experiences. When ERE is accompanied by technological limitations, the impact on students’ satisfaction levels with the education received can be affected (Bokolo, 2021). Thus, the management of students’ expectations by HEIs has an impact on learning processes.
The discussion on the dimensions that make up the construct of satisfaction in ERE reflects two different aspects. On the one hand, Osmani (2021), Gopal et al. (2021), and Hamdan et al. (2021) understand it as a unidimensional construct, while in Latin America Pérez-Villalobos et al. (2021) measures student satisfaction as a multidimensional construct.
In the context of developed countries, Ho et al. (2021) found that the most relevant predictors of student satisfaction in the ERE context in Hong Kong is related to the effort made by professors. In this sense, the aptitude of the professors as well as methods of assessment provide evidence so that students perceive remote learning as well achieved. The study also shows that students did not experience issues when connecting to learning devices.
Method
This study is developed through a mixed sequential design starting with a qualitative and a quantitative phase (Creswell and Creswell, 2017). This type of design allows exploring under-researched problems and constructing measures to be used in subsequent quantitative analyses (Morse, 2016). Before to start this study, the research committee of the Faculty of Economics and Administrative Sciences at Universidad Cooperativa de Colombia approved it and stated that it did not have a conflict of interest, and it did not require ethical approval for its execution. Moreover, no personal information, such as names or identification number were requested to guarantee the privacy of the participants.
Undergraduate students in Colombia were invited to participate in the qualitative phase through the social media networks. Participants had to be register in face-to-face teaching and have changed to online teaching in the pandemic time. There were no additional exclusion criterion as age or gender; therefore, this study applied a voluntary sampling.
Six focus group were applied online. The average age in the focus group was 20 years old, 18 were male and 20 females. Participants were allocated to the focus groups randomly, pursuing that each one had seven participants. Before the application, participants were informed of the aim of the research and gave their informed consent, additionally, each meeting were recorded with the approbation of participants. The main question in the focus group was how ERD have affected the educational quality.
To increase the rigor of the qualitative analysis, researchers took account the trustworthiness and authenticity criteria. Trustworthiness refers to the truth value, applicability, consistency, and neutrality of the results of a research study (Lincoln and Guba, 1985). On the other hand, authenticity refers to the degree to which researchers capture the multiple perspectives and values of participants (Schwandt et al., 2007). The trustworthiness was established through Investigator triangulation, therefore, each researcher analyzed qualitative data and findings were compared. The authenticity was given by the plural values of the participants. This study has used a wide variety of viewpoints searching the fairness. Especially, it has similar proportion of male and females, students from different semesters, and different social contexts related with type of university (public or private).
Sample description.
The data collected made it possible to study the dimensionality of the instrument developed. For this purpose, samples from each country were randomly divided into two groups. An Exploratory Factor Analysis was applied in the first group, while a Confirmatory Factor Analysis was applied in the second. This cross-validation procedure is recommended by Hair et al. (2014). Polychoric correlation matrices were used to perform the estimations due to the ordinal nature of the items (Jöreskog and Moustaki, 2001). In the AFE, a parallel analysis was used to choose the number of dimensions (Hayton et al., 2004). Additionally, an oblique rotation with unweighted least squares estimation method “uls” was used. Items with weights less than 0.4 were iteratively eliminated (Hair et al., 2014). In the AFC, model fit was assessed using the CFI, TLI, RMSEA and SRMR indices (Hoyle, 2012). For each dimension, the reliability of the measure was studied with the Crombach’s Alpha index, which must be higher than 0.7 to be acceptable (Hair et al., 2014).
Simple sums were calculated with the items grouped in each dimension and these were transformed into standardized values with mean = 0 and deviation = 1. With the scores in each dimension, descriptive statistics were calculated, Pearson correlations between dimensions, countries were compared with one-way ANOVAS with post hoc comparisons of Tukey Bonferroni correction (Hair et al., 2014). In addition, the joint effect of the dimensions was studied with Linear Regression models. Additionally, the possible interaction effect between dimensions was explored.
The analysis was performed in R (R Core Team, 2014) with the package’s “psych” (Revelle, 2011) and “lavaan” (Rosseel, 2012) and the interaction used the PROCESS extension by Hayes (2017) for SPSS.
Findings
Qualitative analysis
Information was analyzed by researchers and grouped into three categories reported recurrently: (1) physical resources such as internet access; (2) perceptions about the quality of classes; and (3) the pedagogical processes employed by teachers. These categories show a strong relationship among them.
In the first category, participants express their concerns about the connectivity related with the physical resources such as internet access and availability of a computer. These barriers affect both teachers and students. It is a structural issue in the emerging countries that could affect the environment in which teacher and student were immersed for the teaching-learning process. Access barriers such as digital inequality mark one of the fundamental concerns regarding the quality of classes. Through the pandemic time, teachers and students used the home resources to keep the learning process but it was inadequate for this pursue. Participants said that in home, the distractive stimuli, obsolete equipment or complex family relations made difficult the development of the lessons.
In addition to the accessing problems, students stated that teachers did not have a training to address online classes. “Without teacher qualification for the use of technological platforms, the quality of education is impacted negatively,” was one comment from the students in the focus group. Since teachers are not trained for this model, they had troubles with the planning and execution of lessons.
The second category focuses on the perception about the quality of classes. Students associate the quality of remote classes with the need for professors to maintain high standards, the contents agreed upon in the syllabus and critical thinking. However, it is evident that students feel a strong uncertainty about how HEIs will respond for the quality of the formative processes. Specifically, they feel that there was no preparation for the transition from face-to-face to remote education, stating that the transition to ERT was hasty and poorly planned. The abrupt change to ERT was focused in transmit the contents under the belief that the student is a recipient of information, decreasing the class practicality and increasing the theoretical approach. Moreover, critical thinking was negative impacted because online classes made difficult the discussion of the ideas. Although academic platforms take tools to incentive the discussion, they were no used or they could not the agile communication between students and teachers.
Finally, in the third category, participants claim about the pedagogical processes employed by teachers. Pedagogical and didactic processes, such as the availability of class material, spaces for dialog to resolve doubts other than remote classes, as well as the way in which the teaching method is reinvented, are determining factors in maintaining the quality of remote classes. Participants highlight the active role of teachers who made an effort to keep the high quality of classes, for they, teachers, were the main support of the learning process. Again, participants stated that teachers did not receive capacitation for the HEIs and they counted with scare resources, nonetheless, teachers achieve to keep the learning process.
The main findings of the qualitative analysis are represented in the Figure 1. Finding of qualitative analysis.
Quantitative analysis
Exploratory factor analysis.
Confirmatory factor analysis.
The effect of the interaction of access limitations. The figure represents the difference found in the relationship between teaching strategies and concerns about quality. TSP = Teaching strategies adopted by the professor; PAL = Perception of access limitations.
Descriptive analysis.
CEQ = Concerns about the educational quality; TSP = Teaching strategies adopted by the professor; PAL = Perception of access limitations.
All values are standardized with zero average and standard deviation 1.
***p < 0.001.
When making cross-country comparisons, significant differences are found in the dimension of Concerns About Educational Quality, F(2,2071) = 25.3, p = 0.024. Peru differs significantly from Colombia and Mexico. There are no differences between Mexico and Colombia. Peruvian students report lower concerns than students in the other countries.
The comparison of the Teacher’s Pedagogical Strategies had significant differences, F(2,2071) = 17.92, p < 0.001. Mexico has a negative perception of the Teacher’s Pedagogical Strategies compared to Colombia and Peru. However, there are no differences between the latter two countries.
Finally, significant differences were found in the Perception of Access Limitations, F(2,2071) = 81.28, p < 0.001. In this case Colombia differs significantly from the other countries. Colombian students report lower Perception of Access Limitations than Mexican and Peruvian students.
Regression models in Colombia–México–Perú.
TSP = Teaching strategies adopted by the professor; PAL = Perception of access limitations.
The figure represents the difference found in the relationship between teaching strategies and concerns about quality.
Finally, perception of Access Limitations showed a moderating effect on the relationship between Teacher’s Pedagogical Strategies and Concerns About Educational Quality, =0.056, p < 0.0001. The effect of pedagogical strategies on concerns about quality is negative; the teacher’s strategies allow to reduce students’ concerns about quality. In the presence of high constraints, =.-016, p < 0.0001, the relationship is weaker than in the presence of medium constraints, =−0.22, p < 0.0001. In turn, the medium access limitation condition shows a weaker effect than the high limitation condition, = −0.28, p < 0.0001. See Figure 1.
Discussion
The decision made by universities to continue classes during the pandemic in order to maintain the relationship with their primary stakeholder group, the students, requires an evaluation of the consequences. This study explored the satisfaction of business students with the quality of education during the emergency remote teaching period. For this purpose, the perception of students in three Latin American countries that implemented similar measures seeking to maintain classes was explored.
The qualitative analysis made it possible to identify three elements related to the perception of satisfaction, the physical resources such as Internet access, perceptions about the quality of classes, and the pedagogical processes employed by teachers. Significant concern is expressed about the effects that ERE may have on quality and on the planification of HEIs to face this challenge. The possible role of teachers is highlighted since their qualification in the use of the platforms and the planning of the teaching-learning process may have an impact on the quality of the classes. In general terms, uncertainty is expressed about the effect that this new teaching modality may have on their training process.
This qualitative description of the satisfaction with the quality of the ERE becomes an approximation to delimit the construct. The quantitative instrument applied was constructed on the three general elements identified. Although the qualitative analysis was carried out only in Colombia, the factor analyses applied in the other countries seem to confirm the generalization of these three dimensions of the construct.
When comparing the construct in the Latin American countries with others reported in the literature and summarized in Table 2, there is support for the multidimensional configuration in line with Pérez-Villalobos et al. (2021). Also, there is support for the contextual nature of the construct reported by El Alfy and Abukari (2020)The measurement of the ERE quality satisfaction construct requires the inclusion of several dimensions that show its breadth and diversity. However, it is important to clarify that the work conducted in Iran, India and Jordan adopted the unidimensional measurement of the construct without exploring its dimensionality. Thus, it is not possible to rule out a multidimensional expression in these countries.
In addition, comparing the countries analyzed in this study, significant differences were found in the Concerns About Educational Quality. Colombian students reported the highest level of concern in this dimension but recognized a greater effort in the development of didactic strategies by teachers. Likewise, in Colombia, students perceive a lower level of access limitations.
Mexican students stand out for their low perception of the didactic strategies implemented by teachers. Among all the countries analyzed, they have the worst evaluation of this dimension. The level of concern for educational quality is similar to that of Colombian students and access limitations had a medium value among the countries compared. Finally, Peruvian students have the lowest level of concern about quality despite reporting the highest level of perceived access limitations among the countries compared. Like Colombian students, they recognize an important effort on the part of teachers in the development of pedagogical strategies.
The quantitative analysis allowed us to explore the effect of student demographic variables and university institutional variables on Concerns About Educational Quality. Older students reported less concern about quality, which may be associated with the point at which the students were in their university careers. More advanced students have developed knowledge and skills that allow them to better assimilate the content taught during the ERE than those students who during the pandemic were at the beginning of their professional careers.
The relationship between the dimensions constructed made it possible to reach a remarkable result. The didactic strategies made it possible to reduce the concerns about quality on the part of the students. In the interaction between student and teacher, described in previous paragraphs, the planning and the way of applying the activities in the class sessions became tools to face the threats related to the perception of quality. The teacher became the central agent of the teaching-learning process, reversing the educational bet where the center is the student. Thus, the teacher’s role during the ERE required an enormous effort to develop didactic strategies.
Additionally, this study was able to provide evidence of the moderating role of access limitations in the formative process. The relationship identified between teaching strategies and quality concerns is modified by the degree of perceived limitations. As shown in the moderation analysis, this relationship is stronger when there are fewer access constraints, but weakens with increasing constraints. Thus, teaching strategies alone were not able to allay potential concerns about quality. Consequently, quality is constrained by the access students have to sustain the teaching-learning process.
Findings of qualitative and quantitative phases highlight the concerns of the students about the quality of the learning process in ERE. There are agreement on the few or none planification of the HEIs when change the modality of the lessons from face-to-face to RER and the active role of the teachers as a main educational agent to keep the high quality. Following to Espino-Diaz et al. (2020) HEIs were not prepared to the challenge and teachers had to take weight of the learning process. Moreover, teachers had no training in online communication or tools for remote education as Karalis (2020) claims.
When the performance of the HEIs in ERE is compared with the planning process of remote education depicted by Barbara et al. (2014) there are clues to understand the source of the concerns of the students. The nine dimensions for planning the remote education were overlooked. Form the strategic approach, the consequence of the ERE is a decreasing in the HEIs reputation, following the statements of Hassan et al. (2019) and Wong et al. (2016).
Nonetheless, findings of this research do not allow to explore the impact on the HEIs reputation. Future research can study its impact of the HEIs reputation as a consequence of the ERE. Also, it is relevant to study the performance of the students in ERE compared to students in virtual, blended and face-to face modalities. Although students express their concerns about ERE there are no evidences of a decreasing in the learning, some gap which future research should explore. Moreover, future research can study the contextual expression of the construct of ERE. The instrument developed has identified three dimensions, nonetheless, it should be validated in different countries. Finally, this study focus on business students, however, it is unclear how ERE has been perceived by students of different careers. A comparison with other students could improve the comprehension of the ERE impact in the student satisfaction.
Conclusions
The present study makes several theoretical contributions. First, it contributes new knowledge about the situation that students faced and still face during the pandemic period. ERE is a different educational situation that cannot be compared to traditional remote teaching and requires careful analysis of its particularities. This study contributes to its understanding. A second theoretical contribution is the exploration of the perception of quality satisfaction in Latin American students. This population has been little studied possibly because the academic resources of universities were focused on maintaining active classes, leaving aside research activities. As a third contribution, we offer a measure with evidence of validity and reliability for students in ERE, thus providing a tool for further deepening the understanding of this new educational context. Finally, it offers support for measuring the construct in a multidimensional manner.
Also, this study offers practical implications for HEIs. First, findings show that education was radically transformed in the wake of the pandemic and generated a leap towards remote training without planning. This article makes evident one of the consequences of this decision for universities; students perceive a decline in educational quality. The results reported allow decision-makers in universities and governmental control posts to gain a better understanding of how the training offered during the pandemic period was experienced and how it affected the institution’s primary stakeholder.
Second, HEIs should focus the attention in the role of teachers as a determining factor in the perception of quality. In order to face this emergency situation, teachers must have sufficient resources, knowledge, and skill materials to maintain the educational process. A repeated comment encountered in our study is that both students and teachers did not have the tools and training to face remote education, which was reflected in the quality of the process.
Finally, access limitations were the major limitation of the educational offer. Differential access to technical means to continue education is the element that can enhance or hinder the perception of quality. In view of this, universities and the State must rethink solutions, especially in emerging countries such as those studied, where the social and digital divide is a structural problem.
The present study has several limitations. First, the focus groups that were used to develop the measurement instrument were only carried out in Colombia. Therefore, focus groups in all the countries included in the study could have provided additional comments to expand the contents of the instrument. Although quantitative analysis shows that the construct is similar in the countries, additional qualitative studies could offer information do not consider in this study. On the other hand, the sampling in quantitative phase was not random, which may limit the generalizability of the results. This limitation can be faced with in captive sample that enable the application of random sampling. In addition, the quantitative analyses are correlational, so it is not possible to offer explanations of causality, only of association in the results; neither are explanations offered for the differences reported between countries, given the exploratory nature of the study. New measures once face-to-face lessons return would allow to identify variations in the student satisfaction.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
