Sage Journals: Discover world-class research

Abstract

As a result of the COVID-19 virus outbreak, there has been a dramatic change in the educational landscape, and traditional classroom instruction has shifted to the online environment. Previous research has yielded contradictory results regarding the effect of different instructional modalities on student achievement. This study aimed to examine the impact of teaching mode (online vs. face-to-face) on academic outcomes in the context of Spanish as a foreign language in China. A comparative analysis was conducted between two cohorts of students, one receiving online courses and the other undergoing in-person instruction, and their respective scores for Spanish tests are reported. Additionally, a questionnaire was administered to the online cohort to gain insight into their learning motivation. The results showed that students who received online instruction outperformed those who received face-to-face instruction. Regarding the potential factors underlying the superior academic outcomes of online learners, examination of questionnaire responses pertaining to their learning experience reveals that students’ motivation, learning strategies, self-regulation, and self-efficacy are crucial factors of successful learning results and may have positively contributed to such outcomes.

Plain Language Summary

How do online and f2f learning differ in SFL students’ academic achievements and experience?

The paper examines the effect of learning modality (online vs. face-to-face) on SFL student’s academic achievements during COVID-19. Previous research indicated mixed findings on the effects of learning modality on student achievement. Few previous studies have focused on the effect of teaching mode in a Spanish as a foreign language setting. Based on naturalistic data, we compared two groups concerning academic achievement. The online group also answered a questionnaire about their attitude toward learning modes, motivation, strategies, and self-reported skills. The study is a mixed design; academic achievements were measured with two exam scores on both groups (before and after the implementation of online teaching), and the answers from the questionnaire also provide some insight into possible reasons for the difference. The main findings are: (1) The online group outperformed the face-to-face group after online learning. (2) Results of the questionnaire are discussed related to the literature. (3) Possible explanations for online learners’ success include learners’ motivation, learning strategies, self-regulation, and self-efficacy; multiple activities in the online course that can improve learners’ participation and provide them instant feedback; students, instructors, and universities’ preparedness for online teaching.

Keywords

face-to-face online learning Spanish as a foreign language (SFL)academic achievements higher education

Introduction

In the digital age, particularly with the outbreak of the COVID-19 virus, the education landscape has undergone a radical transformation and is now rife with potential and obstacles (Adedoyin & Soykan, 2023; Barrot et al., 2021). According to Pokhrel and Chhetri (2021), the pandemic caused the greatest disruption to education systems in human history, affecting approximately 1.6 billion students in over 200 nations. Molock and Parchem (2022) report that 46% of the students in their study claimed that COVID-19 impacted their daily life, which may have affected their academic performance. Conventional educational techniques are unable to address the expanding educational needs of individuals. Therefore, online education has gained prominence in the context of the pandemic.

The shift from traditional classrooms to online education raises important challenges and necessitates a more rapid and diverse transformation of technology innovation and digitalization (Donitsa-Schmidt & Ramot, 2020; Khalil et al., 2020; Varea & González-Calvo, 2021). Due to the outbreak of the pandemic in early 2020, the Chinese Ministry of Education implemented a policy of “suspending the class without stopping learning” in February (W. Zhang et al., 2020). During this period, universities in China were physically closed, and classes were shifted to remote learning. The closure of campuses and implementation of lockdown procedures compelled students enrolled in traditional face-to-face college courses to switch to online instruction abruptly.

Previous studies on language teaching in the pandemic era focused mainly on English (Erarslan, 2021; Kamal et al., 2021; Moorhouse & Kohnke, 2021; Rong & Abdullah, 2022; Sugianto & Ulfah, 2020; Yi & Jang, 2020; Yuan & Liu, 2021), few studies attempted to delve into the situation of other languages. As there are currently a large number of Chinese students learning Spanish as a second foreign language, this study aimed to enrich the empirical repository with data from a Chinese University. The purpose of this study was to investigate the effect of teaching mode (online vs. face-to-face) on academic outcomes in the context of Spanish instruction in China, as well as students’ attitudes, motivation, strategies, and self-reported proficiency.

Literature Review

Online Learning Amid the COVID-19

Though seemingly straightforward, it is necessary to reflect upon the definition of online learning. Online learning refers to a learning environment that employs high technology, such as the Internet and other devices and tools, for instructional delivery and management of academic programs (Huang, 2019; Usher & Barak, 2020). In a broader sense, online learning, also commonly referred to as e-learning, may encompass any form of non-traditional classroom-based learning. Hence, researchers have alluded to online learning as computer-assisted learning, distance education, virtual learning, and web-based learning, among others (Blake, 2011; Easton, 2003; Huang, 2017; Wheeler, 2001). In the covid and post-covid era, face-to-face instruction and online learning are the two primary modes of instruction (Huang, 2019).

The digital transformation of teaching is not a novel phenomenon since it has played a role in higher education for decades (Adedoyin & Soykan, 2023). Online instructional methods have long been acknowledged as an effective tool for learning (Aronoff et al., 2010). The increasing popularity of online education is attributable to its numerous multimedia teaching materials, accessibility, and adaptability, which provide access to content and instruction at any time and from any location (Khalil et al., 2020). It can facilitate learner-to-learner and instructor-to-learner communication via online interaction (Leszczyński et al., 2018) and support personalized and collaborative learning (Tong et al., 2012, p. 529). Further, it increases the availability of learning opportunities for students, the efficiency of knowledge dissemination, and the capacity of instructors to meet the needs of more students (Means et al., 2009).

While many advantages of online learning can be noted, some drawbacks are also pointed out by previous studies, especially amid COVID-19. In addition to technological issues (Arnett, 2020), anxiety (Cao et al., 2020; Lee et al., 2021; Odriozola-González et al., 2020; Saputri et al., 2020), lack of verbal and non-verbal communication (Al-Jarf, 2004), low motivation and efficacy (Day et al., 2021; Ng et al., 2020), lack of self-discipline, poor time management, less participation and interaction (Adarkwah, 2021; Di Pietro et al., 2020; Usher & Barak, 2018; K. Zhang & Wu, 2022) are impediments that prevent students from attaining a satisfactory academic achievement.

Anxiety is one of the most explored aspects of online learning (Cao et al., 2020; Lee et al., 2021; Odriozola-González et al., 2020; Saputri et al., 2020). Students frequently experience academic anxiety, which can be caused by various factors, such as an excessive number of assignments, a poor internet connection, rigid assignment deadlines, uncertainty regarding academic performance, worries about future careers, and other domestic responsibilities (Odriozola-González et al., 2020; Son et al., 2020; Sundarasen et al., 2020). Consequently, students with anxiety delay academic activities (Cao et al., 2020), resulting in poorer academic performance (Mooney & Bligh, 1997).

Moreover, online learning may be challenging because it requires a high level of self-discipline. In addition to the fear of academic degradation, Molock and Parchem (2022) observed that lack of concentration and difficulty adapting to virtual learning are also significant concerns associated with online learning. Also, it appears that online learning involves less interaction than face-to-face. Usher and Barak (2020) identify a lack of interaction, a sense of social isolation, low participation, and a low familiarity among professors as the primary obstacles to online education.

Online Learning and Face-to-Face Learning

Comparing Learning Outcomes Across Delivery Modes

Prior research has compared two teaching modes (online vs. face-to-face) or three, including MOOCs (massive open online course), across a variety of disciplines. Several studies suggest that online instruction may be detrimental to students’ academic achievement, as face-to-face students outperformed their online counterparts (Helms, 2014; Heppen et al., 2017; Johnson & Palmer, 2014) or learning loss is found due to online learning (Engzell et al., 2021; Maldonado & De Witte, 2022; Schult et al., 2022). Helms (2014) found online students had significantly lower grade point averages than students in a face-to-face course. Similarly, Johnson and Palmer (2014) found that students performed better and were more engaged in a face-to-face linguistics course than in an online one. Usher and Barak (2018) compared on-campus, MOOC, and online modes of instruction based on student feedback and comments. According to their findings, the first two groups performed better than the third: MOOC participants submitted more feedback comments and volunteered to evaluate more projects than their peers, and on-campus students provided higher-quality feedback.

Students also perceived a decline in their academic performance caused by online instruction. For example, most students believe their English did not improve during online classes (Klimova, 2021). According to Mooney and Bligh (1997), students performed worse in online courses regarding course persistence and final grades. In addition, principals and teachers reported their impressions of students’ performance during the online course and concurred that students did not make significant progress (Meinck et al., 2022).

In contrast, some studies have reported that online instruction improves learning outcomes (Bourelle et al., 2016; Clark et al., 2021; T. Gonzalez et al., 2020; Kamal et al., 2021; Means et al., 2009; Spitzer & Musslick, 2021). A comprehensive meta-analysis of studies comparing online learning to traditional face-to-face instruction found that students who took all or part of their classes online outperformed those who completed the same course through traditional face-to-face instruction (Means et al., 2009). In the context of a pandemic, T. Gonzalez et al. (2020) discovered that students’ confinement positively impacted their academic performance. Similarly, Meeter (2021) found that students’ performance improved after the lockdown. Kamal et al. (2021) interviewed students to identify the possible factors contributing to the effectiveness of online teaching. The survey revealed that these factors include more leisure time, more frequent breaks, a more comfortable learning environment, and less time spent traveling to and from the university.

Several studies, however, demonstrate that these two teaching methods yield comparable results (Horspool & Lange, 2012; Jensen, 2011; Kemp & Grieve, 2014; Montiel Chamorro, 2018; Ni, 2013; Waldman et al., 2009; Wang et al., 2019; Yen et al., 2018). For instance, Montiel Chamorro (2018) compared the English language learning outcome of students taught in these two settings. The results suggest that online courses can be as good as or better than face-to-face courses, as they offer greater time flexibility, the possibility to revisit content as needed, and the ability to be completed virtually anywhere.

Students’ Perceptions and Preference

While some students view online and face-to-face classes as equally effective and of comparable quality (Horspool & Lange, 2012; Wong, 2020), not all students find online courses satisfying or their preferred mode of instruction.

Numerous studies indicated a stronger preference for the traditional classroom setting (Adarkwah, 2021; Bali & Liu, 2018; K. Singh et al., 2020; Usher & Barak, 2020), particularly among language learners (Alhamami, 2018). Face-to-face learning can enhance learners’ presence and interaction and provide an environment conducive to communication. Even though Kemp and Grieve (2014) discovered a similar test performance in the two modes, students still preferred face-to-face study. Alhamami (2018) investigated the intention of language learners to study a foreign language in both face-to-face and online settings. He discovered that students prefer face-to-face classes to online engagement. Further analysis related this preference to learners’ attitudes, the people around them, and their beliefs about their foreign language learning abilities. In the same vein, Bali and Liu (2018) revealed that face-to-face learning was perceived as better than online learning due to the social presence, interaction, and satisfaction it offers. Half of the business undergraduates with experience in multiple modalities (face-to-face, online, and hybrid) favored face-to-face classes (Blau et al., 2017).

Learners also exhibit an unfavorable attitude toward online education. In a survey conducted by Chaturvedi et al. (2020), 51.6% of students between the ages of 18 and 22 rated online classes negatively. Similarly, Tang et al. (2023) showed that students were generally dissatisfied with online learning, particularly in communication and question-and-answer interaction.

By contrast, some studies demonstrate a positive attitude toward online modes (Khalil et al., 2020; Khanal, 2021; Klimova, 2021). For example, Khalil et al. (2020) found that most medical students are satisfied with online synchronized classes and deem them more time-saving. Larson and Sung (2019) demonstrated no significant differences among different learning modes. Still, a higher rate of student and faculty satisfaction and learning effectiveness was found in blended and online modalities.

In sum, although there is a large body of research on the effectiveness of instructional modes, both on learners’ academic performance and student satisfaction, extant research yields mixed results. Moreover, no previous study has focused on the teaching of a second foreign language in higher education. Hence, the current study aims to address these research questions:

RQ1: How do online and face-to-face teaching modes affect the learners’ academic achievements in the context of Spanish learning?

RQ2: What do students experience while engaging with the online mode regarding attitudes, motivation, strategies, and self-perception?

Research Design

Research Setting and Participants

This study was conducted at a public university in southern China from 2020 to 2021. In response to COVID-19, the university where the research took place temporarily switched to online teaching in 2020, coinciding with the first COVID-19 wave. Naturalistic data were collected during the spring semester of 2020 (online instruction) and 2021 (face-to-face instruction).

To assess the difference between learning modes and students’ academic achievements, the present research follows Mooney and Bligh (1997) by comparing a group of students’ performance in an online course with a similar set of students in a face-to-face course.

The study comprised 46 students majoring in English Philology or Chinese-English translation. All learners were native speakers of Mandarin Chinese and shared similar ages. Due to the same enrollment criteria at the same university, the students shared similar English proficiency levels. Spanish classes include weekly sessions of 3 hr. Spanish as a second foreign language is an elective held for two semesters each academic year.

The course was delivered in a traditional face-to-face mode for the first semester. The 2020 cohort (online group, N = 24) took online courses during the second semester, whereas the 2021 cohort (face-to-face group, N = 22) continued to receive face-to-face instruction. The same lecturer instructed both groups. Detailed mode and gender distribution can be found in Figure 1.

Figure 1.

Demographic information of participants.

The study is part of students’ course evaluation, and they have not received compensation. Additionally, the online group was asked to complete a questionnaire voluntarily. All participants were anonymous, over 18, and they consented to participate by accepting the invitation and completing the survey (two students did not complete the online survey and thus were excluded). Hence, only 22 learners of the online group filled out the questionnaire.

Research Procedure

Data Collection

Spanish as a second foreign language course is a two-semester elective. Each semester is approximately 16 weeks long, and each week consists of two 90-min sessions.

The first semester was in a classroom setting for both groups. At the end of the first semester, both groups were given a test of comparable difficulty (SFL1).

The course is offered in two formats during the second semester: a traditional on-campus face-to-face (F2F) course and an online course. The F2F course is standard classroom instruction (cohort 2021). For one term, cohort 2020 engaged in asynchronous online learning. At the end of the second semester, both groups were given another exam similar to SFL1 (SFL2). Both online and offline classes were conducted twice a week with each session lasting one and a half hours.

The instructional material utilized in both courses consisted of Español moderno (Modern Spanish), a Spanish language textbook specifically designed for Chinese students. This textbook offers a comprehensive approach that integrates text, grammar, vocabulary, and listening exercises. Students of both groups were expected to complete a series of exercises in-class as well as outside of class, with no specified time limit for their completion.

At the end of the second semester, a questionnaire regarding the online learning experience was prepared and distributed to the online group in Chinese. It primarily focused on students’ attitudes toward online learning, students’ learning motivation, students’ learning strategies, and self-perception of academic achievement.

Online Instruction

Online courses were delivered asynchronously using Chaoxing, a multimedia-integrated online platform that serves as a repository for students’ learning materials and a distribution platform for announcements, assignments, and assessments. It has multiple functions and advantages because it is specifically designed for online teaching and learning: materials can be uploaded to the platform using an intuitive and integrated interface, and students can access various resources. It permits diverse assignment formats, including cloze, multiple-choice, listening, reading, and writing. It provides a variety of online activities, including discussions, polls, voting, and assignments. In addition, it can monitor students’ learning activities, such as the number of times they have accessed the course, the number of tasks they have completed, and their scores for each module and activity.

The course contains approximately 132 videos totaling 1,138 min (recorded mainly by the instructor), 42 audio files, 47 documents, 99 additional resources, and 339 activities. Additionally, there are discussion forums where students can pose questions and exchange ideas. In online courses, each unit has an instructional section (explanatory videos and audios), workbook exercises, quizzes, and tests. Typically, students begin by exploring each session’s learning objectives before moving on to the instructional section, where vocabulary, grammar, and texts are introduced and explained in videos. Students were then provided with additional exercises and activities for practice. To engage students in online participation, attempts were made to build rapport with students. Students are encouraged to participate in and respond to the online discussion forum. Using a WeChat group chat, synchronous discussion sessions were created. Students also had the option to interact with teachers via Tencent Meeting for tutoring. (See Supplemental Material for additional description on the design and implementation of the online course).

Questionnaire

The questionnaire is part of a larger project. In this study, the following 16 items are reported: six multiple-choice questions, four three-point Likert Scale questions, five rating questions, and one open-ended question. These items pertained to four major dimensions: (1) Attitudes toward online learning. (2) Learning motivation. (3) Learning strategies. (4) Self-reported proficiency. For some multiple-choice questions, students were also given the option to note their responses and any pertinent comments in an item titled Others.

Data Analysis

Measures

SFL1

Exam results obtained for the examination administered after the first semester. The tests are worth 100 points and do not evaluate listening or speaking.

SFL2

Exam results obtained for the examination administered after the second semester. This exam is similar to SFL1.

Completion of Tasks

The online course contains 339 diverse assignments, of which only 306 are mandatory. This platform keeps track of how many assignments each student has completed. To determine the Completion of tasks, the number of completed tasks is divided by the total number of tasks, and multiply the quotient by 100 to obtain a score out of 100. If a student has completed 153 tasks, for instance, his or her score for this item is 153/306 × 100 = 50.

Time Spent Watching Videos

Time each student spends watching the instruction videos (in min).

Number of Visits

The total number of visits by each student during the course.

Exercises and Assignments

The item computes the average assignment and exercise score for each student. The total score is 100 points.

Statistical Analysis

This statistical analysis for this study was performed in RStudio, Jamovi (The jamovi project, 2022) and SPSSAU (The SPSSAU project, 2022).

For RQ1, the t-test and the correlation test are employed. For related and unrelated samples, t-tests were used to calculate between- and within-group differences before and after the implementation of online instruction to test the learning mode’s effect. Correlation tests were conducted to determine the relationship between variables reflecting students’ learning performance and their academic achievement at the end of the second semester.

The survey analysis addressed the RQ2. Percentages and values are presented for single-select multiple-choice questions. Multi-select multiple-choice questions were evaluated using the Chi-square goodness-of-fit test, and response rate was used to identify items that differ significantly from others. The open-ended question was analyzed qualitatively utilizing text mining techniques. The relationship between students’ self-reported Spanish proficiency scores and academic achievements was analyzed using correlation tests.

Results

This section includes two parts; each provides data for one of the research questions.

Results of RQ 1: Effects of Teaching Mode on Academic Achievements

Between-Groups Comparison

Table 1 presents the Mean and Standard deviation of SFL1 and SFL2 per group. A Welch two sample t-test was conducted to explore differences between two groups in terms of SFL1, and no significant statistical difference was found: t(33.78) = −0.33, p = .74, Cohen’s d = −0.11, 95% CI [−0.67, 0.48].

Table 1.

SFL1 and SFL2 of Both Groups.

	SLF1		SFL2
	Mean	SD	Mean	SD
Face-to-face	85.24	10.81	77.80	15.83
Online	86.13	6.48	87.17	4.88

It was nonetheless interesting to note that even though both groups shared similar scores at the end of the first semester (SFL1), the situation changed in the second semester. At this point, the face-to-face group decreased in their average score. A Welch two sample t-test comparing performance on scores across the two instructions types (in-class vs. online) showed a significant difference in SFL2: t(24.64) = −2.66, p = .013, Cohen’s d = −1.07, 95% CI [−1.91, −0.22]. The online group reported higher scores than the face-to-face group for SFL2.

Within-Group Comparison

Paired sample t-tests were used to compare each group’s SFL1 and SFL2, and the results are presented in Table 2. Regarding inner group development, there was no significant difference between SFL1 and SFL2 for the online group. However, significant declines in SFL2 scores were observed for the classroom-based group. In other words, the on-campus group performed worse after the second semester of classroom instruction.

Table 2.

Paired Sample t-test Results for SFL1 and SFL2 in Each Group. The Results Below the Significance Level of .01 are in Bold and Marked With Asterisks.

Group	Mean difference	SD	t	p	95%	Cohen’s d
Online	1.04	4.99	1.023	.317	−1.065 to 3.149	0.209
Offline	−7.45	9.03	−3.867	.001**	−11.449 to −3.442	0.825

Correlation Analysis

In order to determine whether there is a correlation between students’ online performance and academic achievements (online group), a correlation test was performed. Table 3 displays descriptive statistics for variables considered in RQ1.

Table 3.

Descriptive Statistics for Major Variables in RQ1 (Online Group).

Variables	M	SD	Variance	Median	Std. error	95% Confidence interval for M
Variables	M	SD	Variance	Median	Std. error	Lower bound	Upper bound
SFL1	86.13	6.48	42.01	86.55	1.32	83.53	88.72
Completion of tasks	98.12	2.26	5.11	99.02	0.46	97.22	99.03
Time spent watching videos	1,634.58	466.18	217,320.78	1,483.00	95.16	1,448.08	1,821.09
Number of visits	583.46	160.52	25,766.69	557.00	32.77	519.24	647.68
Exercises and assignments	89.84	4.05	16.43	90.20	0.83	88.22	91.47
SFL2	87.17	4.88	23.80	86.50	1.00	85.22	89.12

Note. This table displays descriptive statistics for each variable. SFL1, Completion of tasks, Exercises and assignments, and SFL2 are scores calculated out of 100. Time spent watching videos is calculated in minutes. Number of visits calculates how many times the students have accessed the course. The statistics estimates are the mean, standard deviation, variance, median, standard error, and 95% CI for the mean value.

Spearman correlation coefficient and Pearson correlation coefficient were used to evaluate the relationship between the variables. A summary of statistical findings can be found in Table 4.

Table 4.

Pearson Correlation Coefficient (r), Spearman Rank Correlation Coefficient (ρ), and p-Value for Variables Related to SFL2 (In bold, Correlation Coefficients at p < .05).

Correlation test	Variables	Coefficient	p-Value
Pearson	SFL1	.647 **	.000
Pearson	Exercises and assignments	.369	.076
Spearman	Completion of tasks	.469 *	.021
	Time spent watching videos	−.324	.123
	Number of visits	−.101	.638

Note. Based on whether the assumptions for the Pearson correlation test were met, either Pearson correlation test or Spearman rank correlation test was selected.

Correlation is significant at the .05 level (two-tailed). **Correlation is significant at the .01 level (two-tailed).

A strong and positive correlation exists between SFL1 and SFL2, according to the data presented in Table 4. Thus, students who earned higher grades before implementing online instruction tend to earn higher grades for the online course. In addition, SFL2 scores positively correlate with the online course tasks they complete. Therefore, students who complete more online assignments and exercises tend to receive higher grades. It appears that online exercises improve students’ academic performance. Other variables, including Time spent watching videos, Number of visits, and average score obtained for online Exercises and assignments, have no correlation with SFL2 scores.

Results of RQ2: Questionnaire Analysis

The questionnaire consisted primarily of four sections containing multiple questions: attitudes toward online learning, motivation, learning strategies, and self-reported proficiency. In each section, the questions are specified in detail.

Attitudes Toward Online Learning

This section included two single-select multiple-choice questions and two multi-select multiple-choice questions. As shown in Figure 2, while 36% of students reported spending about as much during the online learning as they did before the transition, more than half of the students (59%) indicated spending more time on learning during the online course, and a small portion of students (5%) reported spending less time.

Figure 2.

Response to the survey question “Do you spend more or less time learning Spanish now?”

In terms of which teaching method students prefer, as shown in the pie chart below (Figure 3), 9% prefer online-only instruction. Most students prefer the traditional mode (59%), while approximately 32% prefer the blended teaching.

Figure 3.

Response to the survey question “Which learning mode do you prefer?”

The response and popularity rates for the two multi-select multiple-choice questions are summarized in Table 5. The proportional distribution of responses to each multiple-choice question is analyzed using the chi-square goodness-of-fit test. According to the table below, the goodness-of-fit test for Q1 was significant (χ² = 25.538, p = .010), indicating that the proportion of choices for each item is significantly different. The response and popularity rates for “Save time” and “More resources and materials” were notably higher. Other responses include “I can take screenshots and save them whenever I want” and “I can repeatedly watch the video.”

Table 5.

Summary of Responses Rate and Popularity Rate of Different Questions in the Questionnaire: Q1. After a Semester of Study, What are the Benefits of Learning Spanish Online? Q2. What is the Most Negatively Affected Aspect of Learning Spanish Online?

Question	Categories	Response		Popularity rate (n = 22) (%)
Question	Categories	n	Response rate (%)	Popularity rate (n = 22) (%)
Q1: After a semester of study, what are the benefits of learning Spanish online?	Save time	20	38.46	90.91
	More communication with teachers and classmates	5	9.62	22.73
	More resources and materials	13	25.00	59.09
	Diverse learning activities	7	13.46	31.82
	The platform is feature-rich and fun to use	5	9.62	22.73
	Others	2	3.85	9.09
	Goodness of fit: χ² = 25.538, p = .010
Q2: What is the most negatively affected aspect of learning Spanish online?	Grammar	8	26.67	36.36
	Vocabulary	3	10.00	13.64
	Speaking	9	30.00	40.91
	Reading	5	16.67	22.73
	Listening	5	16.67	22.73
	Goodness of fit: χ² = 4.000, p = .406

Regarding the second question, the statistical analysis did not reveal a significant difference between the learners’ responses (χ² = 4.000, p = .406). However, Speaking appears to be the most popular response, followed by Grammar.

Learning Motivation

Two multiple-select multiple-choice questions were included to assess students’ learning motivation. Answers are summarized in Table 6.

Table 6.

Summary of Responses Rate and Popularity Rate of Different Questions in the Questionnaire: Q1. What is Your Purpose in Taking This Course? Q2. How Can Spanish Help You in the Future?

Question	Categories	Response		Popularity rate (n = 22) (%)
Question	Categories	n	Response rate (%)	Popularity rate (n = 22) (%)
Q1: What is your purpose in taking this course?	I like this language.	13	43.33	59.09
	It is mandatory for my BA program.	7	23.33	31.82
	I like Spain.	5	16.67	22.73
	It’s easier to get a job if I learn Spanish.	1	3.33	4.55
	Others	4	13.33	18.18
	Total	30	100	136.36
	Goodness of fit: χ² = 13.333, p = .010
Q2: How can Spanish help you in the future?	Communicate with native speakers of Spanish	10	25.00	45.45
	Understand Spanish songs, ball games, movies, etc.	15	37.50	68.18
	Read Spanish books, newspapers, etc.	12	30.00	54.55
	Others	3	7.50	13.64
	Total	40	100	181.82

Concerning Q1, from the above table, the goodness-of-fit test was significant (χ² = 13.333, p = .010), indicating that the proportion of choices for each item differed significantly. Specifically, the item “I like this language” had a significantly higher response and popularity rate for 13 items. Students who selected Others also expressed their preference for Spanish music, such as “I like Enrique Iglesias” or “I like Gabriel Iglesias.” This item, along with “I like Spain,” demonstrates the learners’ intrinsic interest in Spanish learning, such as their affection for Spain, Spanish, and Spanish culture. The second item, “It is mandatory for my BA program,” indicates the student’s academic concern, as some students were enrolled in this course to satisfy an academic requirement. In addition, students exhibit a high degree of pragmatic or utilitarian motivation, that is, they believe that learning Spanish will enable them to obtain a better job.

Regarding the second question, no differences between items were statistically significant (χ² = 7.800, p = .050). About 37.5% of students expect to use Spanish to engage in cultural activities, such as listening to music, watching football games, and watching films. About 30% of the students desire the ability to read in Spanish. 25% hope to communicate with native speakers. Interestingly, two students stated in Others that they would like to use Spanish when they travel to a Spanish-speaking country.

Learning Strategies

In order to identify patterns in students’ learning strategies, the questionnaire included four three-point Likert scale questions and one open-ended question. Figure 4 displays the responses of students to the first four questions.

Figure 4.

Bar chart of the three-point Likert scale asking for the students’ learning strategies during the online course, with the answers ranging from “disagree” to “agree” with the according sum of percentages of positive or negative responses from the four questions.

The primary objective of the first two questions is to determine whether students received additional Spanish input while undertaking their online courses. Most students reported listening to Spanish songs and watching Spanish films or television shows infrequently or not at all, and 5% of students received additional Spanish input. Since previous research suggests that online instruction may be detrimental to students’ speaking and listening skills, the online courses require students to complete a set of listening and reading assignments. Most students have exhibited a favorable or neutral attitude toward these two types of assignments.

The open-ended question “How do you learn Spanish?” was designed to give students a voice. Regarding this open-ended question, learners’ answers were analyzed using R packages dedicated to text mining (Pedersen, 2021). The answers were tokenized into words, stop words were excluded, and bigrams were extracted. Figure 5 presents the most common bigrams extracted from learners’ responses.

Figure 5.

Bigram network plot of words that appear in students’ answers to the open-ended question “How do you learn Spanish now?” Stop words were excluded. The more times two words appear together, the darker the color.

Figure 5 depicts a network of words derived from the responses of students. Also included are intricate relationships and connections between words. As shown in Figure 4, some words form a large node from which other words branch out. The bigram network emphasizes important terms such as “Spanish,”“review,”“online,”“practice,” and “grammar,” which form node centers with shared meaning. In addition, “review textbook,”“verb conjugation,”“online video,”“memorize words,” and “soccer games” are the bigrams that have appeared more than once.

It seems as though students are primarily concerned with vocabulary and grammar. Bigrams associated with vocabulary include “new vocabulary,”“online dictionary,”“verb conjugation,” and “sentence example.” Additionally, students indicated that “grammar is important” and will purchase additional grammar books to improve their skills.

In addition, a number of traditional strategies are identified in this study, including frequent textbook review, assignment review, exercise repetition, word memorization, and text recitation. These instructional techniques seem to be resembling rote repetition, such as extensive reviewing, repetition, and memorization.

Self-Reported Proficiency

The students’ assessments of their Spanish proficiency were gathered using a questionnaire that measured reading, grammar, vocabulary, listening, and speaking skills on a scale ranging from 1 to 10. The internal consistency of the questionnaire items was deemed high (Cronbach’α = .948) based on the test of reliability statistics (Table 7).

Table 7.

Reliability Statistic Cronbach’s Alpha.

Items	Corrected item-total correlation (CITC)	Cronbach alpha if item deleted	Cronbach’α
Reading	0.891	.929	.948
Grammar	0.929	.926
Vocabulary	0.933	.922
Listening	0.759	.953
Speaking	0.801	.947
Cronbach α (Standardized): .951

The self-reported Spanish proficiency in five skills is depicted in Figure 6. According to the results, students have the highest self-reported scores in reading, grammar, and vocabulary. On the other hand, students were least satisfied with their listening and speaking abilities.

Figure 6.

Boxplot of self-assessment (y-axis) by skills (x-axis). The self-assessment score was on a scale of 1 (worst) to 10 (best).

A correlation test was conducted between learners’ self-reported scores and the SFL2 score, and no correlation between the SFL2 score and the self-reported score for any component. Results are summarized in Table 8. Note that no correlation between skills and final academic achievement has been found. A strong correlation was found between all self-reported skills.

Table 8.

Pearson Correlations Between SFL2 and Five Skills.

Variables	Mean	Std. deviation	SFL2	Reading	Grammar	Vocabulary	Listening	Speaking
SFL2	87.36	5.02	1
Reading	6.68	2.01	0.07	1
Grammar	6.46	1.74	0.13	0.89**	1
Vocabulary	6.27	1.96	0.07	0.88**	0.90**	1
Listening	5.05	2.06	−0.08	0.72**	0.77**	0.79**	1
Speaking	4.91	2.18	0.06	0.78**	0.82**	0.81**	0.60**	1

Correlation is significant at the .05 level (two-tailed). **Correlation is significant at the .01 level (two-tailed).

Discussion

Discussion of the Effect of Instruction Mode

This study aimed to explore differences in learning outcomes between classroom-based and online instruction in the context of Spanish teaching. The results showed a significant relationship between learning mode and academic achievement: online students outperformed their counterparts who received instruction in a traditional classroom setting. The finding corroborates previous studies which report a positive effect of online instruction on learning outcomes (Bourelle et al., 2016; Clark et al., 2021; T. Gonzalez et al., 2020; Means et al., 2009; Spitzer & Musslick, 2021) and contradicts studies which reported a better academic achievement in traditional courses.

It seems that students’ motivation, learning strategies, self-regulation, and self-efficacy are crucial for achieving a successful learning outcome. In addition, the course design, which includes implementing various activities, facilitates interaction and provides more immediate feedback. Moreover, students, instructors, and universities seem to be prepared for a smooth transition from in-person to online instruction.

Students’ Motivation, Self-Regulation, and Self-Efficacy

Previous studies have reported motivation, self-control, and self-efficacy as the main predictors of academic success (Klimova, 2021; Montiel Chamorro, 2018). According to Montiel Chamorro (2018), students who adhere to the pacing instructions and complete all tasks and assignments are more likely to advance in their language learning process.

Motivation is a contributing factor to the learning outcome. The learning of intrinsically motivated students is relatively unaffected by the class delivery mode, as they require minimal supervision and guidance, whereas learning-challenged students encounter difficulties (Pokhrel & Chhetri, 2021). Also, older students are more self-motivated, dependent, and well-organized, more likely to understand what they want to learn, take control over learning, and have skills that allow them to exploit the benefits of online learning (Di Pietro et al., 2020). Research showing an academic loss owing to teaching mode is carried out within elementary or secondary schools (Engzell et al., 2021; Maldonado & De Witte, 2022; Schult et al., 2022). The focus of this study is on a cohort of adult learners who have enrolled in the course to fulfill graduation requirements. Therefore, the sample of learners in this study may exhibit greater motivation and self-discipline compared to those examined in previous research.

Compared to traditional classroom contexts, the online learning environment is more autonomous (Dabbagh & Kitsantas, 2004) and significantly benefits independent students (Di Pietro et al., 2020); therefore, it necessitates that students possess greater maturity, self-discipline (D. Zhang et al., 2004), and self-regulation. Self-regulation refers to the ability to use self-managing behaviors and implement learning processes that, when coupled with motivation, enable students to independently put their self-confidence beliefs into action. Armstrong et al. (2022) concur that self-regulation is crucial to academic success in online learning, and they proposed effort regulation, that is, students’ ability to concentrate on academic tasks despite obstacles or distractions, as a moderator that can attenuate the impact of a change in course modality. Online education may not adversely affect students who can self-regulate their learning, given that students with higher self-regulation tend to have greater learning confidence, time management skills, academic commitment, and utilization of learning strategies. Even though course modality diminished learning opportunities and produced a decline in learning outcomes, students higher in effort regulation were not affected as much by the rapid change in course modality.

The self-efficacy of students may also be relevant. Spence and Usher (2007) found that although course setting was a significant predictor of student achievement, it no longer contributed once self-efficacy was controlled. Chang et al. (2014) discovered that students with high internet self-efficacy performed better on the final exam and were more confident in completing an online course than those with low internet self-efficacy.

Course Design, Interaction, and Feedback

Interaction and feedback are essential components of a successful online course. Change in interaction patterns is one of the most significant obstacles Chinese EFL (English as a Foreign Language) students face in online courses (K. Zhang & Wu, 2022). Similarly, Gaytan (2015) found that two factors favoring faculty and students’ perceptions of online education are more teacher instructions and more immediate and thorough feedback. According to student feedback, the majority of teachers were perceived to allocate minimal teaching time and assign little homework. Thus, students expressed a strong desire for online instructors to dedicate more effort to the actual delivery of online instruction.

While learning any course material online is feasible, the learning outcomes for courses requiring more direct contact and interactions will be less desirable (Franchi, 2020), such as anatomy. In contrast to courses where face-to-face communication and direct interaction are crucial, such as medical education (Dedeilia et al., 2020; Franchi, 2020; Iwanaga et al., 2021), geography (Day et al., 2021), or physical education (García-Tascón et al., 2020; González-Calvo et al., 2022; Varea et al., 2022; Varea & González-Calvo, 2021), in language-related courses, interaction and communication might be achieved by other means.

Online learning can be classified as either synchronous or asynchronous. Synchronous online learning involves real-time interactions between the teacher and students, whereas asynchronous online learning does not adhere to a rigid schedule for different students (V. Singh & Thurman, 2019). Synchronous technology enables instant and direct communication (Yao et al., 2020), whereas asynchronous technology may result in delays between instruction and its reception. Previous studies show synchronous learning seems more effective than asynchronous learning, especially in language teaching (Kohnke & Moorhouse, 2022).

The instruction approach implemented in this study involved asynchronous teaching. The observed improvement in student academic performance may raise inquiries regarding the underlying factors. It is posited that despite the asynchronous nature of the instruction method, the varied assignments, regular question-and-answer sessions, and forum posts engage students and prompt direct and immediate interaction between students and the instructor as well as among students. The evidence suggests that despite the online setting for Spanish language learning, the abundance of online activities has a positive impact on learning outcomes. Similarly, online instruction permits instructors to engage students in a variety of ways (Wong, 2020), and both the quality and quantity of the interaction can be observed in online classes (Ni, 2013). Notably, the present study indicates that some students reported increased interaction with peers and instructors during online classes.

Furthermore, more individual exercises were assigned to the online cohort. Due to the time constraints of face-to-face instruction, students have fewer opportunities to practice in class. In addition, the online course appears not only to entail a final assessment but also includes formative assessments. Even though classroom-based courses also assign homework and tasks for students to complete, online teaching inevitably requires students to engage in more independent exercises. In addition, students receive feedback faster in online classes than in traditional classrooms, a phenomenon that is also observed in the present study and holds the potential to enhance students’ learning outcomes.

Students, Teachers, and Universities’ Preparedness

Student and teachers’ readiness may also be relevant for online courses. Previous research has also highlighted the need for teachers to acquire more training and digital skills (X. Jin, 2022; Sánchez-Cruzado et al., 2021; J. Zhang & Yu, 2023). Even though Westwick and Morreale (2021) noted in their study that universities might not be prepared for the rapid transition to remote teaching, the COVID-19 outbreak occurred unexpectedly, and the classroom-to-online transition was abrupt, Chinese teachers may be well-prepared. Based on the research conducted by Gao and Zhang (2020), English as a Foreign Language educators in China demonstrated comprehensive knowledge regarding the distinctive features, benefits, and limitations of online instruction. Teachers acquired information and communication technology literacy by understanding students’ learning needs, online teaching practice, and the importance of integrating traditional classroom and online teaching methods.

In online courses, students heavily rely on instructors’ intervention (Usher & Barak, 2020), and instructors play a managerial role (Huang, 2019). Before each session, students in this study received clear instruction regarding learning objectives and assignments and were required to comply. Thus, explicit instruction can compensate for some disadvantages of online instruction, such as a lack of self-discipline and commitment.

Teachers’ and students’ preparedness is also made possible by the university and department’s support. Numerous training sessions and classes were designed to prepare the teacher for online teaching. Therefore, online instructors compensate for the lack of physical presence by establishing a virtual environment where all students feel at ease, and teachers can be easily contacted to optimize the learning outcome.

Discussion of Learners’ Attitude, Motivation, Strategies, and Self-Reported Proficiency

Our research also investigates learners’ views regarding online learning, as well as their learning motivation, strategies, and self-reported perceptions of various abilities.

Attitudes Toward Online Learning

Even though students valued the convenience of online instruction, they indicated a preference for face-to-face or blended mode in our study, which is consistent with previous research (Blau et al., 2017; Jensen, 2011). According to the present research, the benefits of online learning include time-saving, flexibility, and material accessibility, as shown in previous studies in the literature (Wong, 2020). Given its flexibility in terms of time arrangement and convenience, online learning enables students to revisit content as needed and complete coursework from everywhere (Montiel Chamorro, 2018).

Among the disadvantages of the online mode is a lack of concentration, which overlaps with similar studies (Jensen, 2011; Klimova, 2021). Research also reported less or ineffective interaction (Yang & Lin, 2020), a lack of connectedness and engagement (Bowers & Kumar, 2015), self-discipline and organizational skills (Kokko et al., 2015), and technological concerns (Dyrbye et al., 2009).

Therefore, scholars also proposed blended learning as an alternative combining the benefits of face-to-face and online learning. It can improve students’ concentration, academic performance, motivation to learn, and provide flexibility and suitability (Blau et al., 2017).

The concern regarding students’ online performance and academic achievements is also evident in this study. However, only a correlation between the number of tasks completed and SFL2 was found. This is partially consistent with the findings of Means et al. (2009), who discovered that studies in which online learners spent more time than classroom-based learners found online learning to be more beneficial. Even though Time spent watching tutorials is not correlated with SFL2 in this study, this can be explained by the fact that this variable only accounts for the time students spent watching instructional videos and does not take into account the time they spent reading online materials or completing assignments. Time invested in education is a crucial learning experience. The total duration of all videos in the online course is 1,138 min, and the average time spent by students was 1,634.58 min. Thus, it is evident that the online group took considerable time to complete the course. Di Pietro et al. (2020) cited less time spent learning as one of the four adverse effects of transitioning from traditional school to distance education. On the contrary, the results of this study suggest that students seem to allocate a greater amount of time to learning.

Learning Motivation

Concerning learners’ motivation, notable is the fact that the motivations for learning Spanish included both extrinsic and intrinsic interests, as well as integrative and instrumental motivation. Within the study conducted, most students expressed a keen interest in Spain, the Spanish language, and the associated cultural aspects such as music and soccer games. Additionally, some students also expressed their academic concerns, as their enrollment in the course was driven by the necessity to satisfy an academic requirement. This is comparable to the findings of Li (2014), who found that Chinese learners of English have a high preventional instrumentality. According to the present study, this may not be the determining factor in enrolling in the Spanish course; this is consistent with Liu and Oga-Baldwin (2022). In China, college students majoring in English are required to take a second foreign language as one of the graduation requirements, and universities typically offer French, German, Japanese, Russian, and Spanish as an option. Liu and Oga-Baldwin (2022) found that making a second foreign language course mandatory in China does not necessarily motivate students.

In addition, students demonstrate a high degree of pragmatic or utilitarian motivation, that is, they believe that learning Spanish will enable them to obtain a better job and they can communicate with locals in Spanish-speaking countries.

Learning Strategies

Students are most concerned with vocabulary, grammar, and texts and tend to practice these skills with great vigor. This is consistent with L. Jin and Cortazzi (2006) asserting that Chinese EFL learners have a tendency to memorize vocabulary, understand grammar rules, and recite texts. Therefore, this may be a common occurrence not only among Chinese EFL or SFL learners but also among Chinese foreign language learners in general.

In addition, it has been observed that students seem to dwell on memorization and repetition when learning Spanish. According to some scholars, imitation and memorization are widely employed by EFL students in China and are regarded as effective (Ding, 2007; L. Jin & Cortazzi, 2006). For instance, Ding (2007) found that successful EFL students attributed their English-learning success to the practice of text memorization and imitation. Similarly, within the context of the study, it was found that students acknowledged the utilization of repetition-based memorization methodologies in learning Spanish. These techniques encompassed activities such as the memorization of vocabulary and texts.

Nonetheless, despite the Chinese tradition of memorization, research has shown that Chinese learners make little use of mnemonic devices to learn English (Foong & Goh, 1997; Gu & Johnson, 1996). In addition, academics have observed that the Chinese students’ habit of repetition and memorization is not anchored in their monotonous mechanism to know but rather in their effort to comprehend.

According to L. Jin and Cortazzi (2006), memorization through repeated practice is a means of comprehension for Chinese language learners, who internalize the input and acquire comprehension through reflective practice. In addition, Watkins (2000) notes that excellent Chinese students do not view memorization and comprehension as separate processes; instead, they believe both processes must work in tandem to provide a high-quality learning outcome. Western students use repetition to determine whether they have retained information, whereas Chinese students utilize repetition for two distinct purposes: to develop a deep impression, to aid in memorization, and to increase comprehension by discovering new meanings.

Moreover, students also report taking other extracurricular activities such as listening to Spanish music and watching Spanish films or television shows, which may help students learn Spanish in addition to their regular classes. T. Gonzalez et al. (2020) also attribute the positive effect of online teaching to students’ continuous use of learning strategies, which, in turn, improves learning efficiency.

Self-Reported Proficiency

Regarding the students’ self-reported competence, learners in our study are most satisfied with their reading, grammar, and vocabulary skills. The research on their learning strategies demonstrates that they tend to place emphasis on these areas. Classroom experience appears to have shaped the belief of some students that grammar is the most critical component of learning a foreign language (Sawir, 2002). Reading is the skill in which students exhibit the most self-confidence. The positive linguistic transfer from L2 (second language) English to L3 (third language) Spanish is a plausible contributing reason. In addition to sharing a comparable learning experience, L2 English and L3 Spanish possess a similar vocabulary. Thus, a facilitative transfer from L2 to L3 may contribute to learners’ attainment in reading activities.

The perception of difficulty was greatest for listening and speaking, consistent with previous research. Listening (Nushi & Orouji, 2020; Walker, 2014; Xu, 2011) and speaking (Tavil, 2010; Ur, 1996, p. 120) have been demonstrated to be the two most important and challenging abilities to teach and master. However, in this study, even though students perceive listening and reading as difficult, some learners actively engage in extracurricular activities such as listening to music and watching movies to improve their listening and speaking skills. These strategies, along with participating in conversations and chatting with native speakers, were the most frequently recommended extracurricular activities to boost listening and speaking skills (P. F. Gonzalez et al., 2015).

Conclusion

This paper addresses two important issues: (1) Which mode of instruction, online or in-class, produces better learning outcomes in Spanish classes? (2) What are the attitudes, motivation, learning strategies, and self-perception of the students enrolled in the online learning course?

Online instruction produced superior academic outcomes than traditional classroom instruction, and academic achievements positively correlate with task completion. The majority of students prefer classroom-based instruction, according to the data. In this study, intrinsic and extrinsic motivations are observed, with integrative motivation more prevalent than instrumental motivation. Students use traditional methods such as repetition and memorization to learn Spanish but participate in culturally-centered extracurricular activities. Reading received the highest self-rating, while listening and speaking received the lowest.

In light of the pandemic, which has had a substantial impact on students’ lives and studies, this study’s findings also offer pedagogical insights. The use of various methods to prompt interaction and feedback, as well as proper guidance and explicit instruction from the instructor, can aid students in achieving a successful learning outcome.

This research has limitations that must be addressed. First, a relatively small sample size is used. Nonetheless, this provides insight into the complexity of language acquisition and the interplay between instruction modes and learners’ academic performance. Second, due to time and space constraints, this study did not collect information on language learners’ reading and speaking abilities. Another major limitation of this study pertains to the disproportionate representation of female participants, potentially introducing a confounding factor in terms of the observed learning outcomes. Future studies could collect data from a more balanced distribution across genders. Given the growing need to include more data in this field, particularly in a multilingual context, it is essential to continue in this vein by expanding the corpus.

Supplemental Material

sj-docx-1-sgo-10.1177_21582440231218114 – Supplemental material for Face-to-Face and Online Learning in Higher Education: Academic Achievements and Learners’ Experience in the Chinese SFL Context

Supplemental material, sj-docx-1-sgo-10.1177_21582440231218114 for Face-to-Face and Online Learning in Higher Education: Academic Achievements and Learners’ Experience in the Chinese SFL Context by Zhao Liu in SAGE Open

Footnotes

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work was supported by a grant from the Fundamental Research Funds for the Central Universities (East China Normal University) [No. 2021 ECNU-HWCBFBLW001] and [No. 43800-20101-222201].

Ethics Statement

Ethics approval was not required for this study according to the local legislation.

ORCID iD

Zhao Liu

Data Availability Statement

Anonymized data may be shared upon request to the corresponding author, subject to restrictions according to the data protection legislation.

Supplemental Material

Supplemental material for this article is available online.

References

Adarkwah

M. A.

(2021). “I’m not against online teaching, but what about us?”: ICT in Ghana post Covid-19. Education and Information Technologies, 26(2), 1665–1685. https://doi.org/10.1007/s10639-020-10331-z

Adedoyin

O. B.

Soykan

(2023). Covid-19 pandemic and online learning: The challenges and opportunities. Interactive Learning Environments, 31, 863–875. https://doi.org/10.1080/10494820.2020.1813180

Alhamami

(2018). Beliefs about and intention to learn a foreign language in face-to-face and online settings. Computer Assisted Language Learning, 31(1-2), 90–113. https://doi.org/10.1080/09588221.2017.1387154

Al-Jarf

R. S.

(2004). The effects of Web-Based Learning on struggling EFL college writers. Foreign Language Annals, 37(1), 49–57. https://doi.org/10.1111/j.1944-9720.2004.tb02172.x

Armstrong

K. E.

Goodboy

A. K.

Shin

(2022). Pandemic pedagogy and emergency remote instruction: Transitioning scheduled in-person courses to online diminishes effective teaching and student learning outcomes. Southern Communication Journal, 87(1), 56–69. https://doi.org/10.1080/1041794x.2021.2011954

Arnett

(2020). Communication pedagogy: The coronavirus pandemic. Journal of Communication Pedagogy, 3, 5–10. https://doi.org/10.31446/jcp.2020.02

Aronoff

S. C.

Evans

Fleece

Lyons

Kaplan

Rojas

(2010). Integrating evidence based medicine into undergraduate medical education: Combining online instruction with clinical clerkships. Teaching and Learning in Medicine, 22(3), 219–223. https://doi.org/10.1080/10401334.2010.488460

Bali

Liu

M. C.

(2018). Students’ perceptions toward online learning and face-to-face learning courses. Journal of Physics Conference Series, 1108, 012094. https://doi.org/10.1088/1742-6596/1108/1/012094

Barrot

J. S.

Llenares

I. I.

Del Rosario

L. S.

(2021). Students’ online learning challenges during the pandemic and how they cope with them: The case of the Philippines. Education and Information Technologies, 26(6), 7321–7338. https://doi.org/10.1007/s10639-021-10589-x

10.

Blake

R. J.

(2011). Current trends in online language learning. Annual Review of Applied Linguistics, 31, 19–35. https://doi.org/10.1017/s026719051100002x

11.

Blau

Mittal

Schirmer

Ozkan

(2017). Differences in business undergraduate perceptions by preferred classroom learning environment. Journal of Education for Business, 92(6), 280–287. https://doi.org/10.1080/08832323.2017.1365679

12.

Bourelle

Knutson

A. V.

Spong

(2016). Sites of multimodal literacy: Comparing student learning in online and face-to-face environments. Computers and Composition, 39, 55–70. https://doi.org/10.1016/j.compcom.2015.11.003

13.

Bowers

Kumar

(2015). Students’ perceptions of teaching and social presence: A comparative analysis of face-to-face and online learning environments. International Journal of Web-Based Learning and Teaching Technologies, 10(1), 27–44. https://doi.org/10.4018/ijwltt.2015010103

14.

Cao

Fang

Hou

Han

Dong

Zheng

(2020). The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Research, 287, 112934. https://doi.org/10.1016/j.psychres.2020.112934

15.

Chang

C.-S.

Liu

E. Z.-F.

Sung

H.-Y.

Lin

C.-H.

Chen

N.-S.

Cheng

S.-S.

(2014). Effects of online college student’s Internet self-efficacy on learning motivation and performance. Innovations in Education and Teaching International, 51(4), 366–377. https://doi.org/10.1080/14703297.2013.771429

16.

Chaturvedi

Vishwakarma

D. K.

Singh

(2020). COVID-19 and its impact on education, social life and mental health of students: A survey. Children and Youth Services Review, 121, 105866. https://doi.org/10.1016/j.childyouth.2020.105866

17.

Clark

A. E.

Nong

Zhu

(2021). Compensating for academic loss: Online learning and student performance during the COVID-19 pandemic. China Economic Review, 68, 101629. https://doi.org/10.1016/j.chieco.2021.101629

18.

Dabbagh

Kitsantas

(2004). Supporting self-regulation in student-centered web-based learning environments. International Journal on E-Learning, 3(1), 40–47.

19.

Day

Chang

I.-C. C.

Chung

C. K. L.

Doolittle

W. E.

Housel

McDaniel

P. N.

(2021). The immediate impact of COVID-19 on postsecondary teaching and learning. Professional Geographer, 73(1), 1–13. https://doi.org/10.1080/00330124.2020.1823864

20.

Dedeilia

Sotiropoulos

M. G.

Hanrahan

J. G.

Janga

Dedeilias

Sideris

(2020). Medical and surgical education challenges and innovations in the COVID-19 era: A systematic review. In Vivo, 34(3 suppl), 1603–1611. https://doi.org/10.21873/invivo.11950

21.

Ding

(2007). Text memorization and imitation: The practices of successful Chinese learners of English. System, 35(2), 271–280. https://doi.org/10.1016/j.system.2006.12.005

22.

Di Pietro

Biagi

Costa

Karpinski

Mazza

. (2020). The likely impact of COVID-19 on education reflections based on the existing literature and recent international datasets. Publications Office of the European Union.

23.

Donitsa-Schmidt

Ramot

(2020). Opportunities and challenges: Teacher education in Israel in the covid-19 pandemic. Journal of Education for Teaching, 46(4), 586–595. https://doi.org/10.1080/02607476.2020.1799708

24.

Dyrbye

Cumyn

Day

Heflin

(2009). A qualitative study of physicians’ experiences with online learning in a masters degree program: Benefits, challenges, and proposed solutions. Medical Teacher, 31(2), e40–e46. https://doi.org/10.1080/01421590802366129

25.

Easton

S. S.

(2003). Clarifying the instructor’s role in online distance learning. Communication Education, 52(2), 87–105. https://doi.org/10.1080/03634520302470

26.

Engzell

Frey

Verhagen

M. D.

(2021). Learning loss due to school closures during the COVID-19 pandemic. Proceedings of the National Academy of Sciences, 118(17), e2022376118. https://doi.org/10.1073/pnas.2022376118

27.

Erarslan

(2021). English language teaching and learning during covid-19: A global perspective on the first year. Journal of Educational Technology and Online Learning, 4(2), 349–367.

28.

Foong

Goh

(1997). Chinese ESL students’ learning strategies: A look at frequency, proficiency, and gender. Hong Kong Journal of Applied Linguistics, 2(1), 39–53.

29.

Franchi

(2020). The impact of the covid-19 pandemic on current anatomy education and future careers: A student’s perspective. Anatomical Sciences Education, 13(3), 312–315. https://doi.org/10.1002/ase.1966

30.

Gao

L. X.

Zhang

L. J.

(2020). Teacher learning in difficult times: Examining foreign language teachers’ cognitions about online teaching to tide over COVID-19. Frontiers in Psychology, 11, 549653. https://doi.org/10.3389/fpsyg.2020.549653

31.

García-Tascón

Sahelices-Pinto

Mendaña-Cuervo

Magaz-González

A. M.

(2020). The impact of the COVID-19 confinement on the habits of PA practice according to gender (male/female): Spanish case. International Journal of Environmental Research and Public Health, 17(19), 1–19. https://doi.org/10.3390/ijerph17196961

32.

Gaytan

(2015). Comparing faculty and student perceptions regarding factors that affect student retention in online education. American Journal of Distance Education, 29(1), 56–66. https://doi.org/10.1080/08923647.2015.994365

33.

González-Calvo

Barba-Martín

R. A.

Bores-García

Hortigüela-Alcalá

(2022). The (virtual) teaching of physical education in times of pandemic. European Physical Education Review, 28(1), 205–224. https://doi.org/10.1177/1356336x211031533

34.

Gonzalez

P. F.

Ochoa

C. A.

Cabrera

P. A.

Castillo

L. M.

Quinonez

A. L.

Solano

L. M.

Espinosa

F. O.

Ulehlova

Arias

M. O.

(2015). EFL teaching in the Amazon region of Ecuador: A focus on activities and resources for teaching listening and speaking skills. English Language Teaching, 8(8), 94–102. https://doi.org/10.5539/elt.v8n8p94

35.

Gonzalez

de la Rubia

M. A.

Hincz

K. P.

Comas-Lopez

Subirats

Fort

Sacha

G. M.

(2020). Influence of COVID-19 confinement on students’ performance in higher education. PLoS One, 15(10), e0239490. https://doi.org/10.1371/journal.pone.0239490

36.

Johnson

R. K.

(1996). Vocabulary learning strategies and language learning outcomes. Language Learning, 46(4), 643–679. https://doi.org/10.1111/j.1467-1770.1996.tb01355.x

37.

Helms

J. L.

(2014). Comparing student performance in online and face-to-face delivery modalities. Journal of Asynchronous Learning Networks, 18(1), 1.

38.

Heppen

J. B.

Sorensen

Allensworth

Walters

Rickles

Taylor

S. S.

Michelman

(2017). The struggle to pass algebra: Online vs. face-to-face credit recovery for at-risk urban students. Journal of Research on Educational Effectiveness, 10(2), 272–296. https://doi.org/10.1080/19345747.2016.1168500

39.

Horspool

Lange

(2012). Applying the scholarship of teaching and learning: Student perceptions, behaviours and success online and face-to-face. Assessment & Evaluation in Higher Education, 37(1), 73–88. https://doi.org/10.1080/02602938.2010.496532

40.

Huang

(2017). Development of an instrument to explore teacher roles based on perceptions of English learners in online learning context. Cross-Cultural Communication, 13(5), 1–11. https://doi.org/10.3968/9711

41.

Huang

(2019). Comparing teacher’s roles of f2f learning and online learning in a blended English course. Computer Assisted Language Learning, 32(3), 190–209. https://doi.org/10.1080/09588221.2018.1540434

42.

Iwanaga

Loukas

Dumont

A. S.

Tubbs

R. S.

(2021). A review of anatomy education during and after the COVID-19 pandemic: Revisiting traditional and modern methods to achieve future innovation. Clinical Anatomy, 34(1), 108–114. https://doi.org/10.1002/ca.23655

43.

Jensen

S. A.

(2011). In-class versus online video lectures: Similar learning outcomes, but a preference for in-class. Teaching of Psychology, 38(4), 298–302. https://doi.org/10.1177/0098628311421336

44.

Jin

Cortazzi

(2006). Changing practices in Chinese cultures of learning. Language Culture and Curriculum, 19(1), 5–20. https://doi.org/10.1080/07908310608668751

45.

Jin

(2022). Learning from struggle: ESP teachers’ reflections on online teaching. In Pun

J. K. H.

Curle

Yuksel

(Eds.), The use of technology in English medium education (pp. 39–51). Springer International Publishing.

46.

Johnson

Palmer

C. C.

(2014). Comparing student assessments and perceptions of online and face-to-face versions of an introductory linguistics course. Online Learning Journal, 19(2), 1–18. https://doi.org/10.24059/olj.v19i2.449

47.

Kamal

M. I.

Zubanova

Isaeva

Movchun

(2021). Distance learning impact on the English language teaching during COVID-19. Education and Information Technologies, 26(6), 7307–7319. https://doi.org/10.1007/s10639-021-10588-y

48.

Kemp

Grieve

(2014). Face-to-face or face-to-screen? Undergraduates’ opinions and test performance in classroom vs. Online learning. Frontiers in Psychology, 5, 1278–1311. https://doi.org/10.3389/fpsyg.2014.01278

49.

Khalil

Mansour

A. E.

Fadda

W. A.

Almisnid

Aldamegh

Al-Nafeesah

Alkhalifah

Al-Wutayd

(2020). The sudden transition to synchronized online learning during the COVID-19 pandemic in Saudi Arabia: A qualitative study exploring medical students’ perspectives. BMC Medical Education, 20(1), 285–310. https://doi.org/10.1186/s12909-020-02208-z

50.

Khanal

(2021). Crisis pedagogy: Student perceptions of pedagogical transition amidst the COVID-19. Pedagogical Research, 6(2), em0094. https://doi.org/10.29333/pr/10826

51.

Klimova

(2021). An insight into online foreign language learning and teaching in the era of Covid-19 pandemic. Procedia Computer Science, 192, 1787–1794. https://doi.org/10.1016/j.procs.2021.08.183

52.

Kohnke

Moorhouse

B. L.

(2022). Facilitating synchronous online language learning through zoom. RELC Journal, 53(1), 296–301. https://doi.org/10.1177/0033688220937235

53.

Kokko

Pesonen

Kontu

Pirttimaa

(2015). Why study online in upper secondary school? Qualitative analysis of online learning experiences. Human Technology, 11(1), 57–70. https://doi.org/10.17011/ht/urn.201505061740

54.

Larson

D. K.

Sung

C.-H.

(2019). Comparing student performance: Online versus blended versus face-to-face. Online Learning, 13(1), 31–42. https://doi.org/10.24059/olj.v13i1.1675

55.

Lee

Solomon

Stead

Kwon

Ganti

(2021). Impact of COVID-19 on the mental health of US college students. BMC Psychology, 9(1), 95. https://doi.org/10.1186/s40359-021-00598-3

56.

Leszczyński

Charuta

Łaziuk

Gałązkowski

Wejnarski

Roszak

Kołodziejczak

(2018). Multimedia and interactivity in distance learning of resuscitation guidelines: A randomised controlled trial. Interactive Learning Environments, 26(2), 151–162. https://doi.org/10.1080/10494820.2017.1337035

57.

(2014). Differences in the motivation of Chinese learners of English in a foreign and second language context. System, 42, 451–461. https://doi.org/10.1016/j.system.2014.01.011

58.

Liu

Oga-Baldwin

W. L. Q.

(2022). Motivational profiles of learners of multiple foreign languages: A self-determination theory perspective. System, 106, 102762. https://doi.org/10.1016/j.system.2022.102762

59.

Maldonado

J. E.

De Witte

(2022). The effect of school closures on standardised student test outcomes. British Educational Research Journal, 48(1), 49–94. https://doi.org/10.1002/berj.3754

60.

Means

Toyama

Murphy

Bakia

Jones

(2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. In: Learning Unbound: Select Research and Analyses of Distance Education and Online Learning. Department of Education. https://files.eric.ed.gov/fulltext/ED505824.pdf

61.

Meeter

(2021). Primary school mathematics during the COVID-19 pandemic: No evidence of learning gaps in adaptive practicing results. Trends in Neuroscience and Education, 25, 100163. https://doi.org/10.1016/j.tine.2021.100163

62.

Meinck

Fraillon

Strietholt

(Eds.). (2022). The impact of the COVID-19 pandemic on education: International evidence from the responses to educational disruption survey (REDS). United Nations Educational, Scientific and Cultural Organization. https://unesdoc.unesco.org/ark:/48223/pf0000380398

63.

Molock

S. D.

Parchem

(2022). The impact of COVID-19 on college students from communities of color. Journal of American College Health, 70, 2399–2405. https://doi.org/10.1080/07448481.2020.1865380

64.

Montiel Chamorro

M. L.

, (2018). Comparing online English language learning and face-to-face English language learning at El Bosque University in Colombia [PhD thesis, Virginia Commonwealth University]. https://scholarscompass.vcu.edu/etd/5343/?utm_source=scholarscompass.vcu.edu%2Fetd%2F5343&utm_medium=PDF&utm_campaign=PDFCoverPages

65.

Mooney

G. A.

Bligh

J. G.

(1997). Information technology in medical education: Current and future applications. Postgraduate Medical Journal, 73(865), 701–704. https://doi.org/10.1136/pgmj.73.865.701

66.

Moorhouse

B. L.

Kohnke

(2021). Responses of the English-language-teaching community to the COVID-19 pandemic. RELC Journal, 52(3), 359–378. https://doi.org/10.1177/00336882211053052

67.

T. K.

Reynolds

Chan

M. Y. H.

X. H.

Chu

S. K. W.

(2020). Business (teaching) as usual amid the COVID-19 pandemic: A case study of online teaching practice in Hong Kong. Journal of Information Technology Education. Research, 19, 775.

68.

A. Y.

(2013). Comparing the effectiveness of classroom and online learning: Teaching research methods. Journal of Public Affairs Education, 19(2), 199–215. https://doi.org/10.1080/15236803.2013.12001730

69.

Nushi

Orouji

(2020). Investigating EFL teachers’ views on listening difficulties among their learners: The case of iranian context. Sage Open, 10(2), 1–16. https://doi.org/10.1177/2158244020917393

70.

Odriozola-González

Planchuelo-Gómez Irurtia

M. J

de Luis-García

(2020). Psychological effects of the COVID-19 outbreak and lockdown among students and workers of a Spanish university. Psychiatry Research, 290, 113108. https://doi.org/10.1016/j.psychres.2020.113108

71.

Pedersen

T. L.

(2021). ggraph: An implementation of grammar of graphics for graphs and networks [Manual]. https://CRAN.R-project.org/package=ggraph

72.

Pokhrel

Chhetri

(2021). A literature review on impact of COVID-19 pandemic on teaching and learning. Higher Education for the Future, 8(1), 133–141. https://doi.org/10.1177/2347631120983481

73.

Rong

Abdullah

M. S. H. B

. (2022). English language education in the mainland of China: A Review of selected research under the COVID-19 pandemic (2020-2021). Open Access Library Journal, 09(02), 1–10. https://doi.org/10.4236/oalib.1108390

74.

RStudio Team. (2022). Rstudio: Integrated Development Environment for R [Manual]. http://www.rstudio.com/

75.

Sánchez-Cruzado

Santiago Campión

Sánchez-Compaña

M. T.

(2021). Teacher digital literacy: The indisputable challenge after COVID-19. Sustainability, 13(4), 1858. https://doi.org/10.3390/su13041858

76.

Saputri

Khan

A. K. B. S.

Kafi

M. A.

(2020). Comparison of online learning effectiveness in the ELE during covid-19 in Malaysia and Indonesia. Pioneer: Journal of Language and Literature, 12(2), 103–119. https://doi.org/10.36841/pioneer.v12i2.700

77.

Sawir

(2002). Language difficulties of international students in Australia: The effects of prior learning experience. International Education Journal, 6(5), 567–580.

78.

Schult

Mahler

Fauth

Lindner

M. A.

(2022). Did students learn less during the COVID-19 pandemic? Reading and mathematics competencies before and after the first pandemic wave. School Effectiveness and School Improvement, 33(4), 544–563. https://doi.org/10.1080/09243453.2022.2061014

79.

Singh

Srivastav

Bhardwaj

Dixit

Misra

(2020). Medical education during the COVID-19 pandemic: A single institution experience. Indian Pediatrics, 57(7), 678–679. https://doi.org/10.1007/s13312-020-1899-2

80.

Singh

Thurman

(2019). How many ways can we define online learning? A systematic literature review of definitions of online learning (1988-2018). American Journal of Distance Education, 33(4), 289–306. https://doi.org/10.1080/08923647.2019.1663082

81.

Son

Hegde

Smith

Wang

Sasangohar

(2020). Effects of COVID-19 on college students’ mental health in the United States: Interview survey study. Journal of Medical Internet Research, 22(9), e21279. https://doi.org/10.2196/21279

82.

Spence

D. J.

Usher

E. L.

(2007). Engagement with mathematics courseware in traditional and online remedial learning environments: Relationship to self-efficacy and achievement. Journal of Educational Computing Research, 37(3), 267–288. https://doi.org/10.2190/ec.37.3.c

83.

Spitzer

M. W. H.

Musslick

(2021). Academic performance of K-12 students in an online-learning environment for mathematics increased during the shutdown of schools in wake of the COVID-19 pandemic. PLoS One, 16(8), e0255629. https://doi.org/10.1371/journal.pone.0255629

84.

Sugianto

Ulfah

(2020). Construing the challenges and opportunities of intercultural language teaching amid Covid-19 pandemic: English teachers’ voices. Journal of English Language Teaching and Linguistics, 5(3), 363–381. https://doi.org/10.21462/jeltl.v5i3.454

85.

Sundarasen

Chinna

Kamaludin

Nurunnabi

Baloch

G. M.

Khoshaim

H. B.

Hossain

S. F. A.

Sukayt

(2020). Psychological impact of covid-19 and lockdown among university students in malaysia: Implications and policy recommendations. International Journal of Environmental Research and Public Health, 17(17), 17. https://doi.org/10.3390/ijerph17176206

86.

Tang

Abuhmaid

A. M.

Olaimat

Oudat

D. M.

Aldhaeebi

Bamanger

(2023). Efficiency of flipped classroom with online-based teaching under COVID-19. Interactive Learning Environments, 31(2), 1077–1088. https://doi.org/10.1080/10494820.2020.1817761

87.

Tavil

Z. M.

(2010). Integrating listening and speaking skills to facilitate English language learners’ communicative competence. Procedia - Social and Behavioral Sciences, 9, 765–770. https://doi.org/10.1016/j.sbspro.2010.12.231

88.

The Jamovi Project. (2022). Jamovi (Version 2.3) [Computer software]. https://www.jamovi.org

89.

The SPSSAU Project. (2022). SPSSAU (Version 22.0) [Online application software]. https://www.spssau.com

90.

Tong

Han

Liu

Yang

Chen

(2012). The analysis of influencing factors of college students’ learning effect in face-to-face, online and blended learning. In Lei

Wang

F. L.

Luo

(Eds.), Network Computing and Information security (Vol. 345, pp. 528–538). Springer Berlin Heidelberg.

91.

(1996). A course in language teaching: Practice and theory. Cambridge University Press.

92.

Usher

Barak

(2018). Peer assessment in a project-based engineering course: Comparing between on-campus and online learning environments. Assessment & Evaluation in Higher Education, 43(5), 745–759.

93.

Usher

Barak

(2020). Team diversity as a predictor of innovation in team projects of face-to-face and online learners. Computers & Education, 144, 103702. https://doi.org/10.1016/j.compedu.2019.103702

94.

Varea

González-Calvo

(2021). Touchless classes and absent bodies: Teaching physical education in times of covid-19. Sport Education and Society, 26(8), 831–845. https://doi.org/10.1080/13573322.2020.1791814

95.

Varea

González-Calvo

García-Monge

(2022). Exploring the changes of physical education in the age of Covid-19. Physical Education and Sport Pedagogy, 27(1), 32–42. https://doi.org/10.1080/17408989.2020.1861233

96.

Waldman

Perreault

Alexander

Zhao

(2009). Comparing the perceptions of online learning between students with experience and those new to online learning. Information Technology, Learning and Performance Journal, 25(2), 20–29.

97.

Walker

(2014). Listening: The most difficult skill to teach. Encuentro, 23, 167–175.

98.

Wang

Kremer

G. E.

Jackson

K. L.

(2019). An investigation of effectiveness differences between in-class and online learning: An engineering drawing case study. International Journal on Interactive Design and Manufacturing (IJIDeM), 13(1), 89–98. https://doi.org/10.1007/s12008-018-0510-8

99.

Watkins

(2000). Learning and teaching: A cross-cultural perspective. School Leadership and Management, 20(2), 161–173. https://doi.org/10.1080/13632430050011407

100.

Westwick

J. N.

Morreale

S. P.

(2021). Advancing an agenda for instructional preparedness: Lessons learned from the transition to remote learning. Communication Education, 70(2), 217–222. https://doi.org/10.1080/03634523.2020.1857416

101.

Wheeler

(2001). Information and communication technologies and the changing role of the teacher. Journal of Educational Media, 26(1), 7–17. https://doi.org/10.1080/135816500120069292

102.

Wong

J. O.

(2020). A pandemic in 2020, Zoom and the arrival of the online educator. International Journal of TESOL Studies, 2(3), 82–99. https://doi.org/10.46451/ijts.2020.09.19

103.

(2011). Anxiety in EFL listening comprehension. Theory and Practice in Language Studies, 1(12), 1709–1717. https://doi.org/10.4304/tpls.1.12.1709-1717

104.

Yang

Lin

(2020). The impact of online teaching on interaction during the pandemic: An exploratory study in CFL classes. International Journal of Chinese Language Teaching, 1, 37–50.

105.

Yao

Rao

Jiang

Xiong

(2020). What role should teachers play in online teaching during the COVID-19 pandemic? Evidence from China. Science Insights Education Frontiers, 5(2), 517–524. https://doi.org/10.15354/sief.20.ar035

106.

Yen

S.-C.

Lee

Enriquez

(2018). Learning online, offline, and in-between: Comparing student academic outcomes and course satisfaction in face-to-face, online, and blended teaching modalities. Education and Information Technologies, 23(5), 2141–2153. https://doi.org/10.1007/s10639-018-9707-5

107.

Jang

(2020). Envisioning possibilities amid the COVID-19 pandemic: Implications from English language teaching in South Korea. TESOL Journal, 11(3), e00543. https://doi.org/10.1002/tesj.543

108.

Yuan

Liu

(2021). Understanding EFL instructor identity changes during online teaching in the COVID-19 pandemic: A case study in China. RELC Journal, 1–17. https://doi.org/10.1177/00336882211066622

109.

Zhang

Zhao

J. L.

Zhou

Nunamaker

J. F.

(2004). Can e-learning replace classroom learning? Communications of the ACM, 47(5), 75–79. https://doi.org/10.1145/986213.986216

110.

Zhang

(2023). Reconceptualising digital pedagogy during the COVID-19 pandemic: A qualitative inquiry into distance teaching in China. Innovations in Education and Teaching International, 60(2), 174–184. https://doi.org/10.1080/14703297.2021.2000473

111.

Zhang

(2022). Synchronous online learning during covid-19: Chinese university EFL students’ perspectives. Sage Open, 12(2), 1–10. https://doi.org/10.1177/21582440221094821

112.

Zhang

Wang

Yang

Wang

(2020). Suspending classes without stopping learning: China’s education emergency management policy in the covid-19 outbreak. Journal of Risk and Financial Management, 13(3), 55. https://doi.org/10.3390/jrfm13030055

113.

Zimmerman

B. J.

(1995a). Attaining reciprocality between learning and development through self-regulation. Human Development, 38(6), 367–372. https://doi.org/10.1159/000278343

114.

Zimmerman

B. J.

(1995b). Self-regulation involves more than metacognition: A social cognitive perspective. Educational Psychologist, 30(4), 217–221. https://doi.org/10.1207/s15326985ep3004_8

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.12 MB