Sage Journals: Discover world-class research

Abstract

Online teaching has developed rapidly in the world, and online teaching evaluation (OTE)-related issues have attracted much attention. With the aid of bibliometric visualization software CiteSpace, a co-occurrence network analysis, clustering analysis, and co-citation analysis are developed based on the 1,285 Web of Science Core Collection database articles related to OTE within the period of 2000 to 2023. This paper attempts to find out the current research hotspots and development tendency in the field of OTE by adopting the method of keyword co-occurrence analysis and document co-citation analysis. Based on the above discussions, it has been figured out that in recent years, the quantity and attention degree of publication about OTE are both developing with a rapid-growth tendency, while the core cluster in authors is few and the institution collaboration is not close enough. Therefore, the research hotspots in OTE mainly involve seven aspects: formative evaluation of online teaching, machine learning, peer evaluation, learning satisfaction, MOOCs, evaluation factors, and online evaluation. The focus of frontier research on OTE lies in AI-based, machine learning-based systems, current studies on OTE in the COVID-19 context. It will help researchers understand the current status and development trends of OTE, hence further optimizing methods of evaluation for online teaching.

Keywords

online teaching evaluation (OTE)bibliometric analysis visualized analysis CiteSpace web of science

Introduction

Since the 21st century, with the development of computer computing and the Internet, people’s learning patterns have undergone unprecedented changes, with various new learning modes emerging like a tide. Among all the learning modes, the most impactful is online teaching, which has emerged with the development of network technology. The term online teaching refers to all forms of education that are separated by physical distance from the learners, including teachers or institutions (Joaquin et al., 2020). It generally includes synonyms such as online learning, e-learning, online education, distance education, and online courses. This mode of teaching decreases the time and space limitations associated with traditional education (Panigrahi et al., 2018). It provides a learning experience without the limitation of time and geography (Conrad, 2002).It can meet learners’ fragmented, diversified, and personalized learning needs, thereby improving access to education and training opportunities, enhancing the quality of learning, reducing costs, and increasing the cost-effectiveness of education (Perna et al., 2014).

In 2020, the COVID-19 pandemic and the subsequent lockdowns caused significant changes in people’s behaviors related to work, study, and the consumption of goods and services (Elyassi, 2021; Estrada et al., 2021). In March 2020, at the height of the COVID-19 pandemic, 90% of the global student population—over 1.6 billion people—experienced school closures (Huang et al., 2024). During the pandemic, to ensure the smooth progress of education and teaching, various educational institutions actively conducted online teaching and learning activities through online course platforms and networked learning spaces. According to the information provided through China Internet Network Information Center, online teaching users in China have already reached 377 million.

But at the same time, we must acknowledge that compared with the traditional classroom teaching, online teaching may bring more problems for both the students and teachers. For students, the lack of non-verbal communication (such as facial expressions and tone) in online teaching (Rasheed et al., 2020) and the absence of teacher-student interaction (Anderson et al., 2014) often result in lower accuracy in online teaching (Hill & Ragan, 2010). Students may also experience feelings of isolation, frustration, anxiety, and confusion (Bowers & Kumar, 2015). For teachers, online teaching requires them to learn additional skills and carefully design their courses (Edmunds et al., 2021; Moore et al., 2011), which can lead to increased time spent on preparing teaching activities (Guri-Rosenblit, 2018). Existing literature generally finds that the effectiveness of online teaching is inferior to classroom teaching (e.g., Bettinger et al., 2017; Huang et al., 2024). Therefore, scientifically evaluating the learning outcomes of online teaching becomes an important task, as it directly affects the quality of online education and its sustainable development.

Meanwhile, by the day-to-day great development of online teaching, the literature research about OTE has grown explosively. By 2023, there were 112 new research articles added on the topic in the Web of Science database. However, at the same time, we have to acknowledge that huge literature resources have outgrown the capabilities of human data analysis, and that traditional research methodologies and techniques cannot fit into meeting scientific data analysis demands of the new era.

Based on this, this paper, after conducting a comprehensive review and understanding of the development of online teaching research, utilizes the scientific knowledge mapping tool CiteSpace to perform co-occurrence network analysis, clustering analysis, and co-citation analysis on the relevant literature in OTE, aiming to provide a clearer presentation of the research hotspots and frontiers in this field. The outcomes of this research aim to address the following questions: (1) Examine the current characteristics of research on OTE; (2) Identify the main research hotspots in OTE; (3) Analyze the evolution patterns and current research frontiers in online teaching research; (4) Provide insights and references for researchers and practitioners in the field of online teaching.

Methods

The next section gives an overview of the data sources, research tools, and steps involved in this research. In the case study, the source of data will be the Web of Science Core Collection. Collected data are analyzed by CiteSpace for visualizing the structure of existing research and then complemented with a qualitative content analysis to identify the most important research hotspots and trends in the area of OTE.

Data Sources

This study utilizes the Web of Science Core Collection database as its data source. Web of Science, developed by Clarivate Analytics, is an information service platform, and its Core Collection is recognized as a globally authoritative citation database, widely encompassing world-class academic research outcomes (K. Li et al., 2018). Its powerful analytical capabilities allow for the rapid identification of high-impact papers, the discovery of research directions that attract the attention of peers, and the revelation of research trends. Consequently, the Web of Science Core Collection is a literature retrieval system widely recognized by researchers worldwide (D. Yang et al., 2023).

Research Tools

This study employs CiteSpace for bibliometric and literature visualization analysis. CiteSpace, developed by Dr. Chaomei Chen at Drexel University, is a versatile, dynamic, and time-slicing tool for analyzing literature knowledge maps, grounded in theories such as scientific development, scientific frontiers, structural holes and information foraging (C. M. Chen, 2006). CiteSpace enables scientometric analysis of literature in specific fields and provides diverse visual knowledge maps (such as collaborator network views, cluster views, timeline views, and timezone views), allowing researchers to better grasp hidden knowledge structures, thereby gaining a more comprehensive understanding of the literature (Jia & Harji, 2023). This facilitates the analysis of the underlying mechanisms and research frontiers in a field of study. Compared to other bibliometric analysis software such as VOSviewer and Bibliometrics, CiteSpace is chosen as the preferred analysis tool due to its comprehensiveness, analytical consistency, and user-friendly interface (Chenya et al., 2022).

Research Steps

Results will be obtained using the software CiteSpace 6.3 R1, and then bibliometric analysis will be performed in accordance with the steps described below and reflected schematically in Figure 1.

(1) Keyword selection: Success in retrieving a comprehensive, representative sample depends on the choice of keywords (A. Liu et al., 2023). Generally, online teaching refers to all forms of education where learners are physically separated from teachers or institutions (Joaquin et al., 2020). Related concepts include Online learning, Online courses, Blended learning, Distance education, Distance learning, Massive Open Online Courses and so on. In some studies on OTE, these concepts are often used interchangeably. Therefore, to comprehensively retrieve relevant literature on OTE from the Web of Science Core Collection, we followed the method of B. Liu and Huang (2020) and selected the title search method. The search formula used was “TI = (online teaching or online learning or distance education or online course or MOOC) AND TI = (assessment or evaluation or quality or assurance).” The time span was set from 2000 to 2023, retrieving 1,344 papers (as of December 31, 2023). The refinement of results was done by selecting the document types to be articles and proceedings papers. Subsequently, the subject content of every paper was reviewed manually. Thereafter, irrelevant and duplicate papers were removed. Thus, finally, 1,285 valid papers were left behind.

(2) CiteSpace Configuration: In the CiteSpace setting interface, we select the year from January 2000 to December 2023, and set “time slicing” as “one year per slice.” Term source for txt processing includes Title, Abstract, Author Keywords (DE), and Keywords Plus (ID). The selection criteria for the select top were set to “Top 50” (selecting the top 50 most cited or most occurring items in each slice). Due to the long processing time for certain node types, we enabled the pruning option and selected Pathfinder to improve processing efficiency and graphic readability. Other settings were left at their default values.

(3) Basic Information Analysis of OTE Research: We conducted a co-occurrence network analysis of institutions, countries, and authors to understand the collaboration network in the field of OTE research.

(4) Research Hotspot Analysis in OTE: We used the LLR test method to identify and cluster highly cited papers in the field of OTE, obtaining key cluster labels and summarizing the research hotspots in this field.

(5) Research Frontiers and Trends in OTE: Based on the keyword co-occurrence map, we got the timezone map and burst keywords map of research into OTE. These summarized the main trends and themes of the key research in different periods.

(6) Conclusion and Suggestions: In this paper, based on the results obtained by the above analysis, the current status of OTE research was assessed, and our suggestions were presented considering the possible future research direction that might take place in this field.

Figure 1.

Flow chart of the study.

Results

In this section, we utilize CiteSpace software to conduct bibliometric and visualization analyses of the literature on OTE. These analyses can be divided into six parts, specifically including Characteristics of the temporal distribution, Analysis of research institutions and countries (regions), Journal analysis, Author publication analysis, Cluster analysis, and Keyword co-occurrence map analysis. From the above bibliometric and visualization analyses, we will be able to understand the past, present, and future of research in the field of OTE.

Characteristics of the Temporal Distribution

Plotting the temporal distribution of literature can effectively assess the research status of a discipline and provide insights into its dynamic development trends (Bicheng et al., 2023). Figure 2 illustrates the annual number of publications in OTE studies according to the Web of Science database, from 2000 to 2023. The column denotes the annual publications. We can identify three developmental stages:

Figure 2.

Number of annual publications from 2000 to 2023.

First, the nascent phase (2000–2010): During this period, the annual number of publications mostly remained below 10, reflecting low scholarly attention to OTE.

Second, the growth phase (2011–2019): The number of publications shows a fluctuating upward trend, indicating a sustained increase in attention to OTE.

Third, the explosive phase (2020–2023): The number of publications surged dramatically, with the annual average exceeding 100 since 2021. This surge suggests that OTE has become a focal point in educational research.

Figure 3 displays the citation trends of publications on OTE over the years. It is evident that scholarly attention to OTE has generally been increasing. The total citation count for 1,285 valid publications is 13,387, with an average of 10.42 citations per paper, indicating that the selected literature is of relatively high quality and has a certain level of influence in the field.

Figure 3.

Citation distribution of OTE related research from 2000 to 2023.

In summary, the overall growth of the literature reflects an exponential growth trend. The exponential increase in annual publications may suggest that the development of this field has not yet reached saturation, indicating that it is rapidly evolving (W. Wang & Lu, 2020). Both the number of publications and the citations reached their height after 2020. This is because the COVID-19 pandemic started to break out at the end of 2019, which sped up the pace of development concerning OTE. More and more scholars may join further in discussion and exchange, advancing its diversified development both in theoretical construction and practical educational applications. In this respect, there are great potentials for the future development of OTE.

Analysis of Research Institutions and Countries (Regions)

The network map of institutional collaboration elucidates the spatial distribution of research influence in this field (X. Wang et al., 2019). This study utilized the institutional co-occurrence analysis feature of CiteSpace to explore the distribution of institutional collaboration networks in the field of OTE research, as shown in Figure 3. To analyze the publication output and collaboration relationships among institutions, we further examined Figure 4, identifying the top 10 institutions with the highest number of publications, as presented in Table 1. The top three institutions in terms of publication volume in the OTE field are: Open University-UK (15 papers), National Taiwan University Science & Technol (13 papers), and Universitat Oberta de Catalunya (11 papers). These institutions hold significant influence in this research area.

Figure 4.

Visualization of institution collaboration network.

Table 1.

Top 10 Institutions with the Highest Number of Publications.

Num.	Institution	Country	Count	Centrality (%)
1	The Open University	UK	15	1.17
2	National Taiwan University of Science and Technology	China	13	1.01
3	Universitat Oberta de Catalunya	Spain	11	0.86
4	Central China Normal University	China	9	0.70
5	University of Florida	USA	9	0.70
6	University of Wisconsin-Madison	USA	9	0.70
7	National Cheng Kung University	China	8	0.62
8	Open University of the Netherlands	Netherland	8	0.62
9	University of Indonesia	Indonesia	8	0.62
10	Tsinghua University	China	8	0.62

In terms of collaboration among institutions, National Taiwan University Science & Technol, National Taiwan Normal University, and National Cheng Kung University, all of which are institutions from Taiwan, China, exhibit higher collaboration density and stronger inter-institutional connections. Additionally, we found that the 1,285 retrieved papers originated from 1,512 research institutions, with 1,137 institutions contributing only one paper each. Only 375 institutions published two or more papers, and the connections between institutions were relatively sparse. This indicates that the institutional distribution of OTE research remains quite dispersed.

Figure 5 illustrates the geographical distribution of countries where institutions have published research on OTE. The regions with the highest publication volumes include the United States (287 papers), China (272 papers), Spain (86 papers), Australia (77 papers), the United Kingdom (58 papers), Chinese Taiwan (53 papers), and Canada (49 papers). It is evident that the United States, China, and several European countries, where education is relatively advanced, are the main hubs for research on OTE. However, we also observe that collaboration across regions in this field remains limited, and a close-knit regional collaboration network has yet to form.

Figure 5.

Visualization of country collaboration network.

Journal Analysis

Journals are one of the effective sources for academic communication and sharing of research outcomes. This study identifies the top 10 journals with the highest publication volumes, as shown in Table 2. The International Review of Research in Open and Distributed Learning has the most publications, with 23 articles, accounting for 1.79% of the total publications. It is followed by Computers & Education and Sustainability, each with 19 papers, making up 1.48% of the total publications. Notably, most of these journals focus on fields such as education, computing, network technology, information technology, and medicine. For example, the presence of journals like Computers & Education, Education and Information Technologies, Internet and Higher Education, and IEEE Access indicates that current research on OTE often integrates computer and information technology. Additionally, journals like BMC Medical Education and Frontiers in Psychology are in the fields of medical education and psychology, respectively. This also reflects the effective application of online teaching in medical education.

Table 2.

Top 10 Journals in Terms of the Number of Published Papers.

Num.	Journal	Count	Centrality (%)	Subject
1	International review of research in open and distributed learning	23	1.79	Educational research
2	Computers & education	19	1.48	Computer science, interdisciplinary applications
3	Sustainability	19	1.48	Green, sustainable science & technology
4	Assessment & evaluation in higher education	18	1.40	Educational research
5	BMC medical education	14	1.09	Educational research
6	Frontiers in psychology	14	1.09	Psychology & multidisciplinary
7	Interactive learning environments	14	1.09	Educational research
8	Education and information technologies	13	1.01	Educational research
9	Internet and higher education	12	0.93	Educational research
10	IEEE access	11	0.86	Computer science & information systems

Author Analysis

This study utilized the author co-occurrence feature of CiteSpace to analyze the most prolific authors and their distribution in the field of OTE, as shown in Figure 6. We also ranked the top 10 most prolific authors by publication volume, as detailed in Table 3. Combining the results from Figure 3 and Table 3, we find that Tsai, Chin-Chung from National Taiwan Normal University tops the list with seven publications. He is followed by Caballe, Santi from Open University of Catalonia and Xhafa, Fatos from Technical University of Catalonia, both with five publications each.

Figure 6.

Visualization of author collaboration network.

Table 3.

The Top 10 Authors with the Largest Number of Publications.

Num.	Author	Institution	Count	Centrality (%)
1	Tsai, Chin-Chung	National Taiwan Normal University	7	0.54
2	Caballe, Santi	Open University of Catalonia	5	0.39
3	Xhafa, Fatos	Technical University of Catalonia	5	0.39
4	Kasiyah	University of Indonesia	4	0.31
5	Fernandez-navarro, Francisco	Universidad Loyola Andalucia	4	0.31
6	Hou, Huei-Tse	National Taiwan University of Science and Technology	4	0.31
7	Gomez-rey, Pilar	Universidad Loyola Andalucia	4	0.31
8	Barbera, Elena	Open University of Catalonia	4	0.31
9	Santoso, Harry Budi	University of Indonesia	4	0.31
10	Huang, Yong	Nanjing University of Science and Technology	3	0.23

Regarding research focus: Tsai’s research mainly addresses peer review issues in online teaching (Y. F. Yang & Tsai, 2010); Caballe, Santi, and Xhafa, Fatos earlier focused on the security in online web learning assessment (Miguel et al., 2015). Additionally, Caballe, Santi, and Xhafa, Fatos have a history of collaborative research.

According to Price’s Law, the publication quantity of core authors M = 0.749 × N_max, where N_max represents the publication quantity of the most prolific author in the field (Y. Chen et al., 2015). If an author’s publication quantity exceeds M, they can be considered a core author in the field. In this study, we calculated M ≈ 5 papers. Based on this criterion, authors with more than five publications can be regarded as core authors. According to Table 3, only Tsai, Chin-Chung, Caballe, Santi, Xhafa, and Fatos meet the criteria and are considered core authors in this field of study. Together, they published 12 papers (of which five papers were co-authored by Caballe, Santi, Xhafa, and Fatos), accounting for 0.93% of the total number of papers. Therefore, the distribution of authors in the field of OTE is relatively dispersed, with few core author clusters.

Cluster Analysis

One of the advantages of CiteSpace in bibliometrics is its ability to identify highly cited literature, classify the literature, and form clusters of key research areas, providing labels for each cluster (J. Li & Chen, 2022). These cluster labels reveal the research hotspots within the field to researchers. To understand the research hotspots in the field of OTE, we used CiteSpace’s default Likelihood Rate (LLR) clustering label extraction method to identify and cluster highly cited literature in the field of OTE, as shown in Figure 7. These clusters include Cluster #0 (Formative Evaluation), Cluster #1 (Machine Learning), Cluster #2 (Peer Assessment), Cluster #3 (Learning Satisfaction), Cluster #4 (MOOC), Cluster #5 (Evaluation Factors), and Cluster #6 (Online Evaluation). Clearly, these clusters represent the research hotspots in the field of OTE. To better analyze each cluster, we conducted a detailed study of each one.

Cluster #0 (Formative Evaluation) includes topics such as formative evaluation (Suen, 2014), evaluation methodologies (Margaryan et al., 2015), case study (Collins et al., 2014), flipped classroom, and collaborative learning, among others. This cluster indicates that formative evaluation helps evaluate the outcomes of online teaching and can provide constructive suggestions for future online teaching practices.

Cluster #1 (Machine Learning) focuses on machine learning (Monllao et al., 2020), artificial intelligence (Hopgood et al., 2007), neural networks (T. Liu et al., 2020), and dynamic security assessment (Ren & Xu, 2019), among others. This cluster emphasizes the impact of artificial intelligence on students’ online learning experiences.

Cluster #2 (Peer Assessment) centers on Peer Assessment (Suen, 2014), high education (Onan, 2021), perceptions (Attardi et al., 2016), and Online assessment (Martin et al., 2019), self assessment, and computer-meditated communication, among others. It highlights that peer review can provide significant feedback for the evaluation of current online teaching practices.

Cluster #3 (Learning Satisfaction), along with factors such as student satisfaction (Alqahtani et al., 2022), academic achievement (Gratton-Lavoie et al., 2009) and science of education system worldwide (Klein et al., 2021), learning outcomes, online course design, innovation, among others. This indicates that students’ satisfaction with online learning after class is a relatively critical indicator in the evaluation of online teaching.

Cluster #4 (MOOC) focuses on massive open online courses (Onan, 2021), user experience on online course platforms (S. Wang et al., 2021), quality matters about online courses (Lowenthal et al., 2015), quality assurance management, and automated assessment, among others. This cluster indicates that MOOC serve as an important vehicle in the development of online teaching.

Cluster #5 (Evaluation Factors) includes studies such as evaluation model (Qiang et al., 2009), evaluation of web-based instruction, and media, among others. It not only requires traditional aspects like students, teachers, teaching content, and media but also needs dynamic evaluation and control of online teaching.

Cluster #6 (Online Evaluation) primarily focuses on usability evaluation (Tsironis et al., 2016) and interactional design (Larmuseau et al., 2019), education, online course quality, instructional design, and teaching quality, among others. This cluster emphasizes the comparative evaluation of different online teaching methods.

As we delved deeper into these clusters, we discovered that there are significant correlations between different clusters. For example, in the study by Qiang et al. (2009) on factors influencing OTE, an evaluation model was used, which is classified under Cluster #5 (Evaluation Factors). This model is also mentioned in Cluster #1 (Machine Learning). Similarly, the research conducted by Mcgrew et al. (2018) in Cluster #6 (Online Evaluation) focused on the outcomes of online teaching for students across 12 states in the U.S., which included a survey on student satisfaction with online teaching, indicating a connection with Cluster #3 (Learning Satisfaction).

Figure 7.

OTE research cluster diagram.

Keyword Co-occurrence Map Analysis

The keyword co-occurrence map offers insights into the hotspots and core research themes within specific fields (Bicheng et al., 2023). We configured the node type as “Keywords” to generate a keyword co-occurrence map in the field of online teaching research. Here, the nodes represent keywords, with their size indicating the frequency of occurrence. The lines between nodes depict the relationships between keywords, with the thickness of these lines reflecting the frequency of their co-occurrence (C. M. Chen, 2006). In CiteSpace, various types of keyword co-occurrence maps are available, such as time-zone maps and timeline maps. Below, we analyze the development trends and research frontiers in OTE using these maps.

Time-Zone Diagram Analysis

The time-zone map primarily visualizes the evolution of document keywords and their interrelationships over time, using time as the horizontal axis and clearly displaying them within a two-dimensional coordinate system (Duan et al., 2023), as illustrated in Figure 8. It is evident that from 2000 to 2023, the theme keywords related to OTE have been continuously updated, with their connections to earlier terms deepening over time. Considering the volume of publications, citation years, and keyword distribution, this period can be roughly divided into three stages.

Figure 8.

Time zone of research on OTE.

2000 to 2010: This period marks the early development of OTE. The prominent keywords during this time include distance learning (Xenos, 2004), high-level learning (Vidovich, 2002), formative evaluation of online teaching (De Freitas et al., 2005), learning feedback (Y. F. Yang & Tsai, 2010), and various challenges, among others. It is clear that online teaching was in its initial stages during this period. Scholars primarily focused on evaluating online teaching environments and learning outcomes, with relatively limited research on the evaluation of the online teaching process itself.

2011 to 2019: This stage represents the growth phase of OTE. The main keywords include student evaluation outcomes after online teaching (Adams & Umbach, 2012), machine learning (Scharkow, 2013), MOOC (Yousef et al., 2015), and student perceptions (Attardi et al., 2016), among others. During this stage, online teaching saw significant progress and development. Researchers began optimizing OTE methods by integrating big data, artificial intelligence, and machine learning.

2020 to 2024: This stage is characterized by the rapid expansion of OTE, primarily influenced by the COVID-19 pandemic. With in-person teaching severely restricted, there was a swift transition to online teaching.

The main keywords during this period include COVID-19 (Maqableh & Alia, 2021), student engagement (Szopiński & Bachnik, 2022), and media and devices (Lah et al., 2022), among others. Researchers built upon the advancements from the previous stage, focusing on enhancing interactions between teachers and students. Emerging technologies such as artificial intelligence, machine learning, and networked learning have also played a critical role in the evolution of online teaching during this time.

Analysis of Burst Keywords

The dynamic characteristics of research topics are reflected by a sudden increase in keywords or cited references (S. Wang et al., 2023). Thus, we can analyze the explosive growth of keywords to identify research trends during specific periods. Figure 9 presents the top 12 burst keywords in online teaching evaluation, detailing their year of first appearance, burst strength, start and end years, and duration. Burst strength measures the degree of the sudden increase in the frequency of a keyword, which typically indicates that the keyword represents a research hotspot during that time. The red line segments illustrate the duration of each keyword burst.

Figure 9.

Top 12 keywords with the strongest citation bursts.

From a time series perspective, “distance education” was the first burst keyword to appear in the field of online teaching evaluation. Research on distance education remained a hot topic from 2002 to 2015, spanning 14 years. During this period, scholars primarily focused on the concepts and methods of distance education evaluation (Iskenderoglu et al., 2012). Between 2014 and 2019, burst keywords such as “massive open online courses,”“peer assessment,”“feedback,” and “higher education” began to emerge. This period witnessed the rapid development of MOOCs in higher education, making MOOCs evaluation research a significant hotspot (Yousef et al., 2014). Researchers also increasingly focused on in-depth studies of peer assessment and feedback in online teaching. Beginning in 2020, emerging keywords such as “machine learning,”“model,” and “COVID-19 pandemic” started to surface, reflecting two key trends: first, the research frontier of online teaching evaluation has shifted towards evaluation systems based on machine learning and models (Onan, 2021); second, research on online teaching evaluation in the context of the COVID-19 pandemic has become a critical area of focus (Maqableh et al., 2021).

Discussion and Conclusions

In this section, we will discuss the bibliometric and visualization analysis results, highlight the limitations of this study, and summarize the main conclusions. Based on these findings, we propose future prospects for research on online teaching. We hope that this study can provide valuable insights to scholars in the field of OTE research.

Discussion

With Internet technology developing at such a rapid pace, online teaching is now an indispensable part of modern education. Research on OTE, in tandem with research on online education, has produced some important results. The rising number of research studies on OTE during the last several years is obviously a reflection of this trend. However, there is still a lack of studies that provide a comprehensive overview of the entire body of research on OTE. This study employs CiteSpace software to perform a bibliometric analysis of 1,285 papers on OTE published in the Web of Science database between 2000 and 2023. The study visualizes the research landscape, delineating the basic characteristics, research hotspots, and evolving trends in this field. This research enables us to identify:

The Overall Trend of OTE Research

Our research indicates that online teaching has exhibited a rapid growth trend. In terms of annual publication volume, from 2000 to 2019, the number of publications and citations increased steadily, reflecting the rising global adoption of online teaching driven by advancements in Internet technology. However, from 2020 to 2023, research literature on OTE experienced exponential growth. This growth can be attributed to two primary factors.

First, the development of Internet technology has transformed online teaching from traditional distance education into a diversified, Internet-based learning model, with MOOC serving as a notable example (Margaryan et al., 2015). These diversified online teaching models have clearly created varied research demands for OTE. Second, the onset of the global COVID-19 pandemic in early 2020 rapidly expanded online teaching from higher education to primary and secondary education, leading to a surge in research on OTE at these education levels (Mohan et al., 2021). These two factors have provided valuable data and case studies for theoretical research on OTE, resulting in a significant increase in research output.

The Research Subject Characteristics of OTE

Our research indicates that institutions and scholars in the field of online teaching research are relatively dispersed. First, the field of OTE lacks highly productive institutions and authors. The top three institutions by publication volume are the Open University-UK, National Taiwan University of Science and Technology, and the Open University of Catalonia. Most of the contributing institutions are located in economically developed countries, such as the United States, China, and Spain. However, only 375 institutions have published two or more papers. Prolific authors in this field include Tsai Chin-Chung from National Taiwan Normal University (seven papers), Caballé Santi from the Open University of Catalonia (five papers), and Xhafa Fatos from the Technical University of Catalonia (five papers). Only nine authors have published more than four papers, indicating that prolific authors are still relatively scarce. Moreover, analysis of institutional, national, and author networks reveals that institutional collaboration, cross-regional collaboration, and author collaboration in this field remain underdeveloped. This suggests that the field of online teaching research has yet to form a core group of authors or a tightly-knit network of institutional collaboration.

Research Hotspots of OTE

Our identification and clustering of highly cited literature in the field of OTE resulted in seven prominent clusters. These clusters can be further categorized into three main areas: the evaluation of new online teaching methods (such as MOOC), the elements and methods of OTE, and new technologies for OTE (such as artificial intelligence and machine learning).

First, the evaluation of MOOC. With the establishment of major MOOC platforms like Coursera, Open Learning, and edX, the MOOC movement has rapidly developed on a global scale, significantly impacting traditional teaching models. This has sparked extensive academic discussions on MOOC course design, implementation, and evaluation (Yousef et al., 2014). Research on MOOC quality evaluation has become a frontier and a hot topic in the field of OTE. Current research in this area primarily focuses on evaluating MOOC online teaching environments and technologies, assessing online courses and teaching processes, and evaluating the outcomes of online teaching.

Second, the elements and methods of OTE. Clusters #5 (evaluation elements), #0 (formative assessment), #0 (student learning satisfaction), and #6 (online assessment) all pertain to this theme. It is evident that the evaluation of online teaching courses and processes has consistently been a focal point in the field of OTE. The evaluation criteria have expanded into a multidimensional and comprehensive system that includes teachers, students, equipment, content, and the environment (Hettiarachchi, 2021). Scholars widely recognize the importance of evaluation in the quality control process of online teaching and generally agree that the evaluation system still has deficiencies in aspects such as orientation, standards, methods, and mechanisms, which urgently need improvement (Y. J. Chen et al., 2024). Unlike traditional teaching, online teaching places greater emphasis on student engagement, the importance of collaborative learning, and the role of self-directed learning. It also views teacher-student interaction and peer communication as essential components of online teaching. Therefore, the evaluation of online teaching processes should increasingly focus on aspects such as teacher-student interaction, teaching activities, and peer communication.

Third, the importance of technologies such as artificial intelligence (AI) and machine learning in research has been increasingly recognized. In recent years, the rise of AI, big data, virtual reality, and other technologies has provided effective methods for project evaluation (Gu et al., 2024) and offered significant opportunities and technical means for research in OTE (Moya & Camacho, 2024). Scholars have integrated AI and machine learning into online teaching management systems, utilizing techniques such as learning analytics and data mining to deeply explore, analyze, and cluster learning data. This integration helps online teaching administrators make informed decisions and allocate educational resources more effectively. It also aids teachers in reflecting on their teaching methods, improving teaching models, and optimizing teaching environments. Additionally, these technologies offer personalized intelligent tutoring and support for learners, helping them improve learning efficiency and academic performance.

The Hot Changes of OTE Research

Our study found that hotspots in OTE have evolved alongside the development of evaluation technologies and changes in the external environment. The evolution of research hotspots can be roughly divided into three stages.

The first stage (2000–2010) was the embryonic stage. During this period, the concepts and models of OTE were developed. Researchers aimed to evaluate the effectiveness of online teaching in its initial stages, provide corresponding feedback based on these evaluations, and establish relevant evaluation models. This feedback was subsequently used to inform further studies on online teaching.

The second phase is the development phase (2011–2019). During this period, with the advancement of computer and internet technologies, new forms of online teaching, such as MOOCs, gained rapid popularity, which led to a shift in the research focus on online teaching evaluation. Key research topics during this phase included evaluations of the MOOC teaching environment and technologies, assessments of online courses and teaching processes, and the evaluation of learning outcomes.

The third stage (2020–2023) was the explosive stage. During this time, researchers began to focus on how to integrate online and offline teaching to enhance student engagement and learning efficiency in the context of the significant external impact of COVID-19. They aimed to ensure the quality of online education while also considering the students’ physical health. At the same time, the development of AI and machine learning technologies has further broadened the research field of online teaching evaluation. More researchers are incorporating AI and machine learning into the study of online teaching evaluation, making it a cutting-edge area of current research.

In summary, OTE research has progressed by building on models and concepts established through early practical research, leveraging emerging technologies such as machine learning and AI, and adapting to changes in the external educational environment. These continuous advancements have driven the ongoing development of the field.

Limitations

This study sourced its data from only one database, which is Web of Science; therefore, a number of high-quality research outputs could have been excluded from the sample. In this regard, future research can improve the sample of the study by extending it over different databases to increase the accuracy of the conclusions. For one thing, although we used multiple keywords to cover key literature in the field of online teaching research as comprehensively as possible, some critical literature is inevitably missed due to the lack of some hits of keywords, such as SPOC, which means Small Private Online Courses, personalized MOOC, and Distributed Open Collaborative Courses. This study only included publications written in the English language. In fact, many non-English publications also present valuable insights. These shortcomings might lead to an omission of some relevant papers on this topic. Therefore, there might be some selection bias in this study. We expect that, in the future, research overcomes such limitations and does more precise analysis and comprehensive comparison of publications in the field.

Research Conclusions

Our research, primarily based on CiteSpace, utilizes bibliometric methods to analyze publication volume and trends in the field of OTE within the Web of Science database from 2000 to 2023. The aim is to provide researchers with insights into the hotspots and trends in OTE. Our findings indicate a significant upward trend in the volume of, and attention to, publications on OTE in recent years. However, there remains a lack of core author clusters and strong institutional collaboration networks. The primary research hotspots in OTE include formative assessment, machine learning, peer review, learning satisfaction, MOOC, evaluation elements, and online assessment. The research frontiers in this field focus on AI and machine learning-based evaluation systems, as well as OTE in the context of the COVID-19 pandemic. This study is a pioneering application of bibliometric analysis to reveal historical research trends in the field of OTE. The results may help future researchers better understand the current state and potential directions of this field.

Future Prospects

First, before 2020, research on OTE primarily focused on higher education. After the outbreak of the COVID-19 pandemic in 2020, online teaching was extended to primary and secondary education worldwide. This led to increased attention on online teaching at the basic education level (Mohan et al., 2021). Although the pandemic is now under control, online teaching continues to significantly enrich learners’ experiences, provide personalized tutoring, and offer instant feedback. Therefore, it is still essential to further strengthen the evaluation of online teaching, particularly at the basic education level. Additionally, given that many current studies highlight the clear limitations of online teaching, future research should focus more on how online teaching can be integrated with traditional teaching methods. Researchers should also explore how online teaching can address the shortcomings of traditional education and improve overall teaching quality.

Second, the development of online teaching formats has given rise to new forms such as Small Private Online Courses (SPOCs), Personalized MOOC, and Distributed Open Collaborative Courses. These new formats emphasize learner autonomy, personalized content, and the enhancement of students’ collaborative and problem-solving abilities.

Rigorous evaluation regarding these new online formats of teaching has become the key focus for future research.

The development concerning big data, artificial intelligence, and machine learning is uninterrupted, which opens a new opportunity for researchers. Scholars might go further in exploring the opportunities and challenges that AI, as represented by tools like ChatGPT, poses for OTE. It can improve the measurability of online teaching effectiveness, enhance the sophistication of teaching evaluation, and contribute to a more plentiful theoretical framework and knowledge system. Online teaching belongs to many disciplines and fields, so in this field, interdisciplinary collaboration is particularly urgent. Research into the integration of several disciplines should be done, seeking commonalities or uniqueness of each field for furthering the field of online teaching.

Fourth, by analyzing the network of author and institution collaborations in the current OTE research field, we determined that there are only a few core clusters of authors and underdeveloped institutional collaboration networks. In addition, the unreasonable distribution of educational resources, imbalance in educational development, and unequal access to the internet have led to noticeable gaps in online teaching research between regions. Therefore, communication and collaboration on online teaching research at an international level should be enhanced between researchers and institutions. In that way, the level of OTE research would increase and be harmonized within the different regions, thereby making it cohesive and balanced.

Footnotes

ORCID iDs

Weidong An

Xihui Chen

Luca Esposito

Author Contributions

Weidong An: Conceptualization, Formal analysis, Writing-original draft.

Xihui Chen: Project administration, Conceptualization, Writing-review & editing.

Luca Esposito: Project administration, Supervision, Writing-review & editing.

Liao Zhu: Data curation, Visualization, Writing-review & editing.

Yiling Liu: Investigation, Resources, Software.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the Industry-University Cooperative Education Program of the Ministry of Education of China (Project No. 230800353210400).

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

Adams

M. J. D.

Umbach

P. D.

(2012). Nonresponse and online student evaluations of teaching: Understanding the influence of salience, fatigue, and academic environments. Research in Higher Education, 53(5), 576–591.

Alqahtani

T. M.

Yusop

F. D.

Halili

S. H.

(2022). Dataset on the relationships between flipped classroom approach, students’ learning satisfaction and online learning anxiety in the context of Saudi Arabian higher education institutions. Data in Brief, 45, 108588.

Anderson

Huttenlocher

Kleinberg

Leskovec

(2014, April). Engaging with massive online courses [Conference session]. Proceedings of the 23rd International Conference on World Wide Web, Seoul, Korea (pp. 687–698).

Attardi

S. M.

Choi

Barnett

Rogers

K. A.

(2016). Mixed methods student evaluation of an online systemic human anatomy course with laboratory. Anatomical Sciences Education, 9(3), 272–285.

Bettinger

E. P.

Fox

Loeb

Taylor

E. S.

(2017). Virtual classrooms: How online college courses affect student success. American Economic Review, 107(9), 2855–2875.

Bicheng

Adnan

Harji

M. B.

Ravindran

(2023). Evolution and hotspots of peer instruction: A visualized analysis using CiteSpace. Education and Information Technologies, 28, 2245–2262.

Bowers

Kumar

(2015). Students’ perceptions of teaching and social presence: A comparative analysis of face-to-face and online learning environments. International Journal of Web-Based Learning and Teaching Technologies, 10(1), 27–44.

Chen

C. M.

(2006). CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature. Journal of the American Society for Information Science and Technology, 57, 359–377.

Chen

Y. J.

Smith

Y. M.

Stidham

A. W.

(2024). Faculty online teaching effectiveness scale (FOTES): Instrument development and content validation. Evaluation and Program Planning, 106, 102474.

10.

Chen

Liu

Wang

(2015). Methodological function of CiteSpace knowledge graph. Scientific Research, 2, 242–253.

11.

Chenya

Aminudin

Mohd

Yap

L. S.

(2022). Intelligent risk management in construction projects: Systematic literature review. Institute of Electrical and Electronics Engineers Access, 10, 72936–72954.

12.

Collins

Weber

Zambrano

(2014). Teaching business ethics online: Perspectives on course design, delivery, student engagement, and assessment. Journal of Business Ethics, 125, 513–529.

13.

Conrad

(2002). Deep in the hearts of learners: insights into the nature of online community. Journal of Distance Education, 17(1), 1–19.

14.

De Freitas

R. L.

Piva

D., Jr.

Mauro

Nakamiti

G. S.

Goncalves

(2005). A CBR system to aid the assessment in engineering online courses [Conference session]. WMSCI 2005: 9th World Multi-Conference on Systemics, Cybernetics and Informatics, Orlando, Florida, USA (pp. 296–299).

15.

Duan

Zhao

(2023). Knowledge graph analysis of artificial intelligence application research in nursing field based on visualization technology. Alexandria Engineering Journal, 76(14), 651–667.

16.

Edmunds

J. A.

Gicheva

Thrift

Hull

(2021). High tech, high touch: The impact of an online course intervention on academic performance and persistence in higher education. The Internet and Higher Education, 49, 100790.

17.

Elyassi

(2021). Economics of the financial crisis: Any lessons for the pandemic downturn and beyond? Contemporary Economics, 15(1), 100–121.

18.

Estrada

M. A. R.

Koutronas

Lee

(2021). Stagpression: The economic and financial impact of the COVID-19 pandemic. Contemporary Economics, 15(Special Issue), 19.

19.

Gratton-Lavoie

Stanley

(2009). Teaching and learning principles of microeconomics online: An empirical assessment. The Journal of Economic Education, 40(1), 3–25.

20.

Chen

Parziale

Tang

(2024). Evaluation of primary-level credit environment, indicator system and empirical analysis: A case study of credit construction in China county and district. Evaluation and Program Planning, 104, 102433.

21.

Guri-Rosenblit

(2018). E-teaching in higher education: An essential prerequisite for e-learning. Journal of New Approaches in Educational Research, 7(2), 93–97.

22.

Hettiarachchi

Damayanthi

Heenkenda

Dissanayake

D. M. S. L. B.

Ranagalage

Ananda

(2021). Student satisfaction with online learning during the COVID-19 pandemic: A study at state universities in Sri Lanka. Sustainability, 13(21), 11749.

23.

Hill

Ragan

L. C.

(2010). Principles of effective online teaching: Best practices in distance education (Distance education report), 1–26.

24.

Hopgood

A. A.

Hirst

A. J.

(2007). Keeping a distance-education course current through eLearning and contextual assessment. Institute of Electrical and Electronics Engineers Transactions on Education, 50(1), 85–96.

25.

Huang

Xiong

Liu

Zhang

(2024). College students’ satisfaction with online teaching during the COVID-19 pandemic: Evidence from Hubei Province, China. Heliyon, 10(7), Article 28579.

26.

Iskenderoglu

T. A.

Palanci

(2012). Opinion of teaching staff in distance education systems, regarding the assessment and evaluation process. Procedia-Social and Behavioral Sciences, 46, 4661–4665.

27.

Jia

Harji

M. B.

(2023). Themes, knowledge evolution, and emerging trends in task-based teaching and learning: A scientometric analysis in CiteSpace. Education and Information Technologies, 28(8), 9783–9802.

28.

Joaquin

J. J. B.

Biana

H. T.

Dacela

M. A.

(2020). The Philippine higher education sector in the time of COVID-19. Frontiers in Education, 5, Article 576371.

29.

Klein

Ivanjek

Dahlkemper

M. N.

Jeličić

Geyer

M. A.

Küchemann

Susac

(2021). Studying physics during the COVID-19 pandemic: Student assessments of learning achievement, perceived effectiveness of online recitations, and online laboratories. Physical Review Physics Education Research, 17(1), Article 010117.

30.

Lah

N. H. C.

Tasir

Jumaat

N. F.

(2022). An evaluation of the online social learning environment instrument (OSLEI) using Rasch model analysis. SAGE Open, 12(2), 21582440221104083.

31.

Larmuseau

Desmet

Depaepe

(2019). Perceptions of instructional quality: Impact on acceptance and use of an online learning environment. Interactive Learning Environments, 27(7), 953–964.

32.

Chen

(2022). CiteSpace: Text mining and visualization in scientific literature (3rd ed., pp. 153–163). Capital University of Economics and Business Press.

33.

Rollins

Yan

(2018). Web of Science use in published research and review papers 1997–2017: A selective, dynamic, cross-domain, content-based analysis. Scientometrics, 115(1), 1–20.

34.

Liu

Urquía-Grande

López-Sánchez

Rodríguez-López

Á.

(2023). Research into microfinance and information and communication technologies: A bibliometric analysis. Evaluation and Program Planning, 97, Article 102215.

35.

Liu

Huang

(2020). The frontier and hotspot of foreign OTE research: Visual analysis based on Web of Science journals from 2000 to 2020. Journal of Northwestern Polytechnical University (Social Science Edition), 4, 32–44.

36.

Liu

Zhang

Peng

(2020, July). An effective multi-node charging scheme for wireless rechargeable sensor networks [Conference session]. IEEE INFOCOM 2020-IEEE Conference on Computer Communications, Toronto, ON, Canada (pp. 2026–2035). IEEE.

37.

Lowenthal

P. R.

Hodges

C. B.

(2015). In search of quality: Using quality matters to analyze the quality of massive, open, online courses (MOOCs). International Review of Research in Open and Distributed Learning, 16(5), 83–101.

38.

Maqableh

Alia

(2021). Evaluation online learning of undergraduate students under lockdown amidst COVID-19 pandemic: The online learning experience and students’ satisfaction. Children and Youth Services Review, 128, 106160.

39.

Margaryan

Bianco

Littlejohn

(2015). Instructional quality of massive open online courses (MOOCs). Computers & Education, 80, 77–83.

40.

Martin

Ritzhaupt

Kumar

Budhrani

(2019). Award-winning faculty online teaching practices: Course design, assessment and evaluation, and facilitation. The Internet and Higher Education, 42, 34–43.

41.

McGrew

Breakstone

Ortega

Smith

Wineburg

(2018). Can students evaluate online sources? Learning from assessments of civic online reasoning. Theory and Research in Social Education, 46(2), 165–193.

42.

Miguel

Caballé

Xhafa

Prieto

(2015). Security in online web learning assessment: Providing an effective trustworthiness approach to support e-learning teams. World Wide Web, 18, 1655–1676.

43.

Mohan

Sen

Shah

Jain

(2021). Prevalence and risk factor assessment of digital eye strain among children using online e-learning during the COVID-19 pandemic: Digital eye strain among kids (DESK study-1). Indian Journal of Ophthalmology, 69(1), 140–144.

44.

Monllao Olive

Huynh

D. Q.

Reynolds

Dougiamas

Wiese

(2020). A supervised learning framework: Using assessment to identify students at risk of dropping out of a MOOC. Journal of Computing in Higher Education, 32(1), 9–26.

45.

Moore

J. L.

Dickson-Deane

Galyen

(2011). E-Learning, online learning, and distance learning environments: Are they the same? The Internet and Higher Education, 14(2), 129–135.

46.

Moya

Camacho

(2024). Leveraging artificial intelligence-powered mobile learning: A pedagogically informed framework. Computers and Education: Artificial Intelligence, 7, 100276.

47.

Onan

(2021). Sentiment analysis on massive open online course evaluations: A text mining and deep learning approach. Computer Applications in Engineering Education, 29(3), 572–589.

48.

Panigrahi

Srivastava

P. R.

Sharma

(2018). Online learning: Adoption, continuance, and learning outcome—a review of literature. International Journal of Information Management, 43, 1–14.

49.

Perna

L. W.

Ruby

Boruch

R. F.

Wang

Scull

Ahmad

Evans

(2014). Moving through MOOCs: Understanding the progression of users in massive open online courses. Educational Researcher, 43(9), 421–432.

50.

Qiang

Ming

(2009, May). A evaluation model of web-based distance education for the deaf undergraduate [Conference session]. In 2009 Second International Conference on Information and Computing Science, Chengdu, China (Vol. 1, pp. 297–300). IEEE.

51.

Rasheed

R. A.

Kamsin

Abdullah

N. A.

(2020). Challenges in the online component of blended learning: A systematic review. Computers & Education, 144, 103701.

52.

Ren

(2019). Transfer learning-based power system online dynamic security assessment: Using one model to assess many unlearned faults. IEEE Transactions on Power Systems, 35(1), 821–824.

53.

Scharkow

(2013). Thematic content analysis using supervised machine learning: An empirical evaluation using German online news. Quality & Quantity, 47(2), 761–773.

54.

Suen

H. K.

(2014). Peer assessment for massive open online courses (MOOCs). International Review of Research in Open and Distributed Learning, 15(3), 312–327.

55.

Szopiński

Bachnik

(2022). Student evaluation of online learning during the COVID-19 pandemic. Technological Forecasting and Social Change, 174, Article 121203.

56.

Tsironis

Katsanos

Xenos

(2016, April). Comparative usability evaluation of three popular MOOC platforms [Conference session]. 2016 IEEE Global Engineering Education Conference (EDUCON), Abu Dhabi, United Arab Emirates (pp. 608–612). IEEE.

57.

Vidovich

(2002). Quality assurance in Australian higher education: Globalisation and ‘steering at a distance’. Higher Education, 43(3), 391–408.

58.

Wang

Chen

(2023). Hot topics and frontier evolution of science education research: A bibliometric mapping from 2001 to 2020. Science & Education, 32(3), 845–869.

59.

Wang

Liu

Song

Xie

(2021). Research on evaluation system of user experience with online live course platform. Institute of Electrical and Electronics Engineers Access, 9, 23863–23875.

60.

Wang

(2020). Visualization analysis of big data research based on CiteSpace. Soft Computing, 24(11), 8173–8186.

61.

Wang

Guo

Yang

Zhu

(2019). Tracking knowledge evolution, hotspots and future directions of emerging technologies in cancers research: A bibliometrics review. Journal of Cancer, 10(12), 2643.

62.

Xenos

(2004). Prediction and assessment of student behaviour in open and distance education in computers using Bayesian networks. Computers & Education, 43(4), 345–359.

63.

Yang

Liu

Zhou

(2023). CiteSpace-based global science, technology, engineering, and mathematics education knowledge mapping analysis. Frontiers in Psychology, 13, Article 1094959.

64.

Yang

Y. F.

Tsai

C. C.

(2010). Conceptions of and approaches to learning through online peer assessment. Learning and Instruction, 20(1), 72–83.

65.

Yousef

A. M. F.

Chatti

M. A.

Schroeder

Wosnitza

(2014, July). What drives a successful MOOC? An empirical examination of criteria to assure design quality of MOOCs [Conference session]. 2014 Institute of Electrical and Electronics Engineers 14th International Conference on Advanced Learning Technologies, Athens, Greece (pp. 44–48).

66.

Yousef

A. M. F.

Chatti

M. A.

Schroeder

Wosnitza

(2015). A usability evaluation of a blended MOOC environment: An experimental case study. International Review of Research in Open and Distributed Learning, 16(2), 69–93.

Mapping the Hotspots and New Trends of Online Teaching Evaluation: Evidence from Bibliometric Visualized Analysis

Abstract

Keywords

Introduction

Methods

Data Sources

Research Tools

Research Steps

Results

Characteristics of the Temporal Distribution

Analysis of Research Institutions and Countries (Regions)

Journal Analysis

Author Analysis

Cluster Analysis

Keyword Co-occurrence Map Analysis

Time-Zone Diagram Analysis

Analysis of Burst Keywords

Discussion and Conclusions

Discussion

The Overall Trend of OTE Research

The Research Subject Characteristics of OTE

Research Hotspots of OTE

The Hot Changes of OTE Research

Limitations

Research Conclusions

Future Prospects

Footnotes

ORCID iDs

Author Contributions

Funding

Declaration of Conflicting Interests

Data Availability Statement

References