Abstract
Massive open online courses (MOOCs) are online-based teaching programs designed to accommodate thousands of students without charging any fees. They began appearing in 2009 and 2010, became popular for a while, but are in decline now. This paper contains bibliometric and systematic reviews of research on MOOCs to see what can be learned from the innovation. The primary goals of these reviews are (1) to bibliometrically chart the research conducted on MOOCs and highlight significant milestones, (2) to reveal themes in MOOC research and discover key lessons, and (3) to surface any management education-specific lessons. The results show an increasing interest in scholarly work on MOOCs that demonstrates an enduring interest in reducing drop-out rates, although remedies have not yet been found. Studies demonstrate the importance of increasing opportunities for engagement and interaction. Few studies have explored MOOCs related to business and management. As universities have sought to monetize MOOCs, they have become less massive and less open as key components like credit and certification have been placed behind pay walls. The paper concludes with a critical discussion of MOOC research that suggests that they were a fad whose time has come and gone.
Keywords
A decade ago, the first author wrote an editorial in the Journal of Management Education in which he reflected on the emergence of Massive Open Online Courses (MOOCs) and what they might mean for management education (Billsberry, 2013). He wondered whether these would turn out to be a passing fad or an educational revolution. For several years, MOOCs thrived, but when COVID-19 emerged, a rapid transition to online teaching was forced upon many universities. A silence fell over the libertarian and altruistic notion of free online education with universities’ pressing financial need to monetize online education (Williams, 2023). Consequently, universities’ interest in MOOCs declined significantly (Shah, 2021a).
The switch to online teaching brought about by the COVID-19 pandemic came at an interesting technological moment. Bandwidth, the perennial bugbear of online teaching (Muirhead, 2000), had expanded with broadband and 4G coverage widespread. Communication and conferencing software became stable with “you’re on mute” becoming the catchphrase of the Zoom age. Everything to do with video-making became user-friendly, relatively cheap, and high-quality, so the individual academic could produce their own materials with minimal support. Uploading to internet platforms was straightforward. So, COVID-19 came at a moment when the enthusiastic amateur could produce impressive online materials and deliver acceptable courses. This is quite a contrast to the experience of MOOCs, which had typically absorbed many resources and needed professional and institutional support (Hollands & Tirthali, 2014).
In the “new normal,” university-created MOOCs are disappearing with M (Massive) O (Open) Online Courses being largely replaced by S (Small) P (Private) Online Courses (SPOCs). In answer to the first author’s question, it seems that MOOCs were a fad. In the wake of the COVID-19 pandemic, MOOCs appear to be a product of a particular moment and now is to the ideal time to take stock of the experience and see what can be learned. Our goals in this paper are (1) to bibliometrically chart the research conducted on MOOCs and highlight significant milestones, (2) to reveal themes in MOOC research and discover key lessons, and (3) to surface any management education-specific lessons.
The remainder of this paper is structured as follows. First, we briefly define MOOCs and look at their growth before summarizing the findings from two recent reviews of MOOC research. Second, we conduct a bibliometric review of MOOC research to identify publication trends in MOOC research, the sources and outlets for MOOC research, author networks, and the most cited papers. In addition, we identify themes in MOOC research both over the full life of MOOC research and the last three years. Third, from the identification of research themes in MOOC research carried out in the bibliometric review, we systematically review the six key themes in MOOC research: learning and pedagogy, learning analytics, open education, online teaching, distance education, and student motivation. The purpose of this systematic review is to draw out the main lessons from MOOC research. Fourth, we explore the lessons from the relatively small amount of research that has focused on business and management MOOCs. The paper concludes with a discussion of the current dynamics influencing MOOCs which is seeing them become less massive and less open.
Massive Open Online Courses
MOOCs are online-based teaching programs designed to accommodate thousands of students without charging any fees. They allow public access to their content and progression is at a pre-defined pace (Liyanagunawardena et al., 2013). Typically, MOOCs contain all the materials needed for its successful completion and they are free (Liu et al., 2021), but not credit-bearing (Hew & Cheung, 2014). There are examples of MOOCs requiring a small fee and others with a more substantial fee, which is usually related to the award of formal credit against the suppliers’ qualifications (Kaplan & Haenlein, 2016; Zhu et al., 2020). The earliest MOOCs appeared in the mid-2000s with “Connectivism and Connective Knowledge” facilitated by George Siemens and Stephen Downes at the University of Manitoba generally credited as the first documented course (Kaplan & Haenlein, 2016; Liyanagunawardena et al., 2013).
During the 2010s, there was significant growth in MOOCs. Shah (2019) reports that in 2018, 11,400 MOOCs were offered by more than 900 different universities. The estimated number of enrolments was 101 million students. Shah (2021b) reports that in 2021, the peak of the COVID-19 pandemic when distance and online learning received a considerable boost, 220 million students studied 19,400 MOOCs delivered by 950 universities. Not surprisingly, with this burgeoning of activity in MOOCs, scholars have been keen to study the phenomenon. Results from these studies have been synthesized in a series of reviews that have set the agenda for MOOC research.
Two recent reviews of MOOCs are worthy of note. Zhu et al. (2020, p. 1685) conducted a “systematic comprehensive review of MOOC research.” They summarized the foci of 13 prior reviews of MOOC research before reporting their own. They reviewed 541 empirical studies published from 2009 to 2019 and provided a descriptive overview of these mainly focusing on the research methods and authorship patterns. They also surfaced the research foci of empirical studies which they allocated to one of four categories: student-focused, design-focused, context and impact, and instructor-focused. They also identified 19 research topics, but stopped short of any critical review of these topics, did not summarize findings, or suggest avenues for future research. Liu et al. (2021) conducted a bibliometric review of 1,078 MOOC studies published between 2008 and 2019. They identified 14 previous macro-perspective reviews of MOOC research and 10 micro-perspective ones. Like Zhu et al. (2020), they are only covered in the most superficial way and their further analysis is limited to a descriptive summary. However, these studies demonstrate the size and growth of this research domain and highlight the need for a critical summary of MOOC research and the identification of areas for future research. To address this gap, we conduct a bibliometric review to map MOOC research. In addition, we systematically review MOOC research by theme to see what can be learned from this body of research. Further, we have a specific interest in business and management-related lessons from MOOC research. As there have been no reviews of MOOCs in this domain, it is timely to present a critical review of contemporary themes in this field.
Method
The first goal of this study is to map research on massive open online courses and reveal significant milestones (e.g., leading authors, most cited papers, leading outlets for MOOC research, countries of origin). In effect, this is a bibliometric analysis of MOOC research. The second goal of this study was to discover lessons from MOOC research, which is a systematic review of the MOOC literature. To these ends, we conducted a systematic search across all disciplines looking for articles on the topic. We decided to include journal articles, reviews, editorials, and book chapters and were unconcerned about the empirical or other nature of these articles as we wanted to capture as full a picture of the field as possible. We adopted a Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) approach and followed the bibliometric analyses guidelines of van Oorschot et al. (2018). We only included papers written in English. The search was run on 20 May 2023.
Search Methodology
Inclusion Criteria
We searched for the phrases “Massive Online Open Course” and “Massive Open Online Course” and two abbreviations “MOOC” and “MOOCs.” The logical operator OR was used between each phrase meaning that an article only had to return one of these phrases in the title, abstract, or keywords to be found. The syntax can be viewed in the Appendix.
Exclusion Criteria
Initial pilot work suggested that the inclusion terms were picking up two irrelevant literatures. One was based on optical science and the other in catalysts in chemistry. To eliminate these from the dataset, the exclusion terms “optic*” and “catalyst*” were added as filters. In addition, an analysis of the pilot output showed that all papers that prior to 2009 were unrelated to MOOCs and focused on research in one of these two subject areas. As a “belt and braces” safety measure, hits before 2009 were also excluded in the syntax. We also excluded hits from 2023 as we decided that only full years should be included for ease of interpretation. The searches were run again with these exclusion criteria in place. Limits were set to exclude all forms of publication that were not in journal articles, reviews, editorials, or book chapters, and those not written in English.
Databases Searched
This search was limited to the Scopus database. Scopus has been shown to have a more complete coverage of business, management, and psychology journals than competing databases (Mingers & Lipitakis, 2010) and tends to return more hits than Web of Science although there is considerable overlap (Martín-Martín et al., 2021). Martín-Martín et al. (2021) have calculated the overlap between Scopus and Web of Science to be 83 and 90% between Scopus and Google Scholar.
Search Results
Adopting the search criteria mentioned above, this search for papers initially yielded 3,909 hits. These comprised 3114 journal articles, 597 book chapters, 145 reviews, and 53 editorials.
Software and Techniques
Bibliometric analyses were run in VOSviewer version 1.6.18 (Van Eck & Waltman, 2010). This is free software designed for bibliometric analysis. It is available at www.vosviewer.com. Additional analyses were run in Microsoft Excel based on the same .csv files extracted from Scopus.
Significant Milestones in MOOC Research, 2009 to 2022
Publication Trends
The number of articles published on MOOCs for each year during the period 2009 to 2022 is shown in Figure 1. We also searched without date limitations and found no articles on MOOCs before this period. As noted above, those that appeared in searches before 2009 related to optical science and catalysts. Given the time delays in academic publishing, the graph shows an initial surge of articles between 2012 and 2014, and then increasing volumes to the current day.
Number of journal articles, reviews, editorials, and book chapters published on MOOCs, 2009 to 2022.
Leading Countries by University Affiliation in MOOC Research
Table 1 shows the leading countries by the university affiliation of authors (as opposed to the location of studies or authors’ nationality) who have published MOOC articles. Every country that appears in the affiliations of an article is credited meaning that an article might be counted more than once. For example, there are 850 articles featuring at least one author affiliated to a university or organization located in the United States. Although the table shows that the USA is the most common location for MOOC scholars, less than a quarter of MOOC articles feature an American affiliated author, which is relatively low for an academic literature. It is a marked drop from the 50.2% North American authorship quoted by Veletsianos and Shepherdson (2016) for the period up to the end of 2014 and the 29.9% reported for empirical research up to the end of 2019 by Zhu et al. (2020). Only five other countries, China, Spain, the United Kingdom, Australia, and India feature as authors on more the 5% of articles suggesting a large geographic spread for authors in this literature that is becoming increasingly global and widespread (Table 2).
Number (and Percentage of Total) of MOOC Publications by Authors’ Country Affiliation.
Outlets for MOOC Research by Number of Articles Published (20 Articles Minimum).
Note. = indicates a tie based on the number of MOOC articles published; ties listed in alphabetical order of journal title.
Leading Journals for MOOC Research
Systematic bibliometric analysis is a tool for identifying the landmark contributions in a literature. Here, we produce a table showing the 28 outlets that have published more than 20 papers on MOOCs. In addition, this table includes an assessment of their influence through their Scopus citations per paper.
Co-Author Networks Analysis
The co-authorship network is depicted in Figure 2. Although there are many unconnected authors (minimum of five articles per author to qualify), it shows considerable interconnection in one established network of authors, which is shown in Figure 3. This level of interconnection is generally regarded as a sign that a literature is mature and there is cross-fertilization between research groups.
Co-authorship network, 2009 to 2022.
Largest author network, 2009 to 2022.
Citation Analysis
Table 3 provides a record of the 20 most cited papers on MOOCs. The list is dominated by International Review of Research in Open and Distance Learning from which seven papers appear. This is an open access refereed online journal specializing in open and distance learning. Computers and Education appears four times. All of the journals listed in this list are education journals (typically with a focus on online education or computer-aided education) or generalist journals.
Most Cited Publications on MOOCs in Scopus.
Note. Google Scholar citations captured on 21st May 2023.
Themes in MOOC Research, 2009 to 2022
There are no set guidelines for conducting analyses of keywords in bibliometric studies (Wilden et al., 2017). We tested various configurations to see which produced a logically interpretable clustering of keywords. After a little trial and error, selecting keywords appearing in at least 20 journals which yielded 64 different keywords, and removing keywords “MOOC,” “Massive Open Online Course,” and four similar versions (so that the themes in MOOC research could be spotlighted) produced an interpretable six cluster solution capturing the main themes of MOOC research between 2009 and 2022. These clusters are graphically represented in Figure 4. A review of the main findings from each cluster follows.
Network analysis of author keywords, 2009 to 2022.
Learning and Pedagogy
Studies of MOOC pedagogy inform design, structure, and teaching practices. The early years of MOOC research distinguished between xMOOCs and cMOOCs: xMOOCs included predefined content and focused on participants’ free access to expertise, whereas cMOOCs built on intensive use of social networks as part of the learning (Castaño et al., 2015), cooperation among MOOC participants to introduce new resources through social networks, and the integration of these resources with previous teaching materials (Fidalgo-Blanco et al., 2016). cMOOCs can be seen as an environment for “new forms of distribution, storage, archiving, and retrieval [and] offer the potential for the development of shared knowledge and forms of distributed cognition” (Kop et al., 2011, p. 78). They were found to be better than xMOOCs for learner motivation, performance (Castaño et al., 2015), retention, participation, and satisfaction (Kop et al., 2011). Fidalgo-Blanco et al. (2016) suggested a way to combine the two modes and produce stronger learning outcomes by running the two models in parallel; material created in the cMOOC would be embedded in the xMOOC. They found that completion rates increased by over 30% and improved student satisfaction (Fidalgo-Blanco et al., 2016).
Instructors see MOOC design and facilitation as highly complex relating to pedagogy (learning objectives, assessments, duration, content, etc.), resources (platform affordance, institutional support, and availability), and logistics (e.g., development time) (Laaser & Toloza, 2017; Zhu et al., 2018). To address these challenges, instructors typically rely on reviewing other MOOCs and help from colleagues, their universities, and platform support personnel (Zhu et al., 2018). Drachsler and Kalz (2016) propose a standardized process for MOOC innovation that includes a complex development and evaluation cycle across three levels: micro (learner and teacher), meso (institution), and macro (across MOOC providers). In their review of 76 MOOCs, Margaryan et al. (2015) found that although they were well-packaged, the quality of the instructional design was low in terms of five first principles of instruction: problem centered, activation, demonstration, application, integration. In addition, the design of these MOOCs provided little support for learners’ diversity; a design feature criticism echoed by Knox (2014) which is particularly pertinent given these devices aim for “massive” audiences.
Two main learning approaches are central to the design of MOOCs: flipped classroom (Tolks, et al., 2016; K. Wang & Zhu, 2019) and blended/hybrid design (Gynther, 2016). In a flipped classroom model, a self-directed individual learning phase precedes an engagement with the class and/or teacher (Tolks et al., 2016). This design was shown to outperform traditional classroom teaching in terms of student performance and student experience of student–student interaction, available learning materials, and active learning results (K. Wang & Zhu, 2019). Particularly in language learning, the flipped classroom model showed the following benefits compared with traditional learning: flipped classroom students gained significantly better oral proficiency (speech fluency, complexity and accuracy) requiring 25% less face time and demonstrated more (out of class) time investment in their learning and more positive attitudes toward the course (J. Wang et al., 2018).
MOOC research also showed the benefits of a hybrid/blended teaching model, which complements online learning with in-person activities. This hybrid design offers several benefits. It supports the greatest diversity of learners and scaffolds engagement with networked (peer-to-peer) and emergent (flexible student-directed) learning contexts (Anders, 2015). It provides cognitive benefits such as broadened thinking perspective, raised cultural awareness, and the sharing of learning strategies, affective benefits such as a strong sense of community, and motivation for learning gain. Participants also increased action tendencies toward trying out the learning platform’s functions, new courses, and learning strategies, and had greater cognizance of the benefits and procedures of the MOOC study group (Chen & Chen, 2015). Blended and online learning have also been found to be better than traditional learning for student performance and satisfaction (Larionova et al., 2018). For humanities subjects, blended teaching was better than pure online teaching supplemented by tutoring support, but for technical subjects blended and supported online teaching were equally effective (Larionova, et al., 2018). A hybrid design for engineering MOOCs enables practical and effective in-person labs by using non-costly and easy to build robots, which make use of household-available technologies, thus serving as an alternative or complementary to virtual labs (López-Rodríguez & Cuesta, 2016). But Israel (2015) showed that blended learning experiments found modest improvements in performance and lower satisfaction rates among blended learning students.
Learning Analytics
Learning analytics (LA) is “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Siemens, 2011, cited by Pursel et al., 2016, p. 204). Early research on MOOCs associated learning analytics with education data mining (EDM), which it defined as the development and adaptation of “statistical, machine-learning and data-mining methods to study education-related data generated by students and instructors” (Liñán & Pérez, 2015, p. 100). Despite both approaches involving some similar methods (e.g., prediction, clustering, relationship mining, outlier detection, text mining, etc.), their differences suggest distinctly different perspectives and foci (Siemens & Baker, 2012). Over the years, the distinctions between LA and EDM have become blurred and all methods making use of data resulting from online education have been grouped under the umbrella of learning analytics, including EDM. The two main functions of LA, apart from overall evaluations of MOOC effectiveness, are to predict student completion (or dropout) and performance and to assist with student assessment and learning. We briefly highlight the key findings in each of these areas.
There is strong interest in predicting MOOC dropouts (Moreno-Marcos et al., 2020). Students’ engagement with MOOC features generally predicts completion and early engagement seems particularly important: early engagement with tasks such as peer grading, engagement with the MOOC at 2 weeks (Formanek et al., 2017), or the extent of engagement during the MOOC’s second quartile (Gregori et al., 2018) predict completion more strongly than general engagement. The first two weeks were also found critical for maintaining student engagement. After then, the proportion of active learners and those submitting assessments levels out with less than 3% difference between them (Jordan, 2015). Active participation (such as posting online) increases completion chances more than passive participation (video watching), but the number of posts made by students does not increase completion likelihood (Pursel et al., 2016). However, while completion is associated with participation, some students do not complete despite high forum participation rates (Cohen et al., 2019). Students who pass courses spread their learning over more days compared with students who failed, but the order of engagement in MOOC activities has been shown to account for little significant difference between students who passed and those who failed (Conijn et al., 2018). Learner characteristics and expectations have also been found to predict completion. Previous experience in online education or MOOCs was not found to increase completion probability despite evidence that such experience is associated with better learning strategies (Li, 2019). Finally, learners’ expectations of themselves (to gain a certificate and to be active in the course) also increased completion probability (Pursel et al., 2016). The combination of teachers’ presence throughout the course, their interactions with students, and the quality of videos quality were found to significantly influence course completion (Gregori et al., 2018).
Although data used for predicting performance includes clickstream behavior (i.e., play, pause, slow down or speed up, or jump to another place in the MOOC video), forum posts, course metadata such as information about course modules, video lectures (length, title, module), assignments (including quizzes, homework, essays, human-graded assignments, exams, etc.), and learner demographics (Gardner & Brooks, 2018), the use of clickstream data stands out (Moreno-Marcos et al., 2020). Yang et al. (2017) found that students’ clickstream behavior improved the prediction of student performance based on their quiz performance, and Brinton et al. (2016) were able to take this approach to predict students’ likelihood of being correct on first attempt. In addition, the aggregated number of MOOC activity frequencies undertaken by students was positively related to on-campus exam grade, although this relationship is less clear when controlling for students’ past performance (Conijn et al., 2018).
One of MOOCs’ greatest attractions to providers is their scalability and the potential of large enrolled student numbers. This, however, creates challenges when it comes to assessment given the high ratio of students-to-teachers. The idea of self- and peer-assessment is therefore attractive and LA has been used to examine the effectiveness of each of these methods. While some evidence suggests that self-assessment is relatively consistent with staff assessment (more than 65% of self-graded marks were within 10% of staff marks; Kulkarni et al., 2013), this accuracy can be improved by providing students feedback on their grading. But the jury is out as other evidence suggests that self-assessment is not a reliable way to assess performance and serves better as a learning tool (Admiraal et al., 2015). Peer assessment is a viable solution too, although studies show that when peers, even trained ones, mark assessments, their marks are significantly different to those of paid instructors (Formanek et al., 2017). One study on an Astronomy MOOC found that peer assessors tend to “grade towards the mean”; a tendency also observed in a study by Piech et al. (2013). When moderated, the accuracy of peer marking improves (Suen, 2014). Regardless of its effectiveness, peer-assessment offers a way to engage students in the MOOC, and students who do it perform better in MOOCs (Meek et al., 2017). LA can also be used to assist MOOC students by alerting them to their predicted marks in time for remedial action (Meier et al., 2016), informing instructors to students’ need for tailored intervention (Lu et al., 2017), and supporting students’ self-regulated learning (Wong et al., 2019).
Open Education
Although MOOCs include “open” in their definition and name, there is debate as how “open” the education they provide actually is. “Open” is defined as accessible for free with no restrictions, allowing participants to openly share their experiences and create content in the MOOC, and participants’ learning pathways are not constricted by institutions (Czerniewicz et al., 2017). Open Education Resources (OERs), which preceded MOOCs, were seen as the “pure” form of “open” (Alario-Hoyos et al., 2017). Indeed, MOOC scholars proposed systems to identify OERs suitable to be included in MOOCs (Piedra et al., 2015) and there are MOOCs which train people to use them (Ramírez-Montoya et al., 2017). However, MOOCs usually do not meet all “open” criteria as access to their content is restricted, only accessible to registered users, and it cannot be reused or repurposed by others (Atenas, 2015). Moreover, the learning journey is often restricted and prescribed by the institution. Another aspect of openness of MOOCs and OERs is accessibility and inclusion. Scholars have drawn attention to disabilities and the need to make MOOCs inclusive as a necessary part of being open (Sanchez-Gordon & Luján-Mora, 2016).
Online Teaching in Higher Education
MOOCs have been seen as a setting conducive to advanced peer collaborative learning and even basic technologies (e.g., forums or blogs) have been found to facilitate peer-supported learning (Kellogg et al., 2014). When testing interactive and context-sensitive support through automated peer-help matching and automated facilitation of student interaction, Rosé and Ferschke (2016) discovered that participation in chats involving these technologies reduced the dropout rate by about 50%. Interaction with peers in a writing task (peer assessment or forum entry) has been found to improve students’ learning, particularly their understanding of course concepts (Comer et al., 2014). Furthermore, MOOCs have allowed for the testing of dynamic group formation techniques (e.g., Srba & Bielikova, 2015) where students’ feedback on their group members guided their next group allocation, which showed significantly better performance. Examining a MOOC with multiple social platforms available found that while students’ preferences of social tools varied, the MOOC’s built-in discussion forum had the most engagement (Alario-Hoyos et al., 2017).
Public media discussion of MOOCs rapidly declined between 2012 and 2014 (Kovanovic et al., 2015), however MOOCs still inspired hopes for transformation in higher education in general and for disadvantaged populations in particular. The cost difference to individuals between MOOCs and traditional university courses is staggering; $74 to $272 per MOOC completer, which is substantially lower than costs per completer of regular online courses, which is estimated at $7,000 to $10,000 (Hollands & Tirthali, 2014). Evidence, however, consistently shows that most MOOC learners are highly educated and well-resourced individuals (Laurillard, 2016; Stich & Reeves, 2017). Those of disadvantaged background were found more likely to engage in MOOCs for educational purposes, but are less likely to complete (Stich & Reeves, 2017). Addressing this discrepancy, it was found that when specific disadvantaged groups are recruited to MOOCs relevant to them (e.g., teachers from emerging economies for a teachers’ professional development MOOC), MOOCs reach and successfully support audiences in need (Laurillard, 2016). Evidence shows that MOOCs’ delivery and access have to be transformed if they are to advance global education of less-privileged students: a study of multilingual or non-English MOOCs found that they best address the needs of disadvantaged cohorts when they specifically aim to support those cohorts, provided tailored support informed by partnership with representing community groups, and are delivered in hybrid mode (Laurillard, 2016). In addition, they are effective when resources are free and accessible to the learner irrespective of copyright status (Lambert, 2020). MOOCs’ potential to transform education in developing countries depends on if the hosting countries’ conditions, infrastructure, and policies (e.g., acknowledging accreditation) support it (Oyo & Kalema, 2014), including governments at local, state, and federal levels, country laws, as well as industry, ICT capacity, Internet/mobile technology diffusion, and income and digital divide (Palvia et al., 2018).
Distance Education
Distance learning has been successfully operating for decades (Baran et al., 2016) and the field offers many lessons for MOOC scholarship. MOOCs capitalize on virtual technologies and support peer learner interactions, either by building reputation (Khan et al., 2018) or by peer review (Loizzo & Ertmer, 2016). Although peer review in distance education has produced mixed results (Wen & Tsai, 2006), MOOC research supports its benefits (Meek et al., 2017). Unlike distance education, MOOCs have a propensity for open, unrestricted learning pathways and allow for learning by “lurking”; that is, learning without the students’ engagement or interaction (Loizzo & Ertmer, 2016, p. 1017). MOOCs also empower distance education students. For example, MOOC features (e.g., self-directed videos and self-paced activities) were found beneficial for remedial distance education students of Khan Academy and allow students to be part of the learning experiences as in cMOOCs (Muñoz-Merino et al., 2017). However, the quality of MOOCs’ instructional audio-visual media have been criticized when compared to those of distance education for lacking adequate incentives to use them for collaboration and for insufficient use of student generated videos (Laaser & Toloza, 2017).
Student Motivation
People’s motivation to enroll in MOOCs, engage with them, and complete them is important due to the self-directed aspect of MOOCs. Learners’ motivation to use MOOCs is generally divided into intrinsic and extrinsic forms. Intrinsic motivation relates to the attractiveness of the process itself in terms of enjoyment, curiosity, general interest, growth, and development such as learning and personal challenge (Hew & Cheung, 2014; Loizzo et al., 2017; Milligan & Littlejohn, 2017; Shapiro et al., 2017). Extrinsic motivation typically involves externally-awarded outcomes such as professional development (Salmon et al., 2017; Shapiro et al., 2017; Vivian et al., 2014), benefits to either current or future roles (Milligan & Littlejohn, 2017), or formal certification (Hew & Cheung, 2014). Extrinsic motivations seem to vary across cohorts. When comparing the motivations of students and professionals to use MOOCs as learners across two different MOOCs (data science and clinical trials), Milligan and Littlejohn (2017) found that students use MOOCs to complement their formal learning, whereas professionals were motivated by their current needs at work.
Generally, intrinsic motivation is associated with greater engagement and better performance (Tang et al., 2018). In a 2014 US-based project management MOOC (Tang et al., 2018), learners who indicated predominantly intrinsic motivation (interest, desire to expand knowledge) at the start of the MOOC also demonstrated consistent high engagement throughout the MOOC’s period. Learners who indicated mixed intrinsic and extrinsic (for career advancement) motivation also demonstrated declining engagement as the MOOC progressed, and learners who indicated predominantly extrinsic motivation demonstrated consistently low engagement. MOOC performance was consistent with these patterns, where learners’ performance increased, the more they indicated intrinsic motivation. Although generally intrinsic motivation was associated with better learner performance, at times, so was extrinsic motivation. Learners’ motives and patterns of engagement correlated with their course performance in a digital storytelling MOOC (Phan et al., 2016). Learners who were motivated by earning a continuing professional development certificate (extrinsic), gaining skills, ideas and inspirations (intrinsic), and improving their professional practice (mixed) outperformed the learners who valued these traits less.
Given the self-directed nature of MOOCs, a notable portion of research has examined self-regulated learning; the ability of the learner to control and regulate their own learning through the usage of cognitive and metacognitive strategies such as goal setting, time management, and learning strategies (Zimmerman, 2012). Higher self-regulation has led to better learner outcomes, as follows: strategies for structuring the learner’s environment predicted greater learner satisfaction and perceived learning (Li, 2019). Self-regulated learning strategies such as goal setting and environment structuring (e.g., selecting dedicated learning spaces) were also demonstrated to have high predictive power toward learners’ success, second only to learners’ engagement with tasks (Moreno-Marcos et al., 2020). Goal setting and strategic planning predicted attainment of personal course goals (Kizilcec et al., 2017). Employed learners demonstrated higher self-regulated learning skills than the general public and therefore better performance (Hood et al., 2015). Alario-Hoyos et al. (2017) highlighted that the culprit at preventing MOOC completion is not lack of motivation or the wrong kind of motivation, but lack of self-regulation or learning strategies.
Comparing learners with high self-regulation to those of low self-regulation identified that learners’ motivations and goals shaped how they conceptualized the purpose of the MOOC, which in turn affected their perception of the learning process. High self-regulation learners’ goals and motivations were specifically about contributing to their professional capacity, whereas low self-regulation learners described more general motivation and goals, such as interest and desire to learn (Littlejohn et al., 2016). This affected their engagement with the MOOC’s tasks, where high self-regulation learners sought to use the tasks to develop their skills whereas low self-regulation learners used to them to earn a certificate. Students’ metacognition, that is, their awareness of their learning processes and the strategies they engage in to affect them, have also been found to relate to continuance intention to use MOOCs (Tsai et al., 2018).
Learners’ self-regulation can be supported with MOOC design. Learners’ regulatory focus could be aimed either at positive gain (promotion focus) or at avoiding negative outcomes (prevention focus) (Zhang, 2016). When the MOOC promotes study activities to match the learners’ regulatory focus, their motivation increases and these activities were perceived as helpful, however it is not necessarily enjoyable. For example, advocating against negative outcomes through a message saying “If you do not carefully study an [sic] MOOC, it is more likely that you will not pass the final examination because some contents in the final exam will come from the MOOC” (Zhang, 2016, p. 344) was found to better motivate learners with prevention focus, whereas advocating for positive gain, for example, “If you carefully study an [sic] MOOC, you can not only learn much from the MOOC for your final exam (because some contents in the final exam will come from the MOOC), but this could also be beneficial to your life” (Zhang, 2016, p. 344), motivated learners with a promotion regulatory focus.
Other design elements of MOOCs have been shown to affect learners’ motivation, engagement, and performance. Qualitative evidence suggests that reinforcement of online learning with interaction between learners, either in person (Chen & Chen, 2015) or online (Castaño et al., 2015) supports learner motivation, engagement, and performance. Feedback, even automated, also increases motivation when students are alerted if their current and predicted performance is poor (Meier et al., 2016). Designing MOOCs to maintain learners’ situational interest is also beneficial, as learners’ situational interest, which is their interest in the MOOC triggered and potentially maintained by their environment, plays a crucial role in mediating the impact of their intrinsic motivation and their participation on the learners’ performance (de Barba et al., 2016).
Contemporary Themes in MOOC Research
Drawing from a bibliometric base, the previous section reviewing MOOC research tended to focus on those papers with the highest citations levels, which favors the older papers. To redress the balance and bring the review up-to-date, in this section we focus on contemporary (papers published in 2020, 2021, and 2022) themes in MOOC research.
Contemporary themes in MOOC research were explored through an analysis of the author keywords in 1571 articles in the database which were published in 2020 (473), 2021 (546), and 2022 (552). Using the same method described above that surfaced of themes for the whole period, 2009 to 2022, taking a similar approach for 2020 to 2022 yielded a similar six cluster solution on the same topics. This suggests that the six clusters are enduring themes in MOOC research. To identify hotspots for MOOC research, we produced an author keyword network analysis (minimum of 10 occurrences) and placed a temporal overlay on it. This highlights the average age of the articles in which each of the keywords appears (see Figure 5). The lighter colors indicate the most recently appearing author keywords, which highlights the author keywords around deep learning and satisfaction.
Emerging themes in MOOC research: Author keyword network, 2020 to 2022.
Studies with COVID-19 as a keyword are mainly uncritical reports of the development of MOOCs to educate people about the pandemic and to combat misinformation. More critical accounts raise concerns about the lack of one-to-one engagement (Boltz et al., 2021) or need adaptation to the challenges faced by members of a target community (Claflin et al., 2022) or show through an experimental design that MOOCs produce similar outcomes to traditional courses (Zhou et al., 2020). This latter study is also interesting as the number of participants in the MOOC was 60, which illustrates the trend of reducing participant numbers in MOOCs and the use of the “MOOC label” to describe relatively small courses.
The papers on deep learning apply artificial intelligence, neural networks, and algorithms to predict the performance of students, particularly drop-out rates, which is an enduring problem for MOOCs. Such an approach is facilitated by the large numbers of participants and voluminous clickstream data that allows the machine to train on real data. At present, the main lesson from this stream of research is that deep learning improves upon the predictions from other methods cf. Basnet et al., 2022). No implications for the design of MOOCs appear to have emerged. Sentiment analysis is a similar analytical approach whose purpose is to review qualitative data often using algorithms, text analysis, computational linguistics, and natural language processing to discover positive, neutral, or negative attitudes toward the MOOC. Unsurprisingly, these studies show that participants who express negative comments are more likely to drop out (e.g., Mrhar et al., 2021). The contemporary papers with OER as a keyword are very diverse and do not form a cohesive group.
Turning to the keyword “satisfaction,” Albelbisi et al. (2021) found that system quality leads to higher levels of satisfaction. Arquero et al. (2022) found that a participant’s extrinsic motivation (i.e., their reasons for studying the MOOC) outweighed satisfaction, enjoyment, and reputation in terms of shaping their loyalty to the course. In terms of instructional design, Janelli and Lipnevich (2021) found that taking a pre-test prior to the MOOC did not affect learning outcomes, pre-tests negatively influenced persistence, and, for those who completed the MOOC, pre-tests positively affected learning outcomes. Moore and Blackmon (2022) echoes these suggestions and recommended that researchers should emphasize learner intentions and behaviors instead of overall MOOC completion rates.
Taken overall, contemporary studies of MOOCs tend to replicate findings from earlier studies: For example, higher quality production leads to MOOCs being better received, those with reasons to study are more likely to complete, and greater engagement with other students and teachers leads to better outcomes. The primary difference is the emergence of papers looking at computer-aided techniques to analyze the “big data” that MOOCs generate, although these methods are still in their infancy when applied to MOOCs.
Implications for Management Education
In conducting this systematic review of MOOCs, we were surprised how infrequently we came across business and management MOOCs. Most of the MOOCs that have been studied are in the fields of public health initiatives, education, data analytics, computer programming, humanities, science, engineering, and languages. While the general principles related to MOOCs appear to be relevant across disciplines and therefore management education, we also wanted to see what was known about business and management MOOCs. Our survey of MOOC research surfaced 291 papers in journals that are listed in Scopus as business, management or finance, of which 136 papers were published in the period 2020 to 2022 in the discipline of business, management, and accounting. Although findings from these studies replicated those from the rest of the literature (e.g., Razmerita et al., 2020), a few findings stand out that are particular to business and management.
Al-Atabi and Deboer (2014) show that entrepreneurship can be taught by MOOC and that opportunities for collaboration between students is important. Hockerts (2018) replicated the findings for a MOOC in social entrepreneurship and showed that greater engagement in experiential activities improved outcomes; findings replicated by Calvo et al. (2019). Nayar and Koul (2020) compared how the teaching of negotiation skills altered depending on the teaching methods employed. They found that Generation Z learners were more engaged with blended tools (flipped classroom, massive open online courses, independent study fused with role play simulation) than traditional methods (role plays and lectures). Olsson (2016) asked managers and HR-specialists if the openness that makes collaboration public would hinder the use of MOOCs for professional development courses. They said this was not a hindrance and would be happy for their staff to develop through this medium.
Reading the papers included in this review, we were struck by how infrequently researchers unearthed discipline-specific aspects of MOOCs. The teaching of language skills appears to have particular characteristics that lends itself to teaching by MOOC (J. Wang et al., 2018), but other than that, MOOC research has focused on the features of being massive, open, and online rather than the discipline. Hence, this review captures research that is mainly generic with the discipline of study being (largely) irrelevant to the findings: Lessons learned from these studies into MOOCs are applicable across disciplines including business and management.
Perhaps the biggest surprise with the findings from MOOC research is how unsurprising they are. They echo findings known about teaching generally. For example, teachers’ presence throughout the course improves outcomes. The amount and quality of interactions between teachers and students improves outcomes, as does the quality of the teaching materials. Intrinsically motivated students have greater engagement with the course and perform better. Self-regulation is a crucial factor influencing successful course completion. Feedback increases student motivation. Early engagement in learning tasks improves student outcomes. The opening two weeks of the course is critical for establishing student engagement. Regular engagement with the learning resources improves student outcomes. Successful students study more than unsuccessful ones, and the more positive students are about their expectations for the course, the better the outcomes. None of these findings are different to other context forms of teaching such as face-to-face, hybrid, blended, or smaller online courses (e.g., Dean & Fornaciari, 2014; García-Jiménez et al., 2015; Klem & Connell, 2004; Stronge et al., 2007; Sutherland et al., 2018). MOOCs just have larger numbers and universities need to scale-up resources to support and engage students to their expectations. All of these factors are as relevant to management education as they are to any other discipline.
Perhaps the greatest contribution of MOOC research has been accessing large sample sizes and student performance data to examine educational phenomena. In this sense, MOOC research has been useful in confirming much that we already know about teaching and learning. The massiveness of a course only seems to change educational outcomes when it dilutes student-teacher interaction. If, despite a MOOCs massiveness, there is still suitable interaction between students and tutors, it behaves like any other (online) course.
We were unable to surface any significant management education specific findings. Management as a discipline does not appear to have any qualities that make it more or less suitable to MOOCs or the way they are taught. This is good news in one sense because the findings from MOOC research appear applicable to management education. Where the discipline of management is different to others in the university is in the likelihood that MOOCs will emanate from this field. In many universities, business and management are cash cows that support other disciplines (Doherty et al., 2015; Hogan et al., 2021; Pfeffer & Fong, 2004). In such environments, there is likely to be pressure not to give away education that the university is profited from so strongly. We were unable to find any studies looking comparatively across disciplines to discover which disciplines favor MOOCs. Such a study would be particularly interesting if it also revealed the reasoning why particular disciplines opted for MOOCs and why others did not.
Conclusion
Most articles present MOOCs favorably and uncritically and comment on MOOCs’ potential and successes. However, some criticism concerning the learners, the institutions, the theories and their future has been noted in these articles. In particular, the enduring problem with dropout rates refuses to go away (Basnet et al., 2022; Reich & Ruipérez-Valiente, 2019). It seems that MOOCs have not lived up to the hope that they would make high quality education more accessible world-wide, particularly in developing nations. The business model of MOOCs does not support such students as it offers no assistance when learners encounter difficulties (Kalman, 2014). In the US, it was found that most MOOC users were more affluent and better educated than average (Hansen & Reich, 2015), or students in the field already (Swinnerton et al., 2017). Underserved students (e.g., low income, or students of color) were also under-represented in MOOC enrolments, and less likely to complete them (Stich & Reeves, 2017) raising concerns that MOOCs actually worsen societal gaps. MOOCs do not seem to cultivate life-long learning as the vast majority of learners never returned after their first year (Swinnerton et al., 2017). In some cases, the educational outcomes (knowledge, confidence, and satisfaction) of MOOC learners were no better than those of learners in online self-paced modules (Hossain et al., 2015).
MOOCs’ benefits to institutions have also been criticized. The financial overhead required to develop and deliver MOOCs suitable for mass cohorts is considerable and the benefits were not clear (Burd et al., 2015). The future of MOOCs needs them to adopt teaching strategies promoting personalized learning and feedback, but this is problematic when cohorts are massive. By offering accreditation or certification behind pay walls, the number of MOOCs offered by leading platforms, and their revenues, continues to grow (Shah, 2021a). However, hidden in this growth is a decline in the percentage of MOOCs produced by universities, now less than 40% (Shah, 2021a), with the majority being made by other bodies, particularly corporations addressing professional needs such as those delivered as small private online courses (SPOCs).
Universities’ development of MOOCs was largely driven by altruistic values. These courses cost a lot of money to produce (Hollands & Tirthali, 2014), but offered little in the way of direct benefits. Universities were not completely self-sacrificial as they may have hoped for indirect benefits through an enhancement of reputation, being seen to be socially responsible, or increasing their impact with their communities (Ambrosini et al., 2023). However, when the pandemic arrived and many universities faced a financial crisis, such altruistic activity could not be supported. Ironically, universities turned to online ways of teaching to get student income coming in and they did so in a monetized manner. Hence, the shift from MOOCs to SPOCs.
As we have been reviewing these studies, one question stuck remains unanswered: Is there anything different about MOOCs compared to other types of online courses? Their massiveness has lessened as more competition enters the market and as providers increased flexibility and convenience by reducing their length and favoring self-paced over predetermined schedules (Shah, 2018). Their openness has also lessened as providers seek to monetize their investment by putting much assessment and accreditation behind pay walls (Ledwon & Ma, 2022; Shah, 2021a). By and large, and with a few notable exceptions such as those in public health or the teaching of languages, MOOCs appear to have morphed into paid-for online courses. Ten years ago, the first author asked in these pages whether MOOCs would turn out to be a fad or a revolution in management education (Billsberry, 2013). Unless there is a miraculous rebirth sometime soon, the inescapable conclusion to which we find ourselves drawn is that they were a fad. May they rest in peace.
Footnotes
Appendix
The syntax used in the Scopus search was:
(TITLE-ABS-KEY ( mooc OR moocs OR {Massive Online Open Course*} OR {Massive Open Online Course*} ) AND NOT TITLE-ABS-KEY ( optic* OR catalyt* ) ) AND ( LIMIT-TO ( DOCTYPE,"ar" ) OR LIMIT-TO ( DOCTYPE,"ch" ) OR LIMIT-TO ( DOCTYPE,"re" ) OR LIMIT-TO ( DOCTYPE,"ed" ) ) AND ( EXCLUDE ( PUBYEAR,1979) OR EXCLUDE ( PUBYEAR,1986) OR EXCLUDE ( PUBYEAR,1987) OR EXCLUDE ( PUBYEAR,1991) OR EXCLUDE ( PUBYEAR,1992) OR EXCLUDE ( PUBYEAR,1993) OR EXCLUDE ( PUBYEAR,1996) OR EXCLUDE ( PUBYEAR,1997) OR EXCLUDE ( PUBYEAR,2003) OR EXCLUDE ( PUBYEAR,2007) OR EXCLUDE ( PUBYEAR,2023) OR EXCLUDE ( PUBYEAR,2008) ) AND ( LIMIT-TO ( LANGUAGE,"English" ) )
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
