How Different Categories of Gamified Stimuli Affect Massive Open Online Courses Continuance Intention and Learning Performance? Mediating Roles of Internal Experiences

Abstract

Drawing on gamification literature, this study develops a research model to examine whether gamification features as environmental stimulus antecedents to learners’ organismic experiences in using massive open online courses (MOOCs) can affect their response on MOOCs and learning outcomes. The proposed research framework, rooted in the stimulus-organism-response model, provides a strong foundation for understanding these hypothesized relationships. Sample data for this study were collected from learners who had experience in taking the gamified MOOCs provided by the MOOC platform launched by a well-known university in Taiwan, and 307 usable questionnaires were analyzed using structural equation modeling. This study verified that three types of gamification features including achievement-related gamification features, immersion-related gamification features, and social interaction-related gamification features positively influenced learners’ internal experiences in using MOOCs (i.e., cognitive involvement, flow experience, and social presence), which jointly expounded their continuance intention of MOOCs, and this in turn enhanced their perceived impact on learning. Overall, this study’s results offered enough evidence to strongly support all of the hypothesized links and the research model. Besides, the results of the mediation analysis confirmed that learners’ internal experiences and continuance intention of MOOCs fully mediated the effects of their perceived gamification features on perceived impact on learning.

Keywords

massive open online courses gamification features internal experiences continuance intention perceived impact on learning

Get full access to this article

View all access options for this article.

References

Alraimi

K. M.

Ciganek

A. P.

(2015). Understanding the MOOCs continuance: The role of openness and reputation. Computers & Education, 80(January), 28–38. https://doi.org/10.1016/j.compedu.2014.08.006

Anderson

J. C.

Gerbing

D. W.

(1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103(3), 411–423. https://doi.org/10.1037/0033-2909.103.3.411

Aparicio

Oliveira

Bacao

Painho

(2019). Gamification: A key determinant of massive open online course (MOOC) success. Information & Management, 56(1), 39–54. https://doi.org/10.1016/j.im.2018.06.003

Armstrong

J. S.

Overton

T. S.

(1977). Estimating non-response bias in mail surveys. Journal of Marketing Research, 14(3), 396–402. https://doi.org/10.1177/002224377701400320

Arning

Ziefle

(2007). Understanding age differences in PDA acceptance and performance. Computers in Human Behavior, 23(6), 2904–2927. https://doi.org/10.1016/j.chb.2006.06.005

Arpaci

Al-Emran

Al-Sharafi

M. A.

(2020). The impact of knowledge management practices on the acceptance of massive open online courses (MOOCs) by engineering students: A cross-cultural comparison. Telematics and Informatics, 54(November), 101468. https://doi.org/10.1016/j.tele.2020.101468

Aşıksoy

(2018). The effects of the gamified flipped classroom environment (GFCE) on students’ motivation, learning achievements and perception in a physics course. Quality & Quantity: International Journal of Methodology, 52(1), 129–145. https://doi.org/10.1007/s11135-017-0597-1

Bagozzi

R. P.

(1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16(1), 74–94. https://doi.org/10.1007/bf02723327

Bai

Hew

K. F.

Huang

(2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educational Research Review, 30(June), 100322. https://doi.org/10.1016/j.edurev.2020.100322

10.

Bhattacherjee

(2001). Understanding information systems continuance: An expectation–confirmation model. MIS Quarterly, 25(3), 351–370. https://doi.org/10.2307/3250921

11.

Bitrián

Buil

Catalán

(2021). Enhancing user engagement: The role of gamification in mobile apps. Journal of Business Research, 132(August), 170–185. https://doi.org/10.1016/j.jbusres.2021.04.028

12.

Bollen

K. A.

(1989). Structural equations with latent variables. John Wiley & Sons.

13.

Campbell

D. E.

Wright

R. T.

Clay

P. F.

(2010). Deconstructing and operationalizing interactivity: An online advertising perspective. Journal of Information Technology Theory and Application, 11(4), 29–53.

14.

Castellanos-Reyes

(2021). The dynamics of a MOOC’s learner-learner interaction over time: A longitudinal network analysis. Computers in Human Behavior, 123(3), 106880. https://doi.org/10.1016/j.chb.2021.106880

15.

Chang

C. C.

(2013). Examining users’ intention to continue using social network games: A flow experience perspective. Telematics and Informatics, 30(4), 311–321. https://doi.org/10.1016/j.tele.2012.10.006

16.

Chang

J. W.

Wie

H. Y.

(2016). Exploring engaging gamification mechanics in massive online open courses. Journal of Educational Technology & Society, 19(2), 177–203.

17.

Chen

S.-Y.

(2016). Using the sustainable modified TAM and TPB to analyze the effects of perceived green value on loyalty to a public bike system. Transportation Research Part A: Policy and Practice, 88(June), 58–72. https://doi.org/10.1016/j.tra.2016.03.008

18.

Cheng

Y.-M.

(2019). How does task-technology fit influence cloud-based e-learning continuance and impact? Education + Training, 61(4), 480–499. https://doi.org/10.1108/et-09-2018-0203

19.

Cheng

Y.-M.

(2021). Can gamification and interface design aesthetics lead to MOOCs’ success? Education + Training, 63(9), 1346–1375. https://doi.org/10.1108/et-09-2020-0278

20.

Cheung

C. M. K.

Chiu

P. Y.

Lee

M. K. O.

(2011). Online social networks: Why do students use Facebook? Computers in Human Behavior, 27(4), 1337–1343. https://doi.org/10.1016/j.chb.2010.07.028

21.

Cronbach

L. J.

(1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/bf02310555

22.

Cronbach

L. J.

(1971). Test validation. In Thorndike

R. L.

(Ed.), Educational measurement (2nd ed., pp. 443–507). American Council on Education.

23.

Csikszentmihalyi

(1975). Beyond boredom and anxiety: Experiencing flow in work and play. Jossey-Bass Publishers.

24.

Dai

H. M.

Teo

Rappa

N. A.

(2020a). Understanding continuance intention among MOOC participants: The role of habit and MOOC performance. Computers in Human Behavior, 112(November), 106455. https://doi.org/10.1016/j.chb.2020.106455

25.

Dai

H. M.

Teo

Rappa

N. A.

Huang

(2020b). Explaining Chinese university students’ continuance learning intention in the MOOC setting: A modified expectation confirmation model perspective. Computers & Education, 150(1), 103850. https://doi.org/10.1016/j.compedu.2020.103850

26.

Dellarocas

Van Alstyne

(2013). Money models for MOOCs. Communications of the ACM, 56(8), 25–28. https://doi.org/10.1145/2492007.2492017

27.

Deng

Benckendorff

Gannaway

(2019). Progress and new directions for teaching and learning in MOOCs. Computers & Education, 129(February), 48–60. https://doi.org/10.1016/j.compedu.2018.10.019

28.

Deterding

Dixon

Khaled

Nacke

(2011). From game design elements to gamefulness: Defining gamification. In Lugmayr

E. A.

(Ed.), Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (pp. 9–15), Tampere, Finland, September 29–30, 2011. ACM. https://doi.org/10.1145/2181037.2181040

29.

Dichev

Dicheva

(2017). Gamifying education: What is known, what is believed and what remains uncertain: A critical review. International Journal of Educational Technology in Higher Education, 14(9), 1–36. https://doi.org/10.1186/s41239-017-0042-5

30.

Drossos

D. A.

Kokkinaki

Giaglis

G. M.

Fouskas

K. G.

(2014). The effects of product involvement and impulse buying on purchase intentions in mobile text advertising. Electronic Commerce Research and Applications, 13(6), 423–430. https://doi.org/10.1016/j.elerap.2014.08.003

31.

El Kabtane

El Adnani

Sadgal

Mourdi

(2019). Augmented reality-based approach for interactivity in MOOCs. International Journal of Web Information Systems, 15(2), 134–154. https://doi.org/10.1108/ijwis-04-2018-0033

32.

Fianu

Blewett

Ampong

G. O.

(2020). Toward the development of a model of student usage of MOOCs. Education + Training, 62(5), 521–541. https://doi.org/10.1108/et-11-2019-0262

33.

Fornell

Larcker

D. F.

(1981). Evaluating structural equation models with unobservable and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.2307/3151312

34.

Fortin

D. R.

Dholakia

R. R.

(2005). Interactivity and vividness effects on social presence and involvement with a web-based advertisement. Journal of Business Research, 58(3), 387–396. https://doi.org/10.1016/s0148-2963(03)00106-1

35.

F.-L.

Y.-L.

H.-C.

(2009). An investigation of coopetitive pedagogic design for knowledge creation in web-based learning. Computers & Education, 53(3), 550–562. https://doi.org/10.1016/j.compedu.2009.01.004

36.

Gefen

Straub

D. W.

(2004). Consumer trust in B2C e-commerce and the importance of social presence: Experiments in e-products and e-services. Omega, 32(6), 407–424. https://doi.org/10.1016/j.omega.2004.01.006

37.

Goodhue

D. L.

Thompson

R. L.

(1995). Task-technology fit and individual performance. MIS Quarterly, 19(2), 213–236. https://doi.org/10.2307/249689

38.

Guo

Y. M.

Poole

M. S.

(2009). Antecedents of flow in online shopping: A test of alternative models. Information Systems Journal, 19(4), 369–390. https://doi.org/10.1111/j.1365-2575.2007.00292.x

39.

Hair

J. F.

Jr. Black

W. C.

Babin

B. J.

Anderson

R. E.

(2010). Multivariate data analysis: A global perspective (7th ed.). Pearson Education.

40.

Harman

H. H.

(1976). Modern factor analysis (3rd ed. revised). The University of Chicago Press.

41.

Hassan

Dias

Hamari

(2019). How motivational feedback increases user’s benefits and continued use: A study on gamification, quantified-self and social networking. International Journal of Information Management, 46(June), 151–162. https://doi.org/10.1016/j.ijinfomgt.2018.12.004

42.

Hassanein

Head

M. M.

(2006). The impact of infusing social presence in the web interface: An investigation across product types. International Journal of Electronic Commerce, 10(2), 31–55. https://doi.org/10.2753/jec1086-4415100202

43.

Hayes

A. F.

(2009). Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Communication Monographs, 76(4), 408–420. https://doi.org/10.1080/03637750903310360

44.

Hsu

J.-Y.

Chen

C.-C.

Ting

P.-F.

(2018). Understanding MOOC continuance: An empirical examination of social support theory. Interactive Learning Environments, 26(8), 1100–1118. https://doi.org/10.1080/10494820.2018.1446990

45.

Hsu

Y.-P.

(2021). Technological innovation and massive open online courses (MOOCs) in Taiwan higher education. In Hou

A.Y.-C.

Chiang

T.-L.

Chan

S.-J.

(Eds.), Higher education in Taiwan (pp. 183–197). Springer-Verlag. https://doi.org/10.1007/978-981-15-4554-2_10

46.

Huang

(2012). Online experiences and virtual goods purchase intention. Internet Research, 22(3), 252–274. https://doi.org/10.1108/10662241211235644

47.

Huang

Zhang

Liu

(2017). Antecedents of student MOOC revisit intention: Moderation effect of course difficulty. International Journal of Information Management, 37(2), 84–91. https://doi.org/10.1016/j.ijinfomgt.2016.12.002

48.

Hunicke

LeBlanc

Zubek

(2004). Mda: A formal approach to game design and game research. In Proceedings of the AAAI Workshop on Challenges in Game AI (pp. 17–22), July 25–26, 2004. San Jose McEnery Convention Center.

49.

Jang

Kitchen

P. J.

Kim

(2018). The effects of gamified customer benefits and characteristics on behavioral engagement and purchase: Evidence from mobile exercise application uses. Journal of Business Research, 92(November), 250–259. https://doi.org/10.1016/j.jbusres.2018.07.056

50.

Jiang

Chan

Tan

B. C.

Chua

W. S.

(2010). Effects of interactivity on website involvement and purchase intention. Journal of the Association for Information Systems, 11(1), 34–59. https://doi.org/10.17705/1jais.00218

51.

Kang

J.-Y. M.

Mun

J. M.

Johnson

K. K. P.

(2015). In-store mobile usage: Downloading and usage intention toward mobile location-based retail apps. Computers in Human Behavior, 46(May), 210–217. https://doi.org/10.1016/j.chb.2015.01.012

52.

Kasurinen

Knutas

(2018). Publication trends in gamification: A systematic mapping study. Computer Science Review, 27(February), 33–44. https://doi.org/10.1016/j.cosrev.2017.10.003

53.

Khalil

Wong

de Koning

Ebner

Paas

(2018). Gamification in MOOCs: A review of the state of the art. In Proceedings of the 2018 IEEE global engineering education conference (EDUCON) (pp. 1629–1638), Santa Cruz de Tenerife, Canary Islands, April 17–20, 2018. IEEE.

54.

Kiili

(2005a). Content creation challenges and flow experience in educational games: The IT-emperor case. The Internet and Higher Education, 8(3), 183–198. https://doi.org/10.1016/j.iheduc.2005.06.001

55.

Kiili

(2005b). Digital games-based learning: Towards an experiential gaming model. The Internet and Higher Education, 8(1), 13–24. https://doi.org/10.1016/j.iheduc.2004.12.001

56.

Kim

Sung

(2009). Dimensions of purchase-decision involvement: Affective and cognitive involvement in product and brand. Journal of Brand Management, 16(8), 504–519. https://doi.org/10.1057/bm.2008.39

57.

Kline

R. B.

(2011). Principles and practice of structural equation modeling (3rd ed.). Guilford Press.

58.

Koivisto

Hamari

(2019). The rise of motivational information systems: A review of gamification research. International Journal of Information Management, 45(April), 191–210. https://doi.org/10.1016/j.ijinfomgt.2018.10.013

59.

Lambert

S. R.

(2020). Do MOOCs contribute to student equity and social inclusion? A systematic review 2014–18. Computers & Education, 145(February), 103693. https://doi.org/10.1016/j.compedu.2019.103693

60.

Wang

Tan

S. C.

(2018). What makes MOOC users persist in completing MOOCs? A perspective from network externalities and human factors. Computers in Human Behavior, 85(August), 385–395. https://doi.org/10.1016/j.chb.2018.04.028

61.

Lin

F.-R.

Kao

C.-M.

(2018). Mental effort detection using EEG data in E-learning contexts. Computers & Education, 122(July), 63–79. https://doi.org/10.1016/j.compedu.2018.03.020

62.

Liyanagunawardena

T. R.

Adams

A. A.

Williams

S. A.

(2013). MOOCs: A systematic study of the published literature 2008-2012. International Review of Research in Open and Distributed Learning, 14(3), 202–227. https://doi.org/10.19173/irrodl.v14i3.1455

63.

Mardia

K. V.

(1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57(3), 519–530. https://doi.org/10.1093/biomet/57.3.519

64.

McGill

T. J.

Klobas

J. E.

(2009). A task–technology fit view of learning management system impact. Computers & Education, 52(2), 496–508. https://doi.org/10.1016/j.compedu.2008.10.002

65.

Mehrabian

Russell

J. A.

(1974). An approach to environmental psychology. The MIT Press.

66.

Metwally

A. H. S.

Nacke

L. E.

Chang

Wang

Yousef

A. M. F.

(2021). Revealing the hotspots of educational gamification: An umbrella review. International Journal of Educational Research, 109(August), 101832. https://doi.org/10.1016/j.ijer.2021.101832

67.

Michinov

Brunot

Le Bohec

Juhel

Delaval

(2011). Procrastination, participation, and performance in online learning environments. Computers & Education, 56(1), 243–252. https://doi.org/10.1016/j.compedu.2010.07.025

68.

Ngan

O. M. Y.

Tang

T. L. H.

Chan

A. K. Y.

Chen

D. M.

Tang

M. K.

(2018). Blended Learning in anatomy teaching for non-medical students: An innovative approach to the health professions education. Health Professions Education, 4(2), 149–158. https://doi.org/10.1016/j.hpe.2017.11.001

69.

Novak

T. P.

Hoffman

D. L.

Yung

Y. F.

(2000). Measuring the customer experience in online environments: A structural modeling approach. Marketing Science, 19(1), 22–44. https://doi.org/10.1287/mksc.19.1.22.15184

70.

Nunnally

J. C.

(1978). Psychometric theory (2nd ed.). McGraw-Hill.

71.

Pavlou

P. A.

Liang

Xue

(2007). Understanding and mitigating uncertainty in online exchange relationships: A principal-agent perspective. MIS Quarterly, 31(1), 105–136. https://doi.org/10.2307/25148783

72.

Pérez-López

Contero

(2013). Delivering educational multimedia contents through an augmented reality application: A case study on its impact on knowledge acquisition and retention. Turkish Online Journal of Educational Technology, 12(4), 19–28.

73.

Podsakoff

P. M.

MacKenzie

S. B.

Lee

J.-Y.

Podsakoff

N. P.

(2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879

74.

Poondej

Lerdpornkulrat

(2020). Gamification in e-learning: A moodle implementation and its effect on student engagement and performance. Interactive Technology and Smart Education, 17(1), 56–66. https://doi.org/10.1108/itse-06-2019-0030

75.

Preacher

K. J.

Hayes

A. F.

(2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40(3), 879–891. https://doi.org/10.3758/brm.40.3.879

76.

Reychav

(2015). Are your users actively involved? A cognitive absorption perspective in mobile training. Computers in Human Behavior, 44(March), 335–346. https://doi.org/10.1016/j.chb.2014.09.021

77.

Rodrigues

L. F.

Oliveira

Costa

C. J.

(2016). Does ease-of-use contributes to the perception of enjoyment? A case of gamification in e-banking. Computers in Human Behavior, 61(August), 114–126. https://doi.org/10.1016/j.chb.2016.03.015

78.

Ryan

R. M.

Deci

E. L.

(2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67. https://doi.org/10.1006/ceps.1999.1020

79.

Sanchez

J. I.

Korbin

W. P.

Viscarra

D. M.

(1995). Corporate support in the aftermath of a natural disaster: Effects on employee strains. Academy of Management Journal, 38(2), 504–521. https://doi.org/10.5465/256690

80.

Saputro

R. E.

Salam

Zakaria

M. H.

Anwar

(2019). A gamification framework to enhance students’ intrinsic motivation on MOOC. Telkomnika, 17(1), 170–178. https://doi.org/10.12928/telkomnika.v17i1.10090

81.

Shao

Chen

(2021). Understanding individuals’ engagement and continuance intention of MOOCs: The effect of interactivity and the role of gender. Internet Research, 31(4), 1262–1289. https://doi.org/10.1108/intr-10-2019-0416

82.

Short

Williams

Christie

(1976). The social psychology of telecommunications. Wiley.

83.

Smith

J. A.

Sivo

S. A.

(2012). Predicting continued use of online teacher professional development and the influence of social presence and sociability. British Journal of Educational Technology, 43(6), 871–882. https://doi.org/10.1111/j.1467-8535.2011.01223.x

84.

Triantoro

Gopal

Benbunan-Fich

Lang

(2019). Would you like to play? A comparison of a gamified survey with a traditional online survey method. International Journal of Information Management, 49(3), 242–252. https://doi.org/10.1016/j.ijinfomgt.2019.06.001

85.

Tsai

Y.-H.

Lin

C.-H.

Hong

J.-C.

Tai

K.-H.

(2018). The effects of metacognition on online learning interest and continuance to learn with MOOCs. Computers & Education, 121(June), 18–29. https://doi.org/10.1016/j.compedu.2018.02.011

86.

Chen

(2017). Continuance intention to use MOOCs: Integrating the technology acceptance model (TAM) and task technology fit (TTF) model. Computers in Human Behavior, 67(February), 221–232. https://doi.org/10.1016/j.chb.2016.10.028

87.

Hamari

(2020). Does gamification affect brand engagement and equity? A study in online brand communities. Journal of Business Research, 109(March), 449–460. https://doi.org/10.1016/j.jbusres.2019.11.058

88.

Yang

S. J. H.

Huang

J. C. H.

Huang

A. Y. Q.

(2017). MOOCs in Taiwan: The movement and experiences. In Jemni

Kinshuk Khribi

M. K.

(Eds.), Open education: From OERs to MOOCs (pp. 101–116). VerlagSpringer.

89.

Yang

S. J. H.

Huang

J. C. H.

Huang

A. Y. Q.

Chen

I. Y. L.

(2016). MOOCs for K-12 and higher education in Taiwan. In 2016 5th IIAI international congress on advanced applied informatics (IIAI-AAI), Kumamoto, Japan, 10–14 July 2016. IEEE. https://doi.org/10.1109/IIAI-AAI.2016.42

90.

Yousef

A. M. F.

Sumner

(2021). Reflections on the last decade of MOOC research. Computer Applications in Engineering Education, 29(4), 648–665. https://doi.org/10.1002/cae.22334

91.

Zaichkowsky

J. L.

(1985). Measuring the involvement construct. Journal of Consumer Research, 12(3), 341–352. https://doi.org/10.1086/208520

92.

Zaichkowsky

J. L.

(1994). The personal involvement inventory: Reduction, revision, and application to advertising. Journal of Advertising, 23(4), 59–70. https://doi.org/10.1080/00913367.1943.10673459

93.

Zainuddin

Chu

S. K. W.

Shujahat

Perera

C. J.

(2020). The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educational Research Review, 30(June), 100326. https://doi.org/10.1016/j.edurev.2020.100326

94.

Zhang

Gupta

Zhao

(2014). What motivates customers to participate in social commerce? The impact of technological environments and virtual customer experiences. Information & Management, 51(8), 1017–1030. https://doi.org/10.1016/j.im.2014.07.005

95.

Zhang

Shao

Feng

(2021). Gamification and online impulse buying: The moderating effect of gender and age. International Journal of Information Management, 61(December), 102267. https://doi.org/10.1016/j.ijinfomgt.2020.102267

96.

Zhao

Yang

Shu

Liu

(2021). Sustainability orientation, the adoption of 3D printing technologies, and new product performance: A cross-institutional study of American and Indian firms. Technovation, 101(March), 102197. https://doi.org/10.1016/j.technovation.2020.102197

97.

Zhao

Wang

Sun

(2020). Technological environment, virtual experience, and MOOC continuance: A stimulus–organism–response perspective. Computers & Education, 144(January), 103721. https://doi.org/10.1016/j.compedu.2019.103721