Exploring the Relationship Between Social Interaction,Cognitive Processing and Learning Achievements in a MOOC Discussion Forum

Abstract

In the field of learning analytics, mining the regularities of social interaction and cognitive processing have drawn increasing attention. Nevertheless, in MOOCs, there is a lack of investigations on the combination of social and cognitive behavioral patterns. To fill in this gap, this study aimed to uncover the relationship between social interaction, cognitive processing, and learning achievements in a MOOC discussion forum. Specifically, we collected the 3925 participants’ forum data throughout 16 weeks. Social network analysis and epistemic network analysis were jointly adopted to investigate differences in social interaction, cognitive processing between two achievement groups, and the differences in cognitive processing networks between two types of communities. Finally, moderation analysis was employed to examine the moderating effect of community types between cognitive processing and learning achievements. Results indicated that: (1) the high- and low-achieving groups presented significant differences in terms of degree, betweenness, and eigenvector centrality; (2) the stronger cognitive connections were found within the high-achieving group and the instructor-led community; (3) the cognitive processing indicators including insight, discrepancy, and tentative were significantly negative predictors of learning achievements, whereas inhibition and exclusive were significantly positive predictors; (4) the community type moderated the relationship between cognitive processing and learning achievements.

Keywords

Social network analysis (SNA)epistemic network analysis (ENA)MOOC discussion forum learning achievements cognitive learning SC-LIWC

Get full access to this article

View all access options for this article.

References

Akar

Mardikyan

Dalgic

(2019). User roles in online communities and their moderating effect on online community usage intention: An integrated approach. International Journal of Human–Computer Interaction, 35(6), 495–509. https://doi.org/10.1080/10447318.2018.1465325

Al-Ibrahim

Al-Khalifa

H. S.

(2014). Observing online discussions in educational social networks: A case study. In 2014 international conference on web and open access to learning (ICWOAL) (pp. 1–4). IEEE. https://doi.org/10.1109/ICWOAL.2014.7009189

Almatrafi, O., & Johri, A. (2019). Systematic Review of Discussion Forums in Massive Open Online Courses (MOOCs). IEEE Transactions on Learning Technologies, 12(3), 413–428. https://doi.org/10.1109/TLT.2018.2859304

Barak, M., Watted, A., & Haick, H. (2016). Motivation to learn in massive open online courses: Examining aspects of language and social engagement. Computers & Education, 94, 49–60. https://doi.org/10.1016/j.compedu.2015.11.010

Blondel

V. D.

Guillaume

J.-L.

Lambiotte

Lefebvre

(2008). Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 2008(10), P10008. https://doi.org/10.1088/1742-5468/2008/10/P10008

Bowler

Wm. M.

Halbesleben

J. R. B.

Stodnick

Seevers

M. T.

Little

L. M.

(2009). The Moderating effect of communication network centrality on motive to perform interpersonal citizenship. Journal of Managerial Issues, 21(1), 80–96.

Bressler

D. M.

Bodzin

A. M.

Eagan

Tabatabai

(2019). Using epistemic network analysis to examine discourse and scientific practice during a collaborative game. Journal of Science Education and Technology, 28(5), 553–566. https://doi.org/10.1007/s10956-019-09786-8

Brinton

C. G.

Chiang

Jain

Lam

Liu

Wong

F. M. F.

(2014). Learning about social learning in MOOCs: From statistical analysis to generative model. IEEE Transactions on Learning Technologies, 7(4), 346–359. https://doi.org/10.1109/TLT.2014.2337900

Cai

Eagan

Dowell

N. M.

Pennebaker

J. W.

Shaffer

D. W.

Graesser

A. C.

(2017). Epistemic network analysis and topic modeling for chat data from collaborative learning environment. In International Educational Data Mining Society (pp. 104-111). International Educational Data Mining Society. https://eric.ed.gov/?id=ED596584

10.

Carlson

S. E.

Rees Lewis

D. G.

Gerber

E. M.

Easterday

M. W.

(2018). Challenges of peer instruction in an undergraduate student-led learning community: Bi-directional diffusion as a crucial instructional process. Instructional Science, 46(3), 405–433. https://doi.org/10.1007/s11251-017-9442-0

11.

Carroll

D. W.

(2007). Patterns of student writing in a critical thinking course: A quantitative analysis. Assessing Writing, 12(3), 213–227. https://doi.org/10.1016/j.asw.2008.02.001

12.

Caulfield

Collier

Halawa

(2013). Rethinking online community in MOOCs used for blended learning. https://er.educause.edu/articles/2013/10/rethinking-online-community-in-moocs-used-for-blended-learning

13.

Cazabet

Amblard

(2014). Dynamic community detection. In Alhajj

Rokne

(Eds.), Encyclopedia of social network analysis and mining (pp. 404–414). Springer. https://doi.org/10.1007/978-1-4614-6170-8_383

14.

Chesler

N. C.

Ruis

A. R.

Collier

Swiecki

Arastoopour

Williamson Shaffer

(2015). A novel paradigm for engineering education: Virtual internships with individualized mentoring and assessment of engineering thinking. Journal of Biomechanical Engineering, 137(2), 1–8. https://doi.org/10.1115/1.4029235

15.

Chiu, T. K. F., & Hew, T. K. F. (2018). Factors influencing peer learning and performance in MOOC asynchronous online discussion forum. Australasian Journal of Educational Technology, 34(4), 16–28. https://doi.org/10.14742/ajet.3240

16.

Chouchani

Abed

(2020). Online social network analysis: Detection of communities of interest. Journal of Intelligent Information Systems, 54(1), 5–21. https://doi.org/10.1007/s10844-018-0522-7

17.

Coetzee

Fox

Hearst

M. A.

Hartmann

(2014). Should your MOOC forum use a reputation system? In Proceedings of the 17th ACM conference on computer supported cooperative work & social computing - CSCW ’14 (pp. 1176–1187). ACM. https://doi.org/10.1145/2531602.2531657

18.

Coffrin

Corrin

de Barba

Kennedy

(2014). Visualizing patterns of student engagement and performance in MOOCs. In Proceedings of the fourth international conference on learning analytics and knowledge (pp. 83–92). ACM. https://doi.org/10.1145/2567574.2567586

19.

Csanadi

Eagan

Kollar

Shaffer

D. W.

Fischer

(2018). When coding-and-counting is not enough: Using epistemic network analysis (ENA) to analyze verbal data in CSCL research. International Journal of Computer-Supported Collaborative Learning, 13(4), 419–438. https://doi.org/10.1007/s11412-018-9292-z

20.

Dillenbourg

Jermann

(2010). Technology for classroom orchestration. In Khine

M. S.

Saleh

I. M.

(Eds.), New science of learning (pp. 525–552). Springer. https://doi.org/10.1007/978-1-4419-5716-0_26

21.

Douglas

Zielinski

Merzdorf

Diefes-Dux

Bermel

(2019). Meaningful learner information for MOOC instructors examined through a contextualized evaluation framework. International Review of Research in Open and Distributed Learning, 20(1), 204–220.

22.

Dowell

N. M.

Skrypnyk

Joksimovic

Graesser

A. C.

Dawson

Gaševic

Hennis

T. A.

de Vries

Kovanovic

(2015). Modeling learners’ social centrality and performance through language and discourse. International Educational Data Mining Society. https://eric.ed.gov/?id=ED560532

23.

Ekawati

Kohar

A. W.

Imah

E. M.

Amin

S. M.

Fiangga

(2019). Students’ cognitive processes in solving problem related to the concept of area conservation. Journal on Mathematics Education, 10(1), 21–36. https://doi.org/10.22342/jme.10.1.6339.21-36

24.

Eom

S. B.

Wen

H. J.

Ashill

(2006). The determinants of students’ perceived learning outcomes and satisfaction in university online education: An empirical investigation*. Decision Sciences Journal of Innovative Education, 4(2), 215–235. https://doi.org/10.1111/j.1540-4609.2006.00114.x

25.

Fersini

(2017). Chapter 6—Sentiment analysis in social networks: A machine learning perspective. In Pozzi

F. A.

Fersini

Messina

Liu

(Eds.), Sentiment analysis in social networks (pp. 91–111). Morgan Kaufmann. https://doi.org/10.1016/B978-0-12-804412-4.00006-1

26.

Fougt

S. S.

Siebert-Evenstone

Eagan

Tabatabai

Misfeldt

(2018). Epistemic network analysis of students’ longer written assignments as formative/summative evaluation. In Proceedings of the 8th international conference on learning analytics and knowledge (pp. 126–130). ACM. https://doi.org/10.1145/3170358.3170414

27.

Galikyan

Admiraal

(2019). Students’ engagement in asynchronous online discussion: The relationship between cognitive presence, learner prominence, and academic performance. The Internet and Higher Education, 43, 100692. https://doi.org/10.1016/j.iheduc.2019.100692

28.

Gao

Hao

Gao

Zhu

(2013). Developing simplified chinese psychological linguistic analysis dictionary for microblog. In Imamura

Usui

Shirao

Kasamatsu

Schwabe

Zhong

(Eds.), Brain and health informatics (pp. 359–368). Springer International Publishing. https://doi.org/10.1007/978-3-319-02753-1_36

29.

Garrison

D. R.

Anderson

Archer

(1999). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2), 87–105. https://doi.org/10.1016/S1096-7516(00)00016-6

30.

Gašević

Joksimović

Eagan

B. R.

Shaffer

D. W.

(2019). SENS: Network analytics to combine social and cognitive perspectives of collaborative learning. Computers in Human Behavior, 92, 562–577. https://doi.org/10.1016/j.chb.2018.07.003

31.

Gitinabard, N., Xue, L., Lynch, C. F., Heckman, S., & Barnes, T. (2017). A Social Network Analysis on Blended Courses. The Third Workshop on Graph Based Educational Data Mining (GEDM 2017), 22–26. International Educational Data Mining Society. https://drive.google.com/file/d/0B5dV8wf1DAejUGZ0OGdLTElWRUk/view?resourcekey=0-u5bl2w6mYcUwGQT8ncG-1Q

32.

Gunawardena

C. N.

Lowe

C. A.

Anderson

(1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397–431. https://doi.org/10.2190/7MQV-X9UJ-C7Q3-NRAG

33.

Haavind

Sistek-Chandler

(2015). The emergent role of the MOOC instructor: A qualitative study of trends toward improving future practice. International Journal on E-Learning, 14(3), 331–350.

34.

Hansen

D. L.

Shneiderman

Smith

M. A.

, & Himelboim. (2011). Analyzing social media networks with NodeXL. Elsevier. https://doi.org/10.1016/C2009-0-64028-9

35.

Zhou

(2018). Is participating in MOOC forums important for students? A data-driven study from the perspective of the supernetwork. Journal of Data and Information Science, 3(2), 62–77. https://doi.org/10.2478/jdis-2018-0009

36.

Hew

K. F.

(2015). Student perceptions of peer versus instructor facilitation of asynchronous online discussions: Further findings from three cases. Instructional Science, 43(1), 19–38. https://doi.org/10.1007/s11251-014-9329-2

37.

Houston

S. L.

Brady

Narasimham

Fisher

(2017). Pass the idea please: The relationship between network position, direct engagement, and course performance in MOOCs. In Proceedings of the fourth (2017) ACM conference on learning @ scale - L@S ’17 (pp. 295–298). ACM. https://doi.org/10.1145/3051457.3054008

38.

Donald

Giacaman

Zhu

(2020). Towards automated analysis of cognitive presence in MOOC discussions: A manual classification study. In Proceedings of the tenth international conference on learning analytics & knowledge (pp. 135–140). ACM. https://doi.org/10.1145/3375462.3375473

39.

Huang

Dasgupta

Ghosh

Manning

Sanders

(2014). Superposter behavior in MOOC forums. In Proceedings of the first ACM conference on learning @ scale conference (pp. 117–126). ACM. https://doi.org/10.1145/2556325.2566249

40.

Jacomy

Venturini

Heymann

Bastian

(2014). ForceAtlas2, a continuous graph layout algorithm for handy network visualization designed for the Gephi software. PLoS One, 9(6), e98679. https://doi.org/10.1371/journal.pone.0098679

41.

Jiang, S., Fitzhugh, S. M., Warschauer, M. (2014). Social positioning and performance in MOOCs. In Proceedings of graph-based educational data mining workshop at the 7th international conference on educational data mining (pp. 55–58). CEUR-WS.

42.

Joksimovic

Gasevic

Kovanovic

Adesope

Hatala

(2014). Psychological characteristics in cognitive presence of communities of inquiry: A linguistic analysis of online discussions. The Internet and Higher Education, 22, 1–10. https://doi.org/10.1016/j.iheduc.2014.03.001

43.

Joksimović

Manataki

Gašević

Dawson

Kovanović

de Kereki

I. F.

(2016). Translating network position into performance: Importance of centrality in different network configurations. In Proceedings of the sixth international conference on learning analytics & knowledge (pp. 314–323). ACM. https://doi.org/10.1145/2883851.2883928

44.

Kang

Liu

(2018). Analyzing the relationship among learners’ social characteristics, sentiments in a course forum and learning outcomes. In 2018 seventh international conference of educational innovation through technology (EITT) (pp. 210–213). IEEE. https://doi.org/10.1109/EITT.2018.00049

45.

Kissinger

Bennett

(2014). Using connectivism as a framework for redesigning courses: Social & mobile learning affordances (pp. 1015–1019). https://www.learntechlib.org/primary/p/148813/

46.

Kovanović, V., Joksimović, S., Waters, Z., Gašević, D., Kitto, K., Hatala, M., & Siemens, G. (2016). Towards automated content analysis of discussion transcripts: A cognitive presence case. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 15–24). ACM. https://doi.org/10.1145/2883851.2883950

47.

Laat

Lally

Lipponen

Simons

P. R. J.

(2007). Investigating patterns of interaction in networked learning and computer-supported collaborative learning: A role for social network analysis. International Journal of Computer-Supported Collaborative Learning, 2, 87. https://doi.org/10.1007/s11412-007-9006-4

48.

Lengelle

Meijers

Poell

Post

(2013). The effects of creative, expressive, and reflective writing on career learning: An explorative study. Journal of Vocational Behavior, 83(3), 419–427. https://doi.org/10.1016/j.jvb.2013.06.014

49.

Lin

C.-C.

Tsai

C.-C.

(2012). Participatory learning through behavioral and cognitive engagements in an online collective information searching activity. International Journal of Computer-Supported Collaborative Learning, 7(4), 543–566. https://doi.org/10.1007/s11412-012-9160-1

50.

Lin

Liu

(2016). A social network analysis of teaching and research collaboration in a teachers’ virtual learning community: A social network analysis of research collaboration. British Journal of Educational Technology, 47(2), 302–319. https://doi.org/10.1111/bjet.12234

51.

Liu

Kang

Domanska

Liu

Sun

Fang

(2018). Social network characteristics of learners in a course forum and their relationship to learning outcomes. In Proceedings of the 10th international conference on computer supported education (pp. 15–21). SCITEPRESS. https://doi.org/10.5220/0006647600150021

52.

Liu

X. W.

Zhang

(2020). Diversities of learners’ interactions in different MOOC courses: How these diversities affects communication in learning. Computers & Education, 151, 103873. https://doi.org/10.1016/j.compedu.2020.103873

53.

Mazzolini

Maddison

(2007). When to jump in: The role of the instructor in online discussion forums. Computers & Education, 49(2), 193–213. https://doi.org/10.1016/j.compedu.2005.06.011

54.

McKnight

(2014). Information management. Elsevier. https://doi.org/10.1016/C2012-0-06858-5

55.

Moore

(1989). Three types of interaction. American Journal of Distance Education, 3, 1–7. https://doi.org/10.1080/08923648909526659

56.

Moore

R. L.

Oliver

K. M.

Wang

(2019). Setting the pace: Examining cognitive processing in MOOC discussion forums with automatic text analysis. Interactive Learning Environments, 27(5–6), 655–669. https://doi.org/10.1080/10494820.2019.1610453

57.

Morris

Handcock

M. S.

Hunter

D. R.

(2008). Specification of exponential-family random graph models: Terms and computational aspects. Journal of Statistical Software, 24(4), 1548–7660. https://doi.org/10.18637/jss.v024.i04

58.

Mustafaraj

(2015). The visible and invisible in a MOOC discussion forum. In Proceedings of the second (2015) ACM conference on learning @ scale - L@S ’15 (pp. 351–354). ACM. https://doi.org/10.1145/2724660.2728691

59.

Onah, D. F. O., Sinclair, J. E., & Boyatt, R. (2014). Exploring the Use of MOOC Discussion Forums. In Proceedings of London International Conference on Education(LICE-2014) (pp. 1–4). Infonomic Society. https://doi.org/10.13140/RG.2.1.3319.5042

60.

Osman-Gani

Rockstuhl

(2008). Antecedents and consequences of social network characteristics for expatriate adjustment and performance in overseas assignments: Implications for HRD. Human Resource Development Review, 7, 32–57. https://doi.org/10.1177/1534484307312182

61.

Peden

B. F.

Carroll

D. W.

(2008). Ways of writing: Linguistic analysis of self-assessment and traditional assignments. Teaching of Psychology, 35(4), 313–318. https://doi.org/10.1080/00986280802374419

62.

Pennebaker

J. W.

Chung

C. K.

Ireland

Gonzales

Booth

R. J.

(2007). The development and psychometric properties of LIWC2007. LIWC.net. https://doi.org/10.1109/TC.1980.1675499

63.

Pennebaker

J. W.

Graybeal

(2001). Patterns of natural language use: Disclosure, personality, and social integration. Current Directions in Psychological Science, 10(3), 90–93. https://doi.org/10.1111/1467-8721.00123

64.

Ramesh

Goldwasser

Huang

Daume

Getoor

(2014). Understanding MOOC discussion forums using seeded LDA. In Proceedings of the ninth workshop on innovative use of NLP for building educational applications (pp. 28–33). Association for Computational Linguistics. https://doi.org/10.3115/v1/W14-1804

65.

Reimann

(2009). Time is precious: Variable- and event-centred approaches to process analysis in CSCL research. International Journal of Computer-Supported Collaborative Learning, 4(3), 239–257. https://doi.org/10.1007/s11412-009-9070-z

66.

Saqr

Nouri

(2020). High resolution temporal network analysis to understand and improve collaborative learning. In Proceedings of the tenth international conference on learning analytics & knowledge (pp. 314–319). ACM. https://doi.org/10.1145/3375462.3375501

67.

Seih

Y.-T.

Chung

C. K.

Pennebaker

J. W.

(2011). Experimental manipulations of perspective taking and perspective switching in expressive writing. Cognition & Emotion, 25(5), 926–938. https://doi.org/10.1080/02699931.2010.512123

68.

Shaffer

D. W.

(2006). Epistemic frames for epistemic games. Computers & Education, 46(3), 223–234. https://doi.org/10.1016/j.compedu.2005.11.003

69.

Shaffer

D. W.

Collier

Ruis

A. R.

(2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45. https://doi.org/10.18608/jla.2016.33.3

70.

Shaffer

D. W.

Hatfield

Svarovsky

G. N.

Nash

Nulty

Bagley

Frank

Rupp

A. A.

Mislevy

(2009). Epistemic network analysis: A prototype for 21st-century assessment of learning. International Journal of Learning and Media, 1(2), 33–53. https://doi.org/10.1162/ijlm.2009.0013

71.

Shaffer, D., & Ruis, A. (2017). Epistemic network analysis: A worked example of theory-Ûased learning analytics. In C. Lang, G. Siemens, A. Wise, & D. Gasevic (Eds.), Handbook of learning analytics (pp. 175–187). SOLAR, Society for Learning Analytics and Research.

72.

Sharif

Magrill

(2015). Discussion forums in MOOCs. International Journal of Learning, Teaching and Educational Research, 12(1), Article 1. http://ijlter.org/index.php/ijlter/article/view/368

73.

Swinnerton

Hotchkiss

Morris

N. P.

(2017). Comments in MOOCs: Who is doing the talking and does it help?: Comments in MOOCs. Journal of Computer Assisted Learning, 33(1), 51–64. https://doi.org/10.1111/jcal.12165

74.

Tomkin

J. H.

Charlevoix

(2014). Do professors matter? Using an a/b test to evaluate the impact of instructor involvement on MOOC student outcomes. In Proceedings of the first ACM conference on learning @ scale conference (pp. 71–78). ACM. https://doi.org/10.1145/2556325.2566245

75.

Tsai

C.-C.

(1999). Content analysis of Taiwanese 14 year olds’ information processing operations shown in cognitive structures following physics instruction, with relations to science attainment and scientific epistemological beliefs. Research in Science & Technological Education, 17(2), 125–138. https://doi.org/10.1080/0263514990170201

76.

Tucker

C. S.

Pursel

Divinsky

(2014). Mining student-generated textual data in moocs and quantifying their effects on student performance and learning outcomes. Computers in Education Journal, 5(4), 84–95.

77.

Wang

Yang

Wen

Koedinger

Rosé

C. P.

(2015). Investigating how student’s cognitive behavior in MOOC discussion forums affect learning gains. In International Educational Data Mining Society. (pp. 226–233). International Educational Data Mining Society. https://eric.ed.gov/?id=ED560568

78.

Williams, E. A., Zwolak, J. P., Dou, R., & Brewe, E. (2019). Linking engagement and performance: The social network analysis perspective. Physical Review Physics Education Research, 15(2), 020150. https://doi.org/10.1103/PhysRevPhysEducRes.15.020150

79.

Wise

A. F.

Cui

(2018). Learning communities in the crowd: Characteristics of content related interactions and social relationships in MOOC discussion forums. Computers & Education, 122, 221–242. https://doi.org/10.1016/j.compedu.2018.03.021

80.

Wise

A. F.

Cui

Jin

W. Q.

(2017). Honing in on social learning networks in MOOC forums: Examining critical network definition decisions. In Proceedings of the seventh international learning analytics & knowledge conference (pp. 383–392). ACM. https://doi.org/10.1145/3027385.3027446

81.

Wise

A. F.

Cui

Jin

W. Q.

Vytasek

(2017). Mining for gold: Identifying content-related MOOC discussion threads across domains through linguistic modeling. The Internet and Higher Education, 32, 11–28. https://doi.org/10.1016/j.iheduc.2016.08.001

82.

Liu

Mao

Zhang

(2020). Using epistemic network analysis and self-reported reflections to explore students’ metacognition differences in collaborative learning. Learning and Individual Differences, 82, 101913. https://doi.org/10.1016/j.lindif.2020.101913

83.

Yang

(2014). Students motivation in asynchronous online discussions with MOOC mode. American Journal of Educational Research, 2(5), 325–330. https://doi.org/10.12691/education-2-5-13

84.

Yeh

N.-C.

Lin

H.-P.

(2011). The moderating effect of social roles on user behavior in virtual worlds. Online Information Review, 35, 747–769. https://doi.org/10.1108/14684521111176480

85.

Zhang

Chen

Phang

C. W.

(2018, July 1). Role of instructors’ forum interactions with students in promoting MOOC continuance. Journal of Global Information Management (JGIM). Advance online publication. www.igi-global.com/article/role-of-instructors-forum-interactions-with-students-in-promoting-mooc-continuance/204593

86.

Zhang

Allon

Van Mieghem

J. A.

(2017). Does social interaction improve learning outcomes? Evidence from field experiments on massive open online courses (SSRN Scholarly Paper ID 2663647). Social Science Research Network. Advance online publication. https://doi.org/10.2139/ssrn.2663647

87.

Zhang

Liu

Cai

(2019). Exploring primary school teachers’ technological pedagogical content knowledge (TPACK) in online collaborative discourse: An epistemic network analysis. British Journal of Educational Technology, 50(6), 3437–3455. https://doi.org/10.1111/bjet.12751

88.

Zhao

Zhang

Chiclana

Viedma

E. H.

(2019). An incremental method to detect communities in dynamic evolving social networks. Knowledge-Based Systems, 163, 404–415. https://doi.org/10.1016/j.knosys.2018.09.002

89.

Zou

Pan

Cai

Liu

(2021). Exploring the relationship between social presence and learners’ prestige in MOOC discussion forums using automated content analysis and social network analysis. Computers in Human Behavior, 115, 106582. https://doi.org/10.1016/j.chb.2020.106582