Forum discussions in Massive Open Online Courses (MOOCs) play a crucial role in promoting learning engagement and academic achievement. In particular, discussion topics significantly influence learners’ emotional and cognitive engagement. However, the complex interrelationships among these factors remain underexplored. This study introduces an innovative two-step methodological approach to investigate the relationships between topic complexity, emotional engagement, cognitive engagement, and academic achievement in MOOC discussions. Using BERT for engagement detection and developing a Joint Emotion and Cognition Topic Model (JECTM) based on Bayesian networks, we analyzed 27,428 discussion posts from 2857 learners in a psychology MOOC. Our findings reveal three key insights: (1) The proposed two-step approach efficiently detects topics and analyzes their patterns of emotional and cognitive engagement. (2) As topic complexity increases, learners demonstrate higher-order cognitive engagement while experiencing reduced positive emotions along with increased confused and negative emotions. (3) In high-complexity topics, learners who maintain both positive emotions and higher-order cognitive engagement are more likely to achieve academic success than those who have negative emotions or lower-order cognitive engagement. These fine-grained analyses provide valuable insights for optimizing discussion design and interventions. This study also provides implications for the analysis of classroom dialogues and AI tutor-based conversations.
AtapattuT.ThilakaratneM.VivianR.FalknerK. (2019). Detecting cognitive engagement using word embeddings within an online teacher professional development community. Computers & Education, 140(20), Article 103594. https://doi.org/10.1016/j.compedu.2019.05.020
2.
BirjaliM.KasriM.Beni-HssaneA. (2021). A comprehensive survey on sentiment analysis: Approaches, challenges and trends. Knowledge-Based Systems, 226(8), Article 107134. https://doi.org/10.1016/j.knosys.2021.107134
3.
BleiD. M.NgA. Y.JordanM. I. (2003). Latent Dirichlet allocation. The Journal of Machine Learning Research, 3(2), 993–1022.
4.
BloomB. S. (1956). Taxonomy of educational objectives: The classification of educational goals, 1st ed.Longman Group.
5.
BuhrE. E.DanielsL. M.GoeganL. D. (2019). Cognitive appraisals mediate relationships between two basic psychological needs and emotions in a massive open online course. Computers in Human Behavior, 96(9), 85–94. https://doi.org/10.1016/j.chb.2019.02.009
6.
ChenC.-M.LiM.-C.ChangW.-C.ChenX.-X. (2021). Developing a topic analysis instant feedback system to facilitate asynchronous online discussion effectiveness. Computers & Education, 163(4), Article 104095. https://doi.org/10.1016/j.compedu.2020.104095
7.
ChenG.LoC. K.HuL. (2020). Sustaining online academic discussions: Identifying the characteristics of messages that receive responses. Computers & Education, 156(1), Article 103938. https://doi.org/10.1016/j.compedu.2020.103938
8.
ChevrierM.MuisK. R.TrevorsG. J.PekrunR.SinatraG. M. (2019). Exploring the antecedents and consequences of epistemic emotions. Learning and Instruction, 63(6), Article 101209. https://doi.org/10.1016/j.learninstruc.2019.05.006
9.
ChiM. T. H.WylieR. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243. https://doi.org/10.1080/00461520.2014.965823
DalipiF.ZdravkovaK.AhlgrenF. (2021). Sentiment analysis of students’ feedback in MOOCs: A systematic literature review. Frontiers in Artificial Intelligence, 4(1), Article 728708. https://doi.org/10.3389/frai.2021.728708
12.
DemirÖ.CinarM.KeskinS. (2023). Participation style and social anxiety as predictors of active participation in asynchronous discussion forums and academic achievement. Education and Information Technologies, 28(9), 1–22. https://doi.org/10.1007/s10639-022-11517-3
13.
DevlinJ.ChangM.-W.LeeK.ToutanovaK. (2019). Bert: Pre-Training of deep bidirectional Transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis USA, June 2-7, pp. 4171–4186.
DillonJ.BoschN.ChetlurM.WanigasekaraN.AmbroseG. A.SenguptaB.D’MelloS. K. (2016). Student emotion, Co-occurrence, and dropout in a MOOC context. International Educational Data Mining Society. https://eric.ed.gov/?id=ED592723
16.
DingL. (2019). Applying gamifications to asynchronous online discussions: A mixed methods study. Computers in Human Behavior, 91(9), 1–11. https://doi.org/10.1016/j.chb.2018.09.022
17.
D’MelloS.DieterleE.DuckworthA. (2017). Advanced, analytic, automated (AAA) measurement of engagement during learning. Educational Psychologist, 52(2), 104–123. https://doi.org/10.1080/00461520.2017.1281747
DooM. Y.KimJ. (2024). The relationship between learning engagement and learning outcomes in online learning in higher education: A meta-analysis study. Distance Education, 45(1), 60–82. https://doi.org/10.1080/01587919.2024.2303484
20.
DuboviI. (2022). Cognitive and emotional engagement while learning with VR: The perspective of multimodal methodology. Computers & Education, 183(4), Article 104495. https://doi.org/10.1016/j.compedu.2022.104495
21.
ErtmerP. A.SadafA.ErtmerD. J. (2011). Student-content interactions in online courses: The role of question prompts in facilitating higher-level engagement with course content. Journal of Computing in Higher Education, 23(2–3), 157–186. https://doi.org/10.1007/s12528-011-9047-6
22.
GaševićD.JoksimovićS.EaganB. R.ShafferD. W. (2018). SENS: Network analytics to combine social and cognitive perspectives of collaborative learning. Computers in Human Behavior, 92(3), 562–577. https://doi.org/10.1016/j.chb.2018.07.003
23.
GaševićD.KovanovićV.JoksimovićS. (2017). Piecing the learning analytics puzzle: A consolidated model of a field of research and practice. Learning: Research and Practice, 3(1), 63–78. https://doi.org/10.1080/23735082.2017.1286142
24.
GengS.NiuB.FengY.HuangM. (2020). Understanding the focal points and sentiment of learners in MOOC reviews: A machine learning and SC-LIWC-based approach. British Journal of Educational Technology, 51(5), 1785–1803. https://doi.org/10.1111/bjet.12999
25.
GraesserA. C. (2020). Emotions are the experiential glue of learning environments in the 21st century. Learning and Instruction, 70(9), Article 101212. https://doi.org/10.1016/j.learninstruc.2019.05.009
26.
HanZ.-M.HuangC.-Q.YuJ.-H.TsaiC.-C. (2021). Identifying patterns of epistemic emotions with respect to interactions in massive online open courses using deep learning and social network analysis. Computers in Human Behavior, 122(18), Article 106843. https://doi.org/10.1016/j.chb.2021.106843
HouC.ZhuG.ZhengJ.ZhangL.HuangX.ZhongT.LiS.DuH.KerC. L. (2024). Prompt-based and fine-tuned GPT models for context-dependent and -independent deductive coding in social annotation. In Proceedings of the 14th Learning Analytics and Knowledge Conference, Tokyo Japan, March18-22, pp. 518–528. https://doi.org/10.1145/3636555.3636910
IqbalS.RakovicM.ChenG.LiT.Ferreira MelloR.FanY.FiorentinoG.Radi AljohaniN.GasevicD. (2023). Towards automated analysis of rhetorical categories in students essay writings using Bloom’s taxonomy. In LAK23: 13th International Learning Analytics and Knowledge Conference, Austin USA,March 13-17, pp. 418–429. https://doi.org/10.1145/3576050.3576112
31.
KapurM. (2011). Temporality matters: Advancing a method for analyzing problem-solving processes in a computer-supported collaborative environment. International Journal of Computer-Supported Collaborative Learning, 6(1), 39–56. https://doi.org/10.1007/s11412-011-9109-9
32.
KongX.LiuZ.ChenC.LiuS.XuZ.TangQ. (2025). Exploratory study of an AI-supported discussion representational tool for online collaborative learning in a Chinese university. The Internet and Higher Education, 64(2), Article 100973. https://doi.org/10.1016/j.iheduc.2024.100973
LauJ. H.NewmanD.BaldwinT. (2014). Machine reading tea leaves: Automatically evaluating topic coherence and topic model quality. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, Gothenburg Sweden, April 26-30, pp. 530–539. https://doi.org/10.3115/v1/E14-1056
35.
LaveJ.WengerE. (1991). Situated Learning: Legitimate Peripheral participation. Cambridge University Press.
36.
LeeJ.-E.ReckerM. (2021). The effects of instructors’ use of online discussions strategies on student participation and performance in university online introductory mathematics courses. Computers & Education, 162(5), Article 104084. https://doi.org/10.1016/j.compedu.2020.104084
37.
LiuS.LiuS.LiuZ.PengX.YangZ. (2022). Automated detection of emotional and cognitive engagement in MOOC discussions to predict learning achievement. Computers & Education, 181(9), Article 104461. https://doi.org/10.1016/j.compedu.2022.104461
38.
LiuS.LiuS.ShaL.ZengZ.GasevicD.LiuZ. (2024). Annotation Guidelines-based knowledge Augmentation: Towards enhancing large language models for educational text classificationNo. arXiv:2406.00954. arXiv. https://doi.org/10.48550/arXiv.2406.00954
39.
LiuZ.LiuS.ZhangC.SuZ.HuT.LiuS. (2020). Investigating the relationship between learners’ cognitive participation and learning outcome in asynchronous online discussion forums. In Proceedings of the 12th International Conference on Computer Supported Education, Prague Czech Republic, May 2-4, pp. 26–33.
40.
LiuZ.TangQ.OuyangF.LongT.LiuS. (2024). Profiling students’ learning engagement in MOOC discussions to identify learning achievement: An automated configurational approach. Computers & Education, 219(9), Article 105109. https://doi.org/10.1016/j.compedu.2024.105109
41.
LiuZ.YangC.RüdianS.LiuS.ZhaoL.WangT. (2019). Temporal emotion-aspect modeling for discovering what students are concerned about in online course forums. Interactive Learning Environments, 27(5–6), 598–627. https://doi.org/10.1080/10494820.2019.1610449
42.
MalinenS. (2015). Understanding user participation in online communities: A systematic literature review of empirical studies. Computers in Human Behavior, 46(4), 228–238. https://doi.org/10.1016/j.chb.2015.01.004
43.
MerriënboerJ. J. G. V.SwellerJ. (2005). Cognitive load theory and complex learning: Recent developments & future directions. Educational Psychology Review, 17(5), Article 147177.
44.
MohammedM.OmarN. (2020). Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec. PLoS One, 15(3), Article e0230442. https://doi.org/10.1371/journal.pone.0230442
45.
NewmanD.LauJ. H.GrieserK.BaldwinT. (2010). Automatic evaluation of topic coherence. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Los Angeles USA, June 1–6, pp. 100–108. https://www.aclweb.org/anthology/N10-1012
46.
NistorN.DascaluM.TarnaiC.Trausan-MatuS. (2020). Predicting newcomer integration in online learning communities: Automated dialog assessment in blogger communities. Computers in Human Behavior, 105(12), Article 106202. https://doi.org/10.1016/j.chb.2019.106202
47.
OnanA.ToçoğluM. (2020). Weighted word embeddings and clustering‐based identification of question topics in MOOC discussion forum posts. Computer Applications in Engineering Education, 29(4), 675–689. https://doi.org/10.1002/cae.22252
48.
O’RiordanT.MillardD. E.SchulzJ. (2021). Is critical thinking happening? Testing content analysis schemes applied to MOOC discussion forums. Computer Applications in Engineering Education, 29(4), 690–709. https://doi.org/10.1002/cae.22314
49.
OrjiF. A.VassilevaJ. (2023). Predicting the persuasiveness of influence strategies from student online learning behaviour using machine learning methods. Journal of Educational Computing Research, 61(7), 1410–1429. https://doi.org/10.1177/07356331231178873
50.
OuyangF. (2021). Using three social network analysis approaches to understand computer-supported collaborative learning. Journal of Educational Computing Research, 59(7), 1401–1424. https://doi.org/10.1177/0735633121996477
51.
OuyangF.ChenS.LiX. (2021). Effect of three network visualizations on students’ social-cognitive engagement in online discussions. British Journal of Educational Technology, 52(6), 2242–2262. https://doi.org/10.1111/bjet.13126
52.
PanX.HuB.ZhouZ.FengX. (2022). Are students happier the more they learn? – Research on the influence of course progress on academic emotion in online learning. Interactive Learning Environments, 31(10), 1–21. https://doi.org/10.1080/10494820.2022.2052110
53.
PekrunR.Linnenbrink-GarciaL. (2012). Academic emotions and student engagement. In ChristensonS. L.ReschlyA. L.WylieC. (Eds.), Handbook of research on student engagement (pp. 259–282). Springer US. https://doi.org/10.1007/978-1-4614-2018-7_12
54.
PengX.HanC.OuyangF.LiuZ. (2020). Topic tracking model for analyzing student-generated posts in SPOC discussion forums. International Journal of Educational Technology in Higher Education, 17(1), 35. https://doi.org/10.1186/s41239-020-00211-4
55.
PengX.XuQ. (2020). Investigating learners’ behaviors and discourse content in MOOC course reviews. Computers & Education, 143, Article 103673. https://doi.org/10.1016/j.compedu.2019.103673
56.
QiuM.ZhuF.JiangJ. (2013). It is not just what we say, but how we say them: LDA-based behavior-topic model. In Proceedings of the 2013 SIAM International Conference on Data Mining, Austin USA, May 2-4, pp. 794–802. https://doi.org/10.1137/1.9781611972832.88
57.
Ramirez-ArellanoA.Bory-ReyesJ.Hernández-SimónL. M. (2019). Emotions, motivation, cognitive–metacognitive strategies, and behavior as predictors of learning performance in blended learning. Journal of Educational Computing Research, 57(2), 491–512. https://doi.org/10.1177/0735633117753935
58.
RiveraD. A.FrenayM.PaquotM. (2024). The role of MOOC forum discussion tasks in learners’ cognitive engagement. Journal of Computer Assisted Learning, 40(5), 2103, 2120. https://doi.org/10.1111/jcal.13001
59.
ShafferD. W.CollierW.RuisA. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), Article 3. https://doi.org/10.18608/jla.2016.33.3
60.
ShaikhS.DaudpottaS. M.ImranA. S. (2021). Bloom’s learning outcomes’ automatic classification using LSTM and pretrained word embeddings. IEEE Access, 9(6), 117887–117909. https://doi.org/10.1109/ACCESS.2021.3106443
61.
SofferT.CohenA. (2019). Students’ engagement characteristics predict success and completion of online courses. Journal of Computer Assisted Learning, 35(3), 378–389. https://doi.org/10.1111/jcal.12340
62.
SunJ. C.-Y.RuedaR. (2012). Situational interest, computer self-efficacy and self-regulation: Their impact on student engagement in distance education. British Journal of Educational Technology, 43(2), 191–204. https://doi.org/10.1111/j.1467-8535.2010.01157.x
63.
TangH.XingW.PeiB. (2019). Time really matters: Understanding the temporal dimension of online learning using educational data mining. Journal of Educational Computing Research, 57(5), 1326–1347. https://doi.org/10.1177/0735633118784705
WangD.ShanD.ZhengY.GuoK.ChenG.LuY. (2023). Can chatgpt detect student talk moves in classroom discourse? A preliminary comparison with bert. In Proceedings of the 16th International Conference on Educational Data Mining, Bengaluru India, July 11-14, pp. 515–519. https://doi.org/10.5281/zenodo.8115772
66.
WangX.WenM.RoséC. P. (2016). Towards triggering higher-order thinking behaviors in MOOCs. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, Edinburgh Scotland, April 25-29, pp. 398–407. https://doi.org/10.1145/2883851.2883964
67.
WangX.YangD.WenM.KoedingerK.RoséC. P. (2015). Investigating how student’s cognitive behavior in MOOC discussion forums affect learning gains. International Educational Data Mining Society.
68.
WeiX.SaabN.AdmiraalW. (2021). Assessment of cognitive, behavioral, and affective learning outcomes in massive open online courses: A systematic literature review. Computers & Education, 163(7), Article 104097. https://doi.org/10.1016/j.compedu.2020.104097
69.
WigfieldA.EcclesJ. S. (2020). 35 years of research on students' subjective task values and motivation: A look back and a look forward. In ElliotA. J. (Ed.), Advances in motivation science (7, pp. 161–198). Elsevier. https://doi.org/10.1016/bs.adms.2019.05.002
70.
WiseA. F.CuiY. (2018). Learning communities in the crowd: Characteristics of content related interactions and social relationships in MOOC discussion forums. Computers & Education, 122(10), 221–242. https://doi.org/10.1016/j.compedu.2018.03.021
71.
WiseA. F.CuiY.JinW.VytasekJ. (2017). Mining for gold: Identifying content-related MOOC discussion threads across domains through linguistic modeling. The Internet and Higher Education, 32(1), 11–28. https://doi.org/10.1016/j.iheduc.2016.08.001
72.
WiseA. F.KnightS.ShumS. B. (2021). Collaborative learning analytics. In CressU.RoséC.WiseA. F.OshimaJ. (Eds.), International handbook of computer-supported collaborative learning (1st ed., pp. 425–443). Springer International Publishing. https://doi.org/10.1007/978-3-030-65291-3_23
73.
XingW.DuH. (2023). Mining large open online learning networks: Exploring community dynamics and communities of performance. Journal of Educational Computing Research, 61(2), 390–415. https://doi.org/10.1177/07356331221113613
74.
YangB.TangH.HaoL.RoseJ. R. (2022). Untangling chaos in discussion forums: A temporal analysis of topic-relevant forum posts in MOOCs. Computers & Education, 178, Article 104402. https://doi.org/10.1016/j.compedu.2021.104402
75.
YaoL.ZhangY.WeiB.JinZ.ZhangR.ZhangY.ChenQ. (2017). Incorporating knowledge graph embeddings into topic modeling. In Thirty-First AAAI Conference on Artificial Intelligence, San Francisco USA, February4-9. https://doi.org/10.1609/aaai.v31i1.10951
76.
ZimmermanB. J.SchunkD. H. (2011). Self-regulated learning and performance: An introduction and an overview. In Handbook of self-regulation of learning and performance. Routledge.
77.
ZouW.HuX.PanZ.LiC.CaiY.LiuM. (2021). Exploring the relationship between social presence and learners’ prestige in MOOC discussion forums using automated content analysis and social network analysis. Computers in Human Behavior, 115(11), 106582. https://doi.org/10.ghmph7
