Debugging is essential for identifying and rectifying errors in programming, yet time constraints and students’ trivialization of errors often hinder progress. This study examines differences in debugging challenges and strategies among students with varying computational thinking (CT) competencies using weekly coding journals from an online undergraduate CT course. Participants used Scratch, a block-based programming language, and their journals from five assignments were analyzed using Term Frequency-Inverse Document Frequency and Structural Topic Modeling. High-performing students engaged with diverse topics and specific blocks tied to their weekly projects while low-performing students faced repetitive and broad challenges, such as understanding motion blocks and broadcast concepts. These patterns reveal that low-performing students struggle particularly during the ‘diagnose the fault’ phase of debugging, often hindering their progress in the final stage. Such challenges highlight the necessity for targeted educational interventions to improve the debugging proficiency and overall CT skills. The study underscores the importance of further research into students’ logical thinking processes during code review and debugging, suggesting the use of think-aloud protocols and detailed tracking of debugging practices for deeper insights. This research contributes to the field by showing that differentiated instruction and strategic support can enhance debugging skills across different student performance levels.
Abu DeebF.HickeyT. (2023). Impact of reflection in auto-graders: An empirical study of novice coders. Computer Science Education, 34(3), 473–494. https://doi.org/10.1080/08993408.2023.2262877
2.
AgboF. J.OyelereS. S.SuhonenJ.AdewumiS. (2019). A systematic review of computational thinking approach for programming education in higher education institutions. In Proceedings of the 19th Koli Calling international conference on computing education research. ACM. https://doi.org/10.1145/3364510.3364521
3.
AhmadzadehM.EllimanD.HigginsC. (2005). An analysis of patterns of debugging among novice computer science students. In Proceedings of the 10th Annual SIGCSE conference on innovation and technology in computer science education. ACM. https://doi.org/10.1145/1067445.1067472
4.
AhnJ.SungW.BlackJ. B. (2022). Unplugged debugging activities for developing young learners’ debugging skills. Journal of Research in Childhood Education, 36(3), 421–437. https://doi.org/10.1080/02568543.2021.1981503
5.
ArakawaK.HaoQ.GreerT.DingL.HundhausenC. D.PetersonA. (2021). In situ identification of student self-regulated learning struggles in programming assignments. In Proceedings of the 52nd ACM technical symposium on computer science education (pp. 467–473). ACM. https://doi.org/10.1145/3408877.3432357
6.
BafnaP.PramodD.VaidyaA. (2016). Document clustering: TF-IDF approach. In 2016 International conference on electrical, electronics, and optimization techniques (ICEEOT). IEEE. https://doi.org/10.1109/iceeot.2016.7754750
7.
BaniassadE.ZamprognoL.HallB.HolmesR. (2021). Stop the (autograder) insanity: Regression penalties to deter autograder overreliance. In Proceedings of the 52nd ACM technical symposium on computer science education (pp. 1062–1068). ACM. https://doi.org/10.1145/3408877.3432430
8.
BarrV.StephensonC. (2011). Bringing computational thinking to K-12: What is involved and what is the role of the computer science education community?Acm Inroads, 2(1), 48–54. https://doi.org/10.1145/1929887.1929905
9.
Berardi-ColettaB.BuyerL. S.DominowskiR. L.RellingerE. R. (1995). Metacognition and problem solving: A process-oriented approach. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(1), 205–223. https://doi.org/10.1037/0278-7393.21.1.205
10.
BerkalievZ.DeviS.FasshauerG. E.HickernellF. J.KartalO.LiX.McCrayP.WhitneyS.ZawojewskiJ. S. (2014). Initiating a programmatic assessment report. Primus, 24(5), 403–420. https://doi.org/10.1080/10511970.2014.893939
11.
BorregoM.FosterM. J.FroydJ. E. (2014). Systematic literature reviews in engineering education and other developing interdisciplinary fields. Journal of Engineering Education, 103(1), 45–76. https://doi.org/10.1002/jee.20038
12.
BrennanK.ResnickM. (2012). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 annual meeting of the American educational research association, Vancouver, Canada, Vol. 1, p. 25.
13.
ChenO.KalyugaS.SwellerJ. (2015). The worked example effect, the generation effect, and element interactivity. Journal of Educational Psychology, 107(3), 689–704. https://doi.org/10.1037/edu0000018
GaoX.HewK. F. (2023). A flipped systematic debugging approach to enhance elementary students’ program debugging performance and optimize cognitive load. Journal of Educational Computing Research, 61(5), 1064–1095. https://doi.org/10.1177/07356331221133560
16.
García-PeñalvoF. J.MendesA. J. (2018). Exploring the computational thinking effects in pre-university education. Computers in Human Behavior, 80(7), 407–411. https://doi.org/10.1016/j.chb.2017.12.005
17.
GroverS.PeaR. (2013). Computational thinking in K–12 a review of the state of the field. Educational Researcher, 42(1), 38–43. https://doi.org/10.3102/0013189x12463051
18.
HsuT. C.ChangS. C.HungY. T. (2018). How to learn and how to teach computational thinking: Suggestions based on a review of the literature. Computers & Education, 126(3), 296–310. https://doi.org/10.1016/j.compedu.2018.07.004
19.
HuN.ZhangT.GaoB.BoseI. (2019). What do hotel customers complain about? Text analysis using structural topic model. Tourism Management, 72, 417–426. https://doi.org/10.1016/j.tourman.2019.01.002
20.
HuangM.FanR.ParkE.CheonJ. (2022). Exploring instructional strategies for computational thinking concepts and practices in higher education. In Proceeding of society for information technology & teacher education international conference (pp. 27–32). Association for the Advancement of Computing in Education (AACE).
21.
JayathirthaG.FieldsD.KafaiY. (2024). Distributed debugging with electronic textiles: Understanding high school student pairs’ problem-solving strategies, practices, and perspectives on repairing physical computing projects. Computer Science Education, 1–35. https://doi.org/10.1080/08993408.2023.2297738
22.
JemmaliC.KleinmanE.BunianS.AlmedaM. V.RoweE.Seif El-NasrM. (2020). MAADS: Mixed-methods approach for the analysis of debugging sequences of beginner programmers. In Proceedings of the 51st ACM technical symposium on computer science education (pp. 86–92). ACM. https://doi.org/10.1145/3328778.3366824
23.
JonassenD. H.HungW. (2006). Learning to troubleshoot: A new theory-based design architecture. Educational Psychology Review, 18(1), 77–114. https://doi.org/10.1007/s10648-006-9001-8
24.
KafaiY.BiswasG.HutchinsN.SnyderC.BrennanK.HaduongP.DanishJ. (2020). Turning bugs into learning opportunities: Understanding debugging processes, perspectives, and pedagogies. In GresalfiM.HornI. S. (Eds.), The interdisciplinarity of the learning sciences, 14th international conference of the learning sciences (ICLS) 2020 (Vol. 1, pp. 374–381). International Society of the Learning Sciences.
25.
KafaiY. B.DeLiemaD.FieldsD. A.LewandowskiG.LewisC. (2019). Rethinking debugging as productive failure for CS education. In Proceedings of the 50th ACM technical symposium on computer science education. https://doi.org/10.1145/3287324.3287333
26.
KimC.DinçE.LeeE.BaabdullahA.ZhangA. Y.BellandB. R. (2023). Revisiting analogical reasoning in computing education: Use of similarities between robot programming tasks in debugging. Journal of Educational Computing Research, 61(5), 1036–1063. https://doi.org/10.1177/07356331221142912
27.
KlepschM.SeufertT. (2020). Understanding instructional design effects by differentiated measurement of intrinsic, extraneous, and germane cognitive load. Instructional Science, 48(1), 45–77. https://doi.org/10.1007/s11251-020-09502-9
28.
LamagnaE. A. (2014). A freshman seminar on problem solving and algorithmic thinking. Journal of Computing Sciences in Colleges, 29(6), 29–38.
29.
LeeE.KafaiY. B.VasudevanV.DavisR. L. (2014). Playing in the Arcade: Designing tangible interfaces with MaKey MaKey for Scratch games. In Playful user interfaces (pp. 277–292). Springer. https://doi.org/10.1007/978-981-4560-96-2_13
30.
LiA.EndresM.WeimerW. (2022). Debugging with stack overflow: Web search behavior in novice and expert programmers. In Proceedings of the 2022 IEEE/ACM 44th international conference on software engineering: Software engineering education and training (ICSE-SEET) (pp. 69–81). IEEE. https://doi.org/10.1109/icse-seet55299.2022.9794240
31.
LiC.ChanE.DennyP.Luxton-ReillyA.TemperoE. (2019). Towards a framework for teaching debugging. In Proceedings of the twenty-first Australasian computing education conference. https://doi.org/10.1145/3286960.3286970
32.
LiaoY. K.BrightG. W. (1991). Effects of computer programming on cognitive outcomes: A meta-analysis. Journal of Research on Computing in Education, 7(3), 251–268. https://doi.org/10.2190/e53g-hh8k-ajrr-k69m
33.
LyeS. Y.KohJ. H. L. (2014). Review on teaching and learning of computational thinking through programming: What is next for K-12?Computers in Human Behavior, 41(4), 51–61. https://doi.org/10.1016/j.chb.2014.09.012
34.
LyonJ. A.MaganaA. (2020). Computational thinking in higher education: A review of the literature. Computer Applications in Engineering Education, 28(5), 1174–1189. https://doi.org/10.1002/cae.22295
35.
MacannV.YadavA. (2023). Debugging beyond the code: Teachers’ perceptions of debugging as a CT practice impacting interdisciplinary teaching and learning. In Proceedings of the ACM conference on global computing education (Vol. 1, pp. 119–125). ACM.
36.
MaloneyJ.ResnickM.RuskN.SilvermanB.EastmondE. (2010). The Scratch programming language and environment. ACM Transactions on Computing Education, 10(4), 1–15. https://doi.org/10.1145/1868358.1868363
37.
MargariaT. (2018). From computational thinking to constructive design with simple models. Proceedings of the 8th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, Part I Modeling (ISoLA 2018), LNCS (pp. 261–278). Springer. https://doi.org/10.1007/978-3-030-03418-4_16
38.
MarkandanN.OsmanK.HalimL. (2022). Integrating computational thinking and empowering metacognitive awareness in STEM education. Frontiers in Psychology, 13, Article 872593. https://doi.org/10.3389/fpsyg.2022.872593
39.
MartinezP.LopezJ.RodríguezF. J.WigginsJ. B.BoyerK. E. (2020, February). Novice debugging in block-based and hybrid environments. In Proceedings of the 51st ACM technical symposium on computer science education (p. 1291). ACM. https://doi.org/10.1145/3328778.3372642
40.
MarwanS.DombeA.PriceT. W. (2020). Unproductive help-seeking in programming: What it is and how to address it. In Proceedings of the 2020 ACM conference on innovation and technology in computer science education (pp. 54–60). ACM.
41.
McCauleyR.FitzgeraldS.LewandowskiG.MurphyL.SimonB.ThomasL.ZanderC. (2008). Debugging: A review of the literature from an educational perspective. Computer Science Education, 18(2), 67–92. https://doi.org/10.1080/08993400802114581
42.
MurphyL.LewandowskiG.McCauleyR.SimonB.ThomasL.ZanderC. (2008). Debugging: The good, the bad, and the quirky--a qualitative analysis of novices' strategies. ACM SIGCSE Bulletin, 40(1), 163–167. https://doi.org/10.1145/1352322.1352191
43.
ParkinsonM. M.HermansS.GijbelsD.DinsmoreD. L. (2024). Exploring debugging processes and regulation strategies during collaborative coding tasks among elementary and secondary students. Computer Science Education, 1–28. https://doi.org/10.1080/08993408.2024.2305026
PerscheidM.SiegmundB.TaeumelM.HirschfeldR. (2017). Studying the advancement in debugging practice of professional software developers. Software Quality Journal, 25(1), 83–110. https://doi.org/10.1007/s11219-015-9294-2
46.
QaiserS.AliR. (2018). Text mining: Use of TF-IDF to examine the relevance of words to documents. International Journal of Computer Applications, 181(1), 25–29. https://doi.org/10.5120/ijca2018917395
47.
RamosJ. (2003). Using tf-idf to determine word relevance in document queries. Proceedings of the first instructional conference on machine learning, 242(1), 29–48.
48.
RichK. M.StricklandC.BinkowskiT. A.FranklinD. (2019). A K-8 debugging learning trajectory derived from research literature. In Proceedings of the 50th ACM technical symposium on computer science education (pp. 745–751). ACM. https://doi.org/10.1145/3287324.3287396
49.
RobertsM. E.StewartB. M.TingleyD. (2019). Stm: An R package for structural topic models. Journal of Statistical Software, 91(2), 1–40. https://doi.org/10.18637/jss.v091.i02
50.
RomeroM.LepageA.LilleB. (2017). Computational thinking development through creative programming in higher education. International Journal of Educational Technology in Higher Education, 14(1), 42. https://doi.org/10.1186/s41239-017-0080-z
51.
SunC.YangS.BeckerB. (2024). Debugging in computational thinking: A meta-analysis on the effects of interventions on debugging skills. Journal of Educational Computing Research, 62(4), 1087–1121. https://doi.org/10.1177/07356331241227793
52.
SwellerJ. (2020). Cognitive load theory and educational technology. Educational Technology Research & Development, 68(1), 1–16. https://doi.org/10.1007/s11423-019-09701-3
53.
Van GogT.RummelN.RenklA. (2019). Learning how to solve problems by studying examples. In DunloskyJ.RawsonK. A. (Eds.), The Cambridge handbook of cognition and education (pp. 183–208). Cambridge University Press.
54.
Van MerrienboerJ. J.SwellerJ. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17(2), 147–177. https://doi.org/10.1007/s10648-005-3951-0
55.
VoogtJ.FisserP.GoodJ.MishraP.YadavA. (2015). Computational thinking in compulsory education: Towards an agenda for research and practice. Education and Information Technologies, 20(4), 715–728. https://doi.org/10.1007/s10639-015-9412-6
56.
VourletsisI.PolitisP.KarasavvidisI. (2021). The effect of a computational thinking instructional intervention on students’ debugging proficiency level and strategy use. Research on E-Learning and ICT in Education, 15–34. https://doi.org/10.1007/978-3-030-64363-8_2
57.
WhalleyJ.SettleA.Luxton-ReillyA. (2021). Novice reflections on debugging. In Proceedings of the 52nd ACM technical symposium on computer science education (pp. 73–79). ACM.
58.
WongG. K.JiangS. (2018, December). Computational thinking education for children: Algorithmic thinking and debugging. In 2018 IEEE international conference on teaching, assessment, and learning for engineering (TALE) (pp. 328–334). IEEE. https://doi.org/10.1109/tale.2018.8615232
XuZ.RitzhauptA. D.TianF.UmapathyK. (2019). Block-based versus text-based programming environments on novice student learning outcomes: A meta-analysis study. Computer Science Education, 29(2-3), 177–204. https://doi.org/10.1080/08993408.2019.1565233
61.
YanW.IsraelM.LiuT. (2021). Elementary students’ debugging behaviors in a game-based environment. In Proceedings of the 17th ACM conference on international computing education research (pp. 441–442). ACM.
62.
YehK. C.XieY.KeF. (2011, October). Teaching computational thinking to non-computing majors using spreadsheet functions. In 2011 Frontiers in education conference (FIE) (pp. F3J). IEEE. https://doi.org/10.1109/fie.2011.6142980
63.
YuenT. T.RobbinsK. A. (2015). A qualitative study of students’ computational thinking skills in a data-driven computing class. ACM Transactions on Computing Education (TOCE), 14(4), 1–19. https://doi.org/10.1145/2676660
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
