Exploring Debugging Challenges and Strategies Using Structural Topic Model: A Comparative Analysis of High and Low-Performing Students

Abstract

Debugging is essential for identifying and rectifying errors in programming, yet time constraints and students’ trivialization of errors often hinder progress. This study examines differences in debugging challenges and strategies among students with varying computational thinking (CT) competencies using weekly coding journals from an online undergraduate CT course. Participants used Scratch, a block-based programming language, and their journals from five assignments were analyzed using Term Frequency-Inverse Document Frequency and Structural Topic Modeling. High-performing students engaged with diverse topics and specific blocks tied to their weekly projects while low-performing students faced repetitive and broad challenges, such as understanding motion blocks and broadcast concepts. These patterns reveal that low-performing students struggle particularly during the ‘diagnose the fault’ phase of debugging, often hindering their progress in the final stage. Such challenges highlight the necessity for targeted educational interventions to improve the debugging proficiency and overall CT skills. The study underscores the importance of further research into students’ logical thinking processes during code review and debugging, suggesting the use of think-aloud protocols and detailed tracking of debugging practices for deeper insights. This research contributes to the field by showing that differentiated instruction and strategic support can enhance debugging skills across different student performance levels.

Keywords

computational thinking debugging strategies debugging challenges block-based programming structural topic model

Get full access to this article

View all access options for this article.

References

Abu Deeb

Hickey

(2023). Impact of reflection in auto-graders: An empirical study of novice coders. Computer Science Education, 34(3), 473–494. https://doi.org/10.1080/08993408.2023.2262877

Agbo

F. J.

Oyelere

S. S.

Suhonen

Adewumi

(2019). A systematic review of computational thinking approach for programming education in higher education institutions. In Proceedings of the 19th Koli Calling international conference on computing education research. ACM. https://doi.org/10.1145/3364510.3364521

Ahmadzadeh

Elliman

Higgins

(2005). An analysis of patterns of debugging among novice computer science students. In Proceedings of the 10th Annual SIGCSE conference on innovation and technology in computer science education. ACM. https://doi.org/10.1145/1067445.1067472

Ahn

Sung

Black

J. B.

(2022). Unplugged debugging activities for developing young learners’ debugging skills. Journal of Research in Childhood Education, 36(3), 421–437. https://doi.org/10.1080/02568543.2021.1981503

Arakawa

Hao

Greer

Ding

Hundhausen

C. D.

Peterson

(2021). In situ identification of student self-regulated learning struggles in programming assignments. In Proceedings of the 52nd ACM technical symposium on computer science education (pp. 467–473). ACM. https://doi.org/10.1145/3408877.3432357

Bafna

Pramod

Vaidya

(2016). Document clustering: TF-IDF approach. In 2016 International conference on electrical, electronics, and optimization techniques (ICEEOT). IEEE. https://doi.org/10.1109/iceeot.2016.7754750

Baniassad

Zamprogno

Hall

Holmes

(2021). Stop the (autograder) insanity: Regression penalties to deter autograder overreliance. In Proceedings of the 52nd ACM technical symposium on computer science education (pp. 1062–1068). ACM. https://doi.org/10.1145/3408877.3432430

Barr

Stephenson

(2011). Bringing computational thinking to K-12: What is involved and what is the role of the computer science education community? Acm Inroads, 2(1), 48–54. https://doi.org/10.1145/1929887.1929905

Berardi-Coletta

Buyer

L. S.

Dominowski

R. L.

Rellinger

E. R.

(1995). Metacognition and problem solving: A process-oriented approach. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(1), 205–223. https://doi.org/10.1037/0278-7393.21.1.205

10.

Berkaliev

Devi

Fasshauer

G. E.

Hickernell

F. J.

Kartal

McCray

Whitney

Zawojewski

J. S.

(2014). Initiating a programmatic assessment report. Primus, 24(5), 403–420. https://doi.org/10.1080/10511970.2014.893939

11.

Borrego

Foster

M. J.

Froyd

J. E.

(2014). Systematic literature reviews in engineering education and other developing interdisciplinary fields. Journal of Engineering Education, 103(1), 45–76. https://doi.org/10.1002/jee.20038

12.

Brennan

Resnick

(2012). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 annual meeting of the American educational research association, Vancouver, Canada, Vol. 1, p. 25.

13.

Chen

Kalyuga

Sweller

(2015). The worked example effect, the generation effect, and element interactivity. Journal of Educational Psychology, 107(3), 689–704. https://doi.org/10.1037/edu0000018

14.

Chmiel

Loui

M. C.

(2004). Debugging: From novice to expert. Inroads, 36(1), 17–21. https://doi.org/10.1145/1028174.971310

15.

Gao

Hew

K. F.

(2023). A flipped systematic debugging approach to enhance elementary students’ program debugging performance and optimize cognitive load. Journal of Educational Computing Research, 61(5), 1064–1095. https://doi.org/10.1177/07356331221133560

16.

García-Peñalvo

F. J.

Mendes

A. J.

(2018). Exploring the computational thinking effects in pre-university education. Computers in Human Behavior, 80(7), 407–411. https://doi.org/10.1016/j.chb.2017.12.005

17.

Grover

Pea

(2013). Computational thinking in K–12 a review of the state of the field. Educational Researcher, 42(1), 38–43. https://doi.org/10.3102/0013189x12463051

18.

Hsu

T. C.

Chang

S. C.

Hung

Y. T.

(2018). How to learn and how to teach computational thinking: Suggestions based on a review of the literature. Computers & Education, 126(3), 296–310. https://doi.org/10.1016/j.compedu.2018.07.004

19.

Zhang

Gao

Bose

(2019). What do hotel customers complain about? Text analysis using structural topic model. Tourism Management, 72, 417–426. https://doi.org/10.1016/j.tourman.2019.01.002

20.

Huang

Fan

Park

Cheon

(2022). Exploring instructional strategies for computational thinking concepts and practices in higher education. In Proceeding of society for information technology & teacher education international conference (pp. 27–32). Association for the Advancement of Computing in Education (AACE).

21.

Jayathirtha

Fields

Kafai

(2024). Distributed debugging with electronic textiles: Understanding high school student pairs’ problem-solving strategies, practices, and perspectives on repairing physical computing projects. Computer Science Education, 1–35. https://doi.org/10.1080/08993408.2023.2297738

22.

Jemmali

Kleinman

Bunian

Almeda

M. V.

Rowe

Seif El-Nasr

(2020). MAADS: Mixed-methods approach for the analysis of debugging sequences of beginner programmers. In Proceedings of the 51st ACM technical symposium on computer science education (pp. 86–92). ACM. https://doi.org/10.1145/3328778.3366824

23.

Jonassen

D. H.

Hung

(2006). Learning to troubleshoot: A new theory-based design architecture. Educational Psychology Review, 18(1), 77–114. https://doi.org/10.1007/s10648-006-9001-8

24.

Kafai

Biswas

Hutchins

Snyder

Brennan

Haduong

Danish

(2020). Turning bugs into learning opportunities: Understanding debugging processes, perspectives, and pedagogies. In Gresalfi

Horn

I. S.

(Eds.), The interdisciplinarity of the learning sciences, 14th international conference of the learning sciences (ICLS) 2020 (Vol. 1, pp. 374–381). International Society of the Learning Sciences.

25.

Kafai

Y. B.

DeLiema

Fields

D. A.

Lewandowski

Lewis

(2019). Rethinking debugging as productive failure for CS education. In Proceedings of the 50th ACM technical symposium on computer science education. https://doi.org/10.1145/3287324.3287333

26.

Kim

Dinç

Lee

Baabdullah

Zhang

A. Y.

Belland

B. R.

(2023). Revisiting analogical reasoning in computing education: Use of similarities between robot programming tasks in debugging. Journal of Educational Computing Research, 61(5), 1036–1063. https://doi.org/10.1177/07356331221142912

27.

Klepsch

Seufert

(2020). Understanding instructional design effects by differentiated measurement of intrinsic, extraneous, and germane cognitive load. Instructional Science, 48(1), 45–77. https://doi.org/10.1007/s11251-020-09502-9

28.

Lamagna

E. A.

(2014). A freshman seminar on problem solving and algorithmic thinking. Journal of Computing Sciences in Colleges, 29(6), 29–38.

29.

Lee

Kafai

Y. B.

Vasudevan

Davis

R. L.

(2014). Playing in the Arcade: Designing tangible interfaces with MaKey MaKey for Scratch games. In Playful user interfaces (pp. 277–292). Springer. https://doi.org/10.1007/978-981-4560-96-2_13

30.

Endres

Weimer

(2022). Debugging with stack overflow: Web search behavior in novice and expert programmers. In Proceedings of the 2022 IEEE/ACM 44th international conference on software engineering: Software engineering education and training (ICSE-SEET) (pp. 69–81). IEEE. https://doi.org/10.1109/icse-seet55299.2022.9794240

31.

Chan

Denny

Luxton-Reilly

Tempero

(2019). Towards a framework for teaching debugging. In Proceedings of the twenty-first Australasian computing education conference. https://doi.org/10.1145/3286960.3286970

32.

Liao

Y. K.

Bright

G. W.

(1991). Effects of computer programming on cognitive outcomes: A meta-analysis. Journal of Research on Computing in Education, 7(3), 251–268. https://doi.org/10.2190/e53g-hh8k-ajrr-k69m

33.

Lye

S. Y.

Koh

J. H. L.

(2014). Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41(4), 51–61. https://doi.org/10.1016/j.chb.2014.09.012

34.

Lyon

J. A.

Magana

(2020). Computational thinking in higher education: A review of the literature. Computer Applications in Engineering Education, 28(5), 1174–1189. https://doi.org/10.1002/cae.22295

35.

Macann

Yadav

(2023). Debugging beyond the code: Teachers’ perceptions of debugging as a CT practice impacting interdisciplinary teaching and learning. In Proceedings of the ACM conference on global computing education (Vol. 1, pp. 119–125). ACM.

36.

Maloney

Resnick

Rusk

Silverman

Eastmond

(2010). The Scratch programming language and environment. ACM Transactions on Computing Education, 10(4), 1–15. https://doi.org/10.1145/1868358.1868363

37.

Margaria

(2018). From computational thinking to constructive design with simple models. Proceedings of the 8th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, Part I Modeling (ISoLA 2018), LNCS (pp. 261–278). Springer. https://doi.org/10.1007/978-3-030-03418-4_16

38.

Markandan

Osman

Halim

(2022). Integrating computational thinking and empowering metacognitive awareness in STEM education. Frontiers in Psychology, 13, Article 872593. https://doi.org/10.3389/fpsyg.2022.872593

39.

Martinez

Lopez

Rodríguez

F. J.

Wiggins

J. B.

Boyer

K. E.

(2020, February). Novice debugging in block-based and hybrid environments. In Proceedings of the 51st ACM technical symposium on computer science education (p. 1291). ACM. https://doi.org/10.1145/3328778.3372642

40.

Marwan

Dombe

Price

T. W.

(2020). Unproductive help-seeking in programming: What it is and how to address it. In Proceedings of the 2020 ACM conference on innovation and technology in computer science education (pp. 54–60). ACM.

41.

McCauley

Fitzgerald

Lewandowski

Murphy

Simon

Thomas

Zander

(2008). Debugging: A review of the literature from an educational perspective. Computer Science Education, 18(2), 67–92. https://doi.org/10.1080/08993400802114581

42.

Murphy

Lewandowski

McCauley

Simon

Thomas

Zander

(2008). Debugging: The good, the bad, and the quirky--a qualitative analysis of novices' strategies. ACM SIGCSE Bulletin, 40(1), 163–167. https://doi.org/10.1145/1352322.1352191

43.

Parkinson

M. M.

Hermans

Gijbels

Dinsmore

D. L.

(2024). Exploring debugging processes and regulation strategies during collaborative coding tasks among elementary and secondary students. Computer Science Education, 1–28. https://doi.org/10.1080/08993408.2024.2305026

44.

Pea

R. D.

Kurland

D. M.

(1984). On the cognitive effects of learning computer programming. New Ideas in Psychology, 2(2), 137–168. https://doi.org/10.1016/0732-118x(84)90018-7

45.

Perscheid

Siegmund

Taeumel

Hirschfeld

(2017). Studying the advancement in debugging practice of professional software developers. Software Quality Journal, 25(1), 83–110. https://doi.org/10.1007/s11219-015-9294-2

46.

Qaiser

Ali

(2018). Text mining: Use of TF-IDF to examine the relevance of words to documents. International Journal of Computer Applications, 181(1), 25–29. https://doi.org/10.5120/ijca2018917395

47.

Ramos

(2003). Using tf-idf to determine word relevance in document queries. Proceedings of the first instructional conference on machine learning, 242(1), 29–48.

48.

Rich

K. M.

Strickland

Binkowski

T. A.

Franklin

(2019). A K-8 debugging learning trajectory derived from research literature. In Proceedings of the 50th ACM technical symposium on computer science education (pp. 745–751). ACM. https://doi.org/10.1145/3287324.3287396

49.

Roberts

M. E.

Stewart

B. M.

Tingley

(2019). Stm: An R package for structural topic models. Journal of Statistical Software, 91(2), 1–40. https://doi.org/10.18637/jss.v091.i02

50.

Romero

Lepage

Lille

(2017). Computational thinking development through creative programming in higher education. International Journal of Educational Technology in Higher Education, 14(1), 42. https://doi.org/10.1186/s41239-017-0080-z

51.

Sun

Yang

Becker

(2024). Debugging in computational thinking: A meta-analysis on the effects of interventions on debugging skills. Journal of Educational Computing Research, 62(4), 1087–1121. https://doi.org/10.1177/07356331241227793

52.

Sweller

(2020). Cognitive load theory and educational technology. Educational Technology Research & Development, 68(1), 1–16. https://doi.org/10.1007/s11423-019-09701-3

53.

Van Gog

Rummel

Renkl

(2019). Learning how to solve problems by studying examples. In Dunlosky

Rawson

K. A.

(Eds.), The Cambridge handbook of cognition and education (pp. 183–208). Cambridge University Press.

54.

Van Merrienboer

J. J.

Sweller

(2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17(2), 147–177. https://doi.org/10.1007/s10648-005-3951-0

55.

Voogt

Fisser

Good

Mishra

Yadav

(2015). Computational thinking in compulsory education: Towards an agenda for research and practice. Education and Information Technologies, 20(4), 715–728. https://doi.org/10.1007/s10639-015-9412-6

56.

Vourletsis

Politis

Karasavvidis

(2021). The effect of a computational thinking instructional intervention on students’ debugging proficiency level and strategy use. Research on E-Learning and ICT in Education, 15–34. https://doi.org/10.1007/978-3-030-64363-8_2

57.

Whalley

Settle

Luxton-Reilly

(2021). Novice reflections on debugging. In Proceedings of the 52nd ACM technical symposium on computer science education (pp. 73–79). ACM.

58.

Wong

G. K.

Jiang

(2018, December). Computational thinking education for children: Algorithmic thinking and debugging. In 2018 IEEE international conference on teaching, assessment, and learning for engineering (TALE) (pp. 328–334). IEEE. https://doi.org/10.1109/tale.2018.8615232

59.

Wong

S. K. M.

Yao

Y. Y.

(1992). An information-theoretic measure of term specificity. Journal of the American Society for Information Science, 43(1), 54–61. https://doi.org/10.1002/(SICI)1097-4571(199201)43:1<54::AID-ASI5>3.0.CO;2-A

60.

Ritzhaupt

A. D.

Tian

Umapathy

(2019). Block-based versus text-based programming environments on novice student learning outcomes: A meta-analysis study. Computer Science Education, 29(2-3), 177–204. https://doi.org/10.1080/08993408.2019.1565233

61.

Yan

Israel

Liu

(2021). Elementary students’ debugging behaviors in a game-based environment. In Proceedings of the 17th ACM conference on international computing education research (pp. 441–442). ACM.

62.

Yeh

K. C.

Xie

(2011, October). Teaching computational thinking to non-computing majors using spreadsheet functions. In 2011 Frontiers in education conference (FIE) (pp. F3J). IEEE. https://doi.org/10.1109/fie.2011.6142980

63.

Yuen

T. T.

Robbins

K. A.

(2015). A qualitative study of students’ computational thinking skills in a data-driven computing class. ACM Transactions on Computing Education (TOCE), 14(4), 1–19. https://doi.org/10.1145/2676660

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.97 MB