Effects of Block-Based Visual Programming on K-12 Students’ Learning Outcomes

Abstract

Computer programming is regarded as an important skill for the future. However, many K-12 students face challenges and difficulties in learning traditional text-based programming. Block-based visual programming (BVP) can reduce the difficulty of learning programming and is seen as a potential programming education tool. Nevertheless, the effects of BVP on K-12 student learning outcomes remain unclear, and there is no evidence of meta-analysis. This is the first meta-analysis that aims to examine the effects of BVP on K-12 students’ learning based on 42 effect sizes from 29 independent experimental studies (2000–2023). The result suggests that BVP has an upper-medium effect on learning (SMD = 0.769, 95% CI = [0.554, 0.984], p < .001), particularly in the cognitive outcome (SMD = 0.698, p < .001). Moreover, the BVP tool, study design, and region have moderating effects. The BVP’s effects are larger when: (1) with 61∼100 students, (2) for junior high school, (3) with the learning duration of >1 month, (4) targeting math, (5) among novice students, (6) using Scratch, ScratchJr, Alice, and App Inventor, (7) employing project-based learning and project-based + game-based learning, (8) base on quasi-experimental design, (9) in Africa and Europe, (10) with latter publication years. Finally, discussions, implications, limitations, and future directions are put forward.

Keywords

block-based visual programming programming education K-12 students learning outcomes meta-analysis

Get full access to this article

View all access options for this article.

References

Ahmed

Soomro

A. H.

Shah

Attar

Fatima

Kumari

(2023). An exploration of visual and textual based programming languages: A comparative analysis. International Journal of Electrical Engineering & Emerging Technology, 6(1), 20–23. https://ijeeet.com/index.php/ijeeet/article/view/140

Aivaloglou

Hermans

(2019). Early programming education and career orientation: The effects of gender, self-efficacy, motivation and stereotypes. In Proceedings of the 50th ACM technical symposium on computer science education (pp. 679–685). ACM. https://doi.org/10.1145/3287324.3287358

* Akinyemi

A. L.

Ogundipe

Adelana

O. P.

(2021). Effect of scratch programming language (SPL) on achievement in geometry among senior secondary students in ibadan, Nigeria. Journal of Information and Communication Technology in Education, 8(2), 24–33. https://doi.org/10.37134/jictie.vol8.2.3.2021

Alice.org . (2022). Telling stories. https://www.alice.org/

Aljameel

I. H.

(2022). Exploring the compelling rationale to include programming in the K-12 curriculum. International Journal of Innovative Research in Education, 9(2), 142–155. https://doi.org/10.18844/ijire.v9i2.8460

Armoni

Meerbaum-Salant

Ben-Ari

(2015). From scratch to “real” programming. ACM Transactions on Computing Education, 14(4), 1–15. https://doi.org/10.1145/2677087

* Namli

N. A.

Aybek

(2022). An investigation of the effect of block-based programming and unplugged coding activities on fifth graders’ computational thinking skills, self-efficacy and academic performance. Contemporary Educational Technology, 14(1), ep341. https://doi.org/10.30935/cedtech/11477

Bau

D. A.

Dawson

Pickens

C. S.

(2015). Pencil code: Block code for a text world. In Proceedings of the 14th international conference on interaction design and children (pp. 445–448). ACM. https://doi.org/10.1145/2771839.2771875

Bau

Gray

Kelleher

Sheldon

Turbak

(2017). Learnable programming: Blocks and beyond. Communications of the ACM, 60(6), 72–80. https://doi.org/10.1145/3015455

10.

Ben-Yaacov

Hershkovitz

(2023). Types of errors in block programming: Driven by learner, learning environment. Journal of Educational Computing Research, 61(1), 178–207. https://doi.org/10.1177/07356331221102312

11.

Borenstein

Hedges

L. V.

Higgins

J. P.

Rothstein

H. R.

(2009). Introduction to meta-analysis. John Wiley and Sons.

12.

Brown

N. C.

Mönig

Bau

Weintrop

(2016). Panel: Future directions of block-based programming. In Proceedings of the 47th ACM technical symposium on computing science education (pp. 315–316). ACM. https://doi.org/10.1145/2839509.2844661

13.

* Brown

Mongan

Kusic

Garbarine

Fromm

Fontecchio

(2008). Computer aided instruction as a vehicle for problem solving: Scratch boards in the middle years classroom. In 2008 annual conference & exposition (pp. 13–319). American Society for Engineering Education. https://doi.org/10.18260/1-2--3826

14.

Cai

Fan

(2017). Gender and attitudes toward technology use: A meta-analysis. Computers & Education, 105, 1–13. https://doi.org/10.1016/j.compedu.2016.11.003

15.

Çakiroğlu

Ü.

Çevik

İ.

(2022). A framework for measuring abstraction as a sub-skill of computational thinking in block-based programming environments. Education and Information Technologies, 27(7), 9455–9484. https://doi.org/10.1007/s10639-022-11019-2

16.

Çakıroğlu

Ü.

Mumcu

(2020). Focus-Fight-Finalize (3F): Problem-solving steps extracted from behavioral patterns in block based programming. Journal of Educational Computing Research, 58(7), 1279–1310. https://doi.org/10.1177/0735633120930673

17.

* Calao

L. A.

Moreno-León

Correa

H. E.

Robles

(2015). Developing mathematical thinking with Scratch: An experiment with 6th grade students. In Design for teaching and learning in a networked world: 10th European conference on technology enhanced learning, EC-TEL 2015 (pp. 17–27). Springer. https://doi.org/10.1007/978-3-319-24258-3_2

18.

Caponnetto

Dante

Masotta

La Cerra

Petrucci

Alfes

C. M.

Lancia

(2021). Examining nursing student academic outcomes: A forty-y ear systematic review and meta-analysis. Nurse Education Today, 100(7), Article 104823. https://doi.org/10.1016/j.nedt.2021.104823

19.

* Cetin

Tarık

O. T. U.

(2023). The effect of the modality on students’ computational thinking, programming attitude, and programming achievement. International Journal of Computer Sciences and Engineering Systems, 6(2), 1–14. https://doi.org/10.21585/ijcses.v6i2.170

20.

Chang

C. K.

(2014). Effects of using Alice and Scratch in an introductory programming course for corrective instruction. Journal of Educational Computing Research, 51(2), 185–204. https://doi.org/10.2190/EC.51.2.c

21.

Chao

P. Y.

(2016). Exploring students’ computational practice, design and performance of problem-solving through a visual programming environment. Computers & Education, 95, 202–215. https://doi.org/10.1016/j.compedu.2016.01.010

22.

Cheah

C. S.

(2020). Factors contributing to the difficulties in teaching and learning of computer programming: A literature review. Contemporary Educational Technology, 12(2), ep272. https://doi.org/10.30935/cedtech/8247

23.

Chen

Haduong

Brennan

Sonnert

Sadler

(2019). The effects of first programming language on college students’ computing attitude and achievement: A comparison of graphical and textual languages. Computer Science Education, 29(1), 23–48. https://doi.org/10.1080/08993408.2018.1547564

24.

Cheung

J. C.

Ngai

Chan

S. C.

Lau

W. W.

(2009). Filling the gap in programming instruction: A text-enhanced graphical programming environment for junior high students. ACM SIGCSE Bulletin, 41(1), 276–280. https://doi.org/10.1145/1539024.1508968

25.

Ching

Y. H.

Hsu

Y. C.

Baldwin

(2018). Developing computational thinking with educational technologies for young learners. TechTrends, 62(6), 563–573. https://doi.org/10.1007/s11528-018-0292-7

26.

Chiu

J. I.

Tsuei

(2020). Meta-analysis of children’s learning outcomes in block-based programming courses. In 22nd international conference of HCI international 2020–late breaking posters (pp. 259–266). Springer. https://doi.org/10.1007/978-3-030-60703-6_33

27.

Chou

P. N.

(2020). Using ScratchJr to foster young children’s computational thinking competence: A case study in a third-grade computer class. Journal of Educational Computing Research, 58(3), 570–595. https://doi.org/10.1177/0735633119872908

28.

Code.org. CSTA. ECEP Alliance . (2022). 2022 state of computer science education: Understanding our national imperative. https://advocacy.code.org/stateofcs

29.

Cohen

(1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155

30.

Collis

B. A.

Williams

R. L.

(1987). Cross-cultural comparison of gender differences in adolescents’ attitudes toward computers and selected school subjects. The Journal of Educational Research, 81(1), 17–27. https://doi.org/10.1080/00220671.1987.10885792

31.

Cooper

(2017). Research synthesis and meta-analysis: A step-by-step approach (5th ed.). Sage.

32.

Costa

J. M.

Miranda

G. L.

(2017). Relation between Alice software and programming learning: A systematic review of the literature and meta-analysis. British Journal of Educational Technology, 48(6), 1464–1474. https://doi.org/10.1111/bjet.12496

33.

* de Kereki

I. F.

(2008). Scratch: Applications in computer science 1. In 2008 38th annual frontiers in education conference (pp. 7–11). IEEE. https://doi.org/10.1109/FIE.2008.4720267

34.

* Deng

Lei

Zhou

Zhang

(2020). Pencil Code improves learners' computational thinking and computer learning attitude. Computer Applications in Engineering Education, 28(1), 90–104. https://doi.org/10.1002/cae.22177

35.

Dilmen

Kert

S. B.

Uğraş

(2023). Children’s coding experiences in a block-based coding environment: A usability study on code. Org. Education and Information Technologies, 28(9), 10839–10864. https://doi.org/10.1007/s10639-023-11625-8

36.

Dorgu

T. E.

(2015). Different teaching methods: A panacea for effective curriculum implementation in the classroom. International Journal of Secondary Education, 3(6), 77–87. https://doi.org/10.11648/j.ijsedu.s.2015030601.13

37.

* Erol

Çırak

N. S.

(2022). The effect of a programming tool scratch on the problem-solving skills of middle school students. Education and Information Technologies, 27(3), 4065–4086. https://doi.org/10.1007/s10639-021-10776-w

38.

Erümit

A. K.

(2020). Effects of different teaching approaches on programming skills. Education and Information Technologies, 25(2), 1013–1037. https://doi.org/10.1007/s10639-019-10010-8

39.

Fessakis

Gouli

Mavroudi

(2013). Problem solving by 5–6 years old kindergarten children in a computer programming environment: A case study. Computers & Education, 63, 87–97. https://doi.org/10.1016/j.compedu.2012.11.016

40.

Field

A. P.

Gillett

(2010). How to do a meta-analysis. British Journal of Mathematical and Statistical Psychology, 63(Pt 3), 665–694. https://doi.org/10.1348/000711010X502733

41.

Gal-Ezer

Stephenson

(2014). A tale of two countries: Successes and challenges in K-12 computer science education in Israel and the United States. ACM Transactions on Computing Education, 14(2), 1–18. https://doi.org/10.1145/2602483

42.

Gates

N. J.

March

E. G.

(2016). A neuropsychologist’s guide to undertaking a systematic review for publication: Making the most of PRISMA guidelines. Neuropsychology Review, 26(2), 109–120. https://doi.org/10.1007/s11065-016-9318-0

43.

* Geng

(2023). Environmental design as a component of block-based programming. Computer Applications in Engineering Education, 31(2), 408–420. https://doi.org/10.1002/cae.22591

44.

Glass

G. V.

Smith

M. L.

(1979). Meta-analysis of research on class size and achievement. Educational Evaluation and Policy Analysis, 1(1), 2–16. https://doi.org/10.3102/01623737001001002

45.

Graham

Kiuhara

S. A.

MacKay

(2020). The effects of writing on learning in science, social studies, and mathematics: A meta-analysis. Review of Educational Research, 90(2), 179–226. https://doi.org/10.3102/0034654320914744

46.

Graham

Liu

Bartlett

Harris

K. R.

Aitken

Barkel

Kavanaugh

Talukdar

(2018). Reading for writing: A meta-analysis of the impact of reading interventions on writing. Review of Educational Research, 88(2), 243–284. https://doi.org/10.3102/0034654317746927

47.

Grover

(2021). Teaching and assessing for transfer from block-to-text programming in middle school computer science. In Hohensee

Lobato

(Eds.), Transfer of learning: Progressive perspectives for mathematics education and related fields (pp. 251–276). Springer. https://doi.org/10.1007/978-3-030-65632-4_11

48.

Grover

Pea

(2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43. https://doi.org/10.3102/0013189X12463051

49.

Güler

Kokoç

Önder Bütüner

(2023). Does a flipped classroom model work in mathematics education? A meta-analysis. Education and Information Technologies, 28(1), 57–79. https://doi.org/10.1007/s10639-022-11143-z

50.

Hadad

Shamir-Inbal

Blau

Leykin

(2021). Professional development of code and robotics teachers through small private online course (SPOC): Teacher centrality and pedagogical strategies for developing computational thinking of students. Journal of Educational Computing Research, 59(4), 763–791. https://doi.org/10.1177/0735633120973432

51.

Higgins

J. P.

Thompson

S. G.

Deeks

J. J.

Altman

D. G.

(2003). Measuring inconsistency in meta-analyses. BMJ, 327(7414), 557–560. https://doi.org/10.1136/bmj.327.7414.557

52.

Hill

(2015). Programming environments for children: Creating a language that grows with you. Master’s thesis. University of California. https://escholarship.org/uc/item/7k81b9xz

53.

Homer

Noble

(2017). Lessons in combining block-based and textual programming. Journal of Visual Languages and Sentient Systems, 3(1), 22–39. https://doi.org/10.18293/VLSS2017-007

54.

Hsu

T.-C.

Chang

S.-C.

Hung

Y.-T.

(2018). How to learn and how to teach computational thinking: Suggestions based on a review of the literature. Computers & Education, 126(3), 296–310. https://doi.org/10.1016/j.compedu.2018.07.004

55.

Hsu

T. C.

Hwang

G. J.

(2023). Interaction of visual interface and academic levels with young students’ anxiety, playfulness, and enjoyment in programming for robot control. Universal Access in the Information Society, 22(1), 213–225. https://doi.org/10.1007/s10209-021-00821-3

56.

Hsu

W. C.

Gainsburg

(2021). Hybrid and non-hybrid block-based programming languages in an introductory college computer-science course. Journal of Educational Computing Research, 59(5), 817–843. https://doi.org/10.1177/0735633120985108

57.

Chen

C. H.

C. Y.

(2021). Exploring the effectiveness and moderators of block-based visual programming on student learning: A meta-analysis. Journal of Educational Computing Research, 58(8), 1467–1493. https://doi.org/10.1177/0735633120945935

58.

* Ideris

Baharudin

S. M.

Hamzah

(2019). The effectiveness of scratch in collaborative learning on higher-order thinking skills in programming subject among year-six students. In 4th ASEAN conference on psychology, counselling, and humanities (ACPCH 2018) (pp. 421–425). Atlantis Press. https://doi.org/10.2991/acpch-18.2019.99

59.

Ioannidis

J. P.

Patsopoulos

N. A.

Evangelou

(2007). Uncertainty in heterogeneity estimates in meta-analyses. BMJ, 335(7626), 914–916. https://doi.org/10.1136/bmj.39343.408449.80

60.

Jadad

A. R.

Moore

R. A.

Carroll

Jenkinson

Reynolds

D. M.

Gavaghan

D. J.

McQuay

H. J.

(1996). Assessing the quality of reports of randomized clinical trials: Isblinding necessary? Controlled Clinical Trials, 17(1), 1–12. https://doi.org/10.1016/0197-2456(95)00134-4

61.

Jude

Kajura

Birevu

(2014). Adoption of the SAMR model to asses ICT pedagogical adoption: A case of Makerere university. International Journal of e-Education, e-Business, e-Management and e-Learning, 4(2), 106–115. https://doi.org/10.7763/ijeeee.2014.v4.312

62.

Kalogiannakis

Tzagaraki

Papadakis

(2021). A systematic review of the use of BBC micro: Bit in primary school. In Proceedings of the 10th virtual edition of the international conference new perspectives in science education, Florence, Italy, 18–19 March 2021, pp. 379–384. https://doi.org/10.26352/F318_2384-9509

63.

Kandin

Şendurur

(2023). The effects of goal-based scenarios used for programming education of 5th graders. Interactive Learning Environments, 31(10), 6335–6355. https://doi.org/10.1080/10494820.2022.2036199

64.

Kanuka

Rourke

Laflamme

(2007). The influence of instructional methods on the quality of online discussion. British Journal of Educational Technology, 38(2), 260–271. https://doi.org/10.1111/j.1467-8535.2006.00620.x

65.

Kong

S. C.

Wang

Y. Q.

(2021). The influence of parental support and perceived usefulness on students' learning motivation and flow experience in visual programming: Investigation from a parent perspective. British Journal of Educational Technology, 52(4), 1749–1770. https://doi.org/10.1111/bjet.13071

66.

* Lai

A. F.

Yang

S. M.

(2011). The learning effect of visualized programming learning on 6 th graders' problem solving and logical reasoning abilities. In 2011 international conference on electrical and control engineering (pp. 6940–6944). IEEE. https://doi.org/10.1109/ICECENG.2011.6056908

67.

Lei

Chiu

M. M.

Wang

Xie

(2022). Effects of game-based learning on students’ achievement in science: A meta-analysis. Journal of Educational Computing Research, 60(6), 1373–1398. https://doi.org/10.1177/07356331211064543

68.

Liao

S. M.

(2023). Scratch to r: Toward an inclusive pedagogy in teaching coding. Journal of Statistics and Data Science Education, 31(1), 45–56. https://doi.org/10.1080/26939169.2022.2090467

69.

Lin

Weintrop

(2021). The landscape of Block-based programming: Characteristics of block-based environments and how they support the transition to text-based programming. Journal of Computer Languages, 67(4), Article 101075. https://doi.org/10.1016/j.cola.2021.101075

70.

Lipsey

M. W.

Wilson

D. B.

(2001). Practical meta-analysis. Sage.

71.

* Liu

Zhu

(2023). Exploring the balance between computational thinking and learning motivation in elementary programming education: An empirical study with game-based learning. IEEE Transactions on Games, 15(1), 95–107. https://doi.org/10.1109/TG.2022.3143701

72.

Lye

S. Y.

Koh

J. H. L.

(2014). Review on teaching and learning of computational thinking through programming: What is next for k-12? Computers in Human Behavior, 41(4), 51–61. https://doi.org/10.1016/j.chb.2014.09.012

73.

Zhang

Zhu

Zhao

Wang

(2023). Game-based learning for students’ computational thinking: A meta-analysis. Journal of Educational Computing Research, 61(7), 1430–1463. https://doi.org/10.1177/07356331231178948

74.

Malan

D. J.

Leitner

H. H.

(2007). Scratch for budding computer scientists. ACM Sigcse Bulletin, 39(1), 223–227. https://doi.org/10.1145/1227504.1227388

75.

Mannila

Peltomäki

Salakoski

(2006). What about a simple language? Analyzing the difficulties in learning to program. Computer Science Education, 16(3), 211–227. https://doi.org/10.1080/08993400600912384

76.

Mao

Cui

Chiu

M. M.

Lei

(2022). Effects of game-based learning on students’ critical thinking: A meta-analysis. Journal of Educational Computing Research, 59(8), 1682–1708. https://doi.org/10.1177/07356331211007098

77.

Martin

Sun

Turk

Ritzhaupt

A. D.

(2021). A meta-analysis on the effects of synchronous online learning on cognitive and affective educational outcomes. International Review of Research in Open and Distance Learning, 22(3), 205–242. https://doi.org/10.19173/irrodl.v22i3.5263

78.

Meerbaum-Salant

Armoni

Ben-Ari

(2011). Habits of programming in scratch. In Proceedings of the 16th annual joint conference on innovation and technology in computer science education (pp. 168–172). ACM. https://doi.org/10.1145/1999747.1999796

79.

Merino-Armero

J. M.

González-Calero

J. A.

Cozar-Gutierrez

(2022). Computational thinking in K-12 education. An insight through meta-analysis. Journal of Research on Technology in Education, 54(3), 410–437. https://doi.org/10.1080/15391523.2020.1870250

80.

Mladenović

Žanko

Ž.

Aglić Čuvić

(2021). The impact of using program visualization techniques on learning basic programming concepts at the K–12 level. Computer Applications in Engineering Education, 29(1), 145–159. https://doi.org/10.1002/cae.22315

81.

Mladenović

Žanko

Ž.

Granić

(2021). Mediated transfer: From text to blocks and back. International Journal of Child-Computer Interaction, 29, Article 100279. https://doi.org/10.1016/j.ijcci.2021.100279

82.

Mladenović

Žanko

Ž.

Zaharija

(2024). From blocks to text: Bridging programming misconceptions. Journal of Educational Computing Research, 62(5), 1302–1326. https://doi.org/10.1177/07356331241240047

83.

* Mladenović

Boljat

Žanko

Ž.

(2018). Comparing loops misconceptions in block-based and text-based programming languages at the K-12 level. Education and Information Technologies, 23(4), 1483–1500. https://doi.org/10.1007/s10639-017-9673-3

84.

* Mladenović

Mladenović

Žanko

Ž.

(2020). Impact of used programming language for K-12 students' understanding of the loop concept. International Journal of Technology Enhanced Learning, 12(1), 79–98. https://doi.org/10.1504/IJTEL.2020.103817

85.

Moher

Liberati

Tetzlaff

Altman

D. G.

PRISMA Group . (2010). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. International Journal of Surgery, 8(5), 336–341. https://doi.org/10.1016/j.ijsu.2010.02.007

86.

* Moreno-León

Robles

Román-González

(2016). Code to learn: Where does it belong in the K-12 curriculum? Journal of Information Technology Education: Research, 15(2016), 283–303. https://doi.org/10.28945/3521

87.

* Nam

Kim

Lee

(2010). The effects of scaffolding-based courseware for the Scratch programming learning on student problem solving skill. In Proceedings of the 18th international conference on computers in education (pp. 723–727). Asia-Pacific Society for Computers in Education.

88.

Nikou

S. A.

Economides

A. A.

(2014). Transition in student motivation during a scratch and an app inventor course. In 2014 IEEE global engineering education conference (EDUCON) (pp. 1042–1045). IEEE. https://doi.org/10.1109/EDUCON.2014.6826234

89.

Ning

H. K.

Downing

(2012). Influence of student learning experience on academic performance: The mediator and moderator effects of self-regulation and motivation. British Educational Research Journal, 38(2), 219–237. https://doi.org/10.1080/01411926.2010.538468

90.

Noone

Mooney

(2018). Visual and textual programming languages: A systematic review of the literature. Journal of Computers in Education, 5(2), 149–174. https://doi.org/10.1007/s40692-018-0101-5

91.

Nouri

Zhang

Mannila

Norén

(2020). Development of computational thinking, digital competence and 21st century skills when learning programming in K-9. Education Inquiry, 11(1), 1–17. https://doi.org/10.1080/20004508.2019.1627844

92.

Olsson

Granberg

(2024). Teacher-student interaction supporting students’ creative mathematical reasoning during problem solving using Scratch. Mathematical Thinking and Learning, 26(3), 278–305. https://doi.org/10.1080/10986065.2022.2105567

93.

Ouahbi

Kaddari

Darhmaoui

Elachqar

Lahmine

(2015). Learning basic programming concepts by creating games with scratch programming environment. Procedia-Social and Behavioral Sciences, 191, 1479–1482. https://doi.org/10.1016/j.sbspro.2015.04.224

94.

Papadakis

Kalogiannakis

Orfanakis

Zaranis

(2014). Novice programming environments. Scratch & app inventor: A first comparison. In Proceedings of the 2014 workshop on interaction design in educational environments (pp. 1–7). ACM. https://doi.org/10.1145/2643604.2643613

95.

* Papadakis

Kalogiannakis

Zaranis

Orfanakis

(2016). Using Scratch and App Inventor for teaching introductory programming in secondary education. A case study. International Journal of Technology Enhanced Learning, 8(3/4), 217–233. https://doi.org/10.1504/IJTEL.2016.082317

96.

Poppenk

Köhler

Moscovitch

(2010). Revisiting the novelty effect: When familiarity, not novelty, enhances memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(5), 1321–1330. https://doi.org/10.1037/a0019900

97.

Powers

Ecott

Hirshfield

L. M.

(2007). Through the looking glass: Teaching CS0 with Alice. In Proceedings of the 38th SIGCSE technical symposium on computer science education (pp. 213–217). ACM. https://doi.org/10.1145/1227310.1227386

98.

* Price

T. W.

Barnes

(2015). Comparing textual and block interfaces in a novice programming environment. In Proceedings of the eleventh annual international conference on international computing education research (pp. 91–99). ACM. https://doi.org/10.1145/2787622.2787712

99.

Prince

M. J.

Felder

R. M.

(2006). Inductive teaching and learning methods: Definitions, comparisons, and research bases. Journal of Engineering Education, 95(2), 123–138. https://doi.org/10.1002/j.2168-9830.2006.tb00884.x

100.

Puente

E. L.

(2022). Effect of the use of block-based languages in programming learning. In 2022 international symposium on computers in education (SIIE) (pp. 1–6). IEEE. https://doi.org/10.1109/SIIE56031.2022.9982348

101.

Rich

P. J.

Bartholomew

Daniel

Dinsmoor

Nielsen

Reynolds

Swanson

Winward

Yauney

(2024). Trends in tools used to teach computational thinking through elementary coding. Journal of Research on Technology in Education, 56(3), 269–290. https://doi.org/10.1080/15391523.2022.2121345

102.

* Rodríguez-Martínez

J. A.

González-Calero

J. A.

Sáez-López

J. M.

(2020). Computational thinking and mathematics using scratch: An experiment with sixth-grade students. Interactive Learning Environments, 28(3), 316–327. https://doi.org/10.1080/10494820.2019.1612448

103.

* Ruf

Mühling

Hubwieser

(2014). Scratch vs. Karel: Impact on learning outcomes and motivation. In Proceedings of the 9th workshop in primary and secondary computing education (pp. 50–59). ACM. https://doi.org/10.1145/2670757.267077

104.

* Sáez-López

J. M.

Román-González

Vázquez-Cano

(2016). Visual programming languages integrated across the curriculum in elementary school: A two year case study using “scratch” in five schools. Computers & Education, 97(2), 129–141. https://doi.org/10.1016/j.compedu.2016.03.003

105.

Scherer

Siddiq

Sánchez Viveros

(2019). The cognitive benefits of learning computer programming: A meta-analysis of transfer effects. Journal of Educational Psychology, 111(5), 764–792. https://doi.org/10.1037/edu0000314

106.

Scherer

Siddiq

Sánchez Viveros

(2020). A meta-analysis of teaching and learning computer programming: Effective instructional approaches and conditions. Computers in Human Behavior, 109, Article 106349. https://doi.org/10.1016/j.chb.2020.106349

107.

Shin

I. S.

Chung

J. Y.

(2009). Class size and student achievement in the United States: A meta-analysis. KEDI Journal of Educational Policy, 6(2), 3–19. https://doi.org/10.22804/kjep.2009.6.2.001

108.

Singh

Wei

L. L.

Shanmugam

Gunasekaran

S. S.

Dorairaj

S. K.

Tenaga

Uniten

(2008). Designing computer games to introduce programming to children. In Proceedings of the 4th international conference on information technology and multimedia (pp. 643–647). ACM.

109.

Soomro

A. H.

110.

Sterne

J. A.

Sutton

A. J.

Ioannidis

J. P.

Terrin

Jones

D. R.

Lau

Carpenter

Rücker

Harbord

R. M.

Schmid

C. H.

Tetzlaff

Deeks

J. J.

Peters

Macaskill

Schwarzer

Duval

Altman

D. G.

Moher

Higgins

J. P.

(2011). Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ, 343. https://doi.org/10.1136/bmj.d4002

111.

Stewart

Baek

(2023). Analyzing computational thinking studies in scratch programming: A review of elementary education literature. International Journal of Computer Sciences and Engineering Systems, 6(1), 35–58. https://doi.org/10.21585/ijcses.v6i1.156

112.

* Sukirman Pramudita

D. A.

Afiyanto

Utaminingsih (2022). Block-based visual programming as a tool for learning the concepts of programming for novices. International Journal of Information and Education Technology, 12(5), 365–371. https://doi.org/10.18178/ijiet.2022.12.5.1628

113.

Sun

Guo

Zhou

(2022). Developing K-12 students’ programming ability: A systematic literature review. Education and Information Technologies, 27(5), 7059–7097. https://doi.org/10.1007/s10639-022-10891-2

114.

Sun

Zhou

(2021). Which way of design programming activities is more effective to promote K-12 students' computational thinking skills? A meta-analysis. Journal of Computer Assisted Learning, 37(4), 1048–1062. https://doi.org/10.1111/jcal.12545

115.

* Tabet

Gedawy

Alshikhabobakr

Razak

(2016). From Alice to Python. Introducing text-based programming in middle schools. In Proceedings of the 2016 ACM conference on innovation and technology in computer science education (pp. 124–129). ACM. https://doi.org/10.1145/2899415.2899462

116.

* Tekerek

Altan

(2014). The effect of scratch environment on student’s achievement in teaching algorithm. World Journal on Educational Technology, 6(2), 132–138.

117.

* Totan

H. N.

Korucu

A. T.

(2023). The effect of block based coding education on the students' attitudes about the secondary school students' computational learning skills and coding learning: Blocky sample. Participatory Educational Research, 10(1), 443–461. https://doi.org/10.17275/per.23.24.10.1

118.

Unahalekhaka

Bers

M. U.

(2022). Evaluating young children’s creative coding: Rubric development and testing for ScratchJr projects. Education and Information Technologies, 27(5), 6577–6597. https://doi.org/10.1007/s10639-021-10873-w

119.

UNESCO . (2023). Global education monitoring report summary 2023: Technology in education: A tool on whose terms? UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000386147_hin

120.

UNESCO . (2024). Global education monitoring report: Youth report–technology in education: A tool on our terms. UNESCO. https://doi.org/10.54676/ELSQ4648

121.

van Der Meulen

Hartendorp

Voorn

Hermans

(2022). The perception of teachers on usability and accessibility of programming materials for children with visual impairments. ACM Transactions on Computing Education, 23(1), 1–21. https://doi.org/10.1145/3561391

122.

* Vasilopoulos

I. V.

Van Schaik

(2019). Koios: Design, development, and evaluation of an educational visual tool for Greek novice programmers. Journal of Educational Computing Research, 57(5), 1227–1259. https://doi.org/10.1177/0735633118781776

123.

Vera-Baceta

M. A.

Thelwall

Kousha

(2019). Web of science and Scopus language coverage. Scientometrics, 121(3), 1803–1813. https://doi.org/10.1007/s11192-019-03264-z

124.

* Wang

T. C.

Mei

W. H.

Lin

S. L.

Chiu

S. K.

Lin

J. M. C.

(2009). Teaching programming concepts to high school students with Alice. In 39th IEEE Frontiers in education conference (pp. 1–6). IEEE. https://doi.org/10.1109/FIE.2009.5350486

125.

Webb

Davis

Bell

Katz

Y. J.

Reynolds

Chambers

D. P.

Sysło

M. M.

(2017). Computer science in K-12 school curricula of the 2lst century: Why, what and when? Education and Information Technologies, 22(2), 445–468. https://doi.org/10.1007/s10639-016-9493-x

126.

Weintrop

(2019). Block-based programming in computer science education. Communications of the ACM, 62(8), 22–25. https://doi.org/10.1145/3341221

127.

Weintrop

(2021). The role of block-based programming in computer science education. Understanding Computing Education, 1, 71–78. https://rpf.io/seminar-proceedings-2020

128.

Weintrop

Killen

Munzar

Franke

(2019). Block-based comprehension: Exploring and explaining student outcomes from a read-only block-based exam. In Proceedings of the 50th ACM echnical symposium on computer science education (pp. 1218–1224). ACM. https://doi.org/10.1145/3287324.3287348

129.

Weintrop

Wilensky

(2015). To block or not to block, that is the question: Students’ perceptions of blocks-based programming. In Proceedings of the 14th international conference on interaction design and children (pp. 199–208). ACM. https://doi.org/10.1145/2771839.2771860

130.

Weintrop

Wilensky

(2019). Transitioning from introductory block-based and text-based environments to professional programming languages in high school computer science classrooms. Computers & Education, 142, Article 103646. https://doi.org/10.1016/j.compedu.2019.103646

131.

* Weintrop

Wilensky

(2017). Comparing block-based and text-based programming in high school computer science classrooms. ACM Transactions on Computing Education, 18(1), 1–25. https://doi.org/10.1145/3089799

132.

Weng

O. L.

Cui

Leung

(2023). Creativity development with problem-based digital making and block-based programming for science, technology, engineering, arts, and mathematics learning in middle school contexts. Journal of Educational Computing Research, 61(2), 304–328. https://doi.org/10.1177/07356331221115661

133.

Wilson

M. L.

(2023). The impact of technology integration courses on preservice teacher attitudes and beliefs: A meta-analysis of teacher education research from 2007–2017. Journal of Research on Technology in Education, 55(2), 252–280. https://doi.org/10.1080/15391523.2021.1950085

134.

Wolber

(2011). App inventor and real-world motivation. In Proceedings of the 42nd ACM technical symposium on computer science education (pp. 601–606). ACM. https://doi.org/10.1145/1953163.1953329

135.

Wolber

Abelson

Friedman

(2015). Democratizing computing with app inventor. GetMobile: Mobile Computing & Communications, 18(4), 53–58. https://doi.org/10.1145/2721914.2721935

136.

Xinogalos

Satratzemi

Malliarakis

(2017). Microworlds, games, animations, mobile apps, puzzle editors and more: What is important for an introductory programming environment? Education and Information Technologies, 22(1), 145–176. https://doi.org/10.1007/s10639-015-9433-1

137.

Wang

(2023). A meta-analysis of the effectiveness of programming teaching in promoting K-12 students’ computational thinking. Education and Information Technologies, 28(6), 6619–6644. https://doi.org/10.1007/s10639-022-11445-2

138.

Ritzhaupt

A. D.

Tian

Umapathy

(2019). Block-based versus text-based programming environments on novice student learning outcomes: A meta-analysis study. Computer Science Education, 29(2-3), 177–204. https://doi.org/10.1080/08993408.2019.1565233

139.

Yilmaz

F. G. K.

(2023). Augmented intelligence in programming learning: Examining student views on the use of ChatGPT for programming learning. Computers in Human Behavior: Artificial Humans, 1(2), Article 100005. https://doi.org/10.1016/j.chbah.2023.100005

140.

(2023). Does motion-sensing technology enhance students’ learning? A meta-analysis. Journal of Educational Computing Research, 61(7), 1359–1388. https://doi.org/10.1177/07356331231176741

141.

Gao

Wang

(2021). The effect of educational games on learning outcomes, student motivation, engagement and satisfaction. Journal of Educational Computing Research, 59(3), 522–546. https://doi.org/10.1177/0735633120969214

142.

* Yünkül

Durak

Çankaya

Mısırlı

Z. A.

(2017). Scratch yazılımının öğrencilerin bilgisayarca düşünme becerilerine etkisi. Necatibey Faculty of Education Electronic Journal of Science and Mathematics Education, 11(2), 502–517. https://doi.org/10.17522/balikesirnef.373424

143.

Zhan

Tong

Liang

Guo

Lan

(2022). The effectiveness of gamification in programming education: Evidence from a meta-analysis. Computers and Education: Artificial Intelligence, 3(1), Article 100096. https://doi.org/10.1016/j.caeai.2022.100096

144.

Zheng

Long

Zhong

Gyasi

J. F.

(2022). The effectiveness of technology-facilitated personalized learning on learning achievements and learning perceptions: A meta-analysis. Education and Information Technologies, 27(8), 11807–11830. https://doi.org/10.1007/s10639-022-11092-7