This study implemented a data-driven approach to identify Chinese high school students’ common errors in a Java-based introductory programming course using the data in an automated assessment tool called the Mulberry. Students’ error-related behaviors were also analyzed, and their relationships to success in introductory programming were investigated. This study identified 15 common compilation errors and 6 common test errors. The results showed that these common errors accounted for a large proportion of all errors, so identifying the common errors is important to help students succeed in introductory programming courses. Based on these common errors, five underlying student difficulties were identified and are discussed. In addition, after analyzing existing measures of students’ error-related behaviors, we developed a measure called improvement rate to quantify students’ success in fixing errors. The results of our study suggest that students’ competence of improving code is important to their success in introductory programming. We recommend researchers design and develop automated assessment tools that provide feedback messages for common student errors and instructors who explicitly teach knowledge and skills of improving code in class.
AltadmriA.BrownN. C. (2015). 37 Million compilations: Investigating novice programming mistakes in large-scale student data. In Proceedings of the 46th ACM technical symposium on computer science education (pp. 522–527). New York, NY: Association for Computing Machinery. doi:10.1145/2676723.2677258
2.
AusubelD. P. (2000). The acquisition and retention of knowledge: A cognitive view.
Dordrecht, The Netherlands:
Kluwer Academic Publishers.
3.
BeckerB. A. (2016). An effective approach to enhancing compiler error messages. In Proceedings of the 47th ACM technical symposium on computing science education—SIGCSE ’16 (pp. 126–131). New York, NY: ACM. doi:10.1145/2839509.2844584
4.
BennedsenJ.CaspersenM. E. (2005). An investigation of potential success factors for an introductory model-driven programming course. In Proceedings of the 2005 international workshop on computing education research—ICER ’05 (pp. 155–163). New York, NY: ACM. doi:10.1145/1089786.1089801
BonarJ.SolowayE. (1985).
Preprogramming knowledge: A major source of misconceptions in novice programmers.Human-Computer Interaction,
1(2), 133–161. doi:10.1207/s15327051hci0102_3
7.
BrownN. C. C.AltadmriA. (2017).
Novice java programming mistakes: Large-scale data vs. educator beliefs.ACM Transactions on Computing Education,
17(2), 1–7. doi:10.1145/2994154
8.
ByrneP.LyonsG. (2001).
The effect of student attributes on success in programming.SIGCSE Bulletin,
33(3), 49–52. doi:10.1145/507758.377467
9.
CarlsenW. (1999). Domains of teacher knowledge. In J. Gess-Newsome & N. G. Lederman (Eds.), Examining pedagogical content knowledge: The construct and its implications for science education (pp. 133–144). Dordecht, The Netherlands: Kluwer Academic Publishers.
10.
ClancyM. J.LinnM. C. (1999).
Patterns and pedagogy.ACM SIGCSE Bulletin,
31(1), 37–42. doi:10.1145/384266.299673
11.
DennyP.Luxton-ReillyA.TemperoE. (2012). All syntax errors are not equal. In Proceedings of the 17th ACM annual conference on innovation and technology in computer science education—ITiCSE ’12 (pp. 75–80).New York, NY: ACM. doi:10.1145/2325296.2325318
12.
du BoulayB. (1986).
Some difficulties of learning to program.Journal of Educational Computing Research,
2(1), 57–73. doi:10.2190/3LFX-9RRF-67T8-UVK9
13.
FislerK.KrishnamurthiS.SiegmundJ. (2016). Modernizing plan-composition studies. In Proceedings of the 47th ACM technical symposium on computing science education—SIGCSE ’16 (pp. 211–216).New York, NY: ACM. doi:10.1145/2839509.2844556
14.
Gal-ezerJ.StephensonC. (2014).
A tale of two countries: Successes and challenges in K-12 computer science education in Israel and the United States.ACM Transactions on Computing Education,
14(2), 1–8. doi:10.1145/2602483
15.
Guzdial, M. (1995). Centralized mindset: A student problem with object-oriented programming. ACM SIGCSE Bulletin, 27(1), 182–185. doi:10.1145/199691.199772
16.
GuzdialM. (2015).
Learner-centered design of computing education: Research on computing for everyone.Synthesis Lectures on Human-Centered Informatics,
8(6), 1–165. doi:10.2200/S00684ED1V01Y201511HCI033
17.
GuzdialM. (2016).
Bringing computer science to U.S. schools, state by state.Communications of the ACM,
59(5), 24–25. doi:10.1145/2898963
18.
IhantolaP.AhoniemiT.KaravirtaV.SeppäläO. (2010). Review of recent systems for automatic assessment of programming assignments. In Proceedings of the 10th Koli Calling international conference on computing education research—Koli Calling ’10 (pp. 86–93). New York, NY: ACM. doi:10.1145/1930464.1930480
19.
JacksonJ.CobbM.CarverC. (2005). Identifying top Java errors for novice programmers. In Proceedings frontiers in education 35th annual conference (pp. T4C–T4C). Indianopolis, IN: IEEE.doi:10.1109/FIE.2005.1611967
20.
JadudM. C. (2006). Methods and tools for exploring novice compilation behaviour. In Proceedings of the second international workshop on computing education research (pp. 73–84). New York, NY: ACM.doi:10.1145/1151588.1151600
21.
JadudM. C.DornB. (2015). Aggregate compilation behavior: Findings and implications from 27,698 users. In Proceedings of the eleventh annual international conference on international computing education research (pp. 131–139). New York, NY: ACM.doi:10.1145/2787622.2787718
22.
KaczmarczykL. C.PetrickE. R.EastJ. P.HermanG. L. (2010). Identifying student misconceptions of programming.In Proceedings of the 41st ACM technical symposium on computer science education (pp. 107–111). New York, NY: ACM. doi:10.1145/1734263.1734299
23.
McCallD.KöllingM. (2019).
A new look at novice programmer errors.ACM Transactions on Computational Education,
19(4), 1–38. doi:10.1145/3335814
24.
MillerC. S. (2014).
Metonymy and reference-point errors in novice programming.Computer Science Education,
24(2–3), 123–152. doi:10.1080/08993408.2014.952500
MullerO.GinatD.HabermanB. (2007). Pattern-oriented instruction and its influence on problem decomposition and solution construction. In Proceedings of the 12th annual SIGCSE conference on innovation and technology in computer science education (pp. 151–155). New York, NY: ACM. doi:10.1145/1268784.1268830
27.
PeaR. D. (1986).
Language-independent conceptual “bugs” in novice programming.Journal of Educational Computing Research,
2(1), 25–36. doi:10.2190/689T-1R2A-X4W4-29J2
28.
PetersenA.SpaccoJ.VihavainenA. (2015). An exploration of error quotient in multiple contexts. In Proceedings of the 15th Koli calling conference on computing education research (pp. 77–86). New York, NY: ACM. doi:10.1145/2828959.2828966
29.
PettitR. S.HomerJ.GeeR. (2017). Do enhanced compiler error messages help students? In Proceedings of the 2017 ACM SIGCSE technical symposium on computer science education—SIGCSE ’17 (pp. 465–470). New York, NY: ACM. doi:10.1145/3017680.3017768
30.
QianY.LehmanJ. (2016).
Correlates of success in introductory programming: A study with middle school students.Journal of Education and Learning,
5(2), 73–83. doi:10.5539/jel.v5n2p73
31.
QianY.LehmanJ. (2017).
Students’ misconceptions and other difficulties in introductory programming: A literature review.ACM Transactions on Computing Education,
18(1), 1–1. doi:10.1145/3077618
32.
QianY.LehmanJ. (2018).
Using technology to support teaching computer science: A study with middle school students. Eurasia Journal of Mathematics, Science and Technology Education,
14(12). doi:10.29333/ejmste/94227
33.
QianY.YanP.ZhouM. (2019). Using data to understand difficulties of learning to program: A study with Chinese middle school students. In Proceedings of the ACM conference on global computing education (pp. 185–191). New York, NY: ACM. doi:10.1145/3300115.3309521
34.
ShulmanL. (1986).
Those who understand: Knowledge growth in teaching.Educational Researcher,
15(2), 4–14.
35.
Simon. (2011). Assignment and sequence: Why some students can't recognise a simple swap. In Proceedings of the 11th Koli Calling International Conference on Computing Education Research (pp. 10–15). New York, NY, USA: ACM. doi:10.1145/2094131.2094134
36.
SleemanD.PutnamR. T.BaxterJ.KuspaL. (1986).
Pascal and high school students: A study of errors.Journal of Educational Computing Research,
2(1), 5–23. doi:10.2190/2XPP-LTYH-98NQ-BU77
37.
SorvaJ. (2013).
Notional machines and introductory programming education.ACM Transactions on Computing Education,
13(2), 1–31. doi:10.1145/2483710.2483713
38.
SpohrerJ. C.SolowayE. (1986).
Novice mistakes: Are the folk wisdoms correct?Communications of the ACM,
29(7), 624–632. doi:10.1145/6138.6145
39.
TaberK. S. (2013). Modeling learners and learning in science education.
New York, NY:
Springer.
40.
WatsonC.LiF. W. B. (2014). Failure rates in introductory programming revisited. In Proceedings of the 2014 conference on innovation & technology in computer science education—ITiCSE ’(p. 14). New York, NY: ACM.doi:10.1145/2591708.2591749
41.
WatsonC.LiF. W. B.GodwinJ. L. (2013). Predicting performance in anintroductory programming course by logging and analyzing student programming behavior. In 2013 IEEE 13th international conference on advanced learning technologies (pp. 319–323).Beijing, China: IEEE. doi:10.1109/ICALT.2013.99
42.
WebbM.DavisN.BellT.KatzY.ReynoldsN.ChambersD. P., … SysłoM. M. (2017).
Computer science in K-12 school curricula of the 2lst century: Why, what and when?Education and Information Technologies,
22(2), 445–468. doi:10.1007/s10639-016-9493-x
