We present a privacy-friendly early-detection framework to identify students at risk of failing in introductory programming courses at university. The framework was validated for two different courses with annual editions taken by higher education students (N = 2 080) and was found to be highly accurate and robust against variation in course structures, teaching and learning styles, programming exercises and classification algorithms. By using interpretable machine learning techniques, the framework also provides insight into what aspects of practising programming skills promote or inhibit learning or have no or minor effect on the learning process. Findings showed that the framework was capable of predicting students’ future success already early on in the semester.
AsifR.MerceronA.AliS. A.HaiderN. G. (2017). Analyzing undergraduate students’ performance using educational data mining. Computers & Education, 113, 177–194. https://doi.org/10.1016/j.compedu.2017.05.007
2.
BakerR. S. J. D.YacefK. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–17. https://doi.org/10.5281/zenodo.3554657
BinnsR.Van KleekM.VealeM.LyngsU.ZhaoJ.ShadboltN. (2018). “It’s reducing a human being to a percentage”: Perceptions of justice in algorithmic decisions [Paper presentation]. Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsNew York, NY, USA, April 21, 1–14. https://doi.org/10.1145/3173574.3173951
5.
BlackP.WiliamD. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74. https://doi.org/10.1080/0969595980050102
EdwardsJ. M.FultonE. K.HolmesJ. D.ValentinJ. L.BeardD. V.ParkerK. R. (2018). Separation of syntax and problem solving in introductory computer programming [Paper presentation]. 2018 IEEE Frontiers in Education Conference(FIE), 1–5, San Jose, CA, USA, October 3–6. https://doi.org/10.1109/FIE.2018.8658852
9.
FeldmanM. Q.WangY.ByrdW. E.GuimbretièreF.AndersenE. (2019). Towards answering “Am I on the right track?” Automatically using program synthesis [Paper presentation]. Proceedings of the 2019 ACM SIGPLAN Symposium on SPLASH-E, Athens, Greece, October 25, 13–24. https://doi.org/10.1145/3358711.3361626
Grgić-HlačaN.ZafarM. B.GummadiK. P.WellerA. (2018). The case for process fairness in learning: Feature selection for fair decision making 11. NIPS symposium on machine learning and the law.
13.
HonourW. (1986). Predicting student performance in a beginning computer science class. ACM SIGCSE Bulletin, 18(1), 138–143. https://doi.org/10.1145/953055.5701
14.
KhalifaM.LamR. (2002). Web-based learning: Effects on learning process and outcome. IEEE Transactions on Education, 45(4), 350–356. https://doi.org/10.1109/TE.2002.804395
15.
KleinbaumD. G. (1994). Introduction to logistic regression. In KleinbaumD. G. (Ed.), Logistic regression: A self-learning text (pp. 1–38). Springer. https://doi.org/10.1007/978-1-4757-4108-7_1
RountreeN.RountreeJ.RobinsA.HannahR. (2004). Interacting factors that predict success and failure in a CS1 course [Paper presentation]. Working Group Reports from ITiCSE on Innovation and Technology in Computer Science Education, New York, NY, USA, June 28, 101–104. https://doi.org/10.1145/1044550.1041669
23.
SvetnikV.LiawA.TongC.CulbersonJ. C.SheridanR. P.FeustonB. P. (2003). Random forest: A classification and regression tool for compound classification and qsar modeling. Journal of Chemical Information and Computer Sciences, 43(6), 1947–1958. https://doi.org/10.1021/ci034160g
24.
VihavainenA. (2013). Predicting students’ performance in an introductory programming course using data from students’ own programming process [Paper presentation]. 2013 IEEE 13th International Conference on Advanced Learning TechnologiesBeijing, China, July 15–18, 498–499. https://doi.org/10.1109/ICALT.2013.161
25.
VihavainenA.LuukkainenM.KurhilaJ. (2013). Using students’ programming behavior to predict success in an introductory mathematics course [Paper presentation]. Educational data mining 2013, Beijing, China, July 15–18.
26.
WasikS.AntczakM.BaduraJ.LaskowskiA.SternalT. (2018). A survey on online judge systems and their applications. ACM Computing Surveys, 51(1), 1–3. https://doi.org/10.1145/3143560
27.
WatsonC.LiF. W. B. (2014). Failure rates in introductory programming revisited [Paper presentation]. Proceedings of the 2014 Conference on Innovation & Technology in Computer Science Education, New York, NY, USA, June 21, 39–44. https://doi.org/10.1145/2591708.2591749
WilkinsonM. D.DumontierM.AalbersbergI. J.AppletonG.AxtonM.BaakA.BlombergN.BoitenJ.-W.da Silva SantosL. B.BourneP. E.BouwmanJ.BrookesA. J.ClarkT.CrosasM.DilloI.DumonO.EdmundsS.EveloC. T.FinkersR.MonsB. (2016). The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3(1), 160018. https://doi.org/10.1038/sdata.2016.18
30.
XingW.DuD. (2019). Dropout prediction in MOOCs: Using deep learning for personalized intervention. Journal of Educational Computing Research, 57(3), 547–570. https://doi.org/10.1177/0735633118757015
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
