Goal. Studying the effects of embedding human knowledge in intelligent tutoring systems in the context of cyber social systems. Problem. The usage of human- and machine-readable subject-domain models such as thought process graphs for building intelligent tutoring systems for the cognitive knowledge domain on the example of a tutor to teach variable scope. Methodology. In this study, we consider the structure and interactions of system components of the CompPrehension intelligent tutoring system. Then we explain the method of embedding human knowledge into it by constructing a subject domain model and demonstrate it on the example of the model for determining variable scopes. Then we provide results of the preliminary evaluation of the created tutor. Results. The learning gains of study participants after using the tutor for about 12 minutes were statistically significant, which proves that the tutor is effective. The learners who often repeated their errors in subsequent learning problems had smaller learning gains than the other learners; the average time-per-learning problem did not affect the learning gains. The results of the usability survey were positive. Implications. Using cognitive modeling by developing thought process graphs for building intelligent tutoring systems opens the way not just to building effective tutors, but also allows to solve advanced tasks like generating learning problems and pedagogical dialogue, which significantly expand the features of intelligent tutoring systems and bring them closer to the functions of human tutors.
BurnsH. L.CappsC. G. (1988). Foundations of intelligent tutoring systems (pp. 1–19). Hillsdale, NJ, USA: Lawrence Erlbaum Associates, Inc.
4.
DoostmohammadianM.RabieeH. R.KhanU. A. (2020). Cyber-social systems: Modeling, inference, and optimal design. IEEE Systems Journal, 14(1), 73–83. https://doi.org/10.1109/JSYST.2019.2900027
5.
FernandesE.KumarA. N. (2004). A tutor on scope for the programming languages course. SIGCSE Bulletin, 36(1), 90–93. https://doi.org/10.1145/1028174.971332
6.
FislerK.KrishnamurthiS.Tunnell WilsonP. (2017). Assessing and teaching scope, mutation, and aliasing in upper-level undergraduates. In Proceedings of the 2017 ACM SIGCSE technical symposium on computer science education. SIGCSE ’17 (pp. 213–218). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3017680.3017777
7.
GetsevaV.KumarA. N. (2022). An empirical analysis of code-tracing concepts. In Proceedings of the 27th ACM conference on innovation and technology in computer science education, Vol. 1. ITiCSE ’22 (pp. 262–268). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3502718.3524794
8.
GoldmanK.GrossP.HeerenC.HermanG.KaczmarczykL.LouiM. C.ZillesC. (2008). Identifying important and difficult concepts in introductory computing courses using a delphi process. SIGCSE Bulletin, 40(1), 256–260. https://doi.org/10.1145/1352322.1352226
9.
GunesV.PeterS.GivargisT.VahidF. (2014). A survey on concepts, applications, and challenges in cyber-physical systems. KSII Transactions on Internet and Information Systems (TIIS), 8(12), 4242–4268. https://doi.org/10.3837/tiis.2014.12.001
10.
HaglundP.StrombackF.MannilaL. (2022). Understanding students’ failure to use functions as a tool for abstraction—An analysis of questionnaire responses and lab assignments in a CS1 python course. Informatics in Education, 20(4), 583–614. https://doi.org/10.15388/infedu.2021.26
HasanM. A.NoorN. F. M.RahmanS. S. B. A.RahmanM. M. (2020). The transition from intelligent to affective tutoring system: A review and open issues. IEEE Access, 8, 204612–204638. https://doi.org/10.1109/ACCESS.2020.3036990
13.
HorváthI. (2023). Framing supradisciplinary research for intellectualized cyber-physical systems: An unfinished story. Journal of Computing and Information Science in Engineering, 23(6). https://doi.org/10.1115/1.4062327
KazemitabaarM.YeR.WangX.HenleyA. Z.DennyP.CraigM.GrossmanT. (2024). CodeAid: Evaluating a classroom deployment of an LLM-based programming assistant that balances student and educator needs. In Proceedings of the CHI conference on human factors in computing systems. CHI ’24. New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3613904.3642773
16.
KumarA. N. (2016). The effectiveness of visualization for learning expression evaluation: A reproducibility study. In Proceedings of the 2016 ACM conference on innovation and technology in computer science education. ITiCSE ’16 (pp. 192–197). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/2899415.2899427
17.
LeeY.JeongS.KimJ. (2024). Improving LLM classification of logical errors by integrating error relationship into prompts. In Lecture Notes in Computer Science (pp. 91–103). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-63028-6_8
18.
LeeE.SeshiaS. (2011). Introduction to embedded systems: A cyber-physical systems approach.
19.
LeinonenJ.DennyP.MacNeilS.SarsaS.BernsteinS.KimJ.TranA.HellasA. (2023). Comparing code explanations created by students and large language models. In Proceedings of the 2023 conference on innovation and technology in computer science education Vol. 1. ITiCSE 2023 (pp. 124–130). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3587102.3588785
20.
MaL.WangY.XuC.LiX. (2022). Online robotics technology course design by balancing workload and affect. Journal of Integrated Design and Process Science, 26(2), 131–158. https://doi.org/10.3233/JID-210026
21.
MossesP. D. (2021). Fundamental constructs in programming languages. In MargariaT.SteffenB. (Eds.), Leveraging applications of formal methods, verification and validation (pp. 296–321). Cham: Springer International Publishing.
22.
MüllerS.BergandeB.BruneP. (2018). Robot tutoring: On the feasibility of using cognitive systems as tutors in introductory programming education: A teaching experiment. In Proceedings of the 3rd European conference of software engineering education. ECSEE ’18 (pp. 45–49). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3209087.3209093
23.
NamD.MacveanA.HellendoornV.VasilescuB.MyersB. (2024). Using an LLM to help with code understanding. In Proceedings of the IEEE/ACM 46th international conference on software engineering. ICSE ’24. New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3597503.3639187
24.
NapsT. L.RößlingG.AlmstrumV.DannW.FleischerR.HundhausenC.KorhonenA.MalmiL.McNallyM.RodgerS.Velázquez-IturbideJ. A. (2002). Exploring the role of visualization and engagement in computer science education. SIGCSE Bulletin, 35(2), 131–152. https://doi.org/10.1145/782941.782998
OliP.BanjadeR.Lekshmi NarayananA. B.BrusilovskyP.RusV. (2024). Exploring the effectiveness of reading vs. tutoring for enhancing code comprehension for novices. In Proceedings of the 39th ACM/SIGAPP symposium on applied computing. SAC ’24 (pp. 38–47). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3605098.3636007
SchreppM.HinderksA.ThomaschewskiJ. (2017). Construction of a benchmark for the user experience questionnaire (UEQ). International Journal of Interactive Multimedia and Artificial Intelligence, 4, 40–44. https://doi.org/10.9781/ijimai.2017.445
29.
ShenY.AiX.Soosai RajA. G.Leo JohnR. J.SyamkumarM. (2024). Implications of ChatGPT for data science education. In Proceedings of the 55th ACM technical symposium on computer science education Vol. 1. SIGCSE 2024 (pp. 1230–1236). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3626252.3630874
30.
ShethA. (2010). Computing for human experience: Semantics-empowered sensors, services, and social computing on the ubiquitous Web. IEEE Internet Computing, 14(1), 88–91. https://doi.org/10.1109/MIC.2010.4
31.
ShishaklyR.AlmaiahM.LutfiA.AlrawadM. (2024). The influence of using smart technologies for sustainable development in higher education institutions. International Journal of Data and Network Science, 8(1), 77–90. https://doi.org/10.5267/j.ijdns.2023.10.015
32.
SoweS. K.SimmonE.ZettsuK.de VaulxF.BojanovaI. (2016). Cyber-physical-human systems: Putting people in the loop. IT Professional, 18(1), 10–13. https://doi.org/10.1109/MITP.2016.14
33.
StrömbäckF.HaglundP.BerglundA.BerglundE. (2023). The progression of students’ ability to work with scope, parameter passing and aliasing. In Proceedings of the 25th Australasian computing education conference. ACE ’23 (pp. 39–48). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3576123.3576128
34.
SuZ.QiQ.XuQ.GuoS.WangX. (2020). Incentive scheme for cyber physical social systems based on user behaviors. IEEE Transactions on Emerging Topics in Computing, 8(1), 92–103. https://doi.org/10.1109/TETC.2017.2671843
SychevO. (2024). Educational models for cognition: Methodology of modeling intellectual skills for intelligent tutoring systems. Cognitive Systems Research, 87, 101261. https://doi.org/10.1016/j.cogsys.2024.101261
37.
SychevO.PenskoyN.AnikinA.DenisovM.ProkudinA. (2021). Improving comprehension: Intelligent tutoring system explaining the domain rules when students break them. Education Sciences, 11(11). https://doi.org/10.3390/educsci11110719
38.
SychevO.PenskoyN.TerekhovG. (2022). Using software reasoning to determine domain-law violations and provide explanatory feedback: Expressions tutor example. In Proceedings of the 14th international conference on computer supported education - Volume 1: CSEDU (pp. 116–123). SciTePress. INSTICC. https://doi.org/10.5220/0011070100003182
39.
TavčarJ.DuhovnikJ.HorváthI. (2019). From validation of medical devices towards validation of adaptive cyber-physical systems. Journal of Integrated Design and Process Science, 23(1), 37–59. https://doi.org/10.3233/JID190008
40.
WangD.AminM. T.LiS.AbdelzaherT.KaplanL.GuS.PanC.LiuH.AggarwalC. C.GantiR.WangX.MohapatraP.SzymanskiB.LeH. (2014). Using humans as sensors: An estimation-theoretic perspective. In IPSN-14 Proceedings of the 13th international symposium on information processing in sensor networks (pp. 35–46). https://doi.org/10.1109/IPSN.2014.6846739
41.
WeynsD.AnderssonJ.CaporuscioM.FlamminiF.KerrenA.LöweW. (2021). A research agenda for smarter cyber-physical systems. Journal of Integrated Design and Process Science, 25(2), 27–47. https://doi.org/10.3233/JID210010
42.
YilmaB. A.NaudetY.PanettoH. (2019). Introduction to personalisation in cyber-physical-social systems. In DebruyneC.PanettoH.GuédriaW.BollenP.CiuciuI.MeersmanR. (Eds.), On the move to meaningful internet systems: OTM 2018 workshops (pp. 25–35). Cham: Springer International Publishing.
