This study aimed to develop an app called Physics Multimodal Authentic Generated Interactive Content Problem (P-MAGIC) to gather authentic contextual information by using object recognition and real-time sensors to help students learn the principles of centripetal acceleration in Physics. We implemented an instructional-based prompt generator that adjusts the difficulty level of problems in three levels, with the questions themselves being generated by the Generative Pre-trained version 4o (GPT-4o) model. Furthermore, to evaluate our app, we assessed the quality of the generated questions using automatic and human evaluations. The automatic evaluation demonstrated that the questions were relevant, easy to understand, and diverse. Additionally, feedback from teachers showed high ratings for relevance, completeness, and concept accuracy. Interestingly, problems incorporating both text and images, followed by text, images, and graphs, outperformed those that were based on text and graphs or text and tables. This highlights the positive impact of visual elements on conceptual understanding and student engagement. Furthermore, the strong correlation between automatic and human evaluations indicates that automated assessments can reliably reflect human judgement, making them a scalable tool for evaluations. By providing teachers with the tools to create personalized and context-rich problems in a variety of formats could potentially transform physics education.
AbrahamS. S.KhemaniD. (2016). Hybrid of qualitative and quantitative knowledge models for solving physics word problems. In Proceedings of the twenty-ninth Florida artificial intelligence research society conference (FLAIRS-29). https://cdn.aaai.org/ocs/12939/12939-57691-1-PB.pdf
AllisonJ. (2026). RAISE the standard: A framework for transparent reporting of artificial intelligence studies in education. Journal of Educational Computing Research, 64(1), 3–15. https://doi.org/10.1177/07356331251377430
4.
AndersonL. W.KrathwohlD. R. (Eds.), (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives (Complete ed.). Addison Wesley Longman.
BaltruaitisT.AhujaC.MorencyL.-P. (2019). Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2), 423–443. https://doi.org/10.1109/TPAMI.2018.2798607
7.
DevlinJ.ChangM.-W.LeeK.ToutanovaK. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the north American chapter of the association for computational linguistics: Human language technologies, volume 1 (Long and short papers) (pp. 4171–4186). Association for Computational Linguistics. https://aclanthology.org/N19-1423/
8.
DingJ.CenY.WeiX. (2023). Using large language model to solve and explain physics word problems approaching human level. arXiv. https://doi.org/10.48550/arXiv.2309.08182
9.
El-AdawyS.LiaoI.LadV.AbdelhafezM.DourmashkinP. (2024). Streamlining physics problem generation to support physics teachers in using generative artificial intelligence. The Physics Teacher, 62(7), 595–598. https://doi.org/10.1119/5.0201458
10.
FitzpatrickC. L.HallettD.MorrisseyK. R.YıldızN. R.WynesR.AyesuF. (2019). Response sentences, examples, and authenticity do not help children solve real world problems. Learning and Instruction, 61(1), 111–125. https://doi.org/10.1016/j.learninstruc.2018.10.002
11.
GattA.KrahmerE. (2018). Survey of the state of the art in natural language generation: Core tasks, applications and evaluation. Journal of Artificial Intelligence Research, 61(1), 65–170. https://doi.org/10.1613/jair.5477
12.
GhoshK.DasS. (2023). Automatic generation of algebraic representation for physics problems. In Proceedings of the 3rd international conference on educational technology and online learning (ICETOL 2023) full paper proceedings.2023/06/20.
13.
GilbertJ. K. (2010). The role of visual representations in the learning and teaching of science: An introduction. Asia-Pacific Forum on Science Learning and Teaching, 11(1). https://www.eduhk.hk/apfslt/v11_issue1/foreword/index.htm
HorbachA.AldabeI.BexteM.de LacalleO. L.MaritxalarM. (2020). Linguistic appropriateness and pedagogic usefulness of reading comprehension questions. In Proceedings of the twelfth language resources and evaluation conference (LREC 2020) (pp. 1753–1762). European Language Resources Association. https://aclanthology.org/2020.lrec-1.217/
17.
HwangW.-Y.UtamiI. Q. (2024). Using GPT and authentic contextual recognition to generate math word problems with difficulty levels. Education and Information Technologies, 29(13), 17635–17663. https://doi.org/10.1007/s10639-024-12537-x
JewittC.KressG.OgbornJ.TsatsarelisC. (2001). Multimodal teaching and learning: The rhetorics of the science classroom. Continuum.
22.
KahanJ.KoivunenM.-R. (2001). Annotea: An open RDF infrastructure for shared web annotations. In Proceedings of the 10th international conference on world wide web (pp. 623–632). ACM.
23.
KüchemannS.SteinertS.RevengaN.SchweinbergerM.DincY.AvilaK. E.KuhnJ. (2023). Can ChatGPT support prospective teachers in physics task development?Physical Review Physics Education Research, 19(2), 020128. https://doi.org/10.1103/PhysRevPhysEducRes.19.020128
24.
KurdiG.LeoJ.ParsiaB.SattlerU.Al-EmariS. (2020). A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education, 30(1), 121–204. https://doi.org/10.1007/s40593-019-00186-y
25.
LemkeJ. L. (2004). The literacies of science. In SaulE. W. (Ed.), Crossing borders in literacy and science instruction: Perspectives on theory and practice (pp. 33–47). International Reading Association. https://doi.org/10.1598/0872075192.2
26.
LingJ.AfzaalM. (2024). Automatic question-answer pairs generation using pre-trained large language models in higher education. Computers and Education: Artificial Intelligence, 6, 100252. https://doi.org/10.1016/j.caeai.2024.100252
27.
LuoJ.ShanH.ZhangG.YuanG.ZhangS.YanF.LiZ. (2021). Exploiting syntactic and semantic information for textual similarity estimation. Mathematical Problems in Engineering, 2021(1), 4186750. https://doi.org/10.1155/2021/4186750
28.
Marchotridyo and RuskandaF. (2024). Physics assessment generation through pattern matching and large language models. In Proceedings of the 2024 11th international conference on advanced informatics: Concept, theory and application (ICAICTA),2024/09/28. https://doi.org/10.1109/ICAICTA63815.2024.10763293
29.
MazidiK.NielsenR. D. (2014). Linguistic considerations in automatic question generation. In Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 2: Short papers) (pp. 321–326). Association for Computational Linguistics. https://doi.org/10.3115/v1/P14-2053
MooreS.NguyenH. A.BierN.DomadiaT.StamperJ. (2022). Assessing the quality of student-generated short answer questions using GPT-3. In Proceedings of the European conference on technology enhanced learning.
32.
NokelainenP.KurhilaJ.MiettinenM.FloréenP.TirriH. (2003). Evaluating the role of a shared document-based annotation tool in learner-centered collaborative learning. In Proceedings of the 3rd IEEE international conference on advanced learning technologies (ICALT’03) (pp. 200–203). IEEE.
33.
OtaniM.TogashiR.SawaiY.IshigamiR.NakashimaY.RahtuE.HeikkiläJ.SatohS. (2023). Toward verifiable and reproducible human evaluation for text-to-image generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE. https://arxiv.org/abs/2304.01816
34.
PatelA.BhattamishraS.GoyalN. (2021). Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 conference of the north American chapter of the association for computational linguistics: Human language technologies (pp. 2080–2094). Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.naacl-main.168
35.
PolozovO.O’RourkeE.SmithA. M.ZettlemoyerL.GulwaniS.PopovićZ. (2015). Personalized mathematical word problem generation. In Proceedings of the twenty-fourth international joint conference on artificial intelligence (IJCAI-15) (pp. 381–388). AAAI Press. https://www.ijcai.org/Proceedings/15/Papers/060.pdf
36.
RoyS.RothD. (2015). Solving general arithmetic word problems. In Proceedings of the 2015 conference on empirical methods in natural language processing (pp. 1743–1752). Association for Computational Linguistics. https://doi.org/10.18653/v1/D15-1202
37.
ScariaN.ChennaS. D.SubramaniD. (2024). Automated educational question generation at different Bloom’s skill levels using large language models: Strategies and evaluation. In Proceedings of the international conference on artificial intelligence in education, ArXiv, abs/2408.04394.
38.
ScharpfP.SchubotzM.SpitzA.Greiner-PetterA.GippB. (2022). Collaborative and AI-aided exam question generation using wikidata in education. arXiv. https://arxiv.org/abs/2211.08361
39.
ShiD.ZhaoS.LiuQ.ZhangY. (2024). Research and design of automatic questioning system based on question generation. In Proceedings of the 2024 12th international conference on information and education technology (ICET), pp. 182-186, Yamaguchi, Japan. https://doi.org/10.1109/ICIET60671.2024.10542827
40.
SinghalR.GoyalS.HenzM. (2016). User-defined difficulty levels for automated question generation. In Proceedings of the 2016 IEEE 28th international conference on tools with artificial intelligence (ICTAI) (pp. 828–835). IEEE. https://doi.org/10.1109/ICTAI.2016.0129
StefanelA. (2019). Graph in physics education: From representation to conceptual understanding. In PospiechG.MicheliniM.EylonB.-S. (Eds.), Mathematics in physics education (pp. 195–231). Springer. https://doi.org/10.1007/978-3-030-04627-9_9
43.
SteuerT.BongardL.UhligJ.ZimmerG. (2021). On the linguistic and pedagogical quality of automatic question generation via neural machine translation. In Proceedings of the European conference on technology enhanced learning (pp. 289–294). Springer. https://doi.org/10.1007/978-3-030-86436-1_22
44.
Uz BilginC.TokelS. T. (2019). Facilitating contextual vocabulary learning in a mobile-supported situated learning environment. Journal of Educational Computing Research, 57(4), 930–953. https://doi.org/10.1177/0735633118779397
45.
van LeeuwenT. (2005). Introducing social semiotics. Routledge.
46.
VenkteshV.AkhtarM. S.MohaniaM.GoyalV. (2022). Auxiliary task guided interactive attention model for question difficulty prediction. arXiv. https://arxiv.org/abs/2207.01494
47.
WangH.-C.MaslimM.KanC.-H. (2023). A question–answer generation system for an asynchronous distance learning platform. Education and Information Technologies, 28(9), 12059–12088. https://doi.org/10.1007/s10639-023-11675-y
48.
WanselinH.DanielssonK.WikmanS. (2022). Analysing multimodal texts in science—A social semiotic perspective. Research in Science Education, 52, 891–907. https://doi.org/10.1007/s11165-021-10027-5
49.
YangS. J.ChenI. Y.-L.ShaoN. W. (2004). Ontology enabled annotation and knowledge management for collaborative learning in virtual learning community. Journal of Educational Technology & Society, 7(4), 70–81.
50.
ZhangD.WangL.ZhangL.DaiB. T.ShenH. T. (2020). The gap of semantic parsing: A survey on automatic math word problem solvers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(9), 2287–2305. https://doi.org/10.1109/TPAMI.2019.2914054
ZhangR.GuoJ.ChenL.FanY.ChengX. (2021). A review on question generation from natural language text. ACM Transactions on Information Systems, 40(1), 1–43. https://doi.org/10.1145/3468889
ZhaoW. X.ZhouK.LiJ.TangT.WangX.HouY.MinY.ZhangB.ZhangJ.DongZ.DuY.YangC.ChenY.ChenZ.JiangJ.RenR.LiY.TangX.LiuZ.WenJ.-R. (2023). A survey of large language models. arXiv. https://arxiv.org/abs/2303.18223
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
