Following the launch of the generative AI Web application, Ask.SMILE, designed to evaluate the cognitive levels of questions asked, 2559 educators generated 25,973 question-feedback sets over a three-month period, with an average of over 10 questions per participant. Analyses revealed a significant improvement in question quality from initial submissions to later stages. Specifically, participants progressed from Level 1 questions (simple recall) to Level 5 questions (creative and evaluative), based on Bloom’s Taxonomy. However, participants who began with lower-quality questions showed varied progression paths to higher levels. To analyze the data, a Linear Mixed Model (LMM) was used to account for both fixed and random effects, revealing a statistically significant improvement in question quality over time (p < .01). ANOVA results further confirmed the significance of question stage on quality, and a Least Squares Means (LSMEANS) analysis indicated a precise and significant improvement in later-stage questions. Additionally, survey responses highlighted participants’ appreciation for the ongoing development of AI-driven educational tools like Ask.SMILE. Several implications and future research directions are explored, including the potential to expand the platform’s features beyond question evaluation.
AdesholaI.AdepojuA. P. (2023). The opportunities and challenges of ChatGPT in education. Interactive Learning Environments, 32(10), 6159–6172. https://doi.org/10.1080/10494820.2023.2253858
2.
AflaloE. (2021). Students generating questions as a way of learning. Active Learning in Higher Education, 22(1), 63–75. https://doi.org/10.1177/1469787418769120
3.
AliF.ChoyD.DivaharanS.TayH.ChenW. (2023). Supporting self-directed learning and self-assessment using TeacherGAIA, a generative AI chatbot application: Learning approaches and prompt engineering. Learning: Research and Practice, 9(2), 135–147. https://doi.org/10.1080/23735082.2023.2258886
Baidoo-AnuD.AnsahL. O. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52–62. https://doi.org/10.61969/jai.1337500
6.
BeckerR. R. (2000). The critical role of students' questions in literacy development. The Educational Forum, 64(3), 261–271. https://doi.org/10.1080/00131720008984763
7.
BenzonW. L. (2023). ChatGPT intimates a tantalizing future; its core LLM is organized on multiple levels; and it has broken the idea of thinking, Version 3. https://doi.org/10.2139/ssrn.4336442
8.
BloomB. S.EnglehartM. D.FurstE. J.HillW. H.KrathwohlD. R. (1956). The taxonomy of educational objectives, handbook I: The Cognitive domain. David McKay Co., Inc.
9.
BonkC. J.ReynoldsT. H. (1992). Early adolescent composing within a generative-evaluative computerized prompting framework. Computers in Human Behavior, 8(1), 39–62. https://doi.org/10.1016/0747-5632(92)90018-A
10.
BozkurtA. (2023). Generative artificial intelligence (AI) powered conversational educational agents: The inevitable paradigm shift. Asian Journal of Distance Education, 18(1), 198–204. https://doi.org/10.5281/zenodo.7716416
11.
BucknerE.KimP. (2014). Integrating technology and pedagogy for inquiry-based learning: The Stanford mobile inquiry-based learning environment (SMILE). Prospects, 44(1), 99–118. https://doi.org/10.1007/s11125-013-9269-7
12.
ChanC. K. Y.HuW. (2023). Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education. International Journal of Educational Technology in Higher Education, 20(1), 43. https://doi.org/10.1186/s41239-023-00411-8
13.
DemszkyD.LiuJ.HillH. C.JurafskyD.PiechC. (2023). Can automated feedback improve teachers’ uptake of student ideas? Evidence from a randomized controlled trial in a large-scale online course. Educational Evaluation and Policy Analysis, 46(3), 483–505. https://doi.org/10.3102/01623737231169270
KimP. (2024). Trends of AI in education and new competencies for the future at exploring the ethical use of AI TIP conference. YouTube. [Video]. https://youtube/DsUEhxlNku8?si=h2vXfm-P2zNfdg5o
16.
KimP.AnJ. Y. (2016). New evaluation vector through the Stanford mobile inquiry-based learning environment (smile) for participatory action research. Healthcare Informatics Research, 22(3), 164–171. https://doi.org/10.4258/hir.2016.22.3.164
17.
KimP.LeeJ. (2020). Evolution of online learning environments and the emergence of intelligent MOOCs. In ZhangK.BonkC. J.ReevesT. C.ReynoldsT. H. (Eds.), MOOCs and open education in the Global South: Challenges, successes, and opportunities (pp. 329–341). Routledge. https://doi.org/10.4324/9780429398919
18.
LinX. (2023). Exploring the role of ChatGPT as a facilitator for motivating self-directed learning among adult learners. Adult Learning, 35(3), 156–166. https://doi.org/10.1177/10451595231184928
19.
LoC. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410. https://doi.org/10.3390/educsci13040410
20.
MeyerJ. G.UrbanowiczR. J.MartinP. C. N.O’ConnorK.LiR.PengP.-C.BrightT. J.TatonettiN.WonK. J.Gonzalez-HernandezG.MooreJ. H. (2023). ChatGPT and large language models in academia: Opportunities and challenges. BioData Mining, 16(1), 20. https://doi.org/10.1186/s13040-023-00339-9
21.
MohamedA. M. (2023). Exploring the potential of an AI-based Chatbot (ChatGPT) in enhancing English as a foreign language (EFL) teaching: Perceptions of EFL faculty members. Education and Information Technologies, 29(3), 3195–3217. https://doi.org/10.1007/s10639-023-11917-z
22.
PearceK.AlghowinemS.BreazealC. (2023). Build-a-bot: Teaching conversational ai using a transformer-based intent recognition and question answering architecture. Proceedings of the AAAI Conference on Artificial Intelligence, 37(13), 16025–16032. https://doi.org/10.1609/aaai.v37i13.26903
23.
RahmanM. M.WatanobeY. (2023). ChatGPT for education and research: Opportunities, threats, and strategies. Applied Sciences, 13(9), 5783. https://doi.org/10.3390/app13095783
24.
ReynoldsT. H.BonkC. J. (1996a). Computerized prompting partners and keystroke recording devices: Two macro driven writing tools. Educational Technology Research & Development, 44(3), 83–97. https://doi.org/10.1007/BF02300428
25.
ReynoldsT. H.BonkC. J. (1996b). Facilitating college writers' revisions within a generative-evaluative computerized prompting framework. Computers and Composition, 13(1), 93–108. https://doi.org/10.1016/S8755-4615(96)90038-9
26.
RothsteinD.SantanaL. (2011). Make just one change: Teach students to ask their own questions. Harvard Education Publishing Group.
27.
RuoppR.GalS.DraytonB.PfisterM. (Eds.), (1993). LabNet: Toward a community of practice. Lawrence Erlbaum Associates.
ÜnlüZ. K.Dökmeİ. (2020). The effect of technology-supported inquiry-based learning in science education: Action research. Journal of Education in Science Environment and Health, 6(2), 120–133. https://doi.org/10.21891/jeseh.632375
31.
von GlasersfeldE. (1995). A constructivist approach to teaching. In SteffeL. P.GaleJ. (Eds.), Constructivism in education (pp. 3–15). Lawrence Erlbaum Associates.
32.
YoungJ. C.ShishidoM. (2023). Investigating OpenAI’s ChatGPT potentials in generating chatbot’s dialogue for English as a foreign language learning. International Journal of Advanced Computer Science and Applications, 14(6). https://doi.org/10.14569/IJACSA.2023.0140607
33.
YuF. Y.ChangY. L.WuH. L. (2015). The effects of an online student question-generation strategy on elementary school student English learning. Research and Practice in Technology Enhanced Learning, 10(24), 1–16. https://doi.org/10.1186/s41039-015-0023-z
34.
YuH.GuoY. (2023). Generative artificial intelligence empowers educational reform: Current status, issues, and prospects. Frontiers in Education, 8, 1183162. https://doi.org/10.3389/feduc.2023.1183162
