This study developed and validated a comprehensive AI literacy scale for higher education students through a mixed-methods approach. The development process integrated literature review, expert interviews, and BERTopic modeling analysis. Following content validity assessment and pilot testing, the scale was validated with a sample of 400 students. Factor analyses supported a 17-item, four-dimensional structure comprising AI Fundamental Knowledge, AI Impact Assessment, AI Performance Evaluation, and AI Practical Application. The scale demonstrated adequate internal consistency and configural and metric invariance across academic disciplines. Group differences were observed according to academic major, AI course experience, and academic level. Structural equation modeling indicated that AI literacy is positively associated with academic self-efficacy, which is in turn related to multi-item measured creativity and students’ GPA. Mediation analysis indicated that academic self-efficacy mediated the association between AI literacy and creativity, while the indirect association between AI literacy and academic achievement via this pathway was offset by the negative link between creativity and GPA. However, the generalizability of these findings may be constrained by the specific cultural context and sample characteristics.
AbbasQHussainSRasoolS (2019) Digital literacy effect on the academic performance of students at higher education level in Pakistan. Global Social Sciences Review IV(I): 108–116.
2.
AmabileTM (1988) A model of creativity and innovation in organizations. In: StawBMCummingsLL (eds) Research in Organizational Behavior, vol. 10. JAI Press, pp.123–167.
3.
AmabileTM (1996) Creativity in Context: Update to the Social Psychology of Creativity. Westview Press.
4.
AnnapureddyRFornaroliAGatica-PerezD (2025) Generative AI literacy: Twelve defining competencies. Digital Government: Research and Practice6(1): 1–21.
5.
AsioJMR (2024) Artificial intelligence (AI) literacy and academic performance of tertiary level students: a preliminary analysis. Social Sciences, Humanities and Education Journal5(2): 309–321.
6.
BanduraA (1977) Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review84(2): 191–215.
7.
BanduraA (1997) Self-efficacy. Cambridge University Press.
8.
BawdenD (2001) Information and digital literacies: A review of concepts. Journal of Documentation57(2): 218–259.
9.
BuckinghamD (2015) Defining digital literacy - What do young people need to know about digital media?Nordic Journal of Digital Literacy10: 21–35.
10.
CarolusAKochMJStrakaS, et al. (2023) MAILS - meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies. Computers in Human Behavior: Artificial Humans1(2): 100014.
11.
ChanCKYTsiLHY (2024) Will generative AI replace teachers in higher education? A study of teacher and student perceptions. Studies In Educational Evaluation83: 101395.
12.
ChenKTallantACSeligI (2025) Exploring generative AI literacy in higher education: Student adoption, interaction, evaluation and ethical perceptions. Information and Learning Sciences126(1–2): 132–148.
13.
ChoiSE (2014) The effect of culture and arts based experience on individual creativity and performance. PhD Thesis, Ewha Womans University, South Korea.
14.
FloridiLCowlsJBeltramettiM, et al. (2018) AI4People-An ethical framework for a good AI society: Opportunities, risks, principles, and recommendations. Minds and Machines28(4): 689–707.
15.
FornellCLarckerDF (1981) Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research18(1): 39–50.
16.
GrootendorstM (2022) BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv: 2203. 05794.
17.
GuilfordJP (1950) Creativity. American Psychologist5(9): 444–454.
18.
GuilfordJP (1967) The Nature of Human Intelligence. McGraw-Hill.
19.
GuilfordJP (1968) Intelligence has three facets: There are numerous intellectual abilities, but they fall neatly into a rational system. Science160(3828): 615–620.
20.
HairJFRisherJJSarstedtM, et al. (2019) When to use and how to report the results of PLS-SEM. European Business Review31(1): 2–24.
21.
HobeikaEHallitRMalaebD, et al. (2024) Multinational validation of the Arabic version of the Artificial Intelligence Literacy Scale (AILS) in university students. Cogent Psychology11(1): 2395637.
22.
HornbergerMBewersdorffANerdelC (2023) What do university students know about Artificial Intelligence? Development and validation of an AI literacy test. Computers and Education: Artificial Intelligence5: 1–12.
23.
HornbergerMBewersdorffASchiffDS, et al. (2025) A multinational assessment of AI literacy among university students in Germany, the UK, and the US. Computers in Human Behavior: Artificial Humans4: 100132.
24.
HuLBentlerPM (1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling A Multidisciplinary Journal6(1): 1–55.
25.
JinYMartinez-MaldonadoRGaševićD, et al. (2025) GLAT: The generative AI literacy assessment test. Computers and Education: Artificial Intelligence9: 100436.
26.
KaracaOÇalışkanSADemirK (2021) Medical artificial intelligence readiness scale for medical students (MAIRS-MS) - Development, validity and reliability study. BMC Medical Education21(1): 112–119.
27.
KhosoFJAliNAslamN (2023) Use of Chat-GPT and AI tools by undergraduates: Students and teachers’ perspective. Spry Contemporary Educational Practices2(2): 1–24.
28.
KimAYParkIY (2001) Construction and validation of academic self-efficacy scale [In Korean]. Korean Journal of Educational Research39(1): 95–123.
29.
KochMJCarolusAWienrichC, et al. (2024) Meta AI literacy scale: Further validation and development of a short version. Heliyon10(21): e39686.
30.
LaupichlerMCAsterAHaverkampN, et al. (2023) Development of the “Scale for the assessment of non-experts’ AI literacy” – An exploratory factor analysis. Computers in Human Behavior Reports12: 100338.
31.
LintnerT (2024) A systematic review of AI literacy scales. npj Science of Learning9(1): 50.
32.
LiuXZhangLWeiX (2025) Generative artificial intelligence literacy: Scale development and its effect on job performance. Behavioral Sciences15(6): 811.
33.
LivingstoneS (2004) What is media literacy?Intermediair32(3): 18–20.
34.
LongDMagerkoB (2020) What is AI literacy? Competencies and design considerations. In: Proceedings of the 2020 CHI conference on human factors in computing systems, Virtual conference, pp.1–16. Hawaii: Association for Computing Machinery.
35.
LynnMR (1986) Determination and quantification of content validity. Nursing Research35(6): 382–385.
36.
MansoorHMHBawazirAAlsabriMA, et al. (2024) Artificial intelligence literacy among university students—A comparative transnational survey. Frontiers in Communication9: 1478476.
37.
MikalefPGuptaM (2021) Artificial intelligence capability: Conceptualization, measurement calibration, and empirical study on its impact on organizational creativity and firm performance. Information Management58(3): 103434.
38.
Montenegro-RuedaMFernández-CereroJFernández-BataneroJM, et al. (2023) Impact of the implementation of ChatGPT in education: A systematic review. Computers12(8): 1–13.
39.
NgDTKLeungJKLChuSKW, et al. (2021) Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence2: 1–11.
40.
O’DeaXNgDTKO’DeaM, et al. (2026) Factors affecting university students’ generative AI literacy: Evidence and evaluation in the UK and Hong Kong contexts. Policy Futures in Education24: 13–34. DOI: 10.1177/14782103241287401
41.
Stanford University Human-Centered Artificial Intelligence (2021) Artificial Intelligence Index Report 2021. Report, Stanford University Human-Centered Artificial Intelligence.
42.
StreetB (2003) What’s ‘new’ in new literacy studies? Critical approaches to literacy in theory and practice. Current Issues in Comparative Education5(2): 77–91.
43.
TagareDKarkiTYuW (2025) K-12 teachers’ ethical competencies for AI literacy: Insights from a systematic literature review. Computers & Education239: 105435.
44.
TierneyPFarmerSM (2002) Creative self-efficacy: Its potential antecedents and relationship to creative performance. Academy of Management Journal45(6): 1137–1148.
45.
TorranceEP (1965) Scientific views of creativity and factors affecting its growth. Daedalus94: 663–681.
46.
WalterY (2024) Embracing the future of Artificial Intelligence in the classroom: the relevance of AI literacy, prompt engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education21(15): 1–29.
47.
WangBRauPLPYuanT (2023a) Measuring user competence in using artificial intelligence: Validity and reliability of artificial intelligence literacy scale. Behaviour & Information Technology42(9): 1324–1337.
48.
WangJLiuZZhaoL, et al. (2023b) Review of large vision models and visual prompt engineering. Meta-Radiology1(3): 1–13.
49.
WangSSunZChenY (2023c) Effects of higher education institutes’ artificial intelligence capability on students’ self-efficacy, creativity and learning performance. Education and Information Technologies28(5): 4919–4939.
50.
YangYZhangYSunD, et al. (2025) Navigating the landscape of AI literacy education: Insights from a decade of research (2014–2024). Humanities and Social Sciences Communications12(1): 374.
51.
YuanCWTsaiHYSChenYT (2024) Charting competence: A holistic scale for measuring proficiency in artificial intelligence literacy. Journal of Computer Assisted Learning62(7): 1455–1484.
52.
ZhangSZhaoXZhouT, et al. (2024) Do you have AI dependency? The roles of academic self-efficacy, academic stress, and performance expectations on problematic AI usage behavior. International Journal of Educational Technology in Higher Education21(1): 34.
53.
ZimmermanBJ (2000) Self-efficacy: An essential motive to learn. Contemporary Educational Psychology25(1): 82–91.
