Sage Journals: Discover world-class research

Abstract

Generative AI (GenAI) has become popular with many university students since late 2022 and is incorporated in their daily life. To use GenAI tools effectively and equitably, and also to meet industry expectations, students should be supported to develop GenAI literacy skills during their time at university. Even though some universities have provided AI literacy training courses to students, the courses are not yet widely available to students across the higher education sector. Besides, there has been a shortage of GenAI literacy courses in particular. And more importantly, it is unclear what knowledge and skills that need to be included relating to GenAI literacy, from the student perspective. Building a strong understanding of student perceptions in this area is critical, because it will help enhance appropriateness and content relevance and promote student engagement. Adopting the four-dimensional AI literacy framework as the theoretical foundation, this paper aims to address the gaps. It explores the perceptions of university students on GenAI literacy in the UK and Hong Kong contexts. Survey data from 234 students were collected and analyzed using the descriptive analysis and Kruskal–Wallis test. The results show that factors such as country of studies and prior learning about AI greatly impacted GenAI literacy. However, factors such as age and educational level do not have a significant effect. Built upon the findings and the theoretical foundation, this paper proposes a new GenAI literacy framework. This framework is a revised version of the four-dimensional AI literacy framework and proposes the essential GenAI knowledge and skills for university students. The new framework also acknowledges the impact of the macro, meso, and micro factors on student GenAI literacy development.

Keywords

Generative AI literacy AI course AI in education higher education AI ethics

Introduction

Artificial Intelligence (AI) plays a significant role in industry 4.0 and has now been widely used in business. The data from Statista (2023) suggest that the global AI market size will increase from nearly 100 million in 2021 to nearly 2 trillion in 2030. The rapid growth of generative AI (GenAI) has further accelerated AI adoption in major industries. For example, globally over 43% of employees have used ChatGPT and other large language models, such as Google Gemini and Copilot to help them with their daily work. In the context of higher education, it is believed GenAI has a great potential to transform not only the way universities teach but also the way students learn. Recent research has recommended a range of approaches that can be used to facilitate student learning, such as self-paced adaptive quizzes (Dijkstra et al., 2022), peer assessment evaluation (Jia et al., 2021), conversational partner (El Shazly, 2021), and computer code explanation and debugging (MacNeil et al., 2022). Owing to their user-friendly interface, and the fact that users are not required to have specialized AI knowledge and coding skills, GenAI tools quickly become very popular with university students on their studies and daily life.

The fast surge in GenAI in work, studies, and daily life places demands on users’ GenAI literacy skills and competences and will inevitably have a major impact on the employability of university graduates (O’Dea, 2024). As with other disruptive technologies, GenAI has both the positive and negative effects on the society. For example, it shows its efficiency in brainstorming new ideas, creating document outlines, translations, and multimedia production. On the negative side, as previously identified in the literature, there are some major ethical concerns associated with GenAI (O’Dea and O’Dea, 2023). Some students have already used ChatGPT to generate parts of or an entire essay. There are also output issues. Many students are unaware of the importance and necessity of fact-checking. In addition, evaluating and selecting the most appropriate GenAI tools to use, and optimizing the prompts to achieve the desire outputs are among some of the main challenges students are facing.

Consequently, it is important and essential for universities to support students to develop GenAI literacy. During the pre-GenAI era, there has been research on AI literacy (Long and Magerko, 2020; Ng et al., 2021). Some have offered a variety of definitions for the term. Kandlhofer and colleagues (2016) consider AI literacy as technological knowledge and understanding underpinning AI applications and services. Long and Magerko (2020) define AI literacy as competencies that enable users to understand, apply, and evaluate AI tools at work and in everyday life. In addition to these more general definitions, some research emphasizes the specific competencies AI literacy should comprise. For example, Ng and colleagues (2021) propose four competences, namely, knowledge and understanding of AI; use and apply AI; evaluate and create AI; and AI ethics. The definition offered by Zhang and colleagues (2023) includes three main competences, such as AI concept (e.g., the basic understanding of AI); ethical and societal implications; and AI career future. Moreover, it has been recognized that AI literacy competencies and skills are relevant to all users and should not be limited to computer scientists and AI professionals (Laupichler et al., 2022).

However, it is worth noting that AI and GenAI literacy are different. The former emphasizes the technical aspects of AI literacy skills. Users are normally expected to comprehend interdisciplinary skillsets encompassing machine learning, deep learning, mathematics, statistics, and computer science, so that they were able to understand, deploy, and configure AI solutions meaningfully. For this reason, AI tools tend to be mainly limited to users who are Science, Technology, Engineering, and Mathematics (STEM) subject experts. GenAI literacy however focuses more on developing the awareness of the ethical and social implications and learning how to use GenAI critically and responsibly. This is because, as discussed above, GenAI tools are designed for users with “non-STEM” background. They are highly intuitive and can produce text, image, video, or audio-based responses without difficulty. For GenAI users, possessing any of the interdisciplinary skill sets mentioned above can be helpful, but is not essential. Compared with the existing literature on AI literacy, less attention has been paid to GenAI literacy.

Another gap in the literature that needs to be addressed is the insufficient attention to student perspective on GenAI literacy, as existing research primarily focuses on educator perspective. For example, Chiu et al. (2023) discussed the impact of GenAI on practices, policies, and research direction using ChatGPT and Midjourney. Lim et al. (2022) analyzed the theoretical lens to help provide implications for the future of education from the perspective of management educators. However, gaining student views and understanding how they use GenAI is equally important, as this will potentially enable universities to provide students with sufficient GenAI literacy training and resources and support them using GenAI ethically and equitably in their studies and future work.

Using the 4-dimensional AI literacy framework as the theoretical foundation, this paper aims to address the gaps identified by exploring the perceptions of university students on GenAI literacy in the UK and Hong Kong contexts on these four dimensions. The AI literacy framework was proposed by Ng and colleagues (2021) and contains four dimensions, namely knowledge and understanding of AI; use and application of AI; evaluate and create AI; and AI ethics. This framework is felt appropriate mainly because it emphasizes AI literacy and has been applied across different university contexts.

The United Kingdom and Hong Kong are chosen for the paper partially because they are keen to adopt AI in the workplace and to develop AI competences for university graduates. For example, universities in the United Kingdom drew up a set of guidelines to foster students and staff’s AI literates to support their learning/teaching and assessment methods (Department for Education, 2024). Universities in Hong Kong notably have already integrated AI literacy into their existing curriculum (Ng et al., 2023a; Kong et al., 2023). In addition, the United Kingdom and Hong Kong represent a different level of AI capacity internationally in the areas such as talent, infrastructure, research, and development. For instance, the overall ranking on the AI usage for the United Kingdom is the fourth and for Hong Kong is 32nd (Global AI index, 2023). Among the indexes, the commercial responses that focus on the level of startup activities, investment, and business initiatives based on AI are comparable for both regions. This aligned with another report by Stanford University. It shows that Hong Kong has demonstrated the greatest growth in AI adoption in industry across the world and the United Kingdom ranks fourth. The importance of AI literacy education has been recognized, and some universities are offering some AI literacy courses to students. In addition to what has already been identified in the literature, this paper aims to shed further light on how individual differences and perceptions among students may have influenced their GenAI literacy competences and development. Two research questions are proposed as follows:

• How do university students perceive their GenAI literacy level in the United Kingdom and Hong Kong?

• How do the demographic factors (i.e., country/region, prior AI training, gender, and subject discipline) affect university students’ GenAI literacy?

Literature review

Current research on AI literacy

AI literacy is not a brand new topic. It has however received renewed attention after the appearance of ChatGPT since November 2022. One popular trend in this field is the publication of exploratory or scoping review papers, such as those by Laupichler and colleagues (2022), Ng and colleagues (2021, 2023b), and Zawacki-Richter and colleagues (2019). These papers lay the groundwork by summarizing and evaluating the existing literature, identifying gaps, and providing recommendations for future research. These reviews indicate that research in AI literacy is still in its early stage. The field would benefit from more research that is empirically rigorous and practically relevant. Increasingly, there is a call for re-conceptualizing AI literacy, incorporating the recent development in GenAI (Koh and Doroudi, 2023).

In addition, as discussed above, some papers, including the work of Kandlhofer and colleagues (2016), Long and Magerko (2020), and Ng and colleagues (2021), focus on defining AI literacy or proposing skills that need to be encompassed in AI literacy. These definitions are often used as the underpinning reference for developing AI curriculum at either the undergraduate or postgraduate level (Chiu, 2021; Lin and Van Brummelen, 2021; Ng et al., 2023a; Southworth et al., 2023). The courses can be broadly categorized into two groups: those that mainly focus on the technical skills of AI and those that focus on a combination of technical and non-technical skills (Laupichler et al., 2022; Long and Magerko, 2020; Southworth et al., 2023; Ng et al., 2023b). The first category contains topics such as machine learning, deep learning, robotics, and programming concepts. The non-technical skills in the second category include ethical, problem-solving, and creative skills. Users are also expected to develop their awareness of social responsibilities of using GenAI (Kong et al., 2023). For example, University of California (Southworth et al., 2023) developed a multi-disciplinary campus-wide AI curriculum. The curriculum is designed for undergraduate students with diverse technical backgrounds and focuses on developing their technical and ethical skills. The AI courses experienced a high level of enrollment. A similar type of AI course was introduced to postgraduate students studying Radiology at City, University of London (van de Venter et al., 2023). Particular attention was given to the applications of AI in medical care and medical imaging. Students felt that the course offered a valuable opportunity for them to acquire specific AI skills that are relevant to healthcare. The University of Hong Kong dropped the ban on AI usage in August 2023 and introduced new policies to fully integrate GenAI into learning and teaching. Training and online courses were then provided to equip students and staff with the knowledge and tools needed to be creative and efficient in designing teaching and learning activities (HKU, 2023a). Nevertheless, it is worth noting that, despite the recent surge of interest and efforts, research on AI curriculum in the context of higher education is limited. The extent to which students achieved the intended learning outcomes also remains uninvestigated.

The four cognitive domains of AI literacy framework

The theoretical grounding of this paper is the four-dimensional AI literacy framework proposed by Ng and colleagues (2021). Based on the result of an exploratory review, the framework identifies four dimensions for fostering AI literacy, including knowledge and understanding of AI; use and application of AI; evaluate and create AI; and AI ethics.

Knowledge and understanding of AI refers to acquiring the fundamental knowledge of AI and also how to use AI tools. The fundamental knowledge means “a person’s understanding of the function and operation of currently available technology and applications on that technology, for example, an understanding of how to operate a tablet, download an app, and share a screenshot of something made in that app” (Yu and Golden, 2019). Use and apply AI describes individual understanding of how particular AI tools can be used in different scenarios and to assist users to complete tasks efficiently and creatively.

Evaluation and creation of AI is concerned with selecting the most appropriate AI tools, analyzing and evaluating AI outputs. Particular attention is paid to AI outcome’s evaluation. This is because some of the outputs produced by GenAI can be simply made up or incorrect, even though they may appear to be well formatted and authentic. This phenomenon is called hallucination or deepfake (concerning mainly with videos) and has gained wider recognition (Birhane et al., 2023). Compared with hallucinations, deepfakes are more malicious and harder to detect.

AI ethics can be viewed as an umbrella term and is concerned with areas such as fairness, accountability, and transparency. Fairness refers to the explanation regarding the approach to the processing and the outcomes of the processing. Accountability describes the roles and responsibility relating to the AI systems or tools and the outcomes produced. Transparency refers to information accessibility, concerning both the production and deployment of particular AI tools (Memarian and Doleck, 2023). Several prominent issues that have been raised relating to AI ethics include content bias and ownership. For instance, the GenAI outputs are likely to be biased if the training data sets are non-representative. Some known biases include gender and cultural bias (Baker and Hawn, 2021). To date, there does not seem to be a universal rule globally on intellectual property rights (IPR) in the era of GenAI, and some important questions raised and left unanswered. For instance, who owns the copyright of the generative AI productions? And are these productions protected under IPR laws?

This framework is felt appropriate because this study aims to gain an understanding of students’ perceived views on GenAI literacy in higher education and also how they use GenAI in their studies and daily life. Even though the framework was developed at the pre-GenAI era, the four dimensions cover the technical and application skills university students need to develop with regard to AI literacy, and hence are applicable to GenAI. In addition, this 4-dimensional framework has been adopted regularly as the foundation for AI curriculum design (Bellas et al., 2023; Celik, 2023; Southworth et al., 2023) and assessment methods (Laupichler et al., 2022; Moorhouse et al., 2023).

Apart from this 4-dimensional framework, there are other AI literacy frameworks in the literature. For example, the one proposed by Zhang and colleagues (2023) comprises three domains, namely, technical concepts and processes, ethical and societal implications, and career futures in the AI era. Even though both frameworks share similarities, the latter aims at K-12 education and focuses more on developing foundational knowledge and skills. Built upon a bibliometric analysis paper, Cetindamar and colleagues (2022) also proposed an AI framework containing four dimensions, such as technology-related, work-related, human–machine-related, and learning-related capabilities. This framework is however primarily concerned with commercial contexts and is designed to upskill employees.

Methods

Participants and demographic details

Convenience sampling is used in this study so that we can reach a large population of students. Participants were university students ranging from undergraduates to postgraduates studying in the United Kingdom or Hong Kong. In the United Kingdom, an invitation email was circulated among the authors’ academic network, who then helped distribute the email to their students. Similarly, in Hong Kong, the invitation email was sent out to students either directly or through instant messaging software from June to October 2023. The sample consisted of 234 participants.

The average year of study for participants was 1.64 (SD = 0.84) in the United Kingdom and Hong Kong. Most participants were from the United Kingdom (84%) and Hong Kong (13%). There were 127 males (56%) and 92 females (41%). 92 students (41%) studied STEM-related disciplines (e.g., engineering, computer science, and science), and around 29 participants (13%) reported that they have taken a generative AI course. Table 1 demonstrates the demographic information of the participants.

Table 1.

Demographic characteristics of the participants.

Characteristics	N	%
Country/region
United Kingdom	191	84
Hong Kong	30	13
Others	7	3
Gender
Male	127	56
Female	92	41
Others (e.g., non-binary, third gender)	5	2
Prefer not to say	3	1
Educational level
Year 1 undergraduate	50	22
Year 2 undergraduate	48	21
Year 3 undergraduate	41	18
Year 4 undergraduate	11	5
Postgraduate	77	34
Age
17–24	134	59
25–34	50	22
35–44	27	12
45–54	11	5
55 or above	5	2
Subject discipline
Business and management	62	27
Computing, technology, and engineering	60	26
Education	27	12
Sciences	16	7
Social sciences	13	6
Design, creative, and performing arts	7	3
Languages and linguistics	7	3
Law	7	3
Medicine and healthcare	7	3
Others	21	9
Taken a course on using GenAI
No	194	85
Yes	29	13
Prefer not to say	4	2

Study design and procedure

Quantitative data were collected via an online survey platform, Qualtrics. The survey included five sections and 24 items. In addition to the first section, six demographic characteristics of the participants (i.e., country/region of origin, gender, education, age, subject discipline and if taken a course in AI) were asked. These were examined against four dimensions corresponding to the four competencies of the Ng et al. (2021)’s framework, namely, “know and understand AI,” “use and apply AI,” “evaluate and create AI,” and “AI ethics, as explained in the section above (the four cognitive domains of AI literacy framework).

The AI literacy questionnaire has been in prior research (Wu et al., 2023). This research adopted the existing questionnaire, rather than developing a new one. Only 14 questions were chosen to balance response rate and research purpose, which may limit exploratory and confirmatory factor analysis (Hageman and Cheon, 2023); the limitation was addressed in the section later (limitation section). Instead of developing an original scale, the focus of this study was on assessing the reliability and validity of the established measure. The adapted version was validated by expert review for relevance, writing, and clarity and had a Cronbach’s alpha of 0.82, indicating good internal consistency.

Previous research conducted in Hong Kong (Memarian and Doleck, 2023; Ng et al., 2024) similarly reported favorable validity and reliability for AI literacy, further supporting their applicability in diverse learning environments. “Know and understand AI” was measured using five items (Q.9-13). The answers were rated on a five-point scale, ranging from either “strongly agree” to “strongly disagree,” or “very comfortable” to “very scared.” “Use and apply AI” was assessed using four items (Q.14-17). The items explored student experience in using GenAI tools in their studies and daily life (e.g., how often they used the tool, and what they used the tool for). Evaluate and create AI was measured using three items (Q.18-20). AI ethics was measured using two items (Q.21-22). These questions were referenced from Memarian and Doleck (2023) and prior author’s validated questionnaire (Ng et al., 2023). Apart from assessing student perceived AI literacy, some follow-up questions were asked to provide further explanations for the quantitative findings (e.g., understanding how students decide which software to use, reasons for indicating some of their responses).

This study only selected some of the questions, as we were conscious of the dilemma that asking too many questions would lower the response rate, while asking too few questions may not elicit the required information. The 14-question survey was chosen to balance response rate and data requirements. However, this limited selection of questions may hinder exploratory and confirmatory factor analysis (Hageman and Cheon, 2023). To address this, prior studies adopted the Kruskal–Wallis test (p-value) to compare characteristics or specific conditions of a group of people and examine significant differences using normative data among different characteristics (e.g., country/region and gender) (Michalos, 2014; Mulders et al., 2018).

The study was approved by the ethics committee, and the survey was conducted in the United Kingdom and Hong Kong simultaneously between 1st and 30th of June 2023. Before taking the survey, participants were provided with the research background and the purpose of the study. They were also asked for their informed consent to participate in the study.

Data analysis

A mix of descriptive and inferential statistics was used to analyze the results. Descriptive statistical measures such as the frequency, mean, and standard deviation are included. They provide an overview of the evidence-based observations, allowing for a further understanding of the data. In addition, inferential statistics involve drawing conclusions or making inferences based on the observed data. These inferences rely on the information provided by descriptive statistics, which help to summarize and characterize the data set (Braun and Clarke, 2021). The analysis was based on the demographic factors collected via the survey. These demographic factors were used to test the relationships between the respondents’ characteristics and the pattern of answers to the four cognitive domains. Each of the survey questions was analyzed for each demographic factor through the descriptive analysis and Kruskal–Wallis test.

To answer RQ1, we conducted a descriptive analysis to assess the perceptions of university students on AI literacy in the United Kingdom and Hong Kong. To answer RQ2, the Kruskal–Wallis test was used to determine whether the variables of country/region, gender, age, education, and subject and whether they had taken a course in AI had an influence on the attitude to and use of AI to facilitate student learning. A nonparametric test was used in this study since the data was not normally distributed (p < 0.05 in the Kolmogorov–Smirnov test). Therefore, the Kruskal–Wallis test (a nonparametric equivalent of the one-way ANCOVA test) was used when the assumptions had been violated (Çakıroglu et al., 2017). As we considered these six factors as potentially having an influence in the analysis, we applied a Bonferroni correction to adjust the alpha level for the Kruskal–Wallis tests. As a result, the tests were considered to only be statistically significant if the p-value was below 0.00833, as suggested by Wu et al. (2020).

Results

Students’ perceptions (RQ1)

The study investigated the influence of various demographic factors on AI literacy, including age, education, country of origin, gender, and subject discipline and whether participants had taken an AI course. The GenAI literacy survey provided insights into students’ perceptions. The results indicated that students had a positive response to the statement “know and understand AI” (M = 3.59, SD = 1.16). This response reflects their belief in their ability to grasp the fundamental functions of GenAI and utilize GenAI applications. Regarding the application of AI, students expressed confidence in applying GenAI knowledge, concepts, and applications to solve problems in various scenarios, such as writing and generating multimedia (M = 3.24, SD = 1.17). However, students were less confident in evaluating AI applications and comprehending underlying AI concepts to create artifacts (M = 2.45, SD = 1.17).

To further understand the perceptual differences between the demographic groups (i.e., the UK and Hong Kong contexts), we counted the number of students who rated the items greater than three out of five. The proportion of students who had taken a course by country/region shows that 38% of Hong Kong students have taken a course compared to 9% of UK students. However, among the students who had taken a course, 37% were from Hong Kong and 57% were from the United Kingdom. The proportion by gender shows that 21% of male students had taken a course in AI compared to 4% of female students. Also, 76% of the respondents from Hong Kong were male compared to 54% of those from the United Kingdom.

With regard to the Kruskal–Wallis test, we examined whether the variables of country/region, gender, educational level, and subject and whether they had taken a course in AI had an influence on the attitude to and use of AI in student’s study. The analysis shows a significant response for the questions where respondents were questioned on which software they had used, and why and how they decided upon which to use (Table 2).

Table 2.

Kruskal–Wallis analysis on Dimension 2—use and application.

Characteristic	Themes and sub-themes	p-value
Country/region	Software used
	ChatGPT3.5	0.002174*
	POE	1.496212
	Notion AI	0.012638
	What the sofware was used for
	Essays/written reports—for gathering some ideas	0.000000*
	Essays/written reports—for producing part of the entire essay	0.000000*
	Just playing around to see what it does	0.001064*
	Don’t use it at all	0.008018*
	How to decide which software to use
	Access provided by work/university	0.001367*
Gender	Software used
	ChatGPT3.5	0.005100*
	What the software was used for
	Generating computer code	0.000936*
	Just playing around to see what it does	0.001587*
	How to decide which software to use
	Because it is free	0.006805*
	Because of the functionality of the tool	0.005139*
Subject discipline	What the software was used for
	Generating computer code	0.000059*
	Just playing around to see what it does	0.000036*
	Don’t use it at all	0.000649*
Course taken	Software used
	ChatGPT3	0.000844*
	Microsoft Bing.ai	0.000549*
	POE	0.000000*
	Notion AI	0.001565*
	Others	0.002937*
	What the software was used for
	Essays/written reports—for gathering some ideas	0.003000*
	Essays/written reports—for producing part of the entire essay	0.000015*
	Preparing for interview questions	0.000706*
	How to decide which software to use
	Because of the functionality of the tool	0.006082*

Remarks: *p < 0.00833.

Some interesting findings can be found. When it comes to the theme “software used,” it is found that students from Hong Kong and the United Kingdom had different uses for ChatGPT (p-value = 0.002174). After identifying the significance level, we further provide inferences and reasons to support these findings. A possible reason is that this tool is restricted in Hong Kong, and students in Hong Kong need to rely on other GenAI tools and platforms provided by work/university. Further, students of different educational levels may have different habits of using GenAI in areas such as essay and report writing and computer code generation (p-value = 0.000000). Other aspects follow this data analysis method.

Factors affecting university students’ GenAI literacy in the UK and Hong Kong contexts (RQ2)

A chi-squared analysis was used to test for association between each demographic factor and students’ responses to the questions within each dimension. First, the impact of the country/region of origin on AI literacy was found to be ambiguous. This could be due to various factors such as variations in curriculum and technological readiness between the United Kingdom and Hong Kong that may influence the exposure and familiarity with AI technologies. Also, convenience sampling in authors’ academic networks could be a problem, since students were not chosen at random from a larger population. This might result in sampling bias. Whether participants had taken a course in AI, gender, and subject discipline were identified as factors that significantly influenced AI literacy (p < 0.00). In all four cases where the chi-squared result was significant, it shows a large effect size. Age and educational level were not significant in all four dimensions and therefore were not examined in this section.

Country/region of origin

Country/region of study exhibits a relation to all four dimensions, such as knowledge and understanding, use and apply, evaluate and create, and ethics. The results show significant differences between Hong Kong and the UK students in the following areas: basic understanding of AI, confidence in using AI frequency in using AI, and ethics. Put it succinctly, Hong Kong students appear to be more advanced in all four areas of the AI literacy framework. They are more knowledgeable about AI tools. They use them more and have a greater understanding of how to use them. They also intend to employ them more and evaluate the output more and are more aware of AI ethics when using these tools.

Students from Hong Kong universities display a more comprehensive understanding of AI literacy than their counterparts in the United Kingdom (75% vs 67%) when they were asked to rate their view relating to the question “I have a basic idea of what AI is” using a Likert scale rating from strongly agree to strongly disagree. Their knowledge and understanding in this area seemed to have helped boost their confidence in using AI to solve problems (89% vs 57%) and also the frequency in using AI in their studies and daily life (88% vs 27%). Hong Kong students also appear to be more aware of the limitations and capabilities of AI applications relevant to specific tasks compared to UK students (92% vs 66%). The differences in AI knowledge and understanding and experience in using AI between these two student groups had a direct impact on their intention of using AI tools in future. For example, 81% of Hong Kong students said that they planned to use the technology more in the future, compared to 32% of UK students.

With regard to the ethics dimension, students in both countries also show significant differences. Hong Kong students seem to have also developed more awareness in AI ethics and understood the importance and necessity of checking the outputs produced by AI. The Kruskal–Wallis analysis of the ethics dimension shows a statistically significant relationship between Hong Kong students and the statements “I incorporate ethical considerations when deciding whether to use data provided by an AI” (p = 0.00641) and “I critically think about the ethical concerns when deciding whether to use the recommendations or responses created by AI” (p = 0.00147). In addition, 73% of Hong Kong students reported that they changed half or more of the output from generative AI when they used AI tools for university work, compared to 31%, while 8% of Hong Kong students compared to 44% of UK students reported that they used the output unaltered.

Based on the results above, we anticipated that Hong Kong students in the study would have received more training on AI literacy than the UK students, because both groups show a significant difference in their AI literacy. However, the data suggest a very different picture. When they were asked to answer the question “have you taken a course on using generative AI such as ChatGPT,” 37% of Hong Kong students (N = 11) said “yes,” compared with 57% of UK students (N = 17).

Whether taken a course about AI

Students were asked whether they took any training on using GenAI tools such as ChatGPT. Only 12% of respondents (N = 28) reported that they received some training before, and most of them participated in an external self-taught course, such as boot camps or teacher training courses. Only 5.7% (N = 13) claimed that they attended an optional and compulsory course offered by their university as part of a degree program internally. Nevertheless, students who had taken a course in AI reported a greater knowledge of how to use AI tools, a greater use of AI tools in connection to their studies. They not only intended to use the technologies more but also became more aware of the ethical implications of the application of AI to their studies.

Whether students had taken a course in AI emerged as a significantly influential factor. Prior learning in AI plays a key role in shaping students’ positive perceptions, attitude, and behaviors toward AI. This experience can potentially help the respondents to achieve a higher level of cognitive understanding and think critically about their use of AI. For example, in terms of students’ feelings toward AI, knowing how to use AI to solve problems, and how often they use AI, significant results were observed (p < 0.00). This indicates that students who have taken a course in AI tend to have a more positive attitude toward AI. They also possess a better understanding of how to utilize AI for problem-solving and engage with AI technologies more frequently compared to those who have not taken such a course.

The variable also demonstrates significant results in how students critically verify the outputs generated by AI. For instance, among those taking training on AI already, 64% updated approximately half or more of the AI outputs for their university work compared to 33% of those who had not taken a course. They also learnt to refine prompts to enhance the generated outcomes. In addition, the training these students received seemed to have empowered them with the ability to select appropriate AI tools to solve problems across different scenarios, such as drafting proposals, emails, and image editing. These abilities play a crucial role in enabling students to utilize AI tools more effectively, leading to the acquisition of accurate and reliable solutions.

Gender

Gender is also shown to play a significant role in the use of AI by students. Specifically, it demonstrates notable outcomes in the following areas: knowing how to use AI to solve problems, frequency of AI usage, software utilization, and usage intention.

Knowing how to use AI to solve problems

For example, the results indicate that individuals of different genders exhibit varying levels of proficiency in utilizing AI tools and techniques for problem-solving. 72% of males compared to 50% of females agreed that they knew how to use AI tools to solve problems, while 19% of males compared to 40% of females disagreed. This suggests that there may be differences in the acquisition of AI skills and knowledge based on gender, leading to diverse capabilities in leveraging AI for addressing challenges.

Frequency of AI usage

The frequency of AI usage also demonstrates significant differences based on gender. It appears that individuals of different genders vary in their engagement and utilization of AI technologies. When they were asked: “how often you used AI tools over the last semester”? 88.9% of male students (N = 16) reported daily usage compared with 11.1% of female students (N = 2). This suggests that male students may be inclined to use AI more frequently, integrating it into their daily routines, work tasks, or personal endeavors, while female students may have less exposure or interaction with AI tools.

Software utilization and usage intention

In addition, the judgment of the respondents about AI outputs was influenced by gender. 77.3% (N = 17) of male students compared with 22.7% (N = 5) of female students changed about half of the outputs created by AI. This indicates that gender may influence individuals’ awareness, options, and expectations about the results produced by AI. Different gender groups may utilize distinct software tools or platforms when working with AI. Finally, this study shows significant differences between male and female students with regard to their future usage of AI tools. For instance, 52% of male students intend to use AI more in the future compared to 31% of females while 5% of males and 7% of females intend to use them less.

Subject discipline

The subject discipline variable yields significant results in AI literacy. The majority of respondents from computing and technology disciplines (42.9%) reported that they have a basic idea of what AI is, indicating a higher level of familiarity with the subject matter compared to students from other disciplines. Students from business and education disciplines expressed a higher level of comfort with AI compared to students from other subject disciplines. This suggests that students in these disciplines may have more positive confidence toward AI usage significantly (p < 0.00), potentially influenced by their educational level and exposure to AI-related concepts.

When it comes to the four cognitive dimensions, subject discipline demonstrates significant results in terms of students’ knowledge of how to use AI to solve problems and their frequency of AI usage (p < 0.00). Students from computing and technology were more likely to possess the necessary skills to effectively utilize AI tools for problem-solving and reported using AI more frequently compared to students from other disciplines. Further, understanding the limitations of AI and intentions for future AI usage were also influenced by subject discipline, with significant results observed. This implies that students’ awareness of the limitations of AI and their future plans for incorporating AI into their professional or personal lives may vary depending on their subject discipline. However, there were no significant results in relation to students’ awareness and application of ethics in AI, suggesting that subject discipline alone may not strongly influence students’ ethical considerations when it comes to AI.

Regarding software usage, the only significant result was found for ChatGPT, indicating that students across different subject disciplines demonstrated a significant preference for using this specific AI tool. Furthermore, the only significant application identified was for programming, suggesting that students, regardless of their subject discipline, perceive AI as a valuable tool in programming tasks. While the training for computer and technology students aims to explore AI, future efforts are important to further enhance other skills (e.g., collaboration and communication) to effectively utilize AI in their future workplaces.

Overall, it appears that female students are less comfortable with the technology. They also consider themselves less knowledgeable and plan to use AI less than male students in the future. The variable of gender reveals significant results in several key aspects related to AI usage.

Discussion

Adopting the four-dimensional AI framework as the theoretical foundation, this paper aims to answer two questions: How do university students perceive their GenAI literacy level in the United Kingdom and Hong Kong? and how do the demographic factors (i.e., country/region, prior AI training, gender, and subject discipline) affect university students’ GenAI literacy? Student perceptions on the four dimensions were explored using a survey. The findings show that students appear to have developed some basic understandings of GenAI functions and to be able to utilize some commonly used GenAI applications, such as Midjourney, ChatGPT, and Google Gemini. However, they were less confident in evaluating AI applications and comprehending underlying AI concepts to create artifacts. In addition, factors such as country of studies and prior learning about AI appeared to have a major impact on their GenAI literacy, compared with age and educational level. Furthermore, students studied at the Hong Kong universities seemed to have developed better awareness of AI ethics than their counterparts in the United Kingdom. The result indicates that Hong Kong students in the study would have received more training on AI literacy than the UK students.

Generative AI literacy across learner groups

To discuss RQ1, learner differences can be influenced by factors such as prior experience, gender, and subject discipline, according to prior research (e.g., Fisher et al., 2020; Lai and Hong, 2015; Paray and Kumar, 2020). Regarding prior AI course experience, examining its influence can shed light on the role of education and familiarity with AI concepts in shaping perceptions and understanding. Students with prior AI course experience might demonstrate different levels of knowledge and confidence compared to those without such experience. This can offer insights into the effectiveness of AI education and its potential implications for AI literacy and adoption. Second, gender can be another influential factor in AI perceptions. Research has shown that gender biases can exist in various domains, including technology. Exploring the impact of gender on AI perceptions can help identify any potential disparities or biases that may affect how individuals engage with AI technologies. Additionally, understanding gender differences can aid in designing inclusive AI systems that cater to diverse user needs and preferences. Subject discipline, or the academic field or area of expertise, is another important variable to consider. Different subject disciplines may have varying levels of exposure to AI and different perspectives on its applications. Analyzing the influence of subject discipline on AI perceptions can provide insights into how different disciplines perceive and utilize AI technology. It can also inform interdisciplinary collaborations and the integration of AI into various fields.

The need of generative AI literacy

This study, based on the above findings, contributes to the expanding body of evidence and highlights the urgent necessity for universities to offer GenAI literacy education to students. It extends the concept of AI literacy, which comprises four dimensions, and proposes a potential assessment to evaluate students’ GenAI literacy, although this assessment has not been validated again in this study.

Most of the current research examines students’ learning experiences with a focus on understanding AI and the implementation of AI education, according to a systematic review (Casal-Otero et al., 2023). This need arises primarily from industry expectations and the limited availability of AI courses at the university level. On top of AI literacy, the release of GenAI tools such as ChatGPT in November 2022 has accelerated the adoption of AI further. A recent report (IBM, 2022) shows that globally as of 2022, over 35% of businesses had already used AI in their daily operation. An additional 42% of businesses were currently exploring the potential opportunities for AI adoption. These figures are showing a continual upward trend. The top three business processes identified as being fueled by AI technologies include robotic process automation, natural language text understanding, and virtual agents/conversational interfaces (McKinsey, 2022). Consequently, employees with AI literacy skills are in great demand. As such, there is a need to rethink and revise the current ideas of AI literacy skills in the GenAI age.

When implementing GenAI in the classroom, inclusiveness becomes a significant concern (Lee et al., 2021; Xia et al., 2022). Students come from diverse backgrounds, including variations in gender, prior AI experiences, and subject discipline. It is acknowledged that students with prior knowledge (Hornberger et al., 2023) and a background in computer science are likely to have a higher level of GenAI literacy (Kong et al., 2021). As such, another contribution of this study is to investigate how individual differences contribute to variations in GenAI literacy.

This study highlights self-perceived differences in GenAI literacy based on country/region, gender, prior AI experience, and subject disciplines. These findings align with a recent systematic review (O'Dea et al., unpublished study) that examined 369 articles, revealing insufficient availability of AI courses for students studying non-computer science or data science disciplines. Despite the recognition of the importance of AI curriculum in better preparing university students for future careers, some places still lack adequate supply of AI courses. According to a report by UNESCO, only 16 out of 27 member countries have begun incorporating AI into computer curricula (UNESCO, 2023). Additionally, certain universities, such as the University of Hong Kong, have temporarily prohibited students from using ChatGPT or any other AI-based tool for learning, assessment, and coursework (HKU, 2023b). This cautious approach by educators reflects concerns surrounding the use of GenAI in the classroom (Chiu, 2023). The observed GenAI differences between the United Kingdom and Hong Kong serve as a reminder to educators and universities that some institutions or countries may have already started implementing GenAI, and simply prohibiting its use is not a long-term solution.

Furthermore, educators should address the issue of gender diversity, learners’ subject backgrounds, and prior experiences. The lack of gender diversity can have significant implications for the individuals for whom AI-based systems are developed (Casal-Otero et al., 2023). The literature emphasizes the importance of adopting a gender perspective in designing AI-related activities, with specific initiatives aimed at engaging girls (Solyst et al., 2023; Vachovsky et al., 2016). Another research conducted by Zhang and colleagues (2023) on pre-service teachers indicates that female teachers are more likely to experience anxiety about using AI-based tools than male teachers. A similar result was reported by Nouraldeen (2023) who conducted a survey to explore technology readiness among university students. The survey results reveal that male students are more ready than female students to adopt AI technology. More effort is needed to support female learners to foster their GenAI literacy and build more confidence in using the technology.

Also, it is crucial for universities to develop relevant training programs and guidelines on GenAI topics that span across subject disciplines, including integration into non-computer science subjects (Lin and Brummelen, 2021; Kong et al., 2021). They should take into account the needs and prior backgrounds of students to provide appropriate training (Chiu et al., 2023; Ng et al., 2023). It is important to note that students from non-computer science backgrounds, such as education and business, may initially have a lower perception of GenAI literacy. However, AI literacy is actively being taught among pre-service teachers (Lim, 2023) and business students (Cardon et al., 2023). Education students, for instance, need to acquire sufficient AI competencies and receive training in AI-related concepts and skills to become proficient in teaching AI and using AI for instruction in their future workplaces. While business students may not have a technical background, there are many platforms designed for business purposes that facilitate decision-making, prediction generation, and work efficiency enhancement for business leaders and managers. On the other hand, computer and technology students usually possess a strong technical foundation and perceive themselves as more capable of acquiring and retrieving AI knowledge due to their existing technical knowledge base (Ng et al., 2023).

Furthermore, this study has found that current AI training in higher education places a stronger emphasis on technical knowledge, such as machine learning, programming, and robotics, while giving less attention to AI ethics and the evaluation of AI applications (Farina et al., 2024; Schlagwein and Willcocks, 2023; Zohny et al., 2023). Therefore, the revised framework puts forward the following recommendations for AI courses. When designing an AI course, all three dimensions (knowledge, application, and evaluation) should be addressed, and the curriculum should strive for a balanced approach across these dimensions. When developing new AI courses, opportunities should be provided for students to translate their prior knowledge into practical applications. For instance, students should be able to select the most suitable AI tools to assist them in brainstorming new ideas, testing their knowledge and understanding, and supporting their academic research (Cooper, 2023; Kohnke et al., 2023).

At last, the data was collected in the post-pandemic period. The pandemic has catalyzed a significant shift toward online and blended teaching and learning approaches, where educators are increasingly leveraging emerging technologies to enhance their students’ learning outcomes (Ng et al., 2023). Notably, GenAI technology has gained widespread popularity in online learning environments during this period, as it offers various tools and functionalities to assist students in their learning.

GenAI literacy framework

The AI literacy framework developed by Ng and colleagues (2021) includes four dimensions, namely, knowledge and understanding of AI, use and application of AI, evaluation and creation of AI, and AI ethics. Even though it is not stated explicitly, it appears that these dimensions are presented in a sequential order, with the final dimension being AI ethics. Developed upon the findings of this study, the revised framework has been renamed to be GenAI literacy framework and includes also four dimensions (see Figure 1). However, three have been renamed to knowledge, application, and evaluation, with the fourth dimension, AI ethics, being an integral part of each dimension. Further details are provided below. The new framework in addition acknowledges the impact of the external factors at the macro, meso, and micro levels.

Figure 1.

GenAI literacy framework with socio-ecological domains.

First, AI ethics should carry more weight and become an integral part of each individual dimension. Our study shows that university students in both country contexts, regardless of their gender, subject disciplines, and with or without previous training, did not seem to fully understand the ethics of AI, in particular, with regard to the potential ethical concerns with AI adoption in their studies and in the workplace. The finding is reflected in existing research. Even though some institutions have started providing additional AI courses or modules to undergraduate and/or postgraduate students, AI ethics are either missing from the topics covered or not considered as a central theme (Moorhouse et al., 2023). For example, only three universities out of 50 discuss the ethics and limitations of GenAI (Moorhouse et al., 2023).

In this domain, individuals should consider aspects such as data ownership, privacy, and security (Sanderson et al., 2023). They should be mindful of the ethical concerns surrounding the use of recommendations or responses created by GenAI and understand the ethical implications of GenAI applications. It is important for individuals to check whether the content generated by GenAI is appropriate and ethical, ensure inclusivity and diversity in the data used to train GenAI, avoid incorporating others’ work without proper attribution, acknowledge that prompt information may be stored and impact future responses, make accurate references and citations in GenAI-driven responses, and critically consider biases that may arise from the responses provided (Fui-Hoon Nah et al., 2023; Schlagwein and Willcocks, 2023; Zohny et al., 2023). Finally, individuals should clearly indicate which parts of their work were completed with the assistance of AI tools.

Second, for the ease of recollection, we rename three dimensions knowledge and understand AI, use and apply AI, and evaluate and create AI to knowledge, application, and evaluation. In the domain of knowledge, individuals should acquire a comprehensive understanding of GenAI literacy. This includes grasping the basics of how GenAI utilizes training data sets to create language models and being familiar with concepts such as transformers, large language models, and GenAI (Holmes and Miao, 2023). In the domain of application, individuals should be able to write code to solve problems, work with large data sets, and perform basic data analytics tasks. They should also understand the limitations and capabilities of AI applications in specific tasks. Moreover, individuals can employ GenAI tools in activities such as gathering ideas for essays and written reports, generating computer codes, answering exam questions, preparing CVs and covering letters, engaging in email communication, and collaborating with others (Barrett and Pack, 2023; Chiu, 2023; Escalante et al., 2023). Furthermore, in the domain of evaluation, individuals need to develop critical thinking skills. They should be able to make informed decisions about using data provided by GenAI, fact-check the content produced by GenAI, verify answers provided by AI tools, and harness GenAI’s potential to assist in creative tasks by generating new and innovative ideas (Chan, 2023; Mishra et al., 2023). Individuals should modify and update the responses generated by AI tools and choose the most appropriate tools to solve a given task.

Among these dimensions, knowledge is the prerequisite dimension of the other two and refers to the fundamental technical knowledge and basic understanding of AI ethics. Application is concerned with practical application of AI ethics and tools in different study and work scenarios. Evaluation examines whether the chosen AI tools are fit for purpose and can be sustained over time. Particular attention is paid to the ethics of using these tools in the long term. This new framework represents an iterative process for GenAI literacy development. For example, individuals are able to apply and evaluate AI tools in an ethical manner once they develop knowledge and understanding in these areas. On the other hand, their practical experience will motivate and support them to further develop their knowledge and understanding in AI.

And finally, we acknowledge the impact of socio-ecological factors on GenAI literacy and hence add the macro-, meso-, and micro-level factors to the framework (Berkovich, 2014). Macro-level factors are concerned with AI readiness and the associated plan at the country and government level. The guidance and policy offered by the government in supporting AI adoption significantly influences AI literacy development in industry, including education (Chan, 2023). As the data indicated in this study, students from Hong Kong seemed to be more advanced than their counterparts from the United Kingdom with regard to AI knowledge and application. This is likely because the Hong Kong government has a vision to take a lead on AI and subsequently has encouraged children to develop awareness and receive training on AI literacy from a much younger age (Su andYang, 2023). Meso-level factors are mainly concerned with the AI literacy training and support offered at the institutional level. Our data show that students who attended a university-led AI training course appeared to be more confident in using AI tools in their daily life and were keen to either explore more tools or use them more in the future. Micro factors refer to personal level factors, such as individual characteristics, and demographics, including gender, age, and subject disciplines. This study suggests significant differences in some of the areas mentioned.

As shown in the framework, the three hierarchical factors are proposed to incorporate the existing AI literacy framework to promote an inclusive-specific learning environment in higher education from micro to macro levels. The AI readiness policies implemented at the country level have a direct influence on the AI policies and training courses offered at the institutional level. These policies set the tone and framework for AI education and training within universities. The individual differences between students could impact how they develop and foster GenAI literacy. These differences may affect students’ abilities to comprehend AI concepts and apply AI knowledge in various contexts. Recognizing and addressing these individual differences is crucial for promoting inclusive and effective GenAI education (Kong et al., 2023; Xia et al., 2022).

Conclusion

In conclusion, this study addresses the research gap concerning the academic and professional benefits of GenAI tools for university students in the United Kingdom and Hong Kong. By exploring students’ perspectives using Ng’s four-dimensional framework, the study found that factors such as country of study, prior AI course experience, gender, and subject discipline significantly influence AI literacy. Based on these findings, we propose that GenAI literacy should encompass the four dimensions of knowledge, application, evaluation, and AI ethics, building upon the concept of AI literacy. Among these four factors, students demonstrate lower awareness of the importance of AI ethics. To cultivate AI readiness among young learners, educators should engage them more in evaluating the outputs of different GenAI applications and raise their awareness of AI ethics when utilizing these tools. Furthermore, the study incorporates macro-, meso-, and micro-level factors into the existing framework and highlights how individual differences contribute to variations in GenAI literacy. The study emphasizes the necessity of integrating GenAI technologies into higher education curricula and underscores the importance of focusing on AI ethics when engaging with AI tools.

Several limitations are addressed in this research that warrant further investigation in future studies. First, it is important to note that the sample size of the Hong Kong participants in this study was small, thus requiring future research to confirm the observed effects; particularly, the individual institutional contexts need to be taken into consideration, as the characteristics of the student population may vary significantly. Additionally, this finding serves as a reminder to governments and universities that certain countries/regions have already made significant strides in incorporating GenAI into their campuses. Third, the questionnaire adopted in this study is adapted to GenAI situations based on prior studies (Memarian and Doleck, 2023; Ng, Wu et al., 2023). At the same time, exploratory and confirmatory factor analysis may hardly be calculated in this study as this study has relied on few selected questions to determine the responses for each of the factors. Nevertheless, as Ng et al. (2024) designed the questionnaire on AI literacy contexts; this study helps contribute to adapt existing AI literacy measures for the context of generative AI across the United Kingdom and Hong Kong. By developing a GenAI-specific literacy questionnaire, this study provided novel insights into how demographic factors influence students’ understanding and perceptions of this emerging technology in ChatGPT world.

Consequently, there is a need to design learning programs and allocate resources to prevent universities from falling behind. Future research should also assess the effectiveness of these learning programs and resources. Regarding the new framework proposed, future research could delve into comparing GenAI literacy levels among different universities, since this study did not compare the GenAI differences between universities. In addition, further research is needed to explore the impact of micro level in more detail, because the sample size of this study was small, and data were collected only within two country contexts. While this study primarily focuses on students’ characteristics, it is crucial to explore the learning outcomes and how different groups of students may learn differently in programs that incorporate a more comprehensive understanding of AI.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Xianghan O’Dea

Xianghan O’Dea is a Senior Fellow of Higher Education Academy, and Senior Lecturer in Management and Technology Education in the Department of Public Services Management & Organisation at King’s Business School. She has more than 20 years of experience in Higher Education as lecturer, researcher and academic developer. Christine holds a PhD in Education from the University of Hull and was previously Subject Group Leader (Logistics, Transportation, Operations and Analytics) at the Huddersfield Business School, University of Huddersfield. Her research focuses on AI in education (AIEd), with a particular focus on Generative AI and AI literacy; digital transformation and digital education. She has published her research in high-impact journals, such as Studies in Higher Education, British Journal of Educational Technology, and Fudan Journal of the Humanities and Social Sciences. She has also published featured articles regularly in prestigious media, such as Times Higher Education and Wonkhe.

Davy Tsz Kit Ng is an assistant professor at the Department of Mathematics and Information Technology, The Education University of Hong Kong, and also a visiting scholar at the Hong Kong University of Science and Technology. Previously he worked on the IT and STEM Education Panel. He completed his PhD Education focusing on AI in learning and education in the Faculty of Education, the University of Hong Kong. His research interests lie in the areas of AI literacy, STEM Education and technology-enhanced pedagogic innovation.

Michael O’Dea is a Lecturer in Computer Science at the University of York, U.K.. He holds an Ed.D. from the University of Leeds, where his work focused on Technology Enhanced Learning and he is a Senior Fellow of the HEA. His research interests are in AI literacy, Technology Acceptance and Machine Learning.

Viacheslav Shkuratskyy is a postgraduate PhD student and Computer Science Technician at York St John University.His research interests include machine learning, data analytics, cyber security, and IoT.

References

Baker

Hawn

(2021) Algorithmic bias in education. International Journal of Artificial Intelligence in Education 32: 1052–1092.

Barrett

Pack

(2023) Not quite eye to AI: student and teacher perspectives on the use of generative artificial intelligence in the writing process. International Journal of Educational Technology in Higher Education 20(1): 59.

Bellas

Guerreiro-Santalla

Naya

, et al. (2023) AI curriculum for European high schools: an embedded intelligence approach. International Journal of Artificial Intelligence in Education 33(2): 399–426.

Berkovich

(2014) A socio-ecological framework of social justice leadership in education. Journal of Educational Administration 52(3): 282–309.

Birhane

Kasirzadeh

Leslie

, et al. (2023) Science in the age of large language models. Nature Reviews Physics 5(5): 277–280.

Braun

Clarke

(2021) One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology 18(3): 328–352.

Çakıroğlu

Gökoğlu

Öztürk

(2017) Pre-service computer teachers’ tendencies towards the use of mobile technologies: a technology acceptance model perspective. European Journal of Open, Distance and E-Learning 20(1): 176–191.

Cardon

Fleischmann

Aritz

, et al. (2023) The challenges and opportunities of AI-assisted writing: developing AI literacy for the AI age. Business and Professional Communication Quarterly 86(3): 257–295.

Casal-Otero

Catala

Fernández-Morante

, et al. (2023) AI literacy in K-12: a systematic literature review. International Journal of STEM Education 10(1): 29.

10.

Celik

(2023) Towards Intelligent-TPACK: an empirical study on teachers’ professional knowledge to ethically integrate artificial intelligence (AI)-based tools into education. Computers in Human Behavior 138: 107468.

11.

Cetindamar

Kitto

, et al. (2022) Explicating AI literacy of employees at digital workplaces. IEEE Transactions on Engineering Management.

12.

Chan

CKY

(2023) A comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education 20(1): 38.

13.

Chiu

(2021) A holistic approach to the design of artificial intelligence (AI) education for K-12 schools. TechTrends 65(5): 796–807.

14.

Chiu

(2023) The impact of Generative AI (GenAI) on practices, policies and research direction in education: a case of ChatGPT and Midjourney. Interactive Learning Environments 1–17.

15.

Chiu

Xia

Zhou

, et al. (2023) Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Computers and Education: Artificial Intelligence 4: 100118.

16.

Cooper

(2023) Examining science education in chatgpt: an exploratory study of generative artificial intelligence. Journal of Science Education and Technology 32(3): 444–452.

17.

Dijkstra

Genç

Kayal

, et al. (2022) Reading comprehension quiz generation using generative pre-trained transformers. Retrieved from: https://e.humanities.uva.nl/publications/2022/dijk_read22.pdf

18.

El Shazly

(2021) Effects of artificial intelligence on English speaking anxiety and speaking performance: a case study. Expert Systems 38(3): e12667.

19.

Escalante

Pack

Barrett

(2023) AI-generated feedback on writing: insights into efficacy and ENL student preference. International Journal of Educational Technology in Higher Education 20(1): 57.

20.

Farina

Lavazza

(2024) Ethical considerations and policy interventions concerning the impact of generative AI tools in the economy and in society. AI and Ethics: 1–9.

21.

Fisher

Thompson

Brookes

(2020) Gender differences in the Australian undergraduate STEM student experience: a systematic review. Higher Education Research and Development 39(6): 1155–1168.

22.

Fui-Hoon Nah

Zheng

Cai

, et al. (2023) Generative AI and ChatGPT: applications, challenges, and AI-human collaboration. Journal of Information Technology Case and Application Research 25(3): 277–304.

23.

Hageman

Cheon

(2023) Factor analysis of the financial strain index from the welfare, children and families: a three-city study, wave 3. Journal of Financial Therapy 14(1): 3.

24.

Holmes

Miao

(2023) Guidance for Generative AI in Education and Research. France: UNESCO Publishing.

25.

Hornberger

Bewersdorff

Nerdel

(2023) What do university students know about Artificial Intelligence? Development and validation of an AI literacy test. Computers and Education: Artificial Intelligence 5: 100165.

26.

IBM (2022) IBM global AI adoption index. Available at: https://www.snowdropsolution.com/pdf/IBMGlobalAIAdoptionIndex2022.pdf (accessed 18 September 2024).

27.

Jia

Cui

Xiao

, et al. (2021) All-in-one: multi-task learning bert models for evaluating peer assessments. arXiv preprint arXiv:2110.03895.

28.

Kandlhofer

Steinbauer

Hirschmugl-Gaisch

, et al. (2016) Artificial intelligence and computer science in education: from kindergarten to university. In: 2016 IEEE Frontiers in education conference (FIE), Eire, PA, 12–15 October 2016. IEEE, pp. 1–9.

29.

Koh

Doroudi

(2023) Learning, teaching, and assessment with generative artificial intelligence: towards a plateau of productivity. Learning: Research and Practice 9(2): 109–116.

30.

Kohnke

Moorhouse

Zou

(2023) Exploring generative artificial intelligence preparedness among university language instructors: a case study. Computers and Education: Artificial Intelligence 5: 100156.

31.

Kong

Cheung

WMY

Zhang

(2021) Evaluation of an artificial intelligence literacy course for university students with diverse study backgrounds. Computers and Education: Artificial Intelligence 2: 100026.

32.

Kong

Cheung

WMY

Zhang

(2023) Evaluating an artificial intelligence literacy programme for developing university students’ conceptual understanding, literacy, empowerment and ethical awareness. Educational Technology & Society 26(1): 16–30.

33.

Lai

Hong

(2015) Technology use and learning characteristics of students in higher education: do generational differences exist? British Journal of Educational Technology 46(4): 725–738.

34.

Laupichler

Aster

Schirch

, et al. (2022) Artificial intelligence literacy in higher and adult education: a scoping literature review. Computers and Education: Artificial Intelligence 3: 100101.

35.

Lee

Ali

Zhang

, et al. (2021) Developing middle school students' AI literacy. In: Proceedings of the 52nd ACM technical symposium on computer science education, USA, 13–20 March 2021, pp. 191–197.

36.

Lim

(2023) The effects of pre-service early childhood teachers’ digital literacy and self-efficacy on their perception of AI education for young children. Education and Information Technologies 28(5): 1–27.

37.

Lim

Gunasekara

Pallant

, et al. (2022) Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education 21(2).

38.

Lin

Van Brummelen

(2021, May) Engaging teachers to co-design integrated AI curriculum for K-12 classrooms. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1–12.

39.

Long

Magerko

(2020) What is AI literacy? Competencies and design considerations. In: Proceedings of the 2020 CHI conference on human factors in computing systems, Honolulu, HI, 25–30 April 2020, pp. 1–16.

40.

MacNeil

Tran

Mogil

, et al. (2022) Generating diverse code explanations using the gpt-3 large language model. In: Proceedings of the 2022 ACM conference on international computing education research, Switzerland, 7–11 August 2022, pp. 37–39.

41.

McKinsey (2022) McKinsey technology trends outlook 2022. Available at: https://www.mckinsey.com/∼/media/mckinsey/businessfunctions/mckinseydigital/ourinsights/thetoptrendsintech2022/mckinsey-tech-trends-outlook-2022-full-report.pdf (accessed 18 September 2024).

42.

Memarian

Doleck

(2023) Fairness, accountability, transparency, and ethics (fate) in artificial intelligence (AI), and higher education: a systematic review. Computers and Education: Artificial Intelligence 5: 100152.

43.

Michalos

(ed) (2014) Encyclopedia of Quality of Life and Well-Being Research. Dordrecht: Springer Netherlands, Vol. 171.

44.

Mishra

Warr

Islam

(2023) TPACK in the age of ChatGPT and generative AI. Journal of Digital Learning in Teacher Education 39(4): 235–251.

45.

Moorhouse

Yeo

Wan

(2023) Generative AI tools and assessment: guidelines of the world's top-ranking universities. Computers and Education Open 5: 100151.

46.

Mulders

Kleipool

Dingemans

, et al. (2018) Normative data for the patient-rated wrist evaluation questionnaire. Journal of Hand Therapy 31(3): 287–294.

47.

DTK

Leung

JKL

Chu

SKW

, et al. (2021) Conceptualizing AI literacy: an exploratory review. Computers and Education: Artificial Intelligence 2: 100041.

48.

DTK

Lee

Tan

RJY

, et al. (2023a) A review of AI teaching and learning from 2000 to 2020. Education and Information Technologies 28(7): 8445–8501.

49.

DTK

Leung

JKL

, et al. (2023b) Teachers’ AI digital competencies and twenty-first century skills in the post-pandemic world. Educational Technology Research & Development 71(1): 137–161.

50.

Nouraldeen

(2023) The impact of technology readiness and use perceptions on students’ adoption of artificial intelligence: the moderating role of gender. Development and Learning in Organizations: An International Journal 37(3): 7–10.

51.

O’Dea

(2024) Generative AI: is it a paradigm shift for higher education? Studies in Higher Education 49(5): 811–816. doi: 10.1080/03075079.2024.2332944.

52.

O'Dea

(2023) Is artificial intelligence really the next big thing in learning and teaching in higher education? A conceptual paper. Journal of University Teaching and Learning Practice 20(5).

53.

Paray

Kumar

(2020) Does entrepreneurship education influence entrepreneurial intention among students in HEI’s? The role of age, gender and degree background. Journal of International Education in Business 13(1): 55–72.

54.

Sanderson

Douglas

, et al. (2023) AI ethics principles in practice: perspectives of designers and developers. IEEE Transactions on Technology and Society 4(2): 171–187.

55.

Schlagwein

Willcocks

(2023) ‘ChatGPT et al.’: The ethics of using (generative) artificial intelligence in research and science. Journal of Information Technology 38(3): 232–238.

56.

Solyst

Xie

Yang

, et al. (2023) “I would like to design”: black girls analyzing and ideating fair and accountable AI. In: Proceedings of the 2023 CHI conference on human factors in computing systems, Hamburg, Germany, 23–28 April 2023, pp. 1–14.

57.

Southworth

Migliaccio

Glover

, et al. (2023) Developing a model for AI across the curriculum: transforming the higher education landscape via innovation in AI literacy. Computers and Education: Artificial Intelligence 4: 100127.

58.

Yang

(2023) Artificial Intelligence (AI) literacy in early childhood education: an intervention study in Hong Kong. Interactive Learning Environments: 1–15.

59.

UNESCO (2023) Global Education Monitoring Report 2023: Technology in Education-A Tool on Whose Terms? UN.

60.

University of Hong Kong (HKU) (2023a) HKU introduces new policy to fully integrate GenAI in Teaching and Learning. Retrieved from: https://www.hku.hk/press/press-releases/detail/26434.html

61.

University of Hong Kong (HKU) (2023b) About ChatGPT. Retrieved from: https://tl.hku.hk/2023/02/about-chatgpt/

62.

Vachovsky

Chaturapruek

, et al. (2016) Toward more gender diversity in CS through an artificial intelligence summer program for high school girls. In: Proceedings of the 47th ACM technical symposium on computing science education, Memphis, Tennessee, 2–5 March 2016, pp. 303–308.

63.

Chen

Luo

, et al. (2020) The landscape of immune cells infiltrating in prostate cancer. Frontiers in Oncology 10: 517637.

64.

Chen

, et al. (2023) More intensive use and lifetime extension can enable net-zero emissions in China’s cement cycle. Resources, Conservation and Recycling 198: 107144.

65.

Xia

Chiu

Lee

, et al. (2022) A self-determination theory (SDT) design approach for inclusive and diverse artificial intelligence (AI) education. Computers & Education 189: 104582.

66.

PWD

Golden

(2019) Developing TPACK in elementary mathematics education: a framework to design activities with pre-service teachers. Handbook of Research on TPACK in the Digital Age. USA: IGI Global, 47–68.

67.

Zawacki-Richter

Marín

Bond

, et al. (2019) Systematic review of research on artificial intelligence applications in higher education–where are the educators? International Journal of Educational Technology in Higher Education 16(1): 1–27.

68.

Zhang

Lee

Ali

, et al. (2023) Integrating ethics and career futures with technical learning to promote AI literacy for middle school students: an exploratory study. International Journal of Artificial Intelligence in Education 33(2): 290–324.

69.

Zohny

McMillan

King

(2023) Ethics of generative AI. Journal of Medical Ethics 49(2): 79–80.

Factors affecting university students’ generative AI literacy: Evidence and evaluation in the UK and Hong Kong contexts

Abstract

Keywords

Introduction

Literature review

Current research on AI literacy

The four cognitive domains of AI literacy framework

Methods

Participants and demographic details

Study design and procedure

Data analysis

Results

Students’ perceptions (RQ1)

Factors affecting university students’ GenAI literacy in the UK and Hong Kong contexts (RQ2)

Country/region of origin

Whether taken a course about AI

Gender

Knowing how to use AI to solve problems

Frequency of AI usage

Software utilization and usage intention

Subject discipline

Discussion

Generative AI literacy across learner groups

The need of generative AI literacy

GenAI literacy framework

Conclusion

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

References