Sage Journals: Discover world-class research

Abstract

Artificial intelligence (AI) education is becoming increasingly important worldwide. However, there has been no measuring instrument for diagnosing the students’ current perspective. Thus the aim of this study was to develop an instrument that measures student attitudes toward AI. The instrument was developed by verifying the reliability and validity by 8 computer education PhD using a sample of 305 K-12 students. This scale made students’ attitudes toward AI operational and quantifiable. Accordingly, educators can use it to diagnose the current status of students or verify the effectiveness of new AI education methods.

Keywords

evaluation methodologies artificial intelligence in education student attitude

Introduction

Artificial intelligence (AI), according to Coppin (2004, p. 4), is the ability of machines to adapt to new situations, deal with emerging situations, solve problems, answer questions, device plans, and perform various other functions that require some level of intelligence typically evident in human beings. At present, AI is becoming a key variable in the fields of the technology, economy, and politics. The Fourth Industrial Revolution would most probably be determined by the relationship between humans and AI (ETRI, 2019). As a result, AI education is gaining prominence, and many countries, including the United States, Singapore, China, Korea, Australia, and those in the European Union, are exploring ways to effectively integrate AI education into their K-12 curriculum (Chiu, 2021; Heintz, 2021; J. Kim & Park, 2019; Steinbauer et al., 2021).

Scholars are, therefore, primarily concerned about what needs to be taught in AI education (Ali et al., 2019; E.-K. Lee, 2020; Touretzky et al., 2019). For example, Yoo (2019) divided AI-related education into 40 items and examined the importance and performance of each item for graduate students. Yoo found that enhancing openness toward learning about AI was the primary prerequisite for developing other elements of AI education. However, knowledge on openness to learn AI or attitudes toward learning AI are mostly public opinions (Ikkatai et al., 2022; Schepman & Rodway, 2020). While we can gain some insights about motivation, perception, and sentiment toward AI from previous studies that measured attitudes toward various forms of technology-enhanced learning (Cheung & Vogel, 2013; Dunn & Kennedy, 2019) and attitudes toward science, technology, engineering, and mathematics (STEM) education (Cherry et al., 2020; Cukurova et al., 2020; Gaines-Ross, 2016; Gherheș & Obrad, 2018; Manikonda & Kambhampati, 2018), none of them specifically focused on attitude toward AI education.

If students do not develop a positive attitude toward professional skills learning, they are less likely to master it regardless of the effectiveness of their education (Ajzen, 1991; Fredrickson, 2001). Thus, measuring attitudes toward AI can be an important factor in the success or failure of AI education. Our review of the literature showed that many of the studies on learners’ attitudes toward AI for learning either measured only computational thinking (K. Kim, 2019) or used unvalidated modified instruments (K. Kim, 2019; Y. Lee, 2019) and self-developed surveys with open-ended questions (Y. E. Kim & Kim, 2019; Park & Shin, 2017; S. Shin et al., 2017, 2018; N. M. Shin & Kim, 2007). Most importantly, of these studies have not used measurement instrument that have been developed to be reliable and validated. The purpose of our study, therefore, is to develop a standardized instrument that measures K-12 student attitudes toward AI. Specifically, we would like to identify questions that can be used to objectively measure learners’ attitudes toward AI. We would also like to further test the validity of categorizing student attitudes toward AI by the traditional approach that suggests attitude is made of three components—cognitive, affective, and behavioral. Our study, we believe, is significant because it would be the first study that presents K-12 perceptions on AI and brings AI education into operational and quantifiable areas.

Theoretical Background

Attitude

In modern times, attitude is described as a “psychological tendency, expressed by evaluating a particular entity with some degree of favor or disfavor” (Eagly & Chaiken, 1993). The traditional approach defines attitude as consisting of three complementary and not mutually exclusive components (Bagozzi, 1978; Fishbein & Ajzen, 1972; Metsärinne & Kallio, 2016): (1) the emotional component signifies the positive-negative emotional relations or feelings one has toward an object or activity; (2) the behavioral component indicates the action tendencies one has to respond to an object in a particular way; and (3) the cognitive component marks the content of one’s thoughts, such as one’s beliefs regarding what constitutes a fact.

The following approaches are useful in investigating attitude as a determinant of the use of IT or other new technologies: the theory of reasoned action (TRA; Fishbein & Ajzen, 1975), the theory of planned behavior (TPB; Ajzen, 1991), and the technology acceptance model (TAM; Davis et al., 1989). TRA maintains that one’s attitude toward a particular behavior and the subjective norms that condition that behavior encourage or discourage performance of that behavior. TPB adds perceived behavioral control to the two factors affecting intention in TRA. The TAM considers perceived usefulness, which is “the prospective user’s subjective probability that using a specific application system will increase his or her job performance within an organizational context” (Davis et al., 1989), and perceived ease of use, which is “the degree to which the prospective user expects the target system to be free of effort” (Davis et al., 1989, p. 985); both affect attitude and behavioral intention. Indeed, many studies have investigated how attitude predicts and affects behavior (Bohner & Dickel, 2011; Glasman & Albarracín, 2006; Gorges et al., 2012; Petty et al., 2009; Steinmetz et al., 2016).

As for education, these attitude-behavioral intentions may appear as learning intentions as confirmed by studies related to mathematics (Huang et al., 2016), science (Khine, 2015), and engineering education (Alias et al., 2018). According to the TPB (Ajzen, 1991) students’ attitudes toward professional skill learning play an important role in whether they actually acquire that skill, while at the same time positive attitudes toward learning positively influence students’ learning intentions. Accordingly, it has been found that students’ positive attitudes can improve learning achievements (Alias et al., 2018; Cukurova et al., 2020) and help curriculum developers and teachers optimize lessons (Dunlap & Dugger, 1991; Yu et al., 2012). This is also related to the idea that “people’s general attitudes toward AI are likely to play a large role in their acceptance of AI” (Schepman & Rodway, 2020). Therefore, to successfully implement AI education, there is a need to measure and understand students’ attitudes toward AI.

Attitude Measurements

This study draws from the methodology used in previous studies to determine the reliability and validity of instruments used for measuring attitudes. References include attitudes toward technology (Ardies et al., 2013; Choi et al, 2009; C. S. Lee 2008; Svenningsson et al., 2018), mathematics (Hannula et al., 2016), STEM (Benek & Akcay, 2019), engineering (Choi et al., 2009), and science education (Glynn et al., 2009), as well as evaluations of the effects of software (SW) education on attitude (Park & Shin, 2017). Further, we referred to the study by Schepman and Rodway (2020), where a scale was developed to measure the general attitudes toward AI.

Materials and Methods

Research Procedure

First, to solve research problems, related literature and previous studies as shown in Tables 1 and 2 were examined, and 52 preliminary questions for measuring attitudes toward AI were selected. Second, a 3-point Likert scale consisting of the following choices: “essential,” “useful but not essential,” and “not necessary” (Likert, 1932), was used to examine experts’ opinions about the validity of each item. Content validity refers to how well a survey or test measures the construct that it sets out to measure. The data were examined by using a content validity ratio (CVR) test (Lawshe, 1975). A total of 26 items that did not satisfy the CVR value were removed. Third, the remaining 26 questions were gathered into a questionnaire that was administered to 305 upper-elementary, middle, and high school students in Seoul, Korea. Students were asked to respond to statements using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Lastly, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were conducted, and the final questions were selected. Descriptive statistics and distribution of total scores were prepared for student responses. The study procedure is shown in Table 3.

Table 1.

Previous Studies on Attitude Measurement (Attitude Scale).

Measurement target	Country	Purpose	Question		Example item	References
Measurement target	Country	Purpose	Item	Response option	Example item	References
Student (all levels)	Republic of Korea	Attitude toward technology	58	5-point Likert scale	I want to know more about the computer. I will choose a technical career in the future.	C. S. Lee (2008)
Undergraduate student	Republic of Korea	Attitude toward engineering	39	5-point Likert scale	Engineering is important for understanding the direction of society’s development. I enjoy reading books about engineering.	Choi et al. (2009)
Student (all levels)	Republic of Korea	Evaluation of effectiveness of SW education	40	7-point Likert scale	I think SW is very important for the development of our society. I want to keep learning about SW in the future.	Park and Shin (2017)
Student (all levels)	USA	Attitude toward technology	25	5-point Likert scale	Technology is very important in life. There should be more education about technology.	Ardies et al. (2013)
Undergraduate student	USA	Motivation to learn science	25	5-point Likert scale	Learning science is interesting. Knowing science will give me a career advantage. Learning science makes my life more meaningful. I believe I can master science knowledge and skills.	Glynn et al. (2009)
Secondary school student	Turkey	Attitude toward STEM	33	5-point Likert scale	I am curious about science. In the future, I would like to have a job in technology.	Benek and Akcay (2019)
Fifth and ninth grade students	Sweden	Attitudes toward social robots	14	Yes or No	Could you talk to a robot? Could you ask a robot for help with your schoolwork?	Serholt et al. (2014)

Note. SW = software education.

Table 2.

Measurement and Evaluation in Previous Studies of AI Education (AI Education Scale).

Measurement target	Purpose	Question form	Example item	References
7^th grade student	Perception of intelligent robots as companions	Open narrative	How do you solve conflicts between a robot and an individual?	Y. E. Kim and Kim (2019)
Elementary school student	Elementary school students’ images of AI	Open narrative	How do you feel when you think of AI?	S. Shin et al. (2018)
High school student	High school students’ perceptions of AI (emotion and risk perception)	Open narrative	Job risk awareness and ethical risk awareness.	Shin et al. (2017)
Student (all levels)	Perceptions of artificial intelligence technology and AI teachers			Park and Shin (2017)
		Open narrative	What is the perception of AI technology?
		Open narrative	Can AI replace teachers?
Student (all levels)	Perceptions and attitudes toward robots	Open narrative	Image of the robot.Learning about robots.Learning from robots.Learning with robots.	N. M. Shin and Kim (2007)

Table 3.

Research Procedure.

Step	Objectives	Method	Respondent
Preparation of test question	Develop instrument	Specification of research questions	-
Preparation of test question	Develop instrument	Related literature Review	-
Pilot survey	Confirm instrument	Expert validity test	8 Professors of Computer Education
First main survey	Validated instrument	Exploratory factor analysis	119 students
Second main survey	Validated instrument	Confirmatory factor analysis	186 students
Statistical analysis	Summary of responses	Statistical analysis	305 students

Preparation of Test Questions

Two categories of studies were referenced to develop the attitude scale for AI. First, studies related to attitude measurement conducted in Korea, the United States, Turkey, and Sweden were considered (Table 1). The purpose of the studies was to measure attitudes toward SW, STEM, and science education. As a result of analyzing the sub-elements classified in these studies, it was possible to categorize them into nine categories such as learner’s career, experience of interest in each subject, necessity of learning each subject in school, perceived usefulness (for the learner), perceived usefulness (for the society), acceptance of each subject, career related to each subject, each subject for all genders, and perceived ease of use of each subject. These tests had a range of 14 to 58 questions with either true/false and 5- or 7-point Likert scale responses.

Second, the category of studies that we referred to were all AI-based (Table 2). All these studies used open-ended questions and were administered to students at different grade levels to understand their perceptions about AI and robots.

A total of 52 items were set up through the process of referring to the range of these items, integrating common items among the questions related to the research problem, and excluding unrelated items.

Expert Validity Test

Experts who participated in the preliminary item validity evaluation included professors of computer education and doctors who majored in computer education. The panel had eight members, which is acceptable (Lynn, 1986). The CVR was verified through a 3-point Likert scale (Lawshe, 1975). The formula for calculating the CVR is as follows.

CVR = \frac{N_{e} - \frac{2}{N}}{\frac{2}{N}} (\begin{array}{l} N_{e} : Number of fitted responses, \\ N : Total responses \end{array})

The CVR score for this panel should be 0.75, which means that more than seven out of eight respondents designate an item as “essential” (Lawshe, 1975).

The CVR is mostly adopted to select or exclude individual items testing its validity (Wilson et al., 2012). Among the initial 52 items, 26 items did not satisfy the CVR value requirements. Our final student questionnaire, therefore, consisted of the remaining 26 items (listed in Appendix 1).

Main Survey for Students

In public schools in Korea, AI classes are not part of the regular curriculum. Therefore, students with experience in SW education, most relevant to AI education, were surveyed. The survey was conducted twice from November 2019 to December 2019, and in December 2021. Both surveys were conducted with the same tool, which is presented in Appendix 2. Dataset 1 was provided by 119 students from grades 5 to 6, and Dataset 2 was provided by 186 students from grades 6 to 11 (listed in Table 4). All procedures performed in this study were in accordance with the ethical standards of the Institutional Review Board Sungkyunkwan University (approval number SKKU2019-11-012). Consent from the relevant guardians was obtained.

Table 4.

Demographics of Respondents in the Main Study.

	Grade	Boys	Girls	Total
Dataset 1	5th	28	35	119
Dataset 1	6th	27	29	119
Dataset 2	6th	42	39	186
	7th	12	10
	10th	16	0
	11th	46	21
	Total	171	134	305

Exploratory Factor Analysis

EFA is one of a family of multivariate statistical methods that attempts to identify the smallest number of hypothetical constructs that can parsimoniously explain the covariation observed among a set of measured variables (Watkins, 2018). EFA was conducted on Dataset 1, the result of 119 students’ responses, and through this, the validity and relevance of sub-factors and items were reviewed. Prior to factor analysis, the Kaiser–Meyer–Olkin (KMO) test was conducted, and the suitability of factor analysis was identified using Bartlett’s test. Subsequently, to understand the validity of the constituent concept of the 26 selected scale questions, EFA was performed to extract potential factors using the varimax method and principal component analysis. Additionally, Cronbach’s alpha was used to check the reliability of the instrument.

Confirmatory Factor Analysis

CFA is a type of structural equation modeling that deals specifically with measurement models-that is, the relationships between observed measures or indicators and latent variables of factors (Brown, 2015). In such CFA, validity is secured when the standardized λ value is .5 or higher, the average variance extracted (AVE) is .5 or higher, and the construct reliability (CR) is .7 or higher. Additionally, the absolute fit index, an index that absolutely evaluates the degree of conformity between the collected data and the research model, was investigated. The incremental fit index, which compares the index indicating the accuracy of the structural equation model of the study with the model in which correlation between variables was not set, was also investigated.

Results

In the expert validity test, a group of eight experts verified the content validity of 52 items constructed through literature research. They evaluated the content validity according to a 3-point scale. Consequently, ambiguous expressions with the phrases “anyone” or “make everything good,” items about gender differences in attitude toward AI, and 26 negative questions, such as AI is difficult, were rejected as they did not meet the CVR value. Thus, the final survey questionnaire for students included the remaining 26 question items.

The result of EFA using Dataset 1 showed a KMO value of 0.937, indicating that the item selection was good. Additionally, Bartlett’s sphericity was found to be significant (χ² = 2,645.682, df = 325, p = .000); common factors existed and were appropriate. Accordingly, three sub-factors were extracted, and factor 1 was named “Behavioral,” factor 2 was named “Affective,” and factor 3 was named “Cognitive.” Cronbach’s alpha of all factors indicated excellent reliability (Tavakol & Dennick, 2011). The above results and the eigenvalue and cumulative percentage for each factor for the development of the instrument that measures attitudes toward AI are shown in Table 5.

Table 5.

Results of Exploratory Factor Analysis.

Item number	Commonality	Factor			Cronbach’s α
Item number	Commonality	F1	F2	F3	Cronbach’s α
16	0.775	0.837	0.266	0.067	.956
17	0.714	0.818	0.155	0.145
15	0.713	0.812	0.222	0.065
18	0.709	0.763	0.262	0.239
22	0.702	0.717	0.242	0.36
23	0.685	0.684	0.284	0.369
21	0.584	0.672	0.204	0.301
20	0.714	0.651	0.358	0.402
26	0.582	0.625	0.241	0.366
24	0.699	0.613	0.402	0.402
19	0.609	0.607	0.398	0.286
25	0.627	0.583	0.195	0.5
7	0.720	0.264	0.804	0.056	.924
10	0.666	0.234	0.76	0.183
6	0.622	0.078	0.702	0.351
8	0.573	0.304	0.65	0.24
13	0.657	0.449	0.642	0.211
9	0.514	0.257	0.637	0.205
11	0.538	0.164	0.604	0.383
5	0.557	0.192	0.588	0.417
14	0.529	0.431	0.559	0.175
12	0.451	0.249	0.527	0.334
1	0.760	0.287	0.283	0.773	.905
2	0.684	0.236	0.307	0.731
3	0.742	0.296	0.36	0.725
4	0.638	0.313	0.288	0.676
Factor domain		Behavioral	Affective	Cognitive
Eigenvalue		13.788	2.422	1.563
Percentage of variance		53.031	9.315	6.013
Cumulative percentage		53.031	62.347	68.359
KMO		0.937
Bartlett’s test	Approx. χ²	2,645.682
	df	325
	p-Value	.000

Note. Extraction method = principal component analysis; rotation method = varimax with Kaiser normalization.

Bold text indicates the same category.

A CFA was performed using Dataset 2 to confirm the validity of the items. Validity is secured when the standardized λ value is .5 or higher, the AVE is .5 or higher, and the CR is .7 or higher. After performing a CFA on the sub-factors for each component, standardized, and appropriate λ, AVE, and CR values were all secured, as illustrated in Table 6, indicating the model used in the study was valid.

Table 6.

Results of Confirmatory Factor Analysis.

Factor			SE	CR	Standardized coefficient	AVE	Construct reliability
Beh._1	<-	Behavioral	1		.856	.582630194	.943552573
Beh._2	<-	Behavioral	0.923	14.121	.809
Beh._3	<-	Behavioral	0.938	15.316	.847
Beh._4	<-	Behavioral	0.836	15.199	.843
Beh._5	<-	Behavioral	0.857	15.469	.852
Beh._6	<-	Behavioral	0.835	16.434	.879
Beh._7	<-	Behavioral	0.818	14.041	.806
Beh._8	<-	Behavioral	0.836	15.332	.847
Beh._9	<-	Behavioral	0.847	15.295	.846
Beh._10	<-	Behavioral	0.737	14.297	.815
Beh._11	<-	Behavioral	0.83	14.618	.825
Beh._12	<-	Behavioral	0.704	11.777	.721
Aff._1	<-	Affective	1		.828	.539872315	.921073786
Aff._2	<-	Affective	0.899	12.586	.784
Aff._3	<-	Affective	0.979	11.72	.746
Aff._4	<-	Affective	1.091	13.352	.815
Aff._5	<-	Affective	1.062	13.299	.813
Aff._6	<-	Affective	0.846	11.596	.74
Aff._7	<-	Affective	0.962	10.793	.703
Aff._8	<-	Affective	0.774	9.752	.65
Aff._9	<-	Affective	0.998	12.878	.796
Aff._10	<-	Affective	0.96	10.888	.707
Cog._1	<-	Cognitive	1		.907	.709802673	.907188241
Cog._2	<-	Cognitive	0.913	17.098	.857
Cog._3	<-	Cognitive	0.96	17.92	.875
Cog._4	<-	Cognitive	0.98	16.089	.834

Note. SE = standard error; CR = critical ratio; AVE = average variance extracted.

Among the absolute fit indexes that can be recognized, it is appropriate for the root mean square error of approximation (RMSEA) to be between 0.05 and 0.1, and the standardized root mean square residual (SRMR) to be less than or equal to 0.5. Furthermore, among the incremental fit indexes, both the Turker–Lewis index (TLI) and the comparative fit index (CFI) are considered to be 0.9 or higher. In the model adopted for this study, RMSEA was 0.075, SRMR was 0.047, TLI was 0.928, and CFI was 0.937. In other words, all the mentioned goodness of fit were satisfied.

To summarize, 3 factors and 26 questions were found to be the most appropriate. This measurement instrument was named “Student Attitude Toward Artificial Intelligence” (SATAI). The detailed questionnaires are the same as those in Appendix 2.

Discussion

SATAI was developed and validated after a literature review using two groups of respondents. In particular, the results of the EFA for Dataset 1 showed the SATAI scale’s factor structure. The SATAI scale consists of 26 items comprising 3 components (cognitive, affective, and behavioral factors), and each question is measured using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The scores for these items can be summed (ranging from 26 to 130) to represent a student’s attitude toward AI: a higher score indicates a positive attitude toward AI and the likelihood that a student will be an active participant of AI education.

For example, descriptive statistics on the responses of a total of 305 students who participated in this study are as follows. The average of Cognitive Components consisting of four questions was 3.57 (SD = 0.986), the average of Affective Components consisting of 10 questions was 3.72 (SD = 0.841), and the average of Behavioral Components consisting of 12 questions was 3.16 (SD = 1.068). The mean and standard deviation of each item can be confirmed in Table 7.

Table 7.

Descriptive Statistics of the 305 Student Responses.

Factor	M	SD	Factor	M	SD
Cog._1	3.60	1.056	Aff._10	3.70	1.145
Cog._2	3.65	1.054	Beh._1	2.80	1.373
Cog._3	3.66	1.082	Beh._2	2.69	1.349
Cog._4	3.37	1.213	Beh._3	2.92	1.374
Aff._1	3.94	1.034	Beh._4	3.26	1.258
Aff._2	4.04	0.976	Beh._5	3.36	1.244
Aff._3	3.65	1.163	Beh._6	3.32	1.211
Aff._4	3.58	1.139	Beh._7	3.13	1.245
Aff._5	3.65	1.090	Beh._8	3.19	1.247
Aff._6	3.93	1.025	Beh._9	3.35	1.274
Aff._7	3.49	1.159	Beh._10	3.60	1.188
Aff._8	3.56	0.945	Beh._11	3.16	1.261
Aff._9	3.71	1.084	Beh._12	3.17	1.272

Figure 1 shows the distribution of the total score of the questions answered by 305 people. According to this, the total distribution of students tends to gradually increase to 1.3% at 26 to 40, 6% at 41 to 55, and 30% at 56 to 70, and then slightly decreases afterwards. Table 7 and Figure 1 make it possible to confirm the characteristics of AI of the 305 students who responded in this study. That is, “the average of these students” attitude toward AI considering the Affective component was the highest at 3.72, with the Cognitive and the Behavioral Components at 3.57 and 3.16, respectively, and “The distribution of total scores is the highest in the middle” can be seen. Through these examples, we confirmed that the students’ attitudes toward artificial intelligence have become measurable areas. In other words, the distribution scores obtained through the SATAI can be used to operatively diagnose and compare attitude changes in students before and after integrating AI into education.

Figure 1.

Distribution of total scores.

In previous studies, open-ended questions of subjective criteria were used to verify the effectiveness of AI education (Y. E. Kim & Kim, 2019; Park & Shin, 2017; S. Shin et al., 2017, 2018; N. M. Shin & Kim, 2007). Other studies used instruments to measure attitudes toward other subjects, without validating them (K. Kim, 2019; Y. Lee, 2019). The SATAI can be differentiated from conventional research because it was developed specifically for AI education, thus enabling teachers to quantify students’ attitudes toward AI.

Conclusions

This study is relevant in that it develops a scale that measures student attitudes toward AI and tests the validity of that scale. First, the importance of attitudes toward AI education was confirmed through a literature review. Literature revealed that learners of AI education felt that they were encountering a new subject. Thus, measuring their attitudes can provide instructors with ideas for curriculum design that encourage students to have a positive attitude toward AI learning (Ajzen, 1991; Khine, 2015). As for education through AI, AI appeared as a new technology to learners, which reinforced the importance of attitude through the TAM. Hence, in our study, a standardized instrument for measuring a learner’s attitude toward AI was developed with attention to students’ cognitive, affective, and behavioral attitudes. On the one hand, AI education is new, so there are no traditional curriculum or teaching methods (E.-K. Lee, 2020; Touretzky et al., 2019). On the other hand, SATAI can help teachers objectively measure students’ attitude toward AI. Moreover, instructors can use the test scores to design, modify, apply, and customize educational programs to meet learner needs. These advancements contribute to the emerging philosophical cornerstones of AI education. SATAI results, we believe, can be applied to verify the effects of AI-related education for other K-12 students outside the population used in this study.

This study has a few limitations. First, the measure developed can be used to verify the students’ attitudes toward AI, but cannot measure conceptualizations of AI. Therefore, future researchers should develop a scale that measures a learner’s concept of AI. Second, although the results of CFA on this dataset are strong evidence that the tool does not leverage age-related differences, there may be limitations in the generalization of the study by surveying 305 elementary, middle, and high school students. Future studies should include a wider range of participants, including individuals in other regions.

Footnotes

Appendix

Appendix 2.

Student Attitudes Toward AI (SATAI).

Domain	No.	Item
Cognitive components	1	I think that it is important to learn about AI in school.
	2	AI class is important.
	3	I think that lessons about AI should be taught in school.
	4	I think every student should learn about AI in school.
Affective components	5	AI is very important for developing society.
	6	I think AI makes people’s lives more convenient.
	7	AI is related to my life.
	8	I will use AI to solve problems in daily life.
	9	AI helps me solve problems in real life.
	10	I will need AI in my life in the future.
	11	AI is necessary for everyone.
	12	AI produces more good than bad.
	13	AI is worth studying.
	14	I think that most jobs in the future will require knowledge related to AI.
Behavioral components	15	I want to work in the field of AI.
	16	I will choose a job in the field of AI.
	17	I would participate in a club related to AI if there was one.
	18	I like using objects related to AI.
	19	It is fun to learn about AI.
	20	I want to continue learning about AI.
	21	I’m interested in AI-related TV programs or online videos.
	22	I want to make something that makes human life more convenient through AI.
	23	I am interested in the development of AI.
	24	It is interesting to use AI.
	25	I think that there should be more class time devoted to AI in school.
	26	I think I can handle AI well.

Note. The scale uses 5 points for strongly agree, 4 points for agree, 3 points for neutral, 2 points for disagree, and 1 point for strongly disagree.

Acknowledgements

We would like to thank Editage () for English language editing.

Author Contributions

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Woong Suh. The first draft of the manuscript was written by Seongjin Ahn and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript. Woong Suh and Seongjin Ahn are co–first authors.

Availability of Data and Materials

The data that support the findings of this study are available from Sungkyunkwan University, but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of Sungkyunkwan University.

Ethical Approval

All procedures performed in this study were in accordance with the ethical standards of the Institutional Review Board (IRB) at Sungkyunkwan University (approval number SKKU2019-11-012-003).

Informed Consent

Written informed consent was obtained from all the guardians of students who participated in this study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Woong Suh

References

Ajzen

(1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-T

Ali

Payne

B. H.

Williams

Park

H. W.

Breazeal

(2019). Constructionism, ethics, and creativity: Developing primary and middle school artificial intelligence education. Proceedings of the IJCAI 2019. http://robotic.media.mit.edu/wp-content/uploads/sites/7/2019/08/Constructionism__Ethics__and_Creativity.pdf

Alias

Lashari

T. A.

Akasah

Z. A.

Kesot

M. J.

(2018). Self-efficacy, attitude, student engagement: Emphasising the role of affective learning attributes among engineering students. International Journal of Engineering Education, 34(1), 226–235.

Ardies

De Maeyer

Gijbels

(2013). Reconstructing the pupils’ attitude towards technology-survey. Design and Technology Education, 18(1), 8–19.

Bagozzi

R. P.

(1978). The construct validity of the affective, behavioral, and cognitive components of attitude by analysis of covariance structures. Multivariate Behavioral Research, 13(1), 9–31. https://doi.org/10.1207/s15327906mbr1301_2

Benek

Akcay

(2019). Development of STEM attitude scale for secondary school students: Validity and reliability study. International Journal of Education in Mathematics. International Journal of Education in Mathematics, Science and Technology, 7(1), 32–52. https://doi.org/10.18404/ijemst.509258

Bohner

Dickel

(2011). Attitudes and attitude change. Annual Review of Psychology, 62, 391–417. https://doi.org/10.1146/annurev.psych.121208.131609

Brown

T. A.

(2015), Confirmatory factor analysis for applied research. Guilford Publications.

Cheung

Vogel

(2013). Predicting user acceptance of collaborative technologies: An extension of the technology acceptance model for e-learning. Computers and Education, 63, 160–175. https://doi.org/10.1016/j.compedu.2012.12.003

10.

Cherry

Rohit

Ashik

Keerthini

Aishah

Monzon

Poon

D. S.

(2020). Attitudes and perceptions of UK medical students towards artificial intelligence and radiology: A multicentre survey. Insights into Imaging, 11(1), 14. https://doi.org/10.1186/s13244-019-0830-7

11.

Chiu

T. K. F.

(2021). A holistic approach to the design of artificial intelligence (AI) education for K-12 schools. TechTrends, 65(5), 796–807. https://doi.org/10.1007/s11528-021-00637-1

12.

Choi

H.-Y.

Park

H.-M.

Lee

J.-G.

Ryu

S.-M.

(2009). Development of the instrument for assessment the attitude toward engineering. Journal of Korean Institute of Industrial Education, 34(2), 161–178.

13.

Coppin

(2004), Artificial intelligence illuminated. Jones & Bartlett Publishers Learning.

14.

Cukurova

Luckin

Kent

(2020). Impact of an artificial intelligence research frame on the perceived credibility of educational research evidence. International Journal of Artificial Intelligence in Education, 30(2), 205–235. https://doi.org/10.1007/s40593-019-00188-w

15.

Davis

F. D.

Bagozzi

R. P.

Warshaw

P. R.

(1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982–1003. https://doi.org/10.1287/mnsc.35.8.982

16.

Dunlap

D. D.

Dugger

W. E.

(1991). Comparing attitudes towards technology of third and fourth grade students in Virginia relative to their exposure to technology. PATT 5 Conference Proceedings (Vol. 2, pp. 194–201), Eindhoven.

17.

Dunn

T. J.

Kennedy

(2019). Technology enhanced learning in higher education; motivations, engagement and academic achievement. Computers and Education, 137, 104–113. https://doi.org/10.1016/j.compedu.2019.04.004

18.

Eagly

A. H.

Chaiken

(1993). The psychology of attitudes. Harcourt Publishers Brace Jovanovich College Publishers.

19.

ETRI. (2019). Seven AI trends by 2020-beyond perception (ETRI Insight Report). https://ksp.etri.re.kr/ksp/plan-report/doi?id=766

20.

Fishbein

Ajzen

(1972). Attitudes and opinions. Annual Review of Psychology, 23(1), 487–544. https://doi.org/10.1146/annurev.ps.23.020172.002415

21.

Fishbein

Ajzen

(Eds.). (1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Addison-Wesley.

22.

Fredrickson

B. L.

(2001). The role of positive emotions in positive psychology. The broaden-and-build theory of positive emotions. American Psychologist, 56(3), 218–226. https://doi.org/10.1037/0003-066X.56.3.218

23.

Gaines-Ross

(2016, October 24). What do people—not techies, not companies—think about artificial intelligence? Harvard Business Review. https://hbr.org/2016/10/what-do-people-not-techies-not-companies-think-about-artificial-intelligence

24.

Gherheș

Obrad

(2018). Technical and humanities students’ perspectives on the development and sustainability of artificial intelligence (AI). Sustainability, 10(9), 3066. https://doi.org/10.3390/su10093066

25.

Glasman

L. R.

Albarracín

(2006). Forming attitudes that predict future behavior: A meta-analysis of the attitude-behavior relation. Psychological Bulletin, 132(5), 778–822. https://doi.org/10.1037/0033-2909.132.5.778

26.

Glynn

S. M.

Taasoobshirazi

Brickman

(2009). Science motivation questionnaire: Construct validation with nonscience majors. Journal of Research in Science Teaching, 46(2), 127–146. https://doi.org/10.1002/tea.20267

27.

Gorges

Kandler

Bohner

(2012). Internationalization at home: Using learning motivation to predict students’ attitudes toward teaching in a foreign language. International Journal of Educational Research, 53, 107–116. https://doi.org/10.1016/j.ijer.2012.03.001

28.

Hannula

M. S.

Di Martino

Pantziara

Zhang

Morselli

Heyd-Metzuyanim

Lutovac

Kaasila

Middleton

J. A.

Jansen

Goldin

G. A.

(2016). Attitudes, Beliefs, Motivation and Identity in Mathematics Education: An Overview of the Field and Future Directions. (ICME-13 Topical Surveys). Springer International Publishing

29.

Hannula

M. S.

Di Martino

Pantziara

Zhang

Morselli

Heyd-Metzuyanim

Lutovac

Kaasila

Middleton

J.A.

Jansen

Goldin

G. A.

(2016). Attitudes, beliefs, motivation, and identity in mathematics education: An overview of the field and future directions (ICME-13 Topical Surveys) (pp. 1–35). Springer.

30.

Heintz

(2021). Three interviews about K-12 AI education in America, Europe, and Singapore. KI—Künstliche Intelligenz, 35(2), 233–237. https://doi.org/10.1007/s13218-021-00730-w

31.

Huang

N. N.

Chiu

L. J.

Hong

J. C.

(2016). Relationship among students’ problem-solving attitude, perceived value, behavioral attitude, and intention to participate in a science and technology contest. International Journal of Science and Mathematics Education, 14(8), 1419–1435. https://doi.org/10.1007/s10763-015-9665-y

32.

Ikkatai

Hartwig

Takanashi

Yokoyama

H. M.

(2022). Octagon measurement: Public attitudes toward AI ethics. International Journal of Human–Computer Interaction, 1–18. https://doi.org/10.1080/10447318.2021.2009669

33.

Khine

M. S.

(Ed.). (2015). Attitude measurements in science education: Classic and contemporary approaches. IAP.

34.

Kim

Park

(2019). Development of a board game-based gamification learning model for training on the principles of artificial intelligence learning in elementary courses. Journal of the Korean Association of Information Education, 23(3), 229–235. https://doi.org/10.14352/jkaie.2019.23.3.229

35.

Kim

(2019). An artificial intelligence education program development and application for elementary teachers. Journal of the Korean Association of Information Education, 23(6), 629–637. https://doi.org/10.14352/jkaie.2019.23.6.629

36.

Kim

Y.-E.

Kim

H.-C.

(2019). A study on middle school students’ perception on intelligent robots as companions. Journal of Korean Association of Computer Education, 22(4), 35–45. https://doi.org/10.32431/kace.2019.22.4.004

37.

Lawshe

C. H.

(1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x

38.

Lee

C.-S.

(2008). Development of technology attitude scale for Korean pupils. Journal of Korean Practical Arts Education, 14(2), 157–174. https://doi.org/10.17055/jpaer.2008.14.2.157

39.

Lee

E.-K.

(2020). Analysis of artificial intelligence curriculum of domestic and foreign elementary and secondary school. Journal of Computers in Education, 23(1), 37–44.

40.

Lee

(2019). An analysis of the influence of block-type programming language-based artificial intelligence education on the learner’s attitude in artificial intelligence. Journal of the Korean Association of Information Education, 23(2), 189–196. https://doi.org/10.14352/jkaie.2019.23.2.189

41.

Likert

(1932). A technique for the measurement of attitudes. Archives of Psychology, 22, 5–55.

42.

Lynn

M. R.

(1986). Determination and quantification of content validity. Nursing Research, 35(6), 382–385. https://doi.org/10.1097/00006199-198611000-00017

43.

Manikonda

Kambhampati

(2018). Tweeting AI: Perceptions of lay versus expert Twitterati [Conference paper]. Twelfth International AAAI Conference on Web and Social Media. https://www.aaai.org/ocs/index.php/ICWSM/ICWSM18/paper/viewPaper/17843

44.

Metsärinne

Kallio

(2016). How are students’ attitudes related to learning outcomes? International Journal of Technology and Design Education, 26(3), 353–371. https://doi.org/10.1007/s10798-015-9317-0

45.

Park

J. H.

Shin

N. M.

(2017). Students’ perceptions of artificial intelligence technology and artificial intelligence teachers. Journal of Korean Teacher Education, 34(2), 169–192.

46.

Petty

R. E.

Fazio

R. H.

Brinol

(Eds.). (2009), Attitudes: Insights from the new implicit measures. Psychology Press.

47.

Schepman

Rodway

(2020). Initial validation of the general attitudes towards Artificial Intelligence Scale. Computers in Human Behavior Reports, 1, 100014. https://doi.org/10.1016/j.chbr.2020.100014

48.

Serholt

Barendregt

Leite

Hastie

Jones

Paiva

Vasalou

Castellano

(2014, August 25–29). Teachers’ views on the use of empathic robotic tutors in the classroom [Conference paper]. The 23rd IEEE International Symposium on Robot and Human Interactive Communication. https://ieeexplore.ieee.org/document/6926376

49.

Shin

N. M.

Kim

S. A.

(2007). What do robots have to do with student learning? Journal of Korean Association for Educational Information and Media, 13(3), 79–99.

50.

Shin

Lee

J.-K.

(2017). High school students’ perception of artificial intelligence: Focusing on conceptual understanding, emotion and risk perception. Korean Association for Learner-Centered Curriculum and Instruction, 17(21), 289–312. https://doi.org/10.22251/jlcci.2017.17.21.289

51.

Shin

Lee

J.-K.

(2018). Exploring elementary school students’ image of artificial intelligence. Journal of Elementary Science Education, 37(2), 126–146. https://doi.org/10.15267/keses.2018.37.2.126

52.

Steinbauer

Kandlhofer

Chklovski

Heintz

Koenig

(2021). A differentiated discussion about AI education K-12. Kunstliche Intelligenz, 35(2), 131–137. https://doi.org/10.1007/s13218-021-00724-8

53.

Steinmetz

Knappstein

Ajzen

Schmidt

Kabst

(2016). How effective are behavior change interventions based on the theory of planned behavior? Zeitschrift für Psychologie, 224(3), 216–233. https://doi.org/10.1027/2151-2604/a000255

54.

Svenningsson

Hultén

Hallström

(2018). Understanding attitude measurement: Exploring meaning and use of the PATT short questionnaire. International Journal of Technology and Design Education, 28(1), 67–83. https://doi.org/10.1007/s10798-016-9392-x

55.

Tavakol

Dennick

(2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd

56.

Touretzky

Gardner-McCune

Martin

Seehorn

(2019). Envisioning AI for K-12: What should every child know about AI? Proceedings of the AAAI Conference on Artificial Intelligence, Proceedings of the AAAI Conference on Artificial Intelligence, 33, 9795–9799. https://doi.org/10.1609/aaai.v33i01.33019795

57.

Watkins

M. W.

(2018). Exploratory factor analysis: A guide to best practice. Journal of Black Psychology, 44(3), 219–246. https://doi.org/10.1177/0095798418771807

58.

Wilson

F. R.

Pan

Schumsky

D. A.

(2012). Recalculation of the critical values for Lawshe’s content validity ratio. Measurement and Evaluation in Counseling and Development, 45(3), 197–210. https://doi.org/10.1177/0748175612440286

59.

Yoo

(2019). A study on AI education in graduate school through IPA. Journal of the Korean Association of Information Education, 23(6), 675–687. https://doi.org/10.14352/jkaie.2019.23.6.675

60.

K.-C.

Lin

K.-Y.

Han

F.-N.

Hsu

I. Y.

(2012). A model of junior high school students’ attitudes toward technology. International Journal of Technology and Design Education, 22(4), 423–436. https://doi.org/10.1007/s10798-011-9154-8

Development and Validation of a Scale Measuring Student Attitudes Toward Artificial Intelligence

Abstract

Keywords

Introduction

Theoretical Background

Attitude

Attitude Measurements

Materials and Methods

Research Procedure

Preparation of Test Questions

Expert Validity Test

Main Survey for Students

Exploratory Factor Analysis

Confirmatory Factor Analysis

Results

Discussion

Conclusions

Footnotes

Appendix

Acknowledgements

Author Contributions

Availability of Data and Materials

Ethical Approval

Informed Consent

Declaration of Conflicting Interests

Funding

ORCID iD

References