Sage Journals: Discover world-class research

Abstract

This study aims to explore the criteria and success factors for the application of Artificial Intelligence Generated Content (AIGC) in higher education, and guide its practice through the construction of a comprehensive system and framework. This study first identifies seven primary criteria, encompassing technical robustness, integration with existing systems, evidence-based practice, user acceptance and engagement, ethical considerations, collaborative ecosystems, and cultural and contextual sensitivity. These criteria are further refined into 19 subfactors. Utilizing the Decision-Making Trial and Evaluation Laboratory (DEMATEL) method for analysis, the results indicate that user acceptance and engagement occupy a central position in AIGC applications, emerging as the primary factor influencing successful implementation. Simultaneously, the establishment of a collaborative ecosystem is identified as a critical aspect. Additionally, factors such as technical robustness, integration with existing systems, and evidence-based practice not only directly impact user acceptance and engagement but also indirectly affect other elements like the collaborative ecosystem. In terms of specific key success factors, scalability and feedback mechanisms play a crucial role in AIGC implementation. Furthermore, partnerships demonstrate high prominence in higher education AIGC applications, highlighting the importance of building and maintaining strong collaborative relationships for successful implementation. This study provides significant insights into theories in educational technology and offers practical guidance for higher education institutions in their applications.

Plain language summary

Integrating generative AI technology into higher education practice

Keywords

AIGC higher education education digitization key success factors DEMATEL method

Introduction

Artificial Intelligence Generated Content (AIGC), encompassing various forms such as text, images, audio, and video, has ushered in unprecedented changes in higher education. In the current higher education environment, the application prospects of AIGC are widely optimistic. Its most compelling advantage lies in its ability to craft personalized learning experiences for students (Chai et al., 2024). Through deep learning and big data analytics, AI algorithms can precisely grasp each student’s learning style and preferences, thus generating tailored learning content. For instance, Alam (2023) and Huang et al. (2023) demonstrate the successful application of AI tutoring systems in higher education. These systems analyze student learning data to provide personalized feedback and resources, significantly enhancing learning effectiveness and engagement.

Beyond personalized learning, AIGC also facilitates the automated creation of educational content. Traditional content creation often demands considerable time and effort from teachers, a burden that AI algorithms can greatly alleviate. According to Chaudhry and Kazim (2022), AI-generated study guides are not only accurate and comprehensive, but also drastically reduce the time cost of producing learning materials for teachers. Even for the administrative management of the university, AIGC exhibits its efficiency and convenience as well. The daily administrative tasks in higher education institutions are tedious and repetitive, and the introduction of AI can automate these tasks, improving work efficiency (Vaarma & Li, 2024). For example, Igbokwe (2023) shows that implementing AI chatbots can shorten response times to student inquiries and enhance overall administrative efficiency.

However, despite the vast application prospects of AIGC in higher education, it faces several challenges. First, there are concerns about content quality and accuracy. Errors or inaccuracies in AI-generated content can undermine the student learning experience (Li et al., 2023). Second, while AIGC aids in personalized learning, there’s a risk of content being overly generic and lacking deep customization (Chen et al., 2024). Huang, Liu, Dong and Lu (2024) emphasize that educators need to address students’ unique needs and challenges when designing and implementing AIGC-based teaching strategies. Last, ethical and privacy issues are paramount considerations in AIGC’s application in higher education (Chen et al., 2024; Holmes & Porayska-Pomsta, 2022; Wang, 2024). Also, the overreliance on AI systems may lead to a lack of human supervision and accountability (Naseer et al., 2024). Chan and Hu (2023) reveal that many students express concerns about transparency and data privacy regarding AI-generated content in educational settings (Suh & Ahn, 2022). Also, Ivanov et al. (2024) emphasize that Teachers and students differ in their perceptions of the risks and weaknesses of AIGC

In recent years, researchers have explored successful pathways for AIGC application in higher education, proposing innovative ideas such as practical assignment resource development (Yiwen et al., 2023), self-efficacy enhancement (Wang et al., 2021), and smart education (Peters & Green, 2024). These contribute to the digital transformation of higher education. Nonetheless, research gaps remain exist, particularly in analyzing the key success factors of AIGC. Moreover, extant studies haven’t fully explored the balance between technology and education, or the optimal ways to integrate them. Therefore, this study aims to identify and construct a system of key success factors for AIGC applications in higher education. We theoretically explain the priorities and causal relationships among these factors, guiding the scientific operation of AIGC in the higher education sector. Specifically, we address the following research questions:

(1) How to construct a system of key success factors for AIGC application in higher education?

(2) How to balance the relationship between AIGC technology and education to achieve optimal integration?

(3) What are the importance and causal relationships among the key success factors?

To address these questions theoretically, we employ the Socio-Technical System (STS) theory as a framework, systematically analyzing the key success factors influencing the adoption, integration, and impact of AIGC technology in the educational environment. We then build a system encompassing factors like technological robustness, user acceptance, ethical considerations, integration with existing systems, collaborative ecosystems, evidence-based practices, and cultural sensitivity. Furthermore, we introduce the Decision-Making Trial and Evaluation Laboratory (DEMATEL) method as an intelligent system to consider the interdependencies among key success factors. The DEMATEL approach aims to solve complex and interrelated problems, determining causal relationships among these factors. It also depicts the inherent structural relationships of influencing factors and provides potential solutions for the successful application of AIGC in higher education practices.

The remainder of this paper is organized as follows: Section “Literature Review and Theoretical Framework” reviews relevant work and summarizes the theoretical framework. Section “Research Methodology” introduces the proposed methodology and its steps. Section “Data Collection and Analysis” details data collection and analysis. Section “Results and Discussion” presents our research results and discusses the implications; and finally, Section “Conclusions” concludes the study and outlines future research directions.

Literature Review and Theoretical Framework

Application of AIGC in Higher Education

The AIGC is gradually infiltrating every facet of higher education, from learning experiences to administrative processes and research activities, with increasingly significant impacts. First, AIGC demonstrates significant potential to enhance learning experiences (Chen et al., 2024; Huang et al., 2024). Leveraging AI systems to deeply analyze student data, it is now possible to generate customized content that better aligns with individual learning needs and styles (Yang et al., 2023; Shoaib et al., 2024). For instance, Alam (2023) explored how AI-driven tutoring systems improve the way students are educated and how they can be integrated into curricula and classrooms. Furthermore, Yang et al. (2023) found that AIGC aids teachers in relieving the burden of material preparation, thus enhancing their teaching innovation capabilities. Chen et al. (2024) also revealed that AI-assisted learning tools effectively enhance students’ comprehension and retention of complex topics.

In terms of administration, AIGC also exhibits high efficiency. Hannan and Liu (2023) pointed out that higher education institutions can utilize AI algorithms to rapidly generate reports, process enrollment data, and even engage in routine communication with students. For instance, Gill et al. (2024) studied the potential and challenges of ChatGPT in education sectors, discovering varying performances across disciplines such as finance, coding, and mathematics.

In research activities, AIGC provides powerful support to scholars. AI can rapidly scan vast databases, assist in literature reviews (Kacena et al., 2024), identify relevant studies, and summarize key findings, significantly accelerating the initial stages of research. However, as Chubb et al. (2022) noted, while AI aids in information gathering and other narrow tasks, it may also exacerbate bureaucracy and quantitative processes, potentially spreading negative aspects of academic culture.

Additionally, the application of AIGC in higher education faces a series of challenges and ethical considerations. Issues such as inaccuracy, bias, and over-reliance on technology and algorithms are particularly prominent (Kurtz et al., 2024). Disseminating erroneous or misleading information can adversely affect students’ academic growth. Rodway and Schepman (2023) found that students feel moderately comfortable with AI education applications but also show a decreasing trend in course satisfaction; thus, they advise higher education institutions to proceed with caution before making significant investments in AI education applications. Furthermore, ethical issues such as intellectual property rights, data privacy, and AI-induced biases cannot be ignored (Li et al., 2024). These issues must be addressed appropriately to ensure AI system transparency and accountability, maintaining trust and integrity in higher education.

Socio-technical Systems Theory and AIGC Application

Socio-technical Systems (STS) theory, originating in the mid-20th century, provides a comprehensive analytical framework for understanding the interactions between social and technical elements within organizations (Appelbaum, 1997; Sony & Naik, 2020; Trist & Bamforth, 1951). This theory emphasizes the need to optimize both social and technical elements in organizational design, rather than focusing solely on one, to achieve optimal performance (Hyer et al., 1999). Recently, with the rise of AIGC, this theory has demonstrated unique value in guiding AIGC applications (Nah et al., 2023).

For instance, in marketing, from a socio-technical perspective, marketers need to comprehensively assess the fit between AIGC and existing marketing strategies and team dynamics (Mariani et al., 2022). This involves consistency in brand communication, new models of team collaboration, and reshaping customer relationships. STS theory provides a guiding framework for balancing technological capabilities and social dynamic impacts. In content creation and entertainment industries, AIGC brings new productivity (Gao, 2023), yet it also triggers new discussions on the role of human creators, the reconstruction of creative processes, and ethical issues (Bendel, 2023; Wang et al., 2023). STS theory emphasizes seeking a balance in these transformations, maintaining and enhancing human creativity and uniqueness while leveraging AI technology (Ciriello et al., 2024; Sartori & Theodorou, 2022).

In the context of higher education, STS theory provides profound insights into understanding the interactions between AIGC and social system elements such as students, educators, and administrators (Nah et al., 2023; Xiang et al., 2023). The introduction and application of AIGC technology not only transform teaching methods and learning experiences but also affect the role positioning of educators and students, as well as the overall learning environment (Huang et al., 2024). However, this also requires educators to adjust teaching methods and existing educational models to ensure the effective integration of these technologies into the educational process (Wu & Liang, 2023; Xu et al., 2024).

Key Influencing Factors of AIGC Application in Higher Education

As mentioned above, STS theory in higher education provides a comprehensive perspective for understanding the interactions between social and technical factors within organizations. Applying STS theory enables us to gain a deeper understanding of the key influencing factors of generative AI applications in higher education. Below, we identify and analyze the critical success factors of AIGC applications in higher education from both technical and social aspects.

Key Success Factors from a Technical Perspective

From a technical standpoint, the robustness of technology is a primary consideration (Chen et al., 2014; Mell & Grance, 2011). Technical robustness manifests in the reliability, security, and scalability of AIGC systems, ensuring effective, accurate, and secure system operation under various conditions. Algorithm accuracy is a priority, ensuring that the algorithms used by AIGC systems provide accurate and reliable guidance and advice (Ouyang et al., 2022). In the context of higher education, algorithm accuracy is crucial for the success of AIGC applications as it directly impacts the quality of guidance and advice provided by the system (von Winckelmann, 2023). To ensure algorithm accuracy, meticulous design, and rigorous testing are necessary to address potential biases or discrepancies. Baker and Hawn (2022) proposed a framework to shift AI algorithms from unknown biases to known biases. Additionally, data security is a crucial factor, necessitating robust data security measures to protect sensitive student information stored in AIGC systems (Guo et al., 2023), adhering to relevant data protection regulations such as GDPR or FERPA (Koo et al., 2023). Designing systems for scalability is also essential to accommodate increasing user and data volumes without compromising performance and reliability (Du et al., 2023).

Moreover, integration with existing systems and processes is paramount (Haleem et al., 2022; Ni et al., 2024). This includes ensuring seamless integration of AIGC systems with existing educational platforms, student information systems, and counseling frameworks. Compatibility ensures seamless integration with other educational systems and processes (Fırat, 2023), while interoperability enables data and functional exchange with other educational technologies and tools (Krauss et al., 2023). Furthermore, the functionality of AIGC systems must align with the overall goals of institutions, counseling services, and academic programs to ensure meaningful outcomes and contributions to strategic priorities (Liu et al., 2023). Rigorous research and evaluation are necessary to assess the impact of AIGC applications on institutional efficiency and student success (Guo et al., 2023).

Furthermore, evidence-based practices play a vital role in AIGC applications (Al-Hammouri, 2024; Schwartz & Tilling, 2023). This involves conducting rigorous research and evaluations to assess the impact of AIGC applications on student outcomes, satisfaction levels, and academic performance (Slavin, 2020). Through research and evaluation, we can better understand the effectiveness and value of AIGC applications in higher education, enabling continuous improvement based on empirical data, thereby enhancing the design, functionality, and effectiveness of the system (Huang et al., 2024; Xu et al., 2023).

Key Success Factors from a Social Perspective

From the social perspective, user acceptance and participation are crucial considerations (Davis, 1989; Nielsen, 1993; Zhao et al., 2024). First, user interface design should be simple, clear, and user-friendly, enabling students, educators, and counselors to interact with the system easily and access required information (Li et al., 2024). Second, to enhance users’ understanding and confidence in using the system, adequate training and support are necessary (Qin et al., 2020). Training programs should be tailored to user groups’ needs and proficiency levels, providing comprehensive guidance, including system functionality, data interpretation, and troubleshooting techniques (Srinivasa et al., 2022). Additionally, establishing an effective feedback mechanism is crucial for collecting users’ opinions and suggestions, continuously improving the AIGC system based on their experiences and preferences (Fidan & Gencel, 2022; Perikos et al., 2017). Therefore, to enhance user acceptance of AIGC systems, attention should be paid to user interface design, training and support, and feedback mechanisms.

Furthermore, ethical considerations must be taken seriously (Caliskan et al., 2017; Deroncele-Acosta et al., 2024; Jobin et al., 2019;). Measures need to be implemented to mitigate algorithmic biases to ensure fairness and justice in the guidance and advice provided by AIGC systems (Yang et al., 2021). This includes technical means such as algorithmic audits, bias detection algorithms, and curated diverse datasets to reduce biases and promote algorithmic fairness (Hasan et al., 2022). Additionally, enhancing system transparency is crucial, with institutions providing stakeholders with clear explanations of how AIGC systems operate, the data used, and the basis for recommendations (Khowaja et al., 2024). Transparent communication fosters trust and accountability among stakeholders and enables users to better understand and accept AIGC applications (Fui-Hoon Nah et al., 2023). Last, strict data protection measures and compliance are essential to safeguard the privacy of students, teachers, and other relevant personnel (Guo et al., 2023).

Building a collaborative ecosystem is pivotal for the effective implementation and continuous improvement of AIGC applications in higher education (Ito et al., 2010; Wenger, 1998). Engaging stakeholders in the design, implementation, and evaluation of AIGC applications fosters ownership, commitment, and support for technological initiatives, ensuring AIGC solutions meet their needs and expectations (Liu et al., 2023; Wu & Hu, 2024). Strategic partnerships with industry stakeholders, research institutions, and community organizations enhance the effectiveness and sustainability of AIGC applications in higher education (Lian et al., 2024). Simultaneously, establishing communities of practice facilitates knowledge sharing, dissemination of best practices, and continuous improvement, fostering a vibrant ecosystem of AIGC practitioners and advocates (Cousin & Deepwell, 2005; Ryan, 2015). Therefore, a collaborative ecosystem involves promoting cooperation and communication among stakeholders, building partnerships with industry experts, research institutions, and other higher education organizations, and establishing communities of practice around AIGC applications.

Finally, considering different cultural backgrounds, values, and norms, customizing system functionalities to enhance cultural and contextual sensitivity is essential for ensuring successful system implementation (Gay, 2010; Hofstede, 2001). Culturally competent AI algorithms leverage natural language processing (NLP), sentiment analysis, and machine learning techniques to understand and adapt to the nuances of different cultural expressions, idioms, and communication styles (Tran, 2023). Tailoring AIGC applications to the specific needs, preferences, and priorities of different disciplines, departments, and institutional contexts improves their relevance and impact in higher education environments (Chen et al., 2024).

Construction of Success Factor System for AIGC Application in Higher Education

Based on the content in Section “Key Influencing Factors of AIGC Application in Higher Education”, we can outline a system of success factors for AIGC applications in higher education practices, as shown in Table 1. Correspondingly, Table 1 also lists the coding and literature sources for the standards and success factors.

Table 1.

Success Factors System of AIGC Applications in Higher Education.

Code	Success Factor	Description	Source
A	Technical Robustness	The reliability, security, and scalability of AIGC systems, involving ensuring effective, accurate, and secure system operation under various conditions.	Mell & Grance (2011); Che, Mao, & Liu (2014)
A1	Algorithm Accuracy	Ensuring the accuracy and reliability of algorithms used for guidance and consultation recommendations.	Ouyang et al. (2022); von Winckelmann (2023); Baker & Hawn (2022)
A2	Data Security	Implementing stringent measures to protect student data from breaches or unauthorized access.	Guo et al. (2023); Koo et al. (2023)
A3	Scalability	Designing systems that can handle increased user demands and data volumes without compromising performance.	Du et al. (2023)
B	Integration with Existing Systems and Processes	Ensuring seamless interoperability between AIGC systems and other platforms and systems within educational institutions (e.g., Learning Management Systems, Student Information Systems).	Haleem et al. (2022); Ni et al. (2024)
B1	Compatibility	Ensuring seamless integration with existing educational platforms and systems to streamline workflows.	Yu & Gong (2024); Fırat (2023)
B2	Interoperability	Achieving interoperability with third-party applications to enhance functionality and data exchange capabilities.	Yu & Gong (2024); Krauss et al. (2023)
B3	Alignment with Education Goals	Aligning AIGC applications with institutional goals and strategic priorities for maximum impact and relevance.	Liu et al. (2023); Guo et al. (2023)
C	Evidence-Based Practices	Utilizing empirical evidence and research to guide decision-making and continuous improvement of AIGC systems.	Al-Hammouri (2024); Schwartz & Tilling (2023)
C1	Research and Evaluation	Conducting rigorous research to assess the impact and effectiveness of AIGC applications on student outcomes and experiences.	Slavin (2020)
C2	Continuous Improvement	Leveraging evidence-based insights to guide the continuous improvement and optimization of systems, ensuring sustainable progress.	Xu et al. (2023); Huang et al. (2024)
D	User Acceptance and Participation	The willingness of users (students, educators, counselors) to adopt and effectively use AIGC systems.	Davis (1989); Nielsen (1993)
D1	User Interface Design	Developing intuitive interfaces to promote ease of use and accessibility for various user groups.	Li et al. (2024)
D2	Training and Support	Providing comprehensive training and ongoing support to enhance users’ capabilities and confidence.	Qin et al. (2020); Srinivasa et al. (2022)
D3	Feedback Mechanism	Establishing channels for user feedback to facilitate continuous improvement of the system.	Perikos et al. (2017); Fidan & Gencel (2022)
E	Ethical Considerations	Addressing ethical and moral issues related to bias, privacy, transparency, and fairness.	Caliskan, Bryson, & Narayanan (2017); Jobin, Ienca, & Vayena (2019)
E1	Mitigating Bias	Addressing algorithmic bias issues to ensure fairness and justice in guidance and consultation outcomes.	Yang, et al. (2021); Hasan et al. (2022)
E2	Transparency	Promoting transparency in system operations, data usage, and decision-making processes.	Khowaja et al. (2024); Fui-Hoon Nah et al. (2023)
E3	Privacy Protection	Protecting student privacy through rigorous data protection measures and compliance.	Guo et al. (2023)
F	Collaborative Ecosystem	Facilitating collaboration and communication among stakeholders (students, educators, counselors, administrators, developers) within AIGC systems.	Wenger (1998); Ito et al. (2010)
F1	Stakeholder Engagement	Collaborating with stakeholders for shared decision-making, fostering ownership and support for AIGC initiatives.	Liu et al. (2023); Wu & Hu (2024)
F2	Partnerships	Collaborating with strategic partners to leverage external expertise and resources for AIGC development and implementation.	Lian, et al. (2024)
F3	Community Building	Establishing communities of practice to facilitate knowledge sharing, networking, and dissemination of best practices.	Cousin & Deepwell (2005); Ryan (2015)
G	Cultural and Contextual Sensitivity	Understanding and respecting users’ cultural diversity and adapting AIGC systems to meet their specific cultural needs and preferences.	Hofstede (2001); Gay (2010)
G1	Cultural Competence	Designing AIGC systems that respect and respond to students’ cultural diversity, including culturally relevant features.	Tran (2023); Zhai & Wibowo (2023)
G2	Contextual Adaptability	Customizing AIGC applications for different academic disciplines and institutional contexts to enhance their relevance and impact.	Baber et al. (2023); Chen et al. (2024)

Table 1 presents a standards system for successful applications in the field of educational technology, encompassing seven main standards: “Technical Robustness (A),”“Integration with Existing Systems and Processes (B),”“Evidence-Based Practices (C),”“User Acceptance and Participation (D),”“Ethical Considerations (E),”“Collaborative Ecosystem (F),” and “Cultural and Contextual Sensitivity (G).”

Specifically, (1). “Technical Robustness (A)” primarily covers technical performance standards, including “Algorithm Accuracy (A1),”“Data Security (A2),” and “Scalability (A3).” (2). “Integration with Existing Systems and Processes (B)” focuses on standards for integration with educational systems, such as “Compatibility (B1),”“Interoperability (B2),” and “Alignment with Education Goals (B3).” (3). “Evidence-Based Practices (C)” mainly covers teaching practice standards, including “Research and Evaluation (C1)” and “Continuous Improvement (C2).” (4). “User Acceptance and Participation (D)” addresses user experience standards, such as “User Interface Design (D1),”“Training and Support (D2),” and “Feedback Mechanism (D3).” (5). “Ethical Considerations (E)” primarily covers ethical factors in educational technology applications, including “Mitigating Bias (E1),”“Transparency (E2),” and “Privacy Protection (E3).” (6). “Collaborative Ecosystem (F)” encompasses collaboration standards related to educational technology, such as “Stakeholder Engagement (F1),”“Partnerships (F2),” and “Community Building (F3).” (7). “Cultural and Contextual Sensitivity (G)” focuses on application standards in different cultural and contextual settings, including “Cultural Competence (G1)” and “Contextual Adaptability (G2).”

Research Methodology

In this section, we will employ the DEMATEL technique to examine the critical success factors influencing the application of AIGC (Artificial Intelligence Generated Content) in higher education. DEMATEL is a robust method for analyzing complex causal relationships among variables by integrating matrices and graphs (Hsu et al., 2013; Jeng & Tzeng, 2012). Although the study provides valuable insights into AIGC in higher education, it is important to establish a stronger connection to social science research methodologies. While DEMATEL is originally designed for engineering system issues, it has been widely applied in diverse fields, including education. However, it is less common in education research than methods like SEM or regression. The choice of DEMATEL over other methods is due to its ability to handle problems involving multiple objectives and attributes, which is particularly suitable for this study as it involves numerous interrelated factors. Despite its advantages, DEMATEL has certain limitations that need to be discussed.

One limitation of DEMATEL is its reliance on expert judgment, which may introduce potential biases. The analysis is heavily dependent on the assessments provided by scholars and practitioners in the field, and different experts may have varying opinions and perspectives. This can affect the accuracy and reliability of the results. Additionally, the use of a five-point scale to assess the degree of influence among factors may not capture the nuances and complexities of the relationships. Furthermore, DEMATEL may not be as well-known or widely understood in the field of education research compared to other methods like SEM or regression. This may limit the accessibility and interpretability of the results for some researchers and policymakers.

However, given these limitations, DEMATEL remains a valuable tool for this study as it allows us to analyze the importance and causal relationships of the critical success factors influencing the integration of AIGC in higher education. By identifying the causal relationships among these factors, DEMATEL provides a structured framework for understanding which factors are most influential (cause group) and which are more dependent on others (effect group). This analysis not only highlights the key drivers of successful AIGC integration but also offers actionable insights for policymakers and educators to prioritize their efforts effectively.

DEMATEL Analysis Procedures

Step 1: Establishing the Direct Relation Matrix

Based on the system of critical influencing factors presented in Table 1, scholars and practitioners in the field are invited to assess the direct impact of each factor on others and subsequently establish a direct relation matrix (DRM). Specifically, experts compare the degree of influence among factors on a five-point scale: 0 (no influence), 1 (very little influence), 2 (little influence), 3 (significant influence), and 4 (very significant influence). An initial n × n matrix A is obtained, where a_ij represents the degree of influence of criterion i on criterion j.

Step 2: Normalizing the DRM

Based on the DRM A, the normalized DRM N is calculated as:

N = k \cdot A

where

k = \frac{1}{\max (\sum_{j = 1}^{n} a_{ij})}, i, j = 1, 2, \dots, n

This normalization ensures that the sum of each row in the matrix does not exceed 1, allowing for consistent comparison across factors (Hsu et al., 2013).

Step 3: Calculating the Total Relation Matrix

After obtaining the normalized DRM N, the total relation matrix (TRM) T can be calculated as:

T = N {(I - N)}^{- 1}

where ${(I - N)}^{- 1}$ represents the inverse of the matrix obtained by subtracting the normalized matrix N from the identity matrix I. This operation captures the direct and indirect influences among factors, providing a comprehensive view of their interrelationships.

Step 4: Drawing the DEMATEL Relationship Diagram

Given the TRM $T = {(t_{ij})}_{n \times n}$ , the row sum and column sum are represented by vectors D and C respectively, where

D = {(\sum_{j = 1}^{n} t_{ij})}_{n \times 1} = {(t_{i \cdot})}_{1 \times n}

and

C = {(\sum_{i = 1}^{n} t_{ij})}_{1 \times n} = {(t_{\cdot j})}_{n \times 1}

where vector D represents the sum of each row in the TRM T, indicating the overall influence exerted by each factor on others. Vector C represents the sum of each column in the TRM T, indicating the overall influence received by each factor from others.

By summing D and C, the horizontal axis vector (D+C), defined as “Prominence”, is obtained, which indicates the degree of importance of each criterion. Similarly, subtracting C from D yields the vertical axis (D-C), defined as “Net effect”, which classifies factors into cause-and-effect groups. Typically, if (D-C) is negative, the criterion belongs to the effect group; otherwise, it belongs to the cause group. Mapping the (D+C, D-C) dataset thus results in a causal diagram, providing valuable insights for decision-making (Fu et al., 2012; Wang & Zhao, 2023).

Step 5: Mapping the Relationships Among Factors in the Diagram

The final step involves plotting a graphical representation of each factor’s prominence and net effect values on a two-dimensional axis. The x-axis represents prominence, and the y-axis represents the net effect. Directional arrows capture the interrelationships among factors. To clarify this visualization, a threshold is defined, setting a cutoff point for relationships between factors. Arrows are depicted in the final DEMATEL diagram for values in the TRM exceeding this threshold. The threshold θ is calculated as:

θ = mean (T) + S D_{T}

where mean (T) is the mean of all $t_{ij}$ values in the TRM, and $S D_{T}$ is the standard deviation of $t_{ij}$ values (Fu et al., 2012; Wang & Zhao, 2023). Values of $t_{ij}$ greater than θ indicates a significant relationship between two factors, corresponding to an arrow in the DEMATEL diagram. Values above the threshold in the relation matrix are highlighted.

Alignment of DEMATEL with the Purpose of the Paper

The DEMATEL method aligns closely with the purpose of this paper, which is to explore and analyze the critical success factors for integrating Generative AI in higher education. By identifying the causal relationships among these factors, DEMATEL provides a structured framework for understanding which factors are most influential (cause group) and which are more dependent on others (effect group). This analysis not only highlights the key drivers of successful AIGC integration but also offers actionable insights for policymakers and educators to prioritize their efforts effectively. Thus, DEMATEL serves as a powerful tool to bridge the gap between theoretical exploration and practical implementation in the context of Generative AI in higher education.

Data Collection and Analysis

This study employed a DEMATEL analysis to examine the causal relationships between various strategic or tactical factors related to AIGC in higher education. The data collection and analysis processes can be summarized in the following Figure 1.

Figure 1.

Data collection and analysis processes.

Expert selecting

To conduct the DEMATEL analysis, we assembled a team of experts from higher education and artificial intelligence fields. A total of 15 experts, comprising eight from higher education and seven from industry, participated in the study. The rationale for selecting 15 experts was to balance the depth of expertise with manageable data collection efforts. This number of experts is consistent with literature and precedents where similar-sized expert panels have been deemed sufficient for achieving reliable and valid results in comparable contexts.

The experts from higher education were full-time professionals active in university management or teaching, with an average work experience of 8.5 years. Industry practitioners primarily came from software development, specializing in AI system design, development, or testing, with an average work experience of 6.9 years. All experts had an acceptable level of knowledge in AIGC and higher education teaching, research, or management. Table 2 provides information and profile of the expert team.

Table 2.

Expert Information and Profile.

No.	Domain Type	Department	Sub-field	Position	Gender	Expertise	Work Experience (years)
1	Education	Business School	Teacher	Associate Professor	Male	Operations Management	15
2	Education	School of Foreign Languages	Teacher	Lecturer	Female	English Education	8
3	Education	College of Education	Teacher	Lecturer	Female	Mathematics Teaching	5
4	Education	Business School	Teacher	Lecturer	Male	Economics	5
5	Education	School of Science	Teacher	Associate Professor	Female	Physics Education	12
6	Education	Administrative Department	Education Management	Deputy Director	Male	Educational Technology	16
7	Education	School of Tourism	Teacher	Lecturer	Female	Psychological Counseling	3
8	Education	Business School	Teacher	Associate Professor	Male	Marketing	4
9	Industry	Algorithm Design	Engineer	Supervisor	Male	Programming	10
10	Industry	Front-end Development	Engineer	Junior	Male	Programming	3
11	Industry	Back-end Development	Engineer	Intermediate	Male	Database Management	6
12	Industry	Testing Department	Test Engineer	Junior	Female	Software Testing	2
13	Industry	Project Management	Project Manager	Intermediate	Male	Project Management	7
14	Industry	System Architecture	Architect	Senior	Male	System Design	12
15	Industry	Network Security	Security Engineer	Intermediate	Male	Network Security	8

Data collection instrument

A comprehensive data collection instrument was designed specifically for the context of Generative AI in higher education. This instrument included questions aimed at evaluating the mutual influences between various factors related to AIGC. The instrument was adapted from standard DEMATEL analysis questionnaires, with additional questions tailored to our unique study context. Prior to completing the questionnaire, experts underwent training on AIGC background knowledge and the specific context of the study to ensure a clear understanding of the concepts and terminology used.

Validation and Digitization

A validation procedure was conducted to ensure the reliability and validity of the data collection instrument. This involved pilot testing with a small group of experts, analyzing the results, and making necessary adjustments. Following data collection, the qualitative evaluations and expert opinions were digitized into quantitative data suitable for DEMATEL analysis. Specifically, responses were recorded and tabulated to generate direct relation matrices (DRMs). Using arithmetic methods, individual DRMs were combined to obtain consensus DRMs for the entire team.

Tables Summarizing Data Transaction

After calculation, Table 3 presents the TRM for first-level factors, revealing the interconnectedness between success factors. Table 4 details the TRM of the subfactors, highlighting specific interactions. Finally, Table 5 summarizes the prominence and net effect values for both success factors and subfactors, indicating their relative importance and influence on the system.

Table 3.

Total Relation Matrix for First-Level Factors.

	A	B	C	D	E	F	G
A	1.841	2.054	2.077	2.566	1.657	2.264	1.639
B	1.938	1.803	2.035	2.495	1.578	2.207	1.561
C	2.071	2.045	2.016	2.647	1.719	2.309	1.700
D	1.930	1.946	2.120	2.363	1.653	2.226	1.646
E	1.461	1.463	1.604	2.032	1.216	1.677	1.368
F	2.035	2.070	2.099	2.598	1.726	2.130	1.695
G	1.739	1.776	1.961	2.353	1.625	2.022	1.430

Note. The Total Relation Matrix reveals the interconnectedness between success factors, providing insights into their mutual influences.

Table 4.

The Total Relation Matrix of the Subfactors.

	A1	A2	A3	B1	B2	B3	C1	C2	D1	D2	D3	E1	E2	E3	F1	F2	F3	G1	G2
A1	0.182	0.260	0.283	0.298	0.287	0.329	0.305	0.321	0.252	0.267	0.291	0.262	0.281	0.226	0.320	0.334	0.303	0.235	0.334
A2	0.192	0.198	0.257	0.271	0.281	0.284	0.258	0.298	0.228	0.246	0.257	0.249	0.281	0.258	0.311	0.333	0.290	0.231	0.294
A3	0.231	0.272	0.261	0.348	0.345	0.346	0.310	0.352	0.286	0.289	0.305	0.285	0.300	0.245	0.351	0.383	0.352	0.265	0.352
B1	0.236	0.293	0.350	0.296	0.365	0.375	0.334	0.381	0.307	0.315	0.320	0.312	0.314	0.274	0.373	0.402	0.382	0.291	0.383
B2	0.246	0.303	0.339	0.365	0.295	0.374	0.336	0.371	0.309	0.308	0.335	0.319	0.330	0.277	0.384	0.405	0.389	0.289	0.382
B3	0.262	0.269	0.305	0.346	0.348	0.310	0.319	0.366	0.301	0.309	0.335	0.302	0.303	0.259	0.378	0.393	0.370	0.303	0.376
C1	0.248	0.250	0.281	0.296	0.293	0.341	0.246	0.321	0.250	0.270	0.291	0.266	0.276	0.233	0.337	0.346	0.324	0.252	0.321
C2	0.271	0.297	0.333	0.345	0.343	0.376	0.322	0.319	0.302	0.319	0.338	0.304	0.323	0.274	0.377	0.397	0.377	0.300	0.392
D1	0.203	0.236	0.283	0.293	0.295	0.296	0.261	0.302	0.203	0.255	0.269	0.258	0.255	0.223	0.295	0.304	0.297	0.232	0.310
D2	0.207	0.240	0.280	0.303	0.310	0.323	0.290	0.333	0.254	0.228	0.285	0.283	0.267	0.236	0.336	0.345	0.316	0.256	0.329
D3	0.261	0.302	0.343	0.357	0.362	0.393	0.355	0.404	0.311	0.339	0.300	0.359	0.348	0.297	0.397	0.417	0.401	0.315	0.407
E1	0.188	0.221	0.246	0.268	0.265	0.296	0.274	0.302	0.224	0.258	0.282	0.229	0.295	0.241	0.328	0.341	0.316	0.260	0.334
E2	0.220	0.265	0.302	0.321	0.314	0.325	0.308	0.326	0.269	0.264	0.294	0.315	0.253	0.262	0.357	0.371	0.341	0.273	0.354
E3	0.200	0.264	0.258	0.272	0.269	0.304	0.278	0.305	0.236	0.248	0.268	0.281	0.296	0.200	0.333	0.345	0.324	0.254	0.320
F1	0.227	0.262	0.299	0.310	0.290	0.340	0.306	0.342	0.262	0.275	0.310	0.313	0.301	0.260	0.301	0.387	0.360	0.280	0.357
F2	0.231	0.273	0.297	0.317	0.310	0.351	0.309	0.349	0.268	0.290	0.316	0.315	0.316	0.274	0.372	0.323	0.363	0.290	0.365
F3	0.228	0.273	0.302	0.318	0.306	0.348	0.318	0.354	0.273	0.295	0.317	0.324	0.312	0.266	0.364	0.378	0.305	0.295	0.370
G1	0.193	0.224	0.244	0.262	0.255	0.290	0.273	0.300	0.237	0.257	0.264	0.280	0.250	0.223	0.317	0.324	0.318	0.205	0.327
G2	0.215	0.239	0.288	0.302	0.300	0.330	0.289	0.323	0.257	0.264	0.284	0.268	0.260	0.230	0.325	0.339	0.336	0.259	0.282

Note. The detailed subfactor matrix highlights the specific interactions between subfactors, contributing to a nuanced understanding of their influences.

Table 5.

Prominence and Net Effect Values of Success Factors and Subfactors.

Type	Code	D_i	C_i	Prominence (M_i)	Net Effect (R_i)
Success factors	A	14.098	13.015	27.113	1.083
	B	13.618	13.157	26.776	0.461
	C	14.505	13.913	28.418	0.592
	D	13.884	17.052	30.936	−3.169
	E	10.821	11.175	21.996	−0.353
	F	14.352	14.835	29.187	−0.483
	G	12.906	11.038	23.945	1.868
Subfactors	A1	5.372	4.240	9.612	1.131
	A2	5.017	4.940	9.957	0.077
	A3	5.876	5.549	11.424	0.327
	B1	6.302	5.887	12.190	0.415
	B2	6.358	5.833	12.190	0.525
	B3	6.154	6.330	12.484	−0.177
	C1	5.442	5.691	11.133	−0.249
	C2	6.309	6.367	12.676	−0.059
	D1	5.071	5.031	10.102	0.040
	D2	5.420	5.297	10.717	0.123
	D3	6.668	5.662	12.330	1.006
	E1	5.168	5.523	10.692	−0.355
	E2	5.736	5.561	11.297	0.175
	E3	5.254	4.758	10.012	0.496
	F1	5.781	6.556	12.337	−0.774
	F2	5.929	6.868	12.797	−0.939
	F3	5.946	6.464	12.410	−0.519
	G1	5.041	5.087	10.127	−0.046
	G2	5.390	6.590	11.980	−1.199

Note. The prominence (M_i) and net effect (R_i) values reveal the relative importance and influence of each success factor and subfactor. Factors with higher M_i values are considered more critical in the overall system, while R_i values indicate whether a factor has a positive or negative net influence on the system.

Results and Discussion

The DEMATEL methodology generated several relational diagrams, with the x-axis representing prominence values and the y-axis representing net effect values. Each factor on the graph corresponds to a prominence (x) and net effect (y) value, with arrows indicating the direction of influence between the two factors. It is important to note that only significant relationships are labeled. Subsequently, we analyzed the relationships between each success factor and subfactor.

Analysis of Relationships Between Success Factors

Figure 2 illustrates the relationships between success factors. Connecting arrows represent relationships between factors that meet a threshold. In terms of prominence, user acceptance and participation (D) have the highest prominence, followed by the collaborative ecosystem (F), while cultural and contextual sensitivity (G) has the lowest. This suggests that the expert panel considers user acceptance and engagement as the most critical when applying AIGC in higher education. Several studies have highlighted the significance of user acceptance and participation as key factors for the successful implementation and sustained use of technology (e.g., Abusamhadana et al., 2019; Cheung & Vogel, 2013). Our findings align with this and emphasize the primacy of user acceptance and participation in the application of AIGC in higher education.

Figure 2.

Relationship diagram of success factor.

Moreover, the construction of a collaborative ecosystem is also crucial. Existing research shows that multi-stakeholder collaboration and ecosystem support are essential for the success of innovative technologies (Adner, 2006). The high prominence of the collaborative ecosystem in this study reflects this, indicating that establishing and maintaining a robust collaborative network is equally important in the application of AIGC. The low prominence of cultural and contextual sensitivity may suggest that in some technology implementation scenarios, cultural and contextual factors are not the primary considerations. However, this should be interpreted with caution, as Fellows and Liu (2016) pointed out that sensitivity to culture and context often has a significant impact on the success of cross-cultural or global projects. This may imply that in specific higher education environments, the application of AIGC may rely more on technology acceptance and collaboration, with cultural and contextual factors playing a relatively minor role.

From a causal perspective, user acceptance and participation (D) are directly influenced by all other factors, aligning with the core tenets of the Technology Acceptance Model (TAM) (Granić & Marangunić, 2019) and the extended unified theory of acceptance and use of technology (UTAUT2) (Suhail et al., 2024). Both TAM and UTAUT2 posits that perceived usefulness and perceived ease of use are key determinants of user acceptance of new technologies. This study extends this view, showing that user acceptance and participation are not only influenced by technological characteristics but also by multiple external factors, such as technological robustness and integration with existing systems and processes. This provides a more comprehensive perspective on understanding user behavior.

Furthermore, the collaborative ecosystem (F) is influenced by technological robustness (A) and evidence-based practices (C). This finding resonates with certain elements of the Information Systems Success Model (D&M Model), which emphasizes the impact of information quality, system quality, and service quality on the success of information systems (Petter et al., 2008). Technological robustness can be seen as an aspect of system quality, while evidence-based practices are closely related to information quality and service quality. By linking these factors to the collaborative ecosystem, this finding offers a new perspective on understanding the interactive relationships in complex systems.

Finally, technological robustness (A), integration with existing systems and processes (B), evidence-based practices (C), and cultural and contextual sensitivity (G) only influence other factors but are not influenced by them. This conclusion challenges certain assumptions of existing theoretical frameworks to some extent. For instance, traditional technology acceptance models tend to focus solely on users’ direct perceptions of technology, whereas this conclusion emphasizes the indirect influence of external factors on technology acceptance and usage. Additionally, by introducing the factor of cultural and contextual sensitivity, this conclusion broadens our understanding of the factors influencing technology acceptance and usage.

Analysis of Relationships Between Subfactors

Figure 3 shows the relationships between subfactors. First, in terms of prominence, this study found that partnerships (F2) have the highest prominence, echoing the importance of interdisciplinary and cross-sectoral collaboration emphasized in existing research. However, this study further highlights that building strong partnerships is a top priority in the application of AIGC in higher education, providing new focus points for higher education institutions in advancing AIGC. In contrast, existing research focuses more on technological integration and optimization, such as Huang et al. (2024) pointing out that technological integration of AIGC is crucial for enhancing education quality. However, our analysis reveals that the prominence of algorithmic accuracy (A1) is relatively low, indicating that in the context of increasingly mature technology, pure algorithmic improvements are no longer the core issue.

Figure 3.

Relationship diagram of subfactors.

Regarding causal relationships, the figure highlights the crucial roles of scalability (A3) and feedback mechanisms (D3) in the implementation of AIGC in higher education. This finding aligns with existing research on the importance of system scalability and user feedback mechanisms but clarifies their dominant positions in influencing other factors through specific data analysis. For instance, Han and Anderson (2022) mentioned the importance of feedback mechanisms in enhancing user engagement and satisfaction; however, our causal analysis further confirms the core influence of feedback mechanisms (D3) among multiple factors.

Moreover, the analysis reveals interactions between subfactors within integration with existing systems and processes (B) and the collaborative ecosystem (F). Specifically, compatibility (B1) and interoperability (B2) influence Alignment with Education Goals (B3), while stakeholder engagement (F1) and community building (F3) contribute to partnerships (F2). These findings provide new insights for higher education institutions on optimizing resource allocation and enhancing system integration efficiency in AIGC implementation.

Conclusions

This study delves into the standards and success factors for the application of AIGC in higher education, establishing a comprehensive system and framework to guide its practice. Specifically, we identified seven key standards, including technical robustness, integration with existing systems, evidence-based practices, user acceptance and participation, ethical considerations, collaborative ecosystems, and cultural and contextual sensitivity, which were further subdivided into 19 specific subfactors.

Summary of the Main Findings

Through DEMATEL analysis, we found that user acceptance and participation occupy a central position in AIGC applications, serving as the primary factor influencing successful implementation. To illustrate this point, real-world case studies of successful AIGC implementations in universities would greatly enhance the applicability of our findings. For instance, analyzing how certain universities have effectively integrated AIGC tools into their curricula, and the strategies they employed to garner user acceptance, could provide practical insights for others. Similarly, the construction of a collaborative ecosystem was identified as a crucial link, highlighting the importance of multi-party collaboration and ecosystem support. Regarding partnerships, while our study emphasizes their prominence in AIGC adoption, it is crucial to elaborate on how universities can effectively build and sustain these partnerships. Strategies such as establishing clear communication channels, setting shared goals, and fostering a culture of collaboration among stakeholders can be instrumental in achieving this. While cultural and contextual sensitivity may not be primary considerations in certain scenarios, they can still play a significant role in specific educational environments.

Furthermore, success factors such as technical robustness, integration with existing systems, and evidence-based practices not only directly influence user acceptance and participation but also indirectly impact other factors like the collaborative ecosystem. Notably, scalability and feedback mechanisms play a crucial role in AIGC implementation, emphasizing the importance of system performance and user experience. Finally, partnerships exhibited a high prominence in AIGC applications in higher education, indicating that building and maintaining strong relationships is essential for successful implementation (Fu & Ji, 2024). The interaction between factors such as compatibility, interoperability, and alignment with institutional goals also provides new directions for optimizing resource allocation and system integration efficiency.

Theoretical Implications

The findings of this study carry several significant theoretical implications. Firstly, the study emphasizes the pivotal role of user acceptance and participation in the successful deployment of AIGC in educational settings. This aligns with and further substantiates previous research on technology acceptance, underscoring its essential function in complex educational environments (Davis, 1989; Venkatesh et al., 2012). Notably, our findings confirm the critical role of user acceptance identified in these earlier studies, with no notable contradictions. Consequently, educational technology researchers and practitioners must prioritize enhancing user acceptance and participation when promoting and applying AIGC. Strategies such as optimizing user interface design, offering adequate training and support, and establishing effective feedback mechanisms can be employed to achieve this (Al-Nuaimi & Al-Emran, 2021; Farhan et al., 2019).

Second, the study underscores the importance of collaborative ecosystems in AIGC applications, highlighting that constructing and sustaining a robust collaborative network is equally vital. This finding supplements existing literature on educational technology ecosystems, reinforcing the necessity of multi-party collaboration in technological innovation and promotion (Becker et al., 2018; Siemens, 2013). While our study echoes the importance of collaborative ecosystems highlighted in these prior works, it extends the discussion by emphasizing the specific role of partnerships in AIGC adoption. To advance AIGC, higher education institutions should consider forging cross-disciplinary and cross-sector partnerships to collectively drive technological development and application (Eyal & Yarm, 2018).

Last, the study adopts the DEMATEL method to elucidate the intricate relationships among various factors, particularly the crucial roles of scalability and feedback mechanisms in AIGC implementation. This approach offers a novel perspective on comprehending interactions within complex systems (Tzeng & Huang, 2011). For higher education institutions, this has considerable guiding importance in optimizing resource allocation and enhancing system integration efficiency during AIGC implementation (Dillenbourg, 2008).

Practical Implications

The study offers practical insights for integrating AIGC into higher education. First, it proposes a set of standards and success factors to guide AIGC implementation in this sector, which include seven main criteria: technical robustness, integration with existing systems, evidence-based practices, user acceptance and participation, ethical considerations, collaborative ecosystems, and sensitivity to cultural and contextual factors. These guidelines provide higher education institutions with a comprehensive framework for AIGC application. For instance, at Georgia Tech, the university introduced an AI-driven virtual teaching assistant named “Jill Watson” in its online courses. By involving students in a transparent feedback loop and demonstrating how the AI could enhance learning experiences, Georgia Tech achieved high levels of student satisfaction and engagement (Goel & Polepeddi, 2018). Similarly, at the University of Edinburgh, the institution successfully integrated AIGC tools into its digital learning environment by providing training sessions, clear communication about the benefits, and involving staff in the customization of the system (Breines & Gallagher, 2023). These examples emphasize that co-designing with end-users, transparent communication, and ongoing support are key strategies for enhancing user acceptance (Venkatesh et al., 2003).

Second, the prominence of user acceptance and participation reinforces the idea that individuals’ willingness to adopt and engage with AIGC is critical for its success in higher education. This finding aligns with extensive research on technology adoption, particularly the Technology Acceptance Model (TAM), which links perceived usefulness and ease of use with adoption behavior (Davis, 1989). For example, the University of California, Berkeley, implemented a series of “AI sandbox” workshops where faculty members could explore AI tools in a low-risk environment, fostering confidence and reducing resistance (Brown et al., 2020). Such approaches reflect the broader trend in educational technology adoption that emphasizes hands-on experimentation and support to mitigate skepticism (Rogers et al., 2014).

Finally, the study underscores the essential role of collaborative ecosystems in AIGC adoption. The development and sustainability of partnerships between universities, industry, and government play a critical role in ensuring the effective deployment of AIGC technologies. A prominent example is the collaboration between the Massachusetts Institute of Technology (MIT) and IBM, which established the MIT-IBM Watson AI Lab in 2017. This partnership has focused on advancing AI research, including generative AI, while addressing ethical and societal implications (Lopez, 2017). Additionally, the European Union’s “AI4EU” initiative, launched under Horizon 2020, brought together over 80 organizations, including universities, research centers, and industry leaders, to co-develop AIGC solutions while tackling challenges such as scalability, data privacy, and regulatory compliance (Troumpoukis et al., 2024). To sustain such partnerships, universities should establish regular stakeholder forums, such as the Stanford Institute for Human-Centered Artificial Intelligence (HAI) annual conference, which fosters dialogue between academia, industry, and policymakers on AI governance and innovation (Yang et al., 2021). By fostering such partnerships, higher education institutions can better align AIGC adoption with broader institutional goals and societal needs.

Limitations and Future Work

Despite the achievements of this study in exploring the application of AIGC in higher education, there are still some limitations. First, the key success factor system was primarily constructed based on theoretical analysis and literature review, lacking empirical data support. Future research could validate and expand upon the conclusions of this study by collecting and analyzing actual case data. Second, this study mainly focused on general key success factors and did not conduct in-depth discussions on different types of higher education institutions or specific application scenarios. Future research could conduct customized studies on different types of higher education institutions or specific application scenarios to provide more specific and practical guidance.

Footnotes

Acknowledgements

Thanks to the editors and reviewers for their insightful comments.

ORCID iDs

Junbin Wang

Chuanbo Zhang

Ethical Considerations

The studies involving human participants were reviewed and approved by the School of Business, Changshu Institute of Technology.

Informed consent

Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by The Humanities and Social Science Fund of Ministry of Education of China [24YJC630202] and the Project sponsored by the Higher Education Association of Jiangsu Province under Grant [2024CXJG043].

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The datasets generated and/or analyzed during the above study are available from the corresponding author on reasonable request.

References

Abusamhadana

G. A.

Elias

N. F.

Mukhtar

M. U. R. I. A. T. I.

Asma’Mokhtar

(2019). User engagement model in information systems development. Journal of Theoretical and Applied Information Technology, 97(11), 2908–2930.

Adner

(2006). Match your innovation strategy to your innovation ecosystem. Harvard Business Review, 84(4), 98.

Alam

(2023). Harnessing the power of AI to create intelligent tutoring systems for enhanced classroom experience and improved learning outcomes. In G. Rajakumar, K. L. Du, & Á. Rocha (Eds.), Intelligent communication technologies and virtual mobile networks (pp. 571–591). Springer Nature Singapore.

Al-Hammouri

M. M.

(2024). Promoting evidence-based practice in nursing education: the good behavior game as a pedagogical tool for student engagement. Teaching and Learning in Nursing. 19(3), 545–549. https://doi.org/10.1016/j.teln.2024.03.011

Al-Nuaimi

M. N.

Al-Emran

(2021). Learning management systems and technology acceptance models: A systematic review. Education and Information Technologies, 26(5), 5499–5533.

Appelbaum

S. H.

(1997). Socio-technical systems theory: an intervention strategy for organizational development. Management Decision, 35(6), 452–463.

Baber

Nair

Gupta

Gurjar

(2023). The beginning of ChatGPT–a systematic and bibliometric review of the literature. Information and Learning Sciences, 125(7/8), 587–614. https://doi.org/10.1108/ILS-04-2023-0035.

Baker

R. S.

Hawn

(2022). Algorithmic bias in education. International Journal of Artificial Intelligence in Education, 32, 1052–1092.

Becker

S. A.

Brown

Dahlstrom

Davis

DePaul

Diaz

Pomerantz

(2018). NMC horizon report: 2018 higher education edition. Educause.

10.

Bendel

(2023). Image synthesis from an ethical perspective. AI & Society, 40, 437–446.

11.

Breines

M. R.

Gallagher

(2023). A return to Teacherbot: rethinking the development of educational technology at the University of Edinburgh. Teaching in Higher Education, 28(3), 517–531.

12.

Brown

Ertl

Lewis

Luksch

Reker

Schneider

(2020). Artificial intelligence in chemistry and drug design. Journal of Computer-Aided Molecular Design, 34, 709–715.

13.

Caliskan

Bryson

J. J.

Narayanan

(2017). Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334), 183–186.

14.

Chai

C. S.

King

R. B.

Zhou

(2024). Development and validation of the artificial intelligence learning intention scale (AILIS) for university students. SAGE Open, 14(2), 1–16. https://doi.org/10.1177/21582440241242188

15.

Chan

C. K. Y.

(2023). Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education. International Journal of Educational Technology in Higher Education, 20(1), 43.

16.

Chaudhry

M. A.

Kazim

(2022). Artificial Intelligence in Education (AIEd): A high-level academic and industry note 2021. AI and Ethics, 2(1), 157–165.

17.

Chen

Mao

Liu

(2014). Big data: A survey. Mobile Networks and Applications, 19(2), 171–209.

18.

Chen

Wang

(2024). Empowering education development through AIGC: A systematic literature review. Education and Information Technologies, 29(13), 17485–17537.

19.

Cheung

Vogel

(2013). Predicting user acceptance of collaborative technologies: An extension of the technology acceptance model for e-learning. Computers & Education, 63, 160–175.

20.

Chubb

Cowling

Reed

(2022). Speeding up to keep up: exploring the use of AI in the research process. AI & Society, 37(4), 1439–1457.

21.

Ciriello

R. F.

Richter

Mathiassen

(2024). Emergence of creativity in IS development teams: A socio-technical systems perspective. International Journal of Information Management, 74, 102698.

22.

Cousin

Deepwell

(2005). Designs for network learning: A communities of practice perspective. Studies in Higher Education, 30(1), 57–66.

23.

Davis

F. D.

(1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340.

24.

Deroncele-Acosta

Bellido-Valdiviezo

Sánchez-Trujillo

M. D.

Palacios-Núñez

M. L.

Rueda-Garcés

Brito-Garcías

J. G.

(2024). Ten essential pillars in artificial intelligence for university science education: A scoping review. SAGE Open, 14(3), 1–19. https://doi.org/10.1177/21582440241272016.

25.

Dillenbourg

(2008). Integrating technologies into educational ecosystems. Distance Education, 29(2), 127–140.

26.

Zhang

Niyato

Kang

Xiong

Kim

D. I.

Shen

Poor

H. V.

(2023). Exploring collaborative distributed diffusion-based AI-generated content (AIGC) in wireless networks. IEEE Network, 38(3), 178–186. https://doi.org/10.1109/MNET.006.2300223

27.

Eyal

Yarm

(2018). Schools in cross-sector alliances: what do schools seek in partnerships? Educational Administration Quarterly, 54(4), 648–688.

28.

Farhan

Razmak

Demers

Laflamme

(2019). E-learning systems versus instructional communication tools: Developing and testing a new e-learning user interface from the perspectives of teachers and students. Technology in Society, 59, 101192.

29.

Fellows

Liu

(2016). Sensemaking in the cross-cultural contexts of projects. International Journal of Project Management, 34(2), 246–257.

30.

Fidan

Gencel

(2022). Supporting the instructional videos with chatbot and peer feedback mechanisms in online learning: The effects on learning performance and intrinsic motivation. Journal of Educational Computing Research, 60(7), 1716–1741.

31.

Fırat

(2023). Integrating AI applications into learning management systems to enhance e-learning. Instructional Technology and Lifelong Learning, 4(1), 1–14.

32.

Y. H.

(2024). Government-university relationship in China’s AI talent development: A triple helix perspective. SAGE Open, 14(2). https://doi.org/10.1177/21582440241259012

33.

Zhu

Sarkis

(2012). Evaluating green supplier development programs at a telecommunications systems provider. International Journal of Production Economics, 140(1), 357–367.

34.

Fui-Hoon Nah

Zheng

Cai

Siau

Chen

. (2023). Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration. Journal of Information Technology Case and Application Research, 25(3), 277–304.

35.

Gao

(2023). AIGC technology: Reshaping the future of the animation industry. Highlights in Science, Engineering and Technology, 56, 148–152.

36.

Gay

(2010). Culturally responsive teaching: Theory, research, and practice. Teachers College Press.

37.

Gill

S. S.

Patros

Kaur

Fuller

Singh

Arora

Parlikad

A. K.

Stankovski

Abraham

Ghosh

S. K.

Lutfiyya

Kanhere

S. S.

Bahsoon

Rana

Dustdar

Sakellariou

Buyya

(2024). Transformative effects of ChatGPT on modern education: Emerging Era of AI Chatbots. Internet of Things and Cyber-Physical Systems, 4, 19–23.

38.

Goel

A. K.

Polepeddi

(2018). Jill Watson: A virtual teaching assistant for online education. In B. Dede, J. Richards, B. Saxberg (Eds.), Learning engineering for online education (pp. 120–143). Routledge.

39.

Granić

Marangunić

(2019). Technology acceptance model in educational context: A systematic literature review. British Journal of Educational Technology, 50(5), 2572–2593.

40.

Guo

Chen

Wang

(2023). AIGC challenges and opportunities related to public safety: a case study of ChatGPT. Journal of Safety Science and Resilience, 4(4), 329–339.

41.

Haleem

Javaid

Qadri

M. A.

Suman

(2022). Understanding the role of digital technologies in education: A review. Sustainable Operations and Computers, 3, 275–285.

42.

Han

Anderson

C. K.

(2022). The dynamic customer engagement behaviors in the customer satisfaction survey. Decision Support Systems, 154, 113708.

43.

Hannan

Liu

(2023). AI: New source of competitiveness in higher education. Competitiveness Review: An International Business Journal, 33(2), 265–279.

44.

Hasan

Brown

Davidovic

Lange

Regan

(2022). Algorithmic bias and risk assessments: Lessons from practice. Digital Society, 1(2), 14.

45.

Hofstede

(2001). Culture’s consequences: Comparing values, behaviors, institutions, and organizations across nations. Sage Publications.

46.

Holmes

Porayska-Pomsta

(Eds.). (2022). The Ethics of Artificial Intelligence in education: Practices, challenges, and debates. Taylor & Francis.

47.

Hsu

C. W.

Kuo

T. C.

Chen

S. H.

A. H.

(2013). Using dematel to develop a carbon management model of supplier selection in green supply chain management. Journal of Cleaner Production, 56(1), 164–172.

48.

Huang

A. Y.

O. H.

Yang

S. J.

(2023). Effects of artificial Intelligence–Enabled personalized recommendations on learners’ learning engagement, motivation, and outcomes in a flipped classroom. Computers & Education, 194, 104684.

49.

Huang

K. L.

Liu

Y. C.

Dong

M. Q.

(2024). Incorporating AIGC into design ideation: A study on self-efficacy and learning experience acceptance under higher-order thinking. Thinking Skills and Creativity, 52, 101508.

50.

Huang

K. L.

Liu

Y. C.

Dong

M. Q.

C. C.

(2024). Integrating AIGC into product design ideation teaching: An empirical study on self-efficacy and learning outcomes. Learning and Instruction, 92, 101929.

51.

Hyer

N. L.

Brown

K. A.

Zimmerman

(1999). A socio-technical systems approach to cell design: case study and analysis. Journal of Operations Management, 17(2), 179-203.

52.

Igbokwe

I. C.

(2023). Application of artificial intelligence (AI) in educational management. International Journal of Scientific and Research Publications, 13(3), 300–307.

53.

Ito

Baumer

Bittanti

boyd

Cody

Stephenson

B. H.

Horst

H. A.

Lange

P. G.

Mahendran

Martínez

K. Z.

Pascoe

C. J.

Perkel

Robinson

Sims

Tripp

Antin

Finn

Law

Manion

Horst

H. A.

(2010). Hanging out, messing around, and geeking out: Kids living and learning with new media. MIT Press.

54.

Ivanov

Soliman

Tuomi

Alkathiri

N. A.

Al-Alawi

A. N.

(2024). Drivers of generative AI adoption in higher education through the lens of the Theory of Planned Behaviour. Technology in Society, 77, 102521. https://doi.org/10.1016/j.techsoc.2024.102521

55.

Jeng

J. F.

Tzeng

G. H.

(2012). Social influence on the use of clinical decision support systems: revisiting the unified theory of acceptance and use of technology by the fuzzy dematel technique. Computers & Industrial Engineering, 62(3), 819–828.

56.

Jobin

Ienca

Vayena

(2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9), 389–399.

57.

Kacena

M. A.

Plotkin

L. I.

Fehrenbacher

J. C.

(2024). The use of artificial intelligence in writing scientific review articles. Current Osteoporosis Reports, 22(1), 115–121.

58.

Khowaja

S. A.

Khuwaja

Dev

Wang

Nkenyereye

(2024). Chatgpt needs spade (sustainability, privacy, digital divide, and ethics) evaluation: A review. Cognitive Computation, 16(5), 2528–2550.

59.

Koo

Park

Lee

Seo

Moon

Lim

(2023). Uncovering the risks and drawbacks associated with the use of synthetic data for grammatical error correction. IEEE Access. 11, 95747–95756.

60.

Krauss

Streicher

Poxleitner

Altun

Mueller

T. S.

Müller

(2023, June). Best-of-Breed: Service-oriented integration of artificial intelligence in interoperable educational ecosystems. In International workshop on learning technology for education challenges (pp. 267–283). Springer Nature Switzerland.

61.

Kurtz

Amzalag

Shaked

Zaguri

Kohen-Vacs

Gal

Zailer

Barak-Medina

(2024). Strategies for integrating generative AI into higher education: navigating challenges and leveraging opportunities. Education Sciences, 14(5), 503.

62.

T. W.

Hsu

Fowler

Zhang

Zilles

Karahalios

(2023, August). Am I wrong, or is the autograder wrong? Effects of AI grading mistakes on learning. In Proceedings of the 2023 ACM Conference on International Computing Education Research (vol. 1, pp. 159–176). Association for Computing Machinery.

63.

Zheng

Chen

Zong

(2024). User interaction interface design and innovation based on artificial intelligence technology. Journal of Theory and Practice of Engineering Science, 4(3), 1–8.

64.

Dhruv

Jain

(2024, February). Ethical considerations in the use of AI for higher education: A comprehensive guide. In 2024 IEEE 18th International Conference on Semantic Computing (ICSC) (pp. 218–223). IEEE.

65.

Lian

Tang

Xiang

Dong

(2024). Public attitudes and sentiments toward ChatGPT in China: A text mining analysis based on social media. Technology in Society, 76, 102442.

66.

Liu

Ren

Nyagoga

L. M.

Stonier

(2023). Future of education in the era of generative artificial intelligence: Consensus among Chinese scholars on applications of ChatGPT in schools. Future in Educational Research, 1(1), 72–101.

67.

Lopez

(2017). IBM and MIT to pursue joint research in artificial intelligence, establish new MIT-IBM Watson AI Lab. MIT News.

68.

Mariani

M. M.

Perez-Vega

Wirtz

(2022). AI in marketing, consumer research and psychology: A systematic literature review and research agenda. Psychology & Marketing, 39(4), 755–776.

69.

Mell

Grance

(2011). The NIST definition of cloud computing. NIST Special Publication, 800(145), 7.

70.

Nah

Cai

Zheng

Pang

(2023). An activity system-based perspective of generative AI: Challenges and research directions. AIS Transactions on Human-Computer Interaction, 15(3), 247–267.

71.

Naseer

Khalid

M. U.

Ayub

Rasool

Abbas

Afzal

M. W.

(2024). Automated assessment and feedback in higher education using generative AI. In R. C. Sharma & A. Bozkurt (Eds.), Transforming education with generative AI: Prompt engineering and synthetic content creation (pp. 433–461). IGI Global.

72.

Peng

Zheng

Xie

(2024). Embracing the future: Integrating ChatGPT into China’s nursing education system. International Journal of Nursing Sciences. https://doi.org/10.1016/j.ijnss.2024.03.006

73.

Nielsen

(1993). Usability engineering. Academic Press, Inc.

74.

Ouyang

Zheng

Jiao

(2022). Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020. Education and Information Technologies, 27(6), 7893–7925.

75.

Perikos

Grivokostopoulou

Hatzilygeroudis

(2017). Assistance and feedback mechanism in an intelligent tutoring system for teaching conversion of natural language into logic. International Journal of Artificial Intelligence in Education, 27, 475–514.

76.

Peters

M. A.

Green

B. J.

(2024). Wisdom in the age of AI education. Postdigital Science and Education, 6, 1173–1195.

77.

Petter

DeLone

McLean

(2008). Measuring information systems success: Models, dimensions, measures, and interrelationships. European Journal of Information Systems, 17, 236–263.

78.

Qin

Yan

(2020). Understanding user trust in artificial intelligence-based educational systems: Evidence from China. British Journal of Educational Technology, 51(5), 1693–1710.

79.

Rodway

Schepman

(2023). The impact of adopting AI educational technologies on projected course satisfaction in university students. Computers and Education: Artificial Intelligence, 5, 100150.

80.

Rogers

E. M.

Singhal

Quinlan

M. M.

(2014). Diffusion of innovations. In An integrated approach to communication theory and research (pp. 432–448). Routledge.

81.

Ryan

(2015). It ain’t just what you do and the way that you do it: why discourse matters in higher education communities of practice. Higher Education Research & Development, 34(5), 1001–1013.

82.

Sartori

Theodorou

(2022). A sociotechnical perspective for the future of AI: narratives, inequalities, and human control. Ethics and Information Technology, 24(1), 4.

83.

Schwartz

Tilling

(2023). Making evidence-based practice actionable in the social service context: experiences and implications of workplace education. Journal of Workplace Learning, 35(9), 311–328.

84.

Shoaib

Sayed

Singh

Shafi

Khan

Ali

(2024). AI student success predictor: Enhancing personalized learning in campus management systems. Computers in Human Behavior, 158, 108301.

85.

Siemens

(2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400.

86.

Slavin

R. E.

(2020). How evidence-based reform will transform research and practice in education. Educational Psychologist, 55(1), 21–31.

87.

Sony

Naik

(2020). Industry 4.0 integration with socio-technical systems theory: A systematic review and proposed theoretical model. Technology in Society, 61, 101248.

88.

Srinivasa

K. G.

Kurni

Saritha

(2022). Harnessing the Power of AI to Education. In Learning, teaching, and assessment methods for contemporary learners: Pedagogy for the digital generation (pp. 311–342). Springer Nature Singapore.

89.

Suh

Ahn

(2022). Development and validation of a scale measuring student attitudes toward artificial intelligence. SAGE Open, 12(2), 1–12. https://doi.org/10.1177/21582440221100463

90.

Suhail

Adel

Al-Emran

AlQudah

A. A.

(2024). Are students ready for robots in higher education? Examining the adoption of robots by integrating UTAUT2 and TTF using a hybrid SEM-ANN approach. Technology in Society, 77, 102524. https://doi.org/10.1016/j.techsoc.2024.102524

91.

Tran

M. Q.

(2023). Artificial intelligence in the modern era connecting technology, cultural contexts, and ethical considerations. Journal of Computational Social Dynamics, 8(11), 34–41.

92.

Trist

E. L.

Bamforth

K. W.

(1951). Some social and psychological consequences of the longwall method of coal-getting: An examination of the psychological situation and defences of a work group in relation to the social structure and technological content of the work system. Human Relations, 4(1), 3–38.

93.

Troumpoukis

Klampanos

Pantazi

D.-A.

Albughdadi

Baousis

Barrilero

Bojor

Branco

Bruzzone

Chietera

Fournand

Hall

Lazzarini

Luna

Nousias

Perentis

Petrakis

Punjani

Röbl

Stamoulis

Karkaletsis

(2024). European AI and EO convergence via a novel community-driven framework for data-intensive innovation. Future Generation Computer Systems, 160, 505–521.

94.

Tzeng

G. H.

Huang

J. J.

(2011). Multiple attribute decision making: Methods and applications. CRC Press.

95.

Vaarma

(2024). Predicting student dropouts with machine learning: An empirical study in Finnish higher education. Technology in Society, 76, 102474. https://doi.org/10.1016/j.techsoc.2024.102474

96.

Venkatesh

Morris

M. G.

Davis

G. B.

Davis

F. D.

(2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425–478.

97.

Venkatesh

Thong

J. Y. L.

(2012). Consumer acceptance and use of information technology: Extending the unified theory of acceptance and use of technology. MIS Quarterly, 36(1), 157–178.

98.

von Winckelmann

S. L

. (2023). Predictive algorithms and racial bias: a qualitative descriptive study on the perceptions of algorithm accuracy in higher education. Information and Learning Sciences, 124(9/10), 349-371.

99.

Wang

Chen

Wang

(2016). Cloud computing: A comprehensive survey. Mobile Networks and Applications, 19(6), 1092–1120.

100.

Wang

Zhao

(2023). Reducing carbon footprint in a resilient supply chain: examining the critical influencing factors of process integration. International Journal of Production Research, 61(18), 6197–6214.

101.

Wang

Liu

Y. F.

(2021). Factors affecting the adoption of AI-based applications in higher education. Educational Technology & Society, 24(3), 116–129.

102.

Wang

Pan

Yan

Luan

T. H.

(2023). A survey on ChatGPT: AI-generated contents, challenges, and solutions. IEEE Open Journal of the Computer Society, 4, 280–302.

103.

Wang

(2024). Artificial intelligence in dance education: Using immersive technologies for teaching dance skills. Technology in Society, 77, 102579. https://doi.org/10.1016/j.techsoc.2024.102579

104.

Wenger

(1998). Communities of practice: Learning, meaning, and identity. Cambridge University Press.

105.

(2024). Construction of new engineering talent training mode from the perspective of innovation ecology—Optimization and innovation path of University and Industry Cooperation Mechanism. Journal of Human Resource Development, 6(2), 20–27.

106.

Liang

(2023). Empowering vocational education new media marketing teaching reform and practical exploration with AIGC. Journal of Education and Educational Research, 6(2), 82–85.

107.

Xiang

(2023). Interconnected knowledge: Examining the evolution of graduate student innovation ecosystems. Journal of the Knowledge Economy, 15(3), 14036–14075.

108.

Zhang

Yoo

Pan

(2023). Is everyone an artist? A study on user experience of AI-based painting system. Applied Sciences, 13(11), 6496.

109.

Chen

P. F.

Zhang

(2024). Exploring Chinese university educators’ acceptance and intention to use AI tools: An application of the UTAUT2 model. SAGE Open, 14(4), 1–13. https://doi.org/10.1177/21582440241290013

110.

Yang

Tong

(2023). In-depth application of artificial intelligence-generated content AIGC large model in higher education. Adult and Higher Education, 5(19), 9–16.

111.

Yang

S. J.

Ogata

Matsui

Chen

N. S.

(2021). Human-centered artificial intelligence in education: Seeing the invisible through the visible. Computers and Education: Artificial Intelligence, 2, 100008.

112.

Yiwen

Yahui

Jinrong

Tao

Ting

Yingxi

Yanxia

Taiguo

Xian

(2023, December). Human-AI collaboration: A study on anti-chatgpt strategies employed in innovative practical homework towards “one-click-answer” issue in AIGC. In International Conference on Computer Science and Education (pp. 343–353). Springer Nature Singapore.

113.

Gong

(2024). ChatGPT, AI-generated content, and engineering management. Frontiers of Engineering Management, 11(1), 159–166.

114.

Zhai

Wibowo

(2023). A systematic review on artificial intelligence dialogue systems for enhancing English as foreign language students’ interactional competence in the university. Computers and Education: Artificial Intelligence, 4, 100134.

115.

Zhao

Y. P.

Xiao

Y. Y.

Chang

H. D.

Liu

(2024). Factors influencing the acceptance of ChatGPT in high education: An integrated model with PLS-SEM and fsQCA approach. SAGE Open, 14(4). https://doi.org/10.1177/21582440241289835

Aiming to Enhance Higher Education Practice Through Generative AI Integration: A Theoretical Exploration of Critical Success Factors

Abstract

Plain language summary

Keywords

Introduction

Literature Review and Theoretical Framework

Application of AIGC in Higher Education

Socio-technical Systems Theory and AIGC Application

Key Influencing Factors of AIGC Application in Higher Education

Key Success Factors from a Technical Perspective

Key Success Factors from a Social Perspective

Construction of Success Factor System for AIGC Application in Higher Education

Research Methodology

DEMATEL Analysis Procedures

Step 1: Establishing the Direct Relation Matrix

Step 2: Normalizing the DRM

Step 3: Calculating the Total Relation Matrix

Step 4: Drawing the DEMATEL Relationship Diagram

Step 5: Mapping the Relationships Among Factors in the Diagram

Alignment of DEMATEL with the Purpose of the Paper

Data Collection and Analysis

Expert selecting

Data collection instrument

Validation and Digitization

Tables Summarizing Data Transaction

Results and Discussion

Analysis of Relationships Between Success Factors

Analysis of Relationships Between Subfactors

Conclusions

Summary of the Main Findings

Theoretical Implications

Practical Implications

Limitations and Future Work

Footnotes

Acknowledgements

ORCID iDs

Ethical Considerations

Informed consent

Funding

Declaration of Conflicting Interests

Data Availability Statement

References