Abstract
Artificial intelligence-supported online assessment and evaluation (AI-SOAE) systems have the potential to transform assessment and evaluation processes in higher education; however, there is a need for a comprehensive evaluation of the pedagogical, technical, and ethical dimensions of these systems. In this regard, the study systematically examined the characteristics of AI-SOAE systems within the framework of a SWOT analysis. A systematic literature review method was used in the research. Ninety-eight studies published between 2018 and 2025 in the Web of Science (WoS), Scopus, and Education Resources Information Centre (ERIC) databases were analysed in accordance with PRISMA guidelines. The findings show that the strengths of AI-SOAE systems include personalised feedback, advanced assessment processes, increased efficiency, and data-driven decision support mechanisms. In contrast, technical infrastructure limitations, algorithmic biases, data management issues, and negative user experiences were identified as the weaknesses of the systems. The analysis also reveals that AI-SOAE systems offer significant opportunities in terms of pedagogical transformation, digitalisation, and institutional competitiveness; however, they face threats such as data security, ethical concerns, risks to academic integrity, and limited empirical evidence. Consequently, AI-SOAE systems offer context- and application-dependent potential for supporting assessment and evaluation practices in higher education. Effectively realising this potential depends on strengthening technical infrastructure, managing algorithmic risks, and developing transparent, ethical, and institutional policy frameworks. By synthesising the literature in the field of AI-SOAE in a structured manner, this study provides a comprehensive assessment for researchers, educators, and policymakers.
Plain Language Summary
Universities are increasingly using artificial intelligence (AI) to support online assessment and evaluation processes, such as grading, feedback, and exam monitoring. These systems are designed to reduce instructors’ workload, provide faster feedback, and support more data-informed decision making. However, there is still limited clarity about how effective, reliable, and ethically appropriate these systems are when used in real educational contexts. This study reviewed 98 academic studies published between 2018 and 2025 to examine how AI-SOAE systems are currently applied in higher education. Studies were selected from the WoS, Scopus, and ERIC databases and analysed using a structured SWOT approach, which considers strengths, weaknesses, opportunities, and threats. The findings show that AI-supported assessment systems can offer important benefits, such as personalised feedback, improved efficiency, and support for learning-oriented assessment practices. At the same time, the review identified key challenges, including technical infrastructure limitations, data privacy concerns, algorithmic bias, and negative effects on student trust and well-being. While AI-SOAE systems present opportunities for pedagogical innovation and digital transformation, their use also involves ethical, institutional, and governance-related risks. Overall, the study suggests that AI-supported assessment systems should be understood as context-dependent tools rather than universal solutions. Their effective and responsible use requires strong technical infrastructure, transparent policies, ethical guidelines, and institutional oversight. The results of this review provide practical insights for educators, university leaders, and policymakers who are considering the integration of AI into assessment and evaluation practices in higher education.
Keywords
Introduction
Artificial Intelligence (AI) is an interdisciplinary field that aims to replicate and enhance human cognitive processes such as learning, problem-solving, perception, and decision-making through digital systems (Russell & Norvig, 2020). Today, AI is widely used in many fields, including education, significantly transforming teaching processes, learning management, and feedback mechanisms (Zaheer et al., 2024). Particularly in online learning environments where digitalisation is accelerating, AI-supported applications have made teaching more interactive, flexible, and personalised (Arya & Verma, 2024). These applications can analyse data obtained from students’ online interactions and provide content, guidance, and assessment support tailored to individual learning needs. Thus, AI is reshaping not only teaching processes but also assessment approaches, which are an integral part of these processes.
Traditional assessment and evaluation methods are inadequate for objectively, quickly, and personalisedly evaluating student performance in online learning environments (Saputra et al., 2024). Developed to address these limitations, AI-SOAE analyses student achievements in a data-driven manner, identifying learning gaps at an early stage and providing immediate feedback (Jahongir, 2024). In recent years, these systems have gained a new dimension with the integration of generative AI-based large language models. The assessment of open-ended responses, rubric-based feedback generation, and support for formative assessment processes are prominent examples of this transformation (Qiu & Liu, 2025). However, while objectivity and efficiency in assessment processes have increased, new debates have emerged in areas such as academic integrity, originality, measurement validity, and pedagogical ethics (Wu et al., 2025).
AI-based systems reduce teachers’ workload, accelerate assessment processes, and enrich the learning experience by providing students with detailed, personalised feedback (Niu et al., 2022; Novianti, 2025). Adaptive tests provide individualised exam experiences through dynamic question sets that are shaped according to students’ knowledge levels and learning speeds (Naseer & Khawaja, 2025; Seo et al., 2021). In addition, AI-supported monitoring systems used in online examinations are widely employed to maintain academic integrity and enhance assessment reliability (Tripathi & Thakar, 2024).
Despite all these developments, ethical issues such as data privacy, algorithmic bias, and lack of transparency in decision-making processes are becoming increasingly visible (Cacho, 2024). The constant collection of data can create a feeling of being monitored among students, increasing their stress and anxiety levels (Williamson & Eynon, 2020). Therefore, it is critically important that AI-based assessment systems are structured in accordance with ethical principles, supported by robust data security policies, and implemented in line with transparency principles (Selwyn, 2019).
In this context, AI-SOAE systems stand out as an important innovation that enhances objectivity, consistency, and efficiency in education. However, the rapidly evolving nature of AI necessitates a re-examination of the ethical, pedagogical, and methodological dimensions of assessment and evaluation processes. This study aims to provide a comprehensive assessment of the fair, sustainable, and ethical use of these technologies in education by examining the strengths and weaknesses of AI-SOAE systems within the framework of a SWOT analysis, based on the current literature.
Theoretical Framework
The SWOT analysis used in this study is approached not merely as a descriptive classification tool, but as a multi-layered analytical framework integrated with technology acceptance and measurement-evaluation approaches. This approach enables the interpretation of the pedagogical, technical, and ethical dimensions of AI-SOAE systems not only through their outputs, but also through the theoretical mechanisms that produce these outputs.
SWOT
SWOT is an approach developed in the 1950s for corporate strategic analysis, formed from the initial letters of the concepts Strengths, Weaknesses, Opportunities, and Threats (Benzaghta et al., 2021). Adapted over time to different disciplines, this approach is widely used in the field of education, particularly in decision-making and strategic planning processes (C. Zhu & Justice Mugenyi, 2015). SWOT provides a structured analytical framework that enables the systematic evaluation of internal (strengths and weaknesses) and external (opportunities and threats) factors in the integration of new technologies into educational environments (Dar et al., 2024).
In this context, strengths refer to the capacity and advantages that enable a technology to achieve its objectives; weaknesses refer to existing limitations and shortcomings; opportunities refer to favourable environmental conditions; and threats refer to external risks. Although SWOT analysis has been used in various applications in the field of education (Farrokhnia et al., 2024), there appears to be a limited number of studies that directly address AI-SOAE systems in higher education within this framework.
In this study, SWOT analysis is used to comprehensively evaluate the pedagogical, technical and ethical dimensions of AI-SOAE systems; it provides a theoretical foundation to guide researchers, educators and policymakers in system design, implementation and improvement processes. The analysis encompasses AI-supported tools but is limited to applications directly integrated into online examination processes.
Technology Acceptance Model (TAM)
The adoption of AI-SOAE systems in higher education is closely related to the theoretical mechanisms explained by the Technology Acceptance Model (TAM). Developed by Davis (1989), this model suggests that users’ tendency to accept technology is fundamentally dependent on perceived usefulness and perceived ease of use. The SWOT analysis used in this study allows for the conceptual structuring of these two fundamental dimensions in the context of AI-SOAE.
In this regard, the strengths dimension of SWOT is associated with the perceived benefits concept through the contributions of AI-SOAE systems to learning and assessment processes. The weaknesses, on the other hand, correspond to the perceived ease of use dimension through technical limitations, infrastructure problems, and complexities related to the use of the system. Ethical issues addressed in the threats category of SWOT, along with risks related to data security and transparency, are evaluated within the TAM framework as factors that could affect users’ perception of trust and their associated behavioural intentions. Opportunities point to structural conditions that strengthen technology adoption through digital transformation policies, corporate technology investments, and organisational strategies supporting AI integration. In this respect, SWOT analysis is used as a theoretical positioning tool that relates the acceptance processes of AI-SOAE systems to the fundamental components of TAM.
Measurement and Evaluation Approaches
AI-SOAE systems are directly related to measurement and assessment approaches. Validity refers to the extent to which the measurement tool represents the intended learning outcome (Messick, 1995), while AI-based feedback and automatic scoring systems make formative assessment continuous and instantaneous (Black & Wiliam, 1998). Reliability, on the other hand, is based on the consistency and repeatability of measurements, and the systematic data processing structure of AI algorithms aligns with this principle (Crocker & Algina, 1986). Furthermore, generative AI-supported applications enable the measurement of higher-order cognitive skills through authentic assessment (Bearman & Luckin, 2020). This theoretical framework allows for a comprehensive interpretation of the strengths and weaknesses of SWOT analysis in the context of validity, reliability, and assessment transparency.
Literature Review
A review of the literature reveals that while studies focussing on the effects of AI in distance education are limited in number, they present significant findings (Dogan et al., 2023; Gligorea et al., 2023; Göçmez & Okur, 2023; Ouyang et al., 2022; Tang et al., 2023). These studies emphasise contributions such as adaptive learning, personalised content delivery, and increased student engagement. However, it is also frequently noted that ethical principles must be observed (Dogan et al., 2023). Nevertheless, the majority of these studies only indirectly address evaluation processes and do not provide an analytical framework specific to AI-SOAE systems.
Studies directly focussed on AI-SOAE systems are quite limited (González-Calatayud et al., 2021; Memarian & Doleck, 2024; Mudkanna Gavhane & Pagare, 2024). González-Calatayud et al. (2021) examined the impact of AI on student assessments, while Mudkanna Gavhane and Pagare (2024) highlighted the potential of using AI in assessing students’ problem-solving skills. Memarian and Doleck (2024) focussed on the relationship between the “assessment for learning” approach and AI, raising ethical and reliability issues. However, these studies did not directly address the concept of online measurement and assessment. The only systematic review study explicitly using the term AI-SOAE was conducted by Karadağ (2023), which examines the effects of ChatGPT on online assessment processes while highlighting both the opportunities offered by the systems and the ethical and reliability issues.
Recent studies examining the integration of AI-based systems into assessment processes have made significant contributions to the literature. Qiu and Liu (2025) analysed AI’s potential to both generate and answer exam questions in medical education, highlighting its accuracy and formative assessment capabilities in measurement processes. Similarly, Hale et al. (2024) and Rueff et al. (2025) discussed the integration of AI into assessment policies, focussing on the dimensions of academic integrity and ethical integration. Wu et al. (2025) systematically examined attitudes and behaviours towards the use of generative AI in teaching and assessment processes in higher education. Cacho (2024) developed policy recommendations for the balanced and ethical integration of generative AI at the university level. Ghanem et al. (2025) and Khlaif et al. (2024) have presented important findings regarding assessment processes by examining the effects of AI on student achievement, trust, and ethical awareness.
In this context, the present study aims to examine AI-SOAE systems within the framework of SWOT analysis, based on literature published in international academic databases such as WoS, Scopus, and ERIC. It differs from similar studies in that it covers current studies published up to 2025, takes into account applications used on different platforms, and provides a comprehensive SWOT-based evaluation.
The study is expected to systematically evaluate the limited number of studies on AI-SOAE systems, revealing their strengths, weaknesses, opportunities, and threats; thereby contributing to filling the theoretical and practical gaps in this field. This SWOT analysis-based evaluation will enable the development of recommendations for structuring AI-SOAE systems in a more effective, fair, and sustainable manner. The findings are expected to guide the development of innovative assessment approaches in higher education institutions and contribute to shaping policies and practices for AIEd integration. The following research questions will be addressed during the study;
- What are the strengths of AI-SOAE systems?
- What are the weaknesses of AI-SOAE systems?
- What opportunities do AI-SOAE systems offer?
- What threats might be encountered in AI-SOAE systems?
Method
This study was conducted within a multi-stage and structured research process based on a systematic review approach. All stages of the research were planned within a comprehensive methodological workflow, ranging from the formulation of research questions to literature review, study selection process, coding and data extraction, and synthesis of findings. This process aims not only to show which publications were included in the study but also to reveal how the research was produced and which analytical stages were undergone to reach the results. In this regard, the research process followed is visualised in the methodological flow diagram summarised in Figure 1.
The systematic literature review procedure (Tao et al., 2025).
In the first stage, the main objectives of the research and the research questions developed accordingly were determined. In the second stage, a comprehensive literature review was conducted in various digital databases using predefined key search terms. In the third stage, suitable studies were selected in line with the predefined inclusion and exclusion criteria, and the final data set was created. In the fourth stage, the selected studies were systematically coded and data relevant to the research questions were extracted. In the fifth and final stage, the obtained data were compared to form answers to the research questions, synthesised according to themes, and discussed comprehensively. The first stage of the study was described in the previous section. Detailed processes relating to the second, third, and fourth stages are presented in the following sections; findings relating to the fifth stage are addressed in the final section.
Searching Papers
The screening conducted on 13 December 2025 initially examined studies on AI-SOAE systems without any year restrictions. The preliminary assessment revealed that the relevant publications were from 2018 onwards. The final sample covers studies from 2018 to 2025. To identify articles for review, three core databases were used: WoS, Scopus, and ERIC, which cover multidisciplinary research and include publications specific to the field of education (Pranckutė, 2021). WoS and Scopus are among the databases of critical importance for ensuring reliability and validity in systematic literature reviews. These platforms enable in-depth and comprehensive searches thanks to their inclusion of wide-ranging, high-quality peer-reviewed publications (Moed, 2005). Scopus, with its multidisciplinary structure, covers more than 22,000 journals and over 72 million records (Harzing & Alakangas, 2016), while WoS stands out for its inclusion of high-impact and internationally recognised journals (Moed, 2005). ERIC, on the other hand, is an important academic database specific to the field of educational sciences, covering policy documents, theoretical studies, and empirical research, providing access to studies particularly in the areas of educational technologies and measurement and evaluation (ERIC, 2024). The inclusion of ERIC has contributed to a more comprehensive representation of education-related studies on AI-SOAE systems. In this regard, publications with a high impact level offered by WoS and Scopus were evaluated together with ERIC’s field-focussed content, aiming to ensure that the study covered both theoretical and applied literature in a balanced manner (Falagas et al., 2008).
In the literature review, keywords related to distance education, artificial intelligence, and assessment were used. Accordingly, searches were conducted using the same keywords in the WoS, Scopus, and ERIC databases. These keywords were: (“ai” OR “Artificial Intelligence”) AND (“distance education” OR “distance teaching” OR “distance learning” OR “remote education” OR “remote learning” OR “remote teaching” OR “online education” OR “online learning” OR “online teaching” OR “online course” OR “eLearning” OR “e-learning” OR “MOOC*”) AND (“assessment” OR “evaluation” OR “online assessment” OR “online evaluation”) AND (“higher education” OR “university”) were determined to comprehensively address the AI-SOAE system.
Reviewing and Excluding Papers
This study was conducted in accordance with the PRISMA guidelines, the recommended reporting standard for systematic reviews (Page et al., 2021). PRISMA is widely used in systematic reviews in the field of education as it provides a transparent, traceable, and structured reporting process (Ba & Hu, 2023; Crompton et al., 2021). The study selection process was conducted based on the PRISMA flow diagram, and the details are presented in Figure 2.
Prisma of article selection.
The screening process for studies in the field of AI-SOAE was conducted in the WoS, Scopus, and ERIC databases based on the keywords listed in Figure 2. The initial screening without applying any filters yielded a total of 1,005 studies: 154 in WoS, 709 in Scopus, and 142 in ERIC. Subsequently, only articles in English and open access publications were filtered. As a result of this filtering, 764 studies were excluded, leaving a total of 241 articles: 61 from WoS, 141 from Scopus, and 39 from ERIC.
The articles obtained were reviewed by two independent coders at the level of title, abstract, and full text for their relevance to the research topic. At this stage, 95 studies that were not directly related to the AI-SOAE concept, focussed solely on distance education technologies, or did not relate AI to online assessment processes were excluded. Furthermore, 48 studies that were duplicated across databases were excluded to preserve the uniqueness of the sample. As a result of the suitability assessment, a total of 98 articles published between 2018 and 2025 were included in the study. References to these included studies are provided in the Supplementary Material.
In the inclusion and exclusion process, the following criteria were applied: (1) not being an article, (2) not being published in English, (3) not being open access, (4) not being directly related to the research topic, and (5) being a duplicate record. All publications included in the study focus on AI-SOAE at the higher education level. The included articles were also analysed by classifying them according to AI types (e.g., automated assessment systems, learning analytics, chatbots), study types (user-centred, theoretical, model-based), and focus areas. All this screening, elimination, selection process, and criteria were carried out according to the Research Protocol in Appendix 1.
Coding and Extracting Information
To ensure that the screening was complete and reliable, two coders evaluated the titles, abstracts, and full texts of the retrieved articles according to predefined inclusion and exclusion criteria. Initially, both authors worked together on two sample articles to ensure consistency in the coding process. They then independently coded 25% of their work and measured inter-rater reliability using Cohen’s kappa coefficient (Cohen, 1960). Cohen’s kappa values were calculated for each coding domain, yielding a value of 0.86. This value indicates near-perfect agreement, as it falls between 0.81 and 1 (McHugh, 2012). Once the acceptable level of reliability was established, the authors proceeded to code the remaining articles. To ensure consistency and accuracy in the coding process, any disagreements that arose were resolved through thorough discussion.
Limitations
One of the significant limitations of this study is that it only includes articles published in English, available in open access, and indexed in WoS, Scopus, and ERIC. The systematic review process was limited to peer-reviewed journal articles. While this choice aimed to increase methodological consistency and scientific reliability among the studies examined, it resulted in the exclusion of different types of publications that could contribute to the AI-SOAE field, such as conference proceedings, book chapters, technical reports, and policy documents.
Secondly, limiting the search to works published in English aimed to ensure consistent interpretation of conceptual and technical terminology in the fields of artificial intelligence and assessment. However, this criterion may have restricted access to some studies conducted in different linguistic and cultural contexts that could shed light on local practices.
Finally, the inclusion of only open access publications in the review aimed to support the transparency and reproducibility of the study. However, this approach may have resulted in the exclusion of some studies that could make significant contributions to the field but are only available on subscription-based platforms due to access restrictions.
These limitations, while keeping the scope of the study within a specific framework, do not diminish the aim of revealing general trends, methodological approaches, and research focuses in the field of AI-SOAE. For future studies, it is recommended that broader scanning strategies covering different publication types, languages, and access models be employed to provide a more comprehensive perspective on the field.
Data Analysis
Within the scope of the study, 98 studies accessed were analysed using content analysis methods (Appendix 2). In this context, a data collection tool was developed to be used in the content analysis process of the studies included in the research. The dimensions of the data collection tool were determined based on the research questions identified by referring to the SWOT analysis. Accordingly, the dimensions of the data collection tool were determined in parallel with the research questions as the strengths, weaknesses, opportunities, and threats of AI-SOAEs. The distinction between weaknesses and threats in the SWOT analysis was made based on the system’s internal and external factors, in line with the classic SWOT approach. In this context, for example, algorithmic biases were classified as “weaknesses” because they stem from the system’s own functioning, data sets, or technical limitations of algorithms. On the other hand, ethical issues are evaluated in the “threat” category because they are linked to the reactions of external actors (users, managers, policymakers, social perceptions, etc.), legal regulations, and cultural acceptance.
The analysis process was conducted in two stages: first, themes were extracted through open coding, and then these themes were categorised according to the SWOT framework. In other words, a thematic analysis was performed, and then these themes were matched to SWOT categories. This demonstrates that SWOT is used not only as a descriptive tool but also as a synthesising analytical framework.
Findings
The findings obtained in this study are first presented within the framework of the four fundamental dimensions of SWOT analysis (strengths, weaknesses, opportunities, and threats), in parallel with the research questions. Subsequently, the emerging themes are re-examined from a holistic perspective without forming a separate research question, and the ways in which the SWOT dimensions intersect are evaluated at a synthesising level of interpretation. This approach is consistent with the integrative analysis logic commonly used in systematic review studies and aims to reveal the structural relationships between the findings.
Strengths of AI-SOAE Systems
Relevant studies have been analysed to determine the strengths of AI-SOAE systems at the higher education level. The categories and codes obtained because of the analysis are summarised in Table 1.
Strengths of AI-SOAE systems.
According to Table 1, six main categories related to the strengths of AI-SOAE systems in higher education have been identified: “Feedback,”“Personalisation,”“Advanced assessment,”“Validity-reliability,”“Efficiency” and “Data-driven decision support.” Personalisation, instant feedback, advanced assessment, and data-driven decision-making are the most frequently recurring strengths in this theme. This indicates that the relevant systems are becoming increasingly mature in terms of both technical and functional aspects. The intensive use of codes such as “high academic performance,”“high accuracy value,”“advanced performance measurement,” and “comprehensive evaluation” is the most prominent indicator of this situation. Adaptive testing, learning analytics, and deep learning models enable more precise and dynamic assessment of student performance, initiating a qualitative transformation in measurement and evaluation that can be described as an “increase in resolution.” Numerous studies support this, noting that AI-SOAE systems can provide more accurate and objective assessments. Instant feedback mechanisms develop students’ self-regulation skills, leading to meaningful improvements in student performance. Again, in this system, exams go beyond being a measurement tool that merely produces results and become a dynamic process that shapes learning. These two developments show that AI-SOAE systems have moved beyond being a measurement tool and have become an active regulator of the learning process. These results show that AI can be defined as a tool that automates assessment processes while improving the quality of assessment.
Another pattern emerging across the studies is that personalisation and adaptability have become the norm for these systems. AI-SOAE systems shape learning experiences according to students’ individual needs. The heavy use of codes such as “personalised feedback,”“instant feedback,”“adaptive assessment,”“adaptive learning process” and “flexibility” can be seen as evidence of this. Gamified learning scenarios, deep learning-based teaching models, and new-generation adaptive assessment frameworks contribute to this outcome. Furthermore, it can be said that the assessment processes of the relevant systems are beginning to integrate with pedagogical design. In particular, the continuous monitoring of students throughout the entire process and the planning of subsequent learning processes indicate the emergence of dynamic learning ecosystems.
Weaknesses of AI-SOAE Systems
Studies related to identifying the weaknesses of AI-SOAE systems in higher education have been analysed. The findings obtained from the analysis are presented in Table 2.
Weaknesses of AI-SOAE Systems.
According to Table 2, three main categories have been identified regarding the weaknesses of AI-SOAE systems in higher education: “Technical issues,”“Algorithmic bias and limitations,” and “Negative experiences and perceptions.” Technical limitations and algorithmic biases are the most prominent patterns in this theme. In particular, the online monitoring process and the biometric-based multi-layered data collection applications within the process have made user dissatisfaction apparent. Again, in the educational context, it is seen that AI systems still have significant technical and pedagogical limitations. The presence of technical glitches disrupts assessment processes. Finally, one of the most critical issues emerging for surveillance technologies is that these systems increase students’ stress and anxiety levels.
Under this theme, alongside the marked increase in the technical performance of AI-SOAEs, a similar trend is observed in their algorithmic limitations, bias, and ethical vulnerabilities. The prevalence of codes such as “prejudice and discrimination,”“linguistic complexity,”“bias,”“multimedia processing limitations,” and “human rights threats” is noteworthy. This situation indicates that, despite the development of the relevant systems, dimensions such as fairness, transparency, and explainability need to be seriously addressed. In particular, it can be said that performance-oriented models experience problems in adapting to cultural, linguistic, and pedagogical contexts. This situation is a typical reflection of this pattern, where these models have the potential to produce unpredictable outputs despite their high productivity.
Once again, students’ emotional experiences with these systems show a sharp contrast. On the one hand, personalised assessments, instant feedback and adaptive processes create high satisfaction, motivation and a sense of achievement. On the other hand, online surveillance, behavioural monitoring, and automated cheating detection systems increase pressure on students. This situation is evident in the intensity of the codes “stress and anxiety,”“trust issue,” and “student dissatisfaction.” This dual structure shows that, despite AI-SOAE’s pedagogical opportunities, it creates psychosocial costs that play a critical role in the assessment experience.
Opportunities Offered by AI-SOAE Systems
Relevant studies on the opportunities that the use of AI-SOAE systems in higher education could create have been analysed. The findings obtained because of the analysis are presented in Table 3.
Opportunities Offered by AI-SOAE Systems.
Upon examining Table 3, it is observed that four categories have been identified regarding the opportunities that AI-SOAE systems can provide in higher education: “Pedagogical transformation,”“Development of assessment processes,”“Technology integration and digital transformation,” and “Institutional competition and cultural change.” The theme of opportunities highlights the transformative potential of AI-SOAE systems. Pedagogical innovation, internationalisation, and institutional digitalisation processes have been the main areas of opportunity emerging under this theme. The existence of these areas points to the potential of AI-SOAE systems to reshape and transform the assessment culture. Student performance can be measured more accurately with the help of adaptive exams and alternative assessment tools. It also offers strategic opportunities for institutions in terms of digital leadership, competitive ability, and international visibility.
Threats Posed by AI-SOAE Systems
Studies related to identifying threats to the use of AI-SOAE systems in higher education have been analysed. The findings obtained because of the analysis are presented in Table 4.
Threats Theme Presented by AI-SOAE Systems.
According to Table 4, the threats to AI-SOAE systems in higher education include “Lack of information and awareness,”“Lack of guidance,”“Research gaps,”“Data privacy and security,”“Implementation difficulties,”“Academic integrity concerns” and “Human rights, justice and ethical concerns.” The most prominent elements of this theme were data privacy risks, academic integrity issues and ethical concerns. AI-SOAE systems pose serious ethical risks, particularly in terms of student behaviour monitored during examinations and stored biometric data. False positives, over-surveillance, privacy violations and data security breaches are the most critical threat clusters. However, institutional policy deficiencies and inadequate standards further exacerbate these threats.
This theme strongly highlights a lack of corporate strategy and policy. Numerous studies mention that AI-SOAE systems lack sufficient corporate-level guidelines, ethical principles, transparency protocols, data management policies, or usage guidelines. This situation leads to systems being misinterpreted, misapplied or mismanaged rather than simply not working correctly. This creates a catalytic effect that turns the systems’ weaknesses into threats. Even if technological competence increases, weak usage guidelines and policies remain one of the fundamental patterns that limit the sustainability of evaluation processes.
In general, these patterns demonstrate that a uniform, context-free approach is not feasible in AI-SOAE systems. When technological capacity, pedagogy, user experience, and institutional policies are not addressed collectively, the systems either prove ineffective or generate ethical and psychosocial risks.
Comprehensive Synthesis of SWOT Dimensions
This section aims to provide a comprehensive synthesis of the findings presented in response to the study’s key research questions. Within this scope, the intersections between the strengths, weaknesses, opportunities, and threats related to AI-SOAE systems have been addressed within an interpretative framework. They have been evaluated contextually in terms of the field’s structural dynamics, institutional impacts, and potential for pedagogical transformation.
Areas of Strength and Opportunity
Strengths such as high accuracy value, objectivity in evaluation, and advanced performance measurements directly correspond to codes on the opportunities side, such as transforming the understanding of evaluation and improving learning outcomes. This correspondence demonstrates that the high level of accuracy offered by AI models provides significant momentum for redesigning evaluation processes. Consequently, it is another indicator that teaching processes with AI have undergone not only a technical but also a structural transformation.
The strengths codes of personalised feedback, adaptive assessment, adaptive learning process, and flexibility systematically align with the opportunities codes of improvement of learning outcomes, wider participation, and providing relevance. This relationship demonstrates that personalisation mechanisms play a central role in improving learning outcomes. Furthermore, adaptive assessment designs create a more inclusive assessment atmosphere that responds to different learner profiles. This integrated pattern clearly highlights the personalisation capacity at the heart of the major change occurring in teaching processes through AI-SOAE.
Once again, the codes focussing on Strengths—reduced workload, time saving, and resource saving—strongly align with the Opportunities categories of technology integration in education, leadership in digital transformation, and corporate culture change. This alignment demonstrates that increased productivity is a strategic element that directly fuels organisations’ digital transformation capacity. This, in turn, reveals the potential of AI-supported systems to transform the values, functioning, and adaptation capacity of corporate culture.
Finally, the codes for monitoring learning, timely intervention and adaptive learning process on the Strengths side strongly align with the categories of wider participation, data-driven intervention and relevance on the Opportunities side. This alignment demonstrates that data-driven decision support systems play a critical role in creating inclusive and context-sensitive learning environments.
Weaknesses and Threats Intersection Areas
One of the most striking findings of the SWOT analysis is the very strong negative correlation between weaknesses and threats. This indicates that the technical and ethical risks of AI-SOAE directly translate into threat areas.
Weaknesses such as frequently encountered prejudice and discrimination, algorithmic complexity, and inability to handle linguistic diversity codes; threats such as human rights threats, biased assessment, and algorithmic bias codes overlap strikingly. This overlap constitutes the most critical risk area in the AI-SOAE ecosystem. Misclassifications arising particularly in multilingual and culturally diverse contexts directly translate into the risk of rights violations.
The codes of stress and anxiety, trust issues, and student dissatisfaction on the Weaknesses side strongly correspond with the codes of lack of policy, lack of guidelines, and ethics & consent issues on the Threats side. This correspondence demonstrates that lack of trust and anxiety are directly related to institutional policy deficiencies. Trust issues arise when AI-based assessment systems lack transparency. This lack of trust deepens feelings of anxiety and dissatisfaction among students. Unless institutions develop policies and guidelines, this situation becomes a structural threat and creates cascading risks for the student experience.
The codes for complex system structure, infrastructure requirements and data management issues on the weaknesses side are fully consistent with the codes for data privacy, data security and access problems on the threats side. This situation reveals that technical complexity directly increases data security and privacy risks. The structural complexity of AI systems deepens security issues, seriously raising the threat level. Similarly, architectural complexity creates another threat area by making security vulnerability risks inevitable.
Finally, the codes limited performance identification, limited pedagogical knowledge, and algorithmic complexity on the Weaknesses side directly correspond to the codes lack of research for development, lack of experimental research, and differentiation of measurement objectives on the Threats side. In other words, limitations in the research infrastructure and gaps in pedagogical knowledge clearly constrain the theoretical development of the field. Consequently, the inadequacy of development-oriented research reveals a critical area that directly threatens the theoretical development of the field.
In conclusion, the relationships between the SWOT dimensions demonstrate that the AI-SOAE ecosystem is not merely a technical innovation area; it is also a multi-layered transformation process requiring ethical, governance, organisational, and pedagogical decisions. While strengths support opportunities, weaknesses have a structure that directly feeds threats. Therefore, the sustainable development of the field must be based not only on technical accuracy and efficiency but also on transparency, fairness, data security, and open policy frameworks.
Discussion
This study examined the strengths, weaknesses, opportunities, and threats of AI-SOAE systems within the framework of SWOT analysis. In this context, 98 studies obtained from the WoS, Scopus, and ERIC databases were subjected to content analysis. The analysis results provided a comprehensive assessment addressing the pedagogical and technological impacts of AI-SOAE systems together. The discussion section relates the fundamental components of the SWOT analysis to TAM and assessment approaches.
The findings reveal that AI-SOAE systems in higher education have significant potential in terms of personalised feedback, adaptive assessment, advanced performance measurement, and data-driven decision support processes. The prominence of codes such as high academic performance, high accuracy value, and advanced performance measurement indicates that these systems not only increase automation in measurement processes but also create a qualitative transformation in measurement accuracy. This transformation is related to AI’s capacity to pedagogically restructure assessment processes and transform measurement approaches (Luckin & Cukurova, 2019). The integration of AI into measurement and assessment processes and its impact on the validity, reliability, and fairness dimensions of these processes have been discussed in the literature through application-focussed evaluations (Mpolomoka, 2025). The use of learning analytics and deep learning-based models enables the multidimensional monitoring of student performance through processes and learning behaviours, contributing to the production of more consistent and objective assessment results. In this respect, the findings are consistent with studies showing that AI-SOAE systems increase scoring consistency and pedagogical explainability (González-Calatayud et al., 2021; Memarian & Doleck, 2024; Zhang et al., 2023).
The prevalence of instant and personalised feedback mechanisms demonstrates that AI-SOAE systems are highly compatible with the formative assessment approach. Providing timely feedback and identifying learning gaps at an early stage enables the learning process to be regulated and assessment practices to take on a continuous, technology-based structure. This approach aligns with theoretical frameworks that emphasise the supportive role of formative assessment in learning (Black & Wiliam, 1998). It also demonstrates that AI-supported systems implement these feedback loops in a scalable and sustainable manner (Zawacki-Richter et al., 2019). This development transforms examinations from outcome-focussed tools into dynamic pedagogical mechanisms that guide the learning process; it aligns with findings that adaptive assessment systems strengthen the integrity between assessment and learning (Bearman & Luckin, 2020; Gligorea et al., 2023; Kabudi et al., 2021; Seo et al., 2021).
When evaluated within the TAM framework, it is evident that these strengths reinforce the perceived benefits of AI-SOAE systems, thereby supporting their adoption. However, current critiques that overly automated applications, unsupported by context-sensitive pedagogical design, may weaken pedagogical judgement should be considered (Chiu et al., 2021; Pedro et al., 2019; Selwyn, 2019; Williamson & Eynon, 2020). The fact that AI-SOAE systems are mostly developed based on specific linguistic, cultural, and pedagogical norms raises discussions about fairness, inclusivity, and legitimacy in different educational contexts. Especially in multilingual and culturally diverse environments, algorithmic applications disconnected from context can weaken the interpretive and ethical validity of assessment results. This situation is directly related to the principles of context sensitivity, impartiality, and use-based validity emphasised in the assessment literature (Pedro et al., 2019).
Despite advances in the technical capabilities of AI-SOAE systems, they exhibit significant weaknesses due to various technical issues, algorithmic biases, and negative user experiences. Applications that collect biometric and behavioural data integrated with online monitoring, while aiming to increase assessment reliability, can have negative effects on user satisfaction and psychosocial experience. Indeed, the literature reveals that online monitoring systems can undermine perceived fairness and acceptability in measurement processes (Dendir & Maxwell, 2020; Kharbat & Abu Daabes, 2021; Williamson & Eynon, 2020). From the perspective of assessment theories, this situation demonstrates that interpretive and ethical validity, as well as scoring reliability, are integral components of assessment processes (Kane, 2013). The intensity of algorithmic bias, linguistic complexity, and multi-modal processing limitations makes it difficult for AI-SOAE systems to perform consistently across different cultural, linguistic, and pedagogical contexts. These findings are consistent with studies highlighting that performance-oriented AI models operating independently of context can produce unpredictable and unfair outputs, despite high accuracy rates (Blodgett et al., 2020; Kizilcec & Lee, 2022; Selwyn, 2019). The fact that even technically consistent assessment results can violate the principles of impartiality and explainability demonstrates that these weaknesses constitute a fundamental problem area in terms of measurement and assessment. The dual structure observed in students’ emotional experiences with these systems is also noteworthy; personalised feedback and adaptive processes enhance motivation and perceived success, while online surveillance and automated cheating detection mechanisms can increase stress, anxiety, and trust issues. This situation indicates that the assessment experience has become an area of tension between pedagogical benefits and psychosocial costs. When assessed in the TAM context, it is seen that these perceived weaknesses can limit behavioural intent by undermining perceived ease of use and trust dimensions, even if perceived benefits are high.
Current findings also reveal that AI-SOAE systems offer significant opportunities in the contexts of pedagogical transformation, the development of assessment processes, digital transformation, and institutional competitiveness. The ability to assess student performance in a manner sensitive to individual learning pace and cognitive level through adaptive tests and alternative assessment tools supports a shift in the assessment culture from outcome-focussed approaches to process-focussed and formative models. This shift aligns with assessment theories that treat assessment as an integral component of learning and demonstrates that AI-SOAE systems can be used as tools that strengthen construct and content validity (González-Calatayud et al., 2021). The trend towards globalisation necessitates consideration of ethical and cultural implications alongside pedagogical opportunities. Algorithms that are not designed to be culturally neutral can lead to issues of validity, fairness, and inclusivity in different educational contexts. Therefore, ethical validity, algorithmic transparency, and cultural sensitivity are among the fundamental prerequisites for global adoption. At the institutional level, the opportunities for digital transformation and technology integration offered by AI-SOAE systems point to a strategic potential that could increase the competitiveness and international visibility of higher education institutions. Within the TAM framework, this indicates that the likelihood of adoption increases with the strengthening of the perceived benefits and facilitating conditions dimensions (Davis, 1989). However, the literature emphasises that digitalisation initiatives that are not integrated with pedagogical vision, ethical frameworks, and institutional policies may remain superficial (Yuskovych-Zhukovska et al., 2022). The findings obtained within this framework reveal that AI-SOAE systems can only create sustainable and meaningful transformations when integrated with pedagogical design, assessment principles, and institutional strategies. Otherwise, the pedagogical and technological opportunities offered by these systems risk remaining limited to superficial digitalisation applications.
On the other hand, the findings indicate that despite the technical and pedagogical potential of AI-SOAE systems, they face serious threats in terms of data privacy and security, academic integrity, ethics and human rights. The monitoring of behavioural and biometric data in examination processes, while aiming to increase assessment reliability, carries with it the risks of false positives, excessive surveillance and privacy violations. Current literature emphasises that AI-supported surveillance and cheating detection systems, while providing technical accuracy, can undermine perceived fairness and ethical acceptability (Kharbat & Abu Daabes, 2021; Williamson & Eynon, 2020). From the perspective of measurement and evaluation theories, this situation once again highlights that, in addition to reliability, ethical and interpretive validity are also fundamental components of the evaluation process (Kane, 2013). The fact that a significant portion of the threats stem from institutional policy, guidance, and standard deficiencies rather than technology demonstrates the critical importance of the governance dimension. The absence of clear ethical principles, transparency protocols, and data management policies paves the way for the misapplication of systems and the transformation of weaknesses into threats. Indeed, the literature clearly shows that despite a robust technical infrastructure, AI-SOAE systems cannot be operated in a sustainable and fair manner without adequate governance mechanisms (Floridi et al., 2018; Pedro et al., 2019; Selwyn, 2019; Williamson & Eynon, 2020). From a TAM perspective, it is seen that these threats weaken perceived trust, thereby increasing perceived risks and ultimately limiting behavioural intent; ethical uncertainties and privacy concerns can negatively affect adoption even when perceived benefits are high (Davis, 1989). In this context, it is clear that uniform and context-independent AI-SOAE applications are unsustainable. For effective and equitable integration, technological capacity must be addressed holistically through pedagogical design, user experience, and institutional governance mechanisms.
The interaction between the strengths of AI-SOAE systems and the opportunities they offer points to a structural transformation in the understanding of measurement and evaluation. The intersection of strengths such as high accuracy value, objectivity in evaluation, and advanced performance measurement with the redefinition of evaluation processes and the improvement of learning outcomes demonstrates that these systems have become a pedagogical tool beyond technical improvement in measurement. The strengthening of validity and reliability, coupled with the acquisition of a formative function that guides learning, points to a transformation consistent with assessment theories. This transformation is also consistent with recent studies showing that AI and learning analytics-based systems can support continuous feedback, adaptability, and learning-focussed assessment processes (Gligorea et al., 2023; Kabudi et al., 2021; Zhang et al., 2023). This is further supported by studies emphasising that AI-SOAE systems enable more consistent tracking of learning outcomes due to their high levels of accuracy and objectivity (Bearman & Luckin, 2020; González-Calatayud et al., 2021). The intersection of strengths such as personalised feedback, adaptive assessment, and flexibility with opportunities for improving learning outcomes, increasing participation, and contextual appropriateness enhances the institutional applicability of learner-centred and inclusive assessment approaches. In this respect, the findings are consistent with the literature emphasising that assessment designs sensitive to different learner profiles strengthen inclusivity (Gombert et al., 2024). The transformation of assessment from a standardising screening mechanism to an adaptive pedagogical guidance tool demonstrates that AI-SOAE systems can assume a strategic regulatory function guiding the learning ecosystem. From a TAM perspective, this intersection of strengths and opportunities increases perceived benefits, supporting adoption at both the individual and institutional levels. The alignment of strengths such as reduced workload, time saving, and resource saving with opportunities for technology integration and digital leadership in education reveals that increased productivity is both a result and a trigger of organisational digital transformation. However, transformation initiatives not supported by pedagogical and ethical frameworks risk remaining superficial. The convergence of strengths such as monitoring learning, timely intervention, and adaptive learning processes with opportunities for data-driven intervention, wider participation, and relevance demonstrates that data-driven decision support mechanisms play a critical role in the formation of inclusive and context-sensitive learning environments. When considered within this holistic framework, it becomes apparent that AI-SOAE systems can be positioned not merely as technical measurement tools, but as strategic pedagogical regulators guiding the learning ecosystem.
Finally, one of the noteworthy findings of this study is the distinctiveness of structural relationships that explain how weaknesses in AI-SOAE systems directly translate into threats. The overlap between weaknesses such as prejudice and discrimination, algorithmic complexity, and inability to manage linguistic diversity with human rights threats, biased assessment, and algorithmic bias demonstrates how technical limitations evolve into pedagogical and ethical risks. In contexts with high multilingual and cultural diversity, the risk of misclassifications becoming rights violations highlights that not only reliability but also impartiality and ethical validity are fundamental quality criteria in measurement and assessment. This finding is consistent with the literature emphasising that the context-independent use of AI-SOAE systems can generate fairness issues (Blodgett et al., 2020; Kizilcec & Lee, 2022; Pedro et al., 2019; Selwyn, 2019).
Similarly, the overlap between weaknesses such as stress and anxiety, trust issues, and student dissatisfaction with threats such as policy gaps, inadequate guidance, and ethics and consent issues demonstrates that students’ negative emotional experiences are a structural problem stemming from institutional governance deficiencies rather than individual ones. When transparency and explainability are not ensured, AI-SOAE systems increase anxiety and insecurity rather than generating trust. In the TAM context, this reveals that perceived risk and low trust limit behavioural intent, even when perceived benefits are high (Davis, 1989). From a technical perspective, the overlap of weaknesses such as complex system structure, infrastructure requirements, and data management issues with data privacy, data security, and access problems demonstrates that system complexity directly increases privacy and security risks. The architectural complexity of AI-SOAE systems makes security vulnerabilities inevitable, threatening the sustainability of evaluation processes and potentially rendering even technically consistent results ethically and legally indefensible. Furthermore, the convergence of weaknesses such as limited performance identification, restricted pedagogical knowledge base, and algorithmic complexity with a lack of development-oriented and experimental research indicates that the theoretical and methodological development of the field is insufficiently supported. The limited nature of experimental and development-based research weakens the pedagogical depth of AI-SOAE systems, causing this deficiency to become a permanent structural threat.
This comprehensive assessment reveals that the sustainability of AI-SOAE systems depends on their integration with pedagogical knowledge, ethical frameworks and corporate governance rather than technical capacity. Unless holistic approaches are developed that can prevent weaknesses from becoming threats, even systems with high accuracy and automation levels will struggle to deliver fair, reliable, and acceptable assessment practices. In this regard, higher education institutions must consider AI-SOAE systems not only in terms of technical performance criteria but also in conjunction with governance structures based on ethical principles, transparency protocols, and context-sensitive assessment policies. Open guidelines, data management standards, and teaching staff support mechanisms developed at the institutional level play a critical role in ensuring that these systems can be implemented in a fair, reliable, and sustainable manner. Otherwise, despite their pedagogical potential, AI-SOAE applications will have a limited impact in terms of institutional legitimacy and user trust.
Conclusion and Recommendations
This study comprehensively evaluates the current state of AI-SOAE systems in higher education within the framework of a SWOT analysis, considering both pedagogical and technical dimensions. The SWOT framework integrates the conceptual structure of the study along two complementary axes. The pedagogical dimension (student-centred learning, self-regulation skills, formative assessment, and learning sustainability) and the technical dimension (algorithmic accuracy, data security, system transparency, and adaptive assessment). This integrated structure enables a multidimensional examination of the acceptance, use, and impact of AI-SOAE applications on learning outcomes, consistent with TAM and assessment approaches.
AI-SOAE systems demonstrate the potential to make meaningful contributions to the quality of education in higher education. Among their strengths are personalised learning and assessment processes, advanced assessment capabilities that support validity and reliability, and data-driven decision-making mechanisms. Conversely, technical infrastructure deficiencies, algorithmic biases, and user experience issues are emerging as key factors limiting the effectiveness of these systems. In this context, it is critically important not only to resolve technical issues but also to improve algorithms with pedagogical and ethical sensitivity and to develop user-centred designs.
The research highlights that AI-SOAE systems offer significant opportunities in terms of pedagogical transformation, digital integration, and sustainable learning. However, data privacy and security, ethical risks, lack of knowledge and awareness, and structural challenges related to implementation emerge as the main threats that could limit the realisation of these opportunities. Therefore, the creation of sustainable learning environments depends on addressing weaknesses and threats through comprehensive policies, approaches that increase stakeholder awareness, and strategies based on ethical principles, in addition to supporting strengths.
The analysis goes beyond merely describing strengths and weaknesses, also demonstrating how these elements can be strategically linked to opportunities and threats. For example, strengths such as personalised feedback, data-driven decision support, and adaptive assessment have been matched with opportunities for pedagogical transformation, digital integration, and institutional competitiveness. This pairing can support students in developing self-awareness and responsibility in their learning processes. Similarly, strengths such as transparent feedback mechanisms can serve to reduce the lack of confidence and ethical concerns that arise among students. In this respect, SWOT analysis is positioned in the study not only as a descriptive tool but also as a strategic and guiding one.
The results obtained demonstrate that strengths and opportunities support the sustainability of learning. The instant and valid feedback provided by AI-SOAE systems enhances student success. Similarly, regular assessment cycles and adaptive learning pathways reinforce lifelong learning habits. However, weaknesses such as technical infrastructure deficiencies, algorithmic biases, and user knowledge gaps, as well as threats such as policy gaps, uncertainties regarding data privacy, and research gaps, may jeopardise this sustainability. Therefore, the creation of sustainable learning ecosystems appears to be possible not only by developing strengths but also by addressing weaknesses and threats with long-term and holistic strategies. A summary of the results obtained is presented in Figure 3.
SWOT analysis of AI-SOAE.
The study offers a unique contribution by adapting the SWOT analysis as a strategic framework for AI-SOAE systems, as opposed to fragmented assessments in the literature. The joint consideration of pedagogical and technical factors enables a holistic evaluation of the systems’ potential and risks. In this context, the study offers a conceptual and strategic perspective on how AI-SOAE systems can be positioned in higher education. Going beyond describing the current situation, it proposes a guiding framework for sustainable, ethical, and effective AI integration in education. This approach generates actionable and implementable insights for educators, policymakers, and researchers.
At the policy and implementation level, universities must develop institutional policy frameworks that include clear ethical guidelines, data security standards, and transparency principles for the use of AI-SOAE. In terms of educator support, the pedagogical integration of AI-based tools should be strengthened through continuous professional development programmes for faculty members and teacher candidates. In terms of technical improvement, justice-oriented algorithms and transparent model designs aimed at reducing algorithmic bias and accuracy issues should be prioritised. Within the research agenda, it is important to increase empirical studies that include pedagogical and ethical dimensions, are conducted in different cultural contexts, and examine long-term effects. In terms of strategic integration, strengths must be systematically integrated into institutional plans in a way that maximises opportunities and minimises threats.
Supplemental Material
sj-docx-1-sgo-10.1177_21582440261423669 – Supplemental material for Artificial Intelligence Supported Online Assessment and Evaluation in Higher Education: A Systematic Review With SWOT Analysis
Supplemental material, sj-docx-1-sgo-10.1177_21582440261423669 for Artificial Intelligence Supported Online Assessment and Evaluation in Higher Education: A Systematic Review With SWOT Analysis by Ayşin Gaye Üstün, Bünyami Kayalı and Mehmet Yavuz in SAGE Open
Footnotes
Appendix
Literature Review: Related Studies Examined Within the Scope of the Study
| Author(s) | Publication country | Journal | Method | Sample and size | Data collection instrument | AI tool/algorithm |
|---|---|---|---|---|---|---|
| Gil et al. [1] | Spain | Sustainability | Quantitative | Data Set | N/A | Logistic regression (LR), Decision Trees (DT), Artificial Neural Network (ANN) and Support Vector Machines (SVM) |
| Daftary et al. [2] | USA | Pharmacy Education | Mixed | University Student | System Records | ExamSoft, ExamID, ExamMonitor and ProctorU |
| Popchev & Orozova [3] | Bulgaria | Cybernetics And Information Technologies | Quantitative | Data Set | N/A | Orange Data Mining System |
| Guerrero-Roldán et al. [4] | Spain | International Journal of Education Technology in Higher Education | Mixed | University Students (n = 552) | Survey | Naive Bayes (NB), Decision Tree (DT), K-Nearest Neighbors (KNN) and Support Vector Machine (SVM) for Graded Risk Model (GAR model) training |
| Wu et al. [5] | China, Russia | International Journal of Emerging Technologies in Learning | Quantitative | University Students (n = 50) | Multiple Choice Test | LSA |
| Coghlan et al. [6] | Australia | Philosophy & Technology | Qualitative | N/A | N/A | N/A |
| Muzaffar et al. [7] | Saudi Arabia, Pakistan, Saudi Arabia | IEEE Access | Qualitative | Article (n = 53) | N/A | N/A |
| Mutawa & Sruthi [8] | Kuwait | Journal of Engineering Research | Quantitative | University Students (n = 478) | Online Survey | N/A |
| Bagunaid et al. [9] | Australia | Sustainability | Quantitative | OULAD-VLE Dataset | N/A | RNN, DBSCAN, TMR, R-SARSA |
| Hooda et al. [10] | India, Saudi, Arabia, Bangladesh | Hindawi Mathematical Problems in Engineering | Mixed | OULAD-VLE Dataset | N/A | Decision Trees, Random Forest, Support Vector Machine (SVM), XG Reinforcement, Artificial Neural Network (ANN, Improved Fully Connected Network (I-FCN) |
| Martín-Marchante [11] | Spain | Research in Education and Learning Innovation Archives | Quantitative | Instructor (n = 53), University Student (n = 101) | Survey | N/A |
| Sefcik et al. [12] | Australia | Australasian Journal of Educational Technology | Mixed | University Student (n = 253) | Survey, Interview Form | CRIS |
| Hemachandran et al. [13] | India | Computational Intelligence and Neuroscience | Quantitative | Data Set (n = 114) | Midterm exam results and final evaluation scores | logistic regression, K-nearest neighbor, classification and regression trees |
| Falla-Falcón et al. [14] | Spain | International Journal of Environmental Research and Public Health | Quantitative | Specialist Education Inspector (n = 242) | Scale | Fuzzy Logic, SULODITOOL |
| Hamadneh et al. [15] | Saudi Arabia | Sustainability | Quantitative | Data Set (n = 234) | Mid-exams, assignments, attendance, and virtual/face | Artificial Neural Network (ANN). Firefly Algorithm (FFA) |
| Li & Shu [16] | China | Wireless Communications and Mobile Computing | Quantitative | University Student (n = 95) | Achievement test, Survey | Smart Assessment System (http://www.pigai.org/) Intellimetri, E-Rater, Writing Roadmap and My Access |
| Lee et al. [17] | South Korea, Malaysia | International Journal on Advanced Science, Engineering & Information Technology | Quantitative | University Student (n = 198) | Online Survey | N/A |
| Naidu & Sevnarayan [18] | South Africa | Online Journal of Communication and Media Technologies | Qualitative | N/A | N/A | N/A |
| Fidas et al. [19] | Germany, Cyprus, Portugal, South Africa | Education Sciences | Qualitative | University Student (n = 133) | Interview Form, Usability Survey | TRUSTID |
| Hobbs-Koch et al. [20] | Germany | The Journal of Teaching English for Specific and Academic Purposes | Quantitative | University Student | Student Writing Portfolios, Assessment Tests, Student Feedback Forms and teacher observations | QuillBot, Antconc, Coca, Onelook |
| Li [21] | China | Australasian Journal of Educational Technology | Quantitative | University Student (n = 81) | Scale, Interview Form, Evaluation Rubric | ChatGPT |
| Rudolph et al. [22] | Singapore, Civica Asia Pacific | Journal of Applied Learning and Teaching | Qualitative | Sources (n = 166) | N/A | N/A |
| Ryznar [23] | Poland | Washington and Lee Law Review Online | Qualitative | N/A | N/A | N/A |
| Gonsalves [24] | England | Journal of Learning Development in Higher Education | N/A | N/A | N/A | ChatGPT |
| Fergus et al. [25] | England | Journal of Chemical Education | Mixed | Instructor | N/A | ChatGPT |
| Sullivan et al. [26] | Australia | Journal of Applied Learning & Teaching | Qualitative | Article (n = 100) | N/A | N/A |
| Nikolic et al. [27] | Australia | European Journal of Engineering Education | Qualitative | University student, instructor | N/A | ChatGPT |
| Zheng et al. [28] | China, USA | Journal of Research on Technology in Education | Mixed | University student (n = 135) | Survey, Interview | Knowledge Graph Convolutional Network (KGCN) |
| Perla & Vinci [29] | Italy | Open and Interdisciplinary Journal of Technology, Culture and Education | Mixed | University student and instructor (n = 200) | Interview Form, Survey, System Reports | N/A |
| Halkiopoulos & Gkintoni [30] | Greece | Electronics | Qualitative | Article (n = 85) | N/A | N/A |
| Eva et al. [31] | Bangladesh | Journal of Research in Innovative Teaching & Learning | Qualitative | University student (n = 28) Instructor (n = 15) ICT specialist | Interview Form | ChatGPT, Gemini, QuillBot, Turnitin, CopyAI, Inc., Ahrefs, Gradescope |
| Foung et al. [32] | China | Computers and Education: Artificial Intelligence | Qualitative | University student (n = 74) | Interview Form | Just the Word, WeCheck, QuillBot, Grammarly, Grammarly, ChatGPT |
| Jose & Jose [33] | Oman | Electronic Journal of e-Learning, | Qualitative | Instructor (n = 35) | Online Forum Discussion | N/A |
| Markos et al. [34] | Greece | Electronics | Mixed | University student (n = 257) | Questionnaire, Answer Form | ChatGPT, Decision Tree Algorithm |
| Sevnarayan [35] | South Africa | Journal of Pedagogical Research | Qualitative | University student (n = 19), Instructor (n = 5) | Observation Form, Interview Form | ChatGPT |
| Sevnarayan & Potter [36] | South Africa | Journal of Applied Learning & Teaching | Qualitative | University student (n = 12), Administrative Staff (n = 5) | Interview Form | ChatGPT |
| Van den Berg [37] | South Africa | Open Praxis | Qualitative | Instructor (n = 13) | Interview Form | ChatGPT |
| Susnjak & McIntosh [38] | New Zealand, Australia | Education Sciences | Mixed | N/A | N/A | ChatGPT |
| Newton & Xiromeriti [39] | England | Assessment & Evaluation in Higher Education | Mixed | Question Set (n = 114) Article (n = 53) | N/A | ChatGPT |
| Moşteanu [40] | Malta | Quality-Access to Success | Qualitative | University Students and Professors (n = 100) | Interview Form | N/A |
| Hadzhikoleva et al. [41] | Spain | Frontiers in Education | Theoretical/conceptual model development | N/A | N/A | ChatGPT |
| Doğan [42] | USA | Asian Journal of Distance Education | Design Based Approach | K-12 Teachers, Lecturer, Instructional designers, Programme developers (n = 500) | Rubric | ChatGPT |
| Conrad et al. [43] | USA | Open Journals in Education | Mixed | University Students (n = 29) | Scale | EpiBot |
| Yan et al. [44] | Finland, Canada, USA | Canadian Journal of Learning and Technology | Quantitative | Student dataset (n = 8,000) | Interview form | ZPD-KT ve CAP |
| Du & Butkaew [45] | China | Scientific Reports | Quantitative | University student | Academic performance test, System usage logs, Feedback surveys | LSTM, Deep learning |
| Bauer et al. [46] | Germany | Journal of Computer Assisted Learning | Quantitative | University student (n = 395) | Rubric, Achievement test | NLP, BiLSTM-CRF |
| Marino & Cabezuelo [47] | Spain | Algorithms | Quantitative | University student (n = 300) | Achievement test, survey | ChatGPT, Gemini, DeepSeek, Grok, Copilot, Grammarly, QuillBot, ElsaSpeaki Canva, Gamma, Lumen5, Julius |
| Uğur et al. [48] | Türkiye | Turkish Online Journal of Distance Education | Qualitative | Instructor (n = 4) | Interview forms | N/A |
| El Hadbi et al. [49] | Morocco | Data and Metadata | Quantitative | University student (n = 1,420) | Survey | N/A |
| Spivakovskiy et al. [50] | Ukraine | Information Technologies and Learning Tools | Qualitative | Dataset (n = 343) | Survey, Visuals, Feedback system | ChatGPT |
| Tenakwah et al. [51] | Australia | Knowledge Management & E-Learning | Qualitative | University student & Instructor (n = 22) | Homework, ChatGPT responses, Rubrics, Reflective questions | ChatGPT |
| Wen & Pan [52] | China | PeerJ Computer Science | Quantitative | University student (n = 200) | Test, System logs | Fuzzy Bayesian Intelligent Tutoring System (FB-ITS) |
| Klyshbekova & Abbott [53] | UK | Electronic Journal of e-Learning | Quantitative | Dataset | Prompts, Essay, Rubric | ChatGPT |
| Küçükali [54] | Türkiye | Information Technologies and Learning Tools | Mixed | Teacher (n = 53) | Survey, Interview form | ChatGPT |
| Humble et al. [55] | Sweden | Electronic Journal of e-Learning | Qualitative | Instructor (n = 6) | ChatGPT logs | ChatGPT |
| Materazzini et al. [56] | Italy | Information | Quantitative | University student (n = 80) | Reading test and scale | Supervised ML |
| Kılınç [57] | Türkiye | Journal of Education Technology & Online Learning | Qualitative | Document | Document review form | N/A |
| Eli et al. [58] | USA | Digital | Quantitative | Dataset (n = 32,593) | OULAD | Decision Support System |
| Sharif et al. [59] | Pakistan, Australia | Sustainability | Mixed | 42 Document, 12 Instructor | Survey, Document | N/A |
| Liu & Schwieger [60] | USA | Information Systems Education Journal | Quantitative | Dataset (n = 3,000) | N/A | Graph mining, Network clustering |
| Bootchuy & Amornrit [61] | Thailand | Journal of Education and Learning | Quantitative | University student (n = 32) | Test, Interview form | AI Chatbot |
| Juárez et al. [62] | Spain | Algorithms | Quantitative | University student (n = 35) | Automatic assessment + AI feedback + FQI | Agentic RAG |
| Santos et al. [63] | Portugal, Spain | Electronic Journal of e-Learning | Mixed | Expert (n = 45) | Survey | N/A |
| Ari et al. [64] | USA | Research & Practice in Assessment | Quantitative | University student (n = 49) | Test, Quiz | ChatGPT |
| Prinz et al. [65] | Germany | International Endodontic Journal | Quantitative | University student (n = 164) | Survey | ChatGPT |
| Tseng & Lin [66] | Tayvan | The Electronic Journal of e-Learning | Qualitative | University student (n = 15) | Reflective texts | ChatGPT |
| Elfirdoussi et al. [67] | Fass | Engineering, Technology & Applied Science Research | Qualitative | Document | Document review form | N/A |
| Ismail et al. [68] | Uman | Journal of Education and e-Learning Research | Quantitative | University student (n = 74) | Test | Learn Smart, IRT–Rasch (1PL) |
| Dushyanthen et al. [69] | Australia/BK | BMC Medical Education | Mixed | Healthcare worker (n = 21) | Survey, Scale | N/A |
| Casalino et al. [70] | Italy | Evolving Systems | Quantitative | Dataset | Student dataset | DISSFCM |
| Chan et al. [71] | Indonesia | IAES International Journal of Artificial Intelligence | Quantitative | University student (n = 20) | Survey, Scale | ChatGPT, Gemini, Perplexity, Bing, You |
| Musyaffi et al. [72] | Indonesia | Journal of Education and Learning | Quantitative | University student (n = 160) | Survey, Scale | AI based ERP |
| Jogezai et al. [73] | Pakistan | Turkish Online Journal of Distance Education | Qualitative | Instructor (n = 33) | Structured interview form | N/A |
| Nurdiana et al. [74] | Malaysia | International Journal of Evaluation and Research in Education | Quantitative | Adult learners (n = 460) | Survey | ChatGPT, Gemini, Perplexity, Bing, You |
| Cao et al. [75] | China | Scientific Reports (Nature) | Quantitative | Q&A datasets | Student responses | Siamese LSTM + Attention, Word2Vec, Manhattan distance |
| Islam et al. [76] | Indonesia | IAES International Journal of Artificial Intelligence | Quantitative | University student (n = 20) | Student assignments, expert evaluation forms, Likert scales | ChatGPT, Gemini, Perplexity, Bing, You.com |
| Öncü & Süral [77] | Türkiye | Asian Journal of Distance Education | Mixed | Open university students (n = 374) | Survey, Focus group discussion forms | Chatbot |
| Stillman [78] | USA | Journal of Educators Online | Theoretical/conceptual | Document | Document review form, sample questions | N/A |
| Bueno [79] | Philippines | Institutional Multidisciplinary Research and Development Journal | Quantitative | Postgraduate students | Assessment tool | G-SPACE AI Connect |
| Murad et al. [80] | Indonesia | Journal of Educators Online | Quantitative | University student | LMS data, grade records | Gradient Boosted Trees, GLM, DL, DT, RF, SVM |
| Muoaeweed et al. [81] | Saudi Arabia | Journal of Applied Research and Technology | Qualitative | Instructor (n = 13) | Semi-structured interview form | Chatbots, Learning analytics |
| Pozza et al. [82] | Portugal | Education Science | Mixed | University student (n = 989) & Instructor (n = 266) | Survey, Open-ended questions | N/A |
| Almpanis et al. [83] | United Kingdom | Electronic Journal of e-Learning | Qualitative | Instructor (n = 12) | Open-ended online survey | N/A |
| Aybek & Okur [84] | Türkiye | International Journal of Assessment Tools in Education | Quantitative | Open education student data (n = 195.584) | Student datasets | ANN, MLP, RBF |
| Oliveira et al. [85] | Poland | Electronic Journal of e-Learning | Quantitative | LMS student data | Dataset | ANN/ML-based prediction model |
| Aldhahri et al. [86] | Saudi Arabia, Pakistan | PeerJ Computer Science | Quantitative | Student dataset | Educational dataset | XSEJNet (ResNeXt + SE Attention + Jaya Optimization) |
| Yuan [87] | China | AIP Advances | Design Based Approach | Document | Multi-mode content | Hamming distance, semantic feature extraction |
| Alotaibi et al. [88] | Saudi Arabia | Journal of King Saud University – Science | Quantitative | N/A | N/A | CNN |
| Ferrer et al. [89] | Spain | Digital | Mixed | University student (n = 407) | Survey | Generative AI tools such as ChatGPT |
| Rädel-Ablass et al. [90] | Germany | BMC Medical Education | Quantitative | University student (n = 100) | Survey | GPT-4 based chatbot |
| Lu et al. [91] | Vietnam | Education Science | Quantitative | Student (n = 303) | Survey | K-means, Random Forest, SVM, KNN, Naive Bayes, MLP |
| Nhan [92] | Vietnam | Turkish Online Journal of Distance Education | Mixed | Student (n = 155) | Survey, Interview form | ML & NLP based systems |
| Plessis [93] | South Africa | Tydskrif vir Geesteswetenskappe | Qualitative | Document | N/A | ChatGPT |
| Chan et al. [94] | Hong Kong, Singapur | BMC Medical Education | Mixed | Student (n = 70) | Survey, System dataset | ChatGPT |
| Ahmad et al. [95] | India | Proceedings on Engineering Sciences | Quantitative | Student (n = 605) | Survey | K-means clustering, LPNM |
| Zevallos et al. [96] | Peru | Journal of Computer Science | Design Based Approach | Expert (n = 20) | Survey | AI-based virtual learning system; personalisation and academic management automation |
| Lin & Lin [97] | Taiwan | Engineering Proceedings | Design Based Approach | University student (n = 20) | Test | ChatGPT |
| Meng & Guo [98] | China, Argentina | Edelweiss Applied Science and Technology | Design Based Approach | Course (n = 200), Student (n = 50), Teacher (n = 4) | User data | LLM, RAG, Neo4j, Top-k samping |
Note. N/A = Not being applicable.
Ethical Considerations
Ethical approval and informed consent were not required in this review article because it did not involve any human or animal participants.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
All data generated or analysed during this study are included in this manuscript.
Generative Artificial Intelligence
The authors acknowledge the use of ChatGPT-4 and DeepL AI tools in the final editing process for language improvement.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
