Sage Journals: Discover world-class research

Abstract

While the market offers a wide variety of intelligent educational software, inconsistent quality often prevents these tools from effectively meeting practical teaching needs. This makes it difficult for educators to determine the suitability of specific software for their classrooms. To address this challenge, this study developed a set of evaluation indicators for intelligent educational software used in primary and secondary schools. These indicators were established through market research, literature review, and feedback from both subject experts and teachers. By comparing and analyzing evaluations of commonly used software from both teachers and educational technology experts, this study aims to understand the current development level and characteristics of intelligent educational software in China. Initial findings reveal that while such software demonstrates strong performance in certain educational scenarios, it falls short in others. Interestingly, there are subtle differences in how teachers and education professionals perceive the software’s effectiveness in specific teaching contexts.

Keywords

intelligent educational software intelligent education software evaluation primary and secondary school teachers’ teaching dual perspective assessment

Introduction

The market for intelligent educational software (IES) is booming with the development of new information technology such as mobile Internet, big data, brain science, and meta-universe. The new emergence of ChatGPT, a generative artificial intelligence, has sparked intense discussions in the education sector about how IES can empower education and promote the digital transformation of education (State Council (PRC), 2021). In 2021, the Ministry of Education released the “Notice on Implementing the Second Batch of AI-assisted Teacher Team Building Action Pilot Work,” emphasizing the need to enhance the use of intelligent assistants (including platforms, systems, resources, and tools) among teachers and students, drive innovations in teaching and learning methods, alleviate teachers’ workload, and empower them. Specifically, it stressed the importance of leveraging intelligent education platform systems to explore and promote dual-teacher classrooms that foster both human-human and human-machine collaboration (State Council (PRC), 2021). In the context of the digital transformation of education, educators should take the lead in adjusting to the shifts brought about by artificial intelligence, informatization, and other technologies. They should also actively and successfully carry out instruction based on intelligent educational platforms, systems, and software.

Generally, IES refers to computer programs designed to facilitate learning and teaching by using advanced technologies like artificial intelligence (AI), machine learning, and data analytics (Do et al., 2018). The development of these IES aims to provide personalized learning experiences, adapt to the needs of individual students, and enhance educational outcomes. Moreover, the current IES that supports teachers in optimizing brilliant teaching mainly involves four aspects: teaching research, lesson preparation, lecturing, and question-answering about homework. In particular, Qi (2021) notes that the intelligent teaching research platform facilitates the management of teaching research activities from rudimentary to sophisticated (Y. Dai et al., 2022). It also facilitates bidirectional interaction and dynamic regulation of teaching and research plans (Y. Dai et al., 2022), bridging knowledge gaps between novice and experienced teachers in terms of instruction quality and experience (Z. X. Li, 2020), thereby enhancing the potential for profound discussions in teaching and research activities (W. F. Dai et al., 2014). Hence, the development of IES should serve the education itself, such as teaching, learning, and study management (Tian & Li, 2013).

The market for software used by primary and secondary school teachers is complicated and its quality varies. Many educational software products focus too much on features and not enough on meeting teachers’ practical needs. Current research on evaluating this software is limited, as many evaluation criteria don’t consider specific classroom scenarios and are often based on developers’ perspectives rather than teachers’ feedback. This study creates evaluation criteria for educational software based on four teaching scenarios: lesson preparation, lecturing, research, and homework assistance. These criteria are used to gather feedback from teachers and education experts on commonly used software. The study compares this feedback to understand the development and features of educational software, aiming to improve its quality in China. The goal is to recommend appropriate use of educational software, guide AI tool regulations, and support education’s digital transformation. Therefore, this study is based on four application scenarios of teacher instruction: lesson preparing, lecturing, teaching research, and question-answering about homework. It constructs evaluation indicators for IES suitable for “teaching” by primary and secondary school teachers. Using these evaluation indicators as a survey tool, frontline teachers and experts in the field of educational technology are invited to score the currently commonly used IES for teaching in primary and secondary schools. Subsequently, by comparing the dual-perspective evaluation data on functional quality and software design.

Literature Review

The Haugland/Shade Software evaluation standard was created in 1991 by Susan W. Haugland and Daniel D. Shade, U.S. experts in early childhood software evaluation. It focused on technology, children, and teachers and was based on the concept of developmentally appropriate practices proposed by the National Association for the Education of Children (Shade, 1991). In 2006, the Center for Early Childhood Education Best Practices at Western Illinois University developed software evaluation indicators for teachers and parents from three aspects: educational value, teaching design, and usability (Robinson et al., 2006). With the rapid development of computer science and software technology, increasing attention has been paid to the functionality and user experience of educational software. Turkish researcher Kara (2007) developed indicators for assessing educational software based on design aesthetics, student participation, content, and ease of use using the hierarchical analysis method. The “Evaluation Indicators for Educational Software” were created by Columbia City Schools in 2008. They comprise an assessment of the syllabus, software design, and technical compatibility. The software design indicators are broken down into four categories: technology, content, ease of use, and learning styles (Paul, 2008). The assessment indicators of educational software are progressively improved in terms of software functionalities, user experience, and scene adaptability as the user needs of IES are continuously updated. Based on two teaching programs for physics classes in vocational schools, the Greek researcher Tsihouridis et al. (2011) assessed the teaching software in terms of content compatibility, scientific preparation of the content, appropriateness of the content, activation of students’ motivation, and well-designed environment. Technical system quality, information quality, service quality, support system quality, learner quality, instructor quality, and perceived usefulness are the factors that determine e-learning perceived satisfaction, according to Al-Fraihat et al. (2020). Based on the Bloom-Anderson Taxonomy, Barari et al. (2022) created and validated 2 educational standards and 18 essential indicators.

Chen and Zhang (2000) developed six evaluation dimensions for computer-aided instruction software in terms of the development of educational software evaluation in China: ease of use and simplicity of software operation; popularity and compatibility of applicable media; presentation and attractiveness of the course software; accuracy of textbook content; feedback techniques and textbook explanations; and the application of media characteristics. Compared with the previously mentioned evaluation indicators, this indicator focuses on the application of software to media characteristics. S. Z. Wang et al. (2002) created a two-tier system of quality evaluation indicators based on the software quality evaluation system of SSC (Shanghai Software Center) and other relevant evaluation models. The primary indicators, which comprise functionality, reliability, ease of use, efficiency, maintainability, portability, and efficiency, make up the first level of the system. When assessing digital teaching software in 2003, B. L. Sun created the indicator system for assessing domestic network (digital) courseware by using the American Society for Training & Development (ASTD) criteria. The courseware was categorized into the methods and standards for evaluating teaching software were investigated by Y. Z. Sun and Duan in 2005. They created six evaluation aspects and the standards that go along with them: pedagogical, artistic, reliable, usable, open efficiency, and modifiability (Y. Z. Sun & Duan, 2005). These aspects take into consideration the three main pedagogical, artistic, and technical aspects. In 2016, Li Jinyu established an indicator for assessing the usability of educational resource software based on the evidence theory, which mainly considered several aspects involved in the influencing factors of usability, and determined a four-aspect indicator system of functionality, validity, ease of use and experience (J. Y. Li, 2017). Therefore, the above experts and scholars at home and abroad have designed and suggested the dimensions of (intelligent) educational software evaluation from different perspectives or educational needs, as shown in Table 1.

Table 1.

Summary of Dimensions of Educational Software Evaluation in China and Abroad.

Evaluation dimensions	Evaluation indicators
Empowering Teaching	Syllabus, Educational design, Educational values, Educational objectives, Educational content, Educational stimulation
Technical quality and functionality	Compatibility, Efficiency, Ease of use, Usability, Friendliness, Practicality, Fun, Artistic, Functional design, Structural design, Interaction design, Navigation design, Interface design, Scalability, Technical level, Technical support, Functionality guarantee
User experience	Learning styles, Learning needs analysis, Participation, Motivation
The quality of the software itself	Efficiency, Expansibility

The current evaluation methods for educational software are outdated and inadequate for assessing modern intelligent educational tools. Traditional criteria focus too much on curriculum and educational design, while overlooking teaching effectiveness, data use, and management. These criteria also fail to consider important factors like software portability and security features, which are crucial for adapting to different educational settings and preventing misuse. Current indicators are impractical, vague, and not comprehensive enough for real-world application. To keep pace with the rapid advancements in artificial intelligence and digital transformation in China, new evaluation criteria specifically for IES are needed (M. Liu & Yu, 2023).

Method

To fulfill its research aims, this study first established the intelligent educational software evaluation indicator system for primary and secondary teachers (IESEIPSTT). This system is designed as a practical framework enabling educators to assess and select software that best aligns with teaching needs, thereby facilitating the effective adoption of intelligent education practices in schools. Building on this framework, the study further employed a dual-perspective evaluation strategy. Specifically, both frontline teachers and educational experts were invited to assess commonly used IES in primary and secondary classrooms. Through this comparative evaluation, the study seeks to identify the developmental status, performance features, and limitations of current software, thereby generating evidence to guide future innovation in intelligent educational tools and to support policy and practice in China’s broader digital education transformation.

Indicators Construction

The development of the IESEIPSTT was a multi-phase process. Phase I focused on the initial establishment of assessment indicators, which were then revised and refined in Phase II, see Figure 1. The primary dimension of evaluation targeted “Teaching” for primary and secondary school teachers, encompassing aspects like lesson preparation, lecturing, teaching research, and question answering about homework. To ensure comprehensiveness, a multi-party consultation was conducted, involving stakeholders such as IES developers, experts in educational technology and software development, and practicing teachers.

Figure 1.

The IESEIPSTT building process.

Through many rounds of iterations, the indicator system evolved. Initially, primary indicators like Functional Realization, Business Fit Value, and Charisma Attribute Value were defined. Subsequently, secondary and tertiary indicators were refined, and the entire system was revised based on feedback gathered during the consultation process. This iterative approach culminated in the establishment of the final IESEIPSTT evaluation indicators.

(1) “Functional Realization Value (FRV),” which is an assessment of the functional quality of software and software design experience. Under the four secondary dimensions of lesson preparation, lecturing, teaching research, and question-answering about homework, the same secondary indicators are set up under the functional realization value. In terms of the functional quality of software, there are six secondary indicators: functionality, reliability, ease of use, maintainability, efficiency, and portability; used to assess whether software products have the ability to maintain performance, the ability to be modified and the ability to migrate from one environment to another, etc.; in terms of software design experience, there are three secondary indicators: priority, consistency, and defensiveness: used to assess whether software products are in line with the principle of prioritized design in terms of designing around the user experience, preventing misuse and abuse of the perspective of design, prevention of misuse and abuse, etc. Each of the secondary indicators under FRV has a rating range of 1 to 5 points.

(2) “Business Fit Value (BFV)” is an assessment of the matching degree of functionality in different educational scenarios, that is, whether the software functionality meets the needs and effects of the application in specific educational scenarios; the secondary and tertiary indicators of BFV under the four secondary dimensions of lesson preparation, lecturing, teaching research, and question-answering about homework are all different. As shown in Figure 2, for example, in the BFV (primary indicator) under the teaching research secondary dimension, there are seven secondary indicators, such as management of teaching research activities, teaching research social interaction, and AI intelligent teaching research, which evaluate whether the software supports the recommendation of teaching and research activities, and the participation in teaching and research activities. Each of the secondary and tertiary indicators under BFV has a score range of 1 to 5.

(3) “Charisma Attribute Value (CAV),” which is an assessment of the innovative functions and applications of IES, has two secondary indicators, namely, Functional Advanced Value and Business Innovation Value. For each charismatic highlight, a bonus point system is adopted, in which each charismatic highlight is full of 5 points, which is taken from the average score of the function advanced value and business innovation value, both of which have a score range of 1 to 5 points, and support up to 5 charismatic highlights, which is full of 25 points, or 0 points if there is none.

With respect to the development level and characteristics of IESEIPSTT, the main component of the assessment software is the major indicator, or BFV. The development of IES can help teachers realize effective smart teaching and promote the digital transformation of China’s education, as the BFV evaluates whether the software functions meet the application needs and effects of various teaching scenarios. Therefore, this section describes in detail the secondary and tertiary indicators under BFV, aiming to provide operational guidance and reference for the IESEIPSTT. Figure 2 illustrates how the BFV (primary indicators) correspond to different sets of core functionality (secondary indicators) and tertiary indicators with varying quantities and contents (see Table A1 in Appendix A for specific tertiary indicators).

Figure 2.

Business fit value indicator system in IESEIPSTT.

In the process of “teaching,” lesson preparation is a carefully designed teaching structure and teaching activity plan based on certain teaching theories, systematic analysis of the teaching value of the subject content, and the accomplishment of teaching goals before the teaching activities (Hu, 2020). Teaching-oriented lesson planning software firstly needs to meet the premise of providing teachers with high-quality lesson planning resources, and can intelligently recommend relevant lesson planning resources and suggestions; secondly, lesson planning software needs to provide teachers with diversified subject tools and micro-recording platforms; in addition to this, lesson planning software should be able to support online remote collective lesson planning. Finally, lesson planning software can establish a good incentive mechanism to stimulate teachers’ enthusiasm for lesson planning. Finally, the six core functions of lesson planning software are identified as teaching material production, lesson planning resource management, teaching material management, micro-lesson recording, incentives for lesson planning, and prepare lessons collection in the business fit values under the dimension of lesson planning.

Secondly, from the demand of teachers’ teaching, the lecture software firstly needs to have the ability to support various classroom modes, such as online classroom, flipped classroom, mixed classroom, etc.; secondly, interaction is the core of classroom, and the lecture software needs to be equipped with various interaction paths; again, the lecture software is able to collect and analyze the data of the learning situation and provide teachers with reliable data support; in addition to this, the lecture software should be able to provide various teaching aids for teachers to save classroom time and improve classroom efficiency with the support of new technologies, such as face recognition and emotion recognition. In addition, based on the interactive whiteboard and other hardware devices, the lecture software should be able to provide teachers with various teaching aids to help them save classroom time and improve classroom efficiency; finally, with the support of new technologies such as face recognition and emotion recognition, intelligent evaluation of classroom atmosphere and teachers’ teaching effect will be the next development direction of the lecture software. Therefore, among the business fit values under the dimension of lecturing, five core function sets are abstracted: teaching assistance, classroom support, learning data collection and analysis, multi-terminal interconnection and teaching effect evaluation.

At the same time, teaching software for homework answering should give full consideration to the scientific and reasonable nature of homework design, give full play to the functions of homework diagnosis, consolidation, and learning situation analysis, and encourage the arrangement of hierarchical, flexible, and personalized homework (State Council (PRC), 2021). Through the extensive collection of high-quality topics, content upgrading and label positioning to create a resourceful and clear topic database, teachers can accurately find test questions according to their needs when searching, and compose assignments that are hierarchical, personalized, and in line with the teaching requirements through the system’s intelligent questioning mode that matches a variety of questioning methods. After the completion of homework, through the process of intelligent correction and data analysis, teachers can understand the students’ answer situation, ability distribution and knowledge mastery in the detailed data analysis report, which facilitates the next step of teaching plan development. Finally, in the business fit value under the dimension of question-answering about homework, the five core functional collections of homework grouping, homework release management, homework correction, online Q&A, and homework statistical analysis are abstracted.

Finally, teaching and research activity management, that is, assessing whether the smart education software can provide teachers with rich teaching and research resources, as well as the function of recommending various forms of teaching and research activities, and supporting teachers to manage the resources and activities. Therefore, among the business fit values under the teaching and research dimension, the seven core function sets of teaching and research activity management, social interaction, remote interactive teaching and research, AI intelligent teaching and research, teaching and research data analysis, micro lesson recording, and web-based listening and evaluation of lectures are abstracted.

Dual Perspective Assessment Process

In the ongoing discourse surrounding the evaluation of IES, there are two pivotal groups whose perspectives are essential: the teachers who are the primary users of these technologies in educational settings, and the experts who design and evaluate these tools. Teachers, through their hands-on experience, offer valuable practical insights and feedback, while education technology experts provide the scientific and theoretical framework necessary for assessing these systems.

As shown in Figure 3, this study designed a questionnaire based on the IESEIPSTT, aiming to assess the most commonly used IES in teachers’ teaching. At the beginning of the questionnaire, it was explicitly stated that the survey was conducted anonymously and that no personal information of teachers would be collected. Submission of the completed questionnaire was regarded as written informed consent from the participants. Between September and October 2022, the study disseminated the questionnaire via email to primary and secondary school teachers across China. Out of 105 responses, 72 were deemed valid, showcasing a robust reliability and validity test with a Cronbach’s alpha coefficient of .98, and KMO values for various teaching dimensions ranging from 0.62 to 0.94. Participants were informed that they could withdraw from the study at any time. The study received ethical approval from the relevant institutional ethics committee.

Figure 3.

The procedure for collecting evaluation data.

Proponents argue that analyzing these teacher responses is key to identifying the most frequently employed IES. However, the debate intensifies when the research team brings in the perspectives of education experts. By contacting 40 experts in the field, organizing them into groups, and having them evaluate the identified software, the study enriches its dataset with expert assessments. Critics may question whether the expert data—comprising evaluations and scores from these specialists—aligns with the practical insights derived from the teacher data. Nonetheless, supporters maintain that the amalgamation of both teacher and expert data forms a comprehensive dataset, crucial for understanding and enhancing the use of IES. This dual-perspective approach fuels the debate on how best to evaluate and implement educational technologies effectively.

In the data analysis stage, this study conducted statistics on the ratings of some primary indicators under the secondary dimensions of the teacher data and the expert data to assess the overall development level of the current commonly used IES from a dual perspective. As shown in Table 2, the ratings of the primary indicators of BFV under the four secondary dimensions of expert data are all lower than 60 points, and the ratings of the primary indicators of FRV are all higher than 75 points. With scores of 57.60 and 54.96, respectively, the BFV primary indicator of the secondary dimension of homework and Q&A has the highest score among them, while the FRV primary indicator of the secondary dimension of teaching and research has the highest score among them, with scores of 57.60 and 54.96, respectively, the secondary dimension of lectures has the lowest. With scores of 80.91 and 75.29, respectively, the primary indication of FRV in the second level dimension of teaching and research has the highest score, while the secondary dimension of teaching has the lowest score. While the FRV primary indicator received more than 60 points, the four secondary aspects of the teachers’ data under the BFV primary indication likewise received less than 60 points. Among them, the secondary dimension of lecturing has the lowest score 54.47 and the BFV primary indicator of the secondary dimension of teaching and research has the highest score 68.67. Similarly, the FRV primary indicator of the secondary dimension of teaching and research has the highest score and the secondary dimension of lecturing has the lowest score 68.67 ranking points.

Table 2.

Teacher’s and Expert’s Data Scored on Selected Primary Indicators Under Secondary Dimensions.

Scoring subjects	Secondary dimensions
	Teaching research		Lesson preparing		Lecturing		Question-answering about homework
	Primary indicator
	BFV	FRV	BFV	FRV	BFV	FRV	BFV	FRV
Expert data	57.57	80.91	57.47	77.63	54.96	75.29	57.60	80.09
Teacher data	54.47	68.67	53.75	63.54	53.39	62.20	53.61	63.24

As a result, with a score below 60, experts and teachers agree that the functionality of the IES now in use by teachers to “teach” does not satisfy the requirements of applications in particular educational situations. Teachers evaluated the software’s functionality and software design experience (FRV) at a higher level (>70) than did teachers (>60 & <70). In the meantime, the data results demonstrated that the experts’ ratings of BFV and FRV were higher than the instructors’ ratings under the four secondary aspects (teaching research, lesson preparation, lecturing, and question-answering about homework).

This study examines the development level of IES across various teaching scenarios by assessing the software’s core functionality and application effectiveness through visualization techniques. By analyzing the data in Table 2, the research delves into the similarities and differences between teachers’ and experts’ perspectives. The aim is to offer valuable recommendations for improving and advancing IES by integrating insights from both experts and teachers. Additionally, the study investigates the general characteristics of IES currently used in teaching, evaluating its performance from both practical and educational viewpoints. Moreover, this research captures the genuine feedback from frontline teachers regarding the development needs of IES. It uses word clouds to highlight the key attributes of IES and gathers suggestions for future improvements through feedback questionnaires.

Results and Discussion

Emphasis on “Technology” but Neglect of “Scenario-Application Effects” in the Development of IES

As shown in Figure 4, the IES scored high on “FRV” primary indicator and low on “BFV” primary indicator for both teachers’ and experts’ secondary dimensions. It shows that both teachers and experts believe that commonly used IES for teachers’ teaching performs better in terms of functional quality and software design but performs poorly in terms of application effects in specific teaching scenarios. With the development of AI, big data, and other technologies, current IES for teachers has provided a good user experience, emphasizing hardware and interface design quality, functionality richness, and aesthetic appeal. However, in the design and development process, software developers have not yet fully considered whether the functions of the software can be effectively applied in instructional scenarios and truly cater to the needs of teachers’ instruction and development. The function of the software is complicated in highlighting the efficiency of the intelligent drive, and it cannot assist the teachers in carrying out “intelligent instruction” in specific scenarios. In addition, it is worth paying attention to the fact that the two primary indicators of “FRV” and “BFV” of intelligent education software under the secondary dimension of instruction in teachers’ data are the lowest scores, 62.20 and 53.39 respectively. The two primary indicator scores for the secondary dimension of instruction in the experts’ data were also the lowest, at 75.29 and 54.96, respectively. This reflects that the current IES varies in quality, with some designs needing to be more appropriate for teachers’ classroom instruction scenarios, ignoring the feasibility and efficiency of functions in different teaching modes. Therefore, the development and design of IES should not only consider the richness and power of the software functions but also pay attention to whether these functions are needed and can meet the application effect of teaching scenarios.

Figure 4.

Comparison of scores on the primary indicators in teachers’ data and experts’ data under the secondary dimensions.

Differences Between Teachers’ Data and Experts’ Data in Assessing IES: Functional Realization and Scenario Application

As shown in the graph of teachers’ data versus experts’ data at the primary indicator under the secondary dimensions of “FRV” in Figure 5, teachers’ data and experts’ data both had the highest scores for IES in the secondary dimension of teaching research and the lowest scores in the secondary dimension of lecturing. This suggests a convergence between teachers’ and experts’ experiential perceptions of the software’s quality and design. However, there is a difference between teachers’ and experts’ assessments in terms of the software’s performance in business fit application scenarios. In Figure 5, teachers’ data for IES showed the highest scores in teaching research and the lowest in lecturing, while experts’ data peaked in question-answering about homework and dipped in lecturing. As a result, teachers and experts did not agree on the best performance of IES in the four educational scenarios at the primary indicator of “BFV.” This difference may result from the difference in perspective between experts and teachers due to their different experiences. Experts may have assessed the software based on usage experience, focusing on its functional richness and scenario fit. Conversely, teachers evaluated it based on their daily teaching, emphasizing its functional usefulness and teaching efficiency. With intelligent technology, IES in question-answering about homework exhibits diversified, intelligent, and personalized functions, aligning better with actual needs. In the secondary dimension of teaching research, although the functions of IES are yet to be enriched, most of the functions may be able to meet the actual needs of teachers in the process of teaching research and effectively improve the efficiency of teachers’ teaching research. The value of IES needs to be reflected in the actual teaching. Therefore, the development path of IES should not focus on pursuing the richness of software functions but should focus on the practicality and efficiency of software functions in the application scenarios.

Figure 5.

Comparison of teachers’ data and experts’ data on software scores under the primary indicators of secondary dimensions.

Evaluation of the Core Functions of IES From Two Perspectives in Teachers’ Teaching Scenarios

As shown in Figure 6, the overall scores of teachers’ assessment of the software under the SIs of teaching research, lesson preparation, and lecturing giving are higher than those of experts. This may be because teachers assessed the software based on the assessment indicators for software that was used by them in actual teaching research, lesson preparation, and lecturing scenarios. After using the software in real-life scenarios, teachers have a deeper understanding of the practicality of the software and the meaning of the core set of functions. They also think that the experience of a specific function is better, and the experience of using the software makes them feel emotionally and subjectively about the recommended software, which leads them to give it higher scores.

Figure 6.

Comparison of teachers’ and experts’ assessment of the software under the secondary indicators of the secondary dimensions.

Under the teaching research dimension, both experts and teachers gave the SI of teaching research activity management the highest score, and both agreed that there is still much room for improvement in the performance of intelligent education software in terms of AI intelligent teaching research. It is worth noting that the data from the two sides differed significantly in SIs, such as AI intelligent teaching research, teaching research data analysis, and micrographic recording of lessons. China’s education is undergoing digital transformation, integrating with informatization, gradually evolving from digitalization to intelligence. In the field of teaching research, the functions of AI intelligent teaching research and teaching research data analysis of intelligent education software provide more accurate data reference and more brilliant assessment for teaching research, helping the digital transformation of education. Software development companies should strengthen communication with teaching researchers and teachers to optimize the design of intelligent education software functions under the teaching research dimension.

Under the lesson preparation dimension, experts and teachers scored higher in the SIs of instructional material production and instructional resource management and lower in lesson preparation incentives and collective lesson preparation. At this stage, some IES often struggles to support teachers in online remote collective lesson preparation, and teachers need the assistance of other online platforms in the process of remote collective lesson preparation. Switching back and forth between different software not only increases the operating load of the equipment but also increases the complexity of software use in the process of lesson preparation, which puts a higher demand on the user’s learning cost and patience. It is also worth mentioning that software in the dimension of lesson planning tends to ignore the emotional aspects of the design of the teacher, and most of the incentives are only the external motivation for teachers to prepare lessons.

Under the dimension of instruction, experts and teachers scored higher in the SIs of course supports and multiport interconnection and lower in the analysis and collection of learning data and assessment of teaching effectiveness. Currently, some IES provides teachers with data support by collecting and analyzing learning data. However, there is room for improvement in the reliability, comprehensiveness, and relevance of the learning data because the data collection technology and equipment still need to be developed and popularized.

In most of the core function collections under the question-answering about homework dimension, the score of experts’ data is higher than that of teachers’ data, and both data score higher in the core function collection of homework release management and lower in online Q&A about homework. Some IES realizes the online Q&A about homework function mainly in synchronous and asynchronous discussion, that is, students leave a message, and the teacher or other students answer it after seeing it. When the teacher wants to conduct live Q&A, most of them will use other software or platforms. Integrating IES with live Q&A to meet teachers’ and students’ needs, achieving “one-stop service,” and reducing device burden poses optimization challenges for designers.

Addressing Development Challenges in IES: Leveraging Human–Computer Interaction for Innovation

IES has the potential to significantly empower teachers by enhancing both teaching efficiency and effectiveness. As shown in Figure 7, teachers place strong emphasis on “interactive” features, particularly valuing functions such as “multi-screen interaction,”“diverse and novel forms of classroom interaction,” and the integration of elements like “game-based interaction” and “knowledge capsules.” These functions foster dynamic, student-centered learning environments. Moreover, resource-rich features and course assistance functions alleviate teachers’ workload in lesson preparation. Access to high-quality “lesson plans and courseware,”“composition materials,” and “model courses,” combined with convenient utilities such as “import and export” and “cloud storage,” allows teachers to dedicate more time to pedagogy and student engagement.

Figure 7.

Word cloud of charm attributes of “Teaching” IES for primary and secondary school teachers.

Despite these benefits, several challenges hinder IES from fully realizing its empowering potential. As illustrated in Figure 8, “compatibility issues” remain a major concern. While advancements in desktop protection and terminal operation have been made, persistent problems related to security and compatibility continue to frustrate teachers and obstruct seamless classroom integration. Additionally, “payment issues” highlight further limitations. Reliance on schools to provide sufficient financial support for software access generates inequities and restricts equal opportunities among teachers. Furthermore, although teachers recognize the promise of intelligent features, Figure 8 underscores unmet needs, such as “intelligent homework analysis,”“personalized content push,”“seamless login,” and “intelligent recommendations.” These gaps indicate a misalignment between current software functions and teachers’ evolving expectations for genuinely intelligent educational tools.

Figure 8.

Word cloud of suggestions for developing IES to support primary and secondary school teachers.

In conclusion, while IES provides valuable features that support teaching and reduce workload, the challenges identified in Figure 8 must be addressed to ensure that such tools truly empower teachers rather than imposing additional burdens and complexities.

Development Trends and Outlook

Under the promotion and guidance of national policies, the application of many intelligent technologies such as artificial intelligence, 5G Internet, blockchain, big data, and so on in IES has been very significant (B. Q. Liu et al., 2021). The quality and aesthetics of IES have been significantly improved. At the same time, the development of intelligent technology has given rise to the emergence of personalized recommendations, intelligent teaching evaluation, question bank assessment, and other multistate tools, which greatly enrich the function of the software. However, the problem of the relevance of IES in educational scenarios is still prominent, which restricts the in-depth promotion of the current education informatization. All in all, IES should constantly be improved and optimized based on the feedback from teachers and theoretical guidance from educational experts during the development process, and gradually shift the focus from hardware experience to the appropriateness and efficiency of the actual use of educational scenarios.

Development Trend

The application is king and practice-oriented to achieve iterative improvement of IES. With the comprehensive transformation of education in the era of intelligence, the core of intelligent education has gradually shifted from platform and resource construction to service construction (S. Q. Yu & Chen, 2021). According to the China Education News Network (China Education News Web, 2022), Minister Huai Jinpeng emphasized that informatization should be driven by application needs, noting the adage “technology is the foundation, application is the king,” which also applies to commercial applications. The findings of this study indicate that the current quality of IES varies, with software development still emphasizing “technology” over the “effect of scenario application.” Therefore, the public infrastructure functions of IES should be fully considered prior to software development. The creation of IES must ensure usability and operability while also accommodating teachers’ levels of classroom manipulation (Q. C. Yu, 2020). Furthermore, developers should prioritize the practical use of digital technologies, focusing on how technology empowers teaching and transforms education (Shu & Gu, 2022), rather than uncritically pursuing technical complexity and functional abundance. Development should center on meeting the needs of actual teaching scenarios. Based on specific regional and school contexts, pilot projects can be implemented to iteratively update software resources, enhance applicability, apply technology-enabled teaching, and build an “open, shared, and sustainable” IES service.

A well-organized IES service system requires collaboration among the various stakeholders involved. However, the current tripartite model of “government purchase, enterprise provision, and school use” lacks clear responsibilities, positioning, and cooperation mechanisms. Due to immature feedback channels, enterprises are often unable to obtain timely feedback from educational users (F. Wang et al., 2020). Data analysis reveals that the practicality of IES in specific scenarios remains insufficient, particularly under the four secondary dimensions of “teaching” for teachers. To enhance IES services, the relationships among enterprises, universities, governments, schools, and other stakeholders must be reconsidered. Approaches include building information platforms to improve communication and capture user needs, as well as promoting top-level design for regional coordination to support daily teaching practices. Strengthening cooperation between schools and enterprises is also essential for creating a collaborative chain of “frontline schools–enterprises–universities,” thereby maximizing complementary advantages and avoiding “Great Leap Forward”-style development and low-level duplication of infrastructure (Ma et al., 2023). On the one hand, enterprises can provide IES services to primary and secondary schools, creating a ripple effect by building demonstration schools that influence surrounding institutions. On the other hand, enterprises should systematically collect teacher feedback to improve product quality and responsiveness.

Data-driven educational transformation and governance are inevitable trends, and the extensive data generated through IES use holds considerable educational value. Core service providers must leverage their technological strengths to establish standardized data centers, ensuring secure storage and enabling access for both providers and application stakeholders (F. Wang et al., 2020). Developing an “intelligent education brain” based on AI and IoT can effectively restore the flow of data and information within the intelligent education ecosystem (Y. Y, Wang & Zheng, 2021). This would support schools in aggregating and managing large-scale heterogeneous data to build comprehensive databases with educational value, enabling diverse data services, precise governance, and flexible application. In addition, the advancement of multimodal technologies enriches IES data collection. For example, lecture software can integrate emotion recognition technologies to assess classroom emotional dynamics, thereby enhancing the richness of database samples and contributing to emotionally sensitive and adaptive educational services.

As IES evolves, repetitive teaching tasks are gradually automated, alleviating teachers’ workload and enabling them to focus on creative and relational aspects of teaching. IES facilitates time savings in tasks such as homework correction, allowing teachers to dedicate more effort to lesson preparation and student-centered instruction. This shift underscores the transition of IES development from meeting basic needs to pursuing refinement. The integration of multimodal technologies further supports the enrichment of databases through intelligent and emotional classroom assessments. In promoting refined educational software, it is crucial to provide functional explanations and training resources to increase utilization rates and ensure that functions are effectively implemented. Moreover, collaboration between enterprises and schools should be strengthened to co-develop “personalized + refined” IES solutions tailored to the specific needs and characteristics of schools.

Development Challenges

ChatGPT facilitates the innovative development of IES. As a large-scale AI-based dialogue model with advanced information retrieval and organizational capabilities, ChatGPT presents both opportunities and challenges for the future of IES (Aljanabi, 2023). Its emergence may fundamentally transform resource retrieval, knowledge organization, and online Q&A functions within educational platforms. One limitation of existing innovative educational software lies in its motivational mechanisms: machine-generated language and feedback often appear rigid and lack emotional resonance, thereby reducing their effectiveness in inspiring learners. In contrast, ChatGPT is capable of simulating diverse human emotions and conversational tones, offering contextually appropriate responses (Taecharungroj, 2023). Furthermore, it enables intelligent virtual tutoring (Al-Abri, 2025; Limo et al., 2023) and precise recognition services (Deandres-Tame, 2024), thereby expanding possibilities for personalized education (Zhang & Dong, 2022; Zhang et al., 2022, 2023), immersive gamified learning (Zhang & Robb, 2021a, 2021b), and AI-enhanced entrepreneurial and AI literacy education (Wu et al., 2025), supporting intercultural collaboration, innovative skill development, and equitable learning opportunities across diverse educational contexts.

From a regulatory perspective, the entry threshold for IES must be raised. Such software should not be regarded merely as a consumer-oriented auxiliary tool, but rather as an instrument for knowledge production and cognitive enhancement (M. Z. Liu et al., 2020). Although the current market for intelligent educational tools used in teaching demonstrates a positive and diversified development trend, it remains in a stage of “barbaric growth.” In particular, software evaluated under the teaching dimension of this study scores relatively low in terms of quality and functionality, often failing to meet teachers’ expectations for service standards. As more enterprises enter the education informatization sector, the absence of robust policies and regulations exacerbates these challenges. Establishing clear service specifications and strengthening quality supervision are therefore imperative. Many providers remain uncertain about how to meet the individualized needs of teachers and students or how to optimize service processes. Government departments should establish industry norms and set minimum service thresholds to ensure that IES is well-aligned with pedagogical contexts, delivers high-quality services, and prevents the proliferation of substandard products in the education market.

The “intelligence level” of existing software also requires significant improvement. Intelligent educational tools for teaching currently score poorly on core functions such as AI-based teaching, teaching-effect evaluation, online Q&A, and incentive mechanisms for lesson preparation. Word cloud analyses of teachers’ developmental suggestions further highlight strong demand for advanced functions such as “intelligent homework analysis,”“personalized content recommendation,” and “integration of software with hardware,” underscoring the need to enhance the intelligence level of current systems. Future development should leverage the Internet, artificial intelligence, and big data to address deficiencies in existing teaching applications. This entails designing multi-scenario-oriented platforms, employing diverse software tools for intelligent perception and analysis, and meeting teachers’ varied instructional needs (Gao et al., 2021). Moreover, enabling multi-device interoperability and overcoming barriers across devices, systems, and contexts will strengthen ubiquitous connections and facilitate seamless integration among “human–machine–object–environment.” Another critical direction involves the intelligent aggregation of multimodal, high-quality teaching resources. By utilizing knowledge-mapping technologies, systems can identify multi-dimensional correlations among knowledge points and deliver personalized resource recommendations for teachers (Y. Y. Wang & Zheng, 2021).

Finally, emotional support services in IES should be reinforced. While current systems demonstrate considerable strength in enhancing human–computer interaction functions, they remain underdeveloped in providing intelligent emotional support. For instance, the core function of “incentive mechanisms for lesson preparation” under the preparation dimension receives consistently low ratings. Most existing mechanisms rely heavily on extrinsic motivators, which fail to meaningfully enhance teachers’ intrinsic enthusiasm for lesson preparation. To improve this, teachers’ internal motivation must be better supported, while enterprises should collaborate with educational experts to design more sophisticated and effective incentive structures that maximize both impact and efficiency.

Limitation

This study has several limitations that warrant attention. First, the relatively small sample size may constrain the generalizability of the findings. Second, although both teachers and experts were included, potential selection bias among teacher respondents and subjective judgment within expert groups may have influenced the results. Third, the reliance on self-reported data introduces the possibility of response bias shaped by individual perceptions and contextual factors. Finally, the analysis concentrated on specific functions and scenario applications of intelligent educational software, which may not fully reflect its complexity in diverse educational contexts. Future research should expand sample coverage, adopt more objective and mixed-method approaches, and incorporate longitudinal designs to provide a more comprehensive understanding of IES development and application.

Conclusion

This study developed the IESEIPSTT and applied it to evaluate popular IES from both teachers’ and experts’ perspectives. The results demonstrate that current IES development prioritizes “technology” over the effectiveness of scenario application. While teachers and experts largely agreed on the functionality of IES in specific educational contexts, their evaluations diverged regarding performance in business-fit application scenarios due to differing perspectives. Assessments of core functions from both viewpoints revealed multiple issues across various dimensions. To address these shortcomings, future IES development should prioritize practicality and efficiency in real teaching scenarios, promote multi-stakeholder collaboration to enhance service quality, strengthen database construction, refine functional design, and elevate levels of intelligence. Moreover, it is essential to anticipate and respond to emerging challenges, such as those posed by ChatGPT and evolving regulatory frameworks, in order to ensure the sustainable and effective development of intelligent educational software.

Footnotes

Appendix A

Table A1.

Description of Tertiary Indicators Under Each Secondary Indicator.

Secondary dimension	Secondary indicator	Tertiary indicator
Teaching research	Management of teaching research activities	(1) Support for teaching research activities recommendations
		(2) Support for participation in teaching research activities
		(3) Provide a variety of teaching and research activities
		(4) Teaching and Research Resource Library
	Teaching research social interaction	(1) Collective teaching research
		(2) Friends chat
		(3) Circle Dynamics Sharing
	Remote activity teaching research	(1) Teaching research activities schedule and reminders
		(2) Remote audio and video interaction
		(3) Multi-screen annotations and revisions
		(4) Interactive teaching research recordings
		(5) Multiple resources for quick sharing
		(6) Resource acquisition
		(7) Information sent automatically
		(8) Archiving of teaching research activities (materials)
	AI intelligent teaching research	(1) Speech recognition for teachers and students
		(2) Teacher’s content recognition
		(3) Teacher and student behavior analysis
	Teaching research data analysis	(1) Analysis of the teaching research process
	Teaching research data analysis	(2) Analysis of the results (outcomes) of teaching and research
	Micro-lesson recording	(1) Teaching skills training
		(2) Video recording of teachers’ open class competition
		(3) Teachers’ live on-campus classes
	Web-based lecture evaluation	(1) Support for online listening
		(2) Support for online lecture evaluation
		(3) Supporting the exchange of lecture evaluation results
		(4) Supports resource entry of teaching materials
Lesson preparation	Production of teaching materials	(1) Rapid editing to form an instructional design (lesson plan/lesson plan)
		(2) Producing interactive courseware
		(3) Multi-disciplinary tool support
	Teaching resource management	(1) Provide a variety of resources for lesson planning
		(2) Resource storage and management
		(3) Support for multiple categorization of repositories
		(4) Supports intelligent recommendation of resources
		(5) Support for school-based resource management
		(6) Cloud storage of lecture materials
		(7) Multiple account synchronization
	Micro lesson recording	(1) Support audio and video recording
		(2) Video post-production
		(3) Support for micro-learning interactions
		(4) Micro lesson sharing
		(5) Micro-teaching learning status report
	Incentives for lesson planning	(1) Virtual currency exchange
		(2) Upgrade to unlock new content
		(3) Lesson planning progress reminder
	Prepare lessons collectively	(1) Initiate group lesson preparation
		(2) Full documentation of the seminar process
		(3) Analysis of lesson preparation
		(4) Information sharing
lecturing	Teaching aids	(1) Support for group activities
		(2) Classroom session timing
		(3) Instructional content that supports discipline-specific
		(4) AI assistants to aid in classroom teaching
	Course supports	(1) Question and answer interactive
		(2) Real-time attention and answers to student questions
		(3) Roll call interactive
		(4) Support for live online lectures
		(5) Support online board book
		(6) Supporting diverse classroom formats
	Analysis and collection of learning data	(1) Analysis of student learning effectiveness
		(2) Analysis of student attendance
		(3) Evaluation of student performance in the classroom
		(4) Classroom climate analysis and intervention
		(5) Analysis and intervention of student emotions
		(6) Student concentration analysis and intervention
	Multi-port interconnection	(1) Mobile lecture
	Multi-port interconnection	(2) Rapid sharing of content resources
	Assessment of teaching effectiveness	(1) Teaching progress reminder
	Assessment of teaching effectiveness	(2) Intelligent feedback on the effectiveness of classroom teaching
Question-answering about homework	Homework group question	(1) Manual grouping of questions
		(2) Intelligent grouping of questions
		(3) Human-computer collaborative grouping of papers
		(4) Title filtering
	Homework release management	(1) Class assignment release
	Homework release management	(2) Individual assignments targeted
	Homework correction	(1) Automatic correction of objective questions
		(2) Human-computer collaborative review of subjective questions
		(3) Personalized homework evaluation
	Online Q&A about homework	(1) Support for easy-to-fail questions
	Online Q&A about homework	(2) Support for personalized error-targeted explanations
	Statistics and analysis of homework	(1) Analysis of homework completion
		(2) Assignment knowledge points analysis
		(3) High frequency error statistics
		(4) Quality analysis of homework exercises
		(5) Homework Status Report Download and Share

Acknowledgements

We thank the participating teachers and educational technology experts for their valuable contributions to this study.

ORCID iDs

Bo Zhang

Xiaoqing Gu

Ethical Considerations

Prior to beginning the study, ethical approval was obtained from the institutional ethics committee in University College Dublin under approval number: HS-14-69-Zhang-Goodman.

Author Contributions

B.Z. designed the study, did statistical analyses, collected the data, and wrote the manuscript. Y.C. and X.W. performed the statistical analyses and wrote the manuscript. Y.L. did statistical analyses, collected the data. X.G. contributed to the idea inform and designed the study.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the XJTLU Research Development Fund Project (Grant No. RDF-24-02-021).

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The quantitative data collected for this study can be accessed by request, by contacting Author. The authors declare that there is no conflict of interest.

Data Availability Statement

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

Al-Abri

(2025). Exploring ChatGPT as a virtual tutor: A multi-dimensional analysis of large language models in academic support. Education and Information Technologies, 1–36.

Aljanabi

(2023). ChatGPT: Future directions and open possibilities. Mesopotamian Journal of Cybersecurity, 2023, 16–17. https://doi.org/10.58496/mjcs/2023/003

Al-Fraihat

Joy

Sinclair

(2020). Evaluating E-learning systems success: An empirical study. Computers in Human Behavior, 102, 67–86.

Barari

RezaeiZadeh

Khorasani

Alami

(2022). Designing and validating educational standards for E-teaching in virtual learning environments (VLEs), based on revised Bloom’s taxonomy. Interactive Learning Environments, 30(9), 1640–1652.

Chen

Zhang

(2000). Evaluation of CAI material. Distance Education in China, (5), 45–48. https://doi.org/10.13541/j.cnki.chinade.2000.05.014

China Education News Web. (2022, February 24). Promoting education informatisation higher education should play a leading role [News release]. http://paper.jyb.cn/zgjyb../html/2022-02/24/content_605775.htm?div=-1

Dai

W. F.

L. J.

Wang

D. Q.

(2014). Research on the training platform and application of the teachers’ educational technology ability under cloud media. Modern Educational Technology, 24(9), 114–120.

Dai

Liu

(2022). Construction and practice of teachers knowledge sharing training model based on intelligent education platform. China Educational Technology, (1), 134–142.

Deandres-Tame

Tolosana

Vera-Rodriguez

Morales

Fierrez

Ortega-Garcia

(2024). How good is ChatGPT at face biometrics? A first look into recognition, soft biometrics, and explainability. IEEE Access, 12, 34390–34401.

10.

N. V.

Nguyen

H. D.

Mai

T. T.

(2018). Intelligent educational software in discrete mathematics and graph theory. In New trends in intelligent software methodologies, tools and techniques (pp. 925–938). IOS Press.

11.

Gao

J. J.

Wang

X. J.

Shang

J. H.

Zhou

Y. L.

(2021). The construction and practical cases of human-vachine colaboration teaching mode in the era of artificial intelligence. Journal of Distance Education, 39(4), 24–33. https://doi.org/10.15881/j.cnki.cn33-1304/g4.2021.04.003

12.

H. X.

(2020). Advanced lesson planning: A new paradigm of lesson planning based on classroom research. Journal of Northwest Normal University (Social Sciences), 57(4), 45–53. https://doi.org/10/16783/j.cnki.nwnus.2020.04.006

13.

Kara

(2007). Educational software evaluation form for teachers (p. 5). https://doi.org/10.1080/02796015.1991.12085536

14.

J. Y.

(2017). Usability evaluation of educational resources software based on evidence theory [Master’s thesis, Xi’an University of Posts and Telecomunications]. https://kns.cnki.net/kcms2/article/abstract?v=fsvnL9wA1q251MMGOWAaniEUL8YGQpQXjo5EcpR4R6fOGDITUS7iZ5D6cpC8JiBS1LpUds28dEdo5OppR0rrc5siNJ72BXjCeFVEue3OsYh1FsVVgMEuyhwEI1DYj2VdaBRofgRYm8o=&uniplatform=NZKPT&language=CHS

15.

Z. X.

(2020). Intelligent leadership convergence and innovation—Network public service platform construction of Beijing No.12 High School. Information Technology Education in Primary and Secondary Schools, (6), 35–37. https://doi.org/CNKI:SUN:ZXJA.0.2020-06-018

16.

Limo

F. A. F.

Tiza

D. R. H.

Roque

M. M.

Herrera

E. E.

Murillo

J. P. M.

Huallpa

J. J.

Flores

V. A. A.

Castillo

A. G. R.

Peña

P. F. P.

Carranza

C. P. M.

Gonzáles

J. L. A.

(2023). Personalized tutoring: ChatGPT as a virtual tutor for personalized learning experiences. Przestrzen Spoleczna (Social Space), 23(1), 293–312.

17.

Liu

B. Q.

Zhang

J. X.

Zhu

G. M.

J. H.

X. W.

(2021). Intelligence + education: Industry Status, hot spots and development trends research on the development of China’s smart education industry in 2020. E-education Research, 42(11), 55–62. https://doi.org/10.13811/j.cnki.eer.2021.11.008

18.

Liu

(2023). Towards intelligent e-learning systems. Education and Information Technologies, 28(7), 7845–7876.

19.

Liu

M. Z.

H. S.

Tong

Wang

G. C.

(2020). Current status and development suggestions of education informatization service from perspective of government purchasing service. E-education Research, 41(4), 102–108. https://doi.org/10.13811/j.cnki.eer.2020.04.014

20.

M. Y.

Wang

S. H.

(2023). Digital transformation of educational examination: Problems, elements and objectives take the practice of Jiangsu province for example. Modern Educational Technology, 33(4), 24–31.

21.

Ministry of Education. (2021, September 8). Notice on the implementation of the second batch of pilot work on artificial intelligence-facilitated teacher development initiatives [News release]. http://www.moe.gov.cn/srcsite/A10/s7034/202109/t20210915_563278.html

22.

Paul

(2008). Lucas, Ph.D. Columbus city schools instructional software alignment evaluation [EB/OL]. http://www.columbus.k12.oh.us/website.nsf/0c6fc31e841022ec852573af00703e34/b29e941fd108b23385257537006112b6/$FILE/SoftwareEvalfeb2008.pdf

23.

T. Q.

(2021). Exploration of promoting transformation of teaching and research with intelligent platform. Journal of Dalian Education University, 37(1), 1–3.

24.

Robinson

Johanson

Schneider

Hutinger

(2006). Interactive technology literacy curriculum online (ITLC online). Center for Best Practices in Early Childhood Education.

25.

Shade

D. D.

(1991). Developmentally appropriate software. Computer and Young Children, 18, 34–36. https://doi.org/10.4324/9780367198459-reprw144-1

26.

Shu

X. Q.

(2022).The realistic basis and action framework of digital transformation of education. Modern Educational Technology, 32(11), 24–33.

27.

State Council (PRC). (2021, July 24). Opinions on further reducing the burden of homework for students in compulsory education and the burden of out-of-school training [News release]. http://www.gov.cn/zhengce/2021-07/24/content_5627132.htm

28.

Sun

B. L.

(2003). Research on the evaluation indicator system of digital courseware for distance education in China. Chinese Journal of ICT in Education, (12), 60–62.

29.

Sun

Y. Z.

Duan

Y. B.

(2005). Evaluating standards of analyzing teaching software. Journal of Jilin TV & Radio University, (1), 3.

30.

Taecharungroj

(2023). “What can ChatGPT do?” Analyzing early reactions to the innovative AI Chatbot on Twitter. Big Data and Cognitive Computing, 7(1), 35. https://doi.org/10.3390/bdcc7010035

31.

Tian

B. J.

(2013). Content analysis of educational software research. China Educational Technology, (10), 88–92.

32.

Tsihouridis

Batsila

Vavougios

(2011). Enhancing and assisting laboratory teaching of electrical circuits using ICT: An evaluation of educational software (pp. 323–335). In 14th International Conference on Interactive Collaborative Learning,. Piestany, Slovakia. IEEE. https://doi.org/10.1109/ICL.2011.6059599

33.

Wang

X. X.

X. Q.

(2020). Research on ecological development of educational informatization products and services. E-education Research, 41(10), 99–105. https://doi.org/10.13811/j.cnki.eer.2020.10.014

34.

Wang

S. Z.

Xian

Wang

X. S.

(2002). Study on synthetic evaluation method of software quality. Computer Engineering and Design, 23(4), 16–18. https://doi.org/10.16208/j.issn1000-7024.2002.04.006

35.

Wang

Y. Y.

Zheng

Y. H.

(2021). Smart education products: Building a new ecology of smart education based on AloT. Open Education Research, 27(6), 15–23. https://doi.org/10.13966/j.cnki.kfjyyj.2021.06.002

36.

Y. T.

Huang

Zhang

(2025). Syntegrative entrepreneurship education for engineers: AI-Enhanced module through project based learning. Proceedings of ICAIE 2025. IEEE.

37.

Zhang

Liu

Jiang

(2025). Exploring the construction of a standard AI literacy framework for middle school students in the Era of GAI: Focusing on the assessment of AI application and innovation. Proceedings of ICAIE 2025. IEEE.

38.

Q. C.

(2020). The necessity and limitations of technicalized educational practice focusing on the logical orientation technology in the basic theories of education. Educational Research, 41(6), 14–26.

39.

S. Q.

Chen

(2021). Construction of smart education service ecosystem. E-education Research, 42(6), 10. https://doi.org/10.13811/j.cnki.eer.2021.06.001

40.

Zhang

Dong

R. H.

(2022). How natural language processing technology empowers the AIED: The perspective of AI scientist. Journal of East China Normal University (Educational Sciences), 40(9), 19–31. https://doi.org/10.16382/j.cnki.1000-5560.2022.09.003

41.

Zhang

Goodman

(2022). Novel 3D contextual interactive games on a gamified virtual environment support cultural learning through collaboration among intercultural students. Sage Open, 12(2), 21582440221096141.

42.

Zhang

Goodman

(2023). Telecollaboration tool preferences for online intercultural learning in higher education: Perspectives of Chinese international students. Sage Open, 13(2).

43.

Zhang

Robb

(2021a). A comparison of the effects of augmented reality N-Back training and traditional two-dimensional N-Back training on working memory. Sage Open, 11(2).

44.

Zhang

Robb

(2021b). Immersion experiences in a tablet-based markerless augmented reality working memory game: Randomized controlled trial and user experience study. JMIR Serious Games, 9(4), e27036.

Dual Perspective Assessment of the Development Level and Characteristics of Intelligent Educational Software in China

Abstract

Keywords

Introduction

Literature Review

Method

Indicators Construction

Dual Perspective Assessment Process

Results and Discussion

Emphasis on “Technology” but Neglect of “Scenario-Application Effects” in the Development of IES

Differences Between Teachers’ Data and Experts’ Data in Assessing IES: Functional Realization and Scenario Application

Evaluation of the Core Functions of IES From Two Perspectives in Teachers’ Teaching Scenarios

Addressing Development Challenges in IES: Leveraging Human–Computer Interaction for Innovation

Development Trends and Outlook

Development Trend

Development Challenges

Limitation

Conclusion

Footnotes

Appendix A

Acknowledgements

ORCID iDs

Ethical Considerations

Author Contributions

Funding

Declaration of Conflicting Interests

Data Availability Statement

References