Sage Journals: Discover world-class research

Abstract

Learning business Chinese through digital materials has become an inevitable tendency with the growth of the internet and impact of the COVID-19 pandemic. Despite the increasing research-based publications and practices relating to Chinese language textbooks, there is a dearth of assessment indicators weight systems constructed by alternative research tools and statistical methods to evaluate the quality of the growing digital business Chinese materials. To the best of our knowledge, no literature review has been published on using the analytic hierarchy process (AHP) method to construct the assessment indicators of digital business Chinese materials. Therefore, this study aims to fill this gap by reviewing the existing, but limited, literature and providing recommendations for developing a checklist—based on the assessment indicators weight system—to evaluate digital business Chinese materials. The preliminary PC2U (Production, Content, Use, and Usefulness) assessment indicators are constructed based on the current research on business Chinese, CLTML, and relevant assessment indicators. In addition, the analysis hierarchy process (AHP) is used to construct the weight system. The results show that it is crucial to prioritize the access of digital business Chinese content. Of the three criteria in the content dimension, learning needs and credibility are equally important, and both are more important than data update. The assessment indicators are not only beneficial to fill the knowledge gap on business Chinese research, but also helpful for publishers and teachers to compile and select high quality digital business Chinese materials.

Keywords

digital business Chinese material non-native Chinese speaking businessmen/women (NNCSB)analysis hierarchy process (AHP)assessment indicators weight system

Introduction

Learning business Chinese through digital materials has become an inevitable tendency with the growth of the internet and impact of the COVID-19 pandemic. Scarpetta and Quintini (2020) indicated that the COVID-19 crisis has resulted in a significant increase in digital learning by adults, and much of the training that had started as face-to-face learning in classroom environments is being pursued online. This provided impetus for trainers to develop more digital materials, which are cost-effective, time-effective, and eco-friendly. While the notion of business Chinese language learning using digital materials is not a novel idea, rapid developments of digital technology and the severity of the COVID-19 have increased the demand for learning it digitally.

The digital context of business Chinese language learning has prompted the need for featured, hacker-resistant, authentic, reusable, and useful digital materials for online learning, in-person courses, or at-home learning. These features and functions are crucial for those who are non-native Chinese speaking businessmen/women (NNCSB)—as self-directed adult learners—to learn business Chinese by digital materials “whose subject matter is communicative interaction which includes a business-cultural aspect.” Additionally, these features and functions require distinguishing such materials from their paper versions. However, after comparing existing business Chinese textbooks with similar book titles in Chinese, such as rendering business either as commerce into 商業 (shāngyè) or 經貿 (jīngmào), or as business in use into 商用(shāngyòng)/商務(shāngwù), and digital versions, including “Business Chinese” online, “Far East Business Chinese” (遠東商務漢語/Yuǎndōng shāngwù hànyǔ).

App, and “Business Chinese” App, this study found little concerted eﬀort in this direction. Moreover, the gradual increase in digital business Chinese materials has not been competitive in the quality of production, content, usage, and usefulness, except for software and technical improvements. To construct learners’ knowledge schema, Mayer and Moreno (2003) proposed coherence and signaling principles—based on the cognitive load theory of multimedia learning (CLTML)—for designing e-learning materials. The coherence principle refers to eliminating unnecessary information in multimedia materials to reduce cognitive load and improve learning outcomes. The signaling principle refers to appropriate cues that allow learners to focus on important points of multimedia content, aiming to limit learners’ attention from being attracted toward unimportant or irrelevant information, thereby reducing external cognitive load. CLTML has been widely applied in e-learning (Asraj et al., 2011; Curum & Khedo, 2021; Davids et al., 2015; Kirschner et al., 2018; Lambert et al., 2009; Mayer, 2001; Mayer & Moreno, 1998, 2003; van Merriënboer & Ayres, 2005; Westlake, 2019). These studies revealed that materials that had not considered characteristics of leaners’ cognitive load were difficult to learn (Sweller & Chandler, 1994). However, previous relevant research (Cai & Chen, 2014; Chen & Lee, 2012a; Chiu, 2020; Chung, 2018; Li, 2011; Liao, 2019; Li & Chen, 2016; Tu, 2011; Wonchul et al., 2017) and practices (Liu et al., 2013a, 2013b; Liu, Chen et al., 2014; Liu, Zhou et al., 2014) do not delve deep into the problems of cognitive load faced by learners when applying the e-materials and cognitive load caused by the characteristics of Chinese language, nor are they concerned about the assessment indicators for digital business Chinese materials.

Despite the increasing research-based publications and practices relating to Chinese language textbooks, including Chinese language for business purposes, there is a dearth of assessment indicators weight systems constructed by alternative research tools and statistical methods to evaluate the quality of the growing digital business Chinese materials. To the best of our knowledge, no literature review has been published on using the analytic hierarchy process (AHP) method to construct the assessment indicators of digital business Chinese materials. Therefore, this study aims to fill this gap by reviewing the existing, but limited, literature and providing recommendations for developing a checklist—based on the assessment indicators weight system—to evaluate digital business Chinese materials. However, before proceeding with the AHP method and recommendations, this study attempts to set the stage by presenting the current research on business Chinese, CLTML, and assessment indicators for (digital) materials to contribute toward drafting the preliminary assessment indicators for digital business Chinese materials.

For this study, several references to business Chinese, CLTML, and the construction of assessment indicators are included to frame the general discussion; this is necessary because digital business Chinese materials are a subset of the Chinese as a second and foreign language education, and there have been few publications on specifically applying the AHP method to constructing assessment indicators weight systems for such materials. Mostly, the relevant references discussed in this study deal with research findings and arguments on business Chinese, CLTML, and construction of other assessment indicators, which are closely related to the preliminary assessment indicators formerly analyzed by the AHP method.

Literature Review

Current Research Performance on Business Chinese

With respect to the previous research findings, Li (2011) explored the needs of business Chinese L2 learners and found that the learners from a middle to advanced level needed to learn the following topics: introducing business associates, negotiating prices, conducting market surveys, interviewing skills, signing contracts, ordering products/meals. All of these topics would provide a learner with the ability to learn business Chinese, enter the workplace, and successfully communicate. Moreover, understanding a company’s organization and operation and introducing clients to the company are top priorities for most learners. Chen and Lee (2012a) explored the cultural content of contemporary business Chinese textbooks for beginners and found that the textbooks emphasize contemporary social and economic phenomena, economic and trade systems, and the proportion of contemporary social and economic cultural content far exceeds the description of traditional historical culture. In addition, “what one should and should not say” and “what one can and cannot do” in the Chinese workplace are found to have the utmost importance. Furthermore, Chen and Lee (2012b) analyzed the vocabulary in 10 intermediate business Chinese textbooks and found that vocabulary on “trade” and “finance” occupied the largest part in the textbooks. Their findings reflect that most NNCSB go to China for trade and financial management. The NNCSB need business-related Chinese vocabulary and knowledge to do these jobs. Thus, it is necessary to list the vocabulary in the textbooks. Zheng (2012) indicated that China’s economy has been reformed and liberalized since 1978, and with this China’s economic status has become increasingly important in the world. Consequently, new business Chinese words and translations have emerged rapidly and caused business Chinese L2 learners to spend more time learning the new words. Guan (2012) believed that translation can be regarded as a comprehensive application of listening, speaking, reading and writing skills, or as a supplement to the four skills after exploring the application of translation tasks in business Chinese teaching. Translation can be used not only to practice, consolidate and strengthen vocabulary, sentence, and grammar knowledge, but also to effectively detect learners’ comprehension of Chinese language.

According to Yan’s (2013) Master’s thesis, compared with teaching textbooks, self-learning textbooks were rare. In her thesis, “History and the present state of teaching business Chinese in Taiwan,” she suggested adding information on Taiwan’s economic background and market in textbooks and included an analysis of such course materials. According to Liao’s (2011) thesis, “A study on intermediate spoken communication course for business Chinese: Analysis, design, and implementation,” learners are highly affirmative of the teaching design for business Chinese oral communication, and this method allowed learners to accurately grasp the deeper meaning of speech acts, to cultivate capabilities of appropriate responses, and to immerse in real business Chinese situations. Vu (2011) found that the Vietnamese learners of Chinese have a strong demand for “business Chinese,” and they expect to study “authenticity” in business Chinese textbooks, as discussed in Vu’s thesis, “The Research about the need and development of business Chinese in Vietnam.” In connection with the assessment indicators for business Chinese materials, Chung (2018) constructed 43 evaluation indicators categorized into material content and design, based on 5Cs (Communication, Cultures, Connections, Comparisons, and Communities), for business Chinese textbooks. Liao (2019) adopted the Fuzzy Delphi method to construct 33 evaluation items, including learners’ needs, material content, teaching objectives, and audio and video presentation.

In the digital Chinese teaching materials and methods, Liu et al. (2013a) conducted a teaching experiment for a class of six Chinese L2 learners who studied Chinese language for about 1 to 2 years. The results showed that the digital Chinese material in this experiment helped to improve the layout, improve audio and video effects, create a learning website for learners to study freely, and provide teachers with teaching manuals and discs according to the teacher’s and learners’ feedbacks. Later, Liu, Chen et al. (2014) conducted an observational study to understand 16 Chinese L2 learners’ feedback after the learners studied the digital Chinese material. The observations and surveys indicated that dialogues were practical for learners who were interested in practicing business Chinese (listening and speaking) via role playing, and the material was suitable for business Chinese L2 learners above the intermediate level. As for Chinese L2 learners in non-degree programs, approximately 63% of learners believed that digital Chinese material was suitable for intermediate level learners, and approximately 37% of learners thought that the material was suitable for intermediate high-level learners. These learners expressed that dialogues were practical and studying negotiation knowledge via in-class role playing was interesting (Liu, Zhou et al., 2014).

We found that little research focuses on digital Chinese materials, and the quality and quantity of the materials need to be improved. Adding the elements of Taiwan’s economic background and market conditions was suggested; furthermore, the “authenticity” of the business was emphasized. According to literature reviews on published business Chinese, compared with paperback textbooks on business Chinese, the number of e-materials was limited; it was undoubtedly a flaw for Taiwan as a technological island. Although creating digital Chinese materials was distinguished from compiling paperback textbooks, it was impossible to maximize the learning effect of adult learners by only digitizing the content from paperback textbooks and creating excessive animated information, without considering that the learning content exceeded adult learners’ loads of absorbing Chinese knowledge (Cai & Chen, 2014). To solve this problem, this study constructed the assessment indicators of digital Chinese materials for the NNCSB.

The CLTML and Inspiration

With the advent of the information society, e-materials have become increasingly diverse. Effective digital Chinese material should be consistent with business situations, such that NNCSB can effectively absorb and understand business Chinese language, allowing for immediate use in real-time business situations. Generally, effectively absorbing and understanding knowledge was related to the cognitive load of personal information processing (Mayer, 2001; Mayer & Moreno, 2003). Mayer and Moreno (1998) found that the working memory of the human cognitive structure was divided into internal and external loadings. The content of learning material belonged to internal loading, while the design and layout of learning material belonged to external loading.

The learner’s internal loading will be low if the information of the material to be processed is pure and independent. In contrast, if the information received by the learner is too complicated or requires a lot of information to simultaneously interact for producing meaning, the learner’s internal loading will become overwhelmed. Thus, if the content of learning material cannot be modified more simply, it would not lower the learner’s internal cognitive loading (Sweller et al., 1998). For instance, at present, the business Chinese textbooks published in Taiwan and China emphasize improving learners’ cross-cultural communication skills. Tu (2011) recommended that the content of Business Chinese language textbooks must be changed in response to changes in the overall business environment because the economic cooperation and interaction between China and the West have changed. The changes included importing and exporting accessories, trading electromechanical products, high-tech products, and automatic data processing equipment. Although the focus of the textbooks varies, the content contains four themes: (1) “social and economic conditions”; (2) “economic and trade norms”; (3) “communication style”; and (4) “folk etiquette” (Chen & Lee, 2012a). Li (2011) found that business Chinese L2 learners have a great need for learning resume and curriculum vitae writing and job interviewing skills. The previously mentioned content was required to compile into business Chinese materials, in addition to vocabulary, grammar, and sentence patterns. Assumably, for the working memory of NNCSB, the design and layout of digital business Chinese materials belong to external cognitive loading, which directly affects the effectiveness of working memory. Once the external cognitive loading of a learner is reduced, there is an increase in available space in the learner’s working memory. This suggests that vocabulary, grammar, sentence patterns, and business Chinese knowledge, such as social and economic culture, trade, and communication style, which must be learned in digital Chinese language learning materials at different levels, are fixed, and authors need to reduce the external loading of learners by designing user-friendly layouts and interfaces in e-textbooks. In this case, more meaningful information is fed into the learner’s working memory. In other words, if the learning content is fixed, it is necessary for e-textbook compilers to further consider improving the layout and interface of the learning material to reduce the external loading of the learners and increase their working memory space.

On the technical level, nevertheless, the current text content was directly digitized without the consideration of subjects’ characteristics to design a human interactive interface of learning functions (Li & Yang, 2016; Tseng et al., 2007). Personalization, analytics, and an affordable user-interface are features of digital textbooks; however, less than half (45%) of the faculty agreed that digital courses materials provided significant value added content, which unavailable in print (Railean, 2017). These problems highlight the difficulty of using the benefits of digital teaching materials and the impossibility of providing teaching content. Chen (2011) highlighted that up to date, many digital textbooks, of which layout, interface, and broadcast speed were still traditional and unidirectional learning functions. Tang (2012) conducted a content analysis on 30 online Chinese language materials and found that most of the materials provided learners with exercises of listening, speaking, reading, and writing, but most of them were copied from paper books and transferred to webpages without bidirectional and interactive learning functions. Such learning webpages were not very effective. Although the benefits of e-learning were found to be greater than the traditional methods of teaching and learning in the discussions on computer and education, there were some undeniable limitations. However, it seemed that no universal standards for assessing the quality of e-materials on business Chinese language were provided for teachers’ decision on materials used and for publishers’ compilation of the materials. This study believes that while e-learning materials are praised for their advantages, such as convenience, flexibility, and immediacy, they might have their limitations for some learners.

It was imperative to develop suitable digital business Chinese materials based on learners’ knowledge schema and cognitive structure. If business Chinese materials were only digitized, without considering CLTML, it may be impossible for the NNCSB to learn business Chinese language. Similarly, if only learning volumes are valued, and the learning volumes exceed what NNCSB can absorb, it is difficult to effectively reduce their external cognitive loading. To develop a set of quality e-materials on business Chinese language, what evaluation indicators should be used for measurement? This is a question that teachers and publishers found deeply concerning. Evidence has suggested that considering L2 learners’ cognitive characteristics helps lower their cognitive load (Nawal, 2018). This study, therefore, aimed at constructing the assessment indicators of digital Chinese materials on the basis of CLTML, emphasizing the fact that accessibility, interaction, simple operation, and interface of usage is friendly to NNCSB.

The Current Assessment Indicators for Digital Materials and Courses

Digital foreign language materials are key interactive elements in teaching via Internet technology. They are digitalized tools with dazzling and impressive sound and light effects and make learning effective, by integrating teaching principles and learners’ characteristics (Li & Yang, 2016). In order to allow learners to acquire knowledge effectively, Chapelle (2001) believed that clear content and interactive learning must be considered in the production of digital materials. In studies on digital materials assessment, Peng and Hsiung (2015) established three standard modules (digital, material, and nature of science); eight dimensions (material presentation, interface operation, instructions for use, framework, content, science knowledge, and exploration); 23 sub-dimensions; and 51 indicators for evaluating high quality science via the Delphi technique to collect experts’ suggestions. Li and Yang (2016), through content analyses and interviews, found limitations of school hardware and software, production costs, teacher acceptance, and lack of interaction to be problems existing in the current digital textbooks in elementary schools. Kazaine (2017) used content analysis to summarize four quality criteria—formal, didactic, media, and usability—as well as 27 sub-criteria distributed in the four criteria influencing the quality of teaching materials. Işik (2018) developed a checklist for qualifying, external and internal evaluation, with a total of 229 indicators, for English language teaching (ELT) materials to survey if ELT teachers were satisfied with adopting ELT materials for their students. Tarkowski et al. (2019) suggested considering quality of content, curriculum match, pedagogical models, practices proposed, learning outcomes, and users’ evaluation when compiling open digital materials.

Furthermore, Cai and Chen (2014) addressed digital adult Chinese materials, and other studies (Chiu, 2020; Li & Chen, 2016; Wonchul et al., 2017) focused on general Chinese materials on assessment indicators for Chinese materials, in addition to Chung (2018) and Liao’s (2019) studies mentioned previously. Cai and Chen (2014) generalized four main indicators, nine sub-indicators, and sixty items through three-time Delphi surveys to evaluate the quality of websites regarding digital Chinese learning for adult learners. Li and Chen (2016) designed a prototype tool with four levels based on second language acquisition—(1) Language, (2) Cognition, (3) Teaching Approach, and (4) Students—including 17 indicators and 80 items. Wonchul et al. (2017) reviewed research studies on Chinese textbook evaluation checklists conducted in China and Korea to create the checklist with 45 question items. Chiu (2020) found that the evaluation indicators in textbooks for general Chinese students and those for Indonesian students were basically similar. However, the Indonesian vocabulary, grammar, and culture presented was helpful to study language points and comprehend respective cultures.

In practice, American universities, such as Penn State, Lincoln, and Cheyney, conducted the Innovation in Distance Education (IDE) project funded by an AT&T Foundation grant to initiate distance education (Kashyap, 2014; Malik, 2015; The Pennsylvania State University, 1998), which defined five practice categories (learning goals and content presentation, interactions, assessment and measurement, learner support and services, and instructional media and tools) and 25 guiding principles. The current iteration of the iNACOL National Standards for Quality Online Courses provides a list of 52 standards divided among five sections: content (13 standards); instructional design (11 standards); student assessment (7 standards); technology (11 standards); and course evaluation and support (10 standards) (Kennedy et al., 2018; Michigan Virtual University, 2016). Quality Matters in the MarylandOnline, Inc. (MOL) provided schools with the quality matters rubrics and eight standards: (1) course overview and introduction, which was clarified to the learner at the beginning of the course; (2) learning objectives (competencies), describing what learners could do upon completion of the course; (3) assessment and measurement, which were integral to the learning process and were designed to evaluate learner progress in achieving the stated learning objectives or mastering the competencies; (4) instructional materials, enabling learners to achieve stated learning objectives or competencies; (5) course activities and learner interaction, facilitating and supporting learners’ interaction and engagement; (6) course technology, supporting learners’ achievement of course objectives or competencies; (7) learners’ support, implying that the course facilitates learner access to institutional support services essential to learners’ success; (8) accessibility and usability, wherein the course design reflected a commitment to accessibility and usability for all learners. More than 1,300 colleges and universities worldwide used the rubrics and standards to design and review course, regardless of the media used (Kennedy et al., 2018; Quality Matters, 2021; Texas A&M International University, 2021; University of South California, 2020). Notice of call for proposals for Digital Chinese language materials and course development project by Taiwan Ministry of Education clearly listed that learners, course objectives, content outlines, teaching materials and methods, learning evaluation, and promotion are necessary standards for applicants (Office of Global Mandarin Education, 2020).

Synthesizing the current research and assessment indicators, in addition to user-friendly layout and interface discussed previously in CLTML, this study found that criteria with indicators, such as material features (e.g., Cai & Chen, 2014; Chiu, 2020; Chung, 2018; Işik, 2018; Liao, 2019; Office of Global Mandarin Education, 2020; Quality Matters, 2021; Wonchul et al., 2017), security (e.g., Cai & Chen, 2014; Michigan Virtual University, 2016; Quality Matters, 2021), credible and valid content (e.g., Cai & Chen, 2014; Chapelle, 2001; Chiu, 2020; Chung, 2018; Işik, 2018; Liao, 2019; Office of Global Mandarin Education, 2020; Quality Matters, 2021; Tarkowski et al., 2019), data updates (e.g., Cai & Chen, 2014; Kennedy et al., 2018; Michigan Virtual University, 2016; Quality Matters, 2021), learning needs and results (e.g., Cai & Chen, 2014; Chiu, 2020; Chung, 2018; Liao, 2019; Li & Chen, 2016; Michigan Virtual University, 2016; Quality Matters, 2021; Wonchul et al., 2017), and usage frequency (e.g., Cai & Chen, 2014; Kennedy et al., 2018; Quality Matters, 2021) have been commonly considered and categorized into different but similar dimensions depending on academic interests and practical purposes. Therefore, the arguments and suggestions summarized in references were helpful for this study to develop the preliminary assessment indicators for digital business Chinese materials.

The Preliminary Development of the Assessment Indicators for Digital Business Chinese Materials

Undeniably, the developed standards and assessment indicators of digital materials in research and practice have established general rules that appear to be applicable to assess any digital material, with some of them specific to other subjects and users depending on developers’ research interests. However, effective evaluation indicators must be tailored to digital materials for different subjects and users to check the quality. For example, e-users’ habits for digital interfaces were not considered in the assessment indicators for print materials, and the uniqueness of business Chinese as a professional subject was not considered in the assessment indicators for digital materials and general Chinese materials. Therefore, it was crucial for this study to construct the preliminary PC2U assessment indicators based on the current research on business Chinese, CLTML, and relevant assessment indicators. The PC2U stands for Production, Content, Use, and Usefulness, respectively. Table 1 lists the dimensions and criteria of digital business Chinese materials, and Figure 1 shows 11 criteria and 63 indicators in the PC2U.

Table 1.

The Dimensions and Criteria of Digital Business Chinese Materials.

Dimensions	Criteria	Notes
Production	Material features	For example, learning objectives being set for NNCSB of different nationalities with various Chinese levels; e-materials having sustainable use for latent users, etc.
Production	Security	For example, anti-virus, anti-hacker, and users data loss prevention
Content	Credibility	For example, listing frequently-used business Chinese vocabulary, grammar, and patterns, different usage between Taiwan and China, real-life business situations, etc.
	Data update	For example, regular review and update of business information and learning content, etc.
	Learning needs	For example, the topics and content consistent with business places and meeting the needs and characteristics of NNCSB
Usage	Accessibility	For example, e-materials being compatible with computer operating systems and having sustainable use for latent users, etc.; providing an easy search for materials, both free and charged versions beyond time and space limit, etc.
	Interaction	For example, providing contact information about e-material designers and webmasters, instant online Q & A, etc.
	Simple operation	For example, simple operational functions for e-materials, no network congestion while downloading files, good hyperlink usability, etc.
	Interface	For example, appropriate layout of videos, audios, animations, images, and texts, clear interface, etc.
Usefulness	Learning results	For example, users’ preference of e-business Chinese materials over paper materials, users’ satisfaction with the accessibility of e-business Chinese materials, etc.
Usefulness	Usage frequency	For example, NNCSB’s high willingness to reuse e-business Chinese materials, etc.

Figure 1.

The framework of PC2U.

Research Method

Analysis Hierarchy Process and Super Decision

Constructing evaluation indicators aimed to set educational goals, examine educational quality, and provided users in education with reference information (Johnstone, 1981; Mayston & Jesson, 1991). This study aimed to construct the weight system of the PC2U assessment indicators for digital business Chinese materials and provide educational developers with a set of measurement standards for such digital materials to examine the quality. To this end, the analysis hierarchy process (AHP) was used to establish the weight of indicators. Figure 2 shows an example of the design of the AHP questionnaire for the AHP analysis.

Figure 2.

An example for an AHP questionnaire.

Suppose there were three indicators in an AHP questionnaire. Respondents were asked to check which indicator was more important, and how much more important was it in comparison with the other. After collecting the respondents’ answers, the Super Decision software (Figure 3), offered by Creative Decisions Foundation (http://www.superdecisions.com/downloads/), was used for the AHP analysis. During the AHP analysis, it was necessary to check for inconsistencies was necessary (Mu & Pereyra-Rojas, 2017; Saaty, 2016). Inconsistency measured the logical inconsistency of respondents’ judgments, and the inconsistency ratio, in general, should be <0.1 to be considered reasonably consistent (Saaty, 2016).

Figure 3.

The Super Decision’s operational interface.

Expert Option

The AHP is a widely used method by decision makers and researchers to derive priority scales by measuring through pairwise comparisons and relying on the judgments of experts (Russo & Camanho, 2015). Hence, the expert option is necessary for constructing the weight system of the PC2U assessment indicators for digital business Chinese materials, according to their professional knowledge and experience. Table 2 shows the background of 10 collective TCSOL (teaching Chinese to speakers of other languages)-related experts.

Table 2.

The Background of 10 Collective TCSOL-Related Experts.

Alias	Nation	Alias affiliation	Position	Specialty
CChiB	Taiwan	Department of ACL, TT University	Associate professor	Teaching methods and materials in TCSOL, cultural innovation and marketing
CFengM	Taiwan	Chinese language education center, P University	Assistant professor	Chinese-English translation, Teaching Chinese as a second language
CHisnL	Taiwan	Department of technology and service management, NK University	Associate professor	Second language acquisition in adulthood, Adult learning
ClongV	Vietnam	Department of Chinese language, DL University	Lecture	Chinese-English translation, Chinese language education
CYingC	Taiwan	Language education center, TT institute	Assistant professor	Business English, foreign language teaching methods
CYuX	Sweden	Department of Asian studies, S University	Chinese language teacher	Business Chinese, TCSOL
GGuangS	Japan	Department of Chinese language, KT university	Professor	Teaching Chinese as a foreign language, Chinese-Japanese translation
THuaL	Paraguay	TPP University	Chinese language teacher	TCSOL
LYuL	Taiwan	Department of tourism management	Assistant professor	Business English, TCSOL
YChengC	Taiwan	Teaching profession program, C University	Professor	International education, foreign language teaching

TCSOL = teaching Chinese to speakers of other languages.

Results and Discussions

This study utilized the AHP to construct the weight system for the PC2U assessment indicators, including four dimensions (production, content, usage, and usefulness), 11 criteria (material features, security, credibility, data update, learning needs, accessibility, interaction, simple operation, interface, learning results, and usage frequency), and 63 indicators, which were based on the current research on business Chinese, CLTML, and relevant assessment indicators. The assessment indicators weight system of digital business Chinese materials and its ranking are shown in Table 3.

Table 3.

The Assessment Indicators Weight System of Digital Business Chinese Materials and Its Ranking.

Dimensions	Ranking(D)	Criteria (RW)	Ranking(C)	Indicators (RW)	Ranking(I)
(Relative weight, RW/inconsistency)
Production (0.19/0.00)	3	Material features (0.063/ 0.01)	7	1. Clarity of cognitive, affective, and psychomotor goals toward teaching. (0.0063)	46
				2. Clarity of cognitive, affective, and psychomotor goals toward learning. (0.0069)	42
				3. Clarity of potential users of the materials. (0.0063)	46
				4. Clarity of unit title. (0.0038)	52
				5. Clarity of learning points of each unit. (0.0069)	42
				6. Clarity of learning hours of each unit. (0.0038)	52
				7. Supply of users’ mother tongues as assistant languages. (0.0038)	52
				8. Supply of invisible modes to hide users’ mother tongues as assistant languages. (0.0038)	52
				9. Conformity to the principles of creative commons. (0.0038)	52
				10. Auto notice to users of any updates. (0.0038)	52
				11. Use of user satisfaction surveys for modifying the materials. (0.0069)	42
				12. Prospect of potential users to maintain sustainable use of the materials. (0.0069)	42
		Security (0.127/ 0.02)	3	1. Clear separation of the accounts and passwords between material management maintainers and users; prevention of personal data from leaking. (0.0368)	6
				2. Regular checks on data to prevent viruses and hackers. (0.03175)	10
				3. Perfect login system encryption for the Content Management System (CMS). (0.03175)	10
				4. Perfect login system encryption for users. (0.0267)	12
Content (0.39/ 0.00)	1	Credibility (0.156/ 0.01)	1	1. Listed are the most common business-related Chinese vocabulary, patterns, and grammar. (0.00468)	49
				2. Compared and explained are the differences and similarities of business Chinese vocabulary, patterns, and grammar between Taiwan and China. (0.00312)	62
				3. The texts and dialogues reflect authentic business culture and situations in Chinese societies. (0.00936)	35
				4. Included in the content of topics are resumes, autobiographies, and interview skills for employment. (0.00936)	35
				5. Included in the content of topics are communication skills and etiquette by phones. (0.00936)	35
				6. Included in the content of topics are reading and writing on business, and emails. (0.01092)	32
				7. Included in the content of topics is formal and informal business social communication. (0.01248)	27
				8. Included in the content of topics are paper and oral reports for meetings. (0.01248)	27
				9. Included in the content of topics is negotiating with Chinese businessmen. (0.01404)	25
				10. Included in the content of topics are Chinese economic development and tendency, industrial development, trade, and market trend. (0.01404)	25
				11. Included in cultural topics are social phenomena and life style. (0.01248)	27
				12. Included in cultural topics are Chinese business and industrial organization systems, economic and trade laws, currency and bank systems, and policies on Chinese economic plans. (0.01248)	27
				13. Included in cultural topics are Chinese communication styles and the most common words spoken by Chinese people in specific occasions. (0.0156)	22
				14. Included in cultural topics are dos and don’ts in Chinese specific business occasions, and the gift-giving culture. (0.0156)	22
		Data update (0.078/ 0.00)	6	1. Regular checks and update on information about Chinese business situations and learning content. (0.0351)	7
				2. Indication of the last updated date. (0.0078)	40
				3. Regular update on software and operating system (OS). (0.0351)	7
		Learning needs (0.156/ 0.008)	1	1. The content of the topics comprises needs in Chinese business situations. (0.0234)	15
				2. The content of the topics fits the needs and characteristics of business Chinese L2 learners. (0.05148)	2
				3. The design of the number of topics is based on business Chinese L2 learners’ use frequency and learning hours. (0.02496)	13
				4. The content of the topics fits non-native Chinese speaking situations where the business Chinese L2 learners are located. (0.05616)	1
Usage (0.28/ 0.078)	2	Accessibility (0.126/ 0.013)	4	1. It is easy to search the materials using keywords related to “business” and “Chinese”. (0.02142)	16
				2. The materials can be used any time except when data are updated. (0.02142)	16
				3. The materials have both free and charged versions. (0.02394)	14
				4. Prices and discounted prices are shown. (0.01764)	20
				5. The versions of the materials are compatible with different operating systems of computers, iPad, and smartphones. (0.04158)	5
		Interaction (0.059/ 0.005)	8	1. The contact information of the material designers and web maintenance personnel is provided. (0.0077)	41
				2. The online instant messengers and internet forums are provided. (0.0348)	9
				3. Social community webpages, such as Weibo, Facebook, and Twitter), are created for materials. (0.0165)	21
		Simple operation (0.053/ 0.023)	9	1. Easy operational function. (0.0148)	24
				2. Fast and fluent download of videos, files, and other data. (0.0085)	38
				3. Normal hyperlink of functions. (0.0085)	38
				4. The materials ready to be used without installing other accessories and complicated applications. (0.0212)	18
		Interface (0.042/ 0.004)	10	1. Appropriate layout for videos, audios, animation, images, and words. (0.00504)	48
				2. Clear and reasonable navigation and instruction. (0.00462)	50
				3. Human operation of the functional interfaces. (0.0105)	33
				4. Top quality of the words, audios, images, videos, and animation in the materials. (0.01134)	31
				5. Consistency of main page interface with that of the hyperlinked subpages. (0.0105)	33
Usefulness (0.14/ 0.00)	4	Learning results (0.035/ 0.00)	11	1. Users prefer e-business Chinese materials to paper ones. (0.00175)	63
				2. Users are satisfied with the accessibility and the usability of the e-business Chinese materials. (0.00385)	51
				3. The materials are helpful for learners in increasing business Chinese knowledge, such as pinyin, vocabulary, patterns, and grammar. (0.00375)	58
				4. The materials are helpful for learners in learning listening, speaking, reading, writing and translation skills in business Chinese. (0.00375)	58
				5. The materials are helpful for learners in having better cross-cultural comprehension in business Chinese. (0.00375)	58
				6. The materials are helpful for learners in increasing social and communicative proficiency in business Chinese. (0.00375)	58
		Usage frequency (0.105/ 0.00)	5	1. Users show more willingness to reuse the e-business Chinese materials. (0.042)	3
				2. Users show more willingness to use the new or republished edition of the e-business Chinese materials. (0.042)	3
				3. Users show more willingness to introduce and promote the e-business Chinese materials to others. (0.021)	19

Note. Stands for the top 50% of the dimensions, criteria, and indicators. The inconsistency value of the dimensions is 0.045.

The Weight System in Dimensions

As shown in Table 3, the inconsistency value is 0.045, which is less than 0.1, implying that the level of consistency is acceptable. It is of foremost priority, with respect to the four dimensions, to access digital business Chinese content, including credibility, data update, and learning needs. Its weight score is 0.39, higher than the other three. The second priority is the usage, including accessibility, interaction, simple operation, and interface; its weight score is 0.28. The last two are production, containing material features and security, and usefulness, containing learning results and usage frequency; the RWs are 0.19 and 0.14, respectively. Content as an index, according to various developing foreign language textbook checklists (Cai & Chen, 2014; Chung, 2018; Ghaderi Doust, 2016; Karimnia & Jafari, 2017; Liao, 2019; Wonchul et al., 2017), is a key evaluative item; the results of these studies are consistent with the findings of the present study.

The Weight System in Criteria

As shown in Table 3, with respect to the three criteria in the content dimension to assess digital business Chinese, learning needs is as important as the credibility criteria, and the RW of both is 0.156 higher than others. The learning needs criterion implies that the topics and content are consistent with business places and meets the needs and characteristics of NNCSBs. The credibility criterion implies listing frequently-used business Chinese vocabulary, grammar, and patterns, different usages between Taiwan and China, and real-life business situations. The inconsistency value of the content equals 0.00, which means the level of consistency is consistent. For schools and teachers, selecting and utilizing e-learning resources have not only managed to create space for effective learning, but also been able to cater to students’ diverse needs (Egbert, 2017). Learning needs play a vital role in developing language textbooks, but students’ real needs are not considered in these textbooks (Moiinvaziri, 2014). From this, it can be seen that considering learning needs is significant, regardless of the type of teaching textbook.

Few are opposed to the importance of listing frequently used and correct Chinese vocabulary, grammar, and patterns in Chinese language textbooks, as well as digital business Chinese materials. Regarding real-life business situations, Gilmore (2007) found that using authentic materials in foreign language learning has a long history, even though debates exist between authentic and contrived materials. This study found the importance of credibility to assess digital business Chinese materials reaches a consensus.

With technological advances, various computer operating systems were developed. Unlike print textbook, the anti-virus software was essential when computers were connected to the internet (Savage & Barnett, 2015); thus, the security criterion, including anti-virus, anti-hacker, and users’ data loss prevention, was necessary to access digital business Chinese materials. Table 3 shows that the RW of the security criterion is 0.127, which was ranked third. The result proved that considering the security criterion was a priority over the material features. In addition, the inconsistency value of the production dimension equals 0.00, which means the 10 scholars and experts’ consensus is unanimous.

Different from print textbooks of business Chinese language, the accessibility, interaction, simple operation, and interface criteria are aimed for digital business Chinese materials. Accessibility means providing an easy search for materials, both free and charged versions beyond time and space limit. As shown in Table 3, with respect to the four criteria in the usage dimension to access digital business Chinese, the accessibility (RW = 0.126) is more important than the other three, and it ranks fourth among all criteria. Weir (2012) deemed that accessing content anytime from anywhere, with no simultaneous user restrictions, online availability ahead of print, easy searchability, and linking to cited materials are the potential benefits of online resources. This study also found that the advantages of digital business Chinese materials, especially accessibility, should be considered.

The usage frequency criterion in this study describes NNCSBs’ high willingness to reuse e-business Chinese materials, and the learning results criterion describes users’ preference of e-business Chinese materials over paper materials and users’ satisfaction with the accessibility of e-business Chinese materials. The features of e-materials as open educational resources include adaptability and reusability (Huertas et al., 2018). Reusing is important for materials to remain sustainable (Danso, 2018). With respect to the usage frequency and learning results criteria in the usefulness dimension to assess digital business Chinese, this study found that the usage frequency criterion (RW = 0.105) ranked fifth, which is higher than the learning results criterion. The inconsistency value of the usefulness dimension is 0.00, which means the consensus is consistent.

The next priorities in order of importance include the following: data update (RW = 0.078); material features (RW = 0.063); interaction (RW = 0.059); simple operation (RW = 0.053); interface (RW = 0.042); and learning results (RW = 0.035).

The Weight System in Indicators

Table 3 shows that the inconsistency of the 11 criteria for 63 indicators is <0.1, which means the level of consistency is acceptable.

Compiling Country-specific Chinese language learning materials has become valued since Jianxi Cui proposed such a concept (Cui, 2005; Dong, 2017), and there is little doubt of the importance of learning needs in compiling foreign language materials. This study found that the content of the topics fits non-native Chinese speaking situations where the business Chinese L2 learners are located geographically (RW = 0.05616) and the content of the topics fits the needs and characteristics of business Chinese L2 learners (RW = 0.05148) hold the positions of the first and second priority, respectively; thus, the argument is proven.

Assessing and evaluating e-book usage was a complex and challenging task, but the availability of reliable usage statistics to support purchase decisions was vital (Conyers et al., 2017). In addition, digital textbooks that are frequently used can remain sustainable. This study also found that the third priority is given to users show more willingness to reuse the e-business Chinese materials (RW = 0.042); this is equally important to users show more willingness to use the new or republished edition of the e-business Chinese materials (RW = 0.042).

Accessibility is a key item to evaluate e-materials’ quality (Estabrook & Arashiro, 2004; Kazaine, 2017; Rahrouh et al., 2018; Schoor & Körndle, 2012). There are five indicators in the accessibility criterion in this study, and as shown in Table 3, the versions of the materials are compatible with different operating systems of computers, iPad, and smartphones holds the fifth priority (RW = 0.04158).

Security was an indicator to evaluate the effectiveness of open textbooks (The Open University of Hong Kong, 2016). Recently, the popular video-conferencing program Zoom failed to prevent personal information from leaking (Iyengar, 2020; Wakefield, 2020). For information security, the Taiwan Ministry of Education immediately issued official documents to prohibit schools from using Zoom for online teaching. It is thus clear that security is vital to digital devices. This study found that two indicators, clear separation of the accounts and passwords between material management maintainers and users, and prevention of personal data from leaking ranked sixth. Through a Delphi survey, Cai and Chen (2014) found that update digital Chinese materials for adults anytime, anywhere is subordinate as an indicator. E-materials contain newer content was a selective indicator in the 2012 certification indicators of digital learning materials of the Taiwanese Ministry of Education. Nevertheless, being able to provide an immediate data update is one of the advantages of e-materials, and this study found regular checks and updates on information about Chinese business situations and learning content was equally important to regular update on software and operating system (OS), both of which resulted in RW of 0.0351 and ranked seventh. Bikowski and Casal (2018) found that students can have high expectations and feel engaged throughout the term when using a customized interactive digital textbook, and it was important to consider making e-textbooks interactive to better engage students (Li et al., 2018; Maag, 2016; Vilar & Zabukovec, 2017). With respect to the three indicators in the interaction criterion of the usage dimensions to access digital business Chinese, it was found that the online instant messengers and internet forums are provided was the ninth in priority (RW = 0.0348). The 10th priorities were awarded to regular checks on data to prevent viruses and hackers (RW = 0.03175) and perfect login system encryption for the Content Management System (CMS) (RW = 0.03175). The remaining indicator rankings are shown in Table 2.

Each assessment indicator has its own value to improve the quality of digital business Chinese materials. Presently, the profession has begun to note the need to identify effective assessment indicators for Chinese (e-) materials. The studies of Cai and Chen (2014), Chiu (2020), Chung (2018), Li and Chen (2016), Liao (2019), and Wonchul et al. (2017) are examples of such efforts. Despite these gallant efforts to exemplify practices of constructing assessment indicators for Chinese (e-) materials through qualitative or quantitative approaches, little emphasis has been placed on the usefulness of these assessment indicators in the 21st century, beyond research methods and tools to compare these indicators’ priorities. Specifically, efforts are lacking in terms of preparing an effective checklist with weights for assessing digital business Chinese materials. In fact, it is helpful to provide teachers with systematic checklists to evaluate the quality of materials and decide whether to use them (Işik, 2018). Therefore, this study contributes to constructing the assessment indicators weight system for digital business Chinese materials by using the analytic hierarchy process methods and suggests that a checklist, based on Table 3, can be developed for Chinese language institutes, researchers, teachers, and publishers to evaluate the quality of digital business Chinese materials.

Conclusion

Practical Application

Using ideas from earlier sections, such as the proposed assessment indicators weight system, this study presents recommendations for Chinese language teachers and publishers to use the system as a checklist to review the quality of digital business Chinese materials, whether they are published or used as textbooks in class.

According to the RW ranking, the top 50% of the dimensions, criteria, and indicators are recommended as required standards, while other remaining indicators are suggested as selective. In the production dimension, there are two criteria: material features, including 12 selective indicators, and security, including four required indicators. In the content dimension, there are three criteria: credibility, including 14 indicators (eight as required and six as selective); data update, including three indicators (two as required and one as selective); and learning needs, including four required indicators. In the usage dimension, there are four criteria: accessibility, including five required indicators; interaction, including three indicators (two as required and one as selective); simple operation, including four indicators (two as required and two as selective); and interface, including five indicators (one as required and four as selective). In the usefulness dimension, there are two criteria: learning results, including six selective indicators; and usage frequency, including three required indicators.

In the rating standard, “A” means that the material is superior to the verification standard in the item; “B” means that the material meets the verification standard in the item; “C” means that the material does not meet the verification standards in the item; “N/A” means that the material does not show anything that corresponds with the item, or what is shown in the material cannot correspond with the item. In the scoring standard, if the material reaches “A” in an item, it is a required indicator weighted to calculate with six points and is a selective indicator calculated with three points; if the material reaches “B” in an item, it is a required indicator weighted to calculate with four points and is a selective indicator calculated with two points; if scored as “C” in an item, it is calculated by one point, regardless of whether it is a required or selective indicator; and finally, if scored as “N/A” in an item, it is calculated with zero points, regardless of whether it is a required or selective indicator.

Limitation and Suggestions for Future Research

This study initially invited 10 collective TCSOL-related experts (eight in the academic TCSOL and two in the practical TCSOL) to answer the AHP questionnaire for constructing the assessment indicators weight system for digital business Chinese materials. The common advantage of the AHP method and the Delphi Technique is that both rely on the judgments of experts to reach a consensus. The AHP method uses a measurement through experts’ pairwise comparisons to derive priority scales, whereas the Delphi technique refers to a research method in which experts are asked to express their opinions and score each item to obtain a consensus through a four-time survey. The two research methods have their respective contributions to survey research; therefore, both could be combined for future research.

Second, the AHP assumes that each component in different hierarchies must be independent, but most of the components are mutually dependent in real world. Such an assumption in evaluating decision-making problems does not meet human values and activities in real world (Saaty & Begicevic, 2010). To improve the limitations of AHP, the Analytical Network Process (ANP), also developed by Saaty to solve the AHP limitation, considers mutual influences between each component and adjacent levels to use the “supermatrix” to comprehensively analyze components interacting and influencing each other to calculate its mixed weight. Therefore, future research should utilize the ANP to make the evaluation model more complete, instead of using the AHP.

Third, it is advisable to invite those with practical Chinese experience to participate in a future study of constructing assessment indicators. In addition to the theoretical foundation for the development of the assessment indicators for digital business Chinese, experienced Chinese language teachers’ and publishers’ views play an important role to examine the assessment indicators. Thus, future studies should invite more experienced and practical TCSOL teachers, as well as publishers, to participate in the follow-up study and bridge perspectives between theory and practice.

Studies on constructing the assessment indicators for TCSOL materials must be ongoing. Using digital devices to teach Chinese language is trending, especially since the outbreak of the COVID-19 pandemic. However, there are still few rigorous and empirical studies on constructing the assessment indicators for digital TCSOL materials, specifically, the weight system for digital business Chinese materials, even though most non-native Chinese speakers study Chinese for business purposes. Although the population of Chinese learners has gradually extended from adults to teenagers, children, and preschool children, adult Chinese L2 learners remain the majority. Hence, this exploratory study can reap the benefits of digital business Chinese materials in a future study. The TCSOL field is concerned with target learners. Age, nationality, and Chinese language level are key independent variables to influence Chinese L2 learners’ performance. If only a single set of criteria is used to measure all digital Chinese materials, its applicability can be debatable. Therefore, future studies can refer to this study, as well as previous research results on assessment indicators for digital language materials, while constructing assessment indicators weight systems for digital Chinese materials for various Chinese L2 learners.

Footnotes

Acknowledgements

This research paper was made possible through the help and contribution of various people. Especially, I would like to express my very great appreciation to my like-minded friends, Maureen H. Chang (Stanford University) and Dr. Feng-Mei Chao (Chinese Language Education Center, Providence University) for their proofreading during the development of this research paper, as well as Section Editor and Reviewers for their valuable and constructive comments, and Editage () for English language editing, to improve my research paper. I would also like to thank the ten collective TCSOL (teaching Chinese to speakers of other languages)-related experts for their help in answering my questionnaire of analytic hierarchy process. Their willingness to give their time so generously has been very much appreciated. I would also like to extend my thanks to Creative Decisions Foundation for the free Super Decision software, so that I can conduct analytic hierarchy process by it. Finally, I wish to thank my family for their support and encouragement throughout my research.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Taichung University of Education [grant number F106202].

ORCID iD

Qiao-Yu Warren Cai

References

Asraj

A. A.

Freeman

Chandler

P. A.

(2011). Considering cognitive load theory within e-learning environments. In Seddon

P. B.

Gregor

(Eds.), PACIS 2011 proceedings (pp. 1–13). Queensland University of Technology.

Bikowski

Casal

J. E.

(2018). Interactive digital textbooks and engagement: A learning strategies framework. Language Learning & Technology, 22(1), 119–136.

Cai

Q.-Y.

Chen

Y. C.

(2014). The study of constructing the assessment indicators of digital Chinese materials for adults. Journal of Chinese Language Teaching, 11(2), 87–125.

Chapelle

C. A.

(2001). Computer applications in second language acquisition: Foundations for teaching, testing and research. Cambridge University Press.

Chen

L.-Y.

Lee

C. H.-H.

, (2012a). Culture study of fundamental business Chinese teaching material. Journal of Chinese Language Teaching, 9(4), 41–73.

Chen

L.-Y.

Lee

C. H.-H.

(2012b). Lexical analysis of business Chinese teaching material: A study in teacher trainees’ needs and perceptions. Taiwan Journal of Chinese as a Second Language, 5, 15–42.

Chen

Y. C.

(2011). The study of digital English learning materials for adults: Construction and further implication of assessment indicators [Unpublished doctoral dissertation]. National Chung Cheng University.

Chiu

P. Y.

(2020). The study of constructing the assessment indicators of country-specific Mandarin materials-basing on the experience of textbook writing for Indonesian students [Unpublished master’s thesis]. Chung Yuan Christian University.

Chung

Y. U.-S. Y. U. A. N.

(2018). The construction of 5C-based evaluation indicator for business Chinese textbooks [Unpublished mater thesis]. National Taichung University of Education.

10.

Conyers

Lambert

Wong

Jones

Bamkin

Dalton

(2017). E-book usage: Counting the challenges and opportunities. Insight, 30(2), 26–33. https://doi.org/10.1629/uksg.370

11.

Cui

J. Ɑ. N.-X.

(2005). Explorations of Chinese teaching in eastern Canada. Chinese Language Learning, 6, 69–75.

12.

Curum

Khedo

K. K.

(2021). Cognitive load management in mobile learning systems: Principles and theories. Journal of Computers in Education, 8(1), 109–136. https://doi.org/10.1007/s40692-020-00173-6

13.

Danso

(2018). Identification of key indicators for sustainable construction materials. Advances in Materials Science and Engineering, 2018, 1–7. https://doi.org/10.1155/2018/6916258

14.

Davids

M. R.

Halperin

M. L.

Chikte

U. M. E.

(2015). Optimising cognitive load and usability to improve the impact of e-learning in medical education. African Journal of Health Professions Education, 7(2), 147–152. https://doi.org/10.7196/ajhpe.659

15.

Dong

L.-L.

(2017). Compiling Chinese textbooks for specific countries in international Chinese education. In Globalized Chinese language education: Teaching and research (pp. 178–191). Minzu University of China.

16.

Egbert

(2017). Methods of education technology: Principles, practice, and tools. Pressbooks.

17.

Estabrook

Arashiro

(2004). Standards for online courses: Can we do it? Yes we can! In Anne-Marie

(Ed.), Instructional design in the real world: A view from the trenches (pp. 160–183). IGI Global.

18.

Ghaderi Doust

(2016). A comparative study on objectives and components of reading skill in national curriculum of Iran and America (New Jersey) at high school. International Journal of Language Education and Teaching, 4(3), 48–60. https://doi.org/10.18298/ijlet.631

19.

Gilmore

(2007). Authentic materials and authenticity in foreign language learning. Language Teaching, 40, 97–118. https://doi.org/10.1017/s0261444807004144

20.

Guan

D.-X.

(2012). Applications of translation tasks in business Chinese instruction. Teaching Chinese as a Second Language, 12, 6–17. https://doi.org/10.7083/TCASL.201206.0006

21.

Huertas

Biscan

Ejsing

Kerber

Kozlowska

Ortega

S. M.

Lauri

Risse

Schörg

Seppmann

(2018). Considerations for quality assurance of e-learning provision. European Association for Quality Assurance in Higher Education.

22.

Işik

(2018). ELT materials evaluation: A system and criteria. Theory and Practice in Language Studies, 8(7), 797–812. https://doi.org/10.17507/tpls.0807.11

23.

Iyengar

(2020). Zoom CEO apologizes for having ‘fallen short’ on privacy and security. CNN Business. https://edition.cnn.com/2020/04/02/tech/zoom-ceo-apology-privacy/index.html

24.

Johnstone

J. N.

(1981). Indicators of education systems. Kogan Page.

25.

Karimnia

Jafari

F. M.

(2017). Critical ESP textbook evaluation: The case of visual arts textbook. Sustainable Multilingualism, 11, 219–236.

26.

Kashyap

S. G.

(2014). A case study to examine institutional factors facilitating and inhibiting faculty preparation for teaching in an online MBA program [Unpublished doctoral dissertation]. The Pennsylvania State University.

27.

Kazaine

(2017). Evaluating the quality of e-learning material. Environment. Technology. Resources, 2, 74–77.

28.

Kennedy

Tomaselli

Stimson

(2018). National standards for quality online courses (K-12) and QM K-12 secondary and K-12 publisher rubric revision literature review. MarylandOnline.

29.

Kirschner

P. A.

Sweller

Kirschner

Zambrano

R. J.

(2018). From cognitive load theory to collaborative cognitive load theory. International Journal of Computer-Supported Collaborative Learning, 13(2), 213–233. https://doi.org/10.1007/s11412-018-9277-y

30.

Lambert

Kalyuga

Capan

L. A.

(2009). Student perceptions and cognitive load: What can they tell us about e-learning web 2.0 course design? E-Learning, 6(2), 150–163. https://doi.org/10.2304/elea.2009.6.2.150

31.

Liao

Y. I.-H. S. U. A. N.

(2019). The study of constructing the assessment indicators of business Chinese learning materials [Unpublished master thesis]. Chung Yuan Christian University.

32.

Liao

Y. U.-H. U. I.

(2011). A study on intermediate spoken communication course for business Chinese: Analysis, design, and implementation [Unpublished master’s thesis]. National Taiwan Normal University.

33.

H.-Y.

Yang

K.-Y.

(2016). Current situation of and problems in digital textbook instructional design. Tsing Hua Journal of Educational Research, 34(1), 71–104. https://doi.org/10.3966/252190062017063401003

34.

Yamaguchi

Takada

J. I.

(2018). The influence of interactive learning materials on self-regulated learning and learning satisfaction of primary school teachers in Mongolia. Sustainability, 10(4), 1093–1119. https://doi.org/10.3390/su10041093

35.

Liu

Zhou

J. W.

Yao

(2013a). Research on the design and teaching practice for online business Chinese materials. The 8th International Conference on Internet Chinese Education, Pasadena City College, Pasadena, CA, USA.

36.

Liu

Zhou

J. W.

Yao

(2013b). Exploring teaching global business Chinese materials and methods in a class. The International Symposium on Chinese Education for the 60th Anniversary Academic Activities of Chinese College, Jinan University, Guang Zhou.

37.

Liu

Chen

Y. C.

Cai

Q. I. A. O.-Y. U.

Yao

Zhou

J. W.

(2014a). Exploring the teaching design and teaching practice for business Chinese in a Chinese class at a university of northern Taiwan. 4th Cross-Strait Chinese Teacher Forum, World Chinese Language Association and Chinese College Of Beijing, Beijing, China.

38.

Liu

Zhou

J. W.

Cai

Q. I. A. O.-Y. U.

Yao

Chen

Y. C.

(2014b). Material design for business Chinese and its teaching practice in a Chinese class at a university of central Taiwan [Conference paper]. The 11th Conference on World Chinese Language Teaching Research, World Chinese Language Association and National Taipei University of Education, Taipei.

39.

Y.-C.

Chen

J.-Y.

(2016). A preliminary investigation of the criteria for evaluating Chinese teaching materials: A second language acquisition perspective. Taiwan Journal of Chinese as a Second Language, 13, 67–94.

40.

Y.-J.

(2011). Learner’s needs analysis in business Chinese. Journal of Chinese Language Teaching, 8(3), 23–46.

41.

Maag

(2016). Proactive and interactive: An examination of e-textbook and considerations on how to improve them. Ball State University.

42.

Malik

S. K.

(2015). Strategies for maintaining quality in distance higher education. Turkish Online Journal of Distance Education, 16(1), 238–248.

43.

Mayer

R. E.

(2001). Multimedia learning. Cambridge University Press.

44.

Mayer

R. E.

Moreno

(1998). A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Education & Psychology, 90(2), 312–320.

45.

Mayer

R. E.

Moreno

(2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43–52.

46.

Mayston

D. J.

Jesson

D. S. J.

(1991). Educational performance assessment: A new framework of analysis. Policy and Politics, 19, 99–108.

47.

Michigan Virtual University. (2016). Guidelines and model review process for online courses. Michigan Virtual University.

48.

Moiinvaziri

(2014). Students’ voice: A needs analysis of university general English course in Iran. Journal of Language Studies, 14(01), 57–75.

49.

Pereyra-Rojas

(2017). Practical decision making: An introduction to the analytic hierarchy process (AHP) using super decisions v2. Springer International Publishing AG.

50.

Nawal

A. F.

(2018). Cognitive load theory in the context of second language academic writing. Higher Education Pedagogies, 3(1), 385–402. https://doi.org/10.1080/23752696.2018.1513812

51.

Office of Global Mandarin Education. (2020). A guideline to the Taiwan Ministry of Education for promoting the development of digital Chinese language course project. https://ogme.edu.tw/files/%E6%95%99%E8%82%B2%E9%83%A8%E8%BE%A6%E7%90%86%E9%9D%9E%E5%90%8C%E6%AD%A5%E8%8F%AF%E8%AA%9E%E6%95%B8%E4%BD%8D%E6%95%99%E5%AD%B8%E8%AA%B2%E7%A8%8B%E9%96%8B%E7%99%BC%E8%A8%88%E7%95%AB%E5%BE%B5%E4%BB%B6%E9%A0%88%E7%9F%A5.pdf

52.

Peng

W.-H.

Hsiung

C.-T.

(2015). Establishment and evaluation of high quality science e-textbook evaluation indicators. Journal of Textbook Research, 8(2), 1–38.

53.

Quality Matters. (2021). Course design rubric standards (6th ed.). Quality Matters, QM. https://www.qualitymatters.org/qa-resources/rubric-standards/higher-ed-rubric

54.

Rahrouh

Taleb

Mohamed

E. A.

(2018). Evaluating the usefulness of e-learning management system delivery in higher education. International Journal of Economics and Business Research, 16(2), 162–181.

55.

Railean

E. A.

(2017). User interface design of digital textbooks: How screens affect learning. Springer.

56.

Russo

R. D. F. S.

Camanho

(2015). Criteria in AHP: A systematic review of literature. Procedia Computer Science, 55, 1123–1132.

57.

Saaty

R. W.

(2016). Decision making in complex environments. Creative Decisions Foundation.

58.

Saaty

T. L.

Begicevic

(2010). The scope of human values and human activities in decision making. Applied Soft Computing, 10, 963–974. https://doi.org/10.1016/j.asoc.2010.04.002

59.

Savage

Barnett

(2015). Digital literacy for primary teachers. Critical Publishing.

60.

Scarpetta

Quintini

(2020). The potential of online learning for adults: Early lessons from the COVID-19 crisis. OECD. https://www.oecd.org/coronavirus/policy-responses/the-potential-of-online-learning-for-adults-early-lessons-from-the-covid-19-crisis-ee040002/

61.

Schoor

Körndle

(2012). Checklist for a didactically sound design of eLearning content. eLearning Papers, 29, 1–6.

62.

Sweller

Chandler

(1994). Why some material is difficult to learn. Cognition and Instruction, 12(3), 185–233.

63.

Sweller

van Merrienboer

J. J. G.

Paas

F. G. W. C.

(1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296.

64.

Tang

J. T.

(2012). Analysis for the current situation of Chinese language digital teaching materials. Educators and Professional Development, 29(3), 67–82.

65.

Tarkowski

Sitek

Strycharz

Vuorikari

Castaño

M. J.

(2019). Methodological guide for implementation and evaluation of open e-textbook programs: Impact on costs and learning outcomes. European Union.

66.

Texas A&M International University. (2021). The quality matters rubric. https://www.tamiu.edu/qm/about/qmrubric.shtml

67.

The Open University of Hong Kong. (2016). Defining a system of indicators for evaluation the effectiveness of e-learning. http://www.opentextbooks.org.hk/ditatopic/24851

68.

The Pennsylvania State University. (1998). An emerging set of guiding principles and practices for the design and development of distance education. The Pennsylvania State University.

69.

Tseng

S. Y.-L.

Jih

H.-C. S.-C.

, (2007). An analysis on the style and interaction of online Chinese reading materials [Conference paper]. The 5th International Conference on Internet Chinese Education, Overseas Community Affairs Council of Taiwan and National Taiwan Normal University, Taipei.

70.

C.-M.

(2011). Teaching materials for business Chinese: An analysis and suggestions. Journal of Chinese Language Teaching, 8(2), 81–98. https://doi.org/10.6393/JCLT.201108.0081

71.

University of South California. (2020). Quality matters™ rubric. https://sc.edu/about/offices_and_divisions/cte/instructional_design/distributed_learning/quality_matters/index.php

72.

van Merriënboer

J. J. G.

Ayres

(2005). Research on cognitive load theory and its design implications for e-learning. Educational Technology Research and Development, 53(3), 5–13. https://doi.org/10.1007/bf02504793

73.

Vilar

Zabukovec

(2017). Using e-materials for study: Students’ perceptions vs. Perceptions of academic librarians and teachers. Information Research, 22(1), 1–18.

74.

Q. U. Y.-S. O. N.

(2011). The research about the need and development of business Chinese in Vietnam [Unpublished master’s thesis]. National Taiwan Normal University.

75.

Wakefield

(2020). Zoom boss apologises for security issues and promises fixes. BBC News. https://www.bbc.com/news/technology-52133349

76.

Weir

R. O.

(2012). Managing electronic resources: A LITA guide. American Library Association.

77.

Westlake

(2019). Cognitive load theory and multimedia. In Power

(Ed.), Technology and the curriculum: Summer 2019 (pp. 34–50). Power Learning Solutions.

78.

Wonchul

Narae

Sohn

N. H.

Lee

Y. S.

Lee

Kim

S. Y.

(2017). A study on the evaluation checklist for Chinese textbooks. Chinese Language Education and Research, 25(25), 167–201.

79.

Yan

S. J.

(2013). History and present state of teaching business Chinese in Taiwan–With an analysis of course materials [Unpublished mater thesis]. National Cheng Chi University.

80.

Zheng

Y.-W.

(2012). A research on, and the teaching of, translated Chinese new business words: Stock market taken as the research subject. Journal of Applied Chinese, 11, 225–248.

Constructing Assessment Indicators Weight System of Digital Business Chinese Materials

Abstract

Keywords

Introduction

Literature Review

Current Research Performance on Business Chinese

The CLTML and Inspiration

The Current Assessment Indicators for Digital Materials and Courses

The Preliminary Development of the Assessment Indicators for Digital Business Chinese Materials

Research Method

Analysis Hierarchy Process and Super Decision

Expert Option

Results and Discussions

The Weight System in Dimensions

The Weight System in Criteria

The Weight System in Indicators

Conclusion

Practical Application

Limitation and Suggestions for Future Research

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

ORCID iD

References