Sage Journals: Discover world-class research

Abstract

The COVID-19 pandemic has been forcing people to work remotely in virtual teams around the globe. Global virtual teamwork will continue, and people are not sufficiently prepared for this, resulting in reduced team commitment and lower performance. Higher education institutions need to equip their graduates with International Online Collaboration Competencies (IOCCs), but research into these is fragmented, lacking even a definition of these competencies. This study was systematically reviewing empirical studies on IOCCs. 516 studies were reviewed, and data from 14 full texts were analyzed. Six competence domains emerged from the literature. Most studies focused on single domains of IOCCs, and none of the studies covered all domains. Results indicate that this preliminary framework for higher education students provides a first overview of the fragmented literature on IOCCs. Methods to teach and evaluate IOCCs acquisition are underdeveloped but urgently needed to equip professionals for global virtual teamwork.

Keywords

international online collaboration internationalization of higher education International Online Collaboration Competencies virtual teams global teams

Introduction

In light of the COVID-19 pandemic, one strategy to promote flexibility and safety of the workforce and students during lockdown is international online collaboration (IOC) in virtual teams (VTs) (Saavedra, 2020). VTs consist of two or more persons who collaborate interactively to achieve shared goals, while at least one of the team members works at a different location, organization, or at a different time so that communication and coordination are predominantly based on electronic communication media (Hertel et al., 2005). Significantly, collaboration involves mutual interaction and is not simply a matter of division of tasks (Dolmans et al., 2005). The COVID-19 pandemic has been an accelerator of IOC worldwide (Stiedl & Straub, 2020), but VTs were on the rise even before the present crisis because they allow teams to work more flexibly and greener by traveling less (Olaisen & Revang, 2017). Indeed, most working environments today involve virtual characteristics, and what varies is the extent of virtuality, which can be understood as a continuum from low (only face-to-face meetings) to high (only information and communication technology [ICT]-mediated communication) (Mihhalova, 2006). Previously, co-located team members used virtual tools occasionally, whereas now they are required to work mainly virtually.

The importance of building students’ global competency has gained traction recently and has been recognized across many universities (Deardorff & Arasaratnam-Smith, 2017). International Online Collaboration Competencies (IOCCs), a term used to summarily refer to this skill set, are important for professionals in the 21st century (Bersin et al., 2017; Geisinger, 2016) and thus need to be taught to students in higher education. For this purpose, the digitalization of international education is an important opportunity for IOC and the acquisition of IOCCs, as it offers international and intercultural virtual exchange during and after the COVID-19 pandemic. A recent survey reported increased virtual mobility and/or IOC as alternatives to physical student mobility (Marinoni et al., 2020). This deems important because students who have experienced diversity in sociocultural and linguistic environments provide valuable intellectual resources (Senyshyn, 2019) and develop skills for communicating and working effectively with diversity (Arkoudis et al., 2013). However, IOCCs as an outcome promoted by IOC have scarcely been explored (Gilson et al., 2015; Wang & Haggerty, 2011). This impairs measuring the effectiveness of the digitalization of international education and the development of this field (Deardorff, 2006).

Despite the technological sophistication of many students, studies show they are not prepared for online collaboration, because the aspects of virtuality and internationality add additional challenges to collaborating, for example, in communication, technology use, leadership, project management, and cultural diversity (Cleary et al., 2019; Robey et al., 2000). There is, thus, a need to train IOCCs for effectively working in international VTs (Ford et al., 2017; Wang & Haggerty, 2011). Effective or successful IOC is characterized by completed collaborative work or achieved shared goals. Conversely, a lack of competence in IOC can frustrate VT members and threaten VT’s effectiveness (Rosen et al., 2006). Competence refers to having the necessary ability, motivation, skill, and knowledge to guide action in IOC and is related to positive outcomes (Wang & Haggerty, 2011).

This study analyses studies reporting competencies involved in IOC. Research on this topic is fragmented, as most studies focus on specific moderators involved in IOC such as knowledge sharing and trust (Alsharo et al., 2017), communication strategies, trust formation, leadership, technology selection and use, and culture (Cleary et al., 2019; Robey et al., 2000). Few studies cover a broad enough set of IOCCs. Roy (2012) summarized the skills needed for working in virtual environments, that is, skills in relationship building, communication, collaboration, and technology. Despite giving a good overview of skills reported in scholarly journals, the article does not provide competencies needed for IOC that could be used in higher education settings for teaching and evaluating IOCCs. The study conducted by Wang and Haggerty (2011) developed a questionnaire for virtual competence by reviewing the literature. The authors theorized that virtual competence consists of three essential dimensions, namely, virtual self-efficacy, virtual media skill, and virtual social skill. The most recent review on competencies of VT players by Schulze and Krumm (2017) resulted in six competence clusters, that is, media, communication, trust-related, intercultural, self-management, and conflict management. Only one study collected empirical data on the competencies needed for VT members by interviewing them (Hertel et al., 2006). The study resulted in a web-based selection tool for members of VTs, including 39 competencies related to taskwork attributes, teamwork, and tele-cooperation attributes. However, the competencies included in their interview questionnaire were not generated by an objective and systematic review procedure, which is of utmost importance because participants respond to the competencies provided in the questionnaire and are unlikely to add missing competencies due to the individuals’ tendency to avoid dissonance. This might constitute a risk of missing important competencies reported by empirical studies.

Consequently, the authors of this study see a need to conduct a systematic review of empirical studies reporting competencies relevant to IOC. Such a preliminary competence framework would provide guidance on how to facilitate the process of competence development and how to evaluate the acquired competencies. For these purposes, this systematic review pursues the following research questions:

Research Question 1 (RQ1): What competencies do students in higher education need to develop during their studies to achieve effective IOC?

Research Question 2 (RQ2): Which instructional designs are used in higher education to promote the development of IOCCs?

Research Question 3 (RQ3): How can IOCCs and their development be evaluated?

Materials and Methods

Information Sources and Search Strategy

Quality criteria on planning and conducting a systematic review (Sharma et al., 2015) and reporting results (Moher et al., 2015) were followed. To cover all peer-reviewed research and gray literature from different professional backgrounds dealing with education, three databases were included. ERIC covers articles reporting research and education information, including education in management and business and health professions. Web of Science covers education-related articles from social sciences, arts, and humanities. To retrieve studies on IOC in medical professions, health, and life sciences, which are not completely covered by ERIC and Web of Science, PubMed was included to enclose the broad field of education in medical professions, health, and life sciences. The search strategies for all databases were similar, using keywords related to IOCCs, namely, competence, online education or distance education, international educational, and international collaborations (supplemental material, Figure S1). To identify unpublished studies, “snowball” methods like pursuing references of references and electronic citation tracking were applied (Greenhalgh & Peacock, 2005).

Eligibility Criteria and Data Extraction

Eligibility criteria for articles were defined as being written in English or German and reporting original research with a focus on learners in higher education. We excluded studies that did not describe competencies, instructional designs, and/or evaluation methods for IOC. One author conducted the search process and retrieved abstracts and full texts. After duplicates were removed, all abstracts were independently screened by five teams of two blinded researchers each, meaning that the reviewers were unaware of the authors’ names and journals. Data were extracted using a previously developed review guide, including study design, study language, setting, and reported outcomes associated with IOCCs. Results were compared and differences resolved through discussion.

Data extraction of eligible full texts was done by four teams of two independent reviewers each, using a data extraction form developed for the review. Study design, number of participants, described competencies, methods of competency assessment, instructional designs, and study quality were extracted. Discrepancies during any steps of the review process were discussed and resolved within each team. Where ambiguity could not be resolved, a third reviewer assessed the abstract or full text.

Data Analysis

Following an inductive approach, three authors analyzed the competencies emerging from the literature and assigned them to themes, grouping competencies that seemed to be similar, for example, all competencies related to language skills. The principal researcher then analyzed all themes in an iterative process of data reduction. Based on their relationships and connections, the themes were renamed and reorganized into competence domains, resulting in the IOCCs framework. During the analytical process, themes and competence domains were reviewed, discussed, and defined in various meetings with the research team until full agreement was reached on the description of the IOCCs framework.

Quality of the Included Studies

Critical appraisal for study quality and validity in scientific research in education followed the Medical Education Research Study Quality Instrument (MERSQI) (Reed et al., 2007), allowing assessing the methodological rigor of articles and including a comprehensive list of review items. Study design, number of institutions studied, response rate, study type, validity of evaluation instruments, data analysis, and outcomes were evaluated, yielding a maximum score of 18. The articles averaged 8.9 points ranging from 5 to 12.5 (supplemental material, Table S1).

Results

Of the original 516 articles identified, 11 duplicates were removed and the remaining 505 abstracts were screened for eligibility. From this set, 246 studies were excluded because they did not report empirical data relevant for our three key questions on competencies, instructional designs, and evaluation methods used in IOC. Another 107 studies were excluded because they did not focus on learners in higher education, and seven were excluded because they did not include original research. For the remaining studies, 145 full texts were retrieved for further review, of which 22 did not focus on learners in higher education, 80 were not answering our three key questions, and 29 were not retrievable (Figure 1). Four of the included studies were yielded by the database ERIC, one by PubMed, eight by Web of Science, and one article was retrieved by hand search. The included 14 studies were mainly from the United States, China, and European countries (supplemental material, Table S1). Six studies involved language learning, mainly for preservice teachers (n = 4), a term commonly used to refer to student teachers in education and training to become teachers as a profession. Other areas of study included Business School (n = 3), Health Professions, Manufacturing, Geography, and Information Sciences (n = 4). One study did not report the professional field of students.

Figure 1.

Study selection process.

Competencies Involved in Effective IOC

The first research question aimed at reviewing the empirical evidence for IOCCs needed in the increasingly widespread work environment of VTs. Table 1 provides an overview of the competence domains emerging from the literature:

ICT

Intercultural and cultural

Communication and language

Self-management and organization

Collaboration

Domain-specific

Table 1.

Study Results Per Competence Domain: Instructional Design, Assessment/Evaluation Method.

Reference	Instructional design	Assessment/evaluation method
1. Domain: ICT competencies
Mattheos et al. (2001)	Virtual classroom using PBL	Self-assessment at study beginning and observation of IT skills by researchers (self-developed, not validated)
Ertmer et al. (2011)	Collaborative learning using wiki collaboration	Pre- and postassessment with a self-developed and not validated Perceived Comfort Survey
May and Tekkaya (2016)	Online course with a remote lab	Self-developed questionnaire
Sevilla-Pavon and Haba-Osca (2017)	Gamified tele-collaboration	Self-developed questionnaire (pre- and post-test)
Vinagre (2017)	Tele-collaboration	Self-developed questionnaire (post-test)
2. Domain: Intercultural and cultural competencies
Liaw (2006)	Tele-collaboration	Content analysis of e-forum entries based on Byram’s model (Byram, 2008)
Ray et al. (2012)	Collaborative learning	Pre- and postassessment with a self-developed questionnaire containing open-ended content knowledge items
Liaw and Bunn-Le Master (2010)	Tele-collaboration	Content analysis of e-forum entries based on Byram’s model (Byram, 2008)
Ertmer et al. (2011)	Collaborative learning using wiki collaboration	Standardized and validated Miville-Guzman Universality-Diversity Scale (Miville et al., 1999)
Krumm et al. (2013)	No instruction, survey on competencies	Standardized and validated Multicultural Personality Questionnaire (van der Zee and van Oudenhoven, 2000), Standardized and Validated Intercultural Readiness Check (van der Zee and Brinkmann, 2004), Standardised and validated Global Competencies Inventory (Bird et al., 2010a),Standardised and validated Intercultural Effectiveness Scale (Bird et al., 2010b)
Wang et al. (2013)	Collaborative learning using wiki	Self-developed survey
Bartel-Radic et al. (2015)	virtual project	Self-developed survey with 30 items from Bennett’s six DMIS stages (Bennett, 1993).
Bueno-Alastuey and Kleban (2016)	Tele-collaboration	Self-developed questionnaire
Uukkivi (2016)	Not described	Content analysis of e-interviews and reports
Sevilla-Pavon & Haba-Osca (2017)	Gamified tele-collaboration	Self-developed questionnaire (pre- and post-test)
Hertel et al. (2006)	Survey on competencies	Survey with undergraduate students to define competencies, followed-up by interviews with stakeholders to finalize the developed VTCI.
3. Domain: Communication and language competencies
Krumm et al. (2013)	No instruction, survey on competencies	Self-developed questionnaire
Wang et al. (2013)	Collaborative learning using wiki	Self-developed survey
Bueno-Alastuey and Kleban (2016)	Tele-collaboration	Self-developed questionnaire
May and Tekkaya (2016)	Online course, not further explained	Self-developed questionnaire
Sevilla-Pavon and Haba-Osca (2017)	Gamified tele-collaboration	Self-developed questionnaire (pre- and post-test)
Uukkivi (2016)	Not described	Content analysis of e-interviews and reports
4. Domain: Self-management and organizational competencies
Krumm et al. (2013)	No instruction, survey on competencies	No reported questionnaire
Vinagre (2017)	Tele-collaboration	Self-developed questionnaire, observation, and content analysis
Hertel et al. (2006)	Survey on competencies	Survey with undergraduate students to define competencies, followed-up by interviews with stakeholders to finalize the developed VTCI.
May and Tekkaya (2016)	Online course, not further explained	Self-developed questionnaire
5. Domain: Collaboration competencies
Ertmer et al. (2011)	Collaborative learning using wiki collaboration	No questionnaire reported
Krumm et al. (2013)	No instruction, survey on competencies	No questionnaire reported
May and Tekkaya (2016)	Online course, not further explained	No questionnaire reported
Hertel et al. (2006)	Survey on competencies	Survey with undergraduate students to define competencies, followed-up by interviews with stakeholders to finalize the developed VTCI.
6. Domain: Domain-specific competencies
Bueno-Alastuey and Kleban (2016)	Tele-collaboration	Self-developed questionnaire
Ertmer et al. (2011)	Collaborative learning using wiki collaboration	Pre- and postassessment with a self-developed and not validated Perceived Knowledge Survey
Vinagre (2017)	Tele-collaboration	Self-developed questionnaire
Hertel et al. (2006)	Survey on competencies	No questionnaire reported
Ray et al. (2012)	Collaborative learning	Pre- and postassessment with a self-developed questionnaire containing open-ended content knowledge items

ICT = Information and communication technology; DMIS = Developmental Model of Intercultural Sensitivity; PBL = problem-based learning; VTCI = Virtual Team Competency Inventory.

ICT competencies

Five out of 14 studies reported ICT competencies, which represent the first competence domain (Table 1). Ertmer et al. (2011) reported perceived comfort of using technology including Web 2.0 tools (i.e., comfort in completing basic tasks on the computer) for ICT competence. Mattheos et al. (2001) asked students how novice or expert they were in handling computer technology. The authors observed a mismatch between this initial self-assessment and the actual availability of the necessary skills, and recommended a workshop in technology and assessment of ICT competency prior to IOC. Some authors defined ICT competence as using internet technology for communication with international colleagues (May & Tekkaya, 2016) and others as using technology and developing digital literacy through Google+ and Google-based tools (Drive, Docs, Forms) (Sevilla-Pavon & Haba-Osca, 2017). Vinagre (2017) defined ICT competence as familiarity with asynchronous and synchronous ICT tools and knowledge about using web pages and choosing the most adequate ICT tools. All of these studies mentioned individual components of ICT literacy as defined by Sommerville et al. (2008), but none referred to ICT literacy as an outcome.

Intercultural and cultural competencies

The second domain that emerged from the literature comprises intercultural and cultural competencies (supplemental material, Table S2), as reported by 11 studies. Byram’s model of intercultural communication is widely used in foreign language teaching (Liaw, 2006; Liaw & Bunn-Le Master, 2010; Wang et al., 2013). Liaw (2006) identified knowledge about one’s own culture and others’ culture for intercultural communication, interest in knowing other people’s way of life, and introducing one’s own culture to others as the most prevalent competencies. The ability to change perspective was the least prevalent competence (Liaw, 2006). Liaw and Bunn-Le Master (2010) noted the competencies of knowing other people’s way of life, introducing one’s own culture to others, and knowledge about one’s own culture and others’ culture for intercultural communication during tele-collaboration. The study by Krumm et al. (2013) provided a set of 38 intercultural competencies, which also covered other competence domains as defined here, like communication and language competencies. Therefore, only the competencies we considered as specific to the domain of intercultural competencies are covered in this domain.

Communication and language competencies

The third domain emerging from the literature represents communication and language competencies, as reported by six studies (Bueno-Alastuey & Kleban, 2016; Krumm et al., 2013; May & Tekkaya, 2016; Sevilla-Pavon & Haba-Osca, 2017; Uukkivi, 2016; Wang et al., 2013). Characteristics such as openness and perspective-taking (Krumm et al., 2013), honesty, reliability, helpfulness, friendliness, flexibility, neutrality, constructiveness, activeness, positivity, patience, concreteness, objectivity, lack of prejudice, and, at the same time, carefulness and thoughtfulness were found to be supportive in communication (Uukkivi, 2016). Sevilla-Pavon and Haba-Osca (2017) described linguistic competence, attitudes, and views on language learning as communication and language competencies. In contrast, Bueno-Alastuey and Kleban (2016) included listening and speaking skills in a foreign language, pronunciation, intonation, oral fluency, and vocabulary improvement in colloquial language as well as self-confidence in speaking. Wang et al. (2013) reported improved written foreign language skills, such as using complex words and more complicated sentence structure, increased vocabulary, and the shift from an amateur language learner toward more native-like language speaker. May and Tekkaya (2016) defined perceived level or proficiency in presenting results to expert and novice audiences in oral and written form as communication and language competencies.

Self-management and organizational competencies

The fourth domain emerging from the literature review describes self-management and organizational competencies (supplemental material, Table S3), which were reported by four studies (Hertel et al., 2006; Krumm et al., 2013; May & Tekkaya, 2016; Vinagre, 2017). Besides general organizational competencies, such as planning tasks thoroughly in advance and meeting deadlines, some competencies in this domain focus on dealing with difficult situations: reacting calmly and positively to frustrating or difficult situations, coping with stress and ambiguity, dealing with problems that have no clear solution, improvisation and creative problem solving, seeing new and unfamiliar situations as an opportunity to learn, starting energetically after setbacks or personal crises, and not making premature decisions (Vinagre, 2017). Independence is seen as particularly important because supervisory control and social control by other team members are reduced; similarly, persistence is of utmost importance because online collaboration is characterized by frequent interruptions due to other commitments (Hertel et al., 2006).

Collaboration competencies

The fifth domain emerging from previous research comprises collaboration competencies reported by four studies (Ertmer et al., 2011; Hertel et al., 2006; Krumm et al., 2013; May & Tekkaya, 2016) (supplemental material, Table S4). We assigned engaging in international collaborations, a strong sense of responsibility toward the team, treating different opinions equally, and being perceptive to hints and insinuations to this domain. Hertel et al. (2006) and Krumm et al. (2013) also categorized other, already mentioned domains as part of collaboration competencies, including intercultural, communication, and self-management competencies. These competencies were, therefore, not counted as collaboration competencies, but added to the respective domains as defined above. Interpersonal trust was mentioned by Hertel et al. (2006) as the most frequently mentioned precondition for successful online collaboration.

Domain-specific competencies

The final domain emerging from the analyzed studies represents domain-specific competencies, as reported by five studies (Bueno-Alastuey & Kleban, 2016; Ertmer et al., 2011; Hertel et al., 2006; Ray et al., 2012; Vinagre, 2017). These include job expertise and technical knowledge as mandatory for collaboration to achieve shared goals and substantive knowledge in form of domain-specific competencies to interact in a globally interdependent world. Domain-specific competencies vary widely according to the specific profession or learning domain and are therefore not discussed further here.

Instructional Designs Used to Promote IOCCs

Results for the second research question, focusing on instructional designs used to promote IOCCs, showed a variety of designs for supporting one or more domains of IOCCs (Table 1). Five studies did not provide a clear description of the instructional design and learning tasks (Hertel et al., 2005; Krumm et al., 2013; May & Tekkaya, 2016; Ray et al., 2012; Uukkivi, 2016). Following Gustafson and Branch (2002), instructional design is “. . . a system of procedures for developing education and training programs in a consistent and reliable fashion” (p. 17). Instructional designs described in the included studies were tele-collaboration with and without a shared common task artifact, problem-based learning (PBL), and collaborative learning using wikis and online projects.

Tele-collaboration

Tele-collaboration was used in five out of 14 studies focusing on English as Foreign Language (EFL) and Business English, as shown in Table 1. Vinagre (2017) defined tele-collaboration as “. . .learner centred activity that entails engaging students in virtual collaboration with partners in different locations to achieve certain learning or training goals” (p. 35). A collaborative task should therefore share a common task artifact because discussions do not become interactive and collaborative simply because people are put in touch via ICT (Pawan et al., 2003).

Tele-collaboration with a shared common task artifact was implemented in three studies. Reviewing articles, critically analyzing examples from online exchanges, and collaboratively designing guidelines and assessment tools to organize a tele-collaborative project were included in the design developed by Vinagre (2017). Similar tasks described by Bueno-Alastuey and Kleban (2016) were discussions and reflections either on a video or on topics such as different educational systems. Sevilla-Pavon and Haba-Osca (2017) applied a tele-collaboration approach enriched with gamification, where students achieved points for tasks and used avatars to express themselves. Two studies reported tele-collaboration without a shared common task artifact. In these, tasks were joint readings, responding and reflecting in online forums (Liaw & Bunn-Le Master, 2010), and reading articles followed by online discussions about cultural differences (Liaw, 2006).

PBL, collaborative learning using wikis, and online projects

Mattheos et al. (2001) implemented PBL in a virtual classroom based on clinical cases in periodontology. Online synchronous meetings conducted early in the course resulted in a higher level of activity and interaction compared with groups meeting online at a later stage. A timetable of online meetings is necessary to ensure student participation, and the authors emphasized the importance of self-motivation for IOC. Two studies described collaborative learning using a wiki (Ertmer et al., 2011; Wang et al., 2013). Bartel-Radic et al. (2015) followed a project-based approach, which involves virtual multicultural teams developing a project, collaboratively writing a project report, and reflecting on their collaborative experience.

Evaluation Methods for IOCCs

All assessment and evaluation methods are given in Table 1. For the purposes of this article, assessment refers to the process of making observations about the learner’s proficiency and comparing these to a standard, and evaluation refers to the process of making a judgment that gives meaning to observations about the learner’s proficiency, usually by comparing it to expectations (Pangaro & Ten Cate, 2013).

Evaluating ICT competencies

Five studies evaluating ICT competencies used self-developed questionnaires (Table 1). Students rated their competence prior to and after the course (self-developed non-standardized) in two studies (Ertmer et al., 2011; Mattheos et al., 2001). Mattheos et al. (2001) evaluated self-perceived competence using the computer, Ertmer et al. (2011) evaluated ICT competencies by means of a pre- and postsurvey on perceived comfort and perceived knowledge for using technology tools.. May and Tekkaya (2016) used a questionnaire based on proficiency in technical experimentation, whereas Sevilla-Pavon and Haba-Osca (2017) evaluated the use of technology for language learning. The study by Vinagre (2017) assessed self-perceived satisfaction with a 13-item questionnaire, three of which were related to ICT competencies: “the learner is familiar with asynchronous and synchronous ICT tools, knows how to use web pages in order to find partners in other countries, knows how to choose the most adequate ICT tools” (p. 44).

Evaluating intercultural and cultural competencies

For evaluating intercultural and cultural competencies, five studies applied content analysis to online forums, interviews, and reports (Table 1). Three studies (Liaw, 2006; Liaw & Bunn-Le Master, 2010; Wang et al., 2013) applied Byram’s (1997, 2000) model for content analysis, Uukkivi (2016) did not report using a framework, and Vinagre (2017) used a modified coding scheme for individual and social accountability, cognitive and organizational behaviors. Four studies used standardized questionnaires: the Miville-Guzman Universality Diversity Scale in its short form (M-GUDS-s) (Ertmer et al., 2011), a questionnaire based on the Developmental Model of Intercultural Sensitivity (Bartel-Radic et al., 2015), and the Virtual Team Competency Inventory developed by Hertel et al. (2006). Krumm et al. (2013) used a questionnaire based on the Multicultural Personality Questionnaire (van der Zee & van Oudenhoven, 2000), the Intercultural Readiness Check (van der Zee & Brinkmann, 2004), the Global Competencies Inventory (Bird et al., 2010b), and the Intercultural Effectiveness Scale (Bird et al., 2010a). Five studies used self-developed and unvalidated questionnaires (Bueno-Alastuey & Kleban, 2016; Ray et al., 2012; Sevilla-Pavon & Haba-Osca, 2017; Wang et al., 2013). Ertmer et al. (2011) used two self-developed questionnaires in addition to the M-GUDS-s, namely, the self-developed Perceived Comfort Survey and the Perceived Knowledge Survey.

Other evaluation methods

As shown in Table 1, to evaluate communication and language competencies, Bartel-Radic et al. (2015) asked learners to self-assess team processes, communication modes, outcomes, and learning, but did not provide an in-depth description of the questionnaire.

These results indicate both the rare usage of standardized and validated questionnaires for the six competence domains and the need to develop validated questionnaires. Overall, the reporting on the questionnaires used was poor. Only for intercultural competence and for online collaboration competencies were validated questionnaires used. For the other domains, self-developed questionnaires were used, which were neither provided nor described in detail. Studies such as Mattheos et al. (2001) and Sevilla-Pavon and Haba-Osca (2017) relied heavily on self-perceived results of collaboration.

Discussion

This study systematically reviewed empirical studies on IOCCs (RQ1), the instructional designs used to teach IOCCs (RQ2), and the methods for evaluating the competencies (RQ3), eliminating 97% of the originally retrieved articles (Figure 1). The results of the 14 included studies are difficult to compare due to the heterogeneous reporting of competencies, instructional designs, and evaluation methods involved in IOC.

Regarding RQ1, which IOCCs are needed for effective IOC, we conclude: IOCCs express the ability to work effectively and respectfully with diverse teams, the willingness to work toward a common goal, shared responsibility for collaborative works, and the value of individual contributions made by each team member. IOCCs include six competence domains: ICT, intercultural and cultural, communication and language, self-management and organization, collaboration, and domain-specific competencies. Interestingly, most included studies focused on specific, individual domains of IOCCs, such as intercultural competencies or language competencies, and none of the studies covered all six IOCCs domains as identified here. According to Pawlowski and Holtkamp (2012), a range of competencies rather than a single competence domain are needed to solve a problem or task in IOC. Krumm et al. (2013) argued that future research should link specific professional tasks to specific competencies required of VTs. This leads to the first conclusion: Considering single domains of IOCCs in isolation cannot cover the full set of required skills for the growing trend for IOC because virtuality and intercultural settings raise challenges that may not affect traditional teams in the same way. ICT competencies, despite being a prerequisite for IOC, were rarely described in the included studies. VTs seem to require different skills compared with the daily-life usage of ICT and therefore need to be promoted specifically. As Kirschner and van Merriënboer (2013) reported, students might be seen as “digital natives” while having skills that are limited to basic office skills, e-mailing, text messaging, social media, and internet use. This conclusion is supported by Mattheos et al. (2001), who showed that learners overestimate their skills and knowledge in ICT, maybe despite or even because they are using social media and ICT in their daily life. Future research should consider all six competence domains instead of focusing on single domains. There is a need to evaluate the proposed set of IOCCs to finally develop a competence framework needed for effective virtual teamwork.

The studies relevant to RQ2 on instructional designs used mainly task-centered approaches combined with collaborative learning, such as tele-collaboration, PBL, collaborative learning using wikis, or online projects. The instructional designs were not described in detail, and they were not evaluated for the effectiveness of the acquisition of specific competencies. The main challenge is to design collaborative learning to facilitate all domains of IOCCs and to promote collaboration itself in an online environment. The structure of learning tasks is important for enabling collaboration, but how the collaboration was guided or promoted was not reported. This would be important, however, because free and unguided collaboration does not necessarily lead to the desired learning outcomes, and structuring interactions of students in well-defined scripts has been proved to support learning (Dillenbourg, 2002). Based on the results, there is a clear need for research on instructional designs and, for example, collaborative scripts specifically designed for IOC. As collaboration requires interpersonal communication and coordination of information, more cognitive effort is needed for collaboration than for individual learning, that is, so-called “transaction costs.” Whether collaboration is more effective than individual learning on domain-specific real-life tasks depends on whether the distribution advantage is large enough to compensate for these extraneous transaction costs. Future studies should consider this aspect when developing and evaluating instructional designs and the real-life tasks intended to promote IOCCs.

Validated questionnaires for answering RQ3, how IOCCs and their development can be evaluated, are available for intercultural competence and also for virtual teamwork. For the four other competency domains, validated questionnaires are lacking, and the self-developed questionnaires were rarely described in sufficient detail. Questionnaires relying on self-assessed or self-perceived competencies involve the problem of over- or underestimating one’s own competencies, and authors are well aware of this problem (Bartel-Radic et al., 2015; Mattheos et al., 2001; Wang et al., 2013). To date, objective evaluation methods are absent, which might explain why authors use learners’ self-assessment questionnaires. Future research should develop and evaluate questionnaires and assessment instruments for IOCCs going beyond self-assessment to allow for triangulation of competence acquisition.

Our study has several limitations. First, reported results were heterogeneous and synthesis is possible only to a limited degree. Future research in the form of a Delphi consensus study should evaluate the reviewed competencies and propose a competence framework on IOCCs. Based on this framework, researchers, instructional designers, and teachers can build upon and extend the studies to date by conducting more collaborative research while experimenting with teaching and evaluation methods. Documentation of the findings on IOCCs, instructional designs, and evaluation methods should prove informative and valuable for researchers and teachers alike, which was found to be a limitation in the existing studies. Second, we limited studies to English and German language and therefore might have missed relevant findings of studies in other languages.

To conclude, this study provided an extensive overview of IOCCs reported in the literature, instructional designs, and the evaluation methods used as a solid foundation for future research on IOCCs for virtual teamwork. As the pandemic affected international higher education substantially with a decrease in international student mobility due to travel restrictions, the digitalization of international education offers important opportunities for intercultural exchange and a learning field for IOCCs. Our study contributed by defining IOCCs for measuring the effectiveness of the digitalization of international education and for promoting the development of this field. There is a clear need for continued debate and dialogue to validate the proposed set of competencies as well as a need for further research to identify effective strategies for teaching and evaluating IOCCs in digital international education.

Supplemental Material

sj-pdf-1-jsi-10.1177_10283153211016272 – Supplemental material for International Online Collaboration Competencies in Higher Education Students: A Systematic Review

Supplemental material, sj-pdf-1-jsi-10.1177_10283153211016272 for International Online Collaboration Competencies in Higher Education Students: A Systematic Review by Alexandra Kolm, Jascha de Nooijer, Koen Vanherle, Andrea Werkman, Daniela Wewerka-Kreimel, Shelly Rachman-Elbaum and Jeroen J. G. van Merrïnboer in Journal of Studies in International Education

Footnotes

Acknowledgements

We thank Anique Atherley for proofreading the manuscript and the members of the Globalization Research Group of the School of Health Professions Education, Maastricht University, for their valuable comments on an earlier version of this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Alexandra Kolm

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Alexandra Kolm is a lecturer, researcher, and international coordinator at the Department of Health Sciences at the St. Pölten University of Applied Sciences, Austria, since 2012, with a special focus on international online collaboration and international projects.

Jascha de Nooijer is an associate professor and director of education for Health at Maastricht University, School of Health Professions Education, Maastricht, The Netherlands. She is responsible for two bachelor’s programs and nine master’s programs at Maastricht University.

Koen Vanherle is a lecturer, researcher, and international coordinator at Artesis Plantijn University College Antwerp, Belgium, since 2003, with a special focus on an MOOC for dietetic students and international projects.

Andrea Werkman is senior university lecturer at the School of Health Care Studies at the Hanze University of Applied Sciences, Groningen, The Netherlands, with a focus on (digital) education in an international and/or interprofessional context.

Daniela Wewerka-Kreimel is lecturer and deputy academic director at the Department of Health Sciences, St. Pölten University of Applied Sciences GmbH, Austria. Since 2014, she is an expert of the Austrian Federal Ministry of Health for education and training in dietetics.

Shelly Rachman-Elbaum is lecturer, researcher, and practitioner in the field of clinical nutrition and dietetics focusing on online nutrition education, nutritional assessment, nutrigenomics, and nutrition care process at the Hebrew University of Jerusalem and the University of Applied Sciences in Neubrandenburg, Germany.

Jeroen J. G. van Merriënboer is holding a chair in Learning and Instruction at Maastricht University, School of Health Professions Education, Maastricht, The Netherlands. He is a research program director of the Graduate School of Health Professions Education at Maastricht University.

References

Alsharo

Gregg

Ramirez

(2017). Virtual team effectiveness: The role of knowledge sharing and trust. Information and Management, 54(4), 479–490. https://doi-org.ezproxy.ub.unimaas.nl/10.1016/j.im.2016.10.005

Arkoudis

Watty

Baik

Borland

Chang

Lang

Pearce

(2013). Finding common ground: Enhancing interaction between domestic and international students in higher education. Teaching in Higher Education, 18(3), 222–235. https://doi.org/10.1080/13562517.2012.719156

Bartel-Radic

Moos

J. C.

Long

S. K.

(2015). Cross-cultural management learning through innovative pedagogy: An exploratory study of globally distributed student teams. Decision Sciences Journal of Innovative Education, 13(4), 539–562.

Bennett

M. J.

(1993). Towards ethnorelativism: A developmental model of intercultural sensitivity. In Paige

R. M.

(Ed.), Education for the intercultural experience (pp. 21–71). Intercultural Press.

Bersin

McDowell

Rahnema

Van Durme

(2017). The organization of the future: Arriving now—2017 Global human capital trends. https://www2.deloitte.com/us/en/insights/focus/human-capital-trends/2017/organization-of-the-future.html

Bird

Mendenhall

Stevens

M. J.

Oddou

(2010a). Defining the content domain of intercultural competence for global leaders. Journal of Managerial Psychology, 25(8), 810–828. https://doi.org/10.1108/02683941011089107

Bird

Mendenhall

Stevens

M. J.

Oddou

(2010b). Intercultural effectiveness scale technical report. https://www.kozaigroup.com/intercultural-effectiveness-scale-ies/

Bueno-Alastuey

M. C.

Kleban

(2016). Matching linguistic and pedagogical objectives in a telecollaboration project: A case study. Computer Assisted Language Learning, 29(1), 148–166. https://doi.org/10.1080/09588221.2014.904360

Byram

(1997). Teaching and assessing intercultural communicative competence. Multilingual Matters.

10.

Byram

(2000). Assessing intercultural competence in language teaching. Sprogforum, 18(6), 8–13.

11.

Byram

(2008). From foreign language education to education for intercultural citizenship: Essays and reflections (Vol. 17). Multilingual Matters.

12.

Cleary

Slattery

D. M.

Flammia

Minacori

(2019). Developing strategies for success in a cross-disciplinary global virtual team project: Collaboration among student writers and translators. Journal of Technical Writing and Communication, 49(3), 309–337. https://doi.org/10.1177/0047281618775908

13.

Deardorff

D. K.

(2006). Identification and assessment of intercultural competence as a student outcome of internationalization. Journal of Studies in International Education, 10(3), 241–266. https://doi.org/10.1177/1028315306287002

14.

Deardorff

D. K.

Arasaratnam-Smith

L. A.

(2017). Introduction. In Deardorff

D. K.

Arasaratnam-Smith

L. A.

(Eds.), Intercultural competence in higher education: International approaches, assessment and application (1st ed., pp. 27–29). Routledge.

15.

Dillenbourg

(2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In Kirschner

P. A.

(Ed.), Three worlds of CSCL—Can we support CSCL? (pp. 61–91). Open Universiteit Nederland.

16.

Dolmans

D. H.

De Grave

Wolfhagen

I. H.

van der Vleuten

C. P.

(2005). Problem-based learning: Future challenges for educational practice and research. Medical Education, 39(7), 732–741. https://doi.org/10.1111/j.1365-2929.2005.02205.x

17.

Ertmer

P. A.

Newby

T. J.

J. H.

Liu

Tomory

Lee

Y. M.

. . . Sendurur

(2011). Facilitating students’ global perspectives: Collaborating with international partners using web 2.0 technologies. Internet and Higher Education, 14(4), 251–261. https://doi.org/10.1016/j.iheduc.2011.05.005

18.

Ford

R. C.

Piccolo

R. F.

Ford

L. R.

(2017). Strategies for building effective virtual teams: Trust is key. Business Horizons, 60(1), 25–34. https://doi.org/10.1016/j.bushor.2016.08.009

19.

Geisinger

K. F.

(2016). 21st Century skills: What are they and how do we assess them. Applied Measurement in Education, 29(4), 245–249. https://doi-org.ezproxy.ub.unimaas.nl/10.1080/08957347.2016.1209207

20.

Gilson

L. L.

Maynard

M. T.

Young

N. C. J.

Vartiainen

Hakonen

(2015). Virtual teams research: 10 Years, 10 themes, and 10 opportunities. Journal of Management, 41(5), 1313–1337. https://doi.org/10.1177/0149206314559946

21.

Greenhalgh

Peacock

(2005). Effectiveness and efficiency of search methods in systematic reviews of complex evidence: Audit of primary sources. British Medical Journal, 331(7524), 1064–1065. https://doi.org/10.1136/bmj.38636.593461.68

22.

Gustafson

K. L.

Branch

R. M.

(2002). What is instructional design. In Reiser

R. A.

Dempsey

J. V.

(Eds.), Trends and issues in instructional design and technology (pp. 16–25). Prentice Hall.

23.

Hertel

Geister

Konradt

(2005). Managing virtual teams: A review of current empirical research. Human Resource Management Review, 15(1), 69–95. https://doi.org/10.1016/j.hrmr.2005.01.002

24.

Hertel

Konradt

Voss

(2006). Competencies for virtual teamwork: Development and validation of a web-based selection tool for members of distributed teams. European Journal of Work and Organizational Psychology, 15(4), 477–504. https://doi.org/10.1080/13594320600908187

25.

Kirschner

P. A.

van Merriënboer

J. J. G.

(2013). Do learners really know best? Urban legends in education. Educational Psychologist, 48(3), 169–183. https://doi.org/10.1080/00461520.2013.804395

26.

Krumm

Terwiel

Hertel

(2013). Challenges in norm formation and adherence: The knowledge, skills, and ability requirements of virtual and traditional cross-cultural teams. Journal of Personnel Psychology, 12(1), 33–44. https://doi.org/10.1027/1866-5888/a000077

27.

Liaw

M.-L.

(2006). E-learning and the development of intercultural competence. Language Learning & Technology, 10(3), 49–64. http://dx.doi.org/10125/44074

28.

Liaw

M.-L.

Bunn-Le Master

(2010). Understanding telecollaboration through an analysis of intercultural discourse. Computer Assisted Language Learning, 23(1), 21–40. https://doi.org/10.1080/09588220903467301

29.

Marinoni

van’t Land

Jensen

(2020). The impact of COVID-19 on higher education around the world—IAU Global Survey report. https://www.iau-aiu.net/IMG/pdf/iau_covid19_and_he_survey_report_final_may_2020.pdf

30.

Mattheos

Nattestad

Schittek

Attstrom

(2001). A virtual classroom for undergraduate periodontology: A pilot study. European Journal of Dental Education, 5(4), 139–147. https://doi.org/10.1034/j.1600-0579.2001.50401.x

31.

May

Tekkaya

A. E.

(2016). Transnational connected learning and experimentation using live online classes and remote labs for preparing international engineering students for an international working world. International Journal of Engineering Pedagogy, 6(1), 18–28. https://doi.org/10.3991/ijep.v6i1.5287

32.

Mendenhall

M. E.

Stevens

M. J.

Bird

Oddou

G. R.

(2011). Intercultural Effectiveness Scale technical report. https://www.academia.edu/20623353/Intercultural_Effectiveness_Scale_Technical_Report

33.

Mihhalova

(2006). From ordinary to virtual teams: A model for measuring the virtuality of a teamwork. Frontiers of E-Business Research. http://iceb.johogo.com/proceedings/2006/defevent/papers/cr1079.pdf

34.

Miville

M. L.

Gelso

C. J.

Pannu

Liu

Touradji

Holloway

Fuertes

(1999). Appreciating similarities and valuing differences: The Miville-Guzman Universality-Diversity Scale. Journal of Counseling Psychology, 46(3), 291–307. https://doi.org/10.1037/0022-0167.46.3.291

35.

Moher

Shamseer

Clarke

Ghersi

Liberati

Petticrew

. . . Stewart

L. A.

(2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4(1), 1–9. https://doi.org/10.1186/2046-4053-4-1

36.

Olaisen

Revang

(2017). Working smarter and greener: Collaborative knowledge sharing in virtual global project teams. International Journal of Information Management, 37(1), 1441–1448. https://doi.org/10.1016/j.ijinfomgt.2016.10.002

37.

Pangaro

Ten Cate

(2013). Frameworks for learner assessment in medicine: AMEE guide no. 78. Medical Teacher, 35(6), e1197–e1210. https://doi.org/10.3109/0142159X.2013.788789

38.

Pawan

Paulus

T. M.

Yalcin

Chang

C. F.

(2003). Online learning: Patterns of engagement and interaction among in-service teachers. Language Learning & Technology, 7(3), 119–140.

39.

Pawlowski

J. M.

Holtkamp

(2012). Towards an internationalization of the information systems curriculum. In Mattfeld

D. C.

Robra-Bissantz

(Eds.), Proceedings of multi conference business information systems 2012 (Vol. 1, pp. 437–449). GITO mbH Verlag.

40.

Ray

W. C.

Muñiz-Solari

Klein

Solem

(2012). Effective online practices for international learning collaborations. Review of International Geographical Education Online, 2(1), 25–44.

41.

Reed

D. A.

Cook

D. A.

Beckman

T. J.

Levine

R. B.

Kern

D. E.

Wrigh

S. M.

(2007). Association between funding and quality of published medical education research. Journal of the American Medical Association, 298(9), 1002–1009.

42.

Robey

Khoo

H. M.

Powers

(2000). Situated learning in cross-functional virtual teams. Technical Communication: Journal of the Society for Technical Communication, 47(1), 51–66. https://doi.org/DOI:10.1109/47.826416

43.

Rosen

Furst

Blackburn

(2006). Training for virtual teams: An investigation of current practices and future needs. Human Resource Management, 45(2), 229–247.

44.

Roy

S. R.

(2012). Virtual collaboration: The skills needed to collaborate in a virtual environment. Journal of Internet Social Networking & Virtual Communities, 2012, Article 629512. https://doi.org/10.5171/2012.629512

45.

Saavedra

(2020). Educational challenges and opportunities of the coronavirus (COVID-19) pandemic. World Bank. https://blogs.worldbank.org/education/educational-challenges-and-opportunities-covid-19-pandemic

46.

Schulze

Krumm

(2017). The virtual team player: A review and initial model of knowledge, skills, abilities, and other characteristics for virtual collaboration. Organizational Psychology Review, 7(1), 66–95. https://doi.org/10.1177/2041386616675522

47.

Senyshyn

R. M.

(2019). A first-year seminar course that supports the transition of international students to higher education and fosters the development of intercultural communication competence. Journal of Intercultural Communication Research, 48(2), 150–170. https://doi.org/10.1080/17475759.2019.1575892

48.

Sevilla-Pavon

Haba-Osca

(2017). Learning from real life and not books: A gamified approach to business English task design in transatlantic telecollaboration. Iberica, 33, 235–260.

49.

Sharma

Gordon

Dharamsi

Gibbs

(2015). Systematic reviews in medical education: A practical approach—AMEE guide 94. Medical Teacher, 37(2), 108–124. https://doi.org/10.3109/0142159x.2014.970996

50.

Sommerville

M. M.

Smith

Macklin

A. S.

(2008). The ETS iSkills (TM) assessment: A digital age tool. The Electronic Library, 26(2), 158–171. https://doi.org/10.1108/02640470810864064

51.

Stiedl

Straub

(2020). COVID-19 as an accelerator of digital collaboration. https://people-organisation.pwc.at/en/2020/04/covid-19-as-an-accelerator-of-digital-collaboration/

52.

Uukkivi

(2016). Personal factors supporting intercultural communication in e-learning of information sciences. Library Review, 65(1–2), 20–32. https://doi.org/10.1108/lr-01-2015-0002

53.

van der Zee

K. I.

Brinkmann

. (2004). Construct validity evidence for the Intercultural Readiness Check against the Multicultural Personality Questionnaire. International Journal of Selection and Assessment, 12, 285–290. https://doi.org/10.1111/j.0965-075X.2004.283_1.x

54.

van der Zee

K. I.

van Oudenhoven

J. P

. (2000). The Multicultural Personality Questionnaire: A multidimensional instrument of multicultural effectiveness. European Journal of Personality, 14(4), 291–309. https://doi.org/10.1002/1099-0984(200007/08)14:4<291::AID-PER377>3.0.CO;2-6

55.

Vinagre

(2017). Developing teachers’ telecollaborative competences in online experiential learning. System, 64, 34–45. https://doi.org/10.1016/j.system.2016.12.002

56.

Wang

J. H.

Zou

Wang

D. S.

Xing

M. J.

(2013). Students’ perception of a wiki platform and the impact of wiki engagement on intercultural communication. System, 41(2), 245–256. https://doi.org/10.1016/j.system.2013.04.004

57.

Wang

Haggerty

(2011). Individual virtual competence and its influence on work outcomes. Journal of Management Information Systems, 27(4), 299–333. https://doi.org/10.2753/MIS0742-1222270410

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.25 MB