Experiences of a project-based blended learning approach in a community of inquiry from information and communication technology engineering students at Lapland university of applied sciences in Finland

Abstract

The increase in remote work in the Information and Communication Technology (ICT) industry has introduced the need for research into team collaboration platforms and education using a project-based blended learning approach. This study investigates the perceptions of ICT engineering students (N = 56) at Lapland University of Applied Sciences in Finland regarding the social, cognitive, and teaching presences in a blended setting using the Community of Inquiry (CoI) framework. The CoI questionnaire data were analyzed using the Rasch Rating Scale Model, and team collaboration and development platform logs were used to explore platform usage. The results suggest that the students’ overall perception of blended learning was highly positive. Questions associated with discussing through the online medium were relatively harder for the students to agree with, although the online medium was perceived as excellent for social interaction by the students. The results may help institutions develop an understanding of the factors that may improve the quality of blended learning experiences in project-based digital ecosystems.

Keywords

Project-based learning blended learning engineering education community of inquiry

Introduction

The content and implementation method of the curriculum in engineering education are largely guided by industry and business needs. In Information and Communication Technology (ICT), the project approach is prevalent, and in multinational companies, project teams work in a decentralized manner using remote and teamwork tools. It could be argued that the industry’s community practices, methods, and collaboration platforms should also be applied to teaching and learning. In addition to technical skills, social and interpersonal competencies are required to communicate, collaborate, and establish social connections with other individuals and teams (Fitsilis et al., 2018). Collaboration and communication can be seen as key skills for engineers (Kamp, 2016), and teamwork skills are required by the industry (Chowdhury and Murzi, 2020; Hogarth, 2010: 3). These skills can be developed through project-based learning (PjBL) methods in real-life projects (Mativo et al., 2017). In addition, agile strategies facilitate project management and are useful in improving students' collaboration (Noguera et al., 2018; Sakulviriyakitkul et al., 2020).

The Community of Inquiry (CoI) framework introduced by Garrison et al. (1999) in higher education is based on the idea that education is both a collaborative and an individual learning experience. The framework provides a perspective for understanding blended learning experiences, including e-learning experiences in campus-based learning (Garrison, 2016: 103). The CoI constitutes a framework for the creation of a collaborative learning community with three core elements: the teaching, cognitive, and social presences. The overlap of these three elements provides a structure for understanding the dynamics of e-learning experiences (Garrison et al., 2010). Students’ e-learning perceptions concerning three core elements of the CoI framework are widely measured by a CoI survey developed and validated by Arbaugh et al. (2008). CoI research is typically used as a means of researching, evaluating, comparing, and improving learning environments (Stenbom, 2018).

This article is part of broader design-based research (DBR) conducted at Lapland University of Applied Sciences (Lapland UAS) that aims to fit current project-based and face-to-face (FTF) ICT engineering education within the blended learning environment in the context of the methods and tools used by the industry. Students’ blended learning experiences are evaluated through the CoI framework, and the main research questions for this study are as follows:

RQ1

How do students experience blended learning in terms of the teaching, social,and cognitive presences?

RQ2

How do students use the team collaboration and developer platforms in a blended learning context?

Related Literature

Blended learning integrates FTF and online learning activities (Graham, 2006: 4–6), and its purpose is to engage with innovative ways of presenting ideas and to expand thinking and discussion across time and space (Vaughan et al., 2013: 9). Students should be involved in critical discourse and reflection, which can be achieved by creating research communities where students are responsible for constructing meaning and reinforcing their understanding through active participation in the research process (Vaughan, 2010). According to Vaughan et al. (2013: 4), successful blended learning is dependent upon the creation of a collaborative CoI and an understanding of the principles of teaching presence. Learning is situated within purposeful inquiry, where students collaboratively assume a shared responsibility and control to design, facilitate, and direct inquiry. CoI starts from the idea that education is both a collaborative and an individual constructivist learning experience (Vaughan et al., 2013: 10), and its foundation is the interaction needed in the higher educational experiences to foster understanding (Garrison and Vaughan, 2008: 13–30). According to Vaughan et al. (2013: 19–44), the CoI framework contains three elements: the teaching, social and cognitive presences, as shown in Figure 1.

Figure 1.

The CoI by Garrison (2016: 25). Image source: https://www.thecommunityofinquiry.org/framework (licensed under CC BY-SA 4.0).

Garrison et al. (1999) define the Social Presence (SP) as the ability of participants in a CoI to project their characteristics onto the community and to present themselves to others as real people. The SP is implemented in three categories: open communication, group cohesion, and affective/personal expression. Garrison and Vaughan (2008: 11–30) explain that the SP assumes that students have the freedom to express themselves freely in a safe environment and can develop their commitment and belonging to the community. The Cognitive Presence (CP) is the most basic factor in the success of higher education, which, according to Garrison et al. (1999), refers to the extent to which participants in a particular CoI can construct meaning through sustained communication. The CP moves from defining the problem to exploring and integrating solutions, as well as testing the validity of the learning outcome by certain categories, such as triggering event exploration and resolving the problem through collaboration. Lastly, Anderson et al. (2001) define the Teaching Presence (TP) as “the design, facilitation, and direction of cognitive and social processes to realize personally meaningful and educationally worthwhile learning outcomes.” The TP maintains the activities in the community by categories such as design and organization, facilitation of discourse, and direct instructions. The key overlaps among the three presences include setting the climate, supporting discourse, and monitoring and regulating learning. Engagement of the overlaps must facilitate both FTF and online learning (Vaughan et al., 2013: 47).

Arbaugh et al., (2008) developed and validated a CoI framework-based survey tool that comprises 34 question items in total, reflecting the educational experiences of the CP, SP, and TP. The questions are classified into subcategories under each presence, and each item is evaluated using a Likert scale. As an ordinal scale, the mean or standard deviation are not meaningful to calculate (Jamieson, 2004) because it cannot be assumed that the distance between each level of the rating scale, e.g., from “agree” to “strongly agree” and from “agree” to “neutral,” is equal. (Cohen et al., 2007: 605). Abbitt and Boone, (2021) recommend applying Rasch measurement techniques to CoI data, as they both provide unique insights from the survey data, including person measures and item difficulty visualizations, and they address the properties of the CoI data, such as the ordinal and non-linear nature of the rating scale and items being of different difficulties. The Rasch Rating Scale Model (RSM; Andrich, 1978) is a measurement model suitable for rating scale data.

PjBL is an innovative approach based on the idea of learning by doing (Dewey, 1997; Krajcik and Blumenfeld, 2006: 317), where subjects are learned through authentic, engaging, and complex real-world problems (Almulla, 2020; MacLeod and van der Veen, 2020). PjBL has been applied to engineering education mostly in single courses (Chen et al., 2021), e.g., in Intana, (2020) and Sanchez-Romero et al., (2019), but also in interdisciplinary environments (e.g., MacLeod and van der Veen, 2020; Warr and West, 2020) and integrated curriculum (e.g., Edström and Kolmos, 2014). Cyders and Hilterbrane, 2016 found that a team collaboration platform (TCP) both improved interaction within student teams and between teachers and students and enabled students to communicate, collaborate, and seek peer support before requesting help from an instructor. Moreover, such indicators as the frequency of posts, students’ activity, and forum interactions can be useful for predicting student course completion, as noted by Yau and Ifenthaler, (2020).

Methodology

Design-based research

The present study is part of broader research aimed at elucidating the design principles for a digital ecosystem model that meets stakeholders’ requirements and adapts the degree program into a blended learning model. The digital ecosystem includes stakeholders, such as students, instructors, and industry representatives; communication media and platforms; the educational context, curriculum, and applications; methods and discipline standards; and the pedagogy applied. The broader research is conducted by following the principles of DBR (McKenney and Reeves, 2018; Hall, 2020), the goal of which is to connect educational research to real-world problems (Amiel and Reeves, 2008). DBR is a mixed-method approach for applied and theory-building techniques (Reimann, 2011). The research cycle includes four core phases: analysis and exploration, design and construction, evaluation and reflection, and implementation and spread (McKenney and Reeves, 2018).

The project-based curriculum of Lapland UAS was first analyzed and explored according to the DBR methodology (Mielikäinen, 2021). In the analysis and exploration phase, context analysis and literature reviews are used to address the origins, observations, and causes of phenomena. This phase defines the problem and the basis for the design requirements to understand the context. The analysis, experiences, and expectations of the curriculum, as well as the learning methods were collected. The study was conducted using semi-structured interviews and online surveys among third-year ICT engineering students (N = 27), instructors (N = 15), and industry representatives (N = 3) at Lapland UAS. The stakeholder experience was based on the current project-based curriculum model of education, in which the learning process is implemented through FTF learning. Learning was implemented onsite and based on the semester projects, including holistically integrated subjects. The results of the content analysis show that students thought project-based learning was a successful approach, and collaboration increased motivation and deeper learning. The agile method Scrum (Abrahamsson et al., 2017; Rising and Janoff, 2000), with regular reviews as a phasing method for the multidisciplinary semester project, was considered important for outlining the posture and milestones of the learning progress. From the instructors’ viewpoint, successful implementation requires close teamwork and collaboration of instructors. Students were concerned that the loss of sociality and connection with the instructors would adversely affect efficiency when moving from traditional on-campus learning to an online environment, as support and guidance were seen as the starting point for progress and motivation.

Learning was transferred to a blended model, mainly an online TCP-supported process, with special attention paid to the SP, CP, and TP. The students were physically present during the FTF lessons on laboratory work, as well as possibly during self-regulated project work. Sociality and guidance were mainly organized via TCP, and special attention was paid to the availability of support based on the results of the previous study. This study explores students’ blended learning experiences in this new setting through the theoretical lens of CoI. The students’ behavior regarding the TCP and developer platforms is also explored.

The results of the previous phase serve as a basis for the design and construction phase described in the present study, which concludes the first cycle of DBR. The blended learning approach was introduced following Garrison’s (2016: 112) seven design principles of CoI:

CoI-1) Plan for the creation of open communication and trust

CoI-2) Plan for critical reflection and discourse

CoI-3) Establish community and cohesion

CoI-4) Establish inquiry dynamics (purposeful inquiry)

CoI-5) Sustain respect and responsibility

CoI-6) Sustain inquiry that moves to resolution

CoI-7) Ensure assessment is congruent with intended processes and outcomes

After the first phase of the first DBR cycle, the main design principles for this sub-study are presented in Table 1.

Table 1.

Design principles, related CoI principles, and strategies transferred from the analysis and exploration phase of the first DBR cycle (Mielikäinen, 2021).

Design principle from the previous phase	Related CoI principle	Strategy for the present study
PjBL should be applied in an integrated curriculum context	CoI-2	PjBL will be applied with authentic semester project assignments. The outcomes and results will be presented to the public at the end of the semester
	CoI-3
	CoI-4
	CoI-5
	CoI-6
Collaboration in PjBL should be further supported	CoI-3	PjBL will be applied in student project groups to facilitate group cohesion using TCP with private team channels
PjBL must be applied with authentic project management methods and reviews used in the industry. Particular attention should be paid to project management and monitoring in reviews	CoI-2	Guidance will be provided on the division of responsibilities and tasks during the project planning and implementation phases. Fair distribution of tasks will be ensured in reviews. Active monitoring and follow-up at both the project group and individual level will be conducted. A respectful climate will be encouraged in platforms and reviews
	CoI-5
	CoI-6
A team collaboration platform should be used to facilitate communication and promote accessibility	CoI-1	The Mattermost application will be used to support the establishment of a learning climate
	CoI-2
	CoI-3
Particular attention should be paid to the orientation phase and the provision of learning materials	CoI-4	An orientation phase will be organized before the project kick-off. Support material will be provided
	CoI-6
Technologies related to the industry 4.0 concept should be utilized	CoI-4	The project assignment will follow the concept of industry 4.0 with technology selections. Design and evaluation will be conducted in close collaboration with the instructors
	CoI-6

Research setting

The educational context consisted of the integrated curriculum for ICT engineering education at Lapland UAS throughout the semester project period between September 1, 2019, and December 12, 2019. The integrated curriculum in ICT engineering education at Lapland UAS consisted of 30 European Credit Transfer and Accumulation System (ECTS) semesters, which in turn comprised five ECTS courses and a project study unit integrating the disciplines. All second-and third-year students (N = 63) were invited to participate in the study. The students were aged 20 years and up, and most were male. The first-year students studied mainly onsite and were excluded from the present study as FTF interaction is essential for grouping in the early stages (Garrison and Vaughan, 2008: 34). One of the researchers of the present study acted as an instructor. The study’s aim and processes were explained to the students, who were told that participation in the study was voluntary and would not affect their grades. In Finland, ethical review in human sciences is only required for certain research designs (TENK, 2019). In this case, the research does not include any of the factors that require an ethical review.

Aspects of the learning experience were studied in the context of Industry 4.0 in project-based learning. Key elements of the Industry 4.0 concept (Rosin et al., 2020; Bartodziej, 2017), such as the Internet of Things (IoT), robotics, data analytics, 3D printing, augmented reality, and cloud computing, were applied to the integrated curriculum setting. The 2019 fall semester for second-year students consisted of the following courses integrated into the Game Development and Embedded Systems semester project: Software Engineering, Game Engines, Embedded Systems, and Game Physics. During the semester, students implemented games and wireless game controllers based on their ideas. The third-year studies consisted of the following courses integrated into a Mobile Systems project: Mobile Programming, Automation Technology, Cloud Computing, Information Management, and Event-driven Programming. The semester project included building a camera stabilizer and a mobile application that utilizes not only the stabilizer’s capabilities but also cloud services and machine learning solutions. For example, big data technologies such as pattern recognition and dashboards were included in the Basics of Mobile Systems and Event-Driven Programming study units, and technologies related to autonomous robots, such as Proportional Integral Derivative (PID) controls, Kalman filters, and mechanics for stabilizers, were learned and applied in the Automation Technologies and Mobile System Project study units. The semester project comprises 30 ECTS units, including six five-credit units.

The study-year groups were divided into self-regulated project teams of three-to-four students. In the project planning phase, the teams designed the lead-through and phasing following the agile method Scrum. Prioritized backlogs created by the teams define the contents and goals of each two-week sprint. During the sprints, students received support from other teams or instructors to aid progress, including responsible teachers and staff from the research and development laboratory. The teams also followed quality assurance practices, including internal code reviews. Sprint reviews were organized FTF at the end of each sprint, involving the student team and teachers responsible for the integrated courses. Students also self-evaluated the success and challenges of the sprint in Scrum retrospectives. All project groups presented their projects at the closing seminar, and teachers evaluated the parts of the projects related to their courses. The team-specific closing reports included a comprehensive analysis of the project implementation.

Blended methods were applied throughout the projects. The TCP was used to deliver lectures, lessons, and support, including videos and example codes. Mattermost and GitLab were selected as the TCP and development platforms, respectively, because they can be installed on a self-managed server, and they provide authentication via Active Directory. GitLab is an open-source DevOps application that serves as both a version control system and a social programming environment. GitLab version 12.5 was used. Mattermost is an open-source messaging platform, and the free trial version 4.2.3 of Mattermost was used in the setting. For Mattermost, project teams had private channels to which the instructors did not have access. Discussions, peer support, and technical support worked through a common channel, and there was a dedicated channel for informal conversations.

Data collection and analysis

The CoI questionnaire by Arbaugh et al., (2008) was translated from English to Finnish and adapted to the PjBL context. The students’ responses to the translated and adapted CoI questionnaire were collected anonymously using the Webropol 2.0 survey tool. The questionnaire was sent to all 63 second and third-year students, and data were obtained from 56 students, yielding a response rate of 89%. No demographic data were collected along with the survey. The CoI questionnaire included 34 items adapted to fit the project-based context by the author. The rating scale used for all 34 items included the following categories: Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree, and Not Applicable. The categories, from Strongly Disagree to Strongly Agree, were re-coded to the numeric values 1–5, respectively, for further analysis, and Not Applicable was treated as missing. The CoI questions can be seen in Appendix A.

The Rasch RSM (Andrich, 1978) was used to analyze the CoI data, as recommended by Abbitt and Boone, (2021). Each CoI presence was analyzed separately, as each is considered to represent a distinct latent attribute. For each presence, the Rasch model fit was evaluated using the infit mean-square (Infit MNSQ) and outfit mean-square (Outfit MNSQ) for both items and students. MNSQ values of less than 1.4 were treated as acceptable, as recommended by Wright and Linacre, (1994). Elements exhibiting possible misfit with the Rasch model were excluded from the analysis, and the models were re-built with the resulting subset of persons and items. The Rasch item measures in logit units were calculated for each item in the CoI questionnaire to explore the item difficulty in terms of ordering and spacing. Finally, data visualizations of the item difficulty estimates inspired by Wright maps were conducted. As with the Wright maps, the items that are easier to agree with are at the bottom, and the items that are harder to agree with are at the top.

The usage data of TCP and developer platforms, i.e., Mattermost and GitLab, were also collected to provide information on student behavior and productivity throughout the project. The data comprise the number and contents of public messages and commits submitted by the students over the four-month semester project period. The data were extracted from JavaScript Object Notation (JSON) files and aggregated by date, weekday, and hour. The number of messages and commits on each platform was visualized based on the aggregated data.

All data were analyzed in R version 3.6.2 using RStudio version 1.2.5033. The RSMs were conducted using TAM version 3.7 for R (Robitzsch et al., 2021).

Results

(RQ1) Blended learning experience

The distribution of answers for each CoI presence can be seen in Table 2. A vast majority, 82.8%, of the responses were positive (Agree or Strongly Agree), and 2.5% were negative (Disagree or Strongly Disagree). The distribution of answers for each item in the CoI questionnaire is presented in Appendix B.

Table 2.

The number of responses by CoI presence.

Presence	Responses					Items	Missing responses (%)
Presence	Strongly disagree (%)	Disagree (%)	Neutral (%)	Agree (%)	Strongly agree (%)	Items	Missing responses (%)
TP	0.4	2.5	13.8	46.0	37.3	13	1.2
SP	1.2	0.8	14.4	42.5	41.1	9	2.4
CP	0.3	2.1	15.8	50.9	30.9	12	0.3
Total	0.6	1.9	14.7	46.8	36.0	34	1.2

All CoI survey responses were used to build the three Rasch RSMs. The initial models were built using all 56 responses and all items present in each CoI presence. Altogether, six, nine, and eight students exhibited possible misfit with the Rasch model based on the Outfit MNSQ values and were excluded from the final TP, SP, and CP models. The Outfit MNSQ values tended to be higher for students with more negative responses (Disagree or Strongly Disagree) when compared to the other students in the TP (Z = 2.9, p < 0.05), SP (Z = 3.4, p < 0.05), and CP (Z = 3.7, p < 0.05). Moreover, three items (CP23, CP27, and CP29) exhibited possible misfit in the CP and were excluded from the final CP model, and there were no misfitting items on the TP or SP. The final models yielded a person separation reliability of 0.88, 0.86, and 0.87 and an item separation reliability of 0.90, 0.90, and 0.88 for the TP, SP, and CP, respectively.

Rasch item measures and statistics of the final analyses of the TP, SP, and CP are presented in Tables 3, 4, and 5. The total score indicates the sum of the numerical Likert scores, and the total count is the number of students with available data. The Rasch item measure in logit units and its standard error are presented in the columns measure and model SE. Higher values indicate items that are “harder to agree with” and lower values indicate items that are “easier to agree with” by students. Outfit and Infit MNSQ are outlier- and inlier-sensitive Rasch fit statistics.

Table 3.

Rasch item measures and statistics for the TP. Mean outfit = 0.98, mean infit = 1.01.

Category	Item	Total score	Total count	Measure	Model SE	Outfit MNSQ	Infit MNSQ
Design & Organization (DO)	TP1	209	50	−4.21	0.27	1.09	0.89
	TP2	226	50	−5.56	0.30	0.73	0.87
	TP3	218	50	−4.88	0.28	1.04	1.02
	TP4	221	50	−5.13	0.29	1.50	1.47
Facilitation (F)	TP5	215	50	−4.65	0.28	0.97	1.06
	TP6	209	50	−4.21	0.27	1.10	1.22
	TP7	212	50	−4.43	0.27	1.06	1.06
	TP8	199	49	−3.83	0.26	0.64	0.68
	TP9	213	50	−4.50	0.27	1.00	1.00
	TP10	186	49	−2.98	0.25	1.41	1.42
Direct Instructions (DI)	TP11	208	50	−4.14	0.27	0.67	0.71
	TP12	206	50	−4.00	0.26	0.79	0.85
	TP13	196	50	−3.34	0.25	0.79	0.81

Table 4.

Rasch item measures and statistics for the SP. Mean outfit = 0.92, mean infit = 0.97.

Category	Item	Total score	Total count	Measure	Model SE	Outfit MNSQ	Infit MNSQ
Affective Expression (AE)	SP14	196	46	−6.11	0.33	0.94	1.09
	SP15	202	46	−6.80	0.35	1.13	0.86
	SP16	192	45	−6.21	0.34	0.95	1.03
Open Communication (OC)	SP17	186	45	−5.55	0.33	0.86	0.87
	SP18	199	46	−6.45	0.34	0.92	1.10
	SP19	196	47	−5.72	0.32	0.91	0.98
Group Cohesion (GC)	SP20	200	46	−6.56	0.34	0.73	0.69
	SP21	201	46	−6.68	0.34	0.84	1.06
	SP22	173	44	−4.55	0.33	1.04	1.02

Table 5.

Rasch item measures and statistics for the CP. Mean outfit = 0.96, mean infit = 0.99.

	Item	Total score	Total count	Measure	Model SE	Outfit MNSQ	Infit MNSQ
Triggering Events (TE)	CP24	206	48	−6.19	0.31	1.08	1.08
Triggering Events (TE)	CP25	196	48	−5.26	0.30	0.92	0.97
Exploration (E)	CP26	208	48	−6.38	0.31	1.05	1.03
Integration (I)	CP28	174	45	−4.38	0.31	0.98	1.03
Integration (I)	CP30	193	47	−5.42	0.31	0.99	1.04
Resolution (R)	CP31	182	48	−4.89	0.30	0.95	1.01
	CP32	189	46	−5.40	0.31	0.82	0.82
	CP33	196	47	−5.61	0.31	1.02	1.03
	CP34	206	48	−6.19	0.31	0.86	0.93

Teaching presence

Figure 2 presents the order of item difficulty in the TP. Items that were easiest for the students to agree with, i.e., items with lower item difficulty on the logit scale, were primarily associated with the category DO, related to instructor communication. Items relating to the instructors’ actions and feedback were harder to agree with by the students. The items that were the easiest (TP2) and the hardest (TP10, TP13) for the students to agree with were somewhat separate from the other items on the continuum.

Figure 2.

Item difficulty in the TP. DO = Design & Organization, F = Facilitation, DI = Direct Instructions.

Social presence

Figure 3 shows the item difficulty in the SP. Items related to conversing and collaborating through the online medium were the hardest for students to agree with. Item SP22 (“Online discussions helped me develop a sense of collaboration”) was the most difficult for the students to agree with and was separate from the other items on the item difficulty scale. Item SP22 also had the most missing responses. However, the online medium was perceived as an excellent medium for social interaction by the students (SP16), with 89% positive responses. Meanwhile, the items associated with interacting with their team members were the easiest for the students to agree with.

Figure 3.

Item difficulty in the SP. AE = Affective Expression, OC = Open Communication, GC = Group Cohesion.

Cognitive presence

The order of item difficulty can be seen in Figure 4. Items CP23, CP27, and CP29 were excluded from the Rasch model based on the Outfit MNSQ values, as stated previously. Items CP26, CP24, and CP34 form a cluster on the low end of the logit scale and were the easiest for the students to agree with. Item Q28, relating to online discussions as a medium for appreciating different perspectives, was the hardest for the students to agree with.

Figure 4.

Item difficulty in the CP. TE = Triggering Events, E = Exploration, I = Integration, R = Resolution.

(RQ2) Use of TCP and Developer Platforms

The analysis of the data produced by GitLab during the semester project is shown in Figure 5. The data contained 1,154 commits in total, a number that visibly increased towards the end of the semester. The number of commits was the lowest during the fall vacation in week 42, although there were plenty of commits in the previous week. Most commits were made during the two days preceding a project deadline, and a few commits were submitted after the deadline. Commits were made mostly during the weekdays, but some commits were also submitted on the weekends. Commits took place mainly during office hours, and the number of commits was highest at noon. Some commits were also done late at night. The contents of the subject field left in connection with the commits included both professional messages, as well as non-descriptive comments, such as “stuff” or “fix.”

Figure 5.

GitLab usage statistics.

The message data from the Mattermost TCP are presented in Figure 6. Overall, 297 messages were sent to the public communication channels by 35 users, including five teachers. Given that 56 students were enrolled in the integrated semester project, many did not send any messages to the public channels. The fall holiday was reflected by a low number of messages, as with the GitLab data. Communication mainly took place during office hours and on weekdays.

Figure 6.

Mattermost usage statistics.

Discussion

The purpose of the present study was to examine students’ blended learning experiences with project-based learning through the TP, SP, and CP dimensions of the CoI framework. The students mainly agreed with the items in all three presences, suggesting an overall positive learning experience. As such, the differences observed through the Rasch RSMs were relatively small, and even items that were the most difficult for the students to agree with were largely agreed upon. Moreover, the item difficulty visualizations based on the Rasch model may represent the viewpoint of students with a more positive view of the blended learning approach, as students excluded from the Rasch model based on Outfit MNSQ had a higher proportion of negative responses when compared to the other students. Negative experiences may have been, for instance, because the students did not experience belonging with the group as much as other respondents or because PjBL or blended learning was perhaps an inappropriate learning approach for them. The final Rasch models had relatively high separation reliability in terms of both items and persons, suggesting that the model fit the data well.

Students’ responses to items related to the TP were mainly positive, and items related to teacher communication were the easiest for the students to agree with. The most difficult items to agree with were related to the instructors’ actions in developing a sense of community among project team members (TP10) and providing timely feedback (TP13). Cubric, 2013 found that receiving regular feedback from the instructor was a key factor in group cohesion, reinforcing the importance of the TP. The agile framework Scrum was utilized, including sprint reviews in which the instructors participated. One of the advantages of the Scrum approach in PjBL is receiving regular feedback from the instructor (Fernandes et al., 2021). In addition to regular feedback, timing is also essential and could possibly be improved by deploying learning analytics. For instance, Rios et al. (2019) present a data processing pipeline that collects information from the Git repository to understand a learning experience. The pipeline could provide new indicators and triggers for instructors to provide timely feedback. Hepburn et al. (2021) propose a strategic framework to provide continuous feedback through automated, ongoing, peer-led, and teacher-led feedback. In addition to the instructors’ feedback, other media could be further utilized to support group cohesion and enable more timely feedback. Lam, 2015 suggests an extension of the CoI model by introducing a fourth presence called the autonomy presence, in which the students take over the instructor’s role in deciding and sharing the content and initiating and directing the discourse.

The results of the SP model indicate good group cohesion among project teams, and they highlight challenges related to online communication. Items related to communicating with other team members were the easiest to agree with by the students. Social skills and collaboration can be seen as valuable from the perspective of both industry requirements and student wellbeing. Effective communication is the starting point for business-to-university collaboration and business-oriented student projects, as Chen et al., (2013) noted in their research on the impact of collaborative platforms on academia and industry. Moreover, according to Camarinha-Matos et al., (2017), collaboration is essential to most challenges in Industry 4.0. The results of the earlier analysis and exploration phase of the present DBR indicated that students were concerned about losing sociality and disconnecting from instructors when transitioning into a blended learning model (Mielikäinen, 2021). However, the present study suggests that blended learning did not negatively affect sociality, although the SP should be considered when introducing more online-based learning. Even though online communication was stated to be an excellent medium for social interactions by a vast majority of students, items related to their experiences in online discussions were the most difficult to agree with. In addition, very few messages were sent to the public channels of the TCP. One possible explanation is that the students may have opted to use private channels or other online communication platforms in their team communication if they were, for instance, dissatisfied with the TCP provided by the UAS. Switching between means of communication is now a rule rather than an exception (Lee et al., 2021), and using a variety of tools should be relatively easy for ICT students. In addition, the students had the opportunity for self-regulated onsite work during office hours, in which case chat-mediated communication would not be necessary.

The results of the CP model indicate that students were able to learn transferable skills that can be used in real-world applications. Perceptions of using learning outcomes in different contexts were significantly positive, suggesting the development of not only technical skills but also systemic skills. Noguera et al., (2018) observed similar results in the context of agile strategies, where agile methods in course activities were perceived as meaningful by students, and the knowledge acquired was considered transferable to other contexts. Moreover, a study by Garnjost and Lawter, 2019 showed that PjBL was considered more effective in increasing knowledge acquisition and problem-solving when compared to other learner-focused pedagogies. The blended learning approach enabled project activities to take place anywhere and anytime, as can be seen from the GitLab data. Adopting methodologies, tools, and technologies used by the industry allows the students to develop the skills they need in real work environments. Table 6 presents the design principles generated by this study, which complement the design principles produced in the previous phase, as presented in Table 1.

Table 6.

Design principles to be transferred for the overall DBR study.

Key results/Trigger	Design principles	Related CoI-principle
TP10	Involving the instructors in all channels of the TCP	CoI-2
TP13, TP8, CP28, CP31, most students were passive messengers	Including all stakeholders of the digital ecosystem in discursion, feedback, and support activities	CoI-1
		CoI-2
		CoI-3
		CoI-6
TP8, CP28, CP31, most students were passive messengers	Maintaining professional discussion on questions related to tasks, technologies, and concepts	CoI-2
SP19	Encouraging collaboration across project teams	CoI-1
		CoI-2
		CoI-3
		CoI-5
CP26	Providing sufficient material to support learning	CoI-6

First, the involvement of instructors in all TCP channels may foster cohesion. The results in TP10 suggest the need to strengthen the instructor’s role in terms of project team cohesion. Instructors should also have access to the private channels of the project teams. Garrison (2016: 121) suggests that instructors should be less formal and use, e.g., humor to lighten the atmosphere. Second, activating all stakeholders in the digital ecosystem to TCP’s discussion, feedback, and support functions could help provide feedback. The results of TP13, “The instructors provided feedback in a timely fashion,” indicate that the amount and frequency of feedback should be increased, whereas the potential for additional input by the facilitator may be limited. The results in TP8: “The instructors helped keep the project team on task in a way that helped me learn,” CP28: “Online discussions were valuable in helping me appreciate different perspectives,” and CP31: “Reflection on project content and discussions helped me understand fundamental concepts in this project,” as well as the Mattermost log data analysis result, in which the vast majority of students proved passive messengers in public channels, suggest the need to encourage conversation about tasks, technologies, and concepts. Bringing all stakeholders together in the same community in the TCP can encourage students to engage in ongoing professional discussion, as well as the critical reflection and discourse described by Garrison (2016: 112). Further, the results of SP19 indicate a threshold for communicating with other teams, suggesting the need to encourage collaboration across project team boundaries. Establishing a community connected to the digital ecosystem using TCP and developer tools following DevOps’ collaborative principles and philosophy (Lopez-Fernandez et al., 2021) could, after orientation, dismantle silos. Finally, the results in CP26: “I utilized a variety of information sources to explore problems posed in this semester project,” were positive, triggering the design principle to provide sufficient digital material to support learning. Open educational resources and massive open online courses (MOOCs) enable the support of independent learning (Gürdür Broo et al., 2022) alongside the core course material.

There is a clear need to incorporate remote work and industry practices into teaching and learning. More research is needed on blended and project-based learning with the increasing amount of online learning. Further studies are also needed to explore how learning experiences change when a setting such as that in the present study is implemented increasingly online, even completely.

The findings of the present study should be interpreted in consideration of certain limitations. First, the results cannot be directly attributed to blended learning or PjBL alone, as many factors may have influenced the students’ learning experience and no verbal or textual feedback was collected. Second, the items on the CoI survey were translated and adapted to fit the PjBL context, distinguishing the survey from the original CoI survey. This may have introduced challenges related to reflecting the results directly on the CoI framework, as well as comparing the results with other CoI studies. Third, the students’ experiences of blended learning and PjBL may have differed depending on whether they were faced with it for the first time or whether they have already developed a related routine, but data on year of study were not collected. Finally, the study was conducted in a single UAS, limiting the generalizability of the results.

Conclusions

Remote work practices and modern working methods in the industry have enlightened engineering education about the need to adapt e-learning solutions and move from traditional contact lessons to blended learning. Learning working life skills is essential already during higher education studies. The present study provides insights into students’ use of TCP and developer platforms, as well as results from the students’ individual learning experiences through the CoI model. The CoI framework was used to elucidate the learning experiences of ICT engineering students from the perspective of the TP, SP, and CP in the context of TCP-supported blended and project-based learning. The results show that students were widely satisfied in all three CoI subcategories, and differences among the CoI categories, as well as between individual items in the questionnaire, were relatively small. Items related to teacher instruction, interacting with team members, and applying knowledge to real-world problems were the easiest to agree with by the students, and items related to the teachers’ actions in developing a sense of community, timely feedback, and online communication were the most difficult to agree with.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Disclosure

No potential conflict of interest was reported by the authors.

APPENDIX A

Table A1. CoI questionnaire (published and licensed under CC BY-SA 4.0 in https://coi.athabascau.ca/coi-model/coi-survey/) adapted from Arbaugh et al. (2008). Reprinted from The Internet and Higher Education, 11, Arbaugh et al., Developing a community of inquiry instrument: Testing a measure of the Community of Inquiry framework using a multi-institutional sample, 133-136. Copyright (2008), with permission from Elsevier.

Survey item	Statement
TP1	The instructors clearly communicated important project topics
TP2	The instructors clearly communicated important course goals
TP3	The instructors provided clear instructions on how to participate in project learning activities
TP4	The instructors clearly communicated important due dates and time frames for learning activities
TP5	The instructors were helpful in identifying areas of agreement and disagreement on project topics that helped me learn
TP6	The instructors were helpful in guiding the project team towards understanding project topics in a way that helped me clarify my thinking
TP7	The instructors helped to keep project members engaged and participating in productive dialogue
TP8	The instructors helped keep the project team on task in a way that helped me learn
TP9	The instructors encouraged the project team to explore new concepts in this semester project
TP10	Instructor’s actions reinforced the development of a sense of community among project team members
TP11	The instructors helped focus discussion on relevant issues in a way that helped me learn
TP12	The instructors provided feedback that helped me understand my strengths and weaknesses relative to the project’s goals and objectives
TP13	The instructors provided feedback in a timely fashion
SP14	Getting to know other project members gave me a sense of belonging in the project team
SP15	I was able to form distinct impressions about some project team members
SP16	Online or web-based communication is an excellent medium for social interaction
SP17	I felt comfortable conversing through the online medium
SP18	I felt comfortable participating in my own project team discussions
SP19	I felt comfortable interacting with members from other teams
SP20	I felt comfortable disagreeing with other project team members while still maintaining a sense of trust
SP21	I felt that my point of view was acknowledged by other team members
SP22	Online discussions help me to develop a sense of collaboration
CP23	Problems posed increased my interest in project issues
CP24	Project activities piqued my curiosity
CP25	I felt motivated to explore content-related questions
CP26	I utilized a variety of information sources to explore problems posed in this semester project
CP27	Brainstorming and finding relevant information helped me resolve content-related questions
CP28	Online discussions were valuable in helping me appreciate different perspectives
CP29	Combining new information helped me answer questions raised in project activities
CP30	Learning activities helped me construct explanations/solutions
CP31	Reflection on project content and discussions helped me understand fundamental concepts in this project
CP32	I can describe ways to test and apply the knowledge created in this project
CP33	I have developed solutions to project problems that can be applied in practice
CP34	I can apply the knowledge created in this project to my work or other non-school related activities

APPENDIX B

Table B1. Distribution of answers for each item in the CoI questionnaire.

Item	Total	Strongly disagree		Disagree		Neutral		Agree		Strongly agree
Item	Total	Count	%	Count	%	Count	%	Count	%	Count	%
TP1	56	0	0.0	1	1.8	6	10.7	33	58.9	16	28.6
TP2	56	1	1.8	1	1.8	3	5.3	19	33.3	33	57.9
TP3	56	0	0.0	0	0.0	8	14.3	24	42.9	24	42.9
TP4	56	0	0.0	1	1.8	3	5.4	25	44.6	27	48.2
TP5	56	0	0.0	1	1.8	6	10.7	21	37.5	28	50.0
TP6	56	0	0.0	2	3.6	6	10.7	24	42.9	24	42.9
TP7	56	0	0.0	0	0.0	7	12.5	27	48.2	22	39.3
TP8	55	0	0.0	2	3.6	7	12.7	33	60.0	13	23.6
TP9	56	0	0.0	0	0.0	10	17.9	22	39.3	24	42.9
TP10	55	0	0.0	4	7.3	18	32.7	20	36.4	13	23.6
TP11	56	1	1.8	1	1.8	8	14.3	29	51.8	17	30.4
TP12	56	1	1.8	3	5.4	6	10.7	28	50.0	18	32.1
TP13	56	1	1.8	2	3.6	12	21.4	29	51.8	12	21.4
SP14	55	1	1.8	0	0.0	9	16.4	18	32.7	27	49.1
SP15	55	1	1.8	1	1.8	2	3.6	25	45.5	26	47.3
SP16	54	0	0.0	0	0.0	6	11.1	27	50.0	21	38.9
SP17	54	0	0.0	0	0.0	8	14.8	29	53.7	17	31.5
SP18	56	1	1.8	1	1.8	9	16.1	18	32.1	27	48.2
SP19	56	0	0.0	1	1.8	12	21.4	23	41.1	20	35.7
SP20	55	2	3.6	0	0.0	4	7.3	26	47.3	23	41.8
SP21	55	1	1.8	1	1.8	8	14.5	17	30.9	28	50.9
SP22	53	0	0.0	1	1.9	13	24.5	26	49.1	13	24.5
CP23	56	0	0.0	2	3.6	8	14.3	23	41.1	23	41.1
CP24	57	0	0.0	0	0.0	5	8.8	29	50.9	23	40.4
CP25	56	0	0.0	1	1.8	11	19.6	27	48.2	17	30.4
CP26	56	0	0.0	0	0.0	5	8.9	26	46.4	25	44.6
CP27	55	0	0.0	7	12.7	12	21.8	23	41.8	13	23.6
CP28	53	0	0.0	2	3.8	16	30.2	27	50.9	8	15.1
CP29	56	2	3.6	2	3.6	7	12.5	34	60.7	11	19.6
CP30	55	0	0.0	0	0.0	8	14.5	33	60.0	14	25.5
CP31	56	0	0.0	0	0.0	10	17.9	33	58.9	13	23.2
CP32	54	0	0.0	0	0.0	7	13.0	33	61.1	14	25.9
CP33	55	0	0.0	0	0.0	8	14.5	28	50.9	19	34.5
CP34	56	0	0.0	0	0.0	8	14.3	22	39.3	26	46.4

Author Biographies

Maisa Mielikäinen holds a M.Sc. (Tech) and is currently a senior lecturer in the field of ICT engineering education at Lapland University of Applied Sciences. In addition to her lecturing responsibilities, she is also pursuing a doctoral degree at the University of Lapland. She has established herself with a substantial experience in project management education, expertise in student guidance and curriculum development. Prior to her role as a senior lecturer, she held a position as a project and quality manager in the ICT industry.

Essi Viippola holds a M.Sc. and has experience in data analysis in both academic and industrial settings. She is currently pursuing a doctoral degree at the University of Helsinki. Her research interests are in interpretable machine learning.

Tauno Tepsa is a senior lecturer in the ICT engineering education at Lapland University of Applied Sciences. He holds a M.Sc. (Tech) and brings a wealth of experience in the realm of teaching electronics, IoT, and embedded systems. Additionally, he has experience in development projects in both academic and industrial contexts. His area of interest lies in the fields of electronics, embedded systems, automation and robotics, virtual reality, and digital twins.

References

Abbitt

Boone

(2021) Gaining insight from survey data: an analysis of the community of inquiry survey using Rasch measurement techniques. Journal of Computing in Higher Education 33: 367–397.

Abrahamsson

Salo

Ronkainen

, et al. (2017) Agile software development methods: Review and analysis. Available at: https://arxiv.org/abs/1709.08439

Almulla

(2020) The effectiveness of the project-based learning (PBL) approach as a way to engage students in learning. Sage Open 10(3): 215824402093870.

Amiel

Reeves

(2008) Design-based research and educational technology: Rethinking technology and the research agenda. Journal of Educational Technology & Society, 11(4): 29–40.

Anderson

Rourke

Garrison

, et al. (2019) Assessing teaching presence in a computer conferencing context. Online Learning 5(2). DOI: 10.24059/olj.v5i2.1875.

Andrich

(1978) A rating formulation for ordered response categories. Psychometrika 43(4): 561–573.

Arbaugh

Cleveland-Innes

Diaz

, et al. (2008) Developing a community of inquiry instrument: Testing a measure of the community of inquiry framework using a multi-institutional sample. The Internet and Higher Education 11(3–4): 133–136.

Bartodziej

(2017) The Concept Industry 4.0. Cham, Switzerland: Springer. pp. 27–50.

Camarinha-Matos

Fornasiero

Afsarmanesh

(2017) Collaborative networks as a core enabler of industry 4.0. In:Working Conference on Virtual Enterprises, Vicenza, Italy, 18-20 September 2017, pp. 3–17. Cham, Switzerland: Springer.

10.

Chen

Yang

, et al. (2013) S&T collaboration platform for higher education institutions and industry: a case study of Wenzhou, China. Engineering Economics 24(5): 424–436.

11.

Chen

Kolmos

(2021) Forms of implementation and challenges of PBL in engineering education: a review of literature. European Journal of Engineering Education 46(1): 90–115.

12.

Chowdhury

Murzi

(2020) The evolution of teamwork in the engineering workplace from the first industrial revolution to industry 4.0: A literature review. In:2020 ASEE Virtual Annual Conference Content Access. Virtual On Line. 22-26 June 2020. 10.18260/1-2--35318.

13.

Cohen

Manion

Morrison

(2007) Research Methods in Education. England, UK: Routledge.

14.

Cubric

(2013) An agile method for teaching agile in business schools. The International Journal of Management Education 11(3): 119–131.

15.

Cyders

Hilterbrane

(2016) A Classroom Integration of the slack team collaboration tool. In: The Capstone Design Conference, Columbus, OH. 6-8 June 2016.

16.

Dewey

(1997) How We Think. Chelmsford, Massachusetts, United States: Courier Corporation.

17.

Edström

Kolmos

(2014) PBL and CDIO: Complementary models for engineering education development. European Journal of Engineering Education 39(5): 539–555.

18.

Fernandes

Dinis-Carvalho

Ferreira-Oliveira

(2021) Improving the performance of student teams in project-based learning with scrum. Education Sciences 11(8): 444.

19.

Fitsilis

Tsoutsa

Gerogiannis

(2018) Industry 4.0: required personnel competences. Industry 4.0 3(3): 130–133.

20.

Garnjost

Lawter

(2019) Undergraduates’ satisfaction and perceptions of learning outcomes across teacher-and learner-focused pedagogies. The International Journal of Management Education 17(2): 267–275.

21.

Garrison

Anderson

Archer

(1999) Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education 2(2–3): 87–105.

22.

Garrison

Vaughan

(2008). Blended Learning in Higher Education: Framework, Principles, and Guidelines. New York, United States: John Wiley.

23.

Garrison

Cleveland-Innes

Fung

(2010) Exploring causal relationships among teaching, cognitive and social presence: Student perceptions of the community of inquiry framework. The Internet and Higher Education 13(1–2): 31–36.

24.

Garrison

(2016) E-Learning in the 21st Century: A Community of Inquiry Framework for Research and Practice. Milton Park, England: Taylor and Francis.

25.

Graham

(2006) Blended learning systems. In: Bonk

Graham

(eds) The Handbook of Blended Learning: Global Perspectives, Local Designs. Hoboken, New Jersey, United States: Wiley. pp.3–21.

26.

Gürdür Broo

Kaynak

Sait

(2022) Rethinking engineering education at the age of industry 5.0. Journal of Industrial Information Integration 25, 100311. DOI: 10.1016/j.jii.2021.100311.

27.

Hall

(2020) Bridging practice and theory: The emerging potential of design-based research (DBR) for digital innovation in education. Education Research and Perspectives (Online) 47: 157–173.

28.

Hepburn

Borthwick

Kerr

, et al. (2021) A strategic framework for delivering ongoing feedback at scale. Assessment and Evaluation in Higher Education 47, 742–754. DOI: 10.1080/02602938.2021.1959517.

29.

Hogarth

(2010) Adopting Blended Learning for Collaborative Work in Higher Education. Hauppauge, New York, United States: Nova Science Publishers, Incorporated.

30.

Intana

(2020) Teaching requirement engineering using industrial-infused project-based learning. In: 2020 5th International STEM Education Conference (iSTEM-Ed), Hua Hin, Thailand. 4-6 November 2020, pp.82–85. IEEE.

31.

Jamieson

(2004) Likert scales: How to (ab) use them? Medical Education, 38(12): 1217–1218.

32.

Kamp

(2016) Engineering Education in the Rapidly Changing World. Delft, Netherlands: TU Delft, Faculty of Aerospace Engineering.

33.

Krajcik

Blumenfeld

(2006) Project-based learning. In Sawyer

(Ed.) The Cambridge Handbook of the Learning Sciences. New York, NY, USA: Cambridge University Press. pp.317–334.

34.

Lam

(2015) Autonomy presence in the extended community of inquiry. International Journal of Continuing Education and Lifelong Learning 8(1): 39–61.

35.

Lee

Pham

Crosby

, et al. (2021) Digital collaboration in design education: how online collaborative software changes the practices and places of learning. Pedagogy, Culture and Society 29(2), 231–245.

36.

Lopez-Fernandez

Diaz

Garcia

, et al. (2022) DevOps Team structures: Characterization and implications. IEEE Transactions on Software Engineering 48, 3716, 3736, DOI: 10.1109/TSE.2021.3102982.

37.

Mativo

Sochacka

Youngblood

, et al. (2017) Developing real-life problem-based learning (PBL) activities through partnership with industry. In: 2017 ASEE Annual Conference and Exposition, Columbus, OH. 24-28 June 2017.

38.

MacLeod

van der Veen

(2020) Scaffolding interdisciplinary project-based learning: a case study. European Journal of Engineering Education 45(3): 363–377.

39.

McKenney

Reeves

(2018) Conducting Educational Design Research. England, UK: Routledge.

40.

Mielikäinen

(2021) Towards blended learning: Stakeholders’ perspectives on a project-based integrated curriculum in ICT engineering education. Industry and Higher Education 36(1): 74–85.

41.

Noguera

Guerrero-Roldán

Masó

(2018) Collaborative agile learning in online environments: Strategies for improving team regulation and project management. Computers & Education, 116: 110–129.

42.

Reimann

(2011) Design-based research. In: Methodological choice and design. Dordrecht, Netherlands: Springer, pp.37–50.

43.

Ríos

JCC

Kopec-Harding

Eraslan

, et al. (2019) A methodology for using GitLab for software engineering learning analytics. In: The 2019 IEEE/ACM 12th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE), Montreal, QC, Canada. 27 May 2019. pp.3–6. DOI: 10.1109/CHASE.2019.00009.

44.

Rising

Janoff

(2000) The scrum software development process for small teams. IEEE Software, 17(4): 26–32.

45.

Robitzsch

Kiefer

(2021) Package ‘TAM’. Available at: https://rdocumentation.org/packages/TAM/versions/3.7-16 (accessed 8 Oct 2021).

46.

Rosin

Forget

Lamouri

, et al. (2020) Impacts of Industry 4.0 technologies on lean principles. International Journal of Production Research 58(6): 1644–1661.

47.

Sakulviriyakitkul

Sintanakul

Srisomphan

(2020) The design of a learning process for promoting teamwork using project-based learning and the concept of agile software development. International Journal of Emerging Technologies in Learning (IJET) 15(3): 207–222.

48.

Sanchez-Romero

Jimeno-Morenilla

Pertegal-Felices

, et al. (2019) Design and application of project-based learning methodologies for small groups within computer fundamentals subjects. IEEE Access 7: 12456–12466.

49.

Stenbom

(2018) A systematic review of the community of inquiry survey. The Internet and Higher Education 39: 22–32.

50.

TENK. Finnish National Board of Research Integrity (2019) The Ethical Principles of Research with Human Participants and Ethical Review in the Human Sciences in Finland. Budapest, Hungary: Publications of the Finnish National Board on Research Integrity TENK 3/2019. Available at: https://tenk.fi/sites/default/files/2021-01/Ethical_review_in_human_sciences_2020.pdf (accessed 23 Nov 2021).

51.

Vaughan

(2010) A blended community of inquiry approach: Linking student engagement and course redesign. The Internet and Higher Education 13(1–2): 60–65.

52.

Vaughan

Cleveland-Innes

Garrison

(2013) Teaching in Blended Learning Environments: Creating and Sustaining Communities of Inquiry. Athabasca, Canada: Athabasca University Press.

53.

Warr

West

(2020) Bridging academic disciplines with interdisciplinary project-based learning: Challenges and opportunities. Interdisciplinary Journal of Problem-Based Learning 14(1): n1.

54.

Wright

Linacre

(1994) Reasonable mean-square fit values. Rasch Measurement Transactions 8(3): 370.

55.

Yau

JYK

Ifenthaler

(2020) Reflections on different learning analytics indicators for supporting study success. International Journal of Learning Analytics and Artificial Intelligence for Education (iJAI) 2(2): 4–23.