Sage Journals: Discover world-class research

Abstract

Industry and higher education are increasingly utilizing online environments due to digitalization. As a result, the learning experiences in these new digital learning ecosystems as communities must be re-examined critically. This study incorporates the second cycle of the design-based research (DBR) study developing the design principles and theoretical framework for a digital learning ecosystem in Information and Communication Technology (ICT) engineering education in Lapland University of Applied Sciences (UAS), Finland. This cycle examines students’ learning experiences in a project and Industry 4.0-based approach in a digital learning ecosystem with authentic industry assignments and involvement. The study examines the learning experiences of ICT engineering students in a project and Industry 4.0-based approach in a digital environment with authentic industry assignments and involvement. The study was carried out using the Community of Inquiry (CoI) approach. Rasch Rating Scale Model was used to analyse first-, second-, and third-year students’ responses to a translated and adapted CoI questionnaire. Open-ended questions were added to the questionnaire, which was then analysed using content analysis. The results indicate that students perceived project-based learning in an online setting positively. However, the findings point to issues with social interactions and the actual application of learnt knowledge and skills. Challenges in task management and scheduling, as well as receiving feedback, had a somewhat negative impact on the learning experience, particularly during the first year of study. Finally, this paper concludes by presenting a visual model summarizing the design framework developed through a broader DBR study informed by the previous DBR cycles. The findings may benefit practitioners in developing similar communities and ecosystems.

Keywords

project-based learning community of inquiry design-based research online learning engineering education Rasch rating scale model

Introduction

The COVID-19 pandemic has accelerated the introduction of online and blended learning at all levels of education (Ali, 2020), and teleworking is increasingly being used by the software industry (Russo et al., 2021). The benefits of online learning include flexibility, diversity, inclusion, equality, internationalization, accessibility, and sustainability (Ghanem, 2020). Student satisfaction and enrollment have also been found to improve and the dropout rate decrease when transitioning to online or blended learning (Martínez et al., 2020). However, moving to an online learning environment also poses challenges related to collaboration and communication. The challenges further include the need for a high-quality educational infrastructure to support online and distance learning (Long, 2020) and Massive Open Online Course (MOOC) platforms’ shortcomings in interactivity and collaboration (Gamage et al., 2020). The importance of collaboration and teamwork is emphasized in both learning and preparing the students for working life (Boles & Whelan, 2017). Online technologies will most likely serve as a supplement to traditional teaching and will not be able to completely replace it (Kaur et al., 2021). Blended, online, and distance learning are projected to remain in the post-pandemic educational culture (Park et al., 2021; Qadir & Al-Fuqaha, 2020).

Project-based learning (PjBL) (Granado-Alcón et al., 2020; Hanney, 2018), characterized by student autonomy, collaboration, and communication, not only enables the development of technical skills but is also found to increase so-called “soft skills” (Vogler et al., 2018), such as communication and teamwork (Souza et al., 2019; Vogler et al., 2018). Industry 4.0 (Benis et al., 2021; Fitsilis et al., 2018; Gupta et al., 2019) with its key technologies, such as the Internet of Things (IoT), robotics, data analytics, virtual reality, and cloud services, requires Information and Communication Technology (ICT) engineering students to quickly adopt new skills based on industry needs. With the inclusion of new and emerging technologies and methods in the ICT engineering education curriculum, the students’ working life skill development may be supported by providing them with the most authentic learning environment possible, such as working with real-world projects as part of a community involving both peers and industry representatives.

This study represents the second cycle of a broader Design-Based Research (DBR) developing the design framework with accompanying design principles for a digital learning ecosystem in ICT engineering education in a way that supports industry-specific emerging methods and practices. In this second cycle, the students’ learning experience is explored in a completely online environment with industry representation through a theoretical lens of the Community of Inquiry (CoI) framework introduced by Garrison et al. (1999). The present study seeks answers to the following research question: “How do students experience teaching, social, and cognitive presence in an online environment?.” Finally, in addition to answering the research question, the reader is provided with a visual model of a CoI-based design framework of design principles that summarizes the results of the cycles. Currently, to the best of the authors’ knowledge, there are no corresponding reference studies or frameworks pertaining to ICT engineering education within the existing literature.

Design Framework

Blended learning (e.g., Olapiriyakul & Scher, 2006) is combining face-to-face (FTF) and online activities (N. D. Vaughan et al., 2013, p. 8), that is, it combines digital online discussions, development, and resources with traditional classroom learning. Cronje (2020) challenges the definition to also take the context, theory, method, and technology into account and broadens the definition to a combination of the former to optimize learning. A collaborative CoI is needed for a successful blended learning educational experience, where according to N. D. Vaughan et al. (2013, p. 4) the study strategies and techniques should fuse FTF and online learning. CoI is a collaborative approach to thinking and learning while being a fusion of the personal and social (Garrison, 2016, p. 11). The theoretical framework of CoI, presented in Figure 1, describes its three interdependent elements: Social Presence (SP), Cognitive Presence (CP), and Teaching Presence (TP), establishing procedures for critical inquiry and the collaborative construction of personal, meaningful, and shared understanding.

Figure 1.

Theoretical framework of CoI.

In CoI, social presence (SP) is the individuals’ ability to identify themselves with a group and communicate openly in a trusting environment (Garrison, 2016, pp. 22–27). For example, Ng (2022), drawing on earlier research, discovered that SP is crucial for interaction with other students and knowledge acquisition. Students enjoyed collaborating in an online lab during online sessions, as long as the social functions of the online tools were utilized (Ng, 2022). SP has even been found to be a critical factor in improving students’ successful online learning experiences and outcomes (see e.g., Lim, 2023). On the other hand, according to Akyol and Garrison’s (2008) research results from a study on dynamics of an online educational experience through the lens of the CoI, social presence did not affect learning, but it was associated with satisfaction. Mutezo and Maré (2023) even argue that deficiencies in meeting social needs may result in students not being able to focus on learning course content.

According to Anderson et al. (2001), teaching presence (TP) is “the design, facilitation, and direction of cognitive, and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes.”Zhang et al. (2019) found, for example, that timely feedback promotes student participation, as well as progress management, rational arrangement of learning, and online communication and discussions had an encouraging effect on learning. The TP has indeed been found to have also an association with students’ satisfaction (see e.g., Arsenijevic et al., 2023; Kyei-Blankson et al., 2019; Tayeh, 2021). In fact, it has been discovered that the TP benefits both SP and CP, as well as indirectly on learning performance as a facilitator of students’ cognitive thinking and social interaction (Law et al., 2019). However, Padayachee and Campbell (2022) emphasize that TP is not only about the teacher’s presence but about the students’ joint responsibility and control in the teaching and learning dynamics, where the students can be the teachers themselves who guide and supervise their learning in cooperation.

In cognitive presence (CP), learners construct and confirm meaning through sustained reflection and discourse (Garrison et al., 1999). In terms of cognitive presence, teaching, and social presence can be considered to represent the processes needed for epistemic engagement and cognitive presence in online environments (Shea & Bidjerano, 2009). Fostering cognitive presence is important for success in higher education, and requires strategies carefully designed and implemented. According to Moore and Miller’s (2022) literature review on cognitive presence in online courses, useful suggestions for instructors were for example, providing clear participation requirements, versatile integration of technologies and well-structured discussion forums. They also encourage trying different approaches to improve SP, which is also considered to help with cognitive presence (Moore & Miller, 2022).

Factor analyses have been conducted to reconceptualize structure and identify the possible existence of a fourth component, such as Emotional Presence by Cleveland-Innes and Campbell (2012). Other elements, such as Learning Presence (Shea & Bidjerano, 2010) and Autonomous Presence (Lam, 2015) have also been proposed. According to Garrison’s arguments, the issue of regulation has been incorporated into CoI through Shared Metacognition (Garrison, 2016, p. 31), which is a fusion of self-regulation and co-regulation (Garrison, 2016, p. 63) and is located at the intersection of TP and SP (Garrison & Akyol, 2015; N. Vaughan & Wah, 2020).

The present study adheres to the original three CoI elements: TP, SP, and CP. Table 1 presents the CoI presences and categories (Garrison et al., 2006) that have served as a basis for the development of a quantitative CoI survey instrument developed and validated by (in alphabetical order) Arbaugh et al. (2008). The instrument contains 34 questions in English. The questions are divided into presences (CP, SP, and TP) and further into categories. Responses are represented with a five-level Likert rating scale, where 1 = Strongly Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, and 5 = Strongly Agree. Abbitt and Boone (2021) recommend analysing the psychometric properties of the CoI survey with the Rasch Rating Scale Model (Andrich, 1978) to achieve a stronger understanding of the factors of successful online learning experience referred to by Akyol and Garrison (2008) and to account for the non-linearity of the Likert rating scale.

Table 1.

Categories and Indicators.

Presence	Category	Examples of indicators
Cognitive 2 presence (CP)	Triggering Events (TE)	Sense of puzzlement
	Exploration (E)	Information exchange
	Integration (I)	Connecting ideas
	Resolution (R)	Apply new ideas
Teaching presence (TP)	Design and Organization (DO)	Setting curriculum and methods
	Facilitating Discourse (FD)	Sharing personal meaning
	Direct Instruction (DI)	Focusing discussion
Social presence (SP)	Personal/Affective (PA)	Expressing emotions
	Open Communication (OC)	Risk-free expression
	Group Cohesion (GC)	Encouraging collaboration

Source. From “Revisiting methodological issues in transcript analysis: Negotiated coding and reliability” by Garrison et al. (2006), The internet and higher education, 9(1), p. 1-8 (10.1016/j.iheduc.2005.11.001). Copyright year by Elsevier (2006). Reprinted with permission.

The COVID-19 pandemic has substantially increased the need for research related to online learning and the three CoI presences. Aslan (2021) found that cooperative synchronous learning experiences positively influenced IT students’ CoI perceptions and interaction levels. According to a CoI-based survey by Riese and Kann (2021), computer science students who switched to distance learning in the early stages of the pandemic listed advantages such as flexibility, accessibility, and less commuting, but also disadvantages, such as lack of social interactions, decreased motivation, concentration, and study discipline, lack of structure, poorer quality of education, and technical difficulties.

Project-based learning is a method in which the context of learning is provided in real-life problems through learning-by-doing and applying ideas (Llorens et al., 2017). It facilitates authentic learning (McDermott et al., 2017) as a learner-centered approach to construct knowledge through real-life and open-ended problems (Isomöttönen et al., 2019). Project-based learning aims at broader learning outcomes and emphasizes the personal and professional development of students (Edström & Kolmos, 2014). PjBL has been widely used in higher education as a method in broader project studies, where it is often called the capstone project (Deepamala & Shobha, 2018; Joo et al., 2019; Khakurel & Porras, 2020). Further, project-based learning applies mainly to individual subjects in engineering (Hormigo & Rodriguez, 2019; Intana, 2020; Sanchez-Romero et al., 2019; Younis et al., 2019), rather than systematically in the integrations formed by subjects (Pereira et al., 2017) or integrated semesters (Mielikäinen, 2022). According to Llorens et al. (2017), project learning responds to almost all generic skills required in the ICT industry, such as teamwork, decisiveness, proactive approach, innovativeness, ability to find information, and communication skills. The challenges include the lack of technical tools (Gómez-Pablos et al., 2017), “hitchhikers” who contribute little to the project, and coordination difficulties (Shpeizer, 2019). Working remotely in projects requiring physical equipment further introduces challenges related to the equipment and devices (Hou et al., 2021). Various solutions have been developed to support remote learning, including open-source and modular remote laboratory concepts based on global networks (Letowski et al., 2020) and at-home lab kits (DeBoer et al., 2019).

Methodology

Design-Based Research

The present study is a part of the broader DBR study to develop design framework with accompanying design principles for the digital learning ecosystem in ICT engineering education at (see Mielikäinen, 2022; Mielikäinen et al., 2023). DBR was chosen because of its suitability for designing educational interventions (Design-Based Research Collective, 2003) and producing guiding design principles (Gundersen, 2021; van Den Akker, 1999) for complex and practical educational design problems (McKenney & Reeves, 2019, p. 6). In DBR, the iterative development of solutions to complex real-world educational problems provides a context for empirical research (McKenney & Reeves, 2012, p. 8). DBR relies on the role of the researcher as an active player (Barab & Squire, 2004) which is also realized in the present study. DBR is based on iterative cycles, which include steps such as analysis and exploration, design, and construction, evaluation, and reflection, and implementation and spread (McKenney & Reeves, 2019, p. 89). This study presents the second cycle, where mixed methods are used to analyse an intervention’s outcomes and refine the DBR intervention (Design-Based Research Collective, 2003).

During the analysis and exploration phase of the first cycle, the stakeholders’ (students, teachers, R&D personnel, and industry representatives) experiences and expectations about the current integrated curriculum and project-based pedagogy in ICT engineering education were investigated (Mielikäinen, 2022). Based on the results, the students perceived PjBL as a motivating pedagogical model. The reviews included in agile methods provided the students with the desired discipline in terms of schedules and served as feedback channels. Teachers found PjBL to be an effective but relatively time-consuming way to teach. The students suggested a stronger industry involvement in the projects. The following main design principles were needed for the intervention:

PjBL will be applied as an integrated curriculum context, with particular attention paid to the orientation phase.

Collaboration in PjBL should be further supported.

PjBL should be applied with authentic project management methods and reviews used in the industry.

Deploying a team collaboration platform (TCP) is needed to facilitate communication and promote accessibility.

The involvement of industry representatives needs to be promoted.

Emergent technologies related to the Industry 4.0 concept should be utilized.

The PjBL model was transferred to a TCP-supported learning ecosystem, and the blended learning model was applied (Mielikäinen et al., 2023) in accordance with the design principles described above and Garrison’s (2016, p. 112) design principles of CoI. The experiences of the second and third-year students (N = 56) were collected with a CoI questionnaire instrument adapted to the PjBL environment. TCP and developer platform server data were also collected to further explore the students’ behavior on the platforms. Altogether 83% of the responses to the CoI survey were positive (either Agree or Strongly Agree). The results were analysed using the Rasch Rating Scale Model. The results showed that a vast majority of students thought the online medium was an excellent tool for social interaction, but questions associated with discussing through the online medium were relatively harder for the students to agree with. The analysis of the TCP and developer platform data showed that students worked mainly during office hours but also during other times of the day, including weekends, and that the TCP’s common channels were not largely used among the students.

The main design principles and their relationship to the CoI design principles by Garrison (2016, p. 112) are summarized in Table 2.

Table 2.

Design Principles and Strategies Transferred for the Second Cycle.

Design principles	Related CoI principle by Garrison (2016, p. 112)	Strategy for the second cycle of the DBR
Involvement of industry representatives need to be promoted.	Plan for critical reflection and discourse.Establish inquiry dynamics (purposeful inquiry).	Authentic industry-based assignment will be strengthened with the involvement of industry representatives.
Group cohesion with risk-free expression and encouraging cooperation must be maintained.	Plan for the creation of open communication and trust.Plan for critical reflection and discourse.Establish community and cohesion.	Instructors will be involved in the private channels of the project teams. TCP will be switched to one of the most commonly used products in the industry that better supports audio and video transfer and screen sharing.
All stakeholders of the digital learning ecosystem need to be activated in discourse, feedback, and support activities.	Plan for critical reflection and discourse.Establish community and cohesion.Establish inquiry dynamics (purposeful inquiry).	A community related to the digital learning ecosystem utilizing TCP and developer tools according to the principles of open communication, sharing and collaboration culture will be established.
Discussion on task-related questions, technologies, and concepts needs to be maintained.	Plan for critical reflection and discourse.Establish inquiry dynamics (purposeful inquiry).Sustain inquiry that moves to resolution.	Instructors contribute more to maintaining discussion also in private channels of project teams.

The results are transferred as design principles for this second cycle. The present second cycle proceeded through analysis and exploration, design and construction, and evaluation, and reflection phases, and these will be described next.

Context and Participants

In the design and construction phase of this second cycle, all first, second, and third-year ICT engineering students at Lapland UAS were invited to participate in the second DBR cycle described in the present study. The cycle was carried out between January and April 2020, during the COVID-19 pandemic. The curriculum did not include a semester project for the fourth-year students at the time of the study, which is why the fourth-year students were excluded from the analysis. The semester was implemented completely online. The design of the cycle was based on the design principles formulated as a result of the previous cycle

In the semester projects, the courses of the semester were holistically integrated forming a total of 20 to 30 ECTS. Students’ project groups were given an open-ended Industry 4.0-related assignment based on the objectives of the courses. Except for the first-year assignment, the topics came from the industry to provide an authentic learning experience and to promote student engagement and motivation. Industry representatives were invited to join the TCP’s common channels and participate in evaluating the outcome.

The first-year students’ open-ended assignment for the semester project was to implement an autonomous vehicle whose features, such as acceleration, location, battery charge, and power consumption, can be monitored through a web-based interface. The device platform for prototyping was a radio-controlled autonomous vehicle, which was controlled with Arduino and a Raspberry Pi control unit. The architecture consisted of a Python application with WebSockets, Node.js runtime services, and a MongoDB database. The following courses were integrated into the semester project: Simulation Project of Intelligent Technologies; Basics of Electronics; Server Programing; Networks; and Electromagnetism, five ECTS each. The second-year students’ open-ended assignment was to implement a mobile application for the audience of a real snow-cross event. Machine learning may have been optionally included through image recognition to observe the activities of the event. The students applied the following technologies and platforms in their implementations: Android, Google Cloud Platform, AWS, and a robotic building system. The second-year semester project integrated the following courses: Product Development Project; Entrepreneurship and Business; Product Development; System-Oriented Programing; and Information Management. The third-year student’s open-ended assignment was to produce a mobile application and measurement system for a modern, versatile apartment building with sensors. The applied technologies included Android OS, MQTT, IBM Cloud, Raspberry Pi, Pycom, Arduino, and Azure DevOps. The third-year semester project integrated the following courses: IoT Project; Advanced Mobile Programing; Measurement Systems; Management and Leadership; and Professional English for ICT Engineers.

The students worked in teams of 3 to 4 students. The first and second-year students’ project groups were assigned randomly, and the third-year students formed the groups themselves. The first-year students’ project management was based on the Project Management Body of Knowledge (Project Management Institute [PMI], 2017), which includes standard guidelines for project management. The second and third-year students applied the agile software development method Scrum (Gonçalves, 2018; Takeuchi & Nonaka, 1986), and the projects were carried out in 2-week Scrum sprints. The source code was distributed through GitLab following the discipline standards. The devices and components needed were given to the students to be used at home. MS Teams was used as the TCP. In addition to the common channels, the project teams created private channels with the instructors involved. The instructors supported discursion through the TCP using chat, shared screen, and video calls, often also outside of common office hours. Industry representatives were involved in the TCP’s common channels. The project outcomes were presented to the public and industry representatives following the unit’s traditions at the final exhibition of the semester, this time held virtually in MS Teams. Each course was assessed independently from the project outcome according to each course’s assessment criteria.

Data Collection and Analysis

In the analysis and exploration phase of this second cycle, experiences were collected from the students to refine the design principles for the digital learning ecosystem. The data comprise 34 CoI survey questions supplemented with four open, free-text questions about the students’ challenges, successes, and aspirations. Participating in the study was voluntary for the students. An ethical review statement from a human sciences ethics committee was not required based on the Finnish National Board on Research Integrity (TENK. Finnish National Board of Research Integrity, 2019) guidelines. The CoI survey questions were adapted to the semester project context and translated from English to Finnish by the authors. The data collection was conducted using Webropol 2.0 survey tool. The CoI survey questions adapted from Arbaugh et al. (2008) and the open questions are presented in Appendix A.

The survey was delivered to all first, second, and third-year students, of which 34 were first-year students, 26 second-year students, and 32 third-year students. Altogether 79 students responded to the survey, yielding a response rate of 86%. In total, 41% (32 students) of the respondents studied in their first year, 29% (23 students) in their second year, and 30% (24 students) in their third year. The response rates for the three study groups were 94%, 88%, and 75%, respectively. Missing and “Not Applicable” responses were allowed, resulting in a unique response rate for each item. Altogether 77 students (32 first-year, 23 second-year, and 22 third-year students) responded to at least one of the open questions. The students’ responses to each item of the CoI survey are summarized in Appendix B.

The survey data were analysed using the Rasch Rating Scale Model. The three CoI elements, that is, TP, SP, and CP, were analysed separately as each of them represents a separate latent attribute. The model fit for both students and items was assessed using infit and outfit mean-square (MNSQ). Outfit MNSQ values higher than 1.4 were interpreted as misfits and were excluded from the final models based on recommendations by Wright and Linacre (1994). Item difficulty estimates were calculated for each item to examine the item difficulty ordering and the data were visualized. Data were analysed in R version 3.6.2, and TAM version 3.7 for R was used for the Rasch Rating Scale Models.

The students’ responses to the four open questions were analysed using inductive and to some extent deductive content analysis. CoI was used as the classification framework for open questions Q1 to Q3. The data relating to Q1 to Q3 were first coded, after which categories were formed by grouping together codes related to each other through their content or context. The data were then quantified by counting the number of instances of each category. The analysis was carried out for each study group separately. The coding structure is presented in connection with the results in Tables 7 to 9 and in Appendix C. Open question Q4 was analysed by counting the number of positive, neutral, and negative responses.

Results

Rasch Analysis of the CoI Survey

The distribution of the students’ responses to the CoI survey are summarized in Table 3 and Appendix B. In total, 75.6% of the responses were positive (Agree or Strongly Agree), 6.2% were negative (Disagree or Strongly Disagree), and 18.2% were neutral. In addition, 2.6% of the responses were missing. SP had the most positive responses (84.4%), and TP had the most negative responses (7.9%). Third-year students had both the highest percentage of positive responses (79.7%) and the lowest percentage of negative responses (3.3%). First-year students had the lowest percentage of positive responses (73.5%) and second-year students had the highest percentage of negative responses (8.5%). First-year students had the highest percentage of missing responses (4.0%).

Table 3.

The Distribution of Responses by CoI Presence and Study Group.

Presence	Responses					Missing responses (%)
Presence	Strongly disagree (%)	Disagree (%)	Neutral (%)	Agree (%)	Strongly Agree (%)	Missing responses (%)
First-year students	0.9	5.7	19.9	52.7	20.8	4.0
TP	0.0	0.7	25.9	49.1	17.9	4.6
SP	0.1	6.0	10.7	56.6	25.3	2.4
CP	1.4	4.1	20.4	53.7	20.4	4.4
Second-year students	3.0	5.5	17.3	42.3	31.9	1.8
TP	3.0	8.7	20.8	39.6	27.9	0.3
SP	3.5	4.5	10.9	38.3	42.8	2.9
CP	2.6	2.6	18.2	48.3	28.3	2.5
Third-year students	0.5	2.9	16.9	41.2	38.5	1.6
TP	0.7	4.6	20.7	41.6	32.5	2.2
SP	0.5	0.9	7.5	37.1	54.0	1.4
CP	0.4	2.5	20.0	43.9	33.3	1.0

Three Rasch Rating Scale Models were constructed from the CoI survey data, one for each CoI element. The measures and statistics of the final TP, SP, and CP models are presented in Tables 3 to 5. The total score indicates the sum of the numerical Likert scores and the total count is the number of responses. The measure indicates the Rasch item difficulty measure in logit units. Model SE is the standard error of the item measure in logit units. Outfit and infit MNSQ indicate outlier and inlier sensitive MNSQ fit statistics.

Table 4.

Rasch Item Measures and Statistics for TP. Mean outfit = 0.94, mean infit = 0.98.

Category	Item	Total score	Total count	Measure	Model SE	Outfit MNSQ	Infit MNSQ
Design and Organization (DO)	TP1	250	62	−3.7	0.24	1.15	0.97
	TP2	256	63	−3.8	0.24	1.24	1.46
	TP3	244	63	−3.2	0.23	1.04	1.06
Facilitating Discourse (FD)	TP5	254	62	−3.9	0.25	0.85	0.94
	TP6	244	61	−3.5	0.24	0.84	0.93
	TP7	259	63	−4.0	0.25	1.15	1.15
	TP8	239	62	−3.1	0.23	0.88	0.95
	TP9	242	62	−3.2	0.24	0.89	0.89
	TP10	226	59	−3.0	0.24	0.79	0.87
Direct Instructions (DI)	TP11	238	62	−3.0	0.23	0.59	0.65
	TP12	212	58	−2.4	0.23	1.08	1.05
	TP13	231	60	−3.0	0.24	0.76	0.80

Table 5.

Rasch Item Measures and Statistics for SP. Mean outfit = 0.87, mean infit = 0.93.

Category	Item	Total score	Total count	Measure	Model SE	Outfit MNSQ	Infit MNSQ
Personal/Affective (PA)	SP14	289	66	−4.0	0.27	0.82	1.01
	SP15	298	67	−4.4	0.28	0.74	0.88
	SP16	249	66	−1.8	0.20	1.23	1.22
Open Communication (OC)	SP17	276	67	−2.9	0.24	0.95	1.11
Open Communication (OC)	SP18	290	67	−3.8	0.26	0.61	0.77
Group Cohesion (GC)	SP20	289	66	−4.0	0.27	0.98	0.85
	SP21	294	67	−4.1	0.27	0.75	0.86
	SP22	263	65	−2.6	0.23	0.80	0.70

Teaching Presence

The Rasch model for TP is summarized in Table 4. Item TP4 (“The instructors clearly communicated important due dates and time frames for learning activities”) exhibited a possible misfit with the Rasch model based on the outfit MNSQ values and was excluded from the model. Likewise, 16 students were excluded from the final model based on the same criteria. The person and item separation reliabilities were 0.92 and 0.94, respectively.

Figure 2 presents the item difficulty estimates in the TP category. Items associated with Direct Instructions (DI) are on the higher end of the continuum, indicating that these items were the most difficult for the students to agree with relative to the other items on SP. Item TP12 had the highest item difficulty estimate and also the most negative (Strongly Disagree or Disagree) responses within the TP category, as can be seen from Appendix B. Items TP10, TP13, and TP11 are clustered, indicating little difference in item difficulty between these items. Items TP5 and TP7, both associated with Facilitating Discourse, were the easiest to agree with by the students, indicated by lower item difficulty estimates.

Figure 2.

Rasch item difficulty estimates in TP. Higher estimates indicate items that were relatively harder to agree with by the students.

Social Presence

Table 5 summarizes the Rasch model for SP. Item SP19 (“I felt comfortable interacting with team members from other teams”) and 12 students were excluded from the model based on the outfit MNSQ values. The person and item separation reliabilities for the SP model were 0.85 and 0.90, respectively.

Figure 3 presents the items difficulty estimates in the SP category. Items related to online communication (SP16, SP22, SP17) were the most difficult to agree with by the students relative to the other items on SP. Highlighting the relative nature of the item difficulty estimates, 69% of the responses to item SP16 were positive (Agree or Strongly Agree), as can be seen from Appendix B. Items associated with each CoI subcategory, that is, Personal/Affective (PA), Open Communication (OC), and Group Cohesion (GC) are scattered across the continuum. Item SP15 was the easiest to agree with by the students, followed by a cluster of items SP21, SP20, SP14, and SP18.

Figure 3.

Rasch item difficulty estimates in SP. Higher estimates indicate items that were relatively harder to agree with by the students.

Cognitive Presence

The summary of the CP model is presented in Table 6. Item CP27 (“Brainstorming and finding relevant information helped me resolve content-related questions”) and 16 students were excluded from the final model based on outfit MNSQ values. The person and item separation reliabilities were 0.90 and 0.91, respectively.

Table 6.

Rasch Item Measures and Statistics for CP. Mean outfit = 0.98, mean infit = 1.01.

Category	Item	Total score	Total count	Measure	Model SE	Outfit MNSQ	Infit MNSQ
Triggering Events (TE)	CP23	242	60	−3.1	0.25	1.11	1.04
	CP24	252	60	−3.7	0.26	0.60	0.66
	CP25	251	61	−3.4	0.25	1.08	1.01
Integration (I)	CP26	266	61	−4.4	0.27	1.04	1.18
	CP27	234	60	−2.6	0.23	1.42	1.48
	CP28	244	60	−3.1	0.25	1.20	1.16
	CP30	242	62	−2.6	0.23	0.97	1.02
Resolution (R)	CP31	240	60	−3.0	0.24	0.99	1.03
	CP32	237	62	−2.4	0.22	0.56	0.59
	CP33	238	62	−2.4	0.23	0.93	1.06
	CP34	244	61	−2.9	0.24	0.78	0.80

Figure 4 presents the item difficulty estimates in the CP category. Items related to Resolution (R) were relatively harder for the student to agree with, CP32 having the highest item difficulty estimate. Item CP26 related to the versatile use of information sources was the easiest for the students to agree with. Items associated with Triggering Events (TE) related to the students’ interest and motivation were also easy to agree with by the students.

Figure 4.

Rasch item difficulty estimates in CP. Higher estimates indicate items that were relatively harder to agree with by the students.

Content Analysis of the Open Questions

The results of the content analysis of Q1 to Q3 are presented in Tables 7 to 9, after which the students’ responses to Q4 are summarized. Tables also include the structure of coding following the CoI framework.

Table 7.

The Structure of Coding and Results for Q1.

Structure of coding	Number of instances			Example(s)
Structure of coding	First year (N = 32)	Second year (N = 23)	Third year (N = 22)	Example(s)
Group Cohesion (SP)	5	5	4	“It’s easier for me to chat face-to-face with others so this telecommunication feels really unnatural.”“Sometimes I miss that sociality and good friends from school.”
Design & Organization (TP)	5	2	3	“Electronics lectures are impossible from home when there’s no equipment at home.”
Direct Instructions (TP)	4	1	3	“Lack of or poor introduction [to the topic].”
Shared Metacognition (CP & TP)	24	15	20	“Distractions at home and other entertainment sometimes interfere with school chores.”

Table 8.

The Structure of Coding and Results of Q2.

Structure of coding	Number of instances			Example(s)
Structure of coding	First year (N = 32)	Second year (N = 23)	Third year (N = 22)	Example(s)
Group Cohesion (SP)	9	5	6	“Team spirit was great and everyone did their best for the project.”
Direct Instructions (TP)	9	8	6	“The Teams system was awesome; all teachers were reachable through a mobile device.”“Possibility to share your screen with the teacher for help, etc. Announcements were very clear.”
Shared Metacognition (CP & TP)	6	8	4	“Focusing is easier at your own peace.”

Table 9.

The Structure of Coding and Results of Q3.

Structure of coding	Number of instances			Example(s)
Structure of coding	First year (N = 32)	Second year (N = 23)	Third year (N = 22)	Example(s)
Group Cohesion (SP)	2	3	1	“We should have scheduled dates for working together on Teams, and there should have been multiple dates. That would have given more learning to all members.”
Design and Organization (TP)	14	4	1	“I missed condensed materials on the topics covered in the courses. It makes it easier to understand the whole [topic] and to search for information.”
Direct Instructions (TP)	6	2	11	“More common lessons led by teachers”
Facilitating Discourse (TP)	4	4	5	“More joint discussions between the whole class where ideas would be shared etc.”

The results of the content analysis of Q1 (“Which challenges have you experienced during the online learning period this spring?”) are presented in Table 6. The main challenges are associated with Shared Metacognition at the intersection of CP and TP with 59 (76.6%) instances. Students commented on challenges in management and scheduling, online learning as a learning strategy, as well as issues in motivation. Management and scheduling problems were related to for example, lack of daily routines, accumulation of tasks, home distractions, and other environmental stimuli. Online learning as a learning strategy was perceived to be for example, more numbing, requiring more time and initiative, and more challenging for learning through practice. Some responses suspected that the transition to online learning was the main reason for the loss of motivation. Other challenges were associated with Group Cohesion (14 instances, 18.2%) where communication issues and lack of social contacts posed the most issues, Design & Organization (10 instances, 13.0%) where issues included lack of equipment or problems in using them and Direct Instructions (seven instances, 9.1%) where students reported challenges related to lack of support and guidance and trouble using materials offered in English.

Table 8 summarizes the results of the content analysis of Q2 (“What worked well? What things or elements of the semester project would you keep in the future?”). Support and guidance, associated with Direct Instructions, appeared in 23 (29.9%) instances. The TCP with chat made it easier for the students to reach instructors and receive support. Students also found video records of lectures and support videos useful. Group Cohesion was related to 20 (26.0%) instances through team spirit and collaboration, with almost all mentioning MS Teams as the main reason for success. Shared Metacognition appeared in 18 (23.4%) instances in connection to for example, task management and scheduling and online learning due to the opportunity to better manage their own time and focus on peace, for instance.

The results of the content analysis of Q3 (“How would you develop the online learning/working approach?”) are summarized in Table 9. Nearly all ideas were related to the TP category. Ideas related to Design & Organization were mostly from first-year students and included more learning and support materials, better preparation from the instructors, and expanding the recording of the lectures to cover all activities. In Direct Instructions, the suggestions were focused on closer guidance. Ideas in the Facilitating Discourse category included closer interaction with the instructors as well as efficiency and versatility in the use of the platforms. In Group Cohesion, students expressed their wishes for improved collaborative learning. In addition to the results in Table 9 (11.7%) students stated that there was nothing to improve, and eight (10.4%) students said they did not know how to develop the online learning/working environment.

Students were also asked about their opinions on remote project progress and sprint reviews in Q4 (“How did you feel about the remote reviews?”). In total, 61 responses were interpreted as positive, five as neutral, seven as negative, and one included both positive and negative comments. The positive responses included ease and speed of the reviews as well as the possibility to share your screen and make better use of your own computer. Negative responses included the limited possibilities for demonstrating product features and one comment sought more critical feedback from the instructors.

The results of this second cycle suggest that the setting was successful in the online environment and that most of the student responses indicate a positive learning experience in all three core elements of CoI. Overall satisfaction with the learning experience also increased in subsequent years being at its lowest in the first year of study. However, problems related to sociality in SP and shared metacognition at the intersection of TP and CP became more pronounced, as well as the application of knowledge and skills in practice in the CP category proved to be the most challenging. The students’ development proposals, on the other hand, were mainly focused on the TP category. The analysis and reflection phase of this second cycle contributed the design principles described in Table 10.

Table 10.

Design Principles From the Second Cycle. ID Refers to the Questions in the CoI Survey Presented in Appendix A.

Key results/trigger	Design principles	CoI related design principle by Garrison (2016, p. 112)	Strategy
CP27, CP30, lack of learning and support materials	Provide sufficient digital material to support learning	Plan for critical reflection and discourse.Sustain inquiry that moves to resolution.	Online educational resources are suggested to be utilized.
CP32, CP33,SP16, SP22,communication problems, lack of social contacts	A blended learning approach	Establish inquiry dynamics (purposeful inquiry).	A blended learning approach is to be applied with a special focus on the successful implementation of laboratory-related subjects.

As a result of the overall DBR study, a synthesized and summarized visual model of the design framework based on the CoI (Figure 5) can be presented. It illustrates the elements of the digital learning ecosystem in ICT engineering education that supports educational policies and embrace industry-specific emerging methods and practices.

Figure 5.

The design framework of digital learning ecosystem in ICT engineering education.

This visual model summarizes the design principles of both cycles of the DBR study, the first cycle described previously (Mielikäinen, 2022; Mielikäinen et al., 2023) and the second cycle of the DBR study described in the present study. The inner circle of the model represents eight design principles, each with a unique central characteristic that is applied to the digital learning ecosystem and displayed in the outer circle. Additionally, a brief and concise description of each design principle is provided on the outer circle to enhance the comprehensibility of the visual model.

Discussion

The purpose of this study was to gather students’ experiences in an online environment to refine design principles for a digital learning ecosystem in ICT engineering education as part of a broader DBR study. During the second cycle, the CoI framework was incorporated by applying Garrison’s (116, p. 112) CoI design principles to the three elements of CP, SP, and TP in the context of integrated PjBL in an online environment. To support the quantitative results produced by the Rasch rating scale analysis, qualitative data were also analysed through content analysis. CoI was used as an analysis framework in both type of data enabling interpretable comparisons between quantitative and qualitative results through the same scientific lens. Qualitative results supported numerical estimates by providing background information and explanations for quantitative results. The validity of this study has been sought to be improved by carefully reporting on study propositions, interventions, and findings, in order for readers to assess the trustworthiness and confirmability of the results, which correspond to Lincoln and Guba’s (1985, p. 290) criteria for trustworthiness (McKenney & Reeves, 2019, p. 260).

The overall results obtained in this DBR study suggest that the PjBL approach is perceived as a successful strategy by the students, which is in line with the empirical studies, such as Berselli et al. (2020), Chang and Yen (2021), and Coronado et al. (2021). Also, from a transformative perspective, in addition to understanding the evolution of global technology, students gain experiences of success, which increases confidence in their own abilities. The courage to come up with ideas and experiment in a trusting environment, as well as the understanding of the importance of cooperation and community as a resource for achieving goals, is increasing. With digitalization, project teams often work globally in a decentralized manner in industry, and communication takes place on collaboration platforms. COVID-19 has contributed to accelerating the adoption of these pragmatic approaches in both the industrial and scientific communities. However, the learning experience in the community needs to be looked at critically in different contexts.

In terms of TP, the role of the facilitator in keeping the project team committed and maintaining a dialog as well as in helping to make the right choices and solutions was the easiest to agree with. Based on the students’ open responses, chat support in TCP facilitated the availability and reachability of instructors. Recording and sharing lectures were perceived positively. It would also appears that the professionalism and technical competence of instructors has been valued, although open Industry 4.0 problems were not been tried or solved in advance by instructors. Instead, providing individual feedback on strengths and weaknesses in relation to project objectives was considered the most challenging as well as timing the feedback correctly. More interaction between instructors and students was also anticipated in the open responses. In particular, the results of the first-year students confirm the notion that at an early stage, however, the need for close contact is emphasized. Szeto (2015) states in his study of first-year engineering students that the importance of teaching presence is the most essential of the three elements of the CoI framework. Feedback was mainly provided in the context of progress or sprint reviews. In the meantime, instructors participated in project discussions, mainly in the form of technical support. The role of TP is essential, according to Garrison (2016, s. 117), to ensure the transition to the integration and resolution phase. However, increasing the amount of all discussion and feedback has a significant impact on teachers’ resources and coping, which was a concern for teachers based on the responses collected in the analysis and exploration phase of the first iteration of the broader DBR study (Mielikäinen, 2022). Further, students suggested adding learning materials, extending the recording of lectures to all instructors, better preparation, and more general information.

The students made extensive use of information sources to solve the problem. According to a study by Levy and Ramim (2017), instructors saw information acquisition and critical thinking as one of the most important skills a student should have to succeed in online learning. The project also aroused the students’ curiosity and motivated them to explore content issues and problems. Several studies have summed the enhanced motivation for the benefits of PjBL (see e.g., Guo et al., 2020; Hogue et al., 2011; Mills & Treagust, 2003; Shpeizer, 2019; Vasiliene-Vasiliauskiene et al., 2020). However, the responses in open questions did not address CP issues as success factors or problem areas. Based on the quantitative responses, students found resolution issues challenging, which emerged in the online setting described especially in the present study, in contrast to the first cycle of this DBR study, which in the blended learning setting, where the subsequent application of knowledge and skills was perceived as the second easiest to agree with. Some studies have presented PjBL-related challenges, for example, the uncertainty of the project assignment (Hussein, 2021) and a less rigorous understanding of engineering Fundamentals (Mills & Treagust, 2003). In the case of a completely online environment, studying engineering skills without physical experience in the laboratory can negatively affect about the Resolution, which would favor the choice of a blended learning approach by combining FTF and online activities (Garrison, 2016, pp. 100–108) for issues requiring laboratory work. Also, not everyone had the necessary equipment or components at home.

Under the structure of shared metacognition, issues at the intersection of TP and CP (Garrison, 2016, pp. 63–65) were particularly highlighted in open-ended responses. Students report a decrease in motivation, and they report for example, a lack of daily routines as well as challenges in concentration in the home environment. In that respect, the results of this study emphasize the found effect of shared metacognition in CoI. This is in line with for example, the systematic literature review by Rasheed et al. (2020) with the results on the challenges of the online component of blended learning, where self-regulation, as well as the challenges of student isolation, were also reported as challenges. Indeed, online environments require motivation and self-regulated learning (Park et al., 2020; Rasheed et al., 2020). Because motivation initiates, directs, and maintains activities controlling learning (Akyol & Garrison, 2011), it may be reflected in Resolution-related perspectives in CP. Lin et al. (2016) suggest group awareness and peer assistance to foster self-regulation. Also, studying project management, including task management and scheduling, is justified from the very beginning of the studies, contributing to the facilitation of the problems raised in the open answers in the management of study-related tasks and as well as in coping and life management.

Regarding SP, items related to Group Cohesion proved to be the easiest to agree with. Further, students would seem to have been able to create a clear picture of their team members as well as have a sense of belonging to a project team. These observations were also supported by open responses that team spirit was high, and collaboration was largely good. Management and scheduling were also perceived as positive in some responses, as the online approach made it easier to plan and manage one’s own time. The most difficult items to agree with were the claims that online discussions helped to develop a sense of collaboration and to feel comfortable when conversing through the online medium. Thus, online communication works collaboratively, but using it as the only form of communication compared to, for example, face-to-face communication does not necessarily have a positive effect on the learning experience. Especially in the first year, social contacts with other peers are likely to strongly promote group cohesion and reduce feelings of isolation when second and third-year students had already grouped and gotten to know each other. The open responses also suggested promoting group cohesion through collaborative learning. The results seem to indicate the need for social interaction and collaboration, which, according to Ngereja et al. (2020), suggests that knowledge acquisition, reflection, conceptualization, and testing (experience) can best be achieved through interaction. The TCP used can be compared to places of social interaction and networking (Rasheed et al., 2020), which, however, contributes positively to social presence and was perceived as a successful solution in open responses.

Limitations

Although this study was carried out according to the methodology recommended in educational research, certain limitations should be pointed out. This study has been conducted over one semester in a degree program of one university, which limits the generalizability of the results. Alternative disciplines may yield different results when replicating the setting of the present study. Furthermore, a modest sample size might influence the generalizability as well as the reproducibility of the item difficulty hierarchy. It should be noted that while we utilized CoI questionnaires and the Rasch Model as tools for collecting and analysing experiences, our primary focus in this paper was not to refine or further develop them. Translating CoI questionnaire items from English to Finnish may also have changed the nuances or even the meaning of the items. Lastly, attributing the results to the online approach is not without problems as multiple factors, such as the teachers, project topic, team members, or the COVID-19 pandemic may have impacted the results. DBR cannot claim causality with the same convincing rigor as randomized controlled trials (Bakker & van Eerde, 2015).

Conclusions

This paper describes the second cycle of DBR study, mapping the experiences of students with PjBL and an integrated curriculum in a fully online environment. The present study was part of a broader DBR study to develop a design framework alowith accompanying design principles for the digital learning ecosystem in ICT engineering education. The PjBL-based solution of the digital learning ecosystem also successfully incorporated industry presence in the setting. The students’ experiences were analysed through the theoretical lens of the CoI framework with the Rasch Rating Scale Model and content analysis. The Rasch Rating Scale Model was used to produce a hierarchy of item difficulties. The phenomena behind the experiences were mapped out with a few open questions through content analysis. Finally, the presented visual model of the design framework of the design principles for the digital learning ecosystem based on the CoI framework is presented for further discussions and research. Implications are not limited to ICT engineering education but also extend across other areas of engineering and disciplinary fields.

Footnotes

Appendix A

Open questions.

Q1	Which challenges have you experienced during the online learning period this spring?
Q2	What worked well? What things or elements of the semester project would you keep in the future?
Q3	How would you develop the online learning/working approach?
Q4	How did you feel about the remote reviews?

Adapted from Developing a community of inquiry instrument: Testing a measure of the Community of Inquiry framework using a multi-institutional sample by J. B. Arbaugh et al., 2008, The Internet and Higher Education, 11(3-4), pp. 133-136 (10.1016/j.iheduc.2008.06.003). Copyright 2008 by Elsevier. Adapted with permission.

Also adapted from Confirming the subdimensions of teaching, social, and cognitive presences: A construct validity study by S. Cascurlu, 2018, The Internet and Higher Education, 39, p. 1-12 (10.1016/j.iheduc.2018.05.002). Copyright 2018 by Elsevier. Adapted with permission.

Appendix B

Table B1.

Distribution of Answers for Each Item in the CoI Questionnaire.

Item	Total	Strongly disagree		Disagree		Neutral		Agree		Strongly agree
Item	Total	Count	%	Count	%	Count	%	Count	%	Count	%
TP1	78	0	0.0	6	7.7	10	12.8	43	55.1	19	24.4
TP2	79	2	2.5	7	8.9	6	7.6	40	50.6	24	30.4
TP3	79	2	2.5	8	10.1	13	16.5	36	45.6	20	25.3
TP4	78	1	1.3	3	3.8	12	15.4	34	43.6	28	35.9
TP5	78	1	1.3	3	3.8	10	12.8	40	51.3	24	30.8
TP6	77	1	1.3	3	3.9	18	23.4	33	42.9	22	28.6
TP7	78	0	0.0	3	3.8	16	20.5	35	44.9	24	30.8
TP8	78	2	2.6	3	3.8	27	34.6	31	39.7	15	19.2
TP9	77	0	0.0	7	9.1	19	24.7	35	45.5	16	20.8
TP10	74	0	0.0	4	5.4	25	33.8	28	37.8	17	23.0
TP11	78	0	0.0	5	6.4	24	30.8	36	46.2	13	16.7
TP12	73	1	1.4	10	13.7	26	35.6	23	31.5	13	17.8
TP13	73	1	1.4	6	8.2	22	30.1	26	35.6	18	24.7
SP14	78	2	2.6	0	0.0	4	5.1	37	47.4	35	44.9
SP15	79	1	1.3	2	2.5	3	3.8	32	40.5	41	51.9
SP16	78	2	2.6	8	10.3	14	17.9	31	39.7	23	29.5
SP17	79	2	2.5	3	3.8	12	15.2	33	41.8	29	36.7
SP18	78	1	1.3	2	2.6	5	6.4	35	44.9	35	44.9
SP19	72	3	4.2	8	11.1	11	15.3	31	43.1	19	26.4
SP20	77	0	0.0	1	1.3	4	5.2	37	48.1	35	45.5
SP21	78	0	0.0	1	1.3	3	3.8	38	48.7	36	46.2
SP22	76	1	1.3	3	3.9	12	15.8	41	53.9	19	25.0
CP23	77	1	1.3	3	3.9	16	20.8	38	49.4	19	24.7
CP24	77	2	2.6	0	0.0	10	13.0	42	54.5	23	29.9
CP25	78	0	0.0	5	6.4	9	11.5	39	50.0	25	32.1
CP26	78	1	1.3	1	1.3	4	5.1	32	41.0	40	51.3
CP27	76	1	1.3	2	2.6	22	28.9	32	42.1	19	25.0
CP28	76	1	1.3	2	2.6	10	13.2	46	60.5	17	22.5
CP29	79	2	2.5	4	5.1	22	27.8	30	38.0	21	26.6
CP30	78	0	0.0	5	6.4	17	21.8	40	51.3	16	20.5
CP31	74	0	0.0	2	2.7	19	25.7	37	50.0	16	21.6
CP32	77	1	1.3	3	3.9	18	23.4	42	54.5	13	16.9
CP33	75	4	5.3	0	0.0	17	22.7	38	50.7	16	21.3
CP34	76	0	0.0	2	2.6	17	22.4	36	47.4	21	27.6

Appendix C

The Coding Structure for Open Questions Q1 to Q3.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The funding for this publication was provided by Lapland University of Applied Sciences, covering the publication fee and the expenses for language editing.

ORCID iDs

Maisa Mielikäinen

Essi Viippola

References

Abbitt

J. T.

Boone

W. J.

(2021). Gaining insight from survey data: An analysis of the community of inquiry survey using Rasch measurement techniques. Journal of Computing in Higher Education, 1–31.

Akyol

Garrison

D. R.

(2008). The development of a community of inquiry over time in an online course: Understanding the progression and integration of social, cognitive and teaching presence. Journal of Asynchronous Learning Networks, 12(3), 3–22.

Akyol

Garrison

D. R.

(2011). Assessing metacognition in an online community of inquiry. The Internet and Higher Education, 14(3), 183–190.

Ali

(2020). Online and remote learning in higher education institutes: A necessity in light of COVID-19 pandemic. Higher Education Studies, 10(3), 16–25.

Anderson

Rourke

Garrison

Archer

(2001). Assessing teaching presence in a computer conferencing context. Journal of the Asynchronous Learning Network, 5(2), 1–17.

Andrich

(1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561–573.

Arbaugh

J. B.

Cleveland-Innes

Diaz

S. R.

Garrison

D. R.

Ice

Richardson

J. C.

Swan

K. P.

(2008). Developing a community of inquiry instrument: Testing a measure of the community of inquiry framework using a multi-institutional sample. The Internet and Higher Education, 11(3-4), 133–136.

Arsenijevic

Belousova

Tushnova

(2023). Students satisfaction with online higher Education during the COVID-19 pandemic. Education Sciences, 13(4), 364.

Aslan

(2021). The evaluation of collaborative synchronous learning environment within the framework of interaction and community of inquiry: An experimental study. Journal of Pedagogical Research, 5(2), 72–87.

10.

Bakker

van Eerde

(2015). An introduction to design-based research with an example from statistics education. In Bikner-Ahsbahs

Knipping

Presmeg

N. C.

(Eds.), Approaches to qualitative research in mathematics education (pp. 429–466). Springer.

11.

Barab

Squire

(2004). Introduction: Design-based research: Putting a stake in the round. Journal of the Learning Sciences, 13(1), 1–14.

12.

Benis

Amador Nelke

Winokur

(2021). Training the next industrial engineers and managers about industry 4.0: A case study about challenges and opportunities in the COVID-19 Era. Sensors, 21(9), 2905.

13.

Berselli

Bilancia

Luzi

(2020). Project-based learning of advanced CAD/CAE tools in engineering education. International Journal on Interactive Design and Manufacturing, 14(3), 1071–1083.

14.

Boles

Whelan

(2017). Barriers to student success in engineering education. European Journal of Engineering Education, 42(4), 368–381.

15.

Caskurlu

(2018). Confirming the subdimensions of teaching, social, and cognitive presences: A construct validity study. The Internet and Higher Education, 39, 1–12.

16.

Chang

C.-C.

Yen

W.-H.

(2021). The role of learning style in engineering design thinking via project-based STEM course. Asia Pacific Journal of Education, 1–19. https://doi.org/10.1080/02188791.2021.1957776

17.

Cleveland-Innes

Campbell

(2012). Emotional presence, learning, and the online learning environment. International Review of Research in Open and Distance Learning, 13(4), 269–292.

18.

Coronado

J. M.

Moyano

Romero

Ruiz

Rodríguez

(2021). Student long-term perception of project-based learning in civil engineering education: An 18-Year ex-post assessment. Sustainability, 13(4), 1949.

19.

Cronje

(2020). Towards a new definition of blended learning. Electronic Journal of e-Learning, 18(2), 114–121.

20.

DeBoer

Haney

Atiq

S. Z.

Smith

Cox

(2019). Hands-on engagement online: Using a randomised control trial to estimate the impact of an at-home lab kit on student attitudes and achievement in a MOOC. European Journal of Engineering Education, 44(1-2), 234–252.

21.

Deepamala

Shobha

(2018). Effective approach in making capstone project a holistic learning experience to students of undergraduate computer science engineering program. Journal of Technology and Science Education, 8(4), 420.

22.

Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8.

23.

Edström

Kolmos

(2014). PBL and CDIO: Complementary models for engineering education development. European Journal of Engineering Education, 39(5), 539–555.

24.

Fitsilis

Tsoutsa

Gerogiannis

(2018). Industry 4.0: Required personnel competences. Industry 4.0, 3(3), 130–133.

25.

Gamage

Perera

Fernando

(2020). MOOCs lack interactivity and collaborativeness: Evaluating MOOC platforms. International Journal of Engineering Pedagogy (iJEP), 10(2), 94–111.

26.

Garrison

D. R.

(2016). E-learning in the 21st century: A community of inquiry framework for research and practice. Taylor & Francis.

27.

Garrison

D. R.

Akyol

(2015). Toward the development of a metacognition construct for communities of inquiry. The Internet and Higher Education, 24, 66–71.

28.

Garrison

D. R.

Anderson

Archer

(1999). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87–105.

29.

Garrison

D. R.

Cleveland-Innes

Koole

Kappelman

(2006). Revisiting methodological issues in transcript analysis: Negotiated coding and reliability. The Internet and Higher Education, 9(1), 1–8.

30.

Ghanem

(2020, November). E-learning in higher education to achieve SDG 4: Benefits and challenges [Conference session]. 2020 Second international sustainability and resilience conference: technology and innovation in building designs (51154) (pp. 1–6). IEEE.

31.

Gómez-Pablos

V. B.

del Pozo

M. M.

Muñoz-Repiso

A. G. V.

(2017). Project-based learning (PBL) through the incorporation of digital technologies: An evaluation based on the experience of serving teachers. Computers in Human Behavior, 68, 501–512.

32.

Gonçalves

(2018). Scrum. Controlling and Management Review, 62(4), 40–42.

33.

Granado-Alcón

M. D.

Gómez-Baya

Herrera-Gutiérrez

Vélez-Toral

Alonso-Martín

Martínez-Frutos

M. T.

(2020). Project-based learning and the acquisition of competencies and knowledge transfer in higher education. Sustainability, 12(23), 10062.

34.

Gundersen

P. B.

(2021). Exploring the challenges and potentials of working design-based in educational research [Doctoral dissertation, Aalborg Universitet. Det Humanistiske Fakultet.]. PhD-Serien. https://vbn.aau.dk/ws/files/432703538/PHD_PBG_E_pdf.pdf

35.

Guo

Saab

Post

L. S.

Admiraal

(2020). A review of project-based learning in higher education: Student outcomes and measures. International Journal of Educational Research, 102, 101586.

36.

Gupta

Mukhawana

Mashinini

(2019, October). A project based learning tool for industry 4.0 manufacturing engineering education [Conference session]. Proceedings of the international conference on industrial engineering and operations management (pp. 978–984). Toronto, Canada.

37.

Hanney

(2018). Doing, being, becoming: A historical appraisal of the modalities of project-based learning. Teaching in Higher Education, 23(6), 769–783.

38.

Hogue

Kapralos

Desjardins

(2011). The role of project-based learning in IT: A case study in a game development and entrepreneurship program. Interactive Technology and Smart Education, 8(2), 120–134.

39.

Hormigo

Rodriguez

(2019). Designing a project for learning industry 4.0 by applying IoT to urban garden. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, 14(2), 58–65.

40.

Hou

Muheidat

Usher

Prado

Guo

Van Wart

(2021, April). Evaluation of the COVID-19 shock on STEM laboratory courses [Conference session]. 2021 IEEE global engineering education conference (EDUCON) (pp. 86–93). IEEE.

41.

Hussein

(2021). Addressing collaboration challenges in project-based learning: The student’s perspective. Education Sciences, 11(8), 434.

42.

Intana

(2020, November). Teaching requirement engineering using industrial-infused project-based learning [Conference session]. 2020 5th International STEM education conference (ISTEM-Ed) (pp. 82–85). IEEE.

43.

Isomöttönen

Daniels

Cajander

Å.

Pears

Mcdermott

(2019). Searching for global employability: Can capitalize on enabling learning environments? ACM Transactions on Computing Education, 19(2), 1–29.

44.

Joo

Y. J.

Lim

K. Y.

Lee

S. Y.

(2019). Project-based learning in capstone design courses for engineering students: Factors affecting learning outcomes. Issues in Educational Research, 29(1), 123.

45.

Kaur

Garg

Kaur

(2021). Case study: Student’s response towards online learning in engineering education during COVID-19 Pandemic. Journal of Engineering Education Transformations, 34(3), 62–69.

46.

Khakurel

Porras

(2020, November). The effect of real-world capstone project in an acquisition of soft skills among software engineering students. 2020 IEEE 32nd Conference on Software Engineering Education and Training (CSEE&T) (pp. 1–9). IEEE.

47.

Kyei-Blankson

Ntuli

Donnelly

(2019). Establishing the importance of interaction and presence to student learning in online environments. Journal of Interactive Learning Research, 30(4), 539–560.

48.

Lam

J. Y. C.

(2015). Autonomy presence in the extended community of inquiry. International Journal of Continuing Education and Lifelong Learning, 8(1), 39–61.

49.

Law

K. M. Y.

Geng

(2019). Student enrollment, motivation and learning performance in a blended learning environment: The mediating effects of social, teaching, and cognitive presence. Computers & Education, 136, 1–12.

50.

Letowski

Lavayssière

Larroque

Schröder

Luthon

(2020). A fully open source remote laboratory for practical learning. Electronics, 9(11), 1832.

51.

Levy

Ramim

M. M.

(2017). The e-learning skills gap study: Initial results of skills desired for persistence and success in online engineering and computing courses [Conference session]. Proceeding of the Chais 2017 conference on innovative and learning technologies research (pp. 57–68). The Open University of Israel, Raanana, Israel.

52.

Lim

(2023). Exploring the relationships between interaction measures and learning outcomes through social network analysis: The mediating role of social presence. International Journal of Educational Technology in Higher Education, 20(1), 1–17.

53.

Lincoln

Y. S.

Guba

E. G.

(1985). Naturalistic inquiry. Sage.

54.

Lin

J. W.

Lai

Y. C.

Lai

Y. C.

Chang

L. C.

(2016). Fostering self-regulated learning in a blended environment using group awareness and peer assistance as external scaffolds. Journal of Computer Assisted Learning, 32(1), 77–93.

55.

Llorens

Berbegal-Mirabent

Llinàs-Audet

(2017). Aligning professional skills and active learning methods: An application for information and communications technology engineering. European Journal of Engineering Education, 42(4), 382–395.

56.

Long

J. M.

(2020). Anywhere-anytime engineering education in a complete undergraduate program. International Journal on Innovations in Online Education, 4(1), 37.

57.

Martínez

P. J.

Aguilar

F. J.

Ortiz

(2020). Transitioning from face-to-face to blended and full online learning engineering master’s program. IEEE Transactions on Education, 63(1), 2–9.

58.

McDermott

Zarb

Daniels

Nylen

Pears

Isomottonen

Caspersen

(2017, October). The authenticity of ‘authentic’ assessment some faculty perceptions [Conference session]. 2017 IEEE frontiers in education conference (FIE) (pp. 1–9). IEEE.

59.

McKenney

Reeves

T. C.

(2012). Conducting educational design research. Taylor & Francis Group.

60.

McKenney

Reeves

T. C.

(2019). Conducting educational design research (2nd ed.). Routledge.

61.

Mielikäinen

(2022). Towards blended learning: Stakeholders’ perspectives on a project-based integrated curriculum in ICT engineering education. Industry and Higher Education, 36(1),. 74–95.

62.

Mielikäinen

Viippola

Tepsa

(2023). Experiences of a project-based blended learning approach in a community of inquiry from information and communication technology engineering students at Lapland university of applied sciences in Finland. E-Learning and Digital Media, 20427530231164053.

63.

Mills

J. E.

Treagust

D. F.

(2003). Engineering education—Is problem-based or project-based learning the answer. Australasian Journal of Engineering Education, 3(2), 2–16.

64.

Moore

R. L.

Miller

C. N.

(2022). Fostering cognitive presence in online courses: Asystematiccrevieww (2008-2020). Online Learning, 26(1), 130–149.

65.

Mutezo

A. T.

Maré

(2023). Teaching and cognitive presences: The mediating effect of social presence in a developing world context. Cogent Education, 10(1), 2171176.

66.

D. T. K.

(2022). Online lab design for aviation engineering students in higher education: A pilot study. Interactive Learning Environments, 1–18. Epub ahead-of-print. https://doi.org/10.1080/10494820.2022.2034888

67.

Ngereja

Hussein

Andersen

(2020). Does Project-Based Learning (PBL) Promote Student Learning? A performance evaluation. Education Sciences, 10(11), 330.

68.

Olapiriyakul

Scher

J. M.

(2006). A guide to establishing hybrid learning courses: Employing information technology to create a new learning experience, and a case study. The Internet and Higher Education, 9(4), 287–301.

69.

Padayachee

Campbell

A. L.

(2022). Supporting a mathematics community of inquiry through online discussion forums: Towards design principles. International Journal of Mathematical Education in Science and Technology, 53(1), 35–63.

70.

Park

J. J.

Jackson

Vanhoy

(2020). Online engineering education under COVID-19 pandemic environment. International Journal of Multidisciplinary Perspectives in Higher Education, 5(1), 160–166. https://files.eric.ed.gov/fulltext/EJ1273006.pdf

71.

Pereira

M. A. C.

Barreto

M. A. M.

Pazeti

(2017). Application of project-based learning in the first year of an Industrial Engineering Program: Lessons learned and challenges. Production, 27 (spe), e20162238.

72.

Project Management Institute (PMI). (2017). A guide to the project management body of knowledge (PMBOK Guide) (6th ed.). PMI.

73.

Qadir

Al-Fuqaha

(2020). A student primer on how to thrive in engineering education during and beyond COVID-19. Education Sciences, 10(9), 236.

74.

Rasheed

R. A.

Kamsin

Abdullah

N. A.

(2020). Challenges in the online component of blended learning: A systematic review. Computers & Education, 144, 103701.

75.

Riese

Kann

(2021, April). Computer science majors’ experiences of their distance education caused by the COVID-19 pandemic [Conference session]. 2021 IEEE Global Engineering Education Conference (EDUCON) (pp. 393–397). IEEE.

76.

Russo

Hanel

P. H.

Altnickel

van Berkel

(2021, May). The daily life of software engineers during the covid-19 pandemic [Conference session]. 2021 IEEE/ACM 43rd international conference on software engineering: software engineering in practice (ICSE-SEIP) (pp. 364–373). IEEE.

77.

Sanchez-Romero

J. L.

Jimeno-Morenilla

Pertegal-Felices

M. L.

Mora-Mora

(2019). Design and application of project-based learning methodologies for small groups within computer fundamentals subjects. IEEE Access, 7, 12456–12466.

78.

Shea

Bidjerano

(2009). Community of inquiry as a theoretical framework to foster “epistemic engagement” and “cognitive presence” in online education. Computers & Education, 52(3), 543–553.

79.

Shea

Bidjerano

(2010). Learning presence: Towards a theory of self-efficacy, self-regulation, and the development of a communities of inquiry in online and blended learning environments. Computers & Education, 55(4), 1721–1731.

80.

Shpeizer

(2019). Towards a successful integration of project-based learning in higher education: Challenges, technologies and methods of implementation. Universal Journal of Educational Research, 7(8), 1765–1771.

81.

Souza

M. C.

Lima

R. M.

Mesquita

(2019). Development and assessment of engineering project management competences in the context of Project-Based Learning (PBL) [Conference session]. International symposium on project approaches in engineering education (pp. 185–192). http://hdl.handle.net/1822/66282

82.

Szeto

(2015). Community of Inquiry as an instructional approach: What effects of teaching, social and cognitive presences are there in blended synchronous learning and teaching? Computers & Education, 81, 191–201.

83.

Takeuchi

Nonaka

(1986). The new product development game. Harvard Business Review, 64(1), 137–146.

84.

Tayeh

(2021). The role of undergraduate students’ satisfaction in the transition to online education in light of teaching and social presence. Multicultural Education, 7(1), 104–113.

85.

TENK. Finnish National Board of Research Integrity. (2019). The ethical principles of research with human participants and ethical review in the human sciences in Finland. Publications of the Finnish National Board on Research Integrity TENK 3/2019. https://tenk.fi/sites/default/files/202101/Ethical_review_in_human_sciences_2020.pdf

86.

The Community of Inquiry: About The Framework. (n.d.). The Community of Inquiry. https://www.thecommunityofinquiry.org/framework

87.

van Den Akker

. (1999). Principles and methods of development research. In van den Akker

Branch

R. M.

Gustafson

Nieveen

Plomp

(Eds.), Design approaches and tools in education and raining (pp. 1–14). Springer.

88.

Vasiliene-Vasiliauskiene

Vasilis Vasiliauskas

Meidute-Kavaliauskiene

Sabaityte

(2020). Peculiarities of educational challenges implementing project-based learning. World Journal on Educational Technology Current Issues, 12(2), 136–149.

89.

Vaughan

Wah

J. L.

(2020). The community of inquiry framework: Future practical directions-shared metacognition. International Journal of E-Learning & Distance Education, 35(1), 1–25.

90.

Vaughan

N. D.

Cleveland-Innes

Garrison

D. R.

(2013). Teaching in blended learning environments: Creating and sustaining communities of inquiry. Athabasca University Press.

91.

Vogler

J. S.

Thompson

Davis

D. W.

Mayfield

B. E.

Finley

P. M.

Yasseri

(2018). The hard work of soft skills: Augmenting the project-based learning experience with interdisciplinary teamwork. Instructional Science, 46(3), 457–488.

92.

Wright

Linacre

(1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370.

93.

Younis

A. A.

Sunderraman

Metzler

Bourgeois

A. G.

(2019, May). Case study: Using project based learning to develop parallel programing and soft skills [Symposium]. 2019 IEEE international parallel and distributed processing symposium workshops (IPDPSW) (pp. 304–311). IEEE.

94.

Zhang

Chen

Zhao

Wang

(2019, August). Research on teaching presence evaluation indexes of online course [Conference session]. 2019 10th International conference on information technology in medicine and education (ITME) (pp. 481–484). IEEE.

ICT Engineering Students’ Perceptions on Project-Based Online Learning in Community of Inquiry (CoI)

Abstract

Keywords

Introduction

Design Framework

Methodology

Design-Based Research

Context and Participants

Data Collection and Analysis

Results

Rasch Analysis of the CoI Survey

Teaching Presence

Social Presence

Cognitive Presence

Content Analysis of the Open Questions

Discussion

Limitations

Conclusions

Footnotes

Appendix A

Appendix B

Appendix C

Declaration of Conflicting Interests

Funding

ORCID iDs

References