A Protocol for Evaluating Learning Outcomes Using Reverse Jigsaw and Edpuzzle in Doctoral Business Education: A Q-Methodology Study

Abstract

The aim of this Q-methodology study is to explore the impact of innovative pedagogical interventions, specifically the Reverse Jigsaw and Edpuzzle methods, on the learning outcomes of doctoral business students. The Reverse Jigsaw technique, an innovative collaborative learning strategy, emphasizes information sharing and knowledge integration among students. Edpuzzle, as an interactive video learning platform, engages students by allowing educators to embed questions into videos. These methods are known to significantly enhance learning experiences, particularly in the complex environment of higher education. Conducted over an 18-week Qualitative Research course at a business school, the participant pool will consist of 15 students. Focus group interviews will be conducted to extract insights from the participants, followed by the application of Q-sorting procedures to categorize subjective opinions. The findings from the Q-methodology analysis are expected to reveal a diverse spectrum of perspectives held by the doctoral business students. These insights will provide valuable guidance for educators, curriculum designers, and institutions to refine their teaching strategies in the doctoral business education context. This research contributes to the ongoing discourse in relation to pedagogical innovation, and endeavors to elevate the overall quality of education in this specialized field by shedding light on these nuanced viewpoints.

Introduction

In the ever-evolving landscape of higher education, enhancing pedagogical approaches and elevating learning outcomes remains of the utmost importance. The pursuit of effective teaching methodologies that foster deep comprehension and the acquisition of skills is a perpetual endeavor within the sphere of doctoral business education. Traditionally, the evaluation of learning outcomes has heavily relied on quantitative data sourced from exit surveys and preceptor ratings. While informative, this method possesses inherent limitations in capturing the nuanced perspectives of doctoral students. To fill this gap, the current research embarks on a journey to explore and evaluate learning outcomes achieved by the application of two innovative pedagogical approaches, namely, the Reverse Jigsaw technique and Edpuzzle, within the context of doctoral business education. These two pedagogies represent a progressive and technology-enhanced teaching and learning approach, which has become prominent in recent years. The Reverse Jigsaw method is a pedagogical strategy wherein students collaboratively assemble individual pieces of knowledge to construct a comprehensive understanding of a given topic (Hedeen, 2003), whereas Edpuzzle integrates multimedia and interactive elements into the learning process, enabling educators to engage students in a more dynamic and participatory way (Cesare et al., 2021; Gallardo-López et al., 2022; Griggs & Moore, 2023). While these methods have exhibited promise across various educational settings, their effectiveness within the specific context of doctoral business education remains worthy of exploration.

The integration of tools, such as the Reverse Jigsaw and Edpuzzle, within a Business Doctoral Course is instrumental in cultivating the essential qualities demanded by contemporary business leadership. Modern business schools aspire to prepare graduates for leadership roles in a corporate landscape that is characterized by collaboration, active listening, influencing, and adaptability, steering them away from traditional “command and control” paradigms. To achieve this shift, it is imperative that students not only excel in their specialized fields, but also broaden their horizons across various disciplines. The Reverse Jigsaw technique facilitates collaborative problem-solving and knowledge integration, while Edpuzzle promotes active listening and critical thinking based on interactive video content (Hamid, 2022; Ware, 2021). Collectively, these methods equip students with the multifaceted skills required to be effective leaders in the modern business world.

This study seeks to bridge this gap by employing Q-methodology, a research approach that enables the systematic and rigorous exploration of individuals’ perspectives, opinions, and attitudes. Q-methodology offers a nuanced and holistic assessment of the subjective views of doctoral students, providing a deeper understanding of their experience and perception of the use of the Reverse Jigsaw and Edpuzzle to achieve learning outcomes. By comprehensively assessing the effectiveness of these teaching methods, this research aims to offer valuable insights and recommendations for educators, curriculum designers, and administrators in the field of doctoral business education. By investigating students’ perspectives, this study endeavors to contribute to the ongoing discourse on pedagogical innovation and its impact on learning outcomes in order to ultimately advance the quality of education in the realm of doctoral business studies. The focus of this study is to utilize Q-methodology to investigate the effects of innovative teaching strategies, namely the Reverse Jigsaw and Edpuzzle methods, on the engagement levels and learning outcomes of doctoral business students.

Literature Review

Utilization of Edpuzzle in Higher Education

Edpuzzle is an interactive video learning platform designed to transform static content into dynamic educational tools. By allowing educators to embed quizzes, annotations, and voiceovers into video materials, it facilitates deeper learner engagement and comprehension (see Figure 1). This interactivity broadens the platform’s applicability across varied educational contexts, making it a versatile tool for enhancing teaching and learning. One of Edpuzzle’s key pedagogical strengths lies in its ability to foster active learning (Cesare et al., 2021). By embedding questions and providing instant feedback, the platform encourages students to engage critically with content, moving beyond passive observation to active participation. This aligns with evidence-based practices in education that prioritize interaction and real-time learning feedback as central components of effective teaching.

Figure 1.

Sample Screenshot of Edpuzzle Videos.

However, despite its advantages, the integration of Edpuzzle into educational settings presents notable challenges. A primary concern is the time-intensive nature of content creation and customization. Although the platform provides a repository of user-shared materials, adapting these resources to align with specific curricular goals often requires significant effort. Additionally, educators with limited technical expertise may encounter difficulties in leveraging Edpuzzle’s features effectively, underscoring the need for targeted training and support at the institutional level. Accessibility poses another critical issue, particularly in regions with inconsistent internet connectivity or limited access to digital devices. This digital divide restricts equitable access to Edpuzzle’s interactive features, potentially excluding certain learner populations from its benefits. Furthermore, the reliance on screen-based tools raises concerns about screen fatigue among students, emphasizing the need for educators to balance interactive video use with other teaching methods to ensure a sustainable and well-rounded learning experience (Griggs & Moore, 2023; Mawaddah et al., 2022).

To maximize the effectiveness of Edpuzzle, educators should adopt strategic content design approaches. Leveraging pre-existing video resources, such as those available on platforms like YouTube, enables the customization of content length to suit instructional needs, thereby streamlining the preparation process. Additionally, incorporating personal uploads allows educators to align materials with specific learning objectives, focusing their efforts on tailoring and refining content to achieve targeted educational outcomes. Institutions play a crucial role in supporting the effective use of Edpuzzle by investing in digital infrastructure, providing professional development for educators, and ensuring equitable access to necessary technologies. The integration of Edpuzzle aligns with established learning theories such as constructivism, which emphasizes active knowledge construction, and the Cognitive Theory of Multimedia Learning, which advocates the dual-channel processing of visual and auditory stimuli for improved learning outcomes. Empirical studies highlight the platform’s effectiveness in enhancing student engagement, comprehension, and retention through its interactive features, such as embedded assessments and annotations (Cesare et al., 2021; Mawaddah et al., 2022).

Nevertheless, challenges persist, including the time required for content creation, accessibility limitations, and inclusivity concerns. Addressing these barriers requires further research to refine Edpuzzle’s implementation strategies and ensure its benefits are accessible across diverse educational environments. As an evolving tool in the digital pedagogical landscape, Edpuzzle exemplifies the transformative potential of interactive video platforms to enrich teaching and learning practices, supporting active engagement and personalized educational experiences in higher education (Griggs & Moore, 2023; Littlefield, 2019).

The Reverse Jigsaw Cooperative Approach

In the dynamic sphere of educational methodology, the Jigsaw cooperative learning strategy emerges as a transformative model that realigns the conventional norms of the engagement and assimilation of knowledge. Grounded in the philosophy of deep constructivism, this approach revolutionizes the perception of knowledge as being a static accumulation of facts to a dynamic, collaborative construct. Influenced by the pioneering insights of thinkers like Scardamalia and Bereiter, this method views knowledge as an outcome of social interaction and collective intellectual effort. As the principle of Knowledge Building (KB) is at its core, it revolves around a cyclical process of questioning, dialogue, and the refinement of ideas, thereby nurturing an ecosystem that is conducive to critical reasoning, cooperative learning, and the development of problem-solving skills (Hong & Scardamalia, 2014). This is a paradigm that lays the groundwork for an educational environment that promotes continuous intellectual advancement and prepares learners for lifelong education (Cochon Drouet et al., 2023).

Departing from the traditional educational models, the Jigsaw method signifies a paradigmatic shift to a learner-centric instructional approach. Characterized by its emphasis on the collaborative efforts of small groups, this method leverages structured tasks that are underpinned by principles of mutual interdependence, individual accountability, and interactive engagement. Research by scholars such as Johnson and Johnson reinforces the effectiveness of this approach in enhancing students’ learning experience based on increased social interaction and collaborative engagement. These scholars have consistently shown that the Jigsaw method not only improves information acquisition and retention, but also fosters the development of sophisticated cognitive skills, interpersonal communication abilities, and cooperative competencies. Its distinctive structure, which entails students becoming adept in particular areas of a topic before amalgamating their insights with those of their peers, effectively balances individual responsibility and group learning (Abed et al., 2020).

The conceptual foundation of the Jigsaw approach is deeply rooted in the constructivist theory of Jean Piaget, which is based on a comprehensive understanding of the way learners internalize and construct knowledge. Piaget’s theories of assimilation and accommodation represent a valuable framework for the integration of new information into pre-existing cognitive structures. In advocating the active role of learners in their educational journey, this theory emphasizes the assimilation of new knowledge into existing schemas in a contextually relevant manner. This approach highlights the significance of guided instruction by situating learners at the forefront of the educational process, accentuating the importance of integrating social and experiential elements in teaching methodologies (Looi et al., 2008; Park, 2023; Wu et al., 2023).

In contrast, the Reverse Jigsaw teaching technique, an offshoot of the original Jigsaw method, is distinguished by its collaborative structure. Here, learners integrate various pieces of knowledge to develop a comprehensive understanding of a subject. This method promotes active involvement, analytical thinking, and peer-to-peer interactions. Different from the traditional Jigsaw approach, the Reverse Jigsaw commences with each student or group presenting a segment of the larger puzzle, followed by a collective effort to construct a unified comprehension. This inversion of roles from a specialist to a collaborator provides a unique take on cooperative learning, enabling a more integrative and holistic learning experience (see Figure 2). Although the Reverse Jigsaw method shows potential across a range of educational settings, further investigation is needed to establish its efficacy in specific contexts. The effectiveness of this method relies on structured implementation, addressing challenges such as equitable participation and students’ accountability. The instructor plays a crucial role in effectively organizing, facilitating, and guiding the learning process, with clear instruction and consistent support to ensure balanced participation and benefit for all learners (Dewati et al., 2019).

Figure 2.

Implementation of Reverse Jigsaw.

Teaching Design and Planning

Course Design

In this 18-week Qualitative Research course for doctoral business students, the curriculum will focus on essential qualitative research methodologies and their application in business contexts. The course was implemented in Fall 2024. Key topics include foundational theories, research design, data collection techniques, including interviews and ethnography, and analytical strategies such as content analysis and the grounded theory. The aim of the course is to equip students with a comprehensive set of skills for conducting effective qualitative research, with a particular emphasis on practical applications in real-world business scenarios.

Course Implementation

The plan is as detailed below and will be modified based on students’ feedback and observations in order to optimize the learning experience (see Figure 3).

➢ Weeks 1–2: Course Initiation and Baseline Evaluation

(1) Orientation to Methods: Begin the course with a comprehensive orientation session to introduce the Reverse Jigsaw technique and Edpuzzle platform to students and describe how these methods are incorporated into the course.

(2) Initial Assessment: Undertake an initial survey or Q-sort in order to assess students’ preliminary understanding and attitude toward these methodologies and qualitative research.

➢ Weeks 3–6: Early Implementation and Acclimatization

(3) Edpuzzle Introduction: Familiarize students with Edpuzzle based on initial video lessons that cover the key concepts of qualitative research to ensure that they are adept at interacting with the platform and its features.

(4) Inaugural Reverse Jigsaw Session: Execute the first session of the Reverse Jigsaw. Assign students to expert groups, each focused on a different element of the course content. Post independent study, groups will reconvene to share their knowledge.

➢ Weeks 7–12: Progressive Application and Deeper Inquiry

(5) Continued Edpuzzle Assignments: Persist with using Edpuzzle for more intricate course content, gradually elevating the complexity and depth.

(6) Ongoing Reverse Jigsaw Sessions: Facilitate bi-weekly Reverse Jigsaw sessions, ensuring that they align with the course syllabus and rotating group composition to promote varied interactions.

➢ Weeks 13–16: Advanced Engagement and Student-Led Activities

(7) Student-Generated Edpuzzle Content: Motivate students to create their own Edpuzzle content, with a focus on their specific research interests in order to deepen their engagement and comprehension.

(8) Student-Directed Reverse Jigsaw Sessions: Shift to student-led sessions in which they select topics, prepare materials, and guide discussions.

➢ Weeks 17–18: Concluding Evaluation and Feedback

(9) Capstone Project and Presentations: Assign a final project that requires students to apply qualitative research methods learned during the course, incorporating Reverse Jigsaw and Edpuzzle techniques in their presentations.

(10) Final Course Assessment: Implement a concluding Q-sort and survey to evaluate shifts in students’ perception and learning outcomes. Arrange individual interviews to collect detailed feedback on their experience of these teaching methods.

Figure 3.

Course Plan.

Research Design and Execution Planning

Research Design: Measuring Subjectivity

Q-methodology is used in this study to address the need for a more nuanced exploration of doctoral students’ perspective of Reverse Jigsaw and Edpuzzle. Q-methodology is a research approach that consists of a combination of qualitative and quantitative elements to facilitate the systematic study of subjective views, attitudes, and opinions (Burke, 2015; Shinebourne, 2009). It is particularly suitable for use in investigating complex and multifaceted research questions, such as the assessment of pedagogical methods in higher education. Previous literature has established that Q-methodology could be used as one of the tools for course evaluation and assessing learning outcomes, offering a nuanced understanding of student experiences and perceptions (Hensel, 2017; Ramlo, 2015). Q-methodology allows the shared patterns of thought within a diverse group of participants to be identified to shed light on their different perspectives of the effectiveness of specific teaching techniques (Brown, 1993, 1996; Churruca et al., 2021).

Q-methodology is used in this study to overcome the several limitations of traditional Likert scale assessments. Likert scales can oversimplify complex opinions into basic numerical values, potentially failing to capture the full spectrum of a respondent’s views. They provide quantitative data, but miss deeper motivations or thoughts due to the lack of ability to gain insights into the reasons for responses. Additionally, respondents may lean toward providing moderate or patterned responses, skewing the data and affecting reliability. In contrast, Q-methodology allows for a richer expression of views, as participants sort statements based on their personal perspective, which offers a more comprehensive understanding of their opinion of the teaching approach (Hammami et al., 2020; Liu et al., 2024).

Participants

The target population for this study consists of 15 doctoral students enrolled in an elective Qualitative Research course in a business education program. Given the potentially wide geographic distribution of these students, a purposive sampling approach will be employed to select the participants. A diverse group of doctoral students will be recruited to ensure a broad representation of views. The exact number of participants will depend on the saturation of views, which will be determined during the data collection process. Q-methodology is especially well-suited for studies with smaller sample sizes, as it emphasizes delving into the depth and intricacy of individual viewpoints. Typically, Q-methodology studies involve between 12 to 20 participants, although, as indicated by research from Alanazi et al. (2021) and Webler et al. (2009), some investigations might require even fewer participants.

Data Collection

Data collection will primarily consist of two phases:

(1) Development of the Q-set: The inaugural stage involves meticulously assembling a Q-set, a collection of statements designed to capture a spectrum of opinions on the utilization of Reverse Jigsaw and Edpuzzle within doctoral business education. This assemblage will be the result of a rigorous review of pertinent literature, valuable inputs from domain experts, and rich insights gleaned from interviews with both educators and students. Post-course, the instructor will conduct comprehensive semi-structured interviews with the students involved in the Q-study (see Appendix A). These interviews are designed to delve into the depths of the students’ experiences and perspectives regarding the implemented teaching methods. A critical aspect of this phase is the use of content analysis to distill 30 essential statements that succinctly capture the quintessence of the teaching approach. Special focus will be placed on aspects such as the integration of technology, strategies for content delivery, and the synergistic perspectives of educators and students. The ultimate aim is to compile a Q-set that vividly mirrors the diverse and complex viewpoints of the participants, effectively encompassing their perceived advantages, challenges, and the overall impact of these educational methodologies (see Figure 4).

Figure 4.

Steps of Q-Methodology.

(2) Q-sorting: During the Q-sorting phase, students will be presented with these 30 meticulously chosen statements. They will face the task of methodically sorting these statements into three distinct categories: those that notably enhance their learning experience, those with lesser impact, and those deemed neutral. The students will then be required to assign a rating to each statement, spanning from −4 (indicating a strong disagreement) to +4 (signifying a strong agreement). This systematic and balanced rating procedure is engineered to culminate in a detailed Q-sort, as demonstrated in Figure 5. This Q-sort is strategically designed to facilitate a comprehensive and nuanced evaluation of the 30 statements, thereby providing deep insights into the students’ perceptions and critical assessments of the teaching methodologies deployed.

Figure 5.

Example of a Q-Sorting Distribution Framework.

Data Analysis

The Q-sorts will be analyzed using Q Method Software (https://qmethodsoftware.com/), a web-based application specifically designed for Q-methodology studies. Q Method Software operates entirely within the user’s browser, ensuring that all data processing occurs locally to enhance data security and privacy. This platform facilitates comprehensive Q-sort analyses, including factor extraction and rotation, and provides interactive visualizations to aid in the interpretation of results. It will categorize factors, compute their scores, and reveal their interrelationships, extending beyond Likert’s methodology by emphasizing the analysis of interconnected statements. Advanced statistical techniques and factor rotation methods will be applied to ensure accuracy. The critical factors will be identified through a combination of correlation, centroid factor analysis, and manual rotation, providing a detailed and precise evaluation. The collected Q-sorts will be analyzed using Q-methodology, which involves several steps (Alanazi et al., 2021; Brown, 1996):

(1) Data reduction: The Q-sorts will be factor-analyzed to identify patterns of the participants’ shared viewpoints. This factor analysis will generate factors or clusters that represent distinctive viewpoints.

(2) Factor interpretation: The factors obtained from the analysis will be interpreted and labeled to represent coherent and meaningful perspectives.

(3) Factor comparison: The identified factors will be compared and contrasted in order to understand the differences and commonalities among the viewpoints.

(4) Integration of qualitative insights: The qualitative data collected using open-ended questions or follow-up interviews may provide context to enrich the understanding of the identified perspectives.

Validity

This planned Q-study will focus on assessing different types of validity, including content and face validity. The task of improving the content validity will involve two researchers conducting a comprehensive review of the statements. The feedback received during the Q-sorting phase will be crucial in determining the face validity, with expectations of insightful responses. To ensure the clarity and usability of the Q-sorting process, a pilot study will be conducted with three students. This combination of naturalistic and theoretical approaches is expected to enhance the range of the Q-set.

Ethical Considerations

Ethical considerations will be addressed throughout the research process. Informed consent will be obtained from all the participants to ensure that their participation is voluntary and assure them of the confidentiality of their responses. Any potential conflicts of interest will be disclosed at this stage. Additionally, data will be anonymized and securely stored to protect participants’ privacy.

Discussion

The preliminary findings from this study provide early insights into the use of innovative pedagogical approaches in doctoral business education, aligning with the Association to Advance Collegiate Schools of Business (AACSB) accreditation standards. Using Q-methodology, the study explores diverse perspectives among doctoral students, contributing to an initial understanding of how these approaches may support advanced educational practices. These findings appear consistent with the College of Management’s objectives, which emphasize critical thinking, interdisciplinary expertise, and ethical awareness.

The reverse jigsaw method has shown potential in fostering peer-to-peer learning. By leveraging peer-led instructional models, this method aligns with cooperative learning theory, which suggests that peer teaching may enhance retention and support the development of higher-order thinking skills. This approach could address the complexities of doctoral education by encouraging collaboration and adaptability. The integration of technological tools such as Edpuzzle has also supported these initial outcomes. Features including Teacher Assist AI, a question generator, and autograding capabilities for multiple-choice, true/false, and short essay writing assessments may streamline grading, reduce instructor workload, and facilitate timely feedback. Additionally, Edpuzzle’s voice-over feature enables educators to provide targeted guidance, potentially reinforcing critical concepts. The use of signaling within videos may further focus attention on essential content, potentially reducing cognitive load and supporting comprehension (Brame, 2016).

Preliminary data from Edpuzzle’s learning analytics, including metrics such as video completion rates, re-watch frequency, and quiz performance, suggest actionable insights into student engagement and learning behaviors. These analytics could help identify areas where students face challenges, allowing for targeted adjustments to instructional strategies. Consistent with previous research by Kurzweil et al. (2020), shorter video modules (6–12 min) may sustain attention and improve retention. Segmenting longer lectures into smaller, interactive units with embedded quizzes and voice-over explanations appears to create a more engaging and potentially effective learning experience.

One promising aspect of these findings is the application of learning analytics in doctoral education, an area that remains underexplored. Early indications suggest that engagement metrics, such as viewing patterns and interaction data, could support the development of adaptive and personalized teaching strategies, addressing gaps in traditional pedagogical practices. However, these findings should be interpreted cautiously due to certain limitations inherent in the study’s preliminary nature. The purposive sampling approach may limit the generalizability of results, and self-reported data introduces the potential for response bias. Furthermore, the study’s design does not yet capture the long-term effects of these approaches on learning outcomes. These limitations highlight the need for further investigation as the research progresses.

In summary, these preliminary findings suggest that participatory learning models, when combined with technology-enhanced tools, may offer meaningful contributions to doctoral business education. Consistent with the observations of Arkün-Kocadere and Çağlar Özhan (2024), the findings indicate that while human instructors may foster greater engagement, academic performance does not appear significantly affected, suggesting the potential role of AI-generated assistants in supplementing instruction. Future research will aim to validate these preliminary findings, further explore their broader applicability, and assess their long-term implications for higher education.