Abstract
Lecture-free activities afford students with an engaging approach to knowledge acquisition and integration. When peppered throughout a course, experiential activities inject spontaneity, break up familiar patterns, and empower students to take responsibility of their learning. For an instructor, iteration is required to develop effective lecture-free engineering activities, necessitating thoughtful evaluation. The paper adopts Kelly's personal constructs theory, using repertory grid analysis to consider the effectiveness of six unordinary, lecture-free activities. Through a structured comparison of activities, 29 constructs were elicited with inherently subjective, dichotomous poles. The grid was populated ranking each activity between the poles of each construct such as directed learning or creative expression. Using a cluster analysis and descriptive statistics, various themes emerged revealing the author's preference, and connections between seemingly unrelated constructs such as how summative actives use the entire building whereas formative activities are in the classroom. Recommendations are made to generalize the tool to aid instructors in activity evaluation and development through understanding and challenging existing patterns.
Introduction
When engineering pedagogy is an instructor's research focus, the classroom is transformed into a scientific laboratory. Some experimental techniques result in better student performance and an enjoyable experience for both students and the instructor. However, some experiments are not as effective and may necessitate repetition of learning objectives through more traditional methods, such as a lecture. One limitation of transforming the classroom into a laboratory is the time between iterations of the experiment, as the activity often cannot be repeated until the next year with different students. Careful evaluation is a necessary element of the experimental process to document adjustments for the following iteration.
While teaching is primarily delivered through lectures in engineering education, 1 student attitudes towards learning improve where there is an element of active learning. 2 By moving away from lecture-centric activities, instructors empower students to take ownership of their learning. 3 Successful teaching requires innovative lessons, experiential activities, robust assessment, and responsive feedback.3,4 Rather than supplementing lectures with experiential activities, instructors can design entirely lecture-free activities to maximize the benefits of active learning. Peppering lecture-free activities into the curriculum can revitalize a course, break up the familiar classroom patterns, and maintain student engagement at the course level, especially for dry topics or at stressful times of the semester.
There are two important steps in the learning process: acquiring new knowledge and integrating it with previous knowledge. 3 Instructors have greater control over the first task of knowledge acquisition, selecting whether to use a lecture or active learning. The second task is more dependent upon the learner, though the instructor can encourage integration of knowledge through engaging activities, formative assessments, and feedback. Lecture-free activities emphasize integration of knowledge and require students to transition from recipients of lectures to active participants.
Just as student work is assessed, instructor planning and implementation should be evaluated, especially when novel elements are employed. End-of-semester student evaluations provide course-level feedback, but are not specific to individual activities, nor are they reliable measures as they are influenced by the perceived complexity of the course, maturity of the students, number of times the instructor taught the course, and other problematic factors. 5 Student grades are commonly used to evaluate the successful transmission of learning outcomes,6–9 but additional measures are needed to assess engagement or preference. Surveys elicit student preference.6–8,10–12 Interviews, focus groups, and student reflections provide richer data but are more time-consuming.6–8,13 Direct measures are recorded during activities, such as number of interactions or mistakes, timing, and direct observation.6–8,12 Additionally, meaningful data can be elicited from instructor reflections through rubrics and checklists.6,8,14,15
One form of evaluation is the use of a repertory grid, which incorporates the personal constructs theory put forward by Kelly. 16 Stemming from psychology, personal construct analysis provides a method to analyze patterns to describe how individuals make meaning of the world. Individuals constantly evolve the way they construe the world, and Kelly defined constructs as the temporal patterns that are formed. 16 To better understand a person's constructs, repertory grids were developed that use rating scales derived from the constructs. 17 The repertory grid is especially appropriate to assess lecture-free activities, as the grid provides a way to assess qualitative data through quantitative means. 18
In order to evaluate specific lecture-free activities, a repertory grid was developed from auto-constructed accounts of six previous classroom ‘experiments’. The resulting tool can be used to evaluate future activities. This paper discusses (1) the foundation for personal construct analysis, (2) the development of the repertory grid evaluation tool, (3) connections to the literature for themes that emerge from the constructs, (4) application of the tool on six nontraditional, lecture-free activities, and (5) an analysis of the resulting grid.
Part 1: Methodological foundation
Personal construct analysis through the use of repertory grids has been used in education research in myriad ways, such as eliciting students’ views on a topic, assessing an instructor, evaluating a program, and documenting an instructor's beliefs.17,19,20 According to Bell, “education, in Kelly's view, is necessarily experimental. Its ultimate goal is individual fulfilment and the maximizing of individual potential, capitalizing on the need of each individual to question and explore.” 21 The personal and exploratory nature of personal construct analysis allows teachers to better understand their interactions in the classroom. 20
Evidence of repertory grid technique could not be found in engineering education literature. Case and Light list a number of emerging qualitative methods such as grounded theory, ethnography, and phenomenology, but exclude personal construct analysis. 22 Moore et al. used constructivism methodologies that are built upon Kelly's personal construct research, stating “beliefs are personal constructs that … guide instructional decisions and provide a lens through which to understand classroom events.” 4 This could be perceived as a distant cousin of repertory grids.
Hardison and Neimeyer explain that according to personal construct theory, “humans literally construct the meaning of their own lives, by devising, testing, and continuously revising personal theories that help us make sense of the world around us and anticipate our future experiences.” 23 Personal constructs, then can be used to predict behavior. Yorke warns that the conclusions from repertory grids in education research can be distorted. 24 Keen describes the poor test-retest reliability of one instrument that used repertory grids for teaching appraisals across a large population. 19 Given the subjective, evolving nature of an individual's beliefs, repertory grids are not appropriate measures to generalize large populations or predict behavior. Alternative techniques for personal construct analysis include laddering, 25 pyramiding, 26 and self-characterization. 16 An interviewer is required for these co-constructed analyses, so these techniques are not employed in this study.
Similar to using a rubric, repertory grids are numeric measures of qualitative and quantitative characteristics of a particular activity. Rubrics are philosophically objective with an inherent aspect of “right” and “wrong,” and performance is evaluated by level of achievement for each criteria. 11 Unlike rubrics, repertory grids leave room for the subjective nature of the content, such as the individualistic approach to teaching. What is desirable for one person is not necessarily the desire of a second individual. What is desirable one course does not dictate the preferences for the next course. 23 Similarly, when comparing lecture-free activities, there is not one desired set of attributes for an activity. A short activity may be preferable one day whereas a longer activity may be preferred the next day. Repertory grids are inherently, intentionally subjective.
According to Yorke, the three phases of repertory grids are “elicitation of elements and constructs, location of elements on construct dimensions and the method of analysis of the grid.” 24 These phases correspond to parts 2, 4, and 5 of this paper.
Part 2: Development of evaluation tool
In the first phase, the elements and constructs of the grid were defined. The elements for this study are the six lecture-free activities that were previously deployed in the classroom: (1) Comet Caper role-playing on history of engineering, (2) Choose-your-own-adventure strength-of-materials activity, (3) dynamics lecture replacement, (4) scavenger hunt problem development for engineering problem analysis, (5) sustainability role-playing fishbowl, and (6) storybook as introduction to breadboards. Each activity is described in part 4.
From these elements, the constructs were elicited, which are the criteria to evaluate the elements. The constructs should be dichotomous poles such as “approachable” and “aloof” to describe an instructor's demeanor and phrased so there is not one preferable pole.17,20,24 Though originally performed with an interviewer in the counselling setting, auto-constructed grids are found in the educational context.17,18
Kelly recommends eliciting the constructs from the elements by comparing and contrasting three randomly selected elements and repeating the exercise until no new constructs can be derived. 16 Construct definition can be simplified wherein subjects consider only one element at a time instead of three. 20 Studies with large samples use constructs defined by the researchers to ensure homogeneity of the instrument. 21 Since this is an auto-constructed study, simplification of construct definition and homogeneity of the instrument are not required. The traditional method to elicit constructs was employed for this study, wherein three elements were selected at random and compared. This was repeated six times until no additional constructs could be added. The 44 constructs were organized into three temporal categories: pre (set-up), mid (process-oriented), and post (outcomes-oriented). The reorganized constructs were reviewed for duplicates, revised to prevent one preferable pole, and reduced to the 29 two-poled constructs shown in Table 1.
Two-poled constructs.
The 11 constructs relating to the set-up focus on the room configuration, participant characteristics, activity duration, material requirements, and preparation. The nine process-oriented constructs consider the experience for the students and format of the activity, such as pace and noise level. The nine outcome-oriented constructs focus on the desired learning objectives, such as acquisition or integration of knowledge and the long-term impact, which is anecdotally evaluated from unsolicited feedback from former students. One construct considers whether the overall value of the activity is gained during the experience itself or from the information gained from the activity; this is perhaps a more philosophical analysis as education is often more focused on the learning outcomes rather than the experience itself.
The next section connects existing educational evaluation in the literature to the constructs that were elicited for the repertory grid. Though external validation is not required to establish the overall validity of the constructs, as this form of qualitative research is inherently interpretive and subjective (rather than positivist and objective), there is merit in discovering commonality.
Part 3: Connections to literature
The constructs produced during the first phase of grid analysis overlap with constructs developed by a similar personal construct analysis completed by Jankowicz. 17 Using a five-point scale, nine teaching methods were compared using 10 constructs. Student enjoyment, preparation, type of learning, material cost, amount of preparation, and retention of topic are part of both Jankowicz’ grid as well as the grid in this study. Due to a lack of variance for the elements in this study, four of Jankowicz's constructs are not deemed relevant: (1) number of people needed to prepare the activity, (2) instructor enjoyment, (3) necessity for emotional connection of students, and (4) whether student preparation is required.
Lucas and Hanson identified habits that enhance problem-solving ability, encouraging the use of creativity and adaptation. 27 Four constructs in this study address the problem-solving habits, three of which are process-oriented and one is outcome-oriented. The applicable process-oriented constructs are whether the activity: (1) is directed learning or allows creative expression, (2) is playful and nostalgic or serious and studious, and (3) has a specific goal or is open-ended. The outcome-oriented construct is whether the activity encourages artistic creativity or is technical/task focused. While this construct is similar to the directed learning/creative expression process-oriented construct, the outcome-oriented construct assesses how much flexibility there is in expressing the answer. The process-oriented construct considers the number of ways to get a conclusion, whether there is a specific path or flexible path. These four constructs consider how adaptable and creative students are encouraged to be throughout the activity.
Personalization, participation, and productivity are ways to evaluate an activity. 28 The constructs that assess personalization are whether the activity: (1) involves independent work or class-level teamwork, (2) allows passive participation or all students are active, (3) is relaxed/self-paced or pressure/timed, and (4) has a broad audience (is translatable) or is targeted to an engineering audience. The participation is evaluated through the size of the group (small/15 people or large/50 people) and whether the activity requires a center-stage or parallel discovery. Additionally, the location of the activity can allow for additional participation, whether the activity is location-dependent or can be in a flexible setting and whether the activity uses the entire building or a classroom setting. Students engage differently with an activity that takes place outside the classroom than one that stays in the classroom. Finally, the productivity is evaluated in whether the activity is memorable or has a one-day impact, the length of the activity (60 min or multiple classes), and the type of learning outcome: technical or professional, summative or formative, and integrative or develops new knowledge.
In an evaluation of a physical applet, Pejuan et al. use a rubric to consider the technical aspects, design, quality of message, applicability to the course, and underlying learning objectives. 29 The technical aspects and design are collectively considered in the pre- and mid-activity constructs. The learning objectives are specified in the post-activity constructs. The quality of message appears in the construct on whether the activity is fully developed or needs refinement. Ideally, all activities are applicable to the course so this aspect of the rubric is not reflected in the constructs.
Activities that are designed to replace a lecture must maintain a high level of student engagement and replace the voice of the instructor. 3 In order to do this, increased time and engagement is required of the instructor. 3 Additional instructor effort is represented in the pre-activity constructs such as material, preparation, and classroom arrangement. During the activity, additional instructor effort is represented in the construct on whether a facilitator is required or if the activity is self-guided. Constructs that assess whether there is a high level of student engagement include whether the activity is goal-oriented or open-ended, quiet/ordered or loud/chaotic, nostalgic/playful or studious/serious, and whether participants can be passive or must all be active. The construct on whether the activity is unordinary or traditional/familiar evaluates the overall novelty. While some students may appreciate an unordinary activity, it may make other students uncomfortable to participate in a non-traditional activity, so there is no one preferred pole in the construct.
The process of learning requires students to understand the material and apply it to new situations. 30 This is reflected in the post-activity construct on integration compared to development of new knowledge. Activities that replace lectures should accommodate a variety of student abilities, 30 to simultaneously keep advanced students engaged and encourage students with a lower initial ability to proceed. This versatility is reflected in the mid-activity constructs on whether the activity is (1) relaxed/self-paced or pressure/timed, (2) independent work or class-level teamwork, and (3) has a specific goal or is open-ended.
The construct on whether the activity is future-focused or present-focused relates to a concept introduced by Dewey over 100 years ago. Engaging students with real-world situations helps them to absorb technical knowledge. 31 Simulation games allow students to solve real problems with little risk, 10 and envision their future role as an engineer. Games develop practical skills and require mental and physical involvement, requiring players to make decisions and integrate learning across domains. 32 This connects to the construct on whether there is integration of knowledge or new knowledge is required. Additionally, the value of simulation appears in the outcome-based construct on whether the activity is success-oriented/knowledge transmission or process-oriented/experience-centric. Engineering education relies on displayed evidence of learning, which is best achieved through formative and summative assessments. Lecture-free activities and simulations offer valuable, often unnamed learning objectives that result from the process. Some activities are about the journey, not the destination. This is reflected in the success- versus process-oriented construct.
Finally, accreditation requirements necessitate the development of students’ interpersonal communication and teamwork skills, as well as helping students understand the ethical, environmental, safety, and professional responsibilities of an engineer. 33 The outcome-centric constructs that address these concepts are whether the activity: (1) is problem analysis-centric or conversation-driven, (2) develops primarily professional skills or technical knowledge, (3) is future-focused (role of engineering in society) or present-focused (student-centric), and (4) is completed independently or through class-level teamwork.
Collectively, there is an established foundation in the literature for the 29 elicited constructs. The next section describes the six lecture-free activities used as elements for the next phase of repertory grid analysis.
Part 4: Populating lecture-free elements on constructs
In the second phase of repertory grid analysis, each element is given a numerical score out of five for each construct with ‘1’ representing the left pole and ‘5’ representing the right pole. 24 Table 2 summarizes the six elements with four defining constructs.
Six lecture-free activities/elements.
Comet Caper
The Comet Caper is a role-playing activity similar to a murder mystery wherein students have specific roles and are given information prompts throughout the activity, discovering information in real-time. The real-world experience of engineering is hard to convey in the classroom, such as the joy and frustration of working on a team and the adrenaline rush when a problem appears with the deadline looming. Inspired by the European Space Agency's Rosetta satellite that landed on a comet in 2014, the activity requires students to select one of 10 roles for the historic event, such as Program Manager, Systems Engineer, Media, Scientist, etc. Each role is given a different personality, mood, responsibility, information during the activity, and lines of communication as shown for the Technical Program Manager:
Comet Caper character motivation.
Using slips of paper with prompts as shown in below in-line figure, students are guided through the event as it actually happened: the satellite crashed into the comet and the European Space Agency tried to diagnose what happened and how to optimize the rest of the mission. The students run the class, making decisions on how to operate the satellite and what steps are needed for troubleshooting the problem. After 90 min of intense decision making, discussion, and teamwork, including a few dramatic characters to instill passion and comedy, the activity is debriefed and students are given time for reflection. In their reflections, students discussed the stress of real-life engineering, the reliance on team members, how to deal with conflict, how political agencies operate together, and how a person's emotional state can affect the mission.
Sample page from Breadboardia.
Repertory grid.
Participants are typically first- and second-year design students, but have also included engineering educators at workshops in Canada and India. Adults in non-technical fields and high school students can engage with the activity. While 15 is the ideal number for active participation, the activity can be scaled up for a larger class where roles have multiple members, which allows passive participation for more introverted students. A specific room setup and printed materials are required. Also, a facilitator must be attentive during the activity to advance the times, distribute the prompts, encourage quieter students to share their information, and guide the debrief discussion. Casual white-board exit polling indicates that 85% of the 200 participants gave the activity highest marks, with only 2% of participants requesting for the activity not to be repeated.
Choose your own adventure
To summarize introductory strength of materials concepts for a first-year analysis course, an open-ended activity was developed using QR codes in 2015. Emulating the decision-making in the Choose Your Own Adventure book series by R. A. Montgomery, students worked in teams to solve problems. 34 Based on their calculations, students scanned a QR code to go to a room to find the next problem. Replicating the book series, some rooms were dead ends, and students had to repeat the previous step to correct their mistake.
This summative activity utilized the whole building and QR codes were created from room numbers. It is highly interactive and encourages teamwork through problem-solving as the small teams move throughout the building. It would be time-consuming to adapt to a different building or course, and the activity requires advance set-up and printed materials. This activity was performed once (corresponding to the number of times the instructor taught the course).
Dynamics discovery labs
The dynamics discovery labs were offered for second-year students in 2013 to 2014. 35 Students were introduced to momentum and energy concepts in a self-guided model through hands-on examples and structured problem-solving. Using theory and numeric problem-solving, students manipulated objects and checked answers at different stations before proceeding to a more complex module. By the end of the lab period, students completed five problems and were exposed to content from three lectures through self-discovery.
This formative activity requires reusable materials, printed handouts, and a specific room set-up. It is better with a smaller class so that stations are not crowded, but multiple stations can be created to accommodate large classes. Students struggle with this activity as they are used to having the knowledge conveyed through lectures, so a follow-up activity may be necessary to make students more comfortable. It was found that high-performing students (course grade of 85%–100%) preferred a lecture whereas lower performing students (50%–84%) preferred the activity (P < 0.05). 35 There were no statistically significant differences in performance between students who used a lecture and those use completed the discovery activity. This activity was completed twice at two different universities and will be repeated in future dynamics classes. However, it would be time-consuming to adapt to a different course.
Scavenger hunt
This activity requires students in first-year engineering analysis courses to summarize their learning through a scavenger hunt of their own making. Students are given a list of topics (strength of materials, energy, equilibrium equations, and inertia) and must develop their own problems from four different sources (outside, inside, part of the human body, and from a movie). The students then solve the problems they proposed. Students are given multiple classes to find the problems and work on the solutions, making this the only activity of the six that is not completed during one class period.
A graduating student shared that this was their favorite assignment while in engineering, as it helped them to realize how engineering connects to the world. This assignment is time-consuming to grade as each student provides four individual problems, but it can be adapted to any course at any level. Additionally, student retention of material is improved, incorporating both technical content and professional skills to see the role of engineering in the world, model the situation in a problem, and solve it. This activity was repeated many times from 2013 to 2020.
Sustainability fishbowl
In first-year design courses from 2013 to 2015, students engaged in a fishbowl debate about the most sustainable way to transport goods. 36 Initially, small groups discuss the benefits and drawbacks of the three proposed forms of transportation: trains, planes, or trucks. Then, students join larger groups representing the three transportation industries, environmental activists, government agencies, and consumers.
Each larger group determines their position, formulates arguments for their position and against other groups’ positions, and anticipates counter-points. A fishbowl is then held where one person from each group sits in a circle surrounded by the rest of the students to determine which mode of transportation is the most sustainable for transporting goods. In a fishbowl, students can ‘tap in’ to replace the representative, so eventually many students become involved in the debate.
This formative activity helps students to develop critical thinking, especially to anticipate counter-points. They work in teams, research sustainability concepts, and complete a reflection to integrate their knowledge. The fishbowl format allows for many students to be involved and have a large class discussion (without a lecture), while integrating quieter students in the small groups before the fishbowl begins. The outcome is open-ended and little facilitation is needed. Students can incorporate their creativity and simulate the real world. The activity could be completed in any class, with non-technical participants, in about 90 min.
Breadboardia storybook
To introduce students to breadboards and basic electronics concepts, a storybook was developed with a fictional narrative on the right-hand pages and a technical description on the left-hand pages. 37 Figure 3 shows a sample page from the book.
Cluster analysis.
Instead of a didactic lecture, students read the storybook and complete an electronic circuit to light three LEDs. With illustrated pages and step-by-step instructions, students learn new knowledge at a self-guided pace. Additional concepts such as determination, troubleshooting, and civic responsibility are included in the narrative. It was found that students who use the storybook completed the activity significantly faster than students who received a lecture (P < 0.05). 37
The activity requires electronic components and copies of the storybook. Some set-up is required to load the Arduinos and prepare the kits. The activity is appropriate for any class size and welcomes non-technical students, especially those unfamiliar with electronics. The activity was completed a few times from 2020 to 2021 with students from sixth grade through first-year engineering.
Populating the repertory grid
The six elements were assigned numerical scores between 1 and 5 for each construct. A score of 3 implies an equal balance between the poles. Figure 2 shows the populated repertory grid created with WebGrid online software. 38
Part 5: Analysis of the grid
The third phase of repertory grid analysis uses quantitative techniques to analyze the results. Common techniques include analyzing the descriptive data, 39 factor analysis, 40 principal component analysis, 41 and cluster analysis. 42 Given the number of elements and constructs, the analyses performed for this study consist of a comparison of descriptive statistics using JASP software (Table 3) and a cluster analysis using GridSuite software (Table 4 and Figure 3).43,44 Factor analysis is not appropriate as a minimum of 100 elements are required for analysis. 45 Table 3 shows the descriptive statistics for the constructs and elements. All elements and constructs meet assumptions of normality using the z-score for skewness and kurtosis. 46 The mean scores for elements range from 2.38 to 3.48, indicating diversity among elements and balance between the poles. The choose your own adventure is the most balanced with a mean score of 3.0. The dynamics discovery labs and Breadboardia are the least alike with the lowest and highest mean scores, respectively.
Descriptive statistics.
*Indicates the maximum or minimum pole was not selected.
Standard error for elements of 0.434 for skewness and 0.845 for kurtosis.
Standard error for constructs of 0.845 for skewness and 1.741 for kurtosis.
Cluster statistics.
For the constructs, most of the activities accommodate large groups (mean = 4.33) and are playful or nostalgic as opposed to studious or serious (mean = 1.83). The construct for the location of the activity, whether in a classroom or the whole building has the largest standard deviation (2.066) and is purely dichotomous with no 2, 3, or 4 entries. Similarly, whether the audience is broad or targeted has a large standard of deviation (2.041), but no 3 or 4 entries.
The six constructs with the least range are denoted with a ‘*’ in Table 3. Varying from 2 to 5 or 1 to 4, one of the poles for that construct was not selected, indicating that the activities are fully developed, appropriate for various group sizes including large groups, nostalgic (playful), memorable, and unordinary. A preferred pole reveals the author's conscious or unconscious preference when designing lecture-free activities, and documents the potential bias that can exist in auto-constructed research. However, preference towards the unordinary pole as opposed to the traditional pole provides one level of validation, as the elements are all lecture-free activities. It would be problematic if the mean were closer to the traditional pole.
A cluster analysis determines commonality between subgroupings of the elements and constructs. 45 GridSuite software was used to produce the graphical representation in Figure 3 and numerical results in Table 4, using the single linkage hierarchical cluster method (also called nearest neighbor), which helps to identify outliers. 44
There are two clusters among the elements and three clusters among the constructs, indicating commonality. The elements and constructs were reordered, combining the pre-, mid-, and post-construct activities. The poles of 10 constructs were reversed to find the minimum distance between data points, and are denoted with symbol to the left of the text in Figure 3.
The sustainability fishbowl and Comet Caper elements have the most commonality (76% per Table 4) and are clustered together (with an orange bracket at the top of Figure 3). Both activities are conversation based, require the entire class to work together, and cover professional skills. The remaining four elements in the second cluster (blue brackets) have technical outcomes and are subdivided into summative (choose your own adventure and scavenger hunt) and formative (Breadboardia and dynamics discovery lab) activities. The formative activities have the largest disparity in the means (Table 3), and yet still have 60% matching (Table 4).
As seen along the right side of Figure 3, there are three main clusters of constructs and an outlier (the top construct). A low matching score (shown in Table 4) indicates the division between clusters. The outlying construct indicates that the amount of research incorporated in an activity is unrelated to the other constructs, evident in the 42% matching score to the neighboring construct on whether the preparation is required.
The top (orange) cluster in Figure 3 has 21 constructs that could be considered the main body of analysis with two outliers. The amount of set-up needed is the first outlier, which is not surprising as the set-up doesn't obviously correlate with creativity, materials, or other constructs. The second outlier is the playfulness or nostalgia in the activity. This could be because activities such as Breadboardia and choose your own adventure are nostalgic in that they relate to children's storybooks, but they have few other similar characteristics.
A notable result in the first cluster with 100% matching is between the constructs comparing parallel discovery or center stage participation and problem analysis-centric or conversation-driven. This is reasonable as a large group conversation likely involves the entire class. Closely related (92%) are the constructs for class-level teamwork and whether passive participation is possible. Students have the ability to be quiet but engaged in a center-stage conversational style activity or adversely to disengage entirely. Similarly, all students are active in an independent activity that is goal-driven. These constructs don't match entirely because the dynamics discovery lab allows for partners and the sustainability fishbowl has varying levels of teamwork.
Once reversed, the specific goal or open-ended construct matches 83% with whether materials are required. An open-ended activity (such as the sustainability fishbowl) has few material needs, whereas an activity with a specific goal (like Breadboardia) has material requirements. The only pair of constructs in the 83% matching tier without a clear explanation is that a flexible room layout is correlated with integration of knowledge, whereas a specific room configuration is correlated with new knowledge. Perhaps this is because more rigidity is needed to convey new information, and there is more flexibility when the focus is on integration. This is the first unexpected theme that emerged from the analysis and could be explored with the adoption of future activities. A recent pedagogical shift is using classrooms that are specifically designed for active learning with furniture and seating configured for group work.47–49 Providing a low-tech environment emphasizes interactivity, 48 and a thoughtful classroom setup can reduce anxiety associated with active learning. 49 One complaint of active learning is perceived disorganization when reconfiguring into active learning groups, 50 which could be avoided with a permanent setup.
The second (blue) cluster in Figure 3 has two constructs on group size. According to Table 4, they have 83% commonality but display less than 50% commonality to neighboring constructs. The flexibility or rigidity of the group size is related but not identical to whether the activity is appropriate for small or large groups. The lack of correlation between group size and the other constructs is the second interesting theme that emerged from the analysis. Group work is a common frustration that students have with active learning, particularly for introverted students. Strategies to improve group experiences for introverted students include intentional formation (instead of self-selection), size of the group, assigning roles, and developing teamwork skills. 51 The lecture-free activities tend to be shorter, so student resistance to group work is not as prevalent as it is for a multi-semester project, but should be noted. The third (green) cluster contains five constructs, with no obvious thematic commonality. The outlier within this cluster is the construct on whether the activity is memorable, likely because it is the most speculative of all of the constructs. The third theme that emerged from the data, then, is to investigate how memorable each activity is instead of speculating. This would invite students into the evaluation process, which could improve their learning skills and provide valuable feedback to the instructor. 52 The investigation could be extended to examine benefits of active learning, such as long-term retention and engagement, 53 as well as drawbacks such as student resistance and faculty perception. 54 Examples of student resistance include unfamiliarity with fundamental knowledge, struggling with uncertainty, additional effort to actively engage (rather than passively receive), and potential for negative evaluation.55,56 Only by asking students about their experience could a construct on student preference be meaningful. Instructor resistance to active learning is centered on time, preparation, and efficacy. 54 Time and preparation are addressed in the existing constructs, but an additional construct is warranted for efficacy, which could be based on average assignment scores.
To provide confidence in the results, a second form of cluster analysis was performed using the WebGrid online software that employs Shaw's FOCUS algorithm.44,57 Whereas single linkage (employed above) creates chains of clusters, the FOCUS algorithm uses a focusing technique in addition to the distance between elements, so it is not strictly hierarchical. 57 This second clustering analysis produced identical element results and very similar construct results, with one major construct cluster, two smaller clusters, and similar outliers (nostalgic vs. studious, advance setup vs. no preparation).
Conclusion and recommendations
An evaluation of the six lecture-free activities using repertory grid analysis revealed three themes and documents the instructor's preference for activities that are fully developed, appropriate for various group sizes including large groups, nostalgic (playful), memorable, and unordinary. First, activities that present new information tend to have more rigid room requirements whereas integration activities tend to be more flexible. Similarly, formative activities tend to take place in a classroom setting whereas summative activities use the entire building. Second, the two clustered constructs for group size do not have many similarities with the other constructs, suggesting the setup and outcomes of the activity are not as dependent on the number of students involved. Third, an outlying construct on whether the activity is memorable could be investigated with data from students and extended to include preference. Just as professional skill was used in place of soft skill, a revision to the language is needed for the construct on broad/targeted audience, as the purpose of a lecture-free activity is to convert the audience into participants. A more appropriate construct wording would be broad, translatable participant group or targeted engineering participants, though admittedly it sounds awkward. The inclusion of audience in a construct indicates the author's enduring bias towards lecture-centric activities with the instructor at center stage.
As an exploratory, qualitative study, these results are limited to the specific lecture-free activities included in the study for this particular instructor and cannot be generalized to a larger population. However, the constructs that were developed in this study can be used more generally by instructors to evaluate their lecture-free activities and to aid in the ideation process using the emergent patterns. By reversing one of the poles of the correlated pairs of constructs, a new paradigm can be created, such as a formative activity that uses the whole building (as opposed to the correlation between formative activities and the classroom). Challenging the correlations between constructs with the lowest commonality could be especially innovative, for example, to produce a conversation-driven parallel discovery process activity or a center-stage activity that is problem analysis-centric.
Instructors could replicate this method for activity evaluation to discover their own preferences and challenge their patterns. Personal construct analysis could become more generalizable by scouring the literature for different types of experiential activities and performing a quantitative study using factor analysis to see how numerous instructors populate the repertory grid. The results from this type of study could produce guidelines and a validated instrument to help instructors ideate new activities and evaluate existing activities. The constructs could be derived from this study, with additional constructs identified throughout this manuscript to accommodate different university environments and address the shift to online education, such as those shown in Table 5.
Recommended additional constructs.
Evaluation is a necessary part of developing effective experiential activities, especially when they are considered unordinary. The repertory grid analysis provides a novel application of a traditional counselling technique to elicit the instructors’ preferences. While this tool could be used as-is by other instructors, it would be more beneficial for the instructors to develop their own constructs using personal construct analysis to reveal and challenge their preferences and patterns.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the University of Prince Edward Island.
