Team-based learning in large cohorts: Successes and challenges in first year mechanical engineering

Abstract

Maximising engagement, interaction and providing effective feedback for large engineering cohorts is a significant challenge. Team-based learning is a powerful approach that has been shown to be effective in overcoming these issues. However, the use of team-based learning for very large class sizes, particularly in the field of engineering is limited. This study is focused on refining team-based learning for a large (∼350) student first-year mechanical engineering cohort. Team-based learning was trialled and optimised during tutorials over two successive academic semesters with the same group. Quantitative data collection was collected from student attendance and performance, and regular feedback provided qualitative insight. The results indicate that team-based learning enhances engagement, peer-to-peer learning, and exam performance, particularly for the lower quartile (5%–10% mark increase). However careful tailoring of the methodology is required, monitoring group effectiveness is challenging and the use of hybrid team-based learning needs future refinement.

Introduction

Background

First-year undergraduate Mechanical Engineers require a fundamental grounding in key principles which underpin the topics covered in subsequent years. The exact arrangement of this teaching varies by institution; this study is focused on a UK University in which teaching is split into 2 semesters. Amongst other practical elements and units, at this institution students study solid mechanics 1 (SM1) and thermodynamics (T) in semester 1 (October–February) and fluid mechanics (FM) and solid mechanics 2 (SM2) in semester 2 (February–May). These courses are foundational and are required in the first year of either 3 (BEng) or 4 (MEng) degree programme. The cohort size being considered for this study is relatively large in comparison with the average in the UK, with nominally ∼350 students in 2020/2021. It should be noted that all of these students are registered on the same foundational course, and therefore their membership is consistent across all 4 units being compared. The only minor change in the cohort is that a small number of students (less than 10) discontinued their studies during the trial period.

Historically the four courses outlined above are generally considered to be the most difficult of the course, with a perceived lack of interaction despite the provision of a weekly 1-h tutorial hosted by a lead academic, which historically saw a progressive drop in attendance with only ∼1 in 8 students present after the first 3 weeks. This lack of engagement had a noticeable impact on performance, particularly for the academically lower-achieving students who were less likely to ask questions.

This study took place in the first full academic year since the onset of the coronavirus disease 2019 (COVID-19) pandemic. The associated lockdowns and restrictions on in-person interactions imposed by the UK government restricted the style and nature of teaching. The lecture format was set at an institutional level to be 2 h of live online interactive learning and 1 h of in person teaching each week. Pre-recorded material in the form of short communications were also provided via the University's Virtual Learning Environment (VLE). This shift to a blended learning environment was associated with significant challenges, but equally provided huge opportunities to experiment with different teaching approaches.

Problem statement

The principal problem statement is: ‘what techniques/methods can be practically employed to increase interaction/engagement for a very large (>300) cohort of students without overwhelming staff?’ The primary focus is on addressing the gap in student attainment and increasing the support and feedback available for lower-performing students.

Literature review

A review of the literature was used to identify and critique potential methods that could be used to address the problem statement, and thereby solidify the hypothesis for this study.

Engagement in large cohorts

Increasing the degree of engagement with large cohorts has shown to offer substantial benefits to the teaching experience and outcomes for students. There is a strong correlation between engagement and attendance at lectures/tutorials, ¹ and engagement is known to enhance performance in large cohorts. ² Increased engagement leads to improved understanding ³ and student feedback, ⁴ as well as an increase in results both in formative ⁵ and summative assessment. ⁶ Importantly these effects are known to be most beneficial for underrepresented groups, ⁷ including Black and Minority Ethnic students. ⁸

Whilst these studies have shown the benefits of increased engagement, the practical implementation of techniques for achieving this in large cohorts can take many forms. For example, problem-based learning has been shown to be particularly powerful when it is based on collaborative activities ⁹ or when the approach is tailored to the needs of a specific group. ¹⁰ Effective communication is also crucial in facilitating this type of interaction, for example through the use of discussion boards ¹¹ or anonymous question channels. ¹² The use of a blended learning environment has also been shown to facilitate enhanced engagement in large cohorts through adaptive action learning ¹³ or asynchronous discussion. ¹⁴ Blended learning has proved effective for a wide range of subjects, including engineering,^15,16 with flexibility being a key benefit.

Advances in electronic teaching aids have also revolutionised the ability of academics to provide enhanced interaction. For example, E-learning platforms have been shown to be particularly effective when structured carefully and in line with learning outcomes, ¹⁷ albeit with the caveat that these can lead to other problems when used incorrectly. ¹⁸ Equally, online teaching methods of the flipped classroom ¹⁹ or fully virtual courses can provide significant benefits. ²⁰ Technology is also providing new methods for providing formative and summative feedback, ²¹ which is known to be particularly important for first-year undergraduates. ²²

One particularly powerful basis for effective interaction in large groups has been shown to be peer-to-peer learning methods,^23,24 which both strong and weak students have been shown to find equally useful. ²⁵ However, these methods are known to be challenging in terms of assembling and motivating teams, ²⁶ particularly in large cohorts. ²⁷ A growing approach to facilitate this type of interaction is known as team-based learning (TBL) ²⁸ which can be easily scaled up for large cohorts ²⁹ and has been shown to improve knowledge acquisition, participation and engagement. ³⁰ The team-based learning approach was therefore selected for further investigation.

Team-based learning

TBL is defined as ‘an active learning and small group instructional strategy that provides students with opportunities to apply conceptual knowledge through a sequence of activities with immediate feedback’. ³¹ Many different approaches to TBL have been developed, however the key elements of the approach are shown in Figure 1. Preliminary working before the class is followed by an Individual Readiness Assurance Test (IRAT) that can take the form of an online quiz or example problem. This is followed by a Team Readiness Assurance Test (TRAT) in which the students work together to answer the same quiz. Typically, many of the students’ misconceptions are resolved in the TRAT through peer-to-peer discussion. Unlike the IRAT, the TRAT is followed by immediate feedback. The TRAT is followed by a clarification session, in which the instructor can focus on concepts the whole class is finding difficult. Teams may also ask ‘burning questions’ ²⁹ and may ‘appeal’ questions. Typically, students will next be provided with an opportunity to apply their understanding to a set of application exercises. Students are accountable to their team, both informally through social interaction and formally through peer evaluation.

Figure 1.

Schematic overview of team-based learning (TBL) from the literature. ³² it should be noted that the use of the word ‘unit’ in this figure refers to distinct sections of the course. These are generally delivered in sequential order, but a degree of parallel teaching may be required depending upon the given course structure. The numbering sequence has no specific meaning in this generalised schematic, apart from distinguishing the different elements of the course.

As shown in the distinct ‘Units’ in Figure 1, this process is typically repeated several times either in series or parallel depending upon the specific requirements of the course. It should be noted that the numbering sequence used in this diagram has no specific meaning in this generalised diagram beyond distinguishing different elements of the course. However, it is common for sequential TBL activities to build on prior stages/elements/units of the course.

TBL has been shown to be a powerful approach for increasing interaction, engagement and feedback.^28,33 It is an effective approach to diversifying formative assessment and strongly promotes peer-to-peer learning. ³⁴ The method also naturally facilitates dynamic teaching, by providing lecturers with regular feedback on comprehension.^35,36 In addition, it is highly suited to the problem-based learning approach that is typically employed in the teaching of engineering.^37,38 There is also evidence that this methodology is helpful in the transition to University ³⁹ with first-year students being particularly receptive to this type of innovative teaching methodology.^40,41

These attributes make TBL particularly appealing for the problem being investigated in this study. However, one of the key driving factors for the selection of this approach was the work of Vasan et al., who showed this methodology was highly effective for large cohorts of a similar size (356) to that being considered in this study (∼350), albeit in a different discipline (anatomy). ⁴² Of particular interest was the realisation that this approach can help address issues of different degree attainment and increase feelings of inclusivity for underrepresented students in large cohorts.^43,44 The key emerging theme from all these studies was that in order to be effective, the TBL approach needs to be tailored effectively to the environment in which teaching will be performed.^31,45,46

Accordingly, following the selection of TBL as a promising new methodology, the question arises as to how best tailor the approach. This blended learning format was imposed in accordance with COVID-19 restrictions, at a point where Engineering education was having to adapt to provide an effective student experience. ⁴⁷ Fortunately, however, the TBL community had considered the shift to online/blended delivery in advance of the pandemic. ⁴⁸ Consequently, significant clarity, development, and guidance for the blended delivery of TBL was already established, with the key aspects being identified as successful orientation, relevant task design, careful assessment selection and the generation of student accountability. ⁴⁹ River et al. ⁵⁰ performed a comprehensive review of this area, which summarised the results of 9 studies with over 2000 student participants. This highlighted the significant potential advantages of the effective use of technology in TBL, but also highlighted several subtleties in the reliable use of these methods. These insights were used as the basis for the design/methods for the first iteration of TBL which is outlined in the ‘Design/methods – first iteration’ and ‘Design/methods – second iteration’ sections and were considered again following reflection and updated for the second iteration in the ‘Discussion’ and ‘Conclusions’ sections.

Design/methods–first iteration

As outlined above, the overarching teaching format for the TBL trial was decided centrally by the University and was, therefore, the first full-scale implementation of a blended learning policy for the institution. This provided substantial opportunities, however given the lack of prior experience in the implementation, it was decided that all assessments would be formative

The start of course (SM1) began with an introduction to the TBL framework (shown schematically in Figure 2), in the form of a pre-recorded video and a summary at the end of each online lecture. This was intended to provide clear guidance on the structure and expectations, which were found to be critical to the success of TBL in previous studies. ⁵¹

Figure 2.

Schematic overview of team-based learning (TBL) approach used in this study.

The format of the IRAT was selected to be an online quiz which was hosted on the University's VLE. Each week, 10–15 quantitative questions on that week's material were provided. Questions increased in difficulty from the start of the quiz, and the total activity was intended to take a total of 1–2 h to complete. A more challenging bonus question was also included, to provide an opportunity for the highest-achieving students to extend their understanding. Answers were not provided as the IRAT questions were subsequently used as the TRAT during the weekly tutorial.

The SM1 tutorial sessions consisted of four 1-h slots with approximately 90 students in each session. A hybrid approach was implemented via a Microsoft Teams call which was hosted by the lead academic. Students attending in-person were spread over 3 to 4 rooms, with a dedicated academic or Graduate Teaching Assistant (GTA) present in each room, as well as a dedicated online representative for remote learners. The nature of COVID-19 restrictions meant that students were allocated a desk for the duration of in-person time, and these desks were arranged in their tutor groups; a collection of 4–6 well-acquainted students who had a common pastoral tutor. For each group a Teams Channel was established to facilitate effective file sharing and calls.

The start of each session involved a 5-min recap of the course material from that week, which was followed by a 20-min TRAT quiz hosted on the University's VLE. Academics and GTAs were available throughout this period to answer questions on course material or offer clarification. The students were subsequently brought back to a central Teams call for the clarification session which reviewed the numerical answers and working required. This was followed by a hybrid question-and-answer session for the remainder of the session.

All tutorials were recorded and released online after the session to all students, and e-vouchers were awarded for the best individual and team performance each week. Additional questions on the material were also provided in the form of a problem sheet.

Results–first iteration

Critical assessment of the TBL approach was performed via quantitative and qualitative methods. It should be noted that week 10 of the semester was impacted by the UK government-mandated phased-exit of students, which was intended to minimise the spread of COVID-19 over the Christmas vacation (Exodus Week). ⁵² This policy substantially impacted course attendance and therefore this will be taken into consideration during data assessment.

Engagement and TBL activity results

Engagement with the course was assessed via the completion rates of the individual and team quizzes, which was compared with the manually recorded in-person attendance for the non-TBL for tutorials in 2019. No other changes were made to the course content, structure or instructors between 2019 and 2020. These results are shown in Figure 3, with an average attendance of 75% for individual quiz, 79% for team quiz and 35% for 2019. Figure 3 shows that the nominal completion rate for the first 2 weeks is similar for all three activities, with a substantial deviation being observed from week 3 onwards. The TBL approach thus retains engagement and interaction at a much higher level than the previously passive question and answer-based tutorial.

Figure 3.

Engagement in tutorials/quizzes for solid mechanics 1(SM1).

A comparison between the individual and team engagement shows that in general students were more likely to submit the TRAT than the IRAT, possibly associated with peer responsibility associated with the group task as previously observed in the literature. ⁵³

Historically, an increase in attendance is observed in tutorials towards the end of semester as students begin their revision preparations, as seen in the 2019 data. In 2020 Exodus Week and the week prior were both influenced by the requirement for students to return home, but attendance remained higher than that seen in 2019. This highlights the fact that despite this disruption, the TBL approach retains a powerful draw.

To determine the impact of the TBL approach on understanding, a comparison between the results of the average IRAT and TRAT scores each week can be used as shown in Figure 4. This plot highlights that there is a degree of scatter in the average score from one week to the next, albeit with a slight reduction in performance in the last 2 weeks. Weeks 9 and 10 are focused on the most challenging course material and coincided with the Exodus Week(s), so it is possible this consistent drop arises from a combination of these two factors.

Figure 4.

Individual and team performance in solid mechanics 1 (SM1) quizzes.

The key conclusion that can be drawn from Figure 4 is that the TRAT performance is significantly higher than the IRAT, with an average score increase from 57% to 74% (two-tailed T test p = 2 × 10⁻⁴). Examination of individual scores showed the lowest quartile improving 32% on average, with the upper quartile showing a 9% improvement. Thus, even the strongest students on average showed an improvement, consistent with the effect previously observed in the literature, where TBL benefits all members of the cohort with lower performing students improving their scores by the largest amount. ⁵⁴

Feedback

Qualitative feedback on the course was collected via a ‘stop-start-continue’ questionnaire (Appendix 1) at the course mid-point and the centrally organised Online Unit Evaluations (OUEs). In total 271 independent responses were received which were composed of 653 comments. The overall student satisfaction rating for the course was 4.3/5 which is typically perceived to be a high score for a cohort of this size and is 0.5 higher than any other unit taught at the same time. This and the following section of the article contains several representative comments from the students which have been highlighted in quotation marks.

One of the primary themes in the feedback received was that students appreciated the clear organisation and structure of the TBL, as well as the regular email reminders. This was crucial in ‘ensuring that I knew what I had to do, and when’ and meant that students ‘could structure [their] diaries effective[ly] and make the most of the opportunity’ of the TBL sessions.

Additionally, the feedback highlighted that students enjoyed TBL with 83% of comments being positive. The students really enjoyed ‘the interactive sessions and being able to discuss problems with [their] group’. This discussion was found to arise naturally and was particularly effective at highlighting issues and stimulating debate. Equally, the students felt that the ‘recap at the start is really helpful at reminding [them] of what [they] are covering that week’. Several students highlighted that the TBL structure was particularly effective and that the sessions were engaging, with several stressing that ‘the tutorials help consolidate [their] understanding and is the best session of the week by far’. The optional / bonus question designed for stronger students was also found to be particularly effective with many comments outlining that this allowed them ‘to extend [their] understanding and stretch’ comprehension.

Despite the many positive comments, several challenges were also highlighted by the cohort. These were primarily associated with the practicalities of the approach, rather than the methodology itself. For example, students consistently raised concerns that ‘interacting with people working from home was difficult’. Timetabling of the sessions also resulted in issues, with requests for ‘more time between lectures and the tutorial’ as well as more ‘time to complete the quiz during the tutorial’. There were also concerns over an inability to ‘ask questions during the in-person tutorials’ due to time limitations. Finally, many students felt that the workload was too high, with several highlighting that ‘the problem sheets were too much work’. This last concern indicates a lack of clarity on the part of the academics, who should have further stressed the optional nature of the problem sheets.

Focus group findings

A focus group of 10 students from the cohort was formed and met at the start, mid-point and end of the semester to clarify the role of the group, to provide an opportunity for live feedback and reflection on the course, respectively. The focus groups for the first and second iteration of this study were selected randomly from the cohort. Although this approach naturally leads to a bias towards those willing to participate, this was deemed to be the most practical route for obtaining a broadly representative insight into the opinions of the broader group.

The general message that was consistently received from this exercise was that SM1 was the hardest unit being studied and that the TBL approach was proving to be crucial in overcoming this challenge. Similarly, the formative nature of the TBL exercise was identified as being vital as the students felt that making it summative would encourage copying and reduce efficacy.

The focus groups also mirrored the key comments collated from the cohort questionnaires; in that they felt ‘the discussion is incredibly helpful for improving understanding’ and that the ‘bonus question is really useful’. Equally, they highlighted the same issues on timing, with several students feeling that the 20-min submission time was not sufficient. The focus group also stressed the challenges of engaging with remote students, with several highlighting that they had resorted to WhatsApp calls to interact with those unable to attend in person (instead of using the ‘official’ Teams Channels provided).

The mid-point focus group also highlighted that there was a low uptake of the online chat for questions. Students felt that attendees were discouraged from asking questions because their name was linked to the questions, and there was therefore a fear of ‘asking a stupid question or embarrassing’ themselves. Following this feedback an anonymous channel for questioning was opened via a separate online platform, however this was not utilised effectively. It is likely that this change was made too late, and a culture of not asking questions online had already been established in the cohort by this stage. It should be noted that this was in direct contrast to in-person questions which were regular and widespread. However, it also highlights the importance of setting expectations and suitable routes for interaction from the course onset.

Peer review

During the second half of the course a peer review of the approach was conducted by Reviewer (R1) who is a co-author on the paper. It should be noted that R1 is not an expert in SM1 but is highly experienced in TBL and is therefore well placed to assess the effectiveness of the methodology being implemented. R1 attended the tutorial in week 6 and was physically present in a room where 4 teams were working via TBL. The key conclusions from this review were:

There was a good balance of in-person and online support, and both were effective.

All the students had completed the IRAT and student interaction was outstanding.

In general students enjoyed the TBL approach, although one student commented that the workload was too high.

Several students commented that they were very happy to engage with the TBL methodology as they felt that it was of benefit to their learning.

In general, this overwhelmingly positive feedback suggested that minimal changes to the course were required at this stage. However, clarification on the optional nature of the problem sheets was provided in the following 4 lectures to address some of the concerns over workload.

Summary of findings from first iteration

In general, the TBL methodology was found to be well-liked by both students and academics. The approach implemented appears to improve scores and engagement and promotes strong peer-to-peer learning in a manner similar to that previously observed in the literature. ²⁴ Anecdotally there was evidence that the students were improving their professional and interpersonal skills via TBL interactions, and the ‘live’ quiz scores were particularly effective in facilitating dynamic teaching; a recap of challenging material could be provided where a lack of understanding was identified. The TBL approach also assisted with the transition to university, with students engaging with each other, and the course, in a more effective manner than observed in previous years.

One of the key messages that was received from this first iteration was that clear and effective organisation is crucial to success, but that this is challenging to achieve with so many students/academics and GTAs. The approach was also found to be particularly demanding upon staff time, as the tutorial was repeated 4 times in accordance with social distancing policy.

Additionally, students perceived the workload of the current implementation to be too high, and interaction between remote and in-person groups was ineffective, as has been seen in similar studies.^55,56 Another subtle issue was the ability to ask anonymous questions which is well-known to cause concerns in large cohort first-year groups. ⁵⁰

Design/methods – second iteration

The second iteration of the TBL approach was trialled with the same cohort of ∼350 students. All teaching in semester 2 took place online in compliance with Government regulations. This meant that the TBL sessions were on Zoom, with students spending most of the session working in break-out rooms in the same tutor groups (4–6 students). The switch to fully online learning eliminated the issues with hybrid teams that were encountered in semester 1, but made it less natural for the GTAs to work their way around groups. Despite the change to online working, the timetable of 4 repeated sessions was maintained (this was in part to enable a smooth transition back to physical teaching if restrictions eased and in part to make the sessions manageable online).

As a result of feedback from semester 1, a handful of other adjustments were made – some course-wide, and some specific to FM. The course-wide adjustment was to reduce the length of the tutorial sessions to 40 min to reduce both staff and student workload. To accommodate this reduction in time, the introduction and conclusion to the session were shortened – with a simple explanation of what the students were going to do at the start and a poll to decide which one question to go over at the end. This gave more time (30 min as opposed to 20 min) for group work than in semester 1 despite the compressed overall session length.

Feedback around student workload led to three changes being made to FM. First, the individual quiz was removed and the IRAT was implemented by asking students to attend the tutorial session having attempted the tutorial sheet. Second, the expectations were clearly signposted at the start of the semester along with a breakdown of the time students should spend on each component of the course. The total time expected was calculated to be 5–6 h per week (including 1–2 h attempting the tutorial sheet).

Initially, students were not given the numerical answers to any of the tutorial sheet questions until after the TRAT. However, part-way through the semester the approach was adjusted slightly, and students were given the answers to the earlier, more straightforward questions. The aim of this change was to give students confidence in their method before they tackled the harder examples.

Results – second iteration

Engagement

Engagement in the TRAT was measured by counting the number of submissions each week. This is shown in Figure 5, where engagement was high for the first three weeks of the semester before gradually dropping off (there was no quiz in week 4 or week 9.) There are several external factors that may have influenced this. First, anecdotally the ongoing lockdown and fully online nature of the course made it harder for students to motivate and organise themselves. Second, there was a large piece of coursework due in week 6 for one of the students’ other units, and many students spent a lot of time in weeks 4 and 5 working on this to the detriment of all their other units. While a small uplift was observed in live session attendance in week 7, this did not translate into a recovery in quiz engagement. Finally, the Easter break fell between weeks 8 and 9, and students are typically more focused on revision than they are on learning new material. In semester 1, all teaching is completed before the Christmas break and so students are less focused on revision during the latter teaching weeks. Taken together, these three factors explain the differences observed in quiz engagement between SM and FM.

Figure 5.

Engagement in the team quizzes for FM compared with weekly LOIL attendance. FM: fluid mechanics; LOIL: live online interactive learning.

To ascertain whether the drop in engagement was specific to the FM unit or a broader, course-wide phenomenon, engagement was compared with a non-TBL unit – SM2. Both courses used a combination of pre-recorded and live interactive teaching sessions, but the tutorial sessions for SM2 did not have a structured TBL element. The data chosen for this comparison was a week-by-week completion rate of the pre-recorded material for each course, which is shown in Figure 6 (data was captured at the end of the semester). The overall trend of dropping engagement is seen in both courses, suggesting that the factors influencing student engagement were wider than just the FM unit. From the engagement data we therefore conclude that TBL does lead to higher levels of engagement throughout the course, when compared to a pure lecture-based course (as previously observed in the literature ⁵⁷ ). However external factors such as coursework deadlines can still lead to a reduction in attendance, despite the use of TBL.

Figure 6.

End-of-semester completion rate of pre-recorded material by week - comparison of TBL (FM) and non-TBL (SM2) courses. TBL: team-based learning; SM2: solid mechanics 2; FM: fluid mechanics

Feedback

Qualitative feedback was gained via a mid-course ‘stop-start-continue’ exercise, and from two focus group meetings, as well as from the university-run OUE at the end of the Semester. Several representative statements from the cohort have been highlighted in quotation marks in this section.

Overall, the feedback from the students was highly positive. The satisfaction score from the OUE for the unit was 4.78/5 (compared with 4.56 for the same course in the previous year), with comments such as ‘I only wish more of my lecturers structured their modules like [this one].’

The biggest issue that was identified in the student feedback was that the quality of tutor group discussion was inconsistent. In many cases, the tutor groups worked effectively, with one student commenting, ‘working on the tutorial sheet and then discussing it with my tutor group is really beneficial’, and another saying, ‘the group quiz worked well allowing us to go through the worksheets together and ask questions to helpers if needed.’ There were, however, inevitably some groups where one person felt ignored – ‘I am not a big fan of group quizzes because I feel like no one in my tutor listens to me’. In other groups, attendance was poor: ‘Maybe a more interactive [session is] not the best when there's only a couple in your tutor group’.

In a virtual session with up to 30 breakout rooms, it is hard to see which groups are working well together and to keep track on lone attendees from one week to the next. Assigning a ‘secretary’ role to one of the GTAs would help to alleviate these issues by reassigning groups that are not performing well together. More broadly, a course-wide approach to strengthening the bonds between tutor group members would help with this aspect.

Other requests from the ‘stop-start-continue’ feedback were to go through more than one question at the end and to release some of the numerical answers in advance. Time was the limiting factor in answering the first request. As discussed above, selected numerical answers were given for later tutorial sheets. Students appreciated this change, with one commenting in the OUE: ‘I found [lecturer]'s method of setting worksheets the best out of any in this year. There were some questions where you had the answer given at the end, this enables you to gain confidence in your method. Then there was also [quiz] questions where you can really test your knowledge’.

The adjustment in workload due to the refinements from semester 1 was also appreciated: ‘Furthermore, unlike SM1, the number of questions wasn't excessive’.

Focus group findings

The aim of the focus group was to gain a deeper insight into the students’ experience of TBL, particularly to compare their experience in semester 2 to that of the students in semester 1. A different set of 8 students were chosen for the Focus Group in semester 2, and the group met twice.

In the first session, students were asked about their experience of undertaking the quizzes in a purely online environment. The students did not feel this had affected engagement, though they had technical problems discussing their mathematical working. The most common workaround used by the students was to share photos of their working via WhatsApp.

The second focus group was only attended by one student, though others made technical suggestions for course improvements via email. The student who attended the second focus group mainly reflected on the fact that he had lost motivation after getting behind due to the major coursework deadline for another unit in week 5. This account of one student being unable to re-engage after being ‘de-railed’ by coursework is an echo of the wider trends observed in Figures 5 and 6 and underlines the importance of course-wide co-ordination of workload as well as the impact of student organisation on course engagement.

Peer review

As in semester 1, a peer review was conducted by R1, who attended a TBL session part-way through the course. R1 is not a specialist in the subject matter covered in the FM course. R1 observed one team, and noted the following points:

There was some general chit-chat before they started the questions – this was good team bonding, especially in the context of being locked down away from the university.

All (or most) students had their cameras on and all students were contributing their thoughts as they went through the questions. Engagement was excellent overall.

The conversation with the GTA part-way through the session was useful to correct a gap in the students’ understanding.

As noted above, the experience was not uniform across all tutor groups, and the interaction with the GTAs was less natural than in a physical classroom, but the observer saw an example of the TBL approach working well.

Summary of findings from second iteration

The aim in semester 2 was to achieve the benefits of TBL from semester 1, but without increasing student workload. The adjustments made in this regard were well-received by students. Course feedback was generally positive, with high overall student satisfaction. The time with peers is thought to have been valuable to students during an otherwise fairly isolating lockdown period.

The semester 2 trial also showed the influence of external factors on students’ engagement with TBL. Both a large coursework deadline and a general decline in motivation as the lockdown wore on were seen to reduce student engagement in the second half of the semester. It seems that students found it hard to catch up once they got behind. This drop in engagement was also observed in a unit without TBL.

The move to a fully online teaching environment meant that there were a few of the issues experienced in semester 1 with hybrid teams. However, in the virtual environment, it was more difficult to spot which groups were struggling or to monitor interaction in different groups.

Discussion

Comparison between TBL and non-TBL units

It has been shown above that TBL promotes attendance compared with a non-TBL unit. Engagement can also be asynchronous; in both the TBL courses and the non-TBL courses, students were asked to watch pre-recorded videos prior to each online teaching session. The completion rate of these videos at the end of the semester is shown in Figure 7 as a box-and-whisker plot. The mean completion rate is similar across all four courses, and there is no clear trend in the median or 75th percentile. However, both SM1 and FM (the TBL courses) show appreciably higher 25th percentiles than the comparator courses (TF and SM2).

Figure 7.

Completion rate of pre-recorded lectures by the end of each semester for two TBL units (SM1, FM) and comparator units (T, SM2). Mean completion rate is marked as a cross, with median (horizontal line) and 25^th/75th percentiles (top of the box) also shown. TBL: team-based learning; SM2: solid mechanics 2; SM1: solid mechanics 1; FM: fluid mechanics

In terms of performance, the exam results are shown in Figure 8, for SM1 and FM for the previous 5 years. These are nominally for the same exams, at the same point of intake and at the same difficulty level. An outlier to the FM data is shown in Semester 2 2019/2020, where the students were given three weeks to complete the exam, which led to a large increase in average. Despite this, one key observation is that the lower quartile for SM1 increased by around 20% between 2019/2020 and 2020/2021 which corresponded to before and after TBL implementation, respectively.

Figure 8.

Exam attainment over the past 5 years for the two TBL units, (a) SM1 and (b) FM, as well as (c) their comparator units. TBL: team-based learning; SM1: solid mechanics 1; FM: fluid mechanics

Comparing the 2020/2021 marks across the units in Figure 8(c), as with the engagement statistics, the 25th percentile is raised in the exam (by around 5%–10%) for the TBL-based units. This is despite very little difference in the mean and median of the data sets, and a small (2%–6%) reduction in the upper quartile. A reduction of failure rates (a score of less than 40%) was also observed for the TBL units: A 4% drop when compared with the non-TBL units (as shown by the outlier removed lower bounds ^a ). Exam markers from the SM1 and FM units also highlighted that the clarity and quality of students’ exam solutions improved when compared with previous years.

Taken together, the engagement and achievement statistics suggest that TBL improves the performance of the mid- to lower-performing students in the cohort, without lowering the performance of the higher-performing students.

Contribution and connections to the literature

The results of this study provide a valuable new contribution to the pedagogical literature in the field of engineering, in contrast to the majority of existing TBL literature which is primarily focused on healthcare pedagogy. ³⁰ The present study differs from these works not simply by discipline but also by virtue of the large (350 + student) cohort which is associated with additional practical challenges. Accordingly, the work of Vasan et al. ⁴² and Ostafichuk et al. ⁴⁰ served as valuable starting points for this study which has been tailored and refined via the use of two iterative trials.

In a similar manner to that widely implemented in the literature, feedback was collected from the students to assess the impact of the approach presented and suggest routes for improvement.^30,54 This study went beyond many existing studies by collecting qualitative data from multiple routes (unit evaluations, focus groups and through peer review) and combining this with quantitative measures of performance, attendance, and exam achievement to draw firm conclusions from the data. The 5%–10% increase in performance of the lowest quartile is consistent with that previously observed in some of the literature^41,55 although one study shows a reduction in performance of this group. ⁵⁸ The origin of the discrepancy is likely to be associated with tailoring the TBL methodology, which has been repeatedly shown to be crucial in the success of the approach.^31,45,46

A key observation is that the use of TBL substantially increases engagement for large cohort engineering units. This mirrors closely what was previously observed in other disciplines,^1,28,33 and highlights that undergraduate engineers also view the interactive nature of TBL in a positive manner. The blended / hybrid approach implemented also links strongly to the future expectations of teaching in engineering, which is likely to rely heavily on these methods via effective and responsible use of technology-enhanced learning methodologies. ⁵⁹

This study combines the TBL framework with numerical, problem-based learning, a style of questioning that is known to be highly suited for engineering students^37,38 and can be effective for large cohorts.^9,10 The results show that a careful combination of these approaches can be particularly powerful.

The TBL approach promotes peer-to-peer learning^23,24 which is an important skill for undergraduate engineers who will typically work in large teams during their subsequent professional employment. Encouragingly, this study suggests an improvement in this skill set for all students, mirroring the results observed in the medical sector. ⁶⁰

In summary, the results obtained within this study align well with the existing TBL literature, while the specific implementation offers an exciting tailored pedagogical tool within the field of engineering. It is hoped that the lessons learnt and refined methodology will be of substantial practical use for the community going forward.

Conclusions

The results of this study suggest that TBL is an effective approach for large cohort first-year engineering programmes. Both students and staff enjoy the approach, and the methodology enhances student skill sets whilst substantially improving peer-to-peer learning. Student performance improves via TBL, particularly for the lowest performing 25% of the cohort. Engagement is enhanced, including contexts involving hybrid delivery. However, participation can be substantially affected by external factors and one of the key challenges is to maintain this interaction without students perceiving an increased workload.

In terms of key learning points, effective use of technology and clear signposting of structure/workload are essential to the success of TBL in large cohorts. While organisation and preparation can be demanding on staff time, there are opportunities for refinement and improvement. Monitoring the effectiveness of numerous teams is challenging, especially online. The use of hybrid teams remains a substantial challenge. The approach adopted makes it challenging for students to catch up if they fall behind and refinement is required to address this.

Future work

A marked increase in literature on TBL methods delivered in a blended teaching format has been seen recently due to COVID-19. This has revealed new possibilities and capabilities within the framework which have the potential to further enhance the successes achieved in this study. The challenges of achieving online team accountability and coherence can be addressed by careful structuring, while simultaneously providing enhanced opportunities for peer evaluation. ⁶¹ Novel TBL designs specifically tailored for online courses have shown significant potential ⁶² and there are effective methods for managing logistics and timing for blended TBL. ⁶³ Methods have also been established for synchronously running blended TBL at more than one campus. ⁶⁴

Although none of these methods were applied to very large cohorts (300+), or within the field of Engineering, many of the lessons learnt have important implications for the approach presented here. In particular, the development and refinement of new technologies will help reduce the significant demands that were placed on the staff running this trial. The integration of administrative support will also help more effectively monitor groups and help establish teams more efficiently. Advances in technology are also opening new opportunities for facilitating online hybrid-TBL activities such as jigsaw and fishbowl activities^65,66 or backchannels for communication. ⁶⁷ These approaches are intrinsically linked to the growing idea of Hyflex teaching ⁶⁸ which has been shown to reduce the number of students ‘falling behind’ and allows leaders to spot groups that are not working/engaging effectively. ⁶⁹