Abstract
Background
The unprecedented increase in critically ill patients due to the COVID-19 pandemic mandated rapid training in critical care for redeployed staff to work safely in intensive care units (ICU).
Methods
The COVID-19 ICU Remote-Learning Course (CIRLC) is a remote delivery course developed in response to the pandemic. This was a one-day course focused on the fundamentals of Intensive Care. The course used blended learning with recorded lectures and interactive tutorials delivered by shielding and frontline ICU trained professionals. The course was developed within one week and piloted at three NHS Trusts. It was then made publicly available free of charge to redeployed healthcare professionals across the UK and Ireland. An iterative cycle of improvement was used to update the course content weekly. A course confidence questionnaire with quantitative and qualitative questions was used to evaluate effectiveness. Data is reported as n (%), means (SD) and thematic analysis was used for the open questions.
Results
1,269 candidates from 171 organisations completed the course, with 99 volunteer trainers. 96% of respondents rated the course as very or extremely useful. 86% rated the online platform as excellent. Overall confidence improved from 2.7/5 to 3.9/5. Qualitative data showed that the course was pitched at the appropriate level, accessible and built clinicians confidence to work in intensive care.
Conclusion
This model of educational delivery with a rapid iteration cycle was a pragmatic, effective solution to knowledge-based training under social distancing measures. Whilst full course evaluation was not possible, we believe that this work demonstrates practical guidance on educational response in a pandemic as well as highlighting the altruistic nature of the critical care community.
Background
The COVID-19 pandemic pushed healthcare services to their limit with 6.4 million confirmed cases, and 383,262 deaths worldwide 1 within six months. This led to 12,086 intensive care unit (ICU) admissions in the United Kingdom (UK) alone during that period. 2 However, the UK has one of the lowest numbers of ICU beds per capita in Europe, with only 6.6 beds per 100,000 in 2012, compared to an average of 11.5 per 100,000. 3 This surge demanded an increased ICU capacity requiring appropriately trained staff. The workforce expansion included redeployment of non-ICU health professionals, creating a substantial need for rapid, effective training. Conventional face-to-face training was challenging due to social distancing measures, the lack of available trainers, and demand urgency.
Concurrently, experienced ICU staff with a wealth of knowledge were confined to their homes due to government guidance on self-isolation and shielding. The objective of the presented programme was to rapidly develop and scale an interactive distance learning programme that was compliant with social distancing, but also harnessed the unique opportunities and expertise it serendipitously afforded.
Methods
On 18th March 2020 authors EC, AMo and MC from Brunel University London (www.brunel.ac.uk) and 33 N Ltd. (https://33n.co.uk/home) had a virtual meeting to discuss how both organisations may be able to assist in the COVID response following collaboration with the Northwest London Critical Care Network (NWLCCN). We recognised that there was a high work burden of in-house training for redeployed staff that was being implemented within local hospitals by frontline clinicians. Concurrently, scores of highly skilled critical care staff were shielding at home unable to work on the frontline. We identified that mobilising these staff to assist in teaching frontline clinicians using a remote format may help to off load the work burden of frontline clinicians whilst allowing educational delivery that conformed to social distancing rules.
The result was the rapid development of a 1-day online course developed and delivered by clinicians and academics specialising in intensive care. The process and principles that resulted in this development are detailed below.
Underpinning educational principles
Computer-based technology using online learning content is widely used in undergraduate and postgraduate medical education. For example, to support training for healthcare professionals, the NHS has invested heavily in a web-based e-learning platform http://www.e-lfh.org.uk. Educational packages with a “drill-and-practice” approach can be highly efficient for skill-based training such as x-ray interpretation and drug prescribing. 4 However, without interaction with the teacher and other students, learners may feel unmotivated and unsupported. Moreover, the typical lecture followed by homework elements of a course can make the learning experience isolating and impersonal.
Blended learning offers a solution to these problems by combining pre-recorded lectures to be watched by students and assimilation of knowledge through focused group discussion lead by field experts. This ‘flipped classroom’ model contrasts with the traditional model of learning where first exposure to knowledge occurs inside the classroom with further consolidation and application of knowledge achieved through individual learning. 5 The ‘flipped’ approach not only promotes a more personalised learning experience with expert support readily available, but also encourages peer interaction and collaboration.4,6
The overall development of the course was also guided by general standards of course design (e.g. Quality Matters rubric https://www.qualitymatters.org/qa-resources/rubric-standards/higher-ed-rubric) including consideration and alignment of course overview, learning objectives, instructional material, learner interactions, course technology and usability and learner support. Due to the nature of the pandemic and pressure on staff time, assessment of knowledge and implementation in practice was not practicable and subsequently not included.
Content development
The first step was to identify the target audience for the course and to develop the core content. We liaised with local NHS trusts to identify the staff groups that were being redeployed- in the early days of the pandemic this was theatre nurses, operating department technicians, junior doctors and surgeons. We contacted NWLCCN, practice educators and frontline clinicians to determine the essential
Session plans and learning objectives for each topic and for each profession were developed by the core faculty (MC, AMo, AW and EC) comprised of lecturers in critical care from nursing and physiotherapy backgrounds and intensive care doctors, EC, MC and AW are also Fellows of the Higher Education Academy. An example of the learning objectives for the nursing course is shown in Table 1.
To address these learning objectives, we used a blended learning model combining a series of online lectures written and pre-recorded by topic experts (PS, AW, JB, TM, EC, MJ and AH). These were followed by case-based interactive tutorials on the same topic delivered to profession-specific tutorial groups of up to 20 by experienced volunteer ICU staff. Lectures and tutorials lasted around 30 minutes each.
Technical set up
The course utilised technology donated by Blackboard Learn Online (BBL) (https://www.blackboard.com/teaching-learning/learning-management/blackboard-learn) integrating pre-recorded lectures recorded on panopto™ software (https://www.panopto.com/), with online resources, and ‘collaborate’ room function. The collaborate rooms enabled candidates to speak directly to tutors and each other using audio or text functions. This allowed candidates to consolidate their learning whilst drawing on the tutors’ practical expertise.
Administrative support was provided from 33 N Ltd., and the technological support from Brunel University London.
Course registration and advertising
There were two routes to both register for the course and to volunteer as a tutor. Marketing material was developed and distributed via formal and informal networks to recruit candidates and faculty. Formal networks included liaising directly with NHS Trusts and Sustainability and Transformation Partnerships (STPs), and advertising via the Intensive Care Society and The Association of Physiotherapists in Respiratory Care. In addition, we launched a course Twitter™ account (@IcuRemote) which has n = 1,098 followers to date to advertise for both tutors and candidates.
A total of 99 experienced ICU staff consisting of frontline, self-isolating or shielding clinicians and academics taught on the course. To maintain high standards, we reviewed tutors’ credentials and used a train the trainer system to teach on the course. In order to teach on the course, we requested that all physiotherapists and nurses worked at Senior (Agenda for Change Band 6) level or above in ICU or had an academic role delivering ICU education. All tutors were registered with their professional body. All doctors had at least 3 years’ experience working in an ICU. Demographics of the tutors are shown in Table 2.
Learning objectives for nursing specific course.
Demographics of course tutors (n = 99).
All new tutors were paired with core faculty or experienced tutors to observe the tutorials first before delivering them themselves. They were then observed by experienced faculty delivering the sessions. All sessions had two tutors for quality assurance and in case of sickness absence. If any concerns were noted about the quality of tutoring then the tutors were removed from the faculty.
Organisational structure
A course team was created to manage ongoing organisational refinement including administration, IT support, content and tutors. Continual engagement with Trusts and social media facilitated the training of large cohorts of staff. The course was endorsed by the Intensive Care Society (https://www.ics.ac.uk/). The course was a multi-organisational collaboration between Brunel University London, 33 N Ltd, and Blackboard, and piloted within a week of its conception.
Improvement cycle
The rapidly emerging information during the pandemic, and the course format, required a rapid cycle of iteration. Daily interactive feedback sessions with tutors and candidates were used to enable rapid improvement of course structure, user interface and administration. Course content leads were appointed for each stream to review and update the course material weekly to reflect candidate feedback and new clinical guidance. For example, prone positioning of the awake patient was integrated into the course content as this treatment strategy emerged during the pandemic.
Delegate evaluation methodology
From the outset, candidates were asked to give a global score on the course usefulness and use of platform via an online form. After two weeks a more formal evaluation was undertaken with a course questionnaire administered before and after the course (online Appendix A). Its development was informed by Sitzman & Weinhardt, 7 focusing on training utilization (including demographic data on course attendees), training affect and performance indicators (specifically confidence of application to practice). Candidates were given timetabled time during the course to complete this. The questionnaire was hosted on onlinesurveys.ac.uk and submissions were anonymized.
The questionnaire data were predominantly analysed using descriptive statistics. Open comments were invited regarding participants experience of the course and this qualitative data were transferred onto a word document. They were analysed through inductive descriptive thematic analysis. 8 This is an atheoretical approach suitable for a pragmatic evaluation. The process included familiarization of the data, line coding, collation of subthemes and thematic development. Analysis was completed by one experienced researcher who was independent to the teaching of the course (MN). The candidate themes were discussed with other members of the evaluation team who were involved in verbal feedback from participants to ensure resonance prior to confirmation.
The process of course development, delivery and evaluation is displayed diagrammatically in Figure 1.
Diagrammatic representation of the course development and refinement process.
Funding
The course was provided free to all participants. The organisations donated their employees’ time and software to design, build and hold the structure of the course, at an estimated combined cost of £102,700. The course was then delivered through approximately 3,808 hours of volunteer tutors time, approximating a value of £88,148. The course was designed in a modular fashion to enable rapid scalability, with the estimated cost at between £60 and £100 per person.
The outcome
Demographics and course attendance
The course ran 36 times from 25th of March 2020 to 27th of May 2020 and was accessed by 1,269 candidates from 171 healthcare organisations in the UK and Ireland. Attendance ranged from 60-200 candidates/day. Of these, 1,027 completed basic demographic data demonstrating a professional spread of 476 (46%) Physiotherapists, 249 (24%) Doctors, 234 (23%) Nurses, and 68 (7%) unspecified.
More detailed demographic information was available from the 210 participants who completed the pre-post questionnaire and these are shown in Table 3. Of note 32 areas of clinical specialty were represented and 154 (73%) of candidates had either no or <6 months experience on ICU/HDU.
Candidate demographic data.
aOther included: obstetrics & gynaecology, occupational medicine, maxillofacial surgery, otolaryngology, paediatric cardiology, paediatrics, plastic surgery, public health medicine, clinical genetics, clinical neurophysiology, clinical pharmacology and therapeutics, clinical psychiatry and heamatology, clinical oncology.
Evaluation results
Quantitative data
Of the 1,027 candidates who completed the global evaluation, 96% of respondents rated the overall course as very or extremely useful. 86% rated the online platform as excellent.
The questionnaire results, from 210 participants, demonstrate increased overall confidence for all professions- pre-course mean confidence of 2.7/5, post-course mean 3.9/5 (Table 4). While this pattern was broadly replicated differences in topic and profession were noted (Figure 2).
Changes in confidence by topic and profession.
Results of the questionnaire showing the topic with biggest increase in average confidence for each profession before and after the course.
Qualitative questionnaire results
Themes included the pitch of the material, tutor expertise, forum accessibility including BBL, the approachability of the tutors, building confidence in practice and the course as a platform for further learning. Details of each theme and supporting quotes are shown in Table 5.
Derived these from open questions.
Discussion
The COVID-19 pandemic mandated rapid cross skilling of healthcare professionals to be able to manage, treat and care for critically ill patients in the ICU environment. This created extra workload for frontline staff. The CIRLC course demonstrated that during the current and future pandemics, learning technologies can be utilised to create solutions to educate, whilst reducing training burden on frontline clinicians, maintaining social distancing measures and facilitating rapid iteration, in both formal and informal education.
This course was successful in terms of speed of set up and engagement. The results also demonstrated improved confidence in the taught topics and building confidence to practice. This ICS endorsed course has been adopted by Health Education England to be delivered pan-London to address the training needs required to maintain a flexible critical care workforce.
Key lessons and limitations
Utilizing shielding faculty was feasible and enabled recruitment of experts in their field. This optimized use of the available workforce, and gave shielding staff a role in the pandemic. The online blended learning format allowed learners to access the educational material, consider it at their own pace, and check their understanding through the interactive tutorials. This was highly valued by candidates. The novel format could be replicated for induction and training in the post COVID era. However, it is accepted that this approach does not allow for any practical teaching relating to patient management. Future iterations could consider potential follow-up sessions on site to evaluate the translation of learning to practice. The rapid cycle of iteration was essential due to the nature of the technology, new working relationships, multiple educators, and emerging clinical guidance. This process is a useful consideration for any course that needs rapid development and implementation and requires human resources in each area to enact changes in a timely manner. The range of educators required support to ensure quality delivery and up-to-date messaging. Careful consideration of organization and senior faculty support is required to sustain quality. Due to constraints of the pandemic a pragmatic approach to the course evaluation was used. It was not feasible to evaluate in-practice competence of the course participants. This is a major limitation of this work. We estimate that the course cost £60–100/candidate. The cost of running the course needs to be considered for ongoing sustainability and we would recommend costs being factored into professional development funding as part of the pandemic response.
Supplemental Material
sj-pdf-1-inc-10.1177_1751143720972630 - Supplemental material for Covid-19 ICU remote-learning course (CIRLC): Rapid ICU remote training for frontline health professionals during the COVID-19 pandemic in the UK
Supplemental material, sj-pdf-1-inc-10.1177_1751143720972630 for Covid-19 ICU remote-learning course (CIRLC): Rapid ICU remote training for frontline health professionals during the COVID-19 pandemic in the UK by Matthew Camilleri, Xiaoxi Zhang, Meriel Norris, Alex Monkhouse, Alex Harvey, Allison Wiseman, Pratik Sinha, Alex Hemsley, Sophie Tang, Arun Menon, Smruti Sinmayee, Mandy Jones, Jim Buckley, Ruth Johnson, Thomas Medici and Evelyn Corner in Journal of the Intensive Care Society
Footnotes
Acknowledgments
We would like to acknowledge the contribution of all 99 course tutors, and the administration and IT teams involved in the delivery of the course. We would also like to acknowledge Brunel University London, 33N, CLEAR and Blackboard Learn online for providing staff time and software free of charge to allow the delivery of this course.
Competing interests/declarations
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: 33N is a private limited company specialising in healthcare data analytics and training. It is run by working NHS clinicians and provides the Clinically Led Workforce and Activity Redesign Programme. 33 N provided resources for CIRLC for free with no charges or monetary gains.
Blackboard International B.V: Blackboard provides educational services to support teaching & learning solutions. Blackboard’s EdTech platform ‘Learn Ultra’ was provided for CIRLC for free with no charges or monetary gains.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
