Watching me … watching you: Training and assessment in a virtual world

Abstract

Virtual education isn’t new. Teaching has been delivered remotely for many years, although it came to prominence with the enforced move to online learning during the COVID-19 pandemic. Technologies provide tools that are integral to our everyday lives, and training and assessment are no different. Stepping back from the obligatory remote delivery of practical skills training, now we have the opportunity to review and evaluate our own learning (as educators) about the value of resources for training delivery. Combining sound educational principles with the available technologies, we can use remotely delivered learning to enhance our teaching and to increase access to learning, without hindrance because of distance or the type of facility. Remote connections allow expertise and best practice to become suddenly available to a wider cohort of researchers, rapidly spreading new ideas about refinement of in vivo procedures and supporting colleagues to learn and develop. We currently use only a few of the technological tools available, and there is much to learn from other disciplines where virtual and augmented reality are assisting surgeons, aircraft pilots and others daily. By harnessing remote and assisted technologies in teaching, we can also develop the mindset and ability of the biomedical community to use them to augment, or even to replace, animal studies in future and to democratise training globally.

Introduction

Virtual education isn’t new. From the Open University’s radical, disruptive approach to accessing higher education in the 1960s, teaching has been delivered remotely for many years, more recently with an enforced widespread uptake of online learning during the COVID-19 pandemic. Technologies provide tools that are integral to our everyday lives, and training and assessment are no different.

Stepping back from the absolute necessity to deliver practical skills training remotely, now we have the opportunity to review and evaluate our own learning (as educators) about the value of resources and incorporation of non-animal methods for training delivery. Just transferring our classroom materials online, or making a video of a workshop, isn’t necessarily helpful in promoting learning; delivery methods should be chosen thoughtfully to engage students actively, and then evaluated at the end of the course or module. There are some excellent examples of creative, innovative methods of practical skills teaching that were developed collaboratively during the pandemic, such as the #DryLabRealScience project ¹ and Lecture Remotely online science platform, as well as the existing medical training platforms described later.

The design of any course, with its teaching and learning materials, must first be based on sound pedagogical reasoning. Technology is a tool that can be used to add value to teaching rather than just adapting course design to make use of a novel digital modality (even if it is deemed ‘cool’). ² The student user–technology interface must function well, and it is essential that educators themselves have digital media training and support to understand what technologies are available in order to encourage creative integration of technologies into their teaching. ²

Knowles ³ described the principles of adult learning, and these principles guide what we must consider when integrating technologies into daily teaching practice:

Students are empowered to learn when they need to

Adult learners bring and integrate life skills to their learning

Learning is a dialogue between tutor and student

Students can learn at their own pace

Students have a safe learning environment

Tutors shouldn’t make assumptions

Learning should be inclusive and accessible to all

Evaluate the effectiveness of the teaching and learning materials

In addition, the principles of best practice in medical and veterinary education – that the learning should offer opportunities for skills acquisition through deliberate practice, high fidelity of the simulation and the ability to master learning in order to transfer it to practice – apply equally to procedural training in the in vivo research context.

Teaching practical skills remotely presents its own challenges. But further, in the laboratory animal science (LAS) environment, we are educating people to work with sentient animals (usually small rodents or fish), and we also need to imbue a sense of ethical animal use, responsibility in good scientific conduct and a culture of care for the animals and people around them.^4,5 It is not sufficient to pass on just knowledge or even just technical skills when working with research animals: acquisition of professional behaviours is essential.

Many online activities, such as watching prepared videos and virtual reality (VR) simulations, are solitary, and so challenges to educators are to maintain collaboration between students and to enable a personal rapport between tutor and student in order to facilitate dialogue and exchange of ideas, as well as promoting better student engagement and professional growth. ⁶ Synchronous activities are needed in this case, and live video can allow for interaction, questioning and discussion.

Synchronous training or assessment using video headsets

While video broadcasts are static and can seem quite remote, wearable headset technologies enable one-to-one communication. Wearable headset technologies, such as RealWear™, connect tutor and learner directly through a video call in real time, so that practical teaching and assessment can continue almost in the same way as side-by-side working. Lightweight and with excellent image resolution, the headsets have voice-controlled functions, allowing the operator to continue the task uninterrupted, while the observer sees exactly what the wearer sees. We have used these headsets extensively, particularly when access to facilities is restricted, in order to carry out training and to assess competence in practical skills. Using video headsets enabled better and more frequent real-time access to facilities with foot-fall restrictions, such as high health status barriers, or during the recent pandemic.

During COVID-19 lockdown, a significant amount of our routine veterinary support was carried out using RealWear, and this proved highly successful, as the headset wearer was able to move around the facility, carry out tasks, examine animals and talk with the remote correspondent (the vet) almost as if we would when in the same room. The mobility of the operator is what really makes the difference beyond static video calls on a computer, or even a handheld device: the headset camera is at eye level, giving a natural image to the wearer and viewer and allowing incidental conversations with others in the same room too – something which is challenging when using a laptop screen.

The ability to make a video call as and when it is needed has been very helpful in providing support for colleagues as they learn and carry out procedures. The video headset enabled us to carry out basic surgical training and assessment, as the high image quality and ability to focus the camera via voice commands allowed the trainer to have an accurate view of the procedure in detail, while the headset allows easy conversation with the trainer or assessor but doesn’t interfere with the operator’s movements. Similar ‘on your shoulder’ type cameras, such as the Proximie virtual collaboration system (Proximie Ltd, London, UK), have been used in health care to enable an expert surgeon to guide colleagues remotely, providing immediate, live advice as the operation proceeds. Surgeons at Guy’s and St Thomas’ Hospital, London, have also used Proximie to livestream surgical procedures for clinical teaching, providing multiple video feeds and direct voice communication with the lead surgeon. ⁷

In the laboratory animal setting, the convenience of being able to access the headset, make a quick video call and resolve an issue ‘in the moment’ is superb to enable rapid assistance with troubleshooting, or further support as the operator gains confidence and starts to work with less supervision. The remote expert is of course unable to intervene physically. So, the training and task should be at an appropriate level for the trainee’s experience. ⁸

Sometimes, training is required after a study has begun. So, the ability to ‘access the expert’ whenever needed provides confidence for new operators and reassurance for trainers and compliance officers. Using video technology, training could be carried out anywhere in the world, with the expert trainer guiding learners one-to-one wherever they need help and ensuring that knowledge about current best practice is available to everyone.

Other creative and low-cost solutions that have worked well for us in the absence of a RealWear device include using a wearable a body camera in the same way, or even placing a laptop camera close to the workstation to view the procedure in progress. Optical resolution is very good with both methods. Providing good-quality sound to the laptop has been more challenging but, again, easily solved by using an inexpensive external microphone, and this has enabled the tutor and learner to talk freely during training or assessment so that feedback can be given right away. Whichever solution is used, the technologies all require reliable connectivity at both the tutor’s and trainee’s locations.

Interestingly, some operators reported finding that remote assessment seems less intimidating than having the in-person assessor next to them – similar to the ‘social inhibition’ effect described in psychology. ⁸ It would be an interesting topic for a study to investigate potential advantages for increased fairness of assessment.

Remote technology and 3Rs for practical skills training

One of the issues in enabling widespread and rapid adoption of refined methods is poor dissemination of the information and provision of suitable training for researchers and technicians. ⁹ There are many training videos and paper supplementary data available, but not all are peer-reviewed, so assuring quality and currency of the instruction can be a concern. Whilst asynchronous videos are helpful in initial training, the ability to work with the expert, discuss ‘in the moment’ and receive feedback engages the student actively, builds rapport and results in better proficiency. ¹⁰

It is by pooling and sharing our expertise that we will progress science and animal welfare more rapidly across the world. Video headsets have great potential to increase accessibility of learners to specific expertise and to improve uptake of refined techniques: one tutor is able effectively to guide many trainees in different locations, spreading good practice much more rapidly and easily. ² Virtual communication allows sustainable, effective training on demand whilst remaining environmentally responsible.

What can we learn from the use of technology in other disciplines?

Simulators

Simulators allow ‘deliberate practice’ of a task, that is, repetitive performance with feedback in a controlled setting. ¹¹ Whilst simulators are sophisticated, extensively used and valued in health-care settings, ¹² only a few low-fidelity models, such as the Curvet Rat (CITA Rat) and MimickyMouse (VetTech) are currently available for LAS, perhaps due to the expense of developing high-fidelity models (which provide multi-sensory feedback and a range of learning opportunities). However, basic models could be further developed by providing a range of difficulties (e.g. pink vs. pigmented mouse tails) to aid learning, or to develop non-invasive techniques.

We have successfully carried out basic suture training using simulators via video link. The students are sent a ‘learning pack’, including instruments, suture pad and materials. Then, they join a video call with the tutor who demonstrates on their own equipment, watches the student and gives feedback during the workshop.

Physical and online (e.g. Open University Open Science Laboratory, Royal Veterinary College digital labs) simulators provide both a replacement method and a safe learning space for trainees to practice defined techniques without harm to any animals; simulation-based teaching in medical education has reduced risk to patients and has been shown effective in both under- and postgraduate education. ¹³

Simulation is used in standardising evaluation of competence, showing an overall positive impact in medical training ¹⁴ and video-based assessment of basic procedural skills can be made by trained non-specialists, ¹⁵ promoting consistency of approach over a wider range of institutes and reducing the resources required to apply competence assessment for trainees. Using validated methods, such as Directly Observed Procedure Skills (DOPS), ensures consistency of approach and is already being translated into the LAS environment.

Simulation also comprises more than just ‘gadgets’: a holistic approach to training the researcher or animal carer should take into account what skills the students need and form part of the planned training programme. An evidence-based guide to the use of practical simulation can be found in the article by Motola et al. ¹⁶ Task-oriented, process-oriented (teamwork, self-directed learning) and professional-oriented skills (culture of care, communication) are all required to operate in the modern, collaborative and open research environment. All of three of these themes can be modelled to accelerate and standardise learning, and immersive technologies take these capabilities further.^5,17

Immersive technologies

Immersive technologies encompass 360° video, virtual reality (VR) and augmented reality (AR). Whilst 360° videos give the impression of presence in the learning environment, the student views the material passively on a screen, and rapid camera movement around the scene can even be disorienting. However, such learning materials can be useful, for example in virtual tours, facility inductions or orientations to the laboratory. ¹⁸

In VR, the content is created as a computer-generated image for the student to experience in real time, providing a dynamic, adaptive and immersive learning experience where the user moves around and interacts within that virtual setting.

AR takes the user experience further, enabling digital content to be overlayed onto the real world, so that the user can make decisions, access additional information and enhance their ability to carry out a task. ¹⁹

VR simulation

Experiential learning theory ²⁰ emphasises the role of experience in learning: deeper learning is achieved when we take part and experience through multiple senses rather than just hear or see information. ²¹ VR has been used for decades in the aviation industry and has been demonstrated to reduce accidents through improved pilot training.

To access interactive VR scenarios, the learner puts on a head-mounted display (HMD) running the appropriate software and is able to move around and interact with the created setting as they would in real life, giving the feeling of being completely part of their virtual environment and providing the ability to learn from experience, thus creating a powerful learning tool. ²² More sophisticated systems also add other sensations such as sound, touch (haptics) or even olfaction in order to enhance the impression of taking part in the scenario.

VR enables the student to experience or even walk through areas that they would or could not otherwise access. Using a VR headset, the student can complete tasks in their new workplace under instruction, without having to leave their current environment. The VR scenarios are standardised and can be repeated many times by the student, who learns through deliberate practice and the ability to make mistakes in a safe environment. VR set-ups are becoming relatively cheap and straightforward to use, requiring minimal intervention from trainers, which makes this type of learning more cost-effective for an institution. ²³ The more immersive experience of VR also improves accessibility for a diverse range of students who may otherwise have challenges accessing more traditional learning modalities and can be used at various educational levels (e.g. 3D for Science, Spain).

VR is commonly used in teaching both medical and veterinary students (e.g. Digilab Imperial College, London), ²⁴ where the user wearing the headset is able to manipulate the ‘body tissues’, and the artificial intelligence (AI)-driven programmes enable the virtual patient to respond to the manipulations in real time. This replacement technology is equally applicable to LAS experimental surgery and would allow high-quality training without the need to use cadavers or live animals, recognising the ethical issues that the use of animals in teaching presents.^25,26

The capabilities of VR systems are improving rapidly, with newer headsets (e.g. Microsoft HoloLens3, Oculus 2) providing a better all-round field of view, easy interaction with the visual graphics presented, plus optional features such as iris recognition to log the user into their account automatically. VR headsets are able to track the user’s hands and direction of gaze, and some include haptics and sound cues in order to provide a more realistic and responsive experience for the user. For example, commercially available systems are used for developing laparoscopic surgical skills (Minimal Invasive Surgery Trainer-VR ²⁴ ) and in engineering (TeleAdvisor ²⁷ ).

Training for experimental research is often focussed just on the desired procedure but, by taking a more integrated approach to training, for example, a basic knowledge of local anatomy could be obtained before starting training on the procedure itself, and management of potential adverse effects practised but without harm to an animal. Presenting training in this way allows the student to learn in a safe environment and at their own pace, nicely meeting one of the key concepts of adult education.

High-quality scenario software is commercially available, and although most VR involves a single person using the headset, ‘groupware technology’ is available for teamwork scenarios and in gaming. Collaborative learning encourages learner participation, although more research is needed in order to evaluate training outcomes. ²⁸

VR provides immediate feedback and is also able to record the trainee’s progress, providing a structured and consistent situation in which to assess competence.

Adding a storytelling element to the physical VR experience builds non-technical skills in decision making and communication and enables exploration of scenarios where less optimal events occur. These are currently available for ‘in-person’ workshops ²⁹ and in medical VR training, ²³ but a similar concept could be introduced into the LAS VR setting too. These narrative scenarios could promote culture of care, train those developing at senior levels and also form part of training for animal welfare committees to develop understanding of everyday practice in a research animal facility.

AR

AR combines real and virtual objects that interact in real time, allowing them to be tracked and manipulated by the user, for example digital bones or vasculature to be overlaid on a body (e.g. Oxford Medical Simulators) to assist in conducting a surgical procedure. The content can be overlaid using a headset, glasses or smart device either using markers (e.g. QR codes) or being ‘markerless’, using other sensors as cues. The user still sees their physical environment, moving and interacting with the real objects – this is the main difference between AR and VR – and allows this technology to be more mobile in the real workplace.

AR can assist accuracy and precision through integrating with AR glasses (Google Glass, Microsoft HoloLens) or surgical loupes (e.g. MagicLoupe™) to improve the surgeon’s view of the patient by projecting previously collected magnetic resonance or computed tomography images onto the field of vision during the procedure in order to provide additional clinical data. Bringing AR into training allows learning to occur not only in scenarios but also ‘in the moment’ ³⁰ and is particularly suited to acquisition of practical skills in the workplace context. ³¹ Digital learning materials are kept up to date and can even be held on a smartphone. The trainee would only progress to the live animal once competent in the AR setting.

Less sophisticated apps (e.g. Accuvein, Q Medical) are already available to assist visualisation of blood vessels in order to improve accuracy of venous access, for example in paediatric or obese patients. ³² These are relatively inexpensive and, although currently designed for human subjects, could be applied to improve welfare in the research environment, where securing intravenous access in smaller rodents is often quite challenging for some researchers, thus reducing avoidable injury to the animal.

Good trainers assess the body language of their students during learning in order to understand how they are progressing. A frowning student, stepping back or a big smile tells us whether we need to provide more support and encouragement or to finish a successful session. AR can project a whole image of another person (e.g. the trainer) in the viewer’s surroundings, which improves non-verbal communication beyond just head-and-shoulders video calling and enables us to check that learning objectives are being met. Avatars appear quite real to the observer, even influencing behaviours. For example, a study with undergraduate students showed that they were reluctant to sit on a chair where a virtual person had previously been! ³³

Currently, AR is mainly used in clinical anatomical/surgical situations but, given development of analogous laboratory animal content, could equally be applied in the LAS context, with significant refinement potential.

Gamification

We learn best when having fun. So, bringing game-based elements into the learning can improve engagement in meeting objectives and practising skills. ³⁴ Students may be more relaxed in the game situation and more willing to try out new things and to make and learn from mistakes. ³⁵ This approach is most familiar in the quizzes in online modules but can also involve practical skills during VR-based training, building in advancing difficulty as the student masters the task. Feedback from games assists the student to review their progress and could also be used in formative or summative assessment. This approach successfully promotes learning in other contexts, such as schools and corporate training, but has not yet been evaluated in health-care or research settings. ³⁶

Integrating the technology into learning and assessment

Before presenting a learning modality to the student, it is important to ensure that it is simple to access and that clear instructions are available on how to use and access support, otherwise, it can become frustrating. Some of the more realistic medical simulators are quite complicated, so, students may benefit from the presence of a trained assistant. By contrast, commercial VR systems are simple and intuitive to use, so reducing demand on trainers.

To equip a facility with VR technology requires the purchase of the HMD headset (currently around €2000), a laptop and the software, although once purchased, its use is scalable, with relatively little additional resource required to train a number of learners. The systems and software are commercially available, straightforward to operate, safe and don’t require tutor intervention. This can make VR systems more cost-effective to purchase, use and update for multiple users than physical simulators, which take up space and tutor resource. The relative cost of technologies reduces, and capabilities increase markedly with time, which will soon put immersive learning within reach of many facilities.

Existing teachers may be much less willing to spend additional time to identify ways to use new technologies if they don’t themselves have training and support to begin with.^37,38 Training in skills for the tutors in delivering ‘tele-teaching’ is required,^37,39 and many institutes have dedicated digital learning resources groups (e.g. Imperial College, Digilab, Royal Veterinary College Digital Learning Team) to provide dedicated support. The University of Oxford identifies ‘super user’ experts to champion use of their AR technologies. Specific education is needed about what tools are available and how they could be applied in context. Then, teachers can understand, think and begin to scaffold the learning design, creating their own ideas for integration. By involving technology from the start, it naturally becomes integrated into training (for both tutor and student) and can help make the step to using non-animal replacement technologies in research practice much closer. By building in replacements from the outset, we can help to meet society’s ethical expectation that animals will be involved only when really necessary. ²⁵

Competence assessment lends itself easily to technological integration. DOPS assessments for a range of tasks could be applied using an app for tablet or smartphone and carried out online. Then, the result could be automatically uploaded to the institute’s or accrediting body’s database. In the standardised, repeatable scenarios of VR systems, objective assessment of the learner’s performance in a set task is already integrated. Centralisation of the assessment standards would ensure that standards of competence are harmonised across institutes and that all assessments incorporate the most recent 3Rs and good practice developments.

Educational evaluation

Whichever methods we use in teaching, whether or not technologies are employed, it is important to evaluate their effectiveness in helping students to achieve the defined learning outcomes and for their performance in future tasks. ⁴⁰ Kirkpatrick’s model, ⁴¹ while simplistic, suggested that medical training can be evaluated at four different levels, from learner satisfaction with the course to improved patient outcomes.^40,42 We could perhaps substitute ‘improved research quality and animal welfare’ for the latter, as this is the overarching goal of LAS training.

The most important factor in effective training is that the student participates actively in their learning.^20,43 VR and AR technologies have been incorporated into medical training for many years and their value demonstrated for clinical situations requiring decision making ²¹ and manual skills.

All the technologies allow the student to interact with the learning materials and to progress at their own pace, particularly in the case of simulators, VR and AR, where the task can be practised and scenarios replayed as many times as required. The student is able to do this in a safe learning environment, as there are no risks to themselves or the animals (or study data). Having these modalities constantly available would assist continuous professional development and ‘refresher’ training – particularly relevant when procedures are carried out intermittently and skills may fade over time. The competence of operators is essential to ensure that procedures are carried out correctly, reproducibly and with minimal effect on the animal.

Training is a hugely labour-intensive (but worthwhile) activity. So, collaboration on identifying priorities, developing learning materials, sharing good practice and evidence-based discussion will benefit not only our trainees but also the quality of the science and the welfare of any animals involved. Committed, creative tutors can take the best of the technology to enhance their teaching practices.

Sustainability

A modern approach to teaching and learning comprises attention to social responsibility and sustainability goals, including access to quality education and sustainable development through partnerships.^43,44 In the same way that remote technologies allow expertise to be available to different researchers within the same country, they enable expertise in developed countries to be more easily available in the Global South, without the need for continued long-distance travel and its associated resource implications, including carbon footprint. Consideration has to be given to cost and local connectivity, but a combination of hands-on and virtual training means that surgical training is already being delivered to low- and middle-income countries through initiatives such as VRiMS. ⁴⁵ Similar training in LAS for developing countries is offered through the University of Leeds’s excellent partnership programme. ⁴⁴

Through leveraging the available technologies and a willingness to explore remote teaching methods, learning becomes more collaborative and democratised, with advantages for scientific quality and animal welfare worldwide.

The sci-fi future (it’s closer than you think)

It is essential that we train our colleagues of the future to conduct robust science and to safeguard the welfare of the animals involved. Perhaps eventually, some experiments could be carried out entirely using interactive digital content.

Digital and immersive technologies offer the possibility to make high-quality training more widely accessible, to share current good practice quickly and to ‘troubleshoot’ when issues occur. However, making the hardware, infrastructure and technical competence resources consistently and globally available will require effort and investment. The laboratory animal community can be slow to adopt new methods and currently lags behind professional colleagues in other disciplines in terms of new training methodologies. There are a large number of software engineers offering to develop AR software/apps if our community only asks for it! Perhaps in future, international training platforms such as ETPLAS will offer an AR facility for trainees.

AR of course can’t produce ‘real’ objects that have substance in the real world, as envisioned in science fiction – but the combination with other sensations, such as haptics and olfaction, can give a nearly real sensation to the user. This would help users to understand the reaction of the whole animal, rather than focussing on a single area or system of interest, so improving understanding of welfare and science. An aspiration is that computer-generated simulations may remove the need to use animals in some procedures, as new techniques and models can be explored in the virtual world instead.

Conducting procedures on living animals comprises more than just the technical know-how and manual skills. Humans are social, and we need and value the rapport built between student and tutor. As collaborative AR becomes available, several tutors and students with headsets (remote or co-located) will be able to communicate and share ideas and good practice rapidly, perhaps even in a virtual conference. ⁴⁵ The advantages are enormous as expert training now becomes scalable – accessible to many more people, without difficulties of geography, facility health status or accessibility. ⁴⁵

In the sci-fi novel, Rainbows End, ⁴⁶ the protagonist wakes up in the future, having recovered from Alzheimer’s disease, and finds himself in a world where AR is used for daily life, and where teams work together to solve problems worldwide. Both these themes present excellent goals for an optimal future!