Just-in-time training for nasotracheal intubation: A report of a pilot study

Just-in-time (JiT) training is a contemporary clinical teaching and learning strategy that is becoming more popular within the healthcare sector and amongst clinicians. ¹ We report a pilot study of JiT training and comment on our results. The purpose of our study was to explore whether JiT training would be beneficial for clinical teaching and skill refreshment of anaesthetic trainees. JiT training involves simulated practice of a task in close physical and temporal proximity to the actual task required, endowing the learner with an immediately available skill. ² Studies suggest that a marked decay in procedural skills starts anywhere from 2 weeks to several months after training, ³ and there are many skills used by anaesthetists, both simple and complex, that may warrant investigation into whether JiT training can improve competency. Nasotracheal intubation was identified as a complex and relatively linear task amenable to investigation, with mastery of this skill being particularly important, as errors or uncertainty may have a direct negative impact on patient care.

Before commencement, ethical approval was granted through The Prince Charles Hospital Human Research Ethics Committee (HREC/17/QPCH/40). Twenty-four anaesthetic trainees (registrars or residents) at any stage of their training were randomly allocated over an 18-month period to receive/not receive a JiT training session directly before a dental case where nasotracheal intubation was required. The control group received a 15-minute break from theatre, to maintain blinding of the supervising consultant, and the intervention group received 15 minutes to practise nasotracheal intubation. This included reading a simple proforma, followed by a hands-on simulation on a part-task trainer using equipment identical to that used in theatre. All participants were timed when intubating the patient and given a questionnaire to complete within 30 minutes of intubation. Time to intubation of the patient (measured by an independent assessor) was used as a surrogate marker to assess proficiency, and a validated questionnaire assessed self-confidence. ⁴ The questionnaire consisted of six questions measuring confidence on a five-point Likert scale and included ‘How confident were you overall during this procedure?’ and ‘How would you rate your skill level at performing this procedure?’. This was followed by additional questions regarding their frequency of performing nasotracheal intubation, time since last performing this skill, level of training (1 to 5, referring to year-level of experience) and a subjective question focused on whether the participant felt JiT training improved their performance.

We found that the mean time for intubation was not clinically different between the groups and the confidence scores were also similar. An independent samples Mann–Whitney U test also found no significant differences between the two groups for time (U = 81, P = 0.603), nor confidence (U = 69, P = 0.861). Unfortunately, the randomisation resulted in a control group that had a higher level of overall training, making them more experienced than the intervention group, which may have confounded the results. To inform future research, if time is to be used as the outcome measure, our results suggest that to detect an effect size of Cohen’s d = 0.8 with 95% power (alpha = 0.05, one-tailed), G*Power recommends a sample size of 3,684 participants per group in an independent samples t-test (total n = 7368), ⁵ conceding this an unfeasible undertaking.

Valuable lessons were learned through this research regarding outcome measures. Although time is attractive as an outcome measure due to ease of measurement and statistical analysis, investigators must be cautious. It was originally hypothesised that the longer it took to secure the nasotracheal tube, the less proficient a participant was. We have realised through this study, that for this specific procedure, using time as a surrogate marker of proficiency was too tenuous due to multiple variables. A very skillful intubation in a challenging airway may take a longer time, with planning and execution being more important markers of proficiency. Using time as a surrogate marker for proficiency has been successful in other JiT training studies of surgical procedures; however, this would be comparing hours and minutes, rather than seconds. Proven alternatives for short procedures that are robust measures of proficiency are checklists or validated performance scales. ⁶

While no statistically or clinically relevant differences for the outcome measures were found, most of the intervention group felt training had improved their performance. If the control and intervention group had been more evenly matched in terms of skill and experience, then perhaps separation between the group results may have been more likely. More research is needed to perform statistically sound hypothesis testing and address the confounded data. Future research could also consider exploring matched-pair, cluster-randomisation for group allocation, to aid in protecting against confounded groups.

Table 1.

Group characteristics.

	Control group	Intervention group
Previous nasal intubations, mean (SD)	13.67 (17.66)	4.75 (4.94)
Number of intubations in last year, mean (SD)	4.67 (5.38)	3.33 (4.05)
Months since last nasal intubation, mean (SD)	6.50 (4.60)	17.67 (21.27)
Level of training (1–5), mean (SD)	1.83 (0.93)	1.50 (0.52)
Mean (SD) time (s) to intubation	197.56 s (135.87)	217.92 s (144.04)
Trainee confidence score, mean (SD)	21.42 (2.61)	21.17 (4.32)
Consultant took over (%)	17%	42%

SD, standard deviation