Basic echocardiography competence program in intensive care units: A multinational survey of intensive care units accredited by the College of Intensive Care Medicine

Abstract

In 2014, basic critical care echocardiography (BCCE) competence became a mandatory requirement for trainees registered with the College of Intensive Care Medicine (CICM). To determine the proportion of CICM intensive care units (ICUs) that conduct a BCCE competence program and to learn about the barriers/challenges and successful strategies, we conducted a survey of intensivists working in ICUs accredited by CICM for basic/advanced training in Australia, New Zealand, Hong Kong, Singapore, Ireland and India. Following consultations with content experts and a trial phase to improve clarity and minimise ambiguity, an 11-point questionnaire survey was sent to one intensivist from every CICM-accredited ICU by several methods. Participation was voluntary. Consent was implied. No incentives were offered. Results are reported as numbers and percentages. Of the 104 ICUs surveyed, 99 (95.1%) responded, with 75 (75.8%) having no BCCE teaching whatsoever. In the remaining 24 (24.2%) ICUs, the teaching process was widely variable. Only 5/99 (5.1%) ICUs provided a structured BCCE competence program through which trainees performed and archived BCCE scans, maintained a logbook and underwent formative and summative assessments for credentialling. Six more ICUs provided formative assessment but relied on external bodies for competence assessment. Overall, 20/99 (20.2%) ICUs allowed trainees to perform unsupervised scans for clinical management, even if they were not BCCE competent. Nineteen intensivists perceived management errors due to misinterpretation of echocardiographic findings. Very few CICM-accredited ICUs offer a structured BCCE competence program. To fulfil the objective of universal BCCE competence, potential solutions are presented.

Introduction

Basic critical care echocardiography (BCCE) has been recommended as an important modality to evaluate patients in acute circulatory failure. ¹ Consensus recommendations on BCCE training advocate the performance of at least 30 fully mentored transthoracic echocardiography (TTE) examinations to attain competence. ² In 2014, BCCE competence became a mandatory requirement for trainees registered with the College of Intensive Care Medicine (CICM), which is the body responsible for intensive care medicine specialist training and education in Australia and New Zealand. The CICM-mandated BCCE competence pathway involves attending a BCCE introductory course, performing and being ‘signed off’ for 30 scans and passing an online examination. ³ To date, only 14/172 (8.1%) registered trainees have attained competence (personal communication from CICM), the reasons for which have not been evaluated. Internationally, the lack of structured workplace-based ultrasound teaching has been recognised as one of the barriers for competence.^4–6 There are no data regarding BCCE competence programmes available in CICM-accredited intensive care units (ICUs). Since there are CICM-accredited training units in Australia, New Zealand, Hong Kong, Singapore, Ireland and India, we conducted a multinational survey to evaluate the status of BCCE training in these ICUs.

Methods

This was a questionnaire survey of intensivists working in CICM-accredited ICUs in Australia, New Zealand, Hong Kong, Singapore, Ireland and India. The objectives were to evaluate BCCE training and credentialling processes in these ICUs in order to determine the proportion of ICUs that offer a competence programme, and to describe the barriers and challenges in establishing a competence programme.

Details of the design and development of the survey questionnaire can be found in eTable 1 in the Supplemental Material.

The broad research topic (status of BCCE training in CICM-accredited ICUs) and the specific research question (proportion of CICM-accredited ICUs that offer a BCCE competence programme for their trainees) were decided upon following a discussion between the first author (A.R.) and experts in BCCE education. Based on the European Society of Intensive Care Medicine (ESICM) consensus guidelines ² and the American College of Chest Physicians (ACCP)/La Société de Réanimation de Langue Française (SRLF) statement on competence, ⁷ a BCCE competence programme was defined as a longitudinal programme that required a trainee to perform regular BCCE scans with a logbook/reporting form and obtain both formative and summative assessments by a supervisor. The following domains were chosen for the survey: (a) whether the ICU had a longitudinal program for BCCE competence; (b) structure of the program—initial training, number of scans to be performed, logbook and scan archive; (c) assessment process; (d) competence assessment criteria; (e) clinical use of BCCE by trainees prior to competence; and (f) challenges/barriers to establishing a BCCE program.

To design and validate the survey, we followed a standard process recommended by statistical experts. ⁸ A draft questionnaire with eight questions (version 1) was designed by the first author (A.R.) and administered to five intensivists in New South Wales, Australia, to improve the clarity and minimise ambiguity. Based on their feedback, a redrafted 11-point questionnaire (version 2) was developed and administered to a different cohort of five intensivists for validation and reliability (eTable 1 in the Supplemental Material). In both stages, the intensivists were from ICUs accredited solely for foundation training (i.e. pre-basic training). Their responses were not included in the survey results. The final version was approved by the Human Research Ethics Committee, Nepean Blue Mountains Local Health District, Australia (approval number LNR/14/NEPEAN/36).

Distribution of the questionnaire

CICM-accredited ICUs accredited for basic or advanced training were identified from the CICM website. ³ ICUs that only offered foundation training (i.e. pre-basic level) were excluded, since they are typically small ICUs, usually with a limited departmental training structure. Between July 2018 and April 2019, one intensivist from every included ICU was contacted by email/text message/telephone/WhatsApp™ to distribute the questionnaire. They were either from the principal investigator’s contacts or chosen from social/professional websites such as WhatsApp groups or Research Gate (https://www.researchgate.net). Participation was voluntary, and consent was implied. No incentives were offered. For questionnaires sent via email, up to two reminders were sent over a six-month period.

Results

Responses were elicited from 99/104 (95.1%) ICUs, 26 of which provided general or limited general training, while 49 provided some form of additional subspecialty training (including 34 in cardiothoracic intensive care). Thirteen ICUs were classified as rural.

Only 5/99 (5.1%) ICUs provided a complete, structured longitudinal program along the lines of the ESICM/ACCP/SRLF guidelines. These ICUs expected their trainees to perform BCCE scans regularly, archive them for future scrutiny, maintain a logbook and undergo formative and summative assessments before being credentialled (Table 1). Six more ICUs provided all of these aspects of formative assessment but relied on external bodies for competence assessment—either CICM or the Australasian Society for Ultrasound in Medicine for the Certificate in Clinician Performed Ultrasound (CCPU). In this group of 11 ICUs, the trainees’ practical echocardiography skills were assessed by direct observation by an accredited supervisor (five ICUs), cardiac sonographer (three ICUs) or by a review of the trainee’s cases (three ICUs). Six ICUs provided their trainees protected teaching time for echocardiography training during clinical working hours, while five ICUs expected trainees to utilise non-clinical time.

Table 1.

Summary table with details of BCCE teaching program in 24 intensive care units.

HospitalNumber	All CICMtraineesinvited?	Introductoryworkshop?	Traineesexpected toscan patients?	How manyscans?	Logbook?	Scan archive?	Scrutinyof scans (formativeassessment)?	Certification process(summative assessment)?	Methodof competencyassessment
1	Yes	Yes	Yes	30	Yes	Yes	Yes	Yes	Subj
2	Yes	Yes	Yes	30	Yes	Yes	Yes	Yes	Subj plus MCQ
3	Yes	Yes	Yes	30	Yes	Yes	Yes	Yes	OSCE
4	Yes	No	Yes	CB	Yes	Yes	Yes	Yes	Subj
5	No	Yes	Yes	30	Yes	Yes	Yes	Yes	Subj
6	Yes	Yes	Yes	30	Yes	Yes	Yes	No	Subj
7	No	Yes	No	–	No	No	Yes	No	–
8	Yes	Yes	No	–	No	No	No	No	–
9	Yes	Yes	No	–	TB	Yes	No	No	–
10	Yes	Yes	No	–	TB	Yes	No	No	–
11	Yes	Yes	No	–	No	Yes	No	No	–
12	Yes	Yes	No	–	Yes	Yes	Yes	No	Subj
13	Yes	Yes	No	–	No	No	No	No	–
14	Yes	Yes	No	–	No	Yes	No	No	–
15	No	Yes	No	–	No	No	No	No	–
16	Yes	Yes	No	–	No	No	No	No	–
17	Yes	Yes	No	–	TB	Yes	No	No	–
18	Yes	Yes	No	–	TB	Yes	No	No	–
19	No	Yes	Yes	30	Yes	Yes	Yes	No	Subj
20	No	Yes	Yes	30	Yes	Yes	Yes	No	–
21	No	Yes	Yes	30	Yes	Yes	Yes	No	Subj
22	No	Yes	No	–	TB	Yes	No	No	–
23	Yes	Yes	No	–	No	No	No	No	–
24	Yes	Yes	No	–	No	No	No	No	–
Summary(n/24, %)	•Yes=17, 70.8%•No=7, 29.2%	•Yes=23, 95.8%•No=1, 4.2%	•Yes=9, 37.5%•No=15, 62.5%	•30 scans=8, 33.3%•CB=1, 4.2%•Nil=15, 62.5%	•Yes=10•No=9•TB=5	•Yes=17, 70.8%•No=7, 29.2%	•Yes=11, 45.8%•No=13, 54.2%	•Yes=5, 20.8%•No=19, 79.2%	•Subj=7, 29.2%•OSCE=1, 4.2%•Subj + MCQ = 1, 4.2%•Nil=15, 62.5%

Key

CB = Competence-based

TB = Depends on trainee

Method of competence assessment

• “Subj” – Subjective work place evaluation, both real-time and review of archived scans

•  “OSCE” – Comprehensive objective structured clinical examination as per Certificate in Clinician Performed Ultrasound (CCPU) criteria

•  “MCQ” – Multiple choice question

Of the 99 ICUs, 75 (75.8%) did not conduct any BCCE teaching, while 13 ICUs conducted one- to two-day introductory short courses without a follow-up competence program. Common barriers to the establishment of a longitudinal competence program were lack of time for the trainer (54/99) and/or lack of trainee interest (32/99). Of the 99 ICUs, 20 (20.2%) allowed trainees to perform unsupervised scans for clinical management, even before they had attained BCCE competence.

In response to a specific question regarding errors made by trainees from erroneous interpretation of ultrasound, 19 intensivists perceived that there were diagnostic/management errors by non-competent trainees. Examples included patients with missed pericardial tamponade (with two subsequent cardiac arrests), severe right ventricular dilation in the setting of massive pulmonary embolism, left ventricular thrombus, aortic dissection and flail mitral valve leaflet. There were also instances of misdiagnosing a mirror-image artefact as a large pleural effusion (resulting in an unnecessary pleurocentesis) and misdiagnosing the Eustachian valve for a right atrial thrombus. The survey responses did not mention any diagnostic errors made by competent trainees.

Discussion

This survey showed that only a quarter of CICM-accredited ICUs conducted any form of BCCE teaching. Even amongst these ICUs, most programs were limited to introductory courses and/or unstructured teaching. Lack of time and lack of trainee motivation were cited by many respondents as barriers to establishing a program.

The lack of training structure was often coupled with a lack of oversight, with several ICUs permitting non-competent trainees to perform unsupervised scans for the clinical management of patients. Many of these scans were not stored for future quality assurance. With 19 intensivists reporting serious diagnostic errors made by BCCE non-competent trainees, this practice may have both clinical ⁹ and medicolegal implications.^10–12

In contrast, in the 11 ICUs that conducted a longitudinal competence program, the differentiating factor appeared to be the involvement of a committed trainer, typically an intensivist with advanced echocardiography credentials. An alternative approach by three ICUs was to employ a part-time or full-time cardiac sonographer to help mentor trainees, funded either by the department (two ICUs) or by a fee paid by the trainees (one ICU). Trainee motivation and engagement were sustained by establishing ground rules that included short-term goals and clear competence criteria, and regular interactions between the trainer and trainee.

To the best of our knowledge, this is the first study to evaluate the status of BCCE competence programs in Australasian ICUs. Its strengths are the rigorous methodology employed in designing and validating the survey tool and the high response rate. The weaknesses are those common to any survey design, that is, self-declared statements that have not been corroborated independently. Also, given that some ICUs responded several months ago, it is possible that they may have started a new training program since then. There is also potentially a degree of reporting bias due to the random selection of respondents.

With so few ICUs providing a competence program, a carefully constructed approach is required to ensure or assist the implementation of the CICM policy of mandatory BCCE competence training. The survey results suggest that the following strategies are useful in establishing a successful echocardiography competence program: the leadership of enthusiastic echocardiography trainers, utilising trainees for advanced echocardiography programs/cardiac sonographers as mentors for BCCE education and providing protected teaching time for trainees.

Following consultations with experienced critical care ultrasound educators and trainee representatives, we suggest the following options for the CICM to consider: (a) establishing statewide/nationwide networks of expert supervisors to provide online/face-to-face supervision for interested trainees; (b) identifying nodal hospitals in each region with the existing infrastructure for a regional BCCE program; and, (c) facilitating short-term BCCE boot camps in large hospitals where trainees may scan several patients with pathology under supervision. For these measures to succeed, a strong commitment to BCCE is essential.

Although specific to CICM-accredited ICUs, the results of this survey may also be applicable to other comparable jurisdictions. Therefore, other critical care societies may consider evaluating the status of their BCCE competence programs.

Conclusions

Very few CICM-accredited ICUs in Australia and other countries offer a structured BCCE competence program. To fulfil the objective of universal BCCE competence, several potential solutions are presented for the consideration of the CICM.

Supplemental Material

sj-pdf-1-aic-10.1177_0310057X20911663 - Supplemental material for Basic echocardiography competence program in intensive care units: A multinational survey of intensive care units accredited by the College of Intensive Care Medicine

Supplemental material, sj-pdf-1-aic-10.1177_0310057X20911663 for Basic echocardiography competence program in intensive care units: A multinational survey of intensive care units accredited by the College of Intensive Care Medicine by Arvind Rajamani, Sebastian Knudsen, Kerrianne Ngoc Bich Ha Huynh, Stephen Huang, Wai-Tat Wong, Iris Ting, Anthony S McLean, Johnson Chi Wa Ng, Jinal Parmar, Moushumi Salvi, Kollengode Ramanathan and On behalf of the SPARTAN Collaborative (Small Projects, Audits, Research Trials – Australia/New Zealand) in Anaesthesia and Intensive Care