Abstract
Purpose:
Assess quality metrics of modified barium swallow studies (MBSS) with and without a radiologist present during the procedure.
Methods:
Retrospective review of MBSS performed on adult inpatients at a tertiary care hospital 6-months pre- and post-institutional change from having to not having a radiologist present during the examination.
Factors assessed included:
fluoroscopy time; study duration; number of cine loops; number of images; efficiency of collimation (using a 5-point scoring system); time to final report; radiologist-speech language pathologist report disagreement; and recalls for inadequate studies. Statistical analysis was via Welch’s t-test and a test of proportions for continuous and count data under the normal approximation.
Results:
106 and 119 MBSS were analyzed from the radiologist present and radiologist absent periods, respectively. No statistically significant differences were found for: average fluoroscopy time (116.1 s vs. 126.9 s; P = 0.161); study duration (400.4 s vs. 417.3 s; P = 0.453); number of cine loops (9.3 vs. 10.2; P = 0.075); number of images (620.5 vs. 581.1; P = 0.350); or report disagreement. There was improved performance without the radiologist present for collimation (1.92 vs. 1.43; P = 0.003) and fewer non-diagnostic images (6.5 vs. 4.5; P = 0.001). Time to final report was longer with the radiologist absent due to more reports with significant delays. There were no repeated studies because of inadequate technique in either group.
Conclusion:
MBSS performed by technologists without radiologist supervision is not inferior to those performed with radiologist supervision on multiple performance measures. This supports technologist operated MBSS without radiologist supervision, while acknowledging a need to further address radiologist report time delay.
Introduction
The modified barium swallow study (MBSS) has been available since the 1950s and involves fluoroscopic assessment of barium ingestion using various consistencies to evaluate swallowing.1,2 It is one of the most commonly performed swallowing assessments in Canada and has been validated with published evidence-based guidelines for its performance and interpretation.3-5 Traditionally, radiologists and speech-language pathologists (SLPs) collaborate closely during image acquisition. The radiologist provides supervision and anatomic assessment, while the SLP focuses on the swallowing mechanism.
Although many MBSS are still performed with radiologist supervision, several facilities are now performing the procedure without the radiologist present with study review and reporting at a later time. 6 This change in practice poses the question as to whether study quality and/or patient care is being compromised. A 2018 quality and safety case study by the American College of Radiology denounced the breakdown of the collaborative approach in the case of MBSS and calls for its reinstitution. 7 However, the existing literature lacks evidence supporting or opposing the expressed opinions in the said study. As more centers move toward radiologist-absent MBSS, data is needed to validate the quality and safety of practice in this new environment.
A recent change at our institution, from radiologist supervised to non-radiologist supervised MBSS, presents the opportunity to clarify the impact on quality metrics. The objective of this study is to assess the diagnostic quality and radiation safety of MBSS performed pre- and post-institutional change from radiologist present to absent, while technologists and SLPs remained present throughout. These metrics include: fluoroscopy time; examination duration; anatomical structures of irradiation; number of cine loops and images acquired; reporting delay; diagnostic discrepancies between radiologists and SLPs; and repeat examinations within 2 weeks. We hypothesized that MBSS performed without radiologist supervision are not inferior to those performed with radiologist supervision on multiple metrics.
Methods
This study was approved by our institution’s Research Ethics Board. This was a pre- and post-observational study consisting of retrospective reviews of the periods before and after an institutional change from radiologist supervised to non-supervised MBSS at a single fluoroscopic suite in an academic tertiary care hospital. A convenience sample utilizing records from consecutive MBSS performed 6 months prior (March 1, 2019 to August 31, 2019) and 6 months after (October 1, 2019 to March 31, 2020) were reviewed. The month of change (September 2019) was excluded to allow adjustment to the new practice. The inclusion criteria were inpatient adults 18 years of age or older performed at a single fluoroscopy suite. Studies were excluded if the inclusion criteria were not met, contained images of non-MBSS, or records of the study were incomplete. If SLP reports were missing, those studies were excluded from the report agreement part of the study but remained included for assessment of other metrics.
mPower™ Clinical Analytics (Nuance Communications; Burlington, MA) was queried to identify a list of patients and radiology reports of MBSS from the above stated periods. Patient demographics, number of cine loops and images acquired, and final radiologist reports were extracted from IntelliSpace Picture Archiving and Communication Systems (PACS) Radiology version 4.4 (Philips Healthcare Informatics, Inc.; Pleasanton, CA). Examination duration was calculated by the time elapsed from the first to last stored images on PACS. Report delay was calculated by the time elapsed from the last stored image on PACS to the time stamp of the final radiology report on PACS. Cerner Radiology Information System (RIS) (Cerner; North Kansas City, MO) was queried for the fluoroscopy time of each study. Final SLP reports were accessed via the patient’s electronic medical records on Sunrise Clinical Manager (SCM) Release 15.3 (Sunrise; Richmond, BC).
Images that were classified as non-diagnostic included those captured for the purpose of labeling the administered consistency, as well as single images and short cine loops (less than 5 images), with the patient in view and not in any phase of swallow. Cine loops with less than 5 images were accounted for in the total images taken but were not accounted for in the total cine loops taken.
Since MBSS assesses the essential structures involved in the oral, pharyngeal, laryngeal, and cervical esophageal swallow dynamics, 6 any head and neck structures not essential to the swallowing mechanism should ideally be collimated to reduce unnecessary radiation exposure. A 5-point collimation score was employed to quantitatively assess the irradiation of non-essential structures. A single point was awarded for the inclusion of one of the following structures: suprasellar brain; posterior elements of the spine; orbits; anterior oral cavity; and shoulders, for a potential total score of five. A collimation score was calculated for each included MBSS. Examples of the associated imaging appearance for each collimation score from 0 to 5 are provided in Figure 1.
Five-point collimation score and the associated imaging appearance on MBSS. A, 0 points; none of the 5 non-essential head and neck structures (suprasellar brain, posterior elements of the spine, orbits, anterior oral cavity, and shoulders) are included in the field of view. B, 1 point for inclusion of the posterior elements of the spine. C, 2 points for inclusion of the posterior elements of the spine and shoulders. D, 3 points for inclusion of the anterior oral cavity, posterior elements of the spine, and shoulders. E, 4 points for inclusion of the suprasellar brain, anterior oral cavity, posterior elements of the spine, and shoulders. F, 5 points for inclusion of all 5 of the non-essential head and neck structures.
The incidence of disagreement between the radiologist report and SLP report on significant swallowing pathology, (presence of aspiration and penetration), was assessed. The radiologist reports were accessed from PACS while the SLP reports were reviewed on SCM. The radiologist and SLP assessment of each administered consistency was compared. Reports were considered discordant if there was relative “upstaging” of a swallowing event. Specifically, when one assessor (radiologist or SLP) notes aspiration and the other assessor notes penetration or no abnormality, or one assessor notes penetration and the other assessor notes no abnormality. Radiologist and SLP reports were also searched for any mention of technical inadequacy or need for a repeat study.
Additionally, to assess for potential recalls because of technical limitations on the initial MBSS, PACS was searched for repeat examinations within 2 weeks of the initial exam. Those studies and reports were then examined to determine the reason for the repeat exam, specifically regarding potential recall following technical issues on the initial study.
Potential differences in the demographics and outcomes were analyzed using Welch’s t-test if the data were continuous and a test of proportions for count data under the normal approximation. Count data were described as count and proportions. All analyses were performed in the R-statistical software package (R 4.0.0, “Arbor Day”) with the R-Studio GUI (1.2.5042). The differences between the radiologist-present and the radiologist-absent group were tested using a test of proportions under the normality assumption to determine whether the presence or absence of the radiologist affected the rate of disagreement between the final SLP and radiologist reports.
Results
A total of 238 MBSS were reviewed, 110 during the radiologist present (RP) and 128 during the radiologist absent (RA) periods. Five studies were excluded because of incomplete data and 8 studies were excluded because of combination with an esophogram. This resulted in 106 and 119 studies in the RP and RA group, respectively. All studies included for review from both groups were of diagnostic quality and none were reported as non-diagnostic. The average age for each group was similar, with the RA group containing a greater proportion of females compared to the RP group. A summary of the patient demographics in each study group is shown in Table 1.
Demographics of the Radiologist Present and Radiologist Absent groups.
When comparing the RP and RA groups, the mean fluoroscopy time, average study duration, average number of cine loops per study, and average total number of images per study yielded statistically insignificant comparisons (Table 2). There was no statistical evidence that MBSS performed without radiologist supervision were inferior to studies performed with radiologist supervision on these measures.
Comparison of Study Time, Fluoroscopic Measures, Report Delay, and Collimation Scores of the Modified Barium Swallow Studies Performed in the Periods With and Without Radiologist Supervision.
Abbreviation: SD, standard deviation.
Measures that demonstrated a statistically significant difference between the RP and RA groups included the average number of non-diagnostic images taken per study, with the RA group taking fewer non-diagnostic images compared to the RP group (Table 2). The average time to finalize report was much longer for the RA group than the RP group. This was largely because there were 19 reports delayed by more than 24 hours in the RA group, versus 2 in the RP group. When these significantly delayed reports were excluded, there was no significant difference in time to final report between the groups (Table 2).
The collimation scores demonstrated statistically significant differences between the RP and RA groups with the mean score being 1.92 with RP versus 1.43 with RA (P = 0.003). Higher scores, indicating irradiation of more non-essential structures, tended to occur with the radiologist present during the study (Table 2, Figure 2).
Distribution of the 5-point collimation score in the radiologist absent (RA) and radiologist present (RP) groups. There was a greater proportion of lower scores occurring in the RA group (black bars) than the RP group (grey bars). NB: Lower score is better collimation as fewer non-essential structures are included in those examinations.
Excluding 6 studies with missing SLP reports, 300 consistencies from 104 studies and 335 consistencies from 115 studies were reviewed from the radiologist present and radiologist absent periods, respectively (Table 3). The disagreement rates between the SLP and radiologist for swallowing pathologies of aspiration and penetration were not statistically different between the 2 cohorts (Table 3). Consistencies that were ambiguous in disagreement included 9 consistencies from the radiologist present cohort where the SLP noted penetration and the radiologist noted no aspiration but did not specify penetration (4 consistencies), and consistencies that were not mentioned in the radiology report but noted to demonstrate aspiration or penetration by the SLP (5 consistencies). 5 consistencies from the radiologist absent cohort had ambiguous disagreement with 1 consistency not specifically mentioned in the radiology report and noted to have aspiration or penetration by the SLP and 4 consistencies from one study where the cine loops were out of sequence and not interpretable by the radiologist.
Speech Language Pathologist and Radiologist Report Discordance for Modified Barium Swallow Studies Performed With and Without Radiologist Supervision.
17 MBSS from the radiologist present group and 26 studies from the radiologist absent group had repeat studies performed within 2 weeks due to reasons pertaining to clinical patient care. Despite one study having the cine loops from 4 out of 5 consistencies being out of sequence, the radiologist appropriately interpreted the swallowing pathology and did not necessitate a repeat study. No studies were inconclusive or recalled in either group.
Discussion
A recent institutional change from radiologist supervised to non-supervised MBSS offered an opportunity to evaluate the potential impact on patient safety and study quality. We hypothesized that the radiologist-absent MBSS, one where image capture was performed by the radiation technologist, would not be inferior to a radiologist-present MBSS. This proved to be true via our retrospective analysis for the average fluoroscopy time, study duration, and number of cine loops and images taken per study, which did not demonstrate a statistically significant difference when radiologists were not supervising the study. This indicated that absence of the radiologist during the live study does not result in adverse effects of longer study times, more images and cine loops acquired, and greater radiation exposure for the completion of a MBSS.
Comparative analysis revealed positive differences in studies performed by radiation technologists without radiologist supervision on measures of collimation and the number of non-diagnostic images. The collimation scores of studies performed by radiation technologists demonstrated statistically significant differences with 57.9% of studies scoring 0 and 1 and 3.4% of studies scoring 4 or 5, versus only 39.6% of studies scoring 0 and 1 and as many as 13.2% of studies scoring 4 and 5 with the radiologist present. This indicated a tendency to include more non-essential head and neck structures within the irradiated field of view when the MBSS was directly supervised and/or operated by the radiologist. Radiologists also took more non-diagnostic images per MBSS versus radiation technologists, with the majority of these accounted by variable practices in the labelling of ingested consistencies. Radiation technologists consistently used a single image for labelling of ingested consistencies, whereas certain radiologists usually took cine loops for the purpose of labelling with no added diagnostic value. These differences suggest heightened radiation consciousness and/or technical diligence among the technologists.
Out of the assessed quality metrics, the only statistically significant negative difference in the group of studies performed without a radiologist was reporting lag. The time to final report generation was much lengthier in the radiologist absent group, with an average of 942 minutes, or 15.7 hours, versus 126.6 minutes, or 2.1 hours. This occurred because 16% of reports in the RA group were delayed by more than 24 hours, versus 2% in the RP group. When those delayed by more than 24 hours were excluded, there was no statistically significant difference. This may have occurred due to absent communication between technologists and radiologists or lack of a standardized system of assigning studies to the radiologist following completion. We have since endeavored to improve this communication to prevent report delays moving forward at our institution.
Statistical analysis comparing diagnostic discrepancies between the SLP and radiologist report before and after the institutional change demonstrated no significant difference in report disagreement. No study from either period was inconclusive or required patient recall for repeat imaging. Despite one study having out of sequence cine loops in the radiologist absent cohort, the radiologist appropriately interpreted the swallowing pathology and did not necessitate a repeat study. This further supports the maintenance of diagnostic performance and quality of patient care with distant radiology supervision.
This study was a pre- and post- observational study and as such did not influence the mechanics of MBSS examinations at our institution. MBSS examinations were performed collaboratively by technologists and SLPs using standard protocols of lateral videofluoroscopy of the neck during swallowing. Each image and cine loop were to be stored on PACS. General principles of as low as reasonably achievable (ALARA) radiation safety were encouraged including limiting fluoroscopy time and appropriate collimating to limit radiation to the area of clinical interest. There was no further defined protocol for either radiologists or technologists. The only defined difference was that when the radiologist conducted remote review, technologists completed a standardized worksheet in collaboration with SLPs detailing aspiration, penetration, or other concerns during each of the swallowing sequences of the various substances/ consistencies. The radiologists performing the procedure kept their own notes with no predesigned worksheet. Communication between radiologists and SLPs during the procedure often made these notes informally collaborative.
Our study is limited by several factors. First, we have taken a convenience sample, so it is possible that our comparisons were underpowered and unable to detect small effect sizes. However, given the analysis spans over a period of 1 year, the difference is assumed to be minute and likely intangible for the everyday workflow. It is worth noting that there was a skew in the gender distribution between the comparison groups, which may provide a source of bias. Further, not all images taken were fluoro-stored and thus available for analysis. However, the portion of unsaved images is usually negligible. Given that the institutional change does not limit a radiologist from voluntarily participating in a MBSS, assumptions were made that an inconsequential number of radiologists or residents continue to participate in MBSS, knowing that this source of bias would result in an underestimation, rather than overestimation of the actual effect. Additionally, study duration was calculated based on difference between first and last image stored on PACS for a given examination. Although this accurately captures the time of the active swallowing portion of the study, it would not capture differences in patient set-up or potential delays while waiting for a radiologist to join the procedure in the RP group. A future opportunity is to conduct a parallel study in the pediatric population, where issues of radiation safety and performance quality are of paramount importance.
Although this study acknowledges the non-inferiority of radiologist absent MBSS in fluoroscopy time, study duration, number of images and cine loops required, and diagnostic discrepancies, as well as superior performances in the irradiation of non-essential structures and acquisition of non-diagnostic images, it identifies an important weakness in the radiologist absent MBSS and calls attention to further address report delays.
Conclusion
Findings of this study support technologist operated MBSS without radiologist supervision, while acknowledging a need to further address radiologist report time delay.
Footnotes
Acknowledgments
We gratefully acknowledge the involvement of our speech-language-pathologists and radiation technologists in making this study possible.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
