Artificial intelligence for assistance of radiology residents in chest CT evaluation for COVID-19 pneumonia: a comparative diagnostic accuracy study

Abstract

Background

In hospitals, it is crucial to rule out coronavirus disease 2019 (COVID-19) timely and reliably. Artificial intelligence (AI) provides sufficient accuracy to identify chest computed tomography (CT) scans with signs of COVID-19.

Purpose

To compare the diagnostic accuracy of radiologists with different levels of experience with and without assistance of AI in CT evaluation for COVID-19 pneumonia and to develop an optimized diagnostic pathway.

Material and Methods

The retrospective, single-center, comparative case-control study included 160 consecutive participants who had undergone chest CT scan between March 2020 and May 2021 without or with confirmed diagnosis of COVID-19 pneumonia in a ratio of 1:3. Index tests were chest CT evaluation by five radiological senior residents, five junior residents, and an AI software. Based on the diagnostic accuracy in every group and on comparison of groups, a sequential CT assessment pathway was developed.

Results

Areas under receiver operating curves were 0.95 (95% confidence interval [CI]=0.88–0.99), 0.96 (95% CI=0.92–1.0), 0.77 (95% CI=0.68–0.86), and 0.95 (95% CI=0.9–1.0) for junior residents, senior residents, AI, and sequential CT assessment, respectively. Proportions of false negatives were 9%, 3%, 17%, and 2%, respectively. With the developed diagnostic pathway, junior residents evaluated all CT scans with the support of AI. Senior residents were only required as second readers in 26% (41/160) of the CT scans.

Conclusion

AI can support junior residents with chest CT evaluation for COVID-19 and reduce the workload of senior residents. A review of selected CT scans by senior residents is mandatory.