Comparison and verification of two deep learning models for the detection of chest CT rib fractures

Abstract

Background

A high false-positive rate remains a technical glitch hindering the broad spectrum of application of deep-learning-based diagnostic tools in routine radiological practice from assisting in diagnosing rib fractures.

Purpose

To examine the performance of two versions of deep-learning-based software tools in aiding radiologists in diagnosing rib fractures on chest computed tomography (CT) images.

Material and Methods

In total, 123 patients (708 rib fractures) were included in this retrospective study. Two groups of radiologists with different experience levels retrospectively reviewed images for rib fractures in the concurrent mode aided with RibFrac-High Sensitivity (HS) and RibFrac-High Precision (HP). We compared their diagnostic performance against the reference standard in terms of sensitivity and positive predictive value (PPV).

Results

On a per-patient basis, RibFrac-HS exhibited a higher sensitivity compared with RibFrac-HP (mean difference=0.051, 95% CI=0.012–0.090; P = 0.011), whereas the latter significantly outperformed the former in terms of the PPV (mean difference=0.273, 95% CI=0.238–0.308; P < 0.0001). The use of RibFrac-HP significantly improved the junior and the senior groups’ sensitivities respectively by 0.058 (95% CI=0.033–0.083; P < 0.0001) and 0.058 (95% CI=0.034–0.081; P < 0.0001), and decreased the diagnosis time by 206 s (95% CI=191–220; P < 0.0001) and 79 s (95% CI=67–92; P < 0.0001), respectively, when compared to no software assistance.

Conclusion

The sensitivity and efficiency of radiologists in identifying rib fractures can be improved by using RibFrac-HS and/or RibFrac-HP. With an added module for false-positive suppression, RibFrac-HP maintains the sensitivity and increases the PPV in fracture detection compared to Rib-Frac-HS.

Keywords

Rib fracture artificial intelligence computed tomography false positives computer-aided diagnosis

Get full access to this article

View all access options for this article.

References

Gage

Rivara

Wang

, et al. The effect of epidural placement in patients after blunt thoracic trauma. J Trauma Acute Care Surg 2014;76:39–45; discussion 45–36.

Lin

Tung

, et al. Morbidity, mortality, associated injuries, and management of traumatic rib fractures. J Chin Med Assoc 2016;79:329–334.

Flagel

Luchette

Reed

, et al. Half-a-dozen ribs: the breakpoint for mortality. Surgery 2005;138:717–723; discussion 723–715.

Barnea

Kashtan

Skornick

, et al. Isolated rib fractures in elderly patients: mortality and morbidity. Can J Surg 2002;45:43–46.

Huang

Liu

Wang

, et al. Rectifying supporting regions with mixed and active supervision for rib fracture recognition. IEEE Trans Med Imaging 2020;39:3843–3854.

Meng

Wang

, et al. A fully automated rib fracture detection system on chest CT images and its impact on radiologist performance. Skeletal Radiol 2021;50:1821–1828.

Shelmerdine

Langan

Hutchinson

, et al. Chest radiographs versus CT for the detection of rib fractures in children (DRIFT): a diagnostic accuracy observational study. The Lancet Child & Adolescent Health 2018;2:802–811.

Wang

Dong

Pan

, et al. Machine vision-based monitoring methodology for the fatigue cracks in U-Rib-to-deck weld seams. IEEE Access 2020;8:94204–94219.

Chai

Qian

, et al. Development and evaluation of a deep learning algorithm for rib segmentation and fracture detection from multicenter chest CT images. Radiol Artif Intell 2021;3:e200248.

10.

Tajbakhsh

Suzuki

. Comparing two classes of end-to-end machine-learning models in lung nodule detection and classification: MTANNs vs. CNNs. Pattern Recognit 2017;63:476–486.

11.

Goldenberg

Nir

Salcudean

. A new era: artificial intelligence and machine learning in prostate cancer. Nat Rev Urol 2019;16:391–403.

12.

Rodríguez-Ruiz

Krupinski

Mordang

J-J

, et al. Detection of breast cancer with mammography: effect of an artificial intelligence support system. Radiology 2019;290:305–314.

13.

Chung

Han

Lee

, et al. Automated detection and classification of the proximal humerus fracture by using deep learning algorithm. Acta Orthop 2018;89:468–473.

14.

Badgeley

Zech

Oakden-Rayner

, et al. Deep learning predicts hip fracture using confounding patient and healthcare variables. NPJ Digit Med 2019;2:1–10.

15.

Gan

Lin

, et al. Artificial intelligence detection of distal radius fractures: a comparison between the convolutional neural network and professional assessments. Acta Orthop 2019;90:394–400.

16.

Zhou

Q-Q

Wang

Tang

, et al. Automatic detection and classification of rib fractures on thoracic CT using convolutional neural network: accuracy and feasibility. Korean J Radiol 2020;21:869.

17.

Mauf

Held

Gascho

, et al. Flat chest projection in the detection and visualization of rib fractures: a cross-sectional study comparing curved and multiplanar reformation of computed tomography images in different reader groups. Forensic Sci Int 2019;303:109942.

18.

Ronneberger

Fischer

Brox

. U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention . Berlin: Springer, 2015, pp. 234–241.

19.

Zhang

Ren

, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. New York, NY: IEEE, 2016, pp. 770–778.

20.

Paszke

Gross

Massa

, et al. Pytorch: an imperative style, high-performance deep learning library. In: International Conference on Neural Information Processing Systems. CA: Neural Information Processing Systems (NIPS), 2019, pp. 32.

21.

Zhang

Jia

, et al. Improving rib fracture detection accuracy and reading efficiency with deep learning-based detection software: a clinical evaluation. Br J Radiol 2021;94:20200870.

22.

Weikert

Noordtzij

Bremerich

, et al. Assessment of a deep learning algorithm for the detection of Rib fractures on whole-body trauma computed tomography. Korean J Radiol 2020;21:891–899.

23.

Jin

Yang

Kuang

, et al. Deep-learning-assisted detection and segmentation of rib fractures from CT scans: development and validation of FracNet. EBioMedicine 2020;62:103106.