Deep learning-based decision support systems require synthetic images generated by adversarial networks, which require clinical evaluation to ensure their quality.
Objective
The study evaluates perceived realism of high-dimension synthetic spine fracture CT images generated Progressive Growing Generative Adversarial Networks (PGGANs).
Method: The study used 2820 spine fracture CT images from 456 patients to train an PGGAN model. The model synthesized images up to 512 × 512 pixels, and the realism of the generated images was assessed using Visual Turing Tests and Fracture Identification Test. Three spine surgeons evaluated the images, and clinical evaluation results were statistically analysed.
Result: Spine surgeons have an average prediction accuracy of nearly 50% during clinical evaluations, indicating difficulty in distinguishing between real and generated images. The accuracy varies for different dimensions, with synthetic images being more realistic, especially in 512 × 512-dimension images. During FIT, among 16 generated images of each fracture type, 13–15 images were correctly identified, indicating images are more realistic and clearly depict fracture lines in 512 × 512 dimensions.
Conclusion
The study reveals that AI-based PGGAN can generate realistic synthetic spine fracture CT images up to 512 × 512 pixels, making them difficult to distinguish from real images, and improving the automatic spine fracture type detection system.
ChartrandGChengPMVorontsovE, et al.Deep learning: a primer for radiologists. Radiographics2017; 37: 2113–2131.
2.
KerJWangLRaoJ, et al.Deep learning applications in medical image analysis. Ieee Access2017; 6: 9375–9389.
3.
LeeJGJunSChoYW, et al.Deep learning in medical imaging: general overview. Korean J Radiol2017; 18: 570–584.
4.
MiottoRWangFWangS, et al.Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform2018; 19: 1236–1246.
5.
SorinVBarashYKonenE, et al.Creating artificial images for radiology applications using generative adversarial networks (GANs)–a systematic review. Acad Radiol2020; 27: 1175–1185.
6.
KazeminiaSBaurCKuijperA, et al.GANs for medical image analysis. Artif Intell Med2020; 109: 101938.
7.
KarrasTAilaTLaineS, et al.Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196. 2017 Oct 27.
8.
Frid-AdarMKlangEAmitaiM, et al.Synthetic data augmentation using GAN for improved liver lesion classification. In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018) 2018 Apr 4. IEEE, pp.289–293.
9.
RubinMSteinOTurkoNA, et al.TOP-GAN: stain-free cancer cell classification using deep learning with a small training set. Med Image Anal2019; 57: 176–185.
10.
ZhaoHLiHMaurer-StrohS, et al.Synthesizing retinal and neuronal images with generative adversarial nets. Med Image Anal2018; 49: 14–26.
11.
BrownJMCampbellJPBeersA, et al.Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. JAMA Ophthalmol2018; 136: 803–810.
12.
BurlinaPPachecoKDJoshiN, et al.Comparing humans and deep learning performance for grading AMD: a study in using universal deep features and transfer learning for automated AMD analysis. Comput Biol Med2017; 82: 80–86.
13.
TingDSCheungCYLimG, et al.Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. Jama2017; 318: 2211–2223.
14.
YiXWaliaEBabynP. Generative adversarial network in medical imaging: a review. Med Image Anal2019; 58: 101552.
15.
BaurCAlbarqouniSNavabN. MelanoGANs: high resolution skin lesion synthesis with GANs. arXiv preprint arXiv:1804.04338. 2018 Apr 12.
16.
MahapatraDAntonyBSedaiS, et al.Deformable medical image registration using generative adversarial networks. In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018) 2018 Apr 4. IEEE, pp.1449–1453.
17.
NieDTrulloRLianJ, et al.Medical image synthesis with deep convolutional adversarial networks. IEEE Trans Biomed Eng2018; 65: 2720–2730.
18.
GulakalaRMarkertBStoffelM. Rapid diagnosis of COVID-19 infections by a progressively growing GAN and CNN optimisation. Comput Methods Programs Biomed2023; 229: 107262.
19.
KumarAJChongRSCrowstonJG, et al.Evaluation of generative adversarial networks for high-resolution synthetic image generation of circumpapillary optical coherence tomography images for glaucoma. JAMA Ophthalmol2022; 140: 974–981.
20.
AbdelhalimISMohamedMFMahdyYB. Data augmentation for skin lesion using self-attention based progressive generative adversarial network. Expert Syst Appl2021; 165: 113922.
21.
KimMKimSKimM, et al.Realistic high-resolution lateral cephalometric radiography generated by progressive growing generative adversarial network and quality evaluations. Sci Rep2021; 11: 12563.
22.
TeramotoATsukamotoTYamadaA, et al.Deep learning approach to classification of lung cytological images: two-step training using actual and synthesized images by progressive growing of generative adversarial networks. PloS one2020; 15: e0229951.
23.
YooTKChoiJYKimHK, et al.Adopting low-shot deep learning for the detection of conjunctival melanoma using ocular surface images. Comput Methods Programs Biomed2021; 205: 106086.
24.
ZhengCBianFLiL, et al.Assessment of generative adversarial networks for synthetic anterior segment optical coherence tomography images in closed-angle detection. Transl Vis Sci Technol2021; 10: 34.
25.
IqballTWaniMA.Generative adversarial networks for data augmentation in X-ray medical imaging. In: Generative adversarial learning: architectures and applications 2022 Feb 7. Cham: Springer International Publishing, pp.341–355.
26.
Gab AllahAMSarhanAMElshennawyNM. Classification of brain MRI tumor images based on deep learning PGGAN augmentation. Diagnostics2021; 11: 2343.
27.
GadermayrMLiKMüllerM, et al.Domain-specific data augmentation for segmenting MR images of fatty infiltrated human thighs with neural networks. J Magn Reson Imaging2019; 49: 1676–1683.
28.
KazuhiroKWernerRAToriumiF, et al.Generative adversarial networks for the creation of realistic artificial brain magnetic resonance images. Tomography2018; 4: 159–163.
29.
RussTGoerttlerSSchnurrAK, et al.Synthesis of CT images from digital body phantoms using CycleGAN. Int J Comput Assist Radiol Surg2019; 14: 1741–1750.
30.
Ben-CohenAKlangERaskinSP, et al.Cross-modality synthesis from CT to PET using FCN and GAN networks for improved automated lesion detection. Eng Appl Artif Intell2019; 78: 186–194.
31.
DarSUYurtMKaracanL, et al.Image synthesis in multi-contrast MRI with conditional generative adversarial networks. IEEE Trans Med Imaging2019; 38: 2375–2388.
32.
JiangJHuYCTyagiN, et al.Cross-modality (CT-MRI) prior augmented deep learning for robust lung tumor segmentation from small MR datasets. Med Phys2019; 46: 4392–4404.
33.
LeiYHarmsJWangT, et al.MRI-Only based synthetic CT generation using dense cycle consistent generative adversarial networks. Med Phys2019; 46: 3565–3581.
34.
VitaleSOrlandoJIIarussiE, et al.Improving realism in patient-specific abdominal ultrasound simulation using CycleGANs. Int J Comput Assist Radiol Surg2020; 15: 183–192.
35.
DongXLeiYWangT, et al.Automatic multiorgan segmentation in thorax CT images using U-net-GAN. Med Phys2019; 46: 2157–2168.
36.
LiuXGuoSZhangH, et al.Accurate colorectal tumor segmentation for CT scans based on the label assignment generative adversarial network. Med Phys2019; 46: 3532–3542.
37.
SeahJCTangJSKitchenA, et al.Chest radiographs in congestive heart failure: visualizing neural network learning. Radiology2019; 290: 514–522.
38.
XueYXuTZhangH, et al.Segan: adversarial network with multi-scale l 1 loss for medical image segmentation. Neuroinformatics2018; 16: 383–392.
39.
KangEKooHJYangDH, et al.Cycle-consistent adversarial denoising network for multiphase coronary CT angiography. Med Phys2019; 46: 550–562.
40.
KimKHDoWJParkSH. Improving resolution of MR images with an adversarial network incorporating images with different contrast. Med Phys2018; 45: 3120–3131.
41.
LiangXChenLNguyenD, et al.Generating synthesized computed tomography (CT) from cone-beam computed tomography (CBCT) using CycleGAN for adaptive radiation therapy. Physics in Medicine & Biology2019; 64: 125002.
42.
YouCYangQShanH, et al.Structurally-sensitive multi-scale deep neural network for low-dose CT denoising. IEEE access2018; 6: 41839–41855.
43.
SchleglTSeeböckPWaldsteinSM, et al.f-AnoGAN: fast unsupervised anomaly detection with generative adversarial networks. Med Image Anal2019; 54: 30–44.
44.
ParkHYBaeHJHongGS, et al.Realistic high-resolution body computed tomography image synthesis by using progressive growing generative adversarial network: visual turing test. JMIR Med Inform2021; 9: e23328.
45.
HigakiAKawadaYHiasaG, et al.Using a visual turing test to evaluate the realism of generative adversarial network (GAN)-based synthesized myocardial perfusion images. Cureus2022; 14: e30646.
46.
ChuquicusmaMJHusseinSBurtJ, et al.How to fool radiologists with generative adversarial networks? A visual turing test for lung cancer diagnosis. In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018) 2018 Apr 4. IEEE, pp.240–244.
47.
HanCHayashiHRundoL, et al.GAN-based synthetic brain MR image generation. In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018) 2018 Apr 4. IEEE, pp.734–738.
48.
SindhuraDPaiRMBhatSN, et al.Sub-Axial vertebral column fracture CT image synthesis by progressive growing generative adversarial networks (PGGANs). In: 2022 International conference on distributed computing, VLSI, electrical circuits and robotics (DISCOVER) 2022 Oct 14. IEEE, pp.311–315.
