Quick and accurate selection of hand images among radiographs from various body parts using deep learning

Abstract

BACKGROUND:

Although rheumatoid arthritis (RA) causes destruction of articular cartilage, early treatment significantly improves symptoms and delays progression. It is important to detect subtle damage for an early diagnosis. Recent software programs are comparable with the conventional human scoring method regarding detectability of the radiographic progression of RA. Thus, automatic and accurate selection of relevant images (e.g. hand images) among radiographic images of various body parts is necessary for serial analysis on a large scale.

OBJECTIVE:

In this study we examined whether deep learning can select target images from a large number of stored images retrieved from a picture archiving and communication system (PACS) including miscellaneous body parts of patients.

METHODS:

We selected 1,047 X-ray images including various body parts and divided them into two groups: 841 images for training and 206 images for testing. The training images were augmented and used to train a convolutional neural network (CNN) consisting of 4 convolution layers, 2 pooling layers and 2 fully connected layers. After training, we created software to classify the test images and examined the accuracy.

RESULTS:

The image extraction accuracy was 0.952 and 0.979 for unilateral hand and both hands, respectively. In addition, all 206 test images were perfectly classified into unilateral hand, both hands, and the others.

CONCLUSIONS:

Deep learning showed promise to enable efficiently automatic selection of target X-ray images of RA patients.

Keywords

Rheumatoid arthritis deep learning image classification convolutional neural network

Get full access to this article

View all access options for this article.

References

Tutuncu

and Kavanaugh

, Rheumatic disease in the elderly: rheumatoid arthritis, Rheum Dis Clin North Am 33 (2007), 57–70.

Kobak

and Bes

, An autumn tale: geriatric rheumatoid arthritis, Adv Musculoskelet Dis 10 (2018), 3–11.

Karsdal

M.A.

, Woodworth

, Henriksen

, et al., Biochemical markers of ongoing joint damage in rheumatoid arthritis–current and future applications, limitations and opportunities, Arthritis Res Ther 13 (2011), 215.

Schett

and Gravallese

, Bone erosion in rheumatoid arthritis: mechanisms, diagnosis and treatment, Nat Rev Rheumatol 8 (2012), 656–664.

Aletaha

and Smolen

J.S.

, Diagnosis and Management of Rheumatoid Arthritis: A Review, JAMA 320 (2018), 1360–1372.

van der Heijde

, How to read radiographs according to the Sharp/van der Heijde method, J Rheumatol 27 (2000), 261–263.

Duryea

, Jiang

, Zakharevich

and Genant

H.K.

, Neural network based algorithm to quantify joint space width in joints of the hand for arthritis assessment, Med Phys 27 (2000), 1185–1194.

Huo

, Vincken

K.L.

, van der Heijde

, et al., Automatic Quantification of Radiographic Wrist Joint Space Width of Patients With Rheumatoid Arthritis, IEEE Trans Biomed Eng 64 (2017), 2695–2703.

, Ambalathankandy

, Shimada

, et al., Automatic Radiographic Quantification of Joint Space Narrowing Progression in Rheumatoid Arthritis Using POC. IEEE 16th International Symposium on Biomedical Imaging April 8 –April 11, 2019 Venice.

10.

Kato

, Sutherland

, Tanaka

, et al., Fully automatic quantitative software for assessment of minute finger joint space narrowing progression on radiographs: evaluation in rheumatoid arthritis patients with long-term sustained clinical low disease activity. Jpn J Radiol (2020).

11.

Kato

, Yasojima

, Tamura

, et al., Detection of Fine Radiographic Progression in Finger Joint Space Narrowing Beyond Human Eyes: Phantom Experiment and Clinical Study with Rheumatoid Arthritis Patients, Scientific reports 9 (2019), 1–10.

12.

Okino

, Kamishima

, Sutherland

, et al., Radiographic temporal subtraction analysis can detect finger joint space narrowing progression in rheumatoid arthritis with clinical low disease activity, radiologica 59 (2018), 460–467.

13.

Yasaka

, Akai

, Kunimatsu

, Kiryu

and Abe

, Deep learning with convolutional neural network in radiology, J Radiol 36 (2018), 257–272.

14.

Morisi

, Manners

D.N.

, Gnecco

, et al., Multi-class parkinsonian disorders classification with quantitative MR markers and graph-based features using support vector machines, Relat Disord 47 (2018), 64–70.

15.

Zhu

, Li

and Wang

, Design and experiment verification of a novel analysis framework for recognition of driver injury patterns: From a multi-class classification perspective, Anal Prev 120 (2018), 152–164.

16.

Wang

J.J.

, Hu

S.G.

, Zhan

X.T.

, et al., Handwritten-Digit Recognition by Hybrid Convolutional Neural Network based on HfO2 Memristive Spiking-Neuron, Rep 8 (2018), 12546.

17.

, Yin

, Wang

and Xu , Face Detection With Different Scales Based on Faster R-CNN, IEEE Trans Cybern 49 (2019), 4017–4028.

18.

LeCun

, Bengio

and Hinton

, Deep learning, Nature 521 (2015), 436–444.

19.

Ilyas

, Shahzad

and Kim

, Convolutional-Neural Network-Based Image Crowd Counting: Review, Categorization, Analysis, and Performance Evaluation, Sensors (Basel) 20 (2019), 43.

20.

Chartrand

, Cheng

P.M.

, Vorontsov

, et al., Deep Learning: A Primer for Radiologists, Radiographics 37 (2017), 2113–2131.

21.

Bayar

and Stamm

, A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer, Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security June 20, 2016 Vigo.

22.

Glorot

and Bengio

, Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics (2010), pp. 249–256.

23.

Soffer

, Ben-Cohen

, Shimon

, et al., Convolutional Neural Networks for Radiologic Images: A Radiologist’s Guide, Radiology 290 (2019), 590–606.

24.

Abbas

and Khan

N.A.

, DeepMI: Deep Learning for Multiclass Motor Imagery Classification, Conf Proc IEEE Eng Med Biol Soc 2018 (2018), 219–222.

25.

Munir

, Elahi

, Ayub

, et al., Cancer Diagnosis Using Deep Learning: A Bibliographic Review, Cancers (Basel) 11 (2019), 1235.

26.

Gupta

, Harrison

P.J.

, Wieslander

, et al., Deep Learning in Image Cytometry: A Review, A 95 (2019), 366–380.

27.

Anwar

S.M.

, Majid

, Qayyum

, et al., Medical Image Analysis using Convolutional Neural Networks: A Review, J Med Syst 42 (2018), 226.

28.

, Zhang

and Li

, Study of the Application of Deep Convolutional Neural Networks (CNNs) in Processing Sensor Data and Biomedical Images, Sensors (Basel) 19 (2019), 3584.

29.

Luckehe

and von Voigt

, Evolutionary image simplification for lung nodule classification with convolutional neural networks, J Comput Assist Radiol Surg 13 (2018), 1499–1513.

30.

Ahmad

, Muhammad

and Baik

S.W.

, Data augmentation-assisted deep learning of hand-drawn partially colored sketches for visual search, PLoS One 12 (2017), e0183838.

31.

Rajkomar

, Lingam

, Taylor

A.G.

, et al., High-Throughput Classification of Radiographs Using Deep Convolutional Neural Networks, J Digit Imaging 30 (2017), 95–101.

32.

Kim

T.K.

, Yi

P.H.

, Wei

, et al., Deep Learning Method for Automated Classification of Anteroposterior and Posteroanterior Chest Radiographs, J Digit Imaging 32 (2019), 925–930.

33.

Hussain

, Gimenez

, Yi

and Rubin

, Differential Data Augmentation Techniques for Medical Imaging Classification Tasks, AMIA Annu Symp Proc 2017 (2017), 979–984.