A computational method to differentiate rheumatoid arthritis patients using thermography data

Abstract

BACKGROUND:

The traditional rheumatoid arthritis (RA) diagnosis is very complicated because it uses many clinical and image data. Therefore, there is a need to develop a new method for diagnosing RA using a consolidated set of blood analysis and thermography data.

OBJECTIVE:

The following issues related to RA are discussed: 1) Which clinical data are significant in the primary diagnosis of RA? 2) What parameters from thermograms should be used to differentiate patients with RA from the healthy? 3) Can artificial neural networks (ANN) differentiate patients with RA from the healthy?

METHODS:

The dataset was composed of clinical and thermal data from 65 randomly selected patients with RA and 104 healthy subjects. Firstly, the univariate logistic regression model was proposed in order to find significant predictors. Next, the feedforward neural network model was used. The dataset was divided into the training set (75% of data) and the test set (25% of data). The Broyden-Fletcher-Goldfarb-Shanno (BFGS) and non-linear logistic function to transformation nodes in the output layer were used for training. Finally, the 10 fold Cross-Validation was used to assess the predictive performance of the ANN model and to judge how it performs.

RESULT:

The training set consisted of the temperature of all fingers, patient age, BMI, erythrocyte sedimentation rate, C-reactive protein and White Blood Cells (10 parameters in total). High level of sensitivity and specificity was obtained at 81.25% and 100%, respectively. The accuracy was 92.86%.

CONCLUSIONS:

This methodology suggests that the thermography data can be considered in addition to the currently available tools for screening, diagnosis, monitoring of disease progression.

1. Introduction

Rheumatoid arthritis (RA) is a chronic connective tissue disease manifested by pain, swelling, stiffness of the joints and elevated temperature [1]. Clinical evaluation of this disease is based on in-depth analysis and interpretation of symptoms, laboratory parameters and medical imaging examinations [2, 3]. The commonly used parameters are the erythrocyte sedimentation rate (ESR) and C-reactive protein (CRP). However, both of the parameters are non-specific to RA [8]. Indicators of disease activity look promising. They are a combination of disease activity parameters combined in one specific measure. The most commonly used test is DAS 28 [4], which is based on the assessment of 28 joints. Recently, some authors proposed automated screening systems by applying artificial intelligence methods for RA diagnosis [5, 6, 7, 8, 9, 10]. The dataset was used to train the neural network based on the diagnosis criteria of RA, supported by image techniques [11, 12, 13]. Murakami et al. [14] proposed a new quantitative method for automatically detecting bone erosion from hand X-ray images. The selected areas of interest were analysed in terms of presence or absence of bone erosion using a classifier based on deep neural networks. The percentage of true positive results and false-positive results was 80.5% and 0.84%, respectively. Helwan et al. [15] have attempted to develop a new intelligent system for identifying rheumatoid knee arthritis using X-ray image processing techniques and a neural classifier. The identification index was 95.5%. In turn, Naz et al. [16] presented RA classification methods using neural networks based on thermal patterns obtained from passive thermography. The multi-layer feed-forward perceptron algorithm with backpropagation was used for RA recognition. However, the experimenters did not take into account any clinical parameters of RA. Moreover, Umpathy et al. [17] used a k-means algorithm and fuzzy c means algorithm to classify patients with RA and healthy subjects based on the feature extracted from the segmented thermal image. The receiver operating characteristics (ROC) curve depicted a sensitivity of 86.6% and specificity of 79% achieved in the MCP region of the thermal hand image. Thus, there is a need to develop a method for diagnosing RA using a consolidated set of data from blood analysis and thermography. Herein the following issues related to RA disease are discussed: 1) Which clinical data are significant in the primary diagnosis of RA; 2) What parameters from thermograms should be addressed in differentiation patients with RA from the healthy; 3) Can artificial neural networks (ANN) differentiate patients with RA from the healthy?

2. Methods

2.1 Subjects

The study was conducted between January 2017 and December 2018 and included 65 randomly selected patients with RA from the Department of Rheumatology and Internal Medicine (Poland) at the Medical University of Bialystok. The inclusion criteria were: age over 18 years old, disease duration over two years, biological therapy treatment, DAS28 over 2.0 and below 7.5 units at the start of biologic therapy. Exclusion criteria were as follows: age under 18 years, the duration of treatment under two years, a rheumatoid factor below 50 IU/mL, DAS28 below 2.0 units. The control group consisted of 104 healthy participants, which showed no physical signs of RA, and were questioned about personal and family history of arthritis. The Polish Regional Committees have approved this study for Medical and Health Research Ethics (No. R-I-002/16/2016). Blood samples were taken from the ulnar vein. The test material was serum obtained after isolation and separation from a clotted mass of blood. Serum samples were stored at $-$ 70 ${}^{\circ }$ C until determination. Information that was determined by performing a complete blood count included Red Blood Cells (RBC); White Blood Cells (WBC), Blood Platelets Thrombocytes (PLT). The protein and antibody tests included C-reactive protein CRP. Additionally, an ESR, a nonspecific indicator of the presence of inflammation, was determined.

2.2 Measurement protocol for thermography

The thermograms of both hands for each subject were taken with the thermal imaging camera (FLIR, E60bx, Systems Inc., USA) with a resolution of 320 $\times$ 240 pixels. The measurement conditions were air humidity 55%, emissivity 0.98, air and ambient temperature 23 $\pm$ 1 ${}^{\circ }$ C. Dynamic infrared thermography, which was used in our approach, involved thermal provocations tests. In rheumatologic diseases, it is reasonable because inflamed tissue woven reacts differently to a thermal stimulus due to increased synovial vascularization of the joints affected by this disease [2, 3]. The temperature measurement included two steps: 1) hand’s cooling in the water of 0 ${}^{\circ }$ C for 5 seconds; 2) hand’s rewarming for 180 seconds [2, 3]. Before each measurement, the water temperature was continuously monitored with a digital thermometer and controlled with a mercury thermometer. The thermography images were processed according to [2, 3], and the ROI on thermograms was identified as the axis of all the fingers separately (Fig. 1).

Figure 1.

Region of interest (ROI) of each finger individually.

The outputs were: the mean temperature post-cooling; the mean temperature post-rewarming; and total change in the average temperature of all fingers due to rewarming $\mathrm{\Delta }$ Tr [ ${}^{\circ }$ C] [2, 3].

2.3 ANN in differentiating RA from healthy subjects

Firstly, the univariate logistic regression model to find significant predictors was proposed. Then, the feedforward neural network model was used. The dataset was divided into the training set (75% of data) and the test set (25% of data). The Broyden-Fletcher-Goldfarb-Shanno (BFGS) and non-linear logistic function to transformation nodes in the output layer were used for training. The same dataset generated several training-validation sets. The 10 fold Cross-Validation was used to assess the predictive performance of the ANN model and to judge how it performs. The ten independent scores achieved by each instance were averaged to return the CV score of the model architecture. The number of neurons in the hidden layer was as small as possible while maintaining accuracy. Statistical analysis was performed with the R statistical software (version 3.5.2) [19].

3. Results

3.1 Subjects

The subjects were predominantly female (84.38%). The mean age of all subjects was 53.7 (13.8) years old; the mean BMI was 25.1 (7.7) kg/m ${}^2$ . There were no significant differences concerning gender, age, and BMI ( $p>$ 0.05). The demographic and clinical data in both groups are presented in Table 1.

Table 1

Demographic and clinical data for patients group (SD)

Groups	Age [years]	BMI [kg/m ${}^2$ ]	ESR [mm/h]	CRP [mg/L]	WBC [10 ${}^9$ /L]	PLT [10 ${}^9$ /L]	RBC [10 ${}^{12}$ /L]
RA	53.8 (8.4)	24.5 (3.5)	41.0 (23.8)	18.7 (15.8)	7.2 (1.8)	280.5 (113.5)	4.4 (0.6)
Healthy	44.4 (13.0)	26.2 (4.4)	18.2 (6.4)	3.1 (1.3)	5.1 (2.0)	277.6 (88.2)	4.4 (0.5)

RA patients had significantly higher ESR, CRP, WBC, RBC, and PLT than healthy patients ( $p<$ 0.05).

3.2 Thermovision data

The outputs were: the temperature of all fingers post-cooling and post-rewarming, and $\mathrm{\Delta }$ Tr (Table 2). In addition, the average temperature was calculated over the selected ROI of each finger individually.

Table 2

The mean (SD) dynamic thermal imaging outcomes for healthy and RA subjects

Parameters	Thumb	Index finger	Middle finger	Ring finger	Little finger
Healthy
Mean temperature post-cooling [ ${}^{\circ }$ C]	25.1 (1.9)	24.5 (2.3)	24.9 (1.2)	24.2 (1.8)	23.9 (2.2)
Mean temperature post-rewarming [ ${}^{\circ }$ C]	30.6 (2.9)	29.2 (2.8)	29.8 (3.1)	29.4 (3.2)	29.1 (3.0)
$\mathrm{\Delta }$ Tr [ ${}^{\circ }$ C]	5.1 (1.1)	4.5 (0.9)	4.8 (1.7)	5.0 (1.9)	5.4 (1.9)
RA patients
Mean temperature post-cooling [ ${}^{\circ }$ C]	25.2 (2.0)	24.4 (2.0)	24.4 (1.9)	24.2 (2.0)	23.6 (2.1)
Mean temperature post-rewarming [ ${}^{\circ }$ C]	28.6 (3.3)	27.4 (3.6)	27.2 (3.3)	26.9 (3.4)	26.4 (3.3)
$\mathrm{\Delta }$ Tr [ ${}^{\circ }$ C]	3.4 ${}^{\ast }$ (1.8)	3.1 ${}^{\ast }$ (2)	2.8 ${}^{\ast }$ (1.9)	2.8 ${}^{\ast }$ (2.0)	2.8 ${}^{\ast }$ (2.1)

${}^{\ast }p<$ 0.05.

A smaller increase in joint temperature after reheating in patients with RA compared with healthy subjects was observed. It could be explained by the impaired vascular flow and characteristic features of ischaemia of the fingers skin, which was manifested by the much slower heating of these hand areas. The statistically significant difference between RA patients and the healthy was observed for $\mathrm{\Delta }$ Tr ( $p<$ 0.05).

3.3 ANN

Table 3

Results from multivariate logistic regression model containing all explanatory variables

Parameters	Estimate	SE	$z$ value	$p$
Thumb $\mathrm{\Delta }$ Tr [ ${}^{\circ }$ C]	$-$ 0.152	0.069	$-$ 2.196	0.028
Index finger $\mathrm{\Delta }$ Tr [ ${}^{\circ }$ C]	$-$ 0.252	0.081	$-$ 3.13	0.002
Middle finger [ ${}^{\circ }$ C]	$-$ 0.27	0.088	$-$ 3.059	0.002
Ring finger $\mathrm{\Delta }$ Tr [ ${}^{\circ }$ C]	$-$ 0.203	0.087	$-$ 2.325	0.020
Little finger $\mathrm{\Delta }$ Tr [ ${}^{\circ }$ C]	$-$ 0.18	0.081	$-$ 2.231	0.026
Age [years]	0.051	0.013	3.883	0.000
BMI [kg/m ${}^2$ ]	$-$ 0.103	0.043	$-$ 2.394	0.017
ESR [mm/h]	0.143	0.026	5.579	0.000
CRP [mg/L]	0.478	0.097	4.935	0.000
RBC [10 ${}^{12}$ /L]	0.297	0.29	1.022	0.307
WBC [10 ${}^9$ /L]	0.372	0.077	4.804	0.000
PLT [10 ${}^9$ /L]	0.001	0.002	0.561	0.575

Figure 2.

The ANN model: a) architecture of ANN; b) receiver operating characteristic curves (ROC); c) model of differentiation patients with rheumatoid arthritis from the healthy.

Firstly, the univariate logistic regression method to find significant predictors was used (Table 3). The following parameters were significant in RA prediction: $\mathrm{\Delta }$ Tr for all fingers, patient age, BMI, ESR, CRP and WBC. The ANN architecture consists of 10 neurons in the input layer, 1 in the hidden layer, and 1 in the output layer. The decay index of weighs was 0.1; the Kappa was 0.72. The final architecture of the neural network and the Receiver Operating Characteristic Curves is presented in Fig. 2a and b. The model of differentiation patients with RA from the healthy has been presented in Fig. 2c. High levels of sensitivity and specificity were obtained at 81.25% and 100%, respectively. The accuracy was 92.86%.

4. Discussion

RA is globally the most common inflammatory disease characterized by symmetrical arthritis, extraneous lesions, and systemic complications. It limits the daily functioning of the patient and leads to disability and premature death. The problem in the disease diagnosis is the lack of a generally available method of diagnosing RA that would allow for a quick and effective diagnosis and assessment of joint inflammation activity. Infrared thermographic examination opens up new diagnostic possibilities for RA due to the non-invasive method, low cost and availability of thermal imaging cameras. However, the lack of reliable and unambiguous criteria for quantifying changes in inflammatory joints most frequently affects correct disease diagnosis [23, 24, 25]. In clinical practice, inflammatory markers such as ESR and CRP are used to assess RA activity. We proved that ESR, CRP, WBC, RBC, and PLT are significant in RA diagnosis ( $p<$ 0.05). The same was confirmed by authors [20, 21, 22]. Besides the clinical parameters, medical imaging techniques are also beneficial in visualizing joint pathological changes in the RA disease. Sensitive imaging methods allow identifying lesions characteristic of the early stage of the disease, such as inflammation of the soft tissues, synovium and bone marrow. Currently, three basic techniques are used in the imaging diagnostics of RA: conventional radiography (X-ray), ultrasonography and magnetic resonance imaging. In clinical practice, an X-ray examination is prevalent. The other two auxiliary methods are used only when the X-ray examination has failed or does not provide sufficient information about the disease. Due to the limitations of the existing techniques, we decided to verify the usefulness of thermography technique in RA diagnosis. The temperature is a key physical property, which can reflect the articular inflammatory processes. The increase above the typical value could be a classical sign of inflammation. Some authors reported a statistical difference in joints’ absolute temperature between patients affected by rheumatic diseases and healthy controls [26, 28]. This is in agreement with our study, which shows that the reperfusion processes in RA patients and the healthy are different during the thermal provocation. The results suggest higher thermal activity of the healthy tissues due to more perfusion and metabolism, resulting in a lower increase in joint temperature after reheating in patients with RA compared to the healthy. Moreover, we confirmed that the detection of the inflammation status could be achieved using infrared thermography and the $\mathrm{\Delta }$ Tr of hand fingers. Finally, the parameter $\mathrm{\Delta }$ Tr for all fingers, patient age, BMI, ESR, CRP, and WBC were applied to build a model to differentiate patients with RA from the healthy.

5. Conclusion

The proposed ANN model is characterised by high levels of sensitivity, specificity and accuracy. Therefore, we can state that using thermography data in the ANN model increases the method’s precision in differentiating the RA patients from the healthy, which is a new approach in diagnostic methods. In the literature two other studies investigated the thermal patterns between normal or pathological joints [27, 28]; however, the authors did not reach statistical significance. The proposed methodology suggests that the thermography data can be considered in addition to the currently available tools for screening, diagnosis, and monitoring of disease progression [29, 30]. Further studies should be devoted to the comparison of different methods in differentiation RA status.