A New Mobile Application for Standardizing Diabetic Foot Images

Abstract

Background:

We describe the development of a new mobile app called “FootSnap,” to standardize photographs of diabetic feet and test its reliability on different occasions and between different operators.

Methods:

FootSnap was developed by a multidisciplinary team for use with the iPad. The plantar surface of 30 diabetic feet and 30 nondiabetic control feet were imaged using FootSnap on two separate occasions by two different operators. Reproducibility of foot images was determined using the Jaccard similarity index (JSI).

Results:

High intra- and interoperator reliability was demonstrated with JSI values of 0.89-0.91 for diabetic feet and 0.93-0.94 for control feet.

Conclusions:

Similarly high reliability between groups indicates FootSnap is appropriate for longitudinal follow-ups in diabetic feet, with potential for monitoring pathology.

Foot pathology is highly prevalent in people with diabetes. ¹ Diabetic foot pathologies such as ulceration involve changes in the appearance of the foot over time. These changes present the opportunity for advanced computer vision algorithms to help in the detection and monitoring of diabetic foot pathologies. To implement this approach in a longitudinal/follow-up study design requires capturing photographs that are standardized for specific conditions. ² Using a standard “human-operated” camera, however, leads to inconsistency in relation to these conditions, especially when different operators/experimenters are involved across different clinic sites and there are no other apps or software solutions that the authors are aware of, available to achieve this. We have therefore developed a mobile app for the iPad (Apple Inc, Cupertino, CA, USA) to standardize foot photographs with intended future use in diabetic foot lesions. ² In this technology report, we describe the design of the app and present results from a reliability study. The main aim was to test the reliability of the app when used on the same diabetic feet on separate occasions and in the hands of two separate operators. In addition, we aimed to establish these reliability parameters in healthy control feet to provide reference values for comparison against the greater “challenge” posed by imaging diabetic feet and enable an understanding of whether it is “fit for purpose” in diabetic feet.

Methods

The Framework of “FootSnap” Mobile Application

The FootSnap app was designed and created for the iPad with the objective of standardizing the acquisition of diabetic foot photographs for longitudinal/follow-up investigations of the plantar surface of the diabetic foot (https://youtu.be/TJtl0Yc8YCk). ² FootSnap allows for standardization of plantar foot pictures in relation to the distance from the foot and the orientation of the camera relative to the foot. Figure 1 illustrates the process of FootSnap, consists of three main elements:

• New experiment: Add a new participant or select an existing participant

• Production of a “ghost image” of the foot: Generates a picture of the plantar foot surface, in addition to an outline of the foot termed the “ghost image” (Figure 2). The ghost image will then be used in this specific participant as an alignment tool for standardizing distance and orientation in subsequent photographs.

• Capture foot image: Acquires a plantar foot image based on the specific ghost image for that participant. While taking the photograph, a lighting module directs light with standardized luminosity onto the foot. A color background is used to optimize segmentation of the foot by the computer vision algorithms (Figure 2).

Figure 1.

Flow chart of FootSnap. The app will first check if this is a new experiment (as shown in the diamond shape of the flow chart). If this is a new experiment, the app will create a new experiment, capture the foot image, generate a ghost image, and save the images to the internal storage. Otherwise the app will retrieve the existing experiment and load the ghost image. The user will align the foot to the ghost image, capture the foot image, and save the image to the internal storage.

Figure 2.

(A) Illustration of the experimental setup for using FootSnap. (B) An image being captured by the operator using the ghost image facility (white outline around the foot) within FootSnap.

Reproducibility Experiment

Participants

After receiving ethics committee approval from the relevant bodies, 15 diabetes patients (aged between 43 and 74 years) and 15 controls (aged between 20 and 40 years) without diabetes or any foot pathology (ie, a total dataset of 60 feet: 30 diabetic and 30 control feet) took part in this study testing FootSnap’s reliability. Major inclusion criteria for diabetes patients included a history of plantar ulceration and the presence of peripheral neuropathy assessed with the vibration perception threshold (VPT), measured using a Biothesiometer (Medical Instruments, Newbury, OH, USA) and the modified neuropathy disability score (mNDS) testing pain, temperature, vibration sensation and ankle reflexes. ³ Exclusion criteria included active foot ulcer, peripheral vascular disease, and amputation of more than one digit. Controls did not have diabetes or any previous notable foot pathology.

Image Acquisition and Protocol

Participants were positioned supine on a bed with their feet overhanging. The foot was placed through a piece of blue material to standardize the background (Figure 1). This was performed one foot at a time and the participant was asked to adopt a comfortable position while relaxing their legs and feet, which meant the foot was typically slightly externally rotated. Two different operators/experimenters acquired foot photographs from all participants using FootSnap, each on two separate occasions. Each foot photograph on each of the four separate occasions (two operators, each on two separate occasions) involved the participant repositioning on the bed and required realignment of the iPad and restarting of FootSnap by the operator. This approach enabled assessment of intra- (ie, photographs taken on separate occasions by the same operator) and inter- (photographs taken by different operators) operator reliability. Photographs could be reacquired using FootSnap if the operator was not satisfied with the outcome. The alignment of the ghost image on the foot and the subsequent acquired photograph was the only “feedback” operators received. Any duplicate images were removed by the operator to leave on only one image to be taken forward to image analysis.

Image Analysis

The dataset consisted of a total of 240 separate foot images from 30 people (15 diabetes patients and 15 controls without diabetes), with four images acquired for each foot: two captured by operator 1 and two captured by operator 2.

An automated foot detection computer vision algorithm was developed to segment (ie, identify and extract) the foot region from the photograph. First, k-means clustering^4,5 was applied where squared Euclidean distance was computed for each pixel to the centroid cluster and the shortest distance was selected. Clustering was repeated three times to avoid local minima, each with a new set of initial cluster centroid positions. Finally, the largest cluster was segmented and was used for further analysis.

To quantify intraoperator and interoperator reliability from images acquired using FootSnap, the Jaccard similarity index (JSI), a statistical approach to determine the degree of similarity of two datasets ⁶ was calculated as:

$$JSI(A,B)=\frac{|A{\displaystyle \cap }B|}{|A{\displaystyle \cup }B|}$$

A and B denote the segmented regions from the different conditions. A JSI value of 1 indicates that A and B are in perfect agreement.

Results: Reliability of FootSnap

Diabetic Feet

Diabetes patients had a VPT of 42 ± 12 volts (mean ± SD) and a mNDS score of 7 ± 2, indicating moderate-severe diabetic peripheral neuropathy. Deformity was present in 24 of these 30 diabetic feet.

High intraoperator reliability of FootSnap for operator 1 and 2 was demonstrated for diabetic feet with a mean JSI value of 0.89 (range: 0.84-0.93) and 0.91 (range: 0.84-0.95), respectively (Figures 3a, 4a). Interoperator reliability was also demonstrated to be high with a mean JSI value of 0.89 (range: 0.83-0.94) across the 30 diabetic feet (Figures 3a, 5a).

Figure 3.

Inter- and intraoperator reliability for FootSnap in capturing (A) diabetic feet and (B) control feet. Data are mean and SD (N = 60 feet). A Jaccard similarity index value of 1 indicates perfect agreement between the two images in terms of shape description. Intra 1 and Intra 2 are the two different operators.

Figure 4.

Intraoperator reliability for FootSnap for two separate operators capturing (A) diabetic feet and (B) control feet. Values shown are for individual feet (N = 60 across A and B). A Jaccard similarity index value of 1 indicates perfect agreement between the two images in terms of shape description.

Figure 5.

Interoperator reliability (between operators 1 and 2) for FootSnap capturing (A) diabetic feet and (B) control feet. Values shown are for individual feet (N = 60 across A and B). A Jaccard similarity index value of 1 indicates perfect agreement between the two images in terms of shape description.

Control Feet

High intraoperator reliability of FootSnap for operator 1 and 2 was demonstrated by a mean JSI value of 0.94 (range: 0.87-0.98) and 0.93 (range: 0.87-0.97) respectively, across the 30 control feet (Figures 3b, 4b). Interoperator reliability was also demonstrated to be high with a mean JSI value of 0.93 (range: 0.90-0.98) across the 30 control feet (Figures 3b, 5b).

Discussion

Here we describe the development of a highly novel mobile app called “FootSnap,” intended for capturing standardized photographs of diabetic feet within a follow-up study, or integrated as part of state-of-the-art longitudinal clinical monitoring. High reliability was demonstrated in diabetic (JSI value: 0.90) and control (JSI value: 0.94) feet for measurements taken on different occasions (Figures 3 and 4) and in the hands of different operators (Figures 3 and 5). As expected, diabetic feet presented a greater challenge for reliable imaging with FootSnap compared to control feet, likely due to diabetic foot deformities and other foot shape-related issues. Nevertheless, JSI values were only marginally lower in diabetic feet (0.90) compared to those of controls (0.94), indicating that the app is indeed fit-for-purpose in diabetic feet.

There are a number of future development considerations and limitations of FootSnap that should be acknowledged. In the present study we used an iPad, but any modifications in the hardware used over time would require alterations to the software. In future iterations of FootSnap, differences in the reproducibility found in the present study should be further interrogated to understand the source of these errors. This present study illustrates proof of concept for the fundamental conditions required in standardizing foot images, but future developments will determine the sensitivity and specificity of the app for imaging diabetic foot ulcers. Interoperator reliability was high in the present study, which is important for standardizing images across different clinical sites. However, trained operators were used, whereas it should be acknowledged that reliability may be lower in the hands of more naïve operators and future development of FootSnap will need to take this into consideration.

High intra- and interoperator reliability shown in this study is evidence of how FootSnap was successful in standardizing the acquisition of plantar foot pictures for the key parameters of distance and orientation relative to the foot. Standardization of plantar foot photographs with FootSnap will allow the future implementation of advanced computer vision algorithms with these images, which can be used for monitoring changes in diabetic foot shape, texture, and color during longitudinal clinical trials, or as part of state-of-the-art clinical monitoring procedures. This technology may therefore represent the first stage toward a meaningful step forward in the prevention and management of diabetic foot pathologies.