Abstract
Objective
Wound image analysis tools hold promise in helping patients to monitor their wounds. We aim to perform a novel feasibility study on the efficacy of a patient-owned wound surveillance system for diabetic foot ulcer (DFU) care.
Methods
This two-institutional, prospective, single-arm pilot study examined patients with DFU. An artificial intelligence-enabled image analysis app calculating the wound surface area was installed and patients or caregivers were instructed to take pictures of wounds during dressing changes. Patients were followed until wound deterioration, wound healing, or wound stability at 6 months occurred and the outcomes of interest included study adherence, algorithm performance, and user experience.
Results
Between January 2021 and December 2021, 39 patients were enrolled in the study, with a mean age of 61.6 ± 8.6 years, and 69% (n = 27) of subjects were male. All patients had documented diabetes and 85% (n = 33) of them had peripheral arterial disease. A mean follow-up for those completing the study was 12.0 ± 8.5 weeks. At the conclusion of the study, 80% of patients (n = 20) had primary wound healing whilst 20% (n = 5) had wound deterioration. The study completion rate was 64% (n = 25). Usage of the app for surveillance of DFU healing, as compared to physician evaluation, yielded a sensitivity of 100%, specificity of 20%, positive predictive value of 83%, and negative predictive value of 100%. Of those who provided user experience feedback, 59% (n = 10) felt the app was easy to use, 47% (n = 8) would recommend the wound analysis app to others but only 6% would pay for the app out of pocket (n = 1).
Conclusion
Implementation of a patient-owned wound surveillance system is feasible. Most patients were able to effectively monitor wounds using a smartphone app-based solution. The image analysis algorithm demonstrates strong performance in identifying wound healing and is capable of detecting deterioration prior to interval evaluation by a physician. Patients generally found the app easy to use but were reluctant to pay for the use of the solution out of pocket.
Introduction
Currently over 9% of the global population suffers from diabetes with the number of patients affected is expected to increase by over 50% by 2045. 1 Among other concerns, these individuals with diabetes have a 19 to 34% risk of developing foot ulcers. 2 Amidst a well-described constellation of associated factors, including high rates of concomitant peripheral vascular disease, compromised wound healing leads patients with diabetic foot wounds to a 21% amputation rate and nearly 50% 5-year mortality rate.3,4 Prolonged wound care and subsequent interventions also pose significant costs to the healthcare system. In high-income countries, costs per wound episode, while difficult to precisely identify, range from approximately USD$7500 in the UK to USD$15,000 in Singapore to USD$24,200 in the US.5–7
The clinical and systemic burdens of diabetic foot ulcers necessitate the effective treatment of these wounds. However, budgetary constraints as well as logistic and mobility concerns among individuals in this population have made home- and community-based care the mainstay of treatment. 8 Digital health solutions have been used to augment the reach of wound care professionals, with telemedicine comprising the first wave of digitally-enabled home-based treatment programs. 9
More recently, analytic approaches using imaging technologies have expanded the capabilities of remote wound monitoring. At least 18 commercially available wound assessment systems are available, utilizing a number of approaches, including photography, thermal imaging, and ultrasound. 10 Besides these commercially available solutions, a number of de-novo machine learning algorithms to recognize, assess, or predict the healing of DFUs have also been described in the literature by academic groups.11–14 Patients empowered with these semi-autonomous solutions may, in theory, be able to conduct their technology-guided wound surveillance.
However, creating an algorithm in isolation cannot provide practical insights into how it will behave when used. These potential algorithms can be likened to candidate compounds in the realm of drug development. Assessing the models across different levels of strength, including cross-validation, internal validation, and validation with external data, provides data about the algorithm's validity and reliability. However, this validation is somewhat akin to in-vitro testing. The true understanding of these potential algorithms in real-world scenarios, encompassing factors like generalizability and human influences, only emerges through implementation studies. 15 As such, despite the availability of commercial wound care machine learning solutions, as well as several academically developed technologies, only 1% of the published literature discusses the implementation of algorithms into clinical protocols. 8
Given this deficit, we conducted a qualitative study to explore the views of patients, carers, and healthcare providers (HCPs) regarding the use of a novel patient-owned wound surveillance application as part of outpatient management of patients with DFU. 16 Regarding such a patient-owned wound surveillance app, all were open and receptive to the system and workflow for use in DFU care. Four major themes emerged from patients and carers: (a) technology; (b) application features and usability; (c) feasibility of using the wound imaging application; and (d) logistics of care. Four major themes were identified from HCPs: (a) attitudes toward a wound imaging app; (b) preferences regarding functionality; (c) perceived challenges for patients/carers; and (d) perceived barriers for HCPs. 16 Findings from the study demonstrate the potential of digital health and areas to improve and tailor a DFU wound app suitable for implementation in the local population.
As part of the larger ePOWS study (efficacy of a patient-owned wound surveillance system for DFU care), we utilized findings from the above-mentioned qualitative study to customize the wound imaging app for use in the local population as a patient-owned wound imaging system, for patients discharged from hospitals after limb salvage procedures.
Methods
This two-institutional, prospective, single-arm pilot study involved patients from two tertiary hospitals in Singapore (800-bed and 1300-bed hospitals). Patients were recruited for the study if they (1) were older than 21 years, (2) were able to provide informed consent, (3) were discharged from a participating institution with an active wound at or below the level of the ankle, (4) were scheduled for a follow-up at a participating ambulatory center or for home wound management, (5) owned a smartphone compatible with the app function, and (6) were either personally, or had a carer who was, able to train in the use of eKare. Pregnant and breastfeeding women were excluded from the study.
The eKare Insight app (eKare Inc, Fairfax, VA) (eKare or the app) was used to monitor wound healing. The eKare uses machine learning-driven wound imaging analysis on mobile phone photographs to model wound dimensions via digital planimetry.17,18 The eKare Insight app was customized for the study population according to stated preferences from patients, carers, and health care professionals as elucidated in Lo et al. 16 Features included offline access, interfaces in local languages, and push notifications to physicians as requested by patients and carers; a compiled dashboard of findings, integrated subjective wound questionnaires for infection-related symptoms, and health records were incorporated for health care professionals. 16
Study participants had the app installed onto their personal devices. Prior to the commencement of the study, patients were trained, using a standardized protocol, on the use of the eKare tool; each training session concluded with the successful independent capture of a wound image by either the patients or their caregivers. In addition, the app had internal quality constraints such that images could not be captured unless the angle, distance, and focus rendered interpretable data.
Baseline demographic information, medical history, medical management, and wound characteristics were collected from patient health records. Further demographic and functional information were collected using Client Service Receipt Inventory (CSRI) questionnaires.
Participating patients, once discharged, were asked to conduct home dressing changes independently or with home carers and attend outpatient wound care appointments as specified in discharge instructions. Patient caregivers were trained prior to discharge on the management of bandage dressings. For those patients without carers, visiting nurses changed bandage dressings every 2–3 days. Patients with negative pressure wound therapy had dressings changed twice per week. The frequency of app usage was monitored by treating physicians. Patients who were not adherent to the protocol, defined by 3 consecutive missed instances of app use, were encouraged to withdraw in the interest of safety. These patients were then monitored more traditionally by physicians to ensure the regularity of dressing changes.
Patients were instructed to take a picture of their wound at the time of each dressing change. Participating individuals were followed until they experienced the earliest occurrence of the following: (1) wound deterioration, as defined by any increase in the wound surface area between two consecutive measurements or infection requiring admission, antibiotic therapy, or surgical intervention, (2) wound healed, or (3) stable wound status after 6 months. The increase in the surface area was defined as a growth of 1 mm2 on two consecutive measurements. All primary wounds, including those identified by the app as having increased surface area, were evaluated by a physician at each outpatient visit.
Outcomes of interest included (1) adherence to the study protocol, (2) performance of the algorithm in identifying wound healing, and (3) user experience. User experience was collected using Likert ratings in three domains: qualitative ease of use, binary willingness to recommend the app, and binary willingness to pay any amount out of pocket for the use of the app.
Institutional Review Board approval was granted (NHG 2020/01348). Written consent was obtained from each patient enrolled. Statistical analysis was performed using R Version 4.2.3 and RStudio Version 2023.03 software (PBC, Boston, MA). Continuous variables were presented as mean value ± standard deviation (SD). Discrete variables were presented as values with the percentage of the relevant population as appropriate. Descriptive and continuous values were compared using chi-squared and t-tests as appropriate. P values less than or equal to 0.05 were considered statistically significant.
Results
Cohort demographics
Between Jan 2021 and Dec 2021, 39 patients were enrolled in this study. The average age of subjects in the cohort was 61.6 ± 8.6 years and 69.2% (n = 27) of subjects were male. A summary of demographics is shown in Table 1.
Summary of cohort baseline characteristics.
Medical history and management
All study participants (n = 39) had known diabetes diagnoses and of these individuals, 94.9% (n = 37) were being treated pharmacologically with a mean HbA1c of 8.8 ± 2.5% at the time of enrollment. Exclusive of diabetes, the most prevalent comorbidities included hypertension in 84.6% (n = 33), coronary artery disease in 48.7% (n = 19), and chronic kidney disease in 46.2% (n = 18), with 44.4% (n = 8) of those subjects on dialysis. A summary of comorbidities is shown in Table 1.
Of the 39 participants, 84.6% (n = 33) had documented peripheral arterial disease at the time of enrollment, of which 75.8% (n = 25) were on both antiplatelet and statin therapies. The mean ipsilateral toe brachial index at the time of enrollment was 0.58 ± 0.29. Of the total cohort, 61.5% (n = 24) had undergone ipsilateral revascularization prior to commencing the study; 58.3% (n = 14) of these 24 revascularizations took place within 1 month of study involvement. The majority of these revascularizations, 95.8% (n = 23), were endovascular while the case was an open lower extremity bypass. Of the total 39 individual cohorts, 10.3% (n = 4) had undergone a contralateral major amputation prior to study enrollment.
Wound characteristics
All 39 patients were followed for one primary wound. Additionally, secondary wounds were noted in 20.51% (n = 8) of individuals. The most common wound locations included sites of recent minor amputations in 69.2% (n = 27), toes in 15.38% (n = 6), and heels in 7.7% (n = 3). Those with minor amputations healed by secondary intention. Most wounds, 61.5% (n = 24), were superficial and did not penetrate beyond the dermis; the remaining 38.4% (n = 15) were found to be deep in the dermis. In addition, 61.5% (n = 24), were dressed with standard bandage dressings while 38.4% (n = 15) had negative pressure dressings. Of the 39 subjects, 51.3% (n = 24) were on empiric antibiotic regimens. A summary of wound characteristics is shown in Table 2.
Summary of wound characteristics.
Bold type indicates statistical significance.
Follow-up and adherence
During the study period, 35.9% (n = 14) of the 39 individual cohorts withdrew, yielding a 64.1% (n = 25) completion rate. The mean time to study completion for those completing the study was 12.0 ± 8.5 weeks. Of 24 individuals who completed the study, the mean time between outpatient visits was 9.1 ± 5.5 days. The remaining individuals took regular pictures using the app during dressing changes but presented to the clinic after 6 weeks with a healed wound.
Of the 14 patients who withdrew from the study, 64.2% (n = 9) were either unable or unwilling to follow the study protocol, 21.4% (n = 3) cited personal reasons for withdrawing, and 14.2% (n = 2) had mobile phones of insufficient capability.
The completed cohort had a lower rate of deep wounds (24.0% vs. 64.3%, p = .01) and a lower rate of vacuum dressings (24.0% vs. 64.3%, p = .01) than the withdrawn cohort. Otherwise, the characteristics of the two cohorts did not statistically significantly differ.
Clinical outcomes
Of 25 patients who completed the study, 80.0% (n = 20) healed their primary wounds and 20.0% (n = 5) experienced wound deterioration, as determined by physicians during follow-up appointments. Those who healed wounds did so after a mean time of 9.5 ± 8.2 weeks. Figure 1 demonstrates a survival curve for wound healing. Of the 20 healed primary wounds, the eKare app correctly classified wound healing in all cases. Of the 5 deteriorated primary wounds, the eKare app correctly classified wound deterioration in 20.0% (n = 1) before physician classification.
Kaplan–Meier curve for wound deterioration.
When used for detecting wound healing among the 25 patients who completed the study, the app had sensitivity of 100%, specificity of 20%, positive predictive value of 83% and negative predictive value of 100%. A summary of test performance for detecting wound healing is shown in Tables 3a and 3b. A Kaplan–Meier representation of wound deterioration events is shown in Figure 1.
(a) Wound app usage outcomes.
(b) Wound app performance characteristics.
User experience
Of the 25 individuals who completed the study, 68.0% (n = 17) provided feedback on user experience after concluding the study. Of the 17 individuals, 58.8% (n = 10) found the app interface somewhat or very easy to use. Of those who found the app either somewhat or very difficult to use, 71.4% (n = 5) cited personal difficulties using smartphones generally; 57.1% (n = 4) reported asking another individual to take photographs for them.
Of those who completed the survey, 47.1% (n = 8) would recommend the app to others, 17.6% (n = 3) would not recommend app to others, and 35.3% (n = 6) were unsure of whether to recommend the app to others. Those unsure cited the apparent absence of a physician's involvement as the reason for their ambivalence. Of note, those with wound deterioration were less likely to recommend the app than those whose wounds healed (0.0% vs 66.7%, p = .04), though sample sizes were small.
Of the 17 individuals providing feedback, 5.9% (n = 1) stated they would pay for the app's functionality, 70.6% (n = 12) stated they would not pay for the app's functionality, and 23.5% (n = 4) were unsure of whether they would pay for the app's functionality. Those unsure cited the prospective cost of the app as their primary decision variable. A summary of user experience findings is shown in Tables 4a and 4b.
(a) Summary of qualitative ease of use.
(b) Summary of qualitative willingness to recommend and pay.
Discussion
The current state of diabetic foot ulcer care, despite a migration toward large community care, still presents a relatively costly and onerous proposition to elderly patients with multiple comorbidities. These patients currently bear costs and navigate logistics for frequent, long-term outpatient physician evaluation in order to effectively monitor wound healing. The deployment of a patient-owned solution administered in the absence of a wound care expert may augment the efficiency of care by decreasing barriers to care while maintaining a high quality of care.
One of the more unique aspects of this study, as compared to other uses of machine learning in healthcare, is the patient-owned aspect of clinical decision support. The efficacy of the semi-autonomous tool was contingent upon the reliable performance of the patient. Moreover, the algorithm then analyzed the information and sent it to the physician for ultimate medical management. Based on our findings, this three-way circuit between patient, machine, and physician appears to be feasible.
To our knowledge, there is one other contemporaneous pilot study evaluating a commercially available patient-owned analytic wound assessment system. Keegan et al. analyzed a similar machine learning solution among a cohort of 25 individuals in the United States. 19 Differences from our study in the methodology included weekly reminder calls to patients who did not take images and technical support by phone to individuals enrolled. While 28% of patients in Keegan et al. failed to complete the study, an additional 12% of patients were categorized as not engaged, yielding a similar total participation deficit to our 36% withdrawal rate. 19 As with our study, most patients who completed surveys after the study period found the interface easy to use, with 88% responding positively. Most patients in their study, 94%, found the solution useful, as opposed to the 47% recommendation rate in our study. However, in concordance with our subjective user experience findings, patients in the Keegan et al. report also commented on the lack of real-time physician feedback. 19
Swerdlow et al. conducted a pilot study on a patient-owned image capture system for both primary care and active monitoring of preulcerative and ulcerative foot wounds among 15 patients in the United States. 20 The app trialed was not an analytic clinical decision support system but rather a hardware solution with an accompanying software interface to capture foot wounds that, because of anatomic or patient functionality constraints, may have been missed by patients prior to professional evaluation. As with the Keegan et al.’s study, patients were contacted and offered technical support if they did not submit daily photographs. All patients completed the 5-month study and at the end of the study period, participants gave median 10/10 maximum scores when polled on whether the app was useful and whether they would recommend it. 20 Neither Swerdlow et al. nor Keegan et al. polled patients about willingness to pay.19,20
There are at least four other instances of academic studies examining clinical decision support operated by patients. D’Haese et al. developed and piloted an artificial intelligence-enabled tool that predicted the risk of COVID-19 symptom onset using features collected from wearables and a mobile app for subjective entry. 21 Avari et al. developed and tested a glucose bolus recommendation system using data from wearables, glucose monitors, and mobile app entry data and tested using a randomized controlled trial (RCT). 22 Both solutions, like the eKare system, required users to input data. Both solutions similarly had notable troubles with adherence. During our study period, 30.7% (n = 11) of enrolled patients withdrew for reasons unrelated to technological issues. In D’Haese et al., up to 49.0% (n = 376) of enrollees failed to report data, and in Avari et al., 33.3% (n = 18) of patients withdrew for reasons unrelated to adverse medication effects.21,22 Of note, both of the other patient-owned solution studies enrolled cohorts with mean ages younger than 50 years as compared to our cohort's 61.6 years.21,22
Forman et al. developed and trialed on 190 individuals an app-based prediction system for dietary lapses using an app input. 23 Luštrek et al. designed and conducted a 56-patient RCT on a system using data from wearables and voice analysis to recommend physical and psychological wellness practices to heart failure patients. 24 Despite both studies requiring participation, neither suffered significant issues with adherence (withdrawal rates were 15.8% (n = 30) and 0.0% (n = 0), respectively).
While the exact reason behind withdrawal rates may be difficult to elucidate, feedback regarding user experience comports with the literature on the adoption of new technologies among patients. Galavi et al. conducted a systematic review of barriers to the adoption of health technology solutions in home care. 25 They found that significant barriers to entry for patients included such factors as lack of digital literacy, difficulty of use by older adults, forgetfulness, and complexity of existing comorbidities. 25 Lo et al., in a qualitative study also identified the apparent absence of a physician's involvement as a primary concern of patients considering the use of an artificial intelligence-enabled system. 16
Importantly, there is no evidence that the patient-owned treatment paradigm in this study harmed enrollees. 80.0% (n = 4) of the 5 deterioration events were identified by physicians during the course of routine post-acute wound care, while the remaining case was identified by eKare even prior to evaluation by a physician. Of note, however, in this protocol, the algorithm was used in conjunction and not as a replacement for the standard of care by a physician. Furthermore, those who completed the study and healed their wounds did so in a mean of 9.5 ± 8.2 weeks. While it is difficult to comprehensively compare the quality of wounds with those described in the literature, this time to heal is comparable to the mean 9 weeks and 10.8 weeks described by Ince et al. and Zimny et al., respectively.26,27
It is difficult to measure an increase in efficacy from patient-owned wound surveillance, however, due to five significant limitations. First, the study was not able to capture data on adherence to individual dressing changes. As patients were instructed to use the app when indicated per wound care instructions, it was not possible to capture nuances of adherence and relate them to outcomes or satisfaction among participants.
Second, this was a single-arm pilot study designed to prove feasibility. As such, the lack of a control group or randomization renders the study unable to definitively prove the patient-owned surveillance model more beneficial than the standard of care.
Third, a significant portion of enrolled patients withdrew from the study. While the age and health status did not explain the difference between those who completed the study and those who did not, the presence of a responsible feature would influence the efficacy of this solution in the real world. Those who withdrew did have a higher prevalence of deep wounds, perhaps reflecting physical limitations when taking pictures or trepidation regarding deviations in the standard of care. These features highlight the need for appropriate patient selection for patient-owned wound monitoring, and more work is warranted on factors influencing adherence.
Fourth, while this model of wound monitoring is clinically and logistically feasible, a lack of cost analysis obscures whether the system is financially feasible. Patients were largely uninterested in taking on additional costs for the use of the eKare solution. However, increased care efficiency from patients monitoring their own wounds may return value to the system. Further work is required to quantify these effects.
Fifth, there is no publicly available validation performance data for the eKare Insight system that can be compared to real-world performance data with respect to this application. Machine learning models in the real world are expected to have impaired performance when compared to their performance in controlled studies. These discrepancies can occur when models are exposed to new populations, whether temporally or geographically. 28 Performance impairment can also be caused by adherence or quality of data inputs.
Conclusion
In this two-institutional pilot study on the efficacy of a patient-owned wound surveillance system for patients discharged from hospitals with post-diabetic limb salvage, the use of such a system was sensitive (100%) but not specific (20%) for DFU wound healing surveillance, with high positive (83%) and negative (100%) predictive values. The study completion rate was moderately high (64%), with patients preferring a positive feedback loop through active physician involvement via the system. Most patients (59%) had a positive experience with app utilization but only a minority (6%) of them were willing to pay for the app's functionality.
Footnotes
Acknowledgements
Not Applicable.
Contributorship
The authors ZJL, KBH, ET, JZ, SC, HL, WHH, SL, YTC, HYK, KW, and JC contributed to (1) study conception, study design, data acquisition, data analysis, or data interpretation; (2) writing or critical revision; and (3) approval of the submitted version.
Declaration of Conflicting Interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Kyle Wu is the Chief Medical Officer of eKare, Inc. No other authors have relevant disclosures.
Ethical approval
Institutional Review Board approval was granted (NHG 2020/01348). Written consent was obtained from each patient enrolled.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: the Agency for Science, Technology and Research (A*STAR) Skin Research Institute of Singapore Skin Innovation Grant (SIG2002). The National Research Medical Council (NMRC) Research Training Fellowship Award (FLWSHP19nov-0015).
Guarantor
ZJL guarantees the quality and integrity of this work.
