Abstract
Major reported factors associated with the limited effectiveness of home telemonitoring interventions in chronic respiratory conditions include the lack of useful early predictors, poor patient compliance and the poor performance of conventional algorithms for detecting deteriorations. This article provides a systematic review of existing algorithms and the factors associated with their performance in detecting exacerbations and supporting clinical decisions in patients with chronic obstructive pulmonary disease (COPD) or asthma. An electronic literature search in Medline, Scopus, Web of Science and Cochrane library was conducted to identify relevant articles published between 2005 and July 2015. A total of 20 studies (16 COPD, 4 asthma) that included research about the use of algorithms in telemonitoring interventions in asthma and COPD were selected. Differences on the applied definition of exacerbation, telemonitoring duration, acquired physiological signals and symptoms, type of technology deployed and algorithms used were found. Predictive models with good clinically reliability have yet to be defined, and are an important goal for the future development of telehealth in chronic respiratory conditions. New predictive models incorporating both symptoms and physiological signals are being tested in telemonitoring interventions with positive outcomes. However, the underpinning algorithms behind these models need be validated in larger samples of patients, for longer periods of time and with well-established protocols. In addition, further research is needed to identify novel predictors that enable the early detection of deteriorations, especially in COPD. Only then will telemonitoring achieve the aim of preventing hospital admissions, contributing to the reduction of health resource utilization and improving the quality of life of patients.
Keywords
Introduction
Asthma and chronic obstructive pulmonary disease (COPD) have attracted research interest as a major public health problem of increasing concern to healthcare systems worldwide because of high prevalence 1 and rising socioeconomic burden. 2,3
Reducing the impact of exacerbations through the early recognition of symptoms and prompt treatment may reduce the risk of hospitalization, improve health-related quality of life (HRQoL) and control the burden of COPD. 4 In asthma, self-management involving prompt intervention in line with a personalized asthma action plan reduces acute exacerbations. 5,6 Education, self-monitoring and regular review are recommended to improve outcomes and prevent exacerbations in both conditions. 7,8 New information and communication technologies and telemonitoring can potentially support patient self-management.
Home telemonitoring encompasses the non-invasive exchange of information using electronic devices and telecommunication technologies and allows clinical data to be collected remotely on a routine and regular basis. Telehealth-enabled chronic care management services are promoted as being effective in supporting people with long-term conditions at home, 9 and the telemonitoring of patients with chronic respiratory conditions has become a major research field of the respiratory community because of its potential to promote self-management, improve control, increase quality of life and prevent admissions.
Some authors have argued that telemonitoring is a promising alternative – or adjunct – to provision of traditional healthcare services both in asthma and COPD. 10 However, although some studies have shown that telemonitoring may improve some clinical outcomes and reduce healthcare costs, 11,12 the effects of telehealth interventions on emergency department attendance, hospital admissions, duration of admissions, HRQoL, costs and mortality remain less certain. 10,13 –16
Major reported factors associated with the apparent lack of effectiveness of telemonitoring in COPD and asthma include the dearth of useful early predictors of deteriorations using a telemonitoring approach, 17 poor patient compliance with telemonitoring tasks 18,19 and the poor performance of conventional algorithms (typically derived from studies of paper-based symptom diaries 20 ), for detecting exacerbations. 21 Among other factors described in the telehealth literature are poor integration into routine healthcare and into the day-to-day lives of patient. 14,22,23 Developing accurate predictive algorithms with demonstrable clinical reliability is a priority for the future consolidation of telemonitoring of chronic respiratory diseases. 13
Home telemonitoring enables the collection of large data sets at the individual level. The application of predictive analytics to derive effective preventive algorithms using these data sets and data available from existing electronic medical records (including data on antibiotic/steroid use and hospital admission) may improve outcomes, enhance disease management and patients’ experiences and increase effectiveness and cost-efficiency of telemonitoring. 24 In this context, recent advances in data mining and machine learning methods offer the opportunity to combine prior knowledge of the clinical context with telemonitoring data sets to reveal predictive patterns. 25
Given the current lack of an existing systematic synthesis of data in this area, this manuscript aims to provide a thorough review of the algorithms used for detecting exacerbations and supporting clinical decision in patients with respiratory conditions, (more specifically in COPD and asthma) that have been reported in home telemonitoring studies. The study looks at significant service-related factors associated with the performance of home monitoring algorithms such as the technology deployed, data collection rate, system interoperability, the agreed exacerbation definition, trial factors such as sample size and study duration, and the machine learning techniques used in the reviewed studies.
Theoretical background
This section includes a brief overview of the fields of predictive analytics and machine learning and their application to the healthcare context.
Predictive analytics in healthcare
Healthcare system use, clinical trials and real-time telemonitoring systems are providing data sets larger than are manageable by conventional management tools. 26 Data science is an emerging discipline that combines and draws connections between the fields of statistics and information and computer science, which can support the management of these complex and large data sets. In the prediction of health outcomes, conventional statistical analysis is usually applied by calculating scores for risk stratification. Generally, the underlying belief is that a small number of important variables exist on which the model can rely. However, these variables typically interact with each other in a concealed way which means that often they are not retained in the predictive model. 27 Predictive analytics, a subset of data science, is based on inductive inference rather than classical statistics and is well suited to the analysis of high-dimensional data sets and to the automated knowledge discovery process. 28
Predictive analytics refers to the systematic use of statistical or machine learning methods to make predictions and support decision-making. 29 It uses computational techniques rooted in several domains including statistics, database management, artificial intelligence, machine learning, pattern recognition and data visualization. Currently, predictive analytics is being applied in many fields such as public safety 30 and security, 31 cybersecurity 32 and social media. 33 In healthcare, health analytics offers methods that can contribute to the potential improvement of patient care by supporting and enhancing medical decision-making. 25
Within predictive analytics, modelling takes data with a known variable of interest or
Machine learning
Machine learning is the crucial methodology in predictive analytics. Conventional statistical analysis focuses on explaining data and relies on an expert (i.e. human) to formulate and discover cause–effect relationships, driven by a set of predefined assumptions. Machine learning is more data-focused and orientated to generating hypotheses and building predictive models using algorithms. 34 Through machine learning, a computer is programmed to reveal hidden relationships that explain or predict a particular outcome. The resulting predictive model may be improved over time with increased number of cases.
Machine learning works by (a) defining goals, (b) exploring data and (c) training, refining and validating models. Supervised, unsupervised and complementary algorithms can be used in the process. Supervised learning includes two categories of algorithms. If the output consists of one or more continuous variables, regression algorithms are used. Classification algorithms are applied for categorical response values.
35
In the case of telemonitoring and early detection of patients’ deterioration, supervised learning (classification algorithms) can be used to model a health-related outcome (e.g. alarm state because of exacerbation of respiratory symptoms) using a collection of a priori known inputs and outputs (training data) and exploring interactions to produce an inferred function (classifier) that can predict a response (output) given a valid set of predictor values. Common classification algorithms for supervised learning in the healthcare field include artificial neural networks,
36
decision trees,
37
random forests,
38
Bayesian networks,
39
There are three stages involved in machine learning: training, validation and testing of the algorithm. In practice, machine learning algorithms are
Methodology
Search strategy, eligibility criteria and study selection
We undertook a systematic review, reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses standards 47 with searches conducted in Medline, Scopus, Web of Science and Cochrane library in July 2015. Search terms were based on a combination of the following keywords: [telemonitoring OR telecare OR telehealth OR telehomecare OR home monitoring] AND [COPD OR chronic obstructive pulmonary disease OR asthma OR respiratory diseases]. Additional searches were performed in the references cited in the finally selected articles to find relevant additional studies.
The inclusion criteria were as follows: (1) the study involved home telemonitoring of COPD or asthma and described the algorithms used to detect episodes, (2) the study presented results on the performance of algorithms for the automatic prediction and/or detection of respiratory exacerbations, and (3) the article was written in English and published in peer-reviewed journals from 2005. This date was chosen according to the findings of a recent systematic review that reported the majority of the articles that described home telemonitoring interventions were published from 2010 to the present, suggesting that this is a relatively new field in respiratory research. 48
Some small pilot and feasibility studies were included because the expected outcomes were directly related to the design and implementation of algorithms for the early detection of respiratory exacerbations. Conference, posters abstracts and general reviews were not considered. Publications that focused on settings or conditions other than home telemonitoring in COPD and asthma were excluded.
Data extraction and quality assessment
Two authors (DS and MAF) extracted the data from the selected studies and a third author (AL) checked for inconsistencies. The following data were extracted and are summarized in three tables. Table 1 reports the first author’s last name, year of publication, country where the study was conducted, type of patients (e.g. disease) to which telemonitoring was applied, duration, study design, number of subjects that completed the telemonitoring study, admissions and/or exacerbations in the telemonitoring group and the exacerbation criteria applied. Table 2 summarizes the specification of clinical and physiological parameters collected in the telemonitoring (e.g. symptoms and physiological variables), the features and algorithms used for triggering automatic alerts and the personalization strategies. Table 3 describes the technology deployed during the study, provides details about data transmission, patients’ compliance with remote monitoring tasks and study outcomes.
Description of the characteristics of the selected studies for telemonitoring COPD and asthma.
RCT: randomized controlled trial, ATS: American Thoracic Society, ERS: European Respiratory Society, GOLD: Global Initiative for Chronic Obstructive Lung Disease.
Specification of clinical and physiological parameters under telemonitoring and features, algorithms and personalisation strategies.
FEV1: Forced expiratory volume in 1 second, LF: lung function, BP: blood pressure, HR: heart rate, W: weight, T: temperature; RR: respiratory rate, LS: lung sounds; PA: physical activity, SD: standard deviation, FeNO: Fractional exhaled nitric oxide, EBC: exhaled breath condensate, PEF: peak expiratory flow, PDF: probability density function, CDF: cumulative density function, CART: classification and regression tree, MDL: minimum description length.
Equipment, type of monitored data, characteristics of data transmission and compliance of participants with telemonitoring tasks.
FEV1: Forced expiratory volume in 1 second, Se: sensitivity, Sp: specificity, AUC: area under the received operating characteristic curve, PPV: positive predicted value, FeNO: fractional exhaled nitric oxide, EBC: exhaled breath condensate,
The quality of included studies was independently evaluated by two reviewers using a modified version 49 of the tool developed by Hailey et al. 50 Articles were rated from A (high quality and high degree of confidence in the study findings) to E (poor quality and unacceptable uncertainty in the study findings). Disagreements were resolved by consensus.
Because of the diverse outcome measurements and the heterogeneous nature of data collected and clinical diversity of the studies, a meta-analysis was not performed; instead we used a narrative synthesis.
Results
Study selection
Totally, 407 articles were identified with the search strategy. After removing duplicates and review articles, 189 articles were retained. Two researchers screened title, abstract and keywords (DSM and MAF) to determine eligibility for this review. These potentially relevant articles were examined and 32 articles with potential significance for this study were selected and full text was retrieved. The bibliography of each manuscript was screened and four additional relevant studies were identified. Full-text articles were assessed for eligibility, and finally, 20 were included. The majority were from European countries, with two from United States and one each from Canada and Australia.
Study characteristics and quality assessment
Study characteristics are presented in Table 1. Sixteen studies focused on COPD 13,51 –65 and four 66 –69 studies involved patients with asthma. Nine of the studies were controlled trials, six were prospective cohort studies, four were pilot/feasibility studies and one was an uncontrolled before and after studies. The number of telemonitored patients that completed the studies ranged from 5 to 169. Two studies used the same data set. 59 –63
With respect to the quality assessment, 11 studies were rated A (high quality), 7 were rated B (good quality) and 2 were rated D (poor to fair quality).
Home telemonitoring duration, exacerbation criteria and admissions
Duration of the telemonitoring period ranged from 3 to 15 months (Table 1). Ten studies had a duration greater than 6 months, five studies had a duration of 6 months and five studies had a duration shorter than 6 months.
An ‘event-based’ definition of an exacerbation,
70
including self-administration of medication or unscheduled visits to emergency units and/or admissions, was used in nine studies (
The following symptom questionnaires were used in the studies: Clinical COPD Questionnaire (CCQ), 72 exacerbations of chronic pulmonary disease Tool (EXACT)- patient-reported outcome (PRO), 73 St Georges Respiratory Questionnaire (SGRQ), 74 Paediatric Asthma Quality of Life Questionnaire (PAQLQ), 75 Asthma Control Questionnaire (ACQ) 76 along with a variety of ad-hoc questionnaires for COPD and asthma.
The number of admissions in each study caused by worsening in respiratory symptoms ranged from 1 to 148. The maximum number of recorded exacerbations during a study was 191 for a telemonitored group of 49 subjects. The highest average number of exacerbations was 9 per patient.
Factors that influence algorithms’ performance
Clinical data, sensors and devices
Tables 2 and 3 present information about the telemonitoring system deployed during the study period. Clinical telemonitored data consisted of symptoms collected using electronic questionnaires (
Data were recorded at least on a daily basis in all studies. More specifically, data were gathered more than once per day (
Feature extraction, personalization and model validation
There was limited description of how the collected data were processed. Mean (
In the studies that reported predictive analytic models, personalization of the algorithm was generally discussed as a future improvement (
Algorithms for automatic detection of health deterioration
Different strategies were followed to detect early exacerbations of respiratory symptoms. In the simplest approach, the decision support consisted of a basic decision rule based on assigning threshold values to the collected parameters (
In some studies, predictions were personalized by entering patient-specific data or by modelling individual characteristics. In the case of a decision rule based on score thresholds, an individual baseline was established according to different criteria (
When used, the baseline was defined in various ways: as an exacerbation-free period of 14 days (
Results in terms of system performance were not always reported. In COPD, when reported, accuracy in detecting exacerbations ranged from 40% to 94%, sensitivity from 70% to 80% and specificity from 61% to 95%. In asthma, only one study reported accuracy (52%), whilst achieved sensitivity ranged from 65% to 84% and specificity from 77% to 95%.
Discussion
This systematic literature review provides a description of the algorithms used in home telemonitoring interventions in COPD and asthma for supporting clinical decision and summarizes the findings related to factors associated with the performance of home monitoring algorithms. Reviewed studies were heterogeneous in terms of their definition of exacerbation, telemonitoring duration, telemonitored measurements, type of technology deployed and algorithms used. In asthma, the home telemonitoring comprised symptoms and lung function (PEF and FEV1): in COPD a much wider range of physiological measures were used.
Home telemonitoring duration, exacerbation criteria and admissions
A common limitation in the selected studies was the examination of a relatively small group of patients who presented a high rate of exacerbations. Winter periods were extensively selected for trials because exacerbations are more likely than in other seasons. 51 Although this may have enabled recruitment of ‘at-risk’ populations it may have affected generalizability of results. Randomized controlled trials with a larger sample of patients over periods longer than 6 months might improve the robustness of algorithms and the generalizability of their findings. 62,82
The lack of consensus on a definition of an exacerbation 83 and unreported episodes that are poorly labelled in data sets 84 may adversely affect algorithms’ performance. Identifying an exacerbation episode is dependent on establishing a link between the characteristics being measured and exacerbations. Asthma attacks are relatively clearly defined; in COPD the absence of a gold standard definition makes identifying exacerbations difficult. Of the questionnaires CCQ, PAQLQ, ACQ and SQRG widely used in the reviewed studies, only EXACT-PRO was specifically designed and validated for detecting COPD exacerbations. Finally, in most of studies patients were given self-management advise encouraging them to intervene, usually at quite an early stage, if parameters strayed outside of expected. This may have influenced some outcomes (e.g. hospital admission and length of exacerbation) possibly blunting the relationship between the original predictive features and outcomes.
Factors that influence algorithms’ performance
Clinical data, sensors and devices
The combination of symptoms and physiological measurements was the most common approach in the selected works (
Since learning algorithms and deployed predictive models rely on timely reporting of accurate data, good patient compliance with reporting tasks is needed. Perceived ease of use of home telehealth devices is a significant predictor of compliance. 86 Systems that offer a unified interface with contextual data on smartphones and provide interactive user feedback and self-adaptive sensor polling could be a solution to alleviate the patient burden of reporting tasks and optimize the quality of collected data. 87 One study, which used a wearable sensor that automatically transmitted heart rate, temperature and activity levels five times an hour, achieved average reporting rates of 60% though the reason for 40% loss of data was due to patients not wearing the device or technical failure is not reported. 58
Features extraction, personalization and model validation
Although feature extraction can help to build derived values that enhance the subsequent learning and generalization steps of algorithms, these techniques were seldom implemented in the selected studies. Additionally, personalization of predictive models that used machine learning techniques was only implemented in a single study. 56
With respect to the procedure to estimate the prediction error in studies that applied machine learning methods, different approaches were followed in six studies. None applied external validity and a number of internal validity methods based on holdout and cross validation techniques were reported. Holdout approaches work well in large data sets, but can give inaccurate performance estimations because the results are highly dependent on how the training and test sets are chosen. 88 Cross validation and bootstrapping can help to overcome this challenge, but the latter technique was not used in the reviewed works. Cross validation was used in different ways in four studies. It is advisable that none of the same person’s measures are in the training set and in the test set. Leave-one-subject-out cross-validation 88,89 can allow for subject-to-subject variation, 88 but was not carried out for any of the machine learning experiments.
Algorithms for automatic detection of health deterioration and service implications
The evidence that telemonitoring can enhance outcomes and reduce costs remains unclear,
10
in part because of the limited performance of the predictive models used.
13,14,20
Conventional threshold-based algorithms, adopted in the majority of reviewed studies (
If a sensitivity level over 70% is considered acceptable,
54
the performance of current threshold-based algorithms may not be adequate for the real-time detection of events after experimental validation. As an alternative, the data-driven (
However, the use of semantic mapping between gathered remote telemonitoring data and existing patient-specific data and models to complement the predictive model using ontologies 91 and standards 92,93 was not found in the literature included in this review. Furthermore, the evaluation of strategies for connecting different predictive personalized models was not considered in the selected studies. Such ‘ensemble modelling’ could offer a compromise between clinical comprehension and statistical goodness of fit in order to improve the quality and robustness of the whole system. 94
Integration of telemonitoring services with the electronic healthcare systems used routinely by healthcare professionals was not described in any of the studies. Instead, an electronic health record was designed for the specific purpose of each trial. This lack of integration with the existing health record systems does not enable intelligent algorithms to interpret data from multiple sources. 95 In this context, novel systems 96 that automatically data-mine inpatient decision support from electronic medical records and that are predictive of real practice patterns and clinical outcomes are being evaluated with promising results. 97
Study limitations
The search was undertaken by a team with extensive experience of home telemonitoring studies and predictive analytics and using multiple databases. Despite this, the present work has limitations. Due to the heterogeneity of the data, a formal meta-analysis was not possible. The literature search was comprehensive but we may have missed home telemonitoring interventions including asthma or COPD but not specifically indexed as such, or because they were not written in English. A main limitation relates to the heterogeneous interventions, which makes comparisons hard and conclusions difficult to draw. Furthermore, the number of studies is insufficient to measure the potential of novel predictive models.
Conclusions
Our review of the algorithms used in home telemonitoring interventions in COPD and asthma suggests that the development of predictive models with clinically useful levels of accuracy, sensitivity and specificity has not yet been achieved.
A range of physiological measurements, consensus in the definition of exacerbations, achievement of high-compliance rates, the development and personalization of predictive models using machine learning techniques and the strategic importance of interoperability of telemonitoring systems with routine healthcare record systems have been discussed in this work as key factors that influence – or potentially could influence – the performance of systems for the remote monitoring of chronic respiratory patients.
New predictive models considering symptoms and physiological measures are being tested in pilot studies with positive outcomes. However, the algorithms that underpin these models have to be validated in large samples of patients, for longer periods of time and with well-established protocols. Further research is needed to identify new features associated with symptoms and physiological signals that enable the early detection of deteriorations, especially in COPD. Only then will telemonitoring achieve the aim of preventing hospital admissions, contributing to the reduction of health resource utilization and improving quality of life of patients.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
