Abstract
Data from digitally administered patient reported outcomes (PROs) is used more and more in routine healthcare for long-term conditions as a part of daily clinical practice. This literature study reviews empirical studies of digital PRO to examine patients’ reasons for non-use of digitally administered PRO data in routine care. This scoping review searched through PubMed, Embase, Web of Science and PsycINFO databases, reporting on study population, intervention, duration of intervention and motivational factors alongside stated reasons for nonparticipation or dropout for each study. The patients’ reasons for not participating, either from study start or by dropout, were analysed through a thematic approach.
Fifty-one studies were included, published from 2010 to 2019, mostly from Europe and the Unites States covering different long-term conditions. The reasons for non-use are manifold and cover the themes of ability to use PRO, engagement, emotional distress and technical barriers.
Several reasons are given explaining why patients with long-term conditions are not using digitally administered PRO as intended. This should be taken into account in the design phase of digital PRO interventions and considered in conversations with patients during the intervention.
Introduction
Digital patient-reported outcomes in routine care
Patient-reported outcome (PRO) data are being increasingly used in routine care of long-term conditions. 1 Discussion continues about their definition but, for the purposes of this study, PROs are defined as standardized, typically validated, generic or disease-specific questionnaires completed by patients to assess their perceptions of their health situation (including health status and quality of life).2,3 Numerous PRO concepts were originally developed for research or general evaluation purposes.3,4 Currently, PROs are gradually applied in clinical practice 5 and serve as screening tools anticipated to detect disease complications or as clinical monitoring tools to assess the impact of treatment.6,7 PROs are expected to facilitate patient involvement, support patients in self-managing long-term conditions and increase health literacy.6,8 However, a limitation to PROs is that all patients do not benefit equally from their use; in fact, some patients do not engage with PROs at all.7,9,10 As the healthcare system becomes more digitalized, so do PROs, allowing patients to complete questionnaires on a computer or smartphone without a healthcare worker present. 11 This digitalization creates additional risks to patient engagement and of increasing inequality in the healthcare system. 12
Reviews exist of factors affecting enrolment in and engagement with digital health interventions. 13 Similarly, reviews of facilitators and barriers to implementation of PROs have been conducted. 14 However, to the best of our knowledge, no reviews exist of the patients’ reasons for non-use of digital PROs in routine healthcare.
The purpose of this study is to review empirical studies of the use of digitally administered PROs in routine care and examine the stated reasons for patients’ non-use of digital PRO.
Research questions
The research questions are as follows: (1) what are the characteristics of patients who decline to participate in digital PRO (nonparticipation), and (2) what are the patients’ reasons for non-use (nonparticipation from the beginning of the intervention and dropout during intervention) of digitally administered PROs in empirical studies of routine healthcare for outpatients with long-term conditions?
Method
A scoping review was conducted using the framework of Arksey and O’Malley 15 and PRISMA-ScR reporting. 16 A scoping review is a quick way to synthesize existing knowledge, with the aim of mapping key concepts and main sources of available evidence in a research area. It differs from a systematic review by addressing a broader research question, including more study designs, structuring data extraction differently and typically synthesizing evidence qualitatively. 15
Eligibility criteria
The focus of this scoping review is digitally administered PROs used in routine outpatient care for adult patients with long-term conditions, defined as any somatic or psychiatric disease requiring ongoing regular treatment or monitoring at a hospital or medical clinic (Figure 1).
Population, exposure and outcome of study.
We included primary studies using digital PROs as part of or as an entire intervention in routine care or routine care like settings, published in English in peer-reviewed journals from January 2008 to January 2019. We defined routine use as incorporating individual patients’ PRO data into clinical practice as a part of their treatment or care.10,17,18 Studies using PROs only as outcome measures for another clinical intervention are not included.
This review focuses on digitally administered PROs that assessed patients’ self-reported health situation including well-being. They included both digital versions of PROs that were originally designed for administration on paper and new PRO instruments specifically developed for digital media. We excluded studies of home monitoring with medical equipment generating clinical data to limit the diversity of technology and focus on patients’ experiences of their health. However, we included studies adding PRO data to existing home monitoring (e.g. ongoing blood glucose measurement self-monitoring).
Furthermore, this review is limited to research regarding adult patients, to avoid any risk of different characteristics relating to children, for example, parent involvement.
Exclusion
Studies were excluded if the PROs component of the intervention was trivial in comparison with the main purpose of the intervention, no clinicians reviewed PRO data, no results were reported (e.g. study protocols) or the focus was on healthcare professionals, not patients or on prevention in otherwise healthy individuals (we viewed obesity and physical inactivity as risk factors, not as long-term conditions themselves). In addition, we excluded studies that did not report on non-participation or dropout rates and those collecting PRO data only from paper-based instruments.
Search and screening
Relevant studies were identified by searching PubMed, Embase, Web of Science and PsycINFO databases. Search terms: (Patient Reported Outcome OR Self-monitoring OR home-monitoring) AND (digital OR online OR web-based OR app OR internet OR m-health OR e-health) within the last 10 years (January 2009–January 2019). A PRISMA screening process 16 followed (Figure 2).
PRISMA diagram.
Data
We collected data on nonparticipant and dropout rates, reasons for nonparticipation or dropout and characteristics of nonparticipants and dropouts. We also reviewed study population, intervention, type of PROs, intervention duration and use of reminders.
Analysis and reporting
Study population, intervention and duration and use of reminders are reported in Table 1, along with reasons for nonparticipation or dropout for each study. Patients’ reasons for non-use, either from study start or by dropout, were analysed using a thematic approach. 70 All reported reasons were (1) recorded as worded in the article, (2) categorized based on close similarities (e.g. ‘language barriers’ and ‘language problems’) and (3) grouped into broad themes based on overall influence on patients’ interaction with digital PROs. After removing reasons reflecting clinicians’ or organizational perspectives, the remaining themes were labelled as shown in Table 4.
Overview of literature.
RCT: randomized controlled trial; PRO: patient-reported outcome.
Results
Included studies
A total of 51 studies published from 2010 to 2019 were included (Table 2). In all, 31 studies were from Europe, 14 from North America, 3 from Australia and 3 from Asia.
Diagnosis of patients in the included studies.
Interventions in included studies were digital questionnaires or web-based diaries and mobile phone applications or text messages, using a ‘bring your own device’ approach or a provided smartphone/tablet. Intervals for completing questionnaire varied from daily to half yearly. The duration of interventions was 2–3 weeks to 2 years.
Use of reminders
Seven studies did not use reminders because the aim was to explore how patients interacted with the system without being reminded of it.41,45,47,51,58,59,64 This included a French study of a self-assessment website for patients with rheumatoid arthritis (RA), 41 25.7 per cent of whom never accessed the website.
In all, 17 studies used written reminders such as email, SMS or push notifications.21,26,27,29–33,39,42,44,49,54,55,60,65,69 Two studies used telephone reminders, and one reminded people when they arrived at the clinic. 48
Nonparticipation and dropout rates
Nonparticipants were defined as patients who were invited to participate in the study or the intervention but never did for patient-related reasons. Nonparticipants reflect both people who do not want to be part of the intervention and those who do not want to join a study. Dropouts are defined as patients beginning the study or the intervention and then dropping out as defined in the study. The included studies variously defined nonparticipants and dropouts. Table 3 summarizes nonparticipation and dropout rates and underlying definitions.
Nonparticipation and dropout rates in included studies.
Characteristics of nonparticipants
Only four studies described the characteristics of nonparticipants;47,30,53,61 no studies included characteristics of dropouts. Robotham et al. reported no difference between active participants and nonparticipants. In the work by Brochman et al., participants and nonparticipants were of similar age, but nonparticipants included more women, were better educated and were less likely to live alone (6% vs 22% of participants). In the work by Kjær et al., both groups were again of similar age and more likely to be women (41% vs 28% of participants). Michaud et al. found that the average age of nonparticipants was slightly less than that of participants (53.6 vs 55.7), but the proportions of women and educational levels were the same.
Reasons for nonparticipation and dropout
Reasons for nonparticipation and dropout are summarized in Table 4 and presented in detail.
Reasons for non-use.
Ability to use PRO
A total of 13 studies reported that patients did not use the intervention because of ‘health problems’ or ‘health issues’ affecting their ability to engage in the intervention. However, most studies did not provide a definition of health problems. Five studies defined ‘health problem’ as patients feeling ‘too ill’ to participate.24,35,45,47,55 Death was a reported reason for dropout in seven studies. Four studies reported ‘old age’ as a reason for patients to decline participation11,21,30,53 without defining an age limit or explaining how age hindered engagement.
Three studies found that language barriers were a reason for dropout.47,55,68 Several other studies used lack of sufficient language skills as exclusion criteria. An English study of a digital questionnaire for people with severe mental illness found that lack of clinical support was regarded as a reason for dropout. 61
Emotional distress
A Danish study of Internet-based questionnaires found that some patients did not want to engage in self-assessment because they needed distance from their disease. 30 A Canadian study of questionnaires for complex chronic diseases found that some patients dropped out because they felt an increased level of anxiety from reporting symptoms. 64 An American study of a smartphone application for lung transplant recipients found that patients did participate in daily health monitoring because they did not want a reminder of their deteriorating health status. 45 Similarly, a Dutch study of an e-health self-management application for breast cancer patients found that some patients wanted to forget about their cancer and felt that the constant assessment was a burdensome reminder. 52
Engagement
Some studies found that patients did not participate because they were ‘not sick enough’. A Korean study of a smartphone application for daily sleep disturbance for people with breast cancer 55 found that some patients did not use the intervention because they had no symptoms to report. The same was the case in a Dutch study of an e-health self-management application for breast cancer. 52 An American study of knee surgery patients exposed to digital questionnaires every 2–4 weeks for 24–26 weeks found that patients stopped completing the questionnaire when their symptoms are reduced. 42 The authors of a Dutch study from 2010 of Internet-based weekly monitoring of asthma symptoms concluded that there was a reduced need for monitoring when the disease was controlled. 67
In addition, the notion of being ‘too busy’ or lacking time to actively engage in the intervention appeared in eight studies.23–25,30,44,48,50,65 In an English mobile telehealth intervention for people with diabetes, people declined participation because they were ‘too busy’ and did not want additional stress related to their disease. 23 A single study 55 found that none of the 38 responders reported that being ‘too busy’ was a problem.
In a Danish study, some patients did not participate because they saw no personal benefits of doing so. 30 Two American studies – a multicentre trial of cancer patients using a tablet in the waiting room 24 and a cohort of knee surgery patients 42 – found that some patients were simply not interested. In a Dutch study of online assessment of disease activity for people with RA, some patients simply do not want to perform disease assessments. 48
A Canadian study of a diary to be used for 4 weeks by patients with complex chronic disease found that data entry became a tedious task, causing some patients to drop out. 64 Likewise, a French study of a self-assessment website for people with RA found that some patients lost interest in self-assessment over a 12-month period and dropped out. 41
An American study of a mobile headache diary found that people who paid for the application had higher adherence than people who got the app at no cost, 62 which the authors explain as the fee increasing the incentive to use the app.
Technical issues and usability
The included studies frequently found technical problems that comprised technical barriers, such as missing hardware or software and low technical proficiency among patients, and technical errors, such as system failures.
An English study of a text messaging system for patients with hip and knee arthroplasty found that poor eyesight (an issued not related to the long-term condition) was stated as a reason for nonparticipation. 29 Other usability issues occur in a Danish study of an online portal for cancer patients, where it is reported that people dropout because the system was too complicated. 35 A French study of a self-assessment website found that a poorly organized website was a reason for dropout. 41 An American survey of an online questionnaire for people with RA found ‘dissatisfaction with online tool’ and technical frustrations as reasons for dropout. 50 Another American study on a web-based electronic messaging system for people with multiple sclerosis found difficulties in using the system as a reason underlying dropout. 54
Data security and trust
In an English mobile telehealth study of diabetes patients, some nonparticipants did not want to be monitored because of concerns about data security. 23 An American study of an app-based questionnaire for collaborative care patients also found that some nonparticipants had issues with passive data collection. 25 A German study of an app-based questionnaire also found uncertainty about the device and technology. 26 A German study of a weekly self-assessment of people with amyotrophic lateral sclerosis found that some patients felt uncomfortable about reporting data, 51 and a French study of a self-assessment website for people with RA found that ‘fear of Internet’ was a reason for nonparticipation. 41
Discussion
Despite substantial variations in the studies included in this review, most reported problems with both nonparticipation and dropout. As noted earlier in this report, included studies did generally not examine differences between participants and nonparticipants. A wide range of reasons were reported by patients as underlying their nonparticipation and dropping out, so it should come as no surprise that people declining participation are diverse. In general, patients who are unwilling to give informed consent to participate in clinical studies are younger and more likely to be women. 71 We included studies of both routine practice and routine-like practice, meaning that some people might have declined participation because of a lack of willingness to participate in research but not necessarily in the intervention itself.
Patients’ reasons for declining participation and dropping out during studies are manifold. We captured these reasons using a thematic approach. However, most studies simply listed the reasons with no further explanation, making it difficult to fully understand the underlying logic and thus group them. A different grouping structure could have been relevant. For all included studies, nonparticipation and dropout data were secondary outcomes. As a result, the list of reasons presented here should not be regarded as comprehensive.
However, this review suggests that a group of reasons for nonparticipation and dropout are prevalent in many studies, and some could be accounted for in the design process. The following sections reflect upon the themes recommended to address in the design process of the digital PRO intervention.
Ability to use PROs
Digitally administered PROs are intended to be a tool for people with long-term health conditions as well as a tool for healthcare professionals managing their care. Many people with long-standing conditions experience severe symptom exacerbations that consume their energy, strength and mental capacity, are hospitalized or die. However, most included studies did not define health issues that interfered with participation, limiting our understanding of the level of health problems that lead to nonparticipation or dropping out. Other circumstances could explain why people leave a study when it is simply reported that they ‘feel too ill’. Interventions designed for people with long-term conditions should allow participants to continue to use them even when they feel ill. When physical or mental conditions hinder patients’ ability to engage, the intervention may be too burdensome in terms of both time and cognitive requirements, compared to the value experienced by patients.72,73
Engagement
Some conditions are monitored using digital PROs when people are experiencing no or very few symptoms, and they may view the intervention as less important or relevant. Eight studies found that some patients reported being ‘too busy’ to participate, suggesting the possibility of perceived discrepancies between personal benefits of and time required for the intervention. Another potential issue related to time bears examination. The included studies differed in length, intensity and dropout rates. For instance, studies of daily reports for 90 days had a dropout rate of 67 per cent, and studies of quarterly reports for 12 months had a dropout rate of only 19 per cent. In general, studies of daily reports had a slightly higher dropout rate than those with less frequent entries. Similarly, the proportion of people who never participated was greater for interventions requiring daily use, compared with weekly use. Although out of scope for this review, further examination of the relationship between intervention intensity, length and dropout rates deserves further investigation. It could also be valuable to investigate the relationship between questionnaire length and dropout rates.
Emotional distress
Another important concern for digital services and, especially, for monitoring health is emotional distress. 73 Repeatedly answering questions about symptoms and general health could contribute to an increased focus on the disease, causing anxiety and general dissatisfaction with the intervention for some people. Four studies reported on this concern but most did not, and its prevalence is consequently unclear from our findings. It is challenging to accommodate this concern in the design of PROs because their very nature is to present questions that might cause emotional distress for patients. However, this is an important concern that may be relevant for clinicians to discuss with patients before asking them to engage in PROs.
Technical issues and privacy
More than half the studies found that technical software or hardware problems were reasons for nonparticipation or dropout. It is obviously necessary for patients to have access to hardware and software, but access alone is not sufficient because technical problems still occur. This can lead to distrust and participants ‘giving up’ on the system. 74 As with any technical solution, designers should be well aware of this potential.
Issues related to trust in digital devices and data security are becoming more widely known, and privacy concerns are very important to some patients. 75 Only a few included studies addressed concerns with monitoring or distrust in digital devices or websites, but thorough investigation could reveal that these issues are of greater importance. Designers must also be aware of these concerns and able to provide assurances about privacy and data security.
System usability is also a major concern. For instance, if people with poor eyesight are unable to use a digital solution, changes in the system are required. If people simply do not know how to use the system, they may drop out. The included studies examined native apps for smartphones, text messages, new webpages and new features added to existing software. There may be differences between these approaches, as well as between different layouts and additional functionalities. We did not investigate usability in this study, as it is not possible for us to make a comprehensive usability testing of the included systems. This means that we cannot know whether the digital systems, poor usability and user experience is a major reason for dropping out. Poor usability could also be related to burdensome systems. 72 Usability should be accounted for when designing digital PROs.
Digitally administered PROs
All of the reasons for nonparticipation and dropout identified in this review are consistent with previous research into prerequisites for use and acceptability of e-health solutions.73,76 Digitally administered PROs are a digital health intervention and a system that requires patients to self-report symptoms and well-being, often on a daily or weekly basis and without the presence of a healthcare worker. Factors affecting the non-use of these systems touch both on barriers to use observed in other digital health solutions and barriers that are more specific to digitally administered PROs. A need is apparent to focus on the potential for emotional distress and the relationship between patients’ symptoms and the intensity of symptom reporting required from digital PROs.
Limitation
Conducting a comprehensive search is challenging because digital questionnaires are used as outcome measures in many clinical studies and it is often hard to distinguish these from PROs used as part of or an entire intervention. Similarly, many terms can describe PRO interventions. We tried to capture this variety using a range of very broad search terms, but we could have missed some studies. In addition, many studies of digital PROs combine them with other interventions. Most of these studies were excluded from this review. Including these data could have led to different results. Furthermore, this review only included studies that reported on nonparticipation and dropout. Excluding studies that did not do so could have created bias affecting our findings in unknown ways.
Issues not addressed in this review are clinicians’ attitude towards interventions and patients’ ability to engage in interventions. In most studies we included, clinicians invited patients to participate in the intervention; the sample of patients could thus be biased before the recruitment process starts and clinicians themselves may be a significant barrier to the use of digital PROs.71,77 In addition, this review does not distinguish between dropout and early termination and only addresses the dropout definitions used in the examined studies. A more detailed analysis of when and why patients stop the intervention could lead to a more comprehensive and nuanced understanding.
Conclusion
Numerous factors are at stake when patients with long-term conditions do not use digital PROs as intended. This should be reflected when designing digital PRO interventions in collaboration with patients, for example, in participatory design studies. 78 Relevance to patients’ current health situation is of utmost importance, and appropriate actions should be considered when their health situation changes.
Some reasons for nonparticipation stated before the intervention begins could be related to fear or expectations. They may be addressed by providing information to patients and screening for concerns identified in the studies we included. The digital solution is often not a stand-alone solution and interaction with the clinic and surrounding factors can play a great part in a successful intervention.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
