Readiness,acceptance and use of digital patient reported outcome in an outpatient clinic

Abstract

Use of digital patient-reported outcomes is being introduced in care of chronic conditions, including Inflammatory Bowel Disease. The aim is to supplement face-to-face follow-up sessions through symptom screening, and to inform follow-up through questions about mental health and quality of life. However, little is known about who is using this as intended. This study aimed to map differences between users and non-users among people with IBD and explore the mechanisms behind. We administered a questionnaire including the Readiness and Enablement Index for Health Technology (ReadHy) and the Service User Technology Acceptability Questionnaire (SUTAQ) to all people with IBD registered at Silkeborg Regional Hospital. Comparison between users and non-users and cluster analysis was conducted. Effect size (Cohen’s d) was used to estimate magnitude of difference between groups. The user and non-user groups differed most strongly by level of emotional distress (d = 0.45). Cluster analysis of the ReadHy scales showed profiles have different sets of difficulties and reservations towards digital solutions. These difficulties correlated moderately with SUTAQ dissatisfaction and low acceptability. The dimensions of ReadHy may help to better understand particular needs of people with IBD when accessing digital PROs, which may lead to higher acceptability and improved quality of care.

Keywords

ehealth telehealth healthcare service innovation and IT human factors telecare

Introduction

Patient Reported Outcomes (PROs) are measures of outcomes reported directly from individuals. In this study we define digital PROs as digitally administered, standardized questionnaires (either generic or disease-specific) which measure people’s perception of their health and wellbeing.^1,2 Digital PRO concepts have been developed, tested and implemented into routine healthcare services for a wide range of long-term conditions,^3–5 where they can serve as screening, outcome and monitoring tools. PROs are anticipated to facilitate patient involvement, support people in self-managing chronic conditions and increase health literacy.^5–8 Healthcare users answer questions about their lived experience and health situation using a computer or smartphone, without the presence of a healthcare worker.⁵ Use of digital PROs may require both technical competence, independence, self-care, engagement and motivation.⁹

Digital PROs have been introduced for people with Inflammatory Bowel Disease (IBD) in outpatient care in Denmark. An aim of this process is to reduce the number of face-to-face follow-up sessions through periodic symptoms screening. A second aim is to inform follow-up sessions by introducing questions about mental health, quality of life and health-related issues into the consultation.^10,11 The chronic nature of IBD with multiple relapses can have wide-ranging influences on a person’s emotional, physical, and social wellbeing.^12,13 Enhanced focus on these issues can lead to improvements in patient health and well-being,^11,14,15 and digital PROs are believed to improve care of people suffering from IBD by focusing on these topics.¹⁶ However, many barriers to implementation of newly developed digital health solutions, such as digital PROs, into routine practice, have been identified, including lack of education of participants and lack of clarity of the purpose of the system.^17–19 Despite the identified barriers, little is known about who declines participation or who is not offered digital PROs as part of their treatment²⁰; why some do not engage with digital PROs; and how clinicians’ attitudes affect uptake and engagement.

Various authors have attempted to conceptualize patient readiness and attitudes towards digital health solutions, demonstrating the importance of understanding that diverse people have different preferences and needs regarding their health situation.^9,20,21 The Readiness and Enablement Index for Health Technology (ReadHy) framework²³ explores mechanisms behind readiness to use digital health technologies. Readiness is defined as being prepared and willing to use digital health technology.²³ ReadHy builds on the eHealth Literacy Framework²⁴ with inclusion of social and emotional aspects of use, and captures a holistic perspective on patients’ readiness for digital health solutions. The corresponding ReadHy instrument²³ maps an individual’s prerequisites of readiness for and the mechanisms behind usage of digital health technology.

This concept of readiness to engage with digital health solutions describes factors considered essential to be aware of when introducing a new technology to an individual to obtain the highest likelihood of adoption and adherence. The ReadHy instrument is able to measure enablement through health technology over time.²³ ReadHy has been developed to measure readiness for generic or specific health technologies. However, it does not address particular characteristics of a technology or potential perceived benefits, as embedded in the theories of technology acceptance.²⁵

Another approach to investigate attitudes towards e-health technology is assessing the acceptability of a specific technology.²⁶ The Service User Technology Acceptability Questionnaire (SUTAQ) is a six-dimension instrument that measures the perceived acceptability of a particular technology and context. The authors of SUTAQ define acceptability as the sum of the instrument’s six domains (Satisfaction, As substitution, Enhanced Care, Increased Accessibility, Privacy and Discomfort, and Thoughts of personnel).²⁷ SUTAQ was developed in the context of a large telemedicine projects – “The whole system demonstrator” in the U.K.,²⁷ and has been used to evaluate telehealth interventions in a range of large studies to explore factors relating to use and potential use of health technology.^28–31

The inclusion of users’ and non-users’ competence, eHealth literacy, and social and emotional context, in contrast to a focus on perceived ease of use and usefulness of a particular technology, may help to better understand why users do not adopt and use a new technology. Whether these concepts relate to each other and to non-use of digital PROs in diverse settings is yet to be explored.

The aim of this study was to describe characteristics relating to eHealth literacy and technology readiness of current users and non-users of digital PROs, and to explore concepts of readiness for a digital health intervention and perceived acceptability of a current digital PRO solution. This approach may help to uncover determinants of use and non-use of PRO solutions to inform future interventions.

Methods

Setting

This study is part of an explanatory sequential designed mixed-method observational study³² of people with IBD and clinicians at Silkeborg Hospital, Diagnostic Centre, Central Region Denmark. Here, digital PROs data have been used in care for people with IBD for 2 years prior to this study. Digital PROs are used in several ways: as a clinical follow-up to replace a consultation, to prepare for consultations, to inform the consultation, as a way of monitoring disease activity between consultations, or a way for people to contact the clinic. People with IBD are registered in the digital PRO solution, named AmbuFlex,³³ after an initial conversation with their doctor. Non-Danish speakers and people who do not have access to the secure, national digital mail system are excluded. An individual flow is set up for each patient following general guidelines based on health status. Patients with stable remission usually receive the PRO measure (40–50 questions on symptoms, general health and wellbeing, and need of contact) every 2–12 months to check their health status. Up to three reminders are sent. Patients receive a link to the questionnaire in their national secure email system, or they enter a webpage with a logon opportunity. The questionnaire is then presented to the patients and generally regarded as acceptable by most patients.³⁴ If health indicators are low, a nurse will make a call by telephone and if necessary invite the patient to the clinic for follow-up. When used as a preparation for the consultation, the PRO measure is sent 4 days before arrival at the clinic with a reminder the day before arrival. If the patient has not completed the questionnaire, they can do so on arrival at the clinic. Additional information on the tool is published elsewhere.³⁵

Survey

The purpose of the survey was to examine characteristics and readiness of both users and non-users of digital PRO at the clinic where the solution had been running for 2 years. The questionnaire was sent to all patients with IBD (including those not enrolled or excluded from the PRO setup) (n = 827) at the clinic. To target the entire group, the questionnaire was sent directly to each patient through electronic secure mail (n = 784) or on paper to their home address (n = 43) if they did not use electronic mail system in January 2018. Up to two reminders were sent to non-responders. Data collection ended in June 2018. Seventy-four percent were at that time enrolled in the digital PRO solution. Dropout was defined as previously been assigned to the PRO solution, but no longer enrolled by the time we sent out the survey.

Measures

Demographic data (age and gender) were collected though the patient administrative system (PAS). Educational status, duration of IBD, use of IT in general and in healthcare were self-reported through questionnaire and use of the digital PRO system (date of enrolment and dropout) was collected through the PRO database.

To establish the expected readiness for digital PROs, the Readiness and Enablement Index for Health Technology (ReadHy) questionnaire was used. Evidence of acceptable psychometric properties of the ReadHy as a tool for exploring the level of readiness for digital health intervention on an individual and a general level was developed in a sample of people with cancer in Denmark.²³

ReadHy builds on the eHealth Literacy Framework (eHLF)²⁴ and the associated eHealth Literacy Questionnaire (eHLQ),³⁶ and includes dimensions from two other widely used tools: the Health Literacy Questionnaire (HLQ)³⁷ and the Health Education Impact Questionnaire (heiQ).³⁸ The HLQ and the heiQ have undergone extensive psychometric testing in multiple languages across multiple cultures and found to have strong psychometric properties.^37–41 The eHLQ was developed in Denmark where extensive initial testing demonstrated it had excellent psychometric properties according to both classical test theory and Item Response Theory.³⁶

The ReadHy tool consists of 13 dimensions (heiQ3 Self monitoring and insight, heiQ4 Constructive attitudes and approaches, heiQ5 Skills and technique acquisition, heiQ8 Emotional distress, HLQ1 Feeling understood and supported by healthcare providers, HLQ4 Social support for health, eHLQ1 Using technology to process health information, eHLQ2 Understanding of health concepts and language, eHLQ3 Ability to actively engage with digital services, eHLQ4 Feel safe and in control, eHLQ5 Motivated to engage with digital services, eHLQ6 Access to digital services that work, eHLQ7 Digital services that suit individual needs). In this project, an additional dimension from Health Literacy Questionnaire; HLQ3 (Actively managing own health) was also used. Therefore, for this study the ReadHy contained 14 scales with 70 items using a 1–4 scale (strongly disagree, disagree, agree, and strongly agree). One scale (heiQ8 Emotional distress) is reverse scored, where a higher score means worst status (greater emotional distress). For ease of interpretation scoring was inversed for analysis to enable direct comparison with the other dimensions (that is, a higher score indicates better status). For each scale, a mean score was calculated.

As use of digital PRO was only measured by enrolment and dropout, we wanted to explore health technology acceptability among users as a potential factor relating to use and possible future use. We therefore examined acceptability as measured through SUTAQ²⁷ among the users of digital PROs to understand the relationship, if any, between readiness and acceptability. Acceptability of the current technology was measured by SUTAQ.²⁷ The measurement of acceptability of a certain technology is assessed in SUTAQ through 6 scales: Satisfaction, As substitution, Enhanced Care, Increased Accessibility, Privacy and Discomfort, and Thoughts of personnel in a 22-item questionnaire using a scale from 1 to 6.²⁷ SUTAQ has been used in a Danish context^29,30 but has not been psychometrically tested in a Danish telehealth context. All respondents were asked if they were signed up to AmbuFlex. If they answered “yes” or “do not know,” they were given the SUTAQ questionnaire.

Statistical methods

R statistics 3.5.1 was used to calculate descriptive statistics for users and non-users of digital PROs, and comparisons between the two groups on use of digital PROs, IT in general and in healthcare including 95% confidence intervals (95% CI). Descriptive statistics are presented as means and comparisons are made using Welch Two Sample t-test (numerical data; age and difference between ReadHy dimensions), Pearson’s Chi-squared (categorical data; gender, educational level and IT use), and Fishers’ Exact test (length of condition more or less than 2 years). Effect size was measured through Cohen’s d for difference between ReadHy dimensions. An effect size d < 0.2 was regarded small, 0.2–0.5 was regarded medium, and 0.5–0.8 was regarded high.⁴²

To explore if different profiles exist among survey participants, we conducted a cluster analysis of ReadHy scales. Cluster analysis is suitable to identify certain subgroups believed to be present in a sample.⁴³ Clustering methods are used in health literacy research,^34,41 using hierarchical clustering to be able to examine the detailed differences between groups relating to e.g. HLQ-dimensions, e.g. using the Ophelia process.⁴¹ In this study we conducted a quicker non-hierarchical clustering due to the large number of dimensions in the ReadHy instrument and because the purpose was solely to illustrate different groups to examine whether there were differences between users and non-users of digital PROs, not to examine the differences in detail.⁴³ Clusters were defined using k-means clustering method by enforcing different numbers of cluster solutions. If ≥ 50% of the items in a dimension were answered, an average for the scale was calculated. A range of 4–20 cluster solutions were performed and the attributes of the clusters solutions were examined, first by the first author of this paper, later by a group of clinicians to ensure that the clusters resembled experiences in the clinic. The k-means algorithms were run several times for each cluster solution to detect the solution with the lowest total within cluster sum of squares.

The SUTAQ mean scores were calculated for each scale, handling missing items in the same ways as the ReadHy calculation. Mean scores for each scale were calculated for each ReadHy cluster to be able to qualitatively explore the relation between the two used questionnaires and their dimensions, not to conduct confirmatory factor analysis or validity testing of the questionnaires. For this simpler purpose a correlation matrix between ReadHy dimensions and SUTAQ dimensions on the entire sample was conducted using the Pearson’s correlation. A correlation >0.6 was regarded as high, and a correlation between 0.3–0.59 as moderate.⁴⁵ Due to the explorative design, we made no mathematical correction for multiple testing.

Ethics and data protection

The study was reported to the [details omitted for double-anonymised peer review] and approved by [details omitted for double-anonymised peer review].

The survey was conducted via [details omitted for double-anonymised peer review] password protected, secure IT-system, and data stayed within that system throughout analysis. Participants were informed by the survey through the national e-mail system or by mail and gave consent by participation.

Results

The response rate to the survey was 52%. Characteristics are shown in Table 1. Of the people who did not answer, five reported technical problems, and one was found to be deceased. Fourteen percent of those who responded replied that they did not want to participate (declined).

Table 1.

Characteristics of respondents and non-respondents to the survey.

	Mean age	Min-Max age	Female N (%)	PRO user N (%)	Dropout N (%)	Do not Have E-boks (secure national email)^c N (%)
Total (n = 827)	51.4	18–94	466 (56)	611 (74)	20 (2)	38 (5)
Accepted (n = 369)	53.4	20–86	204 (55)	301 (82)	4 (1)	6 (2)
Insufficient data for analysis^a (n = 37)	54.7	23–79	20 (54)	25 (68)	1 (3)	4 (11)
Declined^b (n = 62)	62.5	18–94	38 (61)	36 (58)	3 (5)	11 (18)
Not answered (n = 396)	47.9	18–92	224 (57)	274 (69)	13 (3)	17 (4)

^aLess than half of the questions in each dimension answered.

^bNotified researchers with “I do not want to participate.”

^cIn Denmark all citizens have the option to have a national secure email for all communication with public institutions.

Users and non-users of PROs

Fifty-eight (17%) of the 332 participants who provided sufficient data were not part of the PRO solution (non-users). Only five did not have a national secure e-mail system (exclusion criterion). The non-user group were older and less educated than the PRO users, as seen in Table 2. The PRO users had greater everyday use of IT, whilst the non-users tended to use the Danish public national health portal (sundhed.dk) more, however this difference was not significant.

Table 2.

Characteristics of survey participants

	Non-PRO^a-user (n = 58)	PRO^a-user (n = 274)	Percentage difference (95%CI)
Mean age (years)	60.5 (29–86)	52.5 (20–84)
Female	27 (47%)	157 (57%)	10 (1–17)
Time since diagnosis >2 years	44 (75%)	252 (92%)	17 (8–25)
Everyday use of information technology	31 (53%)	206 (75%)	22 (14–43)
Use of health related information technology	35 (60%)	164 (60%)	0 (-22–15)
Use of Danish national public health portal (sundhed.dk)^c (often or every time)	38 (66%)	145 (53%)	13 (-2–25)
Education 2–5 years post high school	36 (62%)	194 (71%)	9 (0–26)
Not using national secure digital mail system	5 (9%)	1^b

^aPatient Reported Outcomes.

^bShould not be eligible, but appear of unknown reasons in the data.

^cSundhed.dk is a portal for general health information and personal health data, e.g. test results.

The ReadHy profile amongst PRO users and non-users was similar, with only a significant difference observed in HeiQ8 (Emotional distress) (0.29 (95% CI: 0.07–0.51)) with a medium effect size, see Table 3. A small non-significant effect was also seen with heiQ4 (Constructive attitudes and approaches) and eHLQ6 (Access to digital services that works) and eHLQ7 (Digital services that suits individual needs).

Table 3.

ReadHy (Readiness and enablement index for Health technology) profiles for current PRO* users and non-PRO* users

ReadHy scales^c	PRO^a user (mean) (n = 274)	Non-PRO^a user (mean) (n = 58)	Difference (95% CI^b)	Effect size (Cohen’s d)
heiQ3 Self-monitoring and insight	2.97	2.98	0.01 (−0.15–0.12)	0.07
heiQ4 Constructive attitudes and approaches	3.17	3.02	0.15 (−0.02–0.31)	0.34
heiQ5 Skills and technique acquisition	2.93	2.97	0.04 (−0.19–0.10)	0.05
heiQ8 Emotional distress	3.06	2.77	0.29 (0.07–0.51)	0.45
HLQ1 Feeling understood and supported by healthcare providers	2.92	2.80	0.12 (−0.06–0.29)	0.08
HLQ3 Actively managing own health	2.73	2.71	0.02 (−0.19–0.15)	0.03
HLQ4 Social support for health	2.96	2.89	0.07 (−0.10–0.23)	0.09
eHLQ1 Using technology to process health information	2.61	2.55	0.06 (−0.09–0.21)	0.13
eHLQ2 Understanding of health concepts and language	2.93	2.91	0.02 (−0.10–0.14)	0.04
eHLQ3 Ability to actively engage with digital services	2.92	2.87	0.05 (−0.11–0.21)	0.17
eHLQ4 Feel safe and in control	2.94	2.90	0.04 (−0.10–0.18)	0.11
eHLQ5 Motivated to engage with digital services	2.63	2.58	0.05 (−0.19–0.20)	0.12
eHLQ6 Access to digital services that work	2.79	2.68	0.11 (−0.04–0.25)	0.24
eHLQ7 Digital services that suit individual needs	2.64	2.52	0.12 (−0.04–0.27)	0.24

^aPatient Reported Outcomes.

^bConfidence Interval.

^cReadiness and enablement index for Health technology (Prefix of scale names refer to the scales from which they originate; Health Education Impact Questionnaire, Health Literacy Questionnaire and eHealth Literacy Questionnaire.

ReadHy cluster analysis

To explore if digital PROs could be offered to people in a more informed way, a full ReadHy (14 scales) cluster analysis was conducted with all participants with complete data (users and non-users, n = 332). The resulting 6 cluster solutions are shown in Table 4 with additional characteristics shown in Table 5. Together, they describe a range of profiles constructed from the observed combination of scores from the 14 scales plus demographic and other associated data with the cluster with the highest overall ReadHy score presented first:

Table 4.

Six profiles based on ReadHy (Readiness and enablement index for Health technology).

Cluster	Self-monitoring and insight heiQ3	Constructive attitudes and approaches heiQ4	Skills and technique acquisition heiQ5	Emotional distress heiQ8	Feeling understood and supported by healthcare providers HLQ1	Actively managing own health HLQ3	Social support for health HLQ4	Using technology to process health information eHLQ1	Understanding of health concepts and language eHLQ2	Ability to actively engage with digital services eHLQ3	Feel safe and in control eHLQ4	Motivated to engage with digital services eHLQ5	Access to digital services that work eHLQ6	Digital services that suit individual needs eHLQ7
1	3.43	3.74	3.58	3.21	3.29	3.61	3.74	3.50	3.79	3.81	3.63	3.60	3.52	3.57
2	3.17	3.77	3.30	3.67	2.71	2.96	3.32	2.54	3.05	3.15	2.96	2.61	2.83	2.67
3	3.03	3.05	2.91	2.99	2.77	3.00	2.97	2.79	3.01	3.06	3.00	2.86	2.91	2.85
4	2.81	3.00	2.84	2.94	2.60	2.75	2.76	2.26	2.67	2.50	2.78	2.23	2.49	2.23
5	2.70	2.42	2.46	2.08	2.68	2.60	2.53	2.71	2.76	2.92	3.01	2.69	2.72	2.55
6	2.53	2.67	2.34	2.59	2.44	2.43	2.00	1.68	2.30	1.78	2.18	1.60	1.98	1.56

Prefix of scale names refer to the scales from which they originate; Health Education Impact Questionnaire, Health Literacy Questionnaire and eHealth Literacy Questionnaire.

Table 5.

Characteristics of the six profiles presented in Table 4.

Cluster number	Number of participants (%)	Mean age (min-max)	Females	Education level (1–4) (low-high)	PRO^a user	Mean of number of PRO^a responses per user	Use of sundhed.dk^b (0–3) (never-often)
1	25 (8%)	50.7 (25–86)	53.4%	2.92	84%	2.36	2.12
2	70 (21%)	50.8 (20–78)	60.0%	2.75	86%	2.25	1.59
3	101 (30%)	53.9 (21–82)	49.5%	2.80	82%	2.66	1.95
4	80 (24%)	57.1 (21–83)	53.8%	2.28	78%	2.06	1.05
5	39 (12%)	52.1 (21–84)	69.2%	2.56	77%	2.56	1.74
6	17 (5%)	54.7 (29–81)	47.1%	1.53	76%	1.64	0.58
Total	332	53.7 (20–86)	55.4%	2.58	83%	2.37	1.58

^aPatient Reported Outcomes.

^bSundhed.dk is a portal for general health information and personal health data, e.g. test results.

Cluster (1) profile included 8% of the population. This group scored high on all ReadHy scales. They were younger and were somewhat better educated than the rest, and they had a high use of the national health portal.

Cluster (2) profile included 21% of the population. This group scored high on most scales, but relatively low on eHLQ1 (Using technology to process health information), which corresponded to a relatively low use of the national health portal, a low degree of eHLQ5 (Motivation to engage with digital systems and moderate), and eHLQ7 (Digital services that suit individual needs). This group also scored relatively low on HLQ3 (Actively managing health). On the other hand, they scored high on heiQ8 (Emotional distress), which indicates low emotional distress. The group was as young as people in profile 1 and had a greater number of females, but a slightly lower educational level.

Cluster (3) profile represented 30% of the population and resembled the profile of cluster 1 with regard to the relations between the different scales, but had general lower values in all scales. This group was older than people in profile 1 and there were fewer females. They had the greatest number of PRO responses for each user.

Cluster (4) profile had relative low scores in the more digitally-oriented eHLQ scales and social HLQ scales, but high scores on heiQ scales like Emotional distress and Constructive attitudes and approaches. It represented 24% of the population. This was the oldest group with a low educational level and low degree of use of the national health portal.

Cluster (5) profile included 12% of the sample, had the lowest score on heiQ8 (emotional distress), indicating a high level of emotional distress. In addition, heiQ4 (Constructive attitudes and approaches) and heiQ5 (Skills and technique acquisition) had low scores. In contrast, they scored relatively high on eHLQ4 (Feeling safe and in control) and eHLQ3 (Ability to actively engage with digital services). This group had a high percentage of females.

Cluster (6) profile had the lowest overall score. The lowest scores were on the digitally oriented eHLQ scales and on HLQ4 (Social support for health). This group covered only 5% of the population. They had the lowest use of the national health portal and the lowest educational level. They had the lowest level of PRO responses per user.

Acceptability of the digital PRO solution

The correlation between scales in ReadHy and SUTAQ is shown in Table 6.

Table 6.

Correlation matrix of ReadHy (Readiness and enablement index for Health technology) and SUTAQ (Service User Technology Acceptability Questionnaire) scales by Pearson’s correlation. Correlations <0.10 omitted for clarity.

ReadHy^a scales	SUTAQ 1Satisfaction	SUTAQ 2As substitution	SUTAQ3Enhanced care	SUTAQ 4Increased accessibility	SUTAQ 5Privacy and discomfort	SUTAQ 6Thoughts of personnel
eHLQ1Using technology to process health information	0.20	0.26				0.27
eHLQ2Understanding of health concepts and language	0.24	0.25		0.20		0.20
eHLQ3Ability to actively engage with digital services		0.18				0.32
eHLQ4Feel safe and in control	0.51	0.40	0.25	0.26	0.30	0.46
eHLQ5Motivated to engage with digital services	0.46	0.48	0.21		0.32	0.40
eHLQ6Access to digital services that work	0.50	0.45	0.22	0.28	0.32	0.48
eHLQ7Digital services that suit individual needs	0.53	0.51	0.23	0.27	0.29	0.54
heiQ3Self-monitoring and insight
heiQ4Constructive attitudes and approaches				0.18
heiQ5Skills and technique acquisition
heiQ8Emotional distress
HLQ1Feeling understood and supported by healthcare providers	0.20			0.28
HLQ3Actively managing own health
HLQ4Social support for health	0.24			0.19		0.21

^aReadiness and enablement index for Health technology (Prefix of scale names refer to the scales from which they originate; Health Education Impact Questionnaire, Health Literacy Questionnaire and eHealth Literacy Questionnaire.

Only three of the SUTAQ scales (SUTAQ1 (Satisfaction), SUTAQ2 (As Substitution) and SUTAQ6 Thoughts of personnel) were moderately correlated with some eHLQ scales; highest with eHLQ4 (Feel Safe and in control) (r = 0.25–0.51), eHLQ6 (Access to digital services that work) (r = 0.22–0.50) and eHLQ7 (Digital services that suits individual needs) (r = 0.23–0.54), and eHLQ 5 (Motivated to engage with digital services) (r = 0.21–0.48). The SUTAQ scales, in this sample, was divided in two groups; SUTAQ3 (Enhanced care), SUTAQ4 (Increased accessibility) and SUTAQ5 (Privacy and discomfort) to a lesser extent with the eHLQ scales (r = 0.21–0.32), whilst SUTAQ1 (Satisfaction), SUTAQ2 (As substitution) and SUTAQ6 (Thoughts of personnel) was correlated to a higher degree with the eHLQ scales (r = 0.40–0.54).

In addition, SUTAQ 2 and 6 correlated moderately with all eHLQ scales, whilst SUTAQ 1, 4 and 6 are the only ones to correlate weakly with scales from HLQ and heiQ.

Acceptability of the digital PRO solution amongst users, as measured by SUTAQ, is summarised in Table 7 divided into the six clusters.

Table 7.

Means scores on SUTAQ (Service User Technology Acceptability Questionnaire) scale for each of the six profiles illustrated in Table 4 (scale from 0 to 6).

Cluster number	Number of respondents (SUTAQ)	SUTAQ 1: Satisfaction (SD)	SUTAQ 2: As substitution (SD)	SUTAQ 3: Enhanced care (SD)	SUTAQ 4: Increased accessibility (SD)	SUTAQ 5: Privacy and discomfort (SD)	SUTAQ 6: Thoughts of personnel (SD)
1	14	5.57 (0.68)	4.07 (0.74)	4.76 (0.71)	4.43 (0.68)	5.56 (0.56)	4.88 (1.19)
2	46	4.81 (1.06)	3.53 (0.86)	4.12 (0.92)	3.85 (1.14)	5.28 (0.84)	4.58 (1.24)
3	68	4.75 (0.86)	3.82 (0.79)	4.45 (0.74)	4.19 (0.91)	5.28 (0.84)	4.61 (1.15)
4	56	3.94 (1.11)	3.53 (0.77)	3.77 (0.95)	3.38 (1.08)	4.92 (0.89)	4.23 (1.08)
5	28	4.86 (0.84)	3.56 (0.78)	4.03 (0.88)	3.96 (0.84)	5.25 (0.81)	3.96 (1.22)
6	7	3.43 (1.01)	2.81 (1.29)	2.63 (0.91)	2.39 (0.93)	4.96 (1.33)	3.95 (1.19)
Total	219

Profile 6, with only 7 respondents, with the lowest overall ReadHy scores, were also low on some SUTAQ scales [SUTAQ2 (As substitution), SUTAQ3 (Enhanced care) and SUTAQ4 (Increased accessibility)], whilst profile 1, with the highest scores on ReadHy also had the highest scores on SUTAQ. However, the profiles in between had a more diverse SUTAQ profile as for example profile 5, who had a slightly higher score on eHLQ scales relatively to the HLQ and heiQ scales (see Table 4) scores higher on SUTAQ scales, than profile 4, who had lower eHLQ-scores, but higher heiQ and HLQ scores.

Discussion

This study sought to explore characteristics relating to eHealth Literacy, readiness and acceptance among users and non-users of a digital PRO solution. We found that the level of emotional distress is a key difference between users and non-users. Due to unclear primary outcome and uncertainty about the standard deviation of the sample, needed sample size was not calculated prior to this study. However, it is likely that people who have poor mental health are reluctant to engage in a new intervention.^46–48 Also, clinicians understandably hesitate to invite people to engage with digital health interventions, and thereby add additional burden, if they are not feeling well.¹⁸ When the remaining ReadHy scales are considered from a univariate perspective, they did not show any significant difference between users and non-users of digital PROs in this context. However a small effect was seen within heiQ4 (Constructive attitudes and approaches) and eHLQ6 (Access to digital services that works) and eHLQ7 (Digital services that suits individual needs). Besides being a consequence of the enrolment procedure, the absence of further differences could be a consequence of there being few non-users in the analysis. However, the user-group tended to score higher on each domain of eHLQ, a similar pattern to that reported in other studies.^36,49

The creation of profiles contributed to deeper understanding of the mechanisms behind use and non-use. The three profiles with the overall lowest ReadHy score had only 76–78% users while the top three profiles had 84–86% users, indicating the relevance of the profiles when it comes to use of digital PROs. For example, profile 5 showed high emotional distress, which might affect use of digital PROs negatively. However, the same group scored high (above 2.5) on the eHLQ-scales, which are the dimensions with most focus on the digital aspects of the intervention. These people therefore do have the ability to engage with digital services; they feel safe and are somewhat motivated. The analysis also showed a group with the opposite profile. This group (profile 4) had very low motivation to engage in digital services, had low emotional distress, and low use of other health-related technology, including the national healthcare portal. The different profiles have different facilitators and barriers for readiness towards digital health technology. Despite this, the variation in the percentage of users versus non-users within each clusters, is not high. This might relate to the enrolment procedure, where some clinicians enrol all patients regardless. However, awareness of the patients’ individual prerequisites might make them more comfortable with the digital PRO solution and thereby increase future use.

We found a moderate association between some ReadHy scales assessing participants’ general readiness for technology use and SUTAQ scales assessing the perceived acceptability of the specific digital health intervention, i.e. digital PROs. The eHLQ5 (Motivation to engage with digital services) correlated moderately with all SUTAQ scales except SUTAQ4 (Increased access). Both SUTAQ and eHLQ have a specific focus on technology while HLQ and heiQ do not. In that way, the eHLQ matches the technology acceptance theories^50,51 to a higher degree than the contextual dimensions and the social dimensions from HLQ and heiQ. This aligns with other studies that have shown that users of technology are more likely to adopt new technologies,⁵² and the PRO users in our study did have a high use of everyday IT.

It was only possible to test health technology acceptability among those participants who actually received the digital PRO solution, but our study showed a fine match between the patient profiles with high believed readiness to digital technology and acceptability of the current digital PRO solution. Profile 1 from the cluster analysis showed both the highest overall score in ReadHy and in SUTAQ, whilst Profile 6 had the lowest score on both measures. The correlation between the eHLQ scales and SUTAQ was also reflected this pattern. For example, people within Profile 5 (with a high degree of emotional distress) reported that digital PRO solutions enhanced their care and they expressed satisfaction with the intervention to a higher degree than Profile 4 (with a lower degree of emotional distress i.e. high score, but less use of digital tools). While satisfaction with digital services is likely important, there are clearly a range of mechanisms that determine the uptake of a digital PRO solution, and could be taken into account in implementation of a digital PRO solution.

Interestingly, the heiQ8 (Emotional distress) scale in the ReadHy instrument was not correlated with any SUTAQ scale, despite this dimension being the only one in which there was a difference between users and non-users. This indicates that mental health is an independent predictor of use of digital PRO uptake. People can have either high (or low) scores on the ReadHy or SUTAQ, yet not have the mental wellbeing to face and use digital PROs. Clinicians may also be responding to peoples’ emotional distress, promoting face-to-face contact and therefore not enrolling those with high distress in digital PROs, or it could be that people with high emotional distress decline participation. Either way, this study indicates that people might accept the technology despite emotional distress, and therefore this should not be the only indicator for enrolment. Further investigation into the impact of emotional distress on self-management and use of digital technologies could be relevant.

The association between heiQ8 (Emotional distress) and non-use may inform understanding of previous research indicating clinicians can be a barrier for implementation of digital health interventions.^18,53,54 Clinicians often do not ask people to participate in studies if they feel that they are not interested.⁵⁵ In the setting for the current study, clinicians cannot predict who might benefit or not benefit from the PRO solution. Some of the currently enrolled users of PROs might have difficulties with using the system, while some current non-users might benefit from the technology. In this setting, a person may only receive a single questionnaire once a year, that is, the burden on users is not high. Still, it is possible that some find it very troublesome, including having a specific phobia towards technology and would prefer a personal contact with the clinician.⁵⁶ In addition, it might not only be the digital aspect of using PROs that patients find troublesome.³⁴ Other factors associated with digital PRO, such as sitting home alone assessing one’s symptoms, might be more disturbing to some patients than others.^34,57 If clinicians can be assisted in identifying who might need extra help or guidance, for example by screening patients’ technology readiness and in particular their eHealth literacy, this could increase clinicians’ willingness to use digital PRO, as they may better understand where an individual patient may feel challenged by the technology.

The finding that patients’ level of emotional distress affects their likelihood of using digital PRO calls for increased attention to be paid to patients’ emotional wellbeing, as this factor seems to influence their receptiveness to a new technology even if they have a sufficient level of eHealth literacy. Consequently, before trying to introduce digital PRO or other technologies, clinicians need to be aware of and address their patients’ negative thoughts or concerns.

Strengths and limitations

This study is based on a specific conceptual framework that has emerged over many years of research.^23,24,38,58 The framework strengthens the study as it guided the process of exploring mechanisms and concepts behind use and non-use. However, whilst most components of the ReadHy tool have been very well tested, the ReadHy itself has received limited testing. Further testing is warranted to determine if the concept of readiness is effectively measured through the ReadHy. Likewise, the SUTAQ questionnaire is often used in implementing new digital health technologies, but its validity is not well tested. However, this study has used these tools to explore concepts of health technology readiness and acceptance, and has provided insights into associations between technology acceptance and readiness to help focus future validity testing. The value of this paper may not be to provide results that can be directly generalised to other populations, but to explore mechanisms and concepts behind use and non-use of digital PROs, and to enhance understanding of existing conceptual frameworks.

The survey in this study only had a 52% response rate and the percentage of non-users in our study is 18%, while it is 26% in the cohort. Non-respondents were slightly younger than respondents, but we did not target the youngest group, neither did we catch the oldest group. Different ReadHy profiles may have been found in the non-respondent group. In addition, a group of non-users made the effort of writing back to us by post, stating that they would not participate in our survey. These people are non-PRO users, and it would have been valuable to examine their ReadHy profiles to better understand this group and whether additional mechanisms behind non-use exists within this group.

Conclusion

The level of emotional distress amongst people with IBD is different among users and non-users of a digital PRO solution. However, this is not the only indicator for acceptance of the digital PRO solution. An allocation of different profiles based on the ReadHy dimensions can inform clinicians about specific needs amongst people related to their ability to manage their health, their degree of support and their e-health literacy. There may be many factors which influence acceptance of digital PROs, including emotional distress, motivation, and whether the digital service suits individual needs. The association between eHealth Literacy included in the ReadHy tool and the acceptance level measured through the SUTAQ indicates a relationship between the tools and between readiness and enablement for eHealth technologies in general and the acceptance of a specific eHealth technology. This knowledge helps us to better understand mechanisms behind use and non-use of digital PROs, and may lead to a better match between technology and people, greater uptake and less attrition.

Footnotes

Acknowledgements

Thanks to the participating patients of Regional Hospital Silkeborg. Special thanks to Data Manager Andrew Bolas for sending out questionnaires.

Author contributions

The analysis is conducted by AN, LK and RO. The study is set up by all authors together. The manuscript is drafted by the AN, commented by all authors, finished by the AN and approved by all authors.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors are participating in development of digital PRO in the Danish healthcare system.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Danish Health Data Authorities, Central Region Denmark and University of Copenhagen.

Ethics and data protection

The study was reported to the Office for Responsible Conduct of Research in Central Region Denmark in accordance with Danish regulations (no. 1–16-02–563-18) and approved by The Human Ethics Advisory HEAG-H of School of Health and Social Development at Deakin University in accordance with Australian requirements (no. HEAG-H 194_2018).

The survey was conducted via Silkeborg Hospital’s password protected, secure IT-system, and data stayed within that system through analysis. Participants were informed by the survey through the national e-mail system or by mail and gave consent by participation.

ORCID iD

Amalie Søgaard Nielsen

References

Kingsley

Patel

. Patient-reported outcome measures and patient-reported experience measures. BJA Educ 2017; 17(4): 137–144.

Dawson

Doll

Fitzpatrick

, et al. The routine use of patient reported outcome measures in healthcare settings. BMJ2010; 340: c186.

Schougaard

LMV

Larsen

Jessen

, et al. AmbuFlex: tele-patient-reported outcomes (telePRO) as the basis for follow-up in chronic and malignant diseases. Qual Life Res 2016; 2016(25): 525–534.

Kotronoulas

Kearney

Maguire

, et al. What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol 2014; 32(14): 1480–1501.

Chen

Hollis

. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res 2013; 13(1): 211.

Greenhalgh

. The applications of PROs in clinical practice: what are they, do they work, and why? Qual Life Res 2009; 18: 115–123.

Greenhalgh

Dalkin

Gooding

, et al. Functionality and feedback: a realist synthesis of the collation, interpretation and utilisation of patient reported outcome measures data to improve patient care. Health Serv Deliv Res [Internet] 2017; 5(2): 1–280.

Nolte

Osborne

. A systematic review of outcomes of chronic disease self-management interventions. Qual Life Res Int J Qual Life Asp Treat Care Rehabil 2013; 22(7): 1805–1816.

Sanders

Rogers

Bowen

, et al. Exploring barriers to participation and adoption of telehealth and telecare within the whole system demonstrator trial: a qualitative study. BMC Health Serv Res 2012; 12(1): 220.

10.

Walsh

Travis

. What’s app? Electronic health technology in inflammatory bowel disease. Intest Res 2018; 2018(163): 366–373.

11.

Conley

Redeker

. A systematic review of self-management interventions for inflammatory bowel disease: self-management in IBD. J Nurs Scholarsh 2016; 48(2): 118–127.

12.

Eluri

Cross

Martin

, et al. Inflammatory bowel diseases can adversely impact domains of sexual function such as satisfaction with sex life. Dig Dis Sci 2018; 63(6): 1572–1582.

13.

Wong

Sceats

Dehghan

, et al. Depression and health care use in patients with inflammatory bowel disease. J Crohns Colitis 2019; 13(1): 19–26.

14.

Barlow

Cooke

Mulligan

, et al. A critical review of self-management and educational interventions in inflammatory bowel disease. Gastroenterol Nurs 2010; 33(1): 11–18.

15.

Lores

Goess

Mikocka-Walus

, et al. Integrated psychological care is needed, welcomed and effective in ambulatory inflammatory bowel disease management: evaluation of a new initiative. J Crohns Colitis 2019; 13(7): 819–827.

16.

Khan

Dasrath

Farghaly

, et al. Addressing unmet needs of inflammatory bowel disease patients with patient-reported outcomes app: a mixed methods study. Gastroenterology 2015; 148(4): S184.

17.

Powell

Fernandez

Williams

, et al. Enhancing the impact of implementation strategies in healthcare: a research Agenda. Front Public Health 2019; 7: 3.

18.

Ross

Stevenson

Lau

, et al. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci 2016; 11(1): 146.

19.

Murray

Burns

May

, et al. Why is it difficult to implement e-health initiatives? A qualitative study. Implement Sci 2011; 6(1): 6.

20.

Nielsen

Kidholm

Kayser

. Patients’ reasons for non-use of digital patient-reported outcome concepts: a scoping review. Health Inform J 2020; 26(4): 2811–2833.

21.

Kayser

Kushniruk

Osborne

, et al. Enhancing the effectiveness of consumer-focused health information technology systems through eHealth literacy: a framework for understanding users’ needs. JMIR Hum Factors 2015; 2(1): e9.

22.

Batterham

Hawkins

Collins

, et al. Health literacy: applying current concepts to improve health services and reduce health inequalities. Public Health 2016; 132: 3–12.

23.

Kayser

Rossen

Karnoe

, et al. Development of the multidimensional readiness and enablement index for health technology (READHY) tool to measure individuals’ health technology readiness: initial testing in a cancer rehabilitation setting. J Med Internet Res 2019; 21(2): e10377.

24.

Norgaard

Furstrand

Klokker

, et al. The e-health literacy framework: a conceptual framework for characterizing e-health users and their interaction with e-health systems. Knowl Manag E-Learn 2015; 7(4): 35.

25.

Benbasat

Barki

. Quo vadis, TAM? J Assoc Inf Syst 2007; 8(4): 211–218.

26.

Venkatesh

Davis

. A theoretical extension of the technology acceptance model: four longitudinal field studies. Manag Sci 2000; 46(2): 186–204.

27.

Hirani

Rixon

Beynon

, et al. Quantifying beliefs regarding telehealth: development of the whole systems demonstrator service user technology acceptability questionnaire. J Telemed Telecare 2017; 23(4): 460–469.

28.

Torbjornsen

Smastuen

Jenum

, et al. Acceptability of an mHealth app intervention for persons with type 2 diabetes and its associations with initial self-management: randomized controlled trial. JMIR mHealth uHealth 2018; 6(5): e125.

29.

Lee

Sandvei

Hosbond

, et al. Klinisk Integreret Hjemmemonitorering. Fonden for Velfærdsteknologi; 2015.

30.

Sandvei

Kynde

Lee

. Frontløberprojektet - Evaluering af tilfredshed hos borgere og organisatoriske erfaringer. University of Southern Denmark; 2013.

31.

Dario

Luisotto

Dal Pozzo

, et al. Assessment of patients’ perception of telemedicine services using the service user technology acceptability questionnaire. Int J Integr Care 2016; 16(2): 13.

32.

Creswell

Plano Clark

. Designing and conducting mixed methods research. 3rd ed. Los Angeles, CA: Sage; 2018.

33.

AmbuFlex . AmbuFlex. www.ambuflex.dk; 2018.

34.

Nielsen

Appel

Larsen

, et al. Patient perspectives on digital patient reported outcomes in routine care of inflammatory bowel disease. J Patient-Rep Outcomes 2021; 5(1): 92.

35.

Appel

Pedersen

Nielsen

, et al. Telemedicine based on patient-reported outcomes in management of patients with inflammatory bowel disease in a real-life setting––a before and after cohort study. Scand J Gastroenterol IGAS 2022; 2022: 1–7.

36.

Kayser

Karnoe

Furstrand

, et al. A multidimensional tool based on the ehealth literacy framework: development and initial validity testing of the ehealth literacy questionnaire (eHLQ). J Med Internet Res 2018; 20(2): e36.

37.

Osborne

Batterham

Elsworth

, et al. The grounded psychometric development and initial validation of the Health Literacy Questionnaire (HLQ). BMC Public Health 2013; 13(1): 658.

38.

Osborne

Elsworth

Whitfield

. The Health Education Impact Questionnaire (heiQ): an outcomes and evaluation measure for patient education and self-management interventions for people with chronic conditions. Patient Educ Couns 2007; 66(2): 192–201.

39.

Hawkins

Cheng

Elsworth

, et al. Translation method is validity evidence for construct equivalence: analysis of secondary data routinely collected during translations of the Health Literacy Questionnaire (HLQ). BMC Med Res Methodol 2020; 20(1): 130.

40.

Wahl

Osborne

Langeland

, et al. Making robust decisions about the impact of health education programs: Psychometric evaluation of the Health Education Impact Questionnaire (heiQ) in diverse patient groups in Norway. Patient Educ Couns 2016; 99(10): 1733–1738.

41.

Epstein

Osborne

Elsworth

, et al. Cross-cultural adaptation of the health education impact questionnaire: experimental study showed expert committee, not back-translation, added value. J Clin Epidemiol 2015; 68(4): 360–369.

42.

McLeod

. What does effect size tell you? Available at: https://www.simplypsychology.org/effect-size.html (2019, accessed 10 May 2021).

43.

Oliveira

Gomes

, et al. Identifying user profiles from statistical grouping methods. J Inf Syst Eng Manag 2018; 3(1): 6.

44.

Batterham

Buchbinder

Beauchamp

, et al. The OPtimising HEalth LIterAcy (Ophelia) process: study protocol for using health literacy profiling and community engagement to create and implement health reform. BMC Public Health 2014; 14(1): 694.

45.

Bech

. Clinical psychometrics. John Wiley & Sons; 2012.

46.

Brochmann

Zwisler

A.-D.

Kjerholt

, et al. A new internet-based tool for reporting and analysing patient-reported outcomes and the feasibility of repeated data collection from patients with myeloproliferative neoplasms. Qual Life Res 2016; 25(4): 835–846.

47.

Jiang

DeVito Dabbs

Sereika

, et al. Acceptance and use of mobile technology for health self-monitoring in lung transplant recipients during the first year post-transplantation. Appl Clin Inform 2016; 7(2): 430–445.

48.

Melissant

Verdonck-de Leeuw

Lissenberg-Witte

, et al. ‘Oncokompas,’ a web-based self-management application to support patient activation and optimal supportive care: a feasibility study among breast cancer survivors. Acta Oncol Stockh Swed 2018; 57(7): 924–934.

49.

Holt

Overgaard

Engel

, et al. Health literacy, digital literacy and eHealth literacy in Danish nursing students at entry and graduate level: a cross sectional study. BMC Nurs 2020; 19(1): 22.

50.

Venkatesh

. Unified theory of acceptance and use of technology: a synthesis and the road ahead. J Assoc Inform Syst 2013; 17: 328–376.

51.

Venkatesh

Bala

. Technology acceptance model 3 and a research agenda on interventions. Decis Sci 2008; 39(2): 273–315.

52.

CKL

Karsh

B-T

. A systematic review of patient acceptance of consumer health information technology. J Am Med Inform Assoc JAMIA 2009; 16(4): 550–560.

53.

Porter

Gonçalves-Bradley

Ricci-Cabello

, et al. Framework and guidance for implementing patient-reported outcomes in clinical practice: evidence, challenges and opportunities. J Comp Eff Res 2016; 5(5): 507–519.

54.

Schougaard

LMV

de Thurah

Christensen

, et al. Sociodemographic, personal, and disease-related determinants of referral to patient-reported outcome-based follow-up of remote outpatients: a prospective cohort study. Qual Life Res 2020; 29(5): 1335–1347.

55.

Bartlett

Tamblyn

Kawasumi

, et al. Non-participation bias in health services research using data from an integrated electronic prescribing project: the role of informed consent. Acta Bioethica 2005; 11(2): 145–159.

56.

Nahon

Lahmek

Durance

, et al. Risk factors of anxiety and depression in inflammatory bowel disease. Inflamm Bowel Dis 2012; 18(11): 2086–2091.

57.

Torenholt

Saltbæk

Langstrup

. Patient data work: filtering and sensing patient-reported outcomes. Sociol Health Illn 2020; 42(6): 1379–1393.

58.

Osborne

Batterham

Elsworth

, et al. The grounded psychometric development and initial validation of the Health Literacy Questionnaire (HLQ). BMC Public Health 2013; 13(1): 658.