Abstract
Background:
My Diabetes My Way (MDMW) is an electronic personal health record (ePHR) that provides access to educational resources and clinical data to people with diabetes in Scotland. This questionnaire study aims to assess user experience, barriers to access, and inform future development.
Methods:
All active MDMW users (n = 3797) were invited to complete an online questionnaire in May 2015, surveying usage patterns and system utility. A “utility score” was calculated, based on responses to Likert-scale questions and used as the dependent variable within regression analysis, with demographic features as independent predictors. Free-text responses were analyzed thematically and presented using descriptive statistics.
Results:
A total of 1095/3797 (27.5%) active users completed the survey. Of them, 690/1095 (63%) were male. There was representation of all age and socioeconomic groups. Respondents were positive regarding the system utility, which met expectations. The majority of respondents believed that online access to diabetes information has the potential to improve diabetes self-care within the population. The most valued features were personal clinical data associated visualizations. Th main problems cited were data accuracy and system access (ie, log-in procedure). Perceived usefulness of the system was inversely associated with duration of diabetes, which was the only significant predictor of utility score.
Conclusions:
This study has demonstrated that MDMW users find the system useful in supporting diabetes self-management. The system was found to have greatest utility among those most recently diagnosed with diabetes. This study has informed further development of the service, including enhancing data visualization and the need to improve access to the system.
My Diabetes My Way (MDMW) 1 is the NHS Scotland interactive website for people with diabetes and their carers. It contains a variety of multimedia resources aimed at improving self-management, including traditional information leaflets, interactive educational tools, and videos describing complications.
MDMW was launched in 2008, initially as an open access website containing the above resources. From 2010, MDMW has offered its users access to their clinical data via its novel electronic personal health record (ePHR). The service was initially developed by the University of Dundee but is now available to all people with diabetes in Scotland (approximately 300 000). 2 By the end of 2015, there were nearly 17 000 registered users (6% of people with diabetes in Scotland), distributed evenly throughout all NHS boards in Scotland. 3 Use of MDMW is associated with improved glycemic control and other clinical outcomes. 4
In order to enroll, patients must verify their email address and consent to their data being made available to them online. This ePHR links to SCI-Diabetes, NHS Scotland’s flagship diabetes record. 5 This system includes data from primary and secondary care, specialist screening systems (retinopathy screening, podiatry, etc) and laboratories. These data include diagnostic information, demographics, process outcomes, screening results, medication, and clinical correspondence. The system provides a more complete overview of diabetes than would be available from any single data source, such as an isolated primary care or hospital clinic database.
The MDMW ePHR takes a subset of data from SCI-Diabetes, focusing on key diabetes indicators, such as HbA1c, blood pressure, body mass index, and so on. Alongside these data is descriptive text explaining each assessment, detailing why they are recorded and what normal range values are. Further educational materials are presented alongside clinical results and are tailored to those using the service. For example, foot care advice is based on the patient’s recorded foot risk assessment category. History graphs and tables allow individuals to track changes over time for the full duration of their clinical record from multiple electronic data sources. MDMW aims to provide highly tailored information and provides advice based on the Diabetes UK 15 Healthcare Essentials campaign. 6 Patients can also manually enter home-recorded information (weight, blood pressure, etc), or automatically upload blood glucose results. Users receive a monthly MDMW newsletter via email. In addition, users who have not logged in to MDMW for 6 months are contacted directly to assist them in accessing their account.
This study aims to assess users’ experience of MDMW, with regards to perceived benefits and shortcomings. In particular, we aimed to characterize users’ attitudes to accessing their ePHR via online means, assess whether the service is meeting user expectations, and survey users to inform desired future developments of the system.
Methods
A 35-item questionnaire was devised and written in an electronic format for completion online. The questionnaire aimed to gather feedback on 3 main areas, presented on separate pages: the registration process and usage patterns (6 items), the utility of the system (17 Likert-scale items plus one free-text item), and support services for the system (5 items). Finally, the questionnaire offered free-text responses to allow users to identify the best parts of the system, the worst parts, and desirable features that are not currently available. The questionnaire was developed by the principal investigator, in conjunction with the MDMW steering group (consisting of lay and health professional members) and was previously piloted among MDMW users.
An invite was emailed to all active, registered MDMW users (convenience sample, n = 3797) in May 2015. Active users were defined as those that logged in to the system at any point following the registration process. The invite email contained a link to the questionnaire that was live to respondents for a period of 1 month from the date of invite. Survey completion was completely voluntary, with no impact on access to the MDMW site and no incentives to complete. No reminder emails were sent. The survey was hosted by a proprietary company (SurveyMonkey), 7 using a generic template. All questionnaire items were voluntary, with no completeness check or other forms of data validation. Respondents could amend responses to all items prior to final submission. The URL to access the questionnaire contained a deidentified “key,” restricting respondents to a single response and ensuring that the survey was closed to those without the invite email. This “key” also allowed responses to be retrospectively matched to user account details. Demographic variables were then extracted from the MDMW system, thereby removing the need for respondents to provide personal identifiers or information.
Likert-scale responses for system utility were summed and divided by the maximum possible value to provide a “utility score” out of 1 (0 = low utility, 1 = high utility). The utility score was used as the dependent variable within a regression analysis (see below). Likert responses were also collapsed into “agree,” “neutral,” and “disagree” categories, for the purposes of graphical representation. Free-text responses were thematically analyzed by the lead author and coded by theme, thereby enabling descriptive statistics to be calculated. The denominator used to calculate percentages varied according to the number of responses for each item. For free-text responses, responses often contained more than one theme, in which case this was coded to allow aggregates to be calculated appropriately.
Demographic variables were available for all active users and included age, ethnicity, sex, diabetes type and duration, and socioeconomic status (SES). SES was derived from the Scottish Index of Multiple Deprivation (SIMD), obtained from home postcode and expressed in quintiles ranging from most deprived to least deprived. 8 Respondents’ SES was compared with nonrespondents, as was the system usage in the year prior to the questionnaire invite. Multivariable regression analysis was used to identify groups of respondents which felt that the system has greatest utility. Demographic groups were compared using Mann-Whitney U test. Regression was also used to explore demographic predictors of system usage
This project was a service improvement project, therefore was not subject to medical research ethics review. All invites to complete the questionnaire were sent to MDMW users who previously consented to receive unsolicited email from the MDMW team (consent obtained at initial enrolment).
Results
A total of 1095/3797 (27.5%) active users completed the survey. Of them, 690/1095 (63%) were male. And 789/1095 (72%) had T2D, 290/1095 (27%) had T1D, with the remainder having other types of diabetes. Age was normally distributed around the mean of 58 years (SD 12, range 18-88). There was a representation from all socioeconomic groups, however there were a greater number of respondents from the least deprived categories. There was no significant difference in SES between respondents and nonrespondents (see Figure 1).
Scottish Index of Multiple Deprivation (SIMD) quintiles for all MDMW users invited to complete the survey. Percentages calculated for each legend category.
The vast majority of respondents (873/1095, 80%) identified as “white” (British, Irish, or Scottish); 80/1055 (7%) were in other minority groups, with the remainder being unknown. Duration of diabetes ranged from 1 month to 61 years and was skewed toward those more recently diagnosed (median duration 91 months, interquartile range 155). Those with T2D had had diabetes for significantly less time than those with T1D (median months [IQR]: 69 [118] versus 224 [307], P < .001). Those with T2D were also significantly older (years [SD]: 61 [10] versus 50 [13], P < .001).
Approximately a third of users became aware of MDMW via publicity material at the hospital clinic (358/1095, 33%), a third via material at the GP surgery (324/1095, 30%). The majority of the remainder became aware of MDMW via online means (206/1095, 18.8%). Most users access the system from home (876/966, 80%), during the evening or night (611/946, 55%), with no set day on which they do so (938/968, 86%). In the year prior to completing the questionnaire, respondents accessed the system significantly more than nonrespondents (median number of log-ins [IQR]: 5 [8] versus 1 [4], P < .001). Those with type 2 diabetes accessed the system significantly more than those with type 1 diabetes (median log-ins [IQR]: 5 [9] versus 4 [6], P = .001). There were no other significant demographic predictors of system usage, including SES.
Respondents’ agreement to a variety of statements is provided in Figure 2. The statement that elicited the most agreement with users related to the usefulness of the graphs (818/989, 83% agree that they were helpful; 348/989, 35% of whom were in strong agreement). A similar number of users were confident in the security of the data within the system (799/990, 81%; 316/990, 34% of whom were in strong agreement with the statement regarding confidence in data security). A sizable majority were also of the belief that MDMW has the potential to significantly improve diabetes self-care within the population (803/985, 82%), with 331/985 (34%) in strong agreement.
Collapsed responses to statements regarding system utility (n = 985-997), listed by decreasing order of agreement.
Of the 1095 respondents, 916 (84%) answered all 17 items relating to system utility, thereby allowing a utility score to be calculated. The utility score was skewed toward higher scores, with a median of 0.78 (interquartile range 0.14). Utility score was inversely associated with duration of diabetes, which was a highly significant predictor (P < .001). Both duration of diabetes and type of diabetes were significant univariable predictors of utility score, however type of diabetes was not significant when entered simultaneously with duration (Table 1). Socioeconomic status was not predictive of utility score, although there was a (nonsignificant) trend for lower utility scores within the less deprived groups. System usage in the year prior to completing the questionnaire did not predict utility score.
Univariable and Multivariable Predictors of MDMW Utility Score.
Univariable predictors significant to P < .1 entered into multivariable model. “White–British” includes all who identify as “white–British,” “white–Irish,” and “white–Scottish.”
In response to the question “What was the best part of the system and why?,” 614/1095 (56%) provided a response. There were 7 key themes identified, in addition to other comments that were uncategorizable. The most commonly cited feature was being able access to the complete medical record (302/614, 49%), that is, ability to view their complete diabetes record, including biochemical data, results of screening investigations, and clinical correspondence (see Table 2).
Free-Text Responses to “What Was the Best Part of the System and Why?”
Free text coded by theme and key themes presented in order of decreasing prevalence (n = 614).
Respondents also valued the various data visualization tools (eg, line graphs, target charts etc), thereby allowing increased insights into how to interpret personal data, track progress through time, and compare their values against normative reference ranges. Users valued how the system afforded insights into their underlying condition and their data trends through time. They valued being able to access the system at any time and also being able to view an up to date record of their medication.
In response to the question “What was the worst part of the system and why?,” approximately one-third (347/1095, 31%) provided a free-text response. The responses were grouped into 7 key themes (see Table 3).
Free-Text Responses to “What Was the Worst Part of the System and Why?”
Free text coded by theme and key themes presented in order of decreasing prevalence (n = 347).
By far the most common complaint was with difficulty accessing the system (ie, the initial log-in process), cited by just under half of respondents (154/347, 44%). Problems with data were the next most common complaint, including lack of accuracy, missing data, and problems with entering data into the system. The user interface was criticized for being “clunky” and providing information in a format that was, at times, confusing. Last, users experienced frustration in getting help when contacting the system administrators in order to report problems. Only three individuals (1%) cited data security as being a concern.
In response to the question “What new features would you like to see added to the system?,” 253/1095 (23%) individuals responded with free-text suggestions. These responses could be grouped into 8 main categories, which to some extent provided possible solutions to the previously identified problems (see Table 4).
Free-Text Response to “What New Features Would You Like to See Added to the System?”
Free text coded by theme and key themes presented in order of decreasing prevalence (n = 253).
The most desired feature suggested by respondents was additional educational resources (112/253, 44%). Users expressed a desire to upload and download data more readily and that these data were visualized in a more intuitive way, with the use of data validation to avoid data transcription error. Improving ease of access to the system as well as customer support to do so were identified as priorities. An improved user interface as well as a mobile app version were also suggested, which should contain a more user-friendly diary of events. One person suggested the introduction of “gamification” (ie, the use of incentives and competitive features, derived from the video game industry), 9 by way of motivating users.
Discussion
Principal Results
This study has demonstrated that this cohort of MDMW users are generally satisfied with the system and value features such as access to their medical data and data visualizations. Duration of diabetes was inversely associated with perceived usefulness. Interestingly, perceived usefulness was not related to age, gender, or SES, nor was it related to prior system usage. User feedback would suggest that improvements can be made with regards to registering with and accessing the system as well as providing additional resources in an accessible way.
Comparison With Prior Work
MDMW is a useful aid to diabetes self-management, providing access to people with diabetes in Scotland. It is unique in offering access to a complete geographical population, providing information from many diabetes-related sources. MDMW supports the diabetes improvement, self-management, health care quality, and eHealth strategies of the Scottish government.
This study has demonstrated that overall user satisfaction with the MDMW system is high, and that the majority of users feel that the system is a useful adjunct to diabetes self-care. Web-based interventions for those with diabetes have been shown to improve clinical outcomes. 10 Recent analysis of clinical outcomes among a large population of MDMW users has demonstrated improved glycemic control. 4 At a population level, small improvements in glycemic control are associated with considerable long-term savings due to a reduction in diabetes-related complications. 11
Electronic health applications are often developed in isolation and may not reflect the user requirements of the population that they are designed for. 12 The questionnaire has identified areas for improvement and has directly informed development of the website. In particular, the registration process has since been simplified, resulting in improved access to the system, the user interface is being improved and a MDMW app has been developed. Work is also being done to enhance electronic communications and improve data uploading (including data from smartphone apps and wearable sensors) as well as developing patient-specific decision support. This study has identified areas for further improvement, including the availability and accuracy of data, although it should be noted that these problems were only voiced by a minority of users. Users expressed a desire for additional data feeds from other specialties. In Scotland, the diabetes digital landscape is relatively mature when compared to other specialties. The MDMW system architecture allows it to link with other electronic clinical records via its standardized generic interface, thereby allowing such data feeds to be developed in future.
Internet usage patterns can reflect underlying demographic and socioeconomic differences, with the potential to increase health inequalities. 13 For example, those with lower health literacy are less likely to access an ePHR. 14 In this study, no particular sociodemographic subgroup found MDMW to be more or less useful. However, the limitations of the study make it difficult to generalize these findings to the wider population (see below). It is notable that the utility of the site was inversely related to duration of diabetes among the respondents—the reasons for this remain speculative at this stage and should be explored in future studies.
Limitations
The response rate to the questionnaire was low, but is in keeping with other electronic surveys. 15 There was a higher proportion of respondents from less deprived areas, raising the possibility of sampling bias—the prevalence of T2D tends to be higher in more deprived areas, whereas T1D does not follow a deprivation gradient. 16 However, there was no significant difference in respondents’ SES compared with MDMW users that did not respond, suggesting that this sample reflects MDMW users in general. While MDMW users tend to be from less deprived areas, prior system usage was not predicted by SES status.
The proportion of respondents with T1D was higher than the general population, 2 but is in keeping with other users of MDMW. There was limited diversity in the ethnicity of respondents, however this reflects the Scottish diabetes population. 2 Similarly, age distribution and gender balance was similar to the wider population. 2
Conclusion
MDMW is an innovative and unique ePHR that is associated with improved clinical outcomes. This study has demonstrated that the system is a useful addition to diabetes management in this sample of MDMW users. In addition, this study has identified areas for improvement which has informed subsequent development. These findings have relevance beyond the immediate MDMW platform and can guide developers of other systems to ensure patient satisfaction. The MDMW user base continues to grow significantly with >39 000 registered users and >19 000 active users (May 2018), ensuring that future attempts at gaining user feedback will benefit from representation of a broad cross-section of the diabetes community.
Footnotes
Acknowledgements
The authors would like to thank all of the My Diabetes My Way users who participated in this survey. MDMW was created within NHS Scotland and the University of Dundee as a patient portal that is freely available within NHS Scotland.
Abbreviations
CI, confidence interval; ePHR, electronic personal health record; GP, general practitioner; HbA1c, hemoglobin A1c; IQR, interquartile range; MDMW, My Diabetes My Way; NHS, National Health Service; SCI-DC, Scottish Care Information—Diabetes Collaboration; SES, socioeconomic status; SIMD, Scottish Index of Multiple Deprivation; T1D, type 1 diabetes; T2D, type 2 diabetes.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: DJW and SGC are directors of MyWay Digital Health Ltd.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: A spin out company (MyWay Digital Health Ltd) has been created to develop and commercialize MDMW for other markets.
