Abstract
Background:
The electronic personal health record (PHR) is a promising technology for improving the quality of chronic disease management. Until now, evaluations of such systems have provided only little insight into why a particular outcome occurred. The aim of this study is to gain insight into the navigation process (what functionalities are used, and in what sequence) of e-Vita, a PHR for patients with type 2 diabetes mellitus (T2DM), to increase the efficiency of the system and improve the long-term adherence.
Method:
Log data of the first visits in the first 6 weeks after the release of a renewed version of e-Vita were analyzed to identify the usage patterns that emerge when users explore a new application.
Results:
After receiving the invitation, 28% of all registered users visited e-Vita. In total, 70 unique usage patterns could be identified. When users visited the education service first, 93% of all users ended their session. Most users visited either 1 or 5 or more services during their first session, but the distribution of the routes was diffuse.
Conclusions:
In conclusion, log file analyses can provide valuable prompts for improving the system design of a PHR. In this way, the match between the system and its users and the long-term adherence has the potential to increase.
The aging population and increased prevalence of chronic care requires an integral approach to disease management that is well coordinated and consistent with (inter)national care standards to support a shift from institutionalized care to home care.1-3 Disease management may be viewed as a set of interrelated services that spans the continuum from prevention and self-management to intramural care for patients with chronic diseases.4-6 Information and communication technology (eHealth) will play an important role in disease management, for example, in providing online support for self-management, in improving information exchange among professionals and with patients, and in monitoring the performance of the disease management program.7,8
The electronic personal health record (PHR) is a promising technology for improving the quality of chronic disease management.9,10 The Markle Foundation defined a PHR as “an electronic application through which individuals can access, manage, and share their health information and that of others for whom they are authorized, in a private, secure and confidential environment.” 11 Many researchers adopted this definition over the years.12-14 However, PHRs are becoming more complex and potential functions of current PHRs may not only include sharing clinical and personal data (e.g. history, test results, treatment, appointments), but may also include self-management support, patient–provider communication, information about illness, peer support, or monitoring health behavior data. 13
There are several potential benefits of using a PHR. Access to health data, health information and communication applications have the potential to empower patients in managing their diseases. In addition, deploying a PHR may reduce geographical and communication barriers. An ongoing connection between patient and caregiver may even lead to a transition from episodic to continuous care, which in turn has the potential to shorten the time to address disease-related complaints that may arise.12,13
Despite the potential benefits of a PHR, the use of such systems in diabetes care has led to only small improvements in diabetes quality measures that were of marginal clinical relevance, 9 and up to now evaluations have provided only little insight into why a particular outcome did occur.15,16 Therefore, it is necessary to look for new methodological approaches that go beyond a before and after measurement of health outcomes, for example, by exploring the process by which users find the needed information, share information, and gain benefits out of it. 17 This information is valuable in understanding how individuals want to use the system and what they are willing to do with it. 12 In other words, the logic of the content structure should match with the mental models that the users hold, to increase the efficiency of the system and improve the long-term adherence to the PHR.
Log data have the potential to identify the navigation process (what functionalities are used, and in what order) on a PHR.17,18 With these analyses, it is important to investigate not just the amount of use, because more exposure to a PHR will not necessarily lead to improved health outcomes and may even be an indicator for unfocused and strategic use and inefficient systems. 17
The aim of this study is to collect input for increasing the match between users and the system e-Vita, a PHR for patients with type 2 diabetes mellitus (T2DM). To understand the usage patterns that emerge when users navigate over the PHR, we conducted a log file analysis.
Prior studies showed that the attrition starts when users “get lost” in the intervention.18,19 Because a first impression is important, we used the log files of the first visit to the PHR to identify how users explore a new intervention. This information is important in modifying the content and the design to increase the efficiency of e-Vita and, in turn, increase the adherence of users and the chances of experiencing benefits and patient empowerment.
Methods
Parent Study and Participants
The analyses were performed on data collected in the parent study for effectively implementing a PHR (e-Vita) for patients with chronic diseases. In turn, this study is part of 3 larger studies on the effects of using a PHR in primary care for patients with T2DM, heart failure (HF), or chronic obstructive pulmonary disease (COPD). This article focuses on data collected in the T2DM study (ClinicalTrials.gov number NCT01570140).
All participants in this study are diagnosed with T2DM and aged over 18 years. Potential participants were excluded in case of mental retardation or disorders, insufficient knowledge of the Dutch language, cognitive impairment, or a short life expectancy (≤1 year) due to terminal illnesses.
Intervention
The PHR e-Vita is an initiative of the Dutch foundation Care Within Reach, a partnership between Philips and Achmea, a Dutch health insurance company.
According to Van Gemert-Pijnen et al, 20 a web-based intervention can be seen as the whole of the content, system, and services it provides. In this view, interaction is not just content, system, or service, but rather it is an integral part of an intervention. Therefore, we describe the platform e-Vita according to these categories.
Content
The content of e-Vita was created by experts in response of 12 interviews with patients with T2DM about their thoughts and feelings about living with T2DM and its treatment. Also, observations, interviews and interactive sessions were conducted to gain insight into experiences of health care professionals regarding the treatment of patients with T2DM. With this information, a PHR for patients with T2DM in primary care was developed. The main content of the PHR consists of insight into personal health data, self-monitoring health values, education, and a coach for reaching personal health-related goals.
System
When logging on for the first time to e-Vita, every user (a participant in the study who visited e-Vita at least once) sees a pop-up with a brief explanation about e-Vita and the services that can be found on the website. After the pop-up, the user was directed to the home page (Figure 1). From there, users were able to access all functionalities of the PHR.
Screenshot of the home page of e-Vita.
Service
The system e-Vita consists of the following set of interrelated services, which can be accessed via the home page (the numbers in parentheses correspond to the numbers in Figure 1):
(2) Insight into health values, provided by the general practitioner (GP). The data were updated after the annual check-up. All values are explained via an information button.
(3) An online coach for guidance when working on personal, health-related goals.
(4) Self-monitoring personal health values, where users can register the values they measured for blood pressure, waist circumference, weight, and BMI.
(5) An education module with text and movies about T2DM. Part of the offered education will be tailored to the user. The content is provided by an independent foundation and checked by physicians.
(6) Extra information about T2DM, where the user will be directed to an external website.
(7) Account settings where the user can change personal information.
(8) Inbox with personal messages.
Interaction
Users’ interaction with the system was only web-based. When users finished an education module, a message was sent to the users’ health care provider (in most cases, this was the users’ primary care nurse), giving health care providers the opportunity to use the information as a topic of conversation during face-to-face appointments in the general practice. Users received system messages when new education or personal messages with feedback from the coach were available (8). The interaction was unidirectional and users were not able to send their own messages to their health care provider and coach (and vice versa).
Data Collection
In July 2013, a renewed and extended version of e-Vita was released. All participants were informed about this new release and were invited via email to visit the PHR. Every visit to e-Vita was tracked objectively by collecting log data. In this article, we focus on the log data of the first 6 weeks after the release. No major changes were made to e-Vita in this period.
The log files contained anonymous records of actions performed by each user. For every action on e-Vita (button clicks, page views and database transactions), the following information was collected by the web server and added to a log file: (1) the users’ identification number, (2) time and day of the action, (3) the type of action taken, and (4) optional additional information about the action (e.g. what information was viewed by the users or what personal health goals are added). For every user, sessions (actions taken between logging in and logging out to the system) were identified. When a user logged in to e-Vita within half an hour after the last action, this was considered to be the same session.
When logging in to e-Vita, every user had to accept the general conditions, which contained an informed consent for logging for research purposes. By accepting the general conditions, the users gave permission for logging their actions.
Results
General
At the time of the release of the renewed version of e-Vita, 1197 potential participants were invited to register on the PHR. In total, 568 users (46%) agreed to register. After the invitation via email to visit the renewed e-Vita, 161 users visited the platform at least once in the first 6 weeks (28% of the registered users and 13% of the potential participants).
In total, 249 sessions were conducted, an average of 1.5 sessions per visitor in this period. In Figure 2, an overview of the distribution of these sessions over the weeks is given.
Number of sessions per week in the first 6 weeks after the release of the renewed e-Vita.
In the first week after the release, 110 different users visited e-Vita in 143 sessions. In the following weeks, the number of sessions decreased. Overall, most users visited e-Vita on Tuesday or Wednesday in the afternoon.
The First Session
An overview of all usage patterns that were identified for the first visit is given in the appendix. In total, 70 different usage patterns were identified. An overview of the services that were visited as a first step after the login is given in Table 1.
Services That Were Visited as a First Step After the Login (N = 161).
Regarding the first step after the login, 3 main usage patterns were identified. First, of the 161 users, 55 (34%) visited the service for insight into health values directly after the first login. After this step, the user was most likely to follow the structure of the main menu (marked with the numbers 2 to 6 in Figure 1). This route occurred 9 times.
Second, when a user visited the education service as the first step after the login, 93% (42 out of 46 users) ended the session there. In total, 36 of 42 users (86%) viewed fewer than 5 education topics, while 14% viewed 5 or more topics (median is 1 topic).
Third, 17 users ended their session immediately after the first login. Seven users returned for a second visit of the platform in the first 6 weeks after the release. During the second session, 6 of these users visited the service for insight into health values first. The distribution of the other usage patterns was diffuse.
In Table 2, an overview is given of the number of services that are visited during the first session.In their first session, 60 users (37%) visited 1 service after the login. A quarter of all users visited 5 or more services after the first login. The percentages of users who visited 2 to 4 services are lower.
Number of Visited Services During the First Login, Before Ending the Session (N = 161).
Discussion
Principal Results
The aim of this study was to collect input for increasing the match between users and e-Vita, a PHR for patients with T2DM in primary care, to increase the adherence of users, the chances of experiencing benefits and patient empowerment. Therefore, we conducted a log file analysis to gain insight into the usage patterns that emerge when users explore the PHR.
After receiving an invitation to visit the renewed version of e-Vita, only 28% of the registered users visited the PHR at least once in the first 6 weeks. The number of logins decreased over the weeks, which is a common finding in eHealth research, also known as the law of attrition. 21
In terms of the usage patterns that emerged, there are some important findings. First, users were most likely to follow the structure of the main menu. The results of our analyses have thus shown that the layout of the menu structure is important, and that the routes that users take on a PHR probably can be influenced by the sequence in which the services are presented. This information is valuable in marking the intended routes by the developer on a PHR.
Second, when users visit the education service as the first step after the first login, 93% ended their session. When users visit the education service after they visited another service, this pattern was less likely to emerge. There are several possible explanations for this finding. The first explanation might be that the amount of information that is presented in the education service is too overwhelming, causing users to end their session. A second explanation is that the users who visited the education service as the first step after the first login spend more time to explore the available topics and explore the rest of the PHR in the next sessions. Results have shown that the median number of visited education topics is 1, supporting the idea that viewing the education service as a first step after the first login might be too overwhelming. This information can be used to improve the design of the service, for instance, by making the design more clear and compact.
Third, except the route that follows the menu structure on the home page of the PHR, the distribution of the other routes was very diffuse. This is an indication that it might be unclear for users how they should explore the PHR, and might possibly hinder a second visit. Because diffuse patterns may be an indication for unfocused and nonstrategic use, 17 it might therefore be useful for developers to give an explanation to users about the possibilities of a PHR and to guide users over the platform.
Last, when users logged in for the first time, they were likely either to log out after visiting 1 service or to visit 5 or more services on the website, indicating that the first impression of the PHR of users could be more attractive. However, when users overcame this first impression, they made an effort to explore the rest of the PHR. This is a prompt for more persuasive support at the first login, for guiding users over the PHR.
In summary, our results of the log file analyses have shown that the identification of usage patterns can provide us valuable information about how users navigate over a PHR when visiting it for the first time, which is in line with previous research.17,18 Also, the importance of the layout has been demonstrated. This information can be used to make the purpose of e-Vita more evident with the first login. For example, a tutorial could be made to show new users the evident and effective routes.
Limitations
The first limitation of this study is that we did not involve the users in analyzing the log data. In other words, we have not checked our interpretations regarding the usage patterns that emerged and we are not able to derive mental models out of the results. It is therefore important to involve users in the future, to learn more about the mental models the users hold when using an PHR, like Tang et al previously suggested. 12 On the other hand, we were interested in usage patterns in this study, and the log data have provided us with objectively measured and real-time information that would have been hard to be recalled by users after their first session on the PHR.
Second, these data have revealed only the usage in the first 6 weeks after a new release in a relative small sample, and we were not able to track the usage over a longer period of time to see what long-term usage patterns emerge.
Future Research
The results of the analyses have raised several questions for our future research. First, it would be interesting to track the usage of e-Vita over a longer period of time, for example, over 3 to 6 months. In this way, we are able to track changes in usage patterns over time and more definitive usage patterns can be revealed.
Second, it would be interesting to link these patterns to the information about the users, for example demographics (age, educational level, disease history), health values, or opinions about the PHR or the quality of care, to predict what factors influence a return to the PHR, and, in addition, to identify the most effective patterns in terms of the adherence to the PHR, satisfaction about the care that has been delivered, and the development of self-management skills as a result of using the PHR. Third, it would be useful to conduct an interview study concerning the mental models that the users hold when navigating over a PHR.
Conclusions
In conclusion, we have shown that log file analyses can provide valuable prompts for improving the system design of eHealth applications, for example a PHR, to increase both adherence to and the efficiency of eHealth applications.
Footnotes
Appendix
Acknowledgements
We would like to thank Henk Bilo, Yvonne Roelofsen, and Steven Hendriks of Diabetes Centre, Isala, Zwolle, Netherlands, for their support in the collection of the research data.
Abbreviations
COPD, chronic obstructive pulmonary disease; GP, general practitioner; HF, heart failure; PHR, personal health record; T2DM, type 2 diabetes mellitus.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is funded by the Care Within Reach foundation.