Abstract
The objective of this study was to understand perceptions of the usability and value of web-based patient portals among parents of children with a chronic condition (diabetes, juvenile idiopathic arthritis, and cystic fibrosis). The design was a cross-sectional telephone survey with semistructured interviews and was conducted at a tertiary children’s hospital. Parent ratings of their experiences with and the value of a web-based portal and reasons for not using the portal were ascertained. Most parents agreed that the portal information was accurate, timely, and useful. Confidentiality and seeing worrisome health-related information about their child was not a major concern, and parents felt that the portal helped to improve their understanding of and ability to manage their child’s condition. In conclusion, the results of this study support the notion that web-based patient portals have the potential to be a useful tool for parents of children with a chronic disease.
Keywords
Introduction
The medical profession continues to advance the implementation of electronic personal health records (PHRs). It has been 13 years since the Journal
The actual use of PHRs by patients has not measured up to the high level of activity and interest in their development and implementation. The experiences of several large integrated health systems have shown that only a minority of patients who have been offered access to such technology take advantage of it.3,4,6,12 Most notably, the National Health Service in Great Britain recently closed its PHR due to a lack of interest among patients. 13 Although studies in pediatrics are scarce, a study by the authors of parents of children with chronic conditions demonstrated similar results, with only 28 percent of families obtaining an account to access a web-based portal. 14
Potential next steps and future directions for PHRs are outlined in a report published by the Department of Health in Great Britain. Determining how the PHR will best fit into the overall framework of collecting, disseminating, sharing, and using information in order to achieve more integration in healthcare systems and thus improved quality of care will be key to the future success of PHRs. 15 An important consideration will be to understand what patients find helpful, their concerns, and their opinions on the quality of information contained in PHRs.
Studies of adult populations have shown relatively high levels of satisfaction among users of web-based patient portals.3,4,16–19 However, little is known about how parents of children with chronic disease respond to web-based patient portals to access their child’s health-related information. To our knowledge, there has only been one such study of this population, which explored differences in perceived quality of care (e.g. access and coordination) among portal users and nonusers. The results of this study did not support an association between portal use and perceived quality of care. 20
The objective of this study was to measure and understand among parents of children with chronic conditions, their perceptions of the usability and value and their concerns about using a web-based portal to access their child’s health records.
Methods
This study consisted of a telephone survey with parents of children with a chronic condition who used a web-based patient portal to access information about their child’s health. In addition, semistructured interviews were conducted with parents of children with a chronic condition who had enrolled in but did not use the web-based patient portal. This study was conducted at a tertiary children’s hospital and was approved by its Institutional Review Board.
The portals were developed in-house by the hospital’s Department of Information Services in response to family and provider desire for ready access to key portions of patients’ health records. The portals are secure web-based applications through which parents can access laboratory results, medication information, and their child’s visit history. Other capabilities include secure messaging to healthcare providers, the ability to upload documents to share with healthcare providers (e.g. school-related forms), and reminders for laboratories and clinic visits. While the portals shared a common infrastructure and user interface, the content was disease specific, so that parents could easily locate information most critical for their child. They were designed using family input and refined with formal usability methods. 21 Implementation and marketing efforts for the portals are detailed in a previous publication by the authors. 14
It is important to note some particulars about what the authors of this article refer to as a web-based patient portal. First, the information that can be accessed is the patients’ own health-related information and not general health-related information aimed at a particular audience. Second, an important distinction among electronic PHRs is that they can be either stand-alone or integrated with the patient’s electronic health record (EHR). While each has advantages, participants in a working symposium of the American Medical Informatics Association’s College of Medical Informatics concluded that integrated PHRs offer more benefits and are the preferred model. 22 Integrated PHRs will be referred to as web-based patient portals and are the focus of this work.
Study population
The focus of this study was parents of children with one or more of three chronic diseases, namely, cystic fibrosis (CF), diabetes mellitus (DM), and juvenile idiopathic arthritis (JIA), who are patients of three disease-specific outpatient clinics. The population for this study consisted of 530 parents of children with CF, DM, or JIA who had obtained portal accounts by 29 February 2008. Of these 530 parents, 215 parents were identified as “portal users” because they had used a web-based portal three or more times during the 2007 calendar year. The remaining 315 parents who used the portal less than three times were designated “portal nonusers.”
Data collection
All portal users were asked to participate in a telephone survey, which was administered by an independent vendor contracted by the research team. Parents were sent an introductory letter that explained the purpose of the survey and provided contact information for opting out of the study. The telephone interviews were conducted from April through August 2008, with up to 10 attempts made to contact every eligible parent in the population. When interviewers reached a household, they asked to speak to the primary user of the web-based portal.
The questionnaire was developed based on the work of Cimino.23–25 It consisted of 22 Likert scale questions that elicited parent opinions about the portal such as (1) information usefulness, accuracy, and timeliness, (2) ease of use, (3) concerns regarding confidentiality and seeing worrisome information, (4) perceptions of the portal’s effect on outcomes, and (5) satisfaction with the email function. These questions were in the form of a statement that used a 5-point Likert response scale ranging from “strongly disagree” to “strongly agree.” Additional questions addressed frequency of use and overall quality of the portal. Two open-ended questions were included to elicit any additional concerns families might have about the portal and suggestions for improvement. The survey was pretested to ensure that the questions were unambiguous and elicited the requisite information.
Demographic characteristics included insurance type, patient’s age, race, gender, and family residence (large metro, small metro, or rural). Family residence was determined using zip codes and the 2003 Urban Influence Codes by the US Department of Agriculture. 26 Health utilization data included the number of hospitalizations and outpatient visits during the calendar year of 2007. Demographic and healthcare utilization data were obtained from chronic disease registries and hospital administrative databases.
Semistructured interviews were conducted with five randomly selected portal nonusers (i.e. enrolled in the portal but used it less than three times in 2007) from each of the three chronic diseases for a total of 15 interviews. The objective of these interviews was to ascertain reasons for not using the web-based portal.
Statistical and qualitative analysis
To test for demographic differences between responders and nonresponders to the parent telephone survey, chi-square tests were used for categorical variables (race, insurance, gender, place of residence, and disease) and a
Some parents (6%) had more than one child with a chronic condition for which they accessed web-based patient portals. Since the available demographic data were patient-level, demographic data from the child who had been diagnosed first were used for analysis purposes. Healthcare utilization data were handled slightly differently because we believed that higher frequencies of contact with the healthcare system may influence attitudes about the portal. As a result, healthcare utilization was aggregated by summing the number of outpatient visits and hospitalizations across all the children with a chronic condition within the family.
A content analysis was conducted on the responses to the two open-ended questions in the telephone survey and the responses to the semistructured interview questions asked of the portal nonusers. This analysis involved developing themes based on parent comments and developing a resulting codebook, which was used to assign codes to each comment.
Results
A total of 126 (59%) of the 215 parents who were identified as portal users completed the telephone survey. The disposition of those parents who did not complete the survey was as follows: parent could not be reached during the study period (54), parent declined to participate (24), parent opted out of the study after receiving the introductory letter (7), and incorrect phone number (4). Table 1 contains a comparison of the demographic characteristics of responding and nonresponding parents to the telephone survey for which there were no significant differences (i.e. all p-values > 0.05). Item nonresponse was low with most questions (83%) having one or no missing responses. None of the questions had more than four missing responses.
Demographic characteristics of responders and nonresponders to the telephone survey of portal users.
CF: cystic fibrosis; DM: diabetes mellitus; JIA: juvenile idiopathic arthritis.
During the 2007 calendar year.
Trending toward significance with p-value = .07.
Portal use
Parents were asked to indicate the frequency with which they used the portal during the past 2 months. They responded as follows: once per day (2%), once each week (10%), once every 2–3 weeks (19%), once per month (53%), or less than once per month (17%). Parents were also asked how often they used the portal instead of calling their healthcare provider, and they responded as follows: never (36%), sometimes (58%), and usually/always (6%). More than half (56%) of the parents indicated that they always or usually felt comforted that they did not have to rely on others for medical information about their child’s condition.
Table 2 summarizes the reasons that parents did not use the portal. The most common reason was never receiving or losing their password. Other reasons were that parents had no time, felt that the low level of severity of their child’s disease did not warrant portal use, or they received information through other avenues.
Reasons for not using the portal.
n = 15 nonusers.
Parent experience with the portal
A large majority of parents thought the portal information was useful, accurate, and timely. In addition, most agreed that the portal was easy to use and easy to learn, and more than half the parents reported never having difficulties using the portal (Figure 1). More than one-third (39%) of the parents reported using the portal to send emails to healthcare providers. Of these parents, 81 percent agreed the email function was very useful, 74 percent agreed that they always received the information they needed, and 71 percent agreed that they always received a response in a timely manner. When asked to rate the portal overall, parents responded as follows: excellent (15%), very good (53%), good (24%), or fair or poor (8%). Table 3 summarizes parent responses about how the portal could better meet their needs.

Parent ratings on the quality of information contained in the portal and ease of use.
Suggestions for changing the portal to better meet healthcare needs.
Figure 2 summarizes parent responses regarding concerns about using the portal. Specifically, parents were asked about the confidentiality and the potential of the information to cause stress. A large majority indicated that they never felt concerned about the portal’s confidentiality. Only 2 percent of the parents agreed that they sometimes saw information they wish they had not seen. On the contrary, 12 percent agreed that they sometimes saw information in the portal that frightened them, and 11 percent reported that they sometimes see information that they would have preferred to get directly from their provider. The logistic regression models did not result in identifying any significant demographic predictors of parent concerns. Table 4 summarizes parent responses to an open-ended question about their concerns with the portal.

Parent concerns about the portal.
Parent concerns about the portal.
Figure 3 shows that almost 70 percent of parents agreed that the portal improved their ability to manage and to understand their child’s condition, while 62 percent agreed it improved communication with their healthcare providers. To a lesser extent, parents agreed that the portal improved their relationship with their child’s healthcare provider (43%) and how they make healthcare decisions for their child (42%). The logistic regression models showed that a higher number of outpatient visits was associated with parents agreeing that the portal improved how they make healthcare decisions for their child (odds ratio (OR): 1.29, 95% confidence interval (CI): 1.04, 1.60).

Parent reported effect on outcomes.
Discussion
In our study, parents generally found the portal easy to use and the information included in it useful, accurate, and timely. A suggestion, however, for improving the portal was to provide additional explanations of test results and medical terminology. Findings from studies of adults have been similar regarding the quality of the information contained in portals 3,17,18 and portal users having issues understanding medical content and jargon.16–18 While portals are generally found to be easy to use and the health-related information useful, these findings show that there is room for improving how information is presented to make it more understandable.
Our findings support the notion that parental concerns about the confidentiality of a web-based portal or being anxious about the information they saw are few. Review articles have concluded that, in general, patient access to medical records did not produce significant anxiety in patients 27 and concerns about confidentiality were minimal.27,28 On the contrary, a study of adult patients with congestive heart failure found concerns about privacy and confidentiality, 17 and a study in adult primary care found that patients wanted to hear bad news or abnormal results from their healthcare providers first. 18 A possible explanation for the low level of concern expressed by parents in our study is that parents who have been caring for a child with a chronic condition for some time are knowledgeable about their child’s illness. It could be that they are experienced in receiving and processing information, even when the news is not good. An important topic for future research is to determine whether parents of newly diagnosed children need additional coaching, so that they are comfortable accessing the information in their child’s EHR.
Compared to our results, studies in adult primary care showed higher use of web-based messaging with 46 percent
29
and 58 percent
30
of patients communicating with their physician at least once. Another study of parents of children with chronic disease, who were identified using the coding system
There has been little published about the effect of this technology on outcomes, with much of the work addressing health-related outcomes of adult patients with diabetes.28,34–36 A study in adult primary care showed that a large majority of patients felt that access to their EHR helped them to better understand the reasons they needed medication. 16 In addition, a study of parents of children with chronic conditions showed no association between measures of experience of care, such as access and care coordination, with the use of an integrated PHR. 20 In our study, a majority of parents agreed that they better understood their child’s condition and that it improved communication with healthcare providers and their ability to manage their child’s condition. To a much lesser extent, parents reported that the portal improved their relationship with their child’s physician and their ability to make healthcare decisions. These results support the notion that web-based patient portals have the potential to play an important role in the improvement of outcomes in children with chronic conditions such as CF, DM, and JIA. A recent report published by the Institute of Medicine, however, describes the rapid development of this technology in order to engage patients and families in health care but cautions that research to address unintended consequences, particularly with respect to patient safety, is needed. 37
An electronic PHR has increased potential to be beneficial if it is integrated with the patient’s EHR, has tools that empower the patient to act on the information contained within the portal, and is customizable in order to provide the most useful information to patients. An article by Tang et al. 22 describes such potential benefits as including access to credible information that can be used by patients to manage disease and improve health; providing for collaborative disease tracking capabilities between physicians and patients; making it easier for patients to ask questions, set up appointments, and manage prescriptions and referrals; and lowering communication barriers by providing an ongoing connection between patients and healthcare providers. Future studies should address which features of web-based patient portals are most helpful to parents in understanding and managing their child’s condition. They should take a value-added approach to evaluate web-based patient portals in order to determine the features and information that can potentially add to or fill gaps in patients’ current system of care. For example, a qualitative study of patients with diabetes showed that they used a portal to address gaps in the care they received, such as increasing their knowledge of the disease and reliably getting messages to their physicians. 38
There are several limitations to this study. First, there are biases inherent in telephone surveys including nonresponse bias and the fact that the outcomes are parent reported and based on their recollections. Second, this is a single site study of parents of patients with one of three chronic diseases, which limits generalizability. Third, this was primarily a study of users of web-based patient portals. Although our sample of nonusers was limited, it provided some preliminary data regarding reasons for not using the portal. Finally, there is evidence of disparities in patient portal enrollment and use by race and insurance status.12,14,39,40 We were not able to address this given that the majority of the population in our study of patient portal users was White and privately insured.
The responsibility and burden of caring for a child with a serious chronic condition can be stressful and disruptive for parents and families. 41 Providing tools to help them manage their child’s disease has the potential to lessen the burden of caring for a chronically ill child. Understanding perceptions regarding experiences with using web-based patient portals is an important step toward determining whether or not such portals can be useful tools for parents of children with chronic disease. Overall, the parents in this study found the information contained in the portal to be useful, accurate, and timely. In addition, confidentiality and seeing worrisome health-related information about their child was not a major concern. These issues were important to explore as potentially significant barriers for the ongoing development and implementation of web-based patient portals in this population. Beyond these considerations, parents felt that the portal helped to improve their understanding of and ability to manage their child’s condition. In conclusion, the results of this study support the notion that web-based patient portals have the potential to be a useful tool for parents of children with chronic disease.
Footnotes
Funding
The work was supported by a grant from the Cincinnati Children’s Hospital Medical Center’s Research Foundation Outcomes Award.
