Abstract
Patient self-reporting of symptoms and quality of life following surgical interventions is generally delivered in the form of paper-based questionnaires to be completed in the outpatient clinic or at home. A commonly used tool for patient self-reporting of quality of life is the EQ5D health status questionnaire which, while limited in scope, has general applicability across a range of health interventions. In this article we examine the issues relating to online patient self-reporting using this questionnaire and the wider implications for the online reporting of health status.
Introduction
Patient self-reporting of health status following health interventions, especially those involving surgery, is typically delivered in paper form by completing a previously validated questionnaire. 1–3 This is done in an outpatient clinic within the hospital where the patient received treatment, or at home. Glintborg et al. reported that patients’ self-reports are generally reliable. 4 Wolpin et al. commented that traditional paper-based elicitation of quality of life information from patients presents barriers to medical and nursing care processes including delays in receiving information, difficulty in integrating responses with electronic records, and the time required to manually score questionnaires for measurement purposes. 5 Ludemann et al. concluded that the measurement of surgical outcomes using a mail-out questionnaire may not be an accurate method of assessing patient symptoms and satisfaction after surgery. 6
Further work by Gunnars et al. highlighted assessment of a patient’s quality of life as an important complement to traditional objective evaluation measures. 7 The UK NHS White Paper Equity and Excellence: Liberating the NHS specifically advocates the inclusion of patient-reported outcomes (PROMS). 8 Interpretation of quality of life data can be more difficult than interpretation of objective end points such as survival time, or objective response rates. Despite such difficulties, quality of life analyses can provide new insights into the advantages and disadvantages of various treatments not provided by traditional end points.
In addition, Trotter and Morgan commented that there has been increased use of the Internet by patients for health related matters during the period 2000 to 2006. 9 However, this has been mostly for information gathering, rather than as a medium for effective communication with health providers. The increased use of the Internet has aroused interest in using this medium to collect patient-reported outcome information in the form of web-based questionnaires. 10–12 This offers the potential for improved convenience for the patient and improvements in the efficiency of clinical research administration.
Given the importance of PROMS and the paucity of published material in the peer reviewed literature, it is important to review what evidence exists to date and to establish the key questions which need to be addressed in this emerging field. This article then reports results from a pilot study to begin to address some of these questions, with the ultimate aim of serving as a catalyst for new targeted research in PROMS.
Possible advantages arising from web-based administration of health status questionnaires include:
convenience of use by the patient
improved data acquisition, avoiding transcription errors
support for real-time and episodic monitoring of health status
improved accuracy of responses
ease of administration of repeated measures of health status over an extended period following the medical intervention.
However, there are concerns about the uptake of the web as a medium for retrieval of routine information, particularly with respect to response rates, security of communication, patient perception of the appropriateness of this medium to relay health outcome information, and IT literacy among older patients. 13 In addition Nwosu and Cox commented that, while Internet use can be beneficial for patients, it may induce misunderstandings, leading to unrealistic patient expectations and increased medical litigation. 14 This could relate, for instance, to an unstated expectation that reported shortcomings in health status will be addressed, even though the questionnaire is intended for monitoring of an aggregate status rather than for action on individual cases.
Trotti et al. commented that exploratory work suggests that patient-reported outcomes can be used with a high degree of patient engagement and compliance. 15 However, Solomons suggested that patients unfamiliar with the Internet, ‘the information poor’, were unlikely to use the Internet even with assistance. 13 Van Geffen et al. commented that patient reporting is not yet widely accepted, and that the number of systems collecting experiences from patients is still limited. 16 De Langen et al. stated that although differences may exist between reports from patients and reports from healthcare professionals, patient reporting via a reporting system is feasible and can contribute significantly to a reliable pharmocovigilance. 17 Atherton and Sloan commented upon the rising importance of patient-reported outcomes. 18
Basch et al. examined patient versus clinician symptom reporting in a study of 400 cancer patients. 11 The study found that for most systems the agreement between patient and clinician was high. Differences in symptom reporting covered in the study would rarely have changed treatment decisions and dosing, and patients (where differences occurred) typically assigned greater severity to symptoms than did the clinicians. However, Kievit et al. commented that patients’ perceptions of health can be different with equal disease activity, depending on the moment in the disease course. 19 Furthermore, self-reported measures of functionality, disease activity, and general health cannot substitute for objective measures of disease activity (in the case of rheumatoid arthritis) in longitudinal studies, and consequently both need to be measured.
In this article we examine the issues relating to online patient self-reporting with the widely used health status questionnaire EQ5D, 20 drawing conclusions about the wider implications for online reporting of health status.
The research questions addressed in this article are:
Can patient online self-reporting be a feasible alternative to paper-based self-reporting, from the point of view of acceptance by patients, response rates and accuracy of reporting?
What is the validity of self-reporting on the web compared with filling in forms on paper at the clinic?
Are there cultural, literacy, access or other barriers to patient online self-reporting?
Are there ethical and clinical issues arising from patient online self-reporting?
Can patient online self-reporting improve efficiency and effectiveness of patient feedback on health status?
What are the wider implications for the design, implementation and evaluation of web-based systems for patient self-reporting?
These research questions are important because potentially the use of online patient self-reporting can provide further data for improving healthcare delivery and clinical and pharmaceutical research.
Literature review
A range of medical, healthcare and pharmaceutical journals were reviewed with regard to online patient self-reporting and all other forms of patient reporting using the PubMed and Medline databases. 32 A thorough review of articles on web-based patient self-reporting was undertaken. A screening of articles for inclusion was performed by reviewing the titles and abstracts of the articles identified, with a time limit of articles published since 1996 since limited use of the World Wide Web occurred before that date. A set of search criteria with relevant keywords, including patient, self-reporting, online, web-based, and health status, was used to identify potentially relevant articles in the PubMed and Medline databases. Overall only 26 articles were found that directly related to online patient self-reporting of health status. The issues identified in the relevant articles retrieved were categorized by listing the identified issues discussed in the articles and then grouping into similar types of issue. For example, a number of the articles discussed patient Internet access from different perspectives.
Only two articles were found that related directly to patient access to the Internet. Further data regarding general Internet access in the UK were found through examination of the UK Office for National Statistics website. No articles were found that directly concerned website accessibility with regard to online patient self-reporting. From the authors’ experience of website development, appropriate references were found from bodies concerned with general web accessibility standards. Two articles were found that directly concerned the use of clinical terminology with regard to patient self-reporting. Only one article was located that directly related to question phrasing in patient self-reporting. The EQ5D form used in clinical practice in the area studied was used as an example of clarity of response categorization. Only two articles were found that concerned patient perception of self-reporting. Only one article was found that covered the patient classification of symptoms and quality of life with regard to self-reporting. In addition, only one article was located that provided some guidance in completing health status questionnaires. The UK Data Protection Act 1998 was used to outline the requirements relating to data confidentiality and security. Of the other 17 articles found, five concerned the overall use of health status questionnaires, three concerned the reliability and accuracy of health status questionnaires, and nine concerned the general use of the Internet by patients.
Patient access to the Internet
Patient access to the Internet can be a potential barrier to patient online self-reporting. Although the number of households with Internet connections has increased significantly in recent years, there are still households without an Internet connection. 21–23 In 2009, 70 percent of UK households had Internet access. 21 Internet access can be provided for patients without a household Internet connection in outpatient clinics or local libraries. However, this could be viewed as being inconvenient by some patients.
Accessibility/usability of web-based patient self-reporting systems
Any patient online self-reporting system would need to be accessible in terms of patients with visual impairments or other impairments such as reduced fine motor skills. Such a system should also be easy and straightforward for the patient to use. Without due care and consideration of the design of such systems, accessibility and usability could be barriers to patient online self-reporting. Typical website accessibility considerations include: the avoidance of overly patterned backgrounds; good contrast between background and text; reasonably large font sizes in a clear font; and ease of use by screen reader software for users with more severe visual impairments. 24 , 25
Clinical terminology
Clinical terminology used in a patient online self-reporting system needs to be kept to a minimum, or clearly explained in as simple terms as possible. The EQ5D questionnaire in particular uses no clinical terminology. 20 Kessler et al. commented that clinical terms need to be clearly defined when communicating with patients. 26 Basch et al. stated the importance of using suitable language for online patient self-reporting mechanisms. 27
Question phrasing
DeSalvo et al. commented that self-rated health reporting is a valuable tool that can predict important patient outcomes such as mortality and health system utilization. 28 However, despite its value as a patient-centred measure, self-rated health status is not routinely collected in healthcare settings due to the time and cost associated with its collection and interpretation. Typically self-rated health status is a multi-item, multi-scale measure more appropriate for research than for a clinical environment. DeSalvo et al. argued that single-item general health status measurements could serve as a reasonable substitute for multi-item measures of self-related health. 28 Question phrasing for patient online self-reporting needs to be carefully considered so as not to mislead or confuse or bias patients reporting their symptoms of quality of life.
Response categorization
Careful consideration is required with regard to the categorization of responses that the patient can use to self-report symptoms and quality of life. Typically the use of established self-reporting mechanisms, such as EQ5D for quality of life questions and responses, 20 would be an appropriate starting point.
Perceptions of purpose of online self-reporting
It needs to be made clear to patients undertaking online self-reporting exactly how their responses will be treated. In some instances, certain categories of responses may be used to flag further investigation or treatments. 10,11 In other instances, it may be necessary to provide a clear disclaimer that the patient responses are for research purposes only, and patients should report adverse symptoms to their general practitioner or consultant.
Patient classification of symptoms and quality of life
Care needs to be taken with regard to the aspect of the patient’s classification of symptoms or quality of life. 29 It may be difficult for patients to appreciate how the symptoms or quality of life factors that they experience relate to the categories of responses. For example, how would a patient differentiate between mild and moderate pain? In addition, would patients possibly have an expectation of being in discomfort? Providing an initial training session regarding the use of the online patient self-reporting system in an outpatient clinic may alleviate some of such difficulties.
Guidance in completing online forms
Kharod et al. stated that clearly written instructions can improve the accuracy of patient self-reporting. 30 In addition to providing guidance in appropriate ways of completing online self-reporting webpage forms in an outpatient clinic, it can also be appropriate to provide guidance concerning the need for the patient to complete the self-reporting forms. In some instances it might be the case that self-reporting forms are completed with unwanted or inappropriate guidance from relatives or friends, or even that such may actually complete the online self-reporting forms rather than the actual patient.
Data confidentiality and security
Under the UK Data Protection Act 1998, appropriate technical and procedural mechanisms should be applied to the security of personal data. 31 In the context of an online patient self-reporting system, appropriate procedural and technical measures might include the assignment of random usernames and passwords to participating patients. Such anonymized usernames and passwords would be required for access to a patient online self-reporting system. At the server end of the web-based system within the hospital concerned, appropriate usernames and passwords would be required for access to the accumulated patient responses in terms of analysis and reporting of patient-reported outcomes.
Other issues
While the literature review identified a quite substantial range of criteria that require consideration during the development of web-based patient reporting systems, there are further issues for which there is also a lack of published research. The following issues serve as pointers to areas of future work:
the utilization of a validated questionnaire
the use of a disclaimer to avoid misunderstandings by the patient regarding the purpose of the interface
ethical issues requiring management of the patient expectation from the system
the use of appropriate technology and procedures to address data confidentiality and security issues
the specification and design of a user-friendly and accessible web interface.
Methodology
The research methodology employed in this article was to:
apply the relevant findings of the literature review to the design of a web-based patient interface for health status reporting using the EQ5D questionnaire
pilot the web-based questionnaire
carry out structured interviews with the clinicians involved in the study and service user representatives to ascertain their view of the efficacy of patient online self-reporting.
The online questionnaire developed was based upon the EQ5D questionnaire. This consists of five questions relating to: mobility, self-care, usual activities, pain/discomfort, anxiety/depression (all completed by a choice of one of three answers), and overall assessment of health state (indicated on a slider scale). The online questionnaire was evaluated by a focus group panel consisting of six members of a service users research committee affiliated to a UK heart and chest hospital. Three of the group members completed a paper version of the EQ5D questionnaire, followed by the online version. The other three group members completed the online version, followed by the paper version. There was a gap of approximately 30 minutes between the completion of the two versions. The EQ5D questionnaire was completed anonymously as a self-reporting tool by the focus group members. This was used as a single life-cycle point, in order to assess health status at a defined point in time following surgical intervention.
Results
The guidelines derived from the literature review were applied to the prototype web-based reporting system in that an appropriately validated questionnaire (in this case EQ5D) was used a basis for the reporting system design (in order to hopefully achieve appropriate response categorization). Data confidentiality and security of the web-based reporting system incorporated a logon screen requiring a unique and anonymous user identification code and password. The patient perception of the web-based reporting system was considered by inclusion of an appropriate disclaimer that had to be viewed before proceeding to the online questionnaire webpages. Accessibility and usability of the web-based reporting system were aimed for by adopting published web accessibility and usability guidelines.
The equivalence of the entries for the paper-based and web-based reporting system was established by providing the users involved with an anonymous four-digit user identification. This was recorded on the paper version of the form and was used to log into the web-based reporting system. A printout was taken of the responses entered in the web-based reporting system and this was compared with the relevant paper forms (as both contained the same unique four-digit user identification).
The crossover experimental design applied to the small sample (n = 6) of service users research committee group members showed no concerns with validity as the entries were identical on paper and online for the participants. This appeared to indicate that patient online self-reporting could potentially be a feasible and valid alternative to paper-based self-reporting.
Evaluation of prototype online patient self-reporting system
Issues identified regarding the design of the pilot online patient reporting system with regard to barriers to patient online self-reporting by the patient volunteer group included:
Anonymization of usernames and passwords in order to provide for the security of patient data.
The automatic validation of the answer to each question. For example, one respondent did not complete one of the questions on the paper-based form. In contrast, on the web-based patient form, the patient could not proceed to the end of the form unless all the questions were answered.
The design of a slider rule and data capture from this. For example, on the paper-based questionnaire, one respondent entered a response of 65 for overall health state. However, on the online version, this was entered as 70 since the online version only allowed responses in units of 10 out of 100.
The writing of disclaimers in terms of their clarity and the ethical considerations surrounding their use. This has at least two distinct aspects. The first is to manage the expectation of care by indicating that the questionnaire is for passive data acquisition only and not linked to a real-time response unit. The second is to provide guarantees of anonymity. For example, one respondent queried whether responses to the questionnaire might affect the status of their pensions or other allowances.
Riders are required in addition to disclaimers. As an example, when administering the questionnaire following a specific clinical procedure, the question ‘Are you experiencing discomfort?’ may be interpreted by the patient in two ways: either discomfort attributed to the procedure, or discomfort from any cause such as a comorbidity. This ambiguity has to be explicitly clarified at the entry point into the questionnaire.
The automatic saving and restart of the online questionnaire, since some patients might be interrupted during completion of the form online (or in paper-based form).
Date stamping of the patient responses.
Coding of the data for future use in statistical analysis.
Administration of the online questionnaire
The issues that arose during piloting of the web-based questionnaire with regard to ethical and clinical aspects of the administration of online self-reporting included:
possible first login at the outpatient clinic
training issues for patients
accessibility for the visually impaired
clarity of use
partial completion of form
verification issues – the online form automatically verified that patient answers complied with a preset menu, e.g. only one answer out of three can be selected, which is not as easily implemented in paper form due to the potential need for adaptive menus
possible inclusion of training questions for the participants prior to their entry into the questionnaire pages.
Overall it was not possible (due to the limited scope of the study) to determine whether patient online self-reporting could improve the efficiency and effectiveness of patient feedback of health status, other than to provide an indication that it could be a feasible alternative that requires further examination.
Evaluation
In order to further evaluate the pilot online health reporting system, structured interviews were conducted with two relevant clinicians in order to evaluate the pilot system from a clinical perspective. The structured interview questions included:
Would patients be sufficiently aware of the purpose of the online questionnaire?
Would patients be sufficiently aware of what they would need to do in order to report adverse symptoms that might require medical attention?
Is there sufficient guidance within the online questionnaire for patients to be able to complete the questionnaire?
Is there sufficient validation within the online questionnaire to prevent patients from making data entry mistakes (e.g. multiple responses when only one is required)?
Is the presentation of the online questionnaire appropriate for the intended patient group (e.g. appropriately formal and professional)?
Would the online questionnaire be easier/equivalent/more difficult for patients to complete than the paper-based equivalent?
Would patients be more/equivalent/less likely to complete the online questionnaire than the paper-based questionnaire?
Would patients be more/equivalent/less likely to accurately complete the online questionnaire than the paper-based questionnaire?
These questions were asked in order to acquire a wider view of the patient cohort (since the prototype evaluation concerned the views of only a small number of patients).
The interview responses indicated that:
There were some genuine concerns on data security and how information would be utilized without revealing the participant’s identity.
There is strong support for completing the questionnaire electronically, albeit with some degree of training, the level of which will depend on the user’s basic computer literacy.
There is a degree of uncertainty on how to respond to some of the questions in the EQ5D form and whether to focus on specific or general conditions affecting quality of life for both versions of the questionnaire.
Screen resolution can be problematic. This can be improved by reducing the image banner and the blank lines between the text on the screen.
A touch screen format could avoid the need for vertical scrolling which can cause problems for certain individuals with poor basic computer literacy.
The system could be difficult to administer for certain individuals with visual impairment.
Conclusion
There is a dearth of published research into online patient self-reporting. Historically, lack of access to the Internet would have hindered remote electronic self-reporting. However, with UK home Internet access now reaching 70 percent, and forecasted to rise still further, online patient self-reporting is much more feasible in practice. Some patients will still require Internet access either through an outpatient clinic or in local libraries. The results of a small scale pilot study of online patient self-reporting indicate that online self-reporting can be as valid and reliable as paper-based self-reporting, with positive feedback obtained from both patient representatives and the clinicians involved. However, patient training, data confidentiality and the security and accessibility of any online self-reporting system need careful consideration. Further larger scale studies are required before online patient self-reporting can become a viable option for self-reporting of quality of life and other patient self-reporting, in particular, with regard to the impact of online self-reporting on response rates and accuracy. This article sets out key issues for consideration and provides guidance for the design of such systems.