Abstract
Background
Members of the public differ regarding their views on the use and sharing of personal health information.
Objective
This paper describes the methodology employed to develop a telephone-based survey tool to capture views of the public on the collection, use and sharing of personal health information.
Method
A rigorous methodology comprising multiple stages was undertaken to develop a vignette/scenario-based survey instrument. These steps included a review of instruments used in other jurisdictions, focus groups, engagement meetings with healthcare professionals, cognitive testing and piloting the final instrument. Informed by the findings of each survey development phase, draft scenarios and accompanying questions were developed.
Results
The following scenarios were developed: ‘Circle of care,’ ‘Use of information beyond your direct care’ and ‘Digital records.’
Conclusion
The findings from this survey will inform national policy in relation to health information and will inform the development and implementation of eHealth initiatives. In turn, this should support the delivery of high-quality, effective health and social care. The learnings from the development of this survey will contribute to future health information policy and governance in countries or jurisdictions considering the development of a national electronic health record system. Moreover, this research will support public and population health management by encouraging public engagement to support successful implementation of new health information systems.
Introduction
Health Information refers to information collected while an individual is availing of health and social care services, and is primarily used to inform direct patient care. It can be used to inform healthcare management and planning, and, to support research and policy development. 1 However, sharing of personal health information can raise concerns among the public and the preferences and rights of individuals represented within the information need to be considered together with the advantages of sharing health information. 2 Interest in personal health information has gained traction in recent years, coinciding with digitalisation of health care. 3
Sharing health information for secondary purposes provides many potential benefits including supporting high-quality research, informing policy and gaining insights into how to improve patient care and healthcare services.4–6 Various factors can influence views on the use of personal health information, and it is necessary to examine public attitudes and opinions surrounding health information sharing to inform the implementation of initiatives. 7 A recent systematic review of research measuring willingness to share personal health information found that secondary use of health information is an important determinant of individuals’ willingness to share health information. 3 A systematic review found that the public generally acknowledge the benefits of sharing personal health information for quality improvement and research purposes, however this must be balanced with transparency and privacy. 8 Another systematic review on using patient information in research, found that while there was a willingness to share patient information for research, concerns relating to both the security of the information and motivations behind the use of the information also existed. 7 Public concerns relating to privacy and security indicate the need for public awareness and engagement initiatives when introducing and advancing systems to manage personal health information. 9 While significant worries about risks of sharing health information may be present, if adequate security safeguards are implemented, the majority of respondents would be willing to share their health information. 10 Findings from a survey examining patient perspectives on using health information for research purposes concluded that public engagement and education is necessary to ensure that the public’s trust is maintained. 11 These points support the need for continued research and engagement with the public, to allow for the development and successful implementation of new systems relating to sharing health information. Moreover, the need to enhance the practice of health information management through the direct engagement with members of the public is highlighted.
A survey is a suitable method for gathering individual attitudes and opinions from a sample of individuals, 12 including views on initiatives relating to the collection, use and sharing of personal health information.7,13,14 Surveys enable extensive collection of data from a nationally representative sample of individuals 12 and are shown to elicit public opinions on uses of personal health information. 8 More specifically, vignette or scenario-based frameworks which can be tailored, during survey development, have been used to survey individuals on uses of health information.15–17 Vignette surveys present short descriptions of situations or hypothetical accounts to survey respondents to establish their attitudes, opinions and understanding of certain scenarios.18,19 Vignettes have been previously developed to elicit respondents’ views and attitudes towards consent in relation to the uses of personal health information. 16 Moreover, a scenario-based survey administered via telephone was found to be an effective methodology to determine what influences the public’s willingness to consent to share their health information. 16
Given that research highlights the need to reflect the wishes and preferences of the public in policies relating to health information, public acceptance of new initiatives cannot be assumed. As opinions can vary across countries and contexts, it is important to seek the opinions of those who will be impacted by the implementation of new initiatives. The learnings from conducting research to develop this survey may be useful to inform future health information policy and governance in jurisdictions considering the development of national electronic health systems and innovative approaches to the management of personal health records by encouraging public engagement to support successful implementation. Therefore, given that opinions and issues may differ across contexts, this research aimed to develop an instrument tailored to the health information landscape within Ireland. This research aimed to develop a survey on public views on the collection, use and sharing of health information, suitable for administration to a nationally representative sample. The purpose of this article is to detail the methodology for developing this survey.
Method
Research design
The survey will form one part of a wider study exploring public perceptions on collection, use and sharing of health information. A multi-phase approach was adopted to develop a survey to measure attitudes and opinions on the collection, use and sharing of personal health information (see Figure 1). Overview of survey methodology.
Ongoing stakeholder and public engagement was undertaken during survey development. Public engagement involves communicating and consulting with the public on policy development and decision-making to ensure public views are reflected in policy decisions.
20
Figure 2 presents the steps undertaken to develop the survey. Flow chart of survey development.
Development of survey methodology
Phase one: construction of draft questionnaire by review of relevant literature
Prior to survey development, a review of best practice was conducted in relation to public engagement on the collection, sharing and use of health information.
21
This review detailed the methods used in other jurisdictions to consult with the public on health and social care information, in order to inform national policy. Semi-structured interviews were conducted with experts involved in conducting these public engagements to gain further insights into best practices. Each of the identified engagements included a survey. The engagements included in the review were: • National Data Guardian – Review of data security, consent and opt-outs (UK) • Scottish Health Council – Our Voice Citizens’ Panel (Scotland) • Canada Health Infoway – Connecting patients for better health (Canada) • Social Investment Agency – Your Voice, your data, your say (New Zealand) • Data Futures Partnership, Our Data, Our Way (New Zealand).
The Data Futures Partnership (New Zealand) developed a scenario-based/vignette survey which was administered via telephone. A vignette survey describes an event, happening or other scenario and the respondent answers question(s) based on the circumstance described. Respondents choose from or rate multiple hypothetical descriptions of objects (scenarios or vignettes) which vary along attributes and these attributes are presumed to be important determinants of the chosen response. 22 Strengths of this method include that it mimics decision tasks encountered in the real world.23,24
Three focus areas featured across the public engagements reviewed: trust, use of personal health information and consent. Items measuring trust elicited perspectives on the safeguarding of personal information, the technology used to share personal information, and how the information will be used. The questionnaires reviewed also sought to elicit opinions on how personal health information is used in various contexts including for the direct care of the patient, service planning and delivery, supporting national policy on public health improvement, health services research, and use by other government agencies, businesses and community organisations. Finally, the surveys posed questions on the level of control people would like to have in relation to uses of personal health information.
The contents of the questionnaires developed in these countries were compiled into a question bank (64 items) to guide the development of the current survey.
Phase two: focus groups and interviews to inform survey development
Phase two involved engaging with key stakeholders including members of the public and health and social care professionals (HSCPs) through focus groups and semi-structured interviews. The focus group topic guide and interview schedules were informed by phase one.
Focus groups
Two online focus groups (n = 4 and n = 5) were conducted with staff from a national agency in Ireland responsible for monitoring and improving the quality and safety of health and social care services. These focus groups were conducted as a proxy to represent the perceptions of the general public. Focus groups were carried out with staff at the national agency due to difficulties recruiting members of the public as a result of Covid-19 restrictions. The staff that participated in the focus groups did not work in health information and instead were recruited from other areas of the organisation.
Focus group questions and prompts.
Qualitative analysis of the focus group data was carried out. 26 The analysis yielded codes which were grouped according to patterns evident between the codes. The following themes were identified: Privacy and Confidentiality, Complex/Difficult Topic, Patient Safety, Trust, Service Improvement/Efficiency, Ownership, Responsibility, Quality of health information, Data protection, Different levels of access/role based access, Withdrawal, and Informed Consent.
Semi-structured interviews with health and social care professionals (HSCPs)
Semi-structured interview questions with health and social care professionals.
The interviews were coded and the resulting codes were grouped into themes. The following themes were identified: consent, public engagement, uses of health information, challenges with use of health information, integrated care, importance of health information and patient centred.
Phase three: development of scenarios and cognitive interviews
Informed by the findings from phases one and two, draft scenarios and accompanying questions (four scenarios and 33 questions) were developed. The scenarios developed were: ‘the use of health information for direct patient care,’ ‘the use of health information for beyond the care of the patient,’ ‘new technologies in healthcare that will enable more efficient and effective use of health information,’ and ‘sensitive health and social care information.’ These were reviewed by an expert panel of stakeholders who examined whether the scenarios and question items were representative of the content they were intended to measure and that they provided an accurate reflection of the health and social care context in Ireland. The panel was made up of policy makers, people working in health and social care services, patient representatives and representatives of people who use health and social care services. The panel were involved from the outset of the research. Patient representatives were recruited from a national platform for collaborative partnership and engagement with patients, people who use services, family members and carers at a national level. Representatives of people who use health and social care services were recruited through representative and advocacy organisations. Following this expert review the titles of the scenarios, wording of scenarios and questions were updated and modified to ensure accuracy. One scenario relating to sensitive information was dropped and some of the associated questions were incorporated to the remaining scenarios. Following the incorporation of feedback, the revised survey comprised three draft scenarios and 29 questions. The following scenarios were developed: ‘Circle of care,’ ‘Use of information beyond your direct care’ and ‘Digital records.’
Cognitive interviews with patient representatives were undertaken to assess the clarity and appropriateness and validity of the survey scenarios and questions.27,28 The cognitive interviews were conducted online via Zoom. Participants were requested to turn their cameras on to ensure their reactions were observable as they answered the questions. Cognitive interviews involve asking respondents the survey questions and discussing how the respondent developed their answer. Each question’s performance was evaluated. The question-response process (a process that can be conceptualized by four stages: comprehension, retrieval, judgment and response) was examined.
29
The degree of difficulty respondents experience as they formulate an accurate response to the question is also considered. Three main techniques are then used to explore the question and answer process
29
: • Observation – looking for hesitation, confusion, requests for clarification; • Think aloud – the participant is asked to verbalise the thought processes they go through when answering the questions; • Probing – the interviewer asks scripted or spontaneous probes to explore specific issues such as how participants understand a particular word or phrase, what time frame they are thinking about when recalling information, whether there are any missing answer options.
Three rounds of cognitive interviews were carried out with nine patient representatives.
Feedback was considered and incorporated following each round. Following the final round of feedback the survey was sent to a managed service, responsible for the administration of the survey, for approval.
Based on the cognitive testing feedback, the following changes were made to the survey: • Refinement of the survey to reduce repetition and length • More information on the topic was provided in an introduction to each scenario • Greater distinction was made between different secondary uses of health information • Changes were made to the questioning style including not repeating the scale for each question and instead repeating scale for the initial questions until the participant was familiar with the scale • More sensitive demographic questions were moved to the end of the survey as respondents were more comfortable answering these once rapport had been built.
Stakeholder engagement
The survey was developed with input from partner organisations and engagement with key stakeholders at each phase. This group was responsible for the operational aspects of the project, developing the methodology and analysing and reporting on findings. An advisory group was convened which provided feedback in relation to the broader project that this survey relates to. The group was made up of policy makers, people working in health and social care services, patient representatives and representatives of people who use health and social care services. The group advised the core project team on each phase of the development of the survey. Continued stakeholder involvement ensured that the survey instrument achieved satisfactory face and content validity.
Telephone survey methodology
A computer-assisted telephone interviewing (CATI) approach, where the interviewer records and inputs the survey participant’s answers into the questionnaire, was used to administer the survey. CATI is a cost-effective approach to reach a representative sample. CATI interviews have also been found to be less likely to result in socially desirable responses and are therefore more likely to elicit responses to sensitive questions in comparison to personal interviews. 30
Prior to commencement of data collection, interviewers were trained and briefed on the purpose of the project. Interviewers carried out pilot interviews with other team members prior to collecting data.
Piloting the survey and data collection
The survey was conducted between October and December 2020. A pilot of 10 surveys was undertaken from the 19th – 20th of October 2020. 10 participants were recruited to participate in the pilot from a panel of members of the public maintained by the managed service implementing data collection. The purpose of the pilot was to ensure that all components of the survey were easily comprehended by participants. The average length of the pilot survey was 24 min and 25 s.
Participant consent was obtained at the beginning of the survey. Participants were asked to confirm that they understood the purpose of the survey, that participation was voluntary, that they had the right to withdraw from the survey at any time, and that they consented to participate. Once participants consented to participate in the survey, they were asked demographic questions and presented with scenario-based questions in relation to the collection, use and sharing of health information. The scenarios were first read to participants and following this participants were asked a series of questions relating to the scenario presented.
Several amendments were made to the survey following the pilot, based on feedback from interviewers, including shortening of the introduction script, a number of open-ended questions were combined to reduce repetition and minor amendments were made to question wording to improve clarity. See supplemental online material for final survey tool.
Sampling
A sample of 1200 people aged over 18 years was calculated to achieve a nationally representative sample for Ireland, with a plus or minus error at a 95% confidence interval. The sample was quota controlled by gender, age, region, and area and then subsequently weighted by population estimates to match the profile of the population as defined by 2016 Census data. 31 Survey participants were recruited using a random digit dial sample to ensure all households, including ex-directory, were covered. 80% of this sample was accessed through mobile phone numbers and 20% was accessed through landline phone numbers.
Response rate
Overview of sample characteristics.
Data processing, analysis and reporting
Completed questionnaires were entered directly into a customised data entry form using ASKIA software. 32 The data entry form was trialled prior to the commencement of data collection. ASKIA software was also used for tabling of responses. The data were then exported into SPSS Version 25 for further analysis. Missing data were not taken into account during analysis and were excluded from tables and any further analyses. Frequencies and cross tabulations of all questions by demographics and health questions were provided in tables to identify whether frequency of responses varied according to demographic characteristic. Exploratory factor analysis was used to examine the underlying factor structure of the data. Cronbach’s alpha for each factor was calculated to determine the internal consistency and agreement between items on each factor. The findings of the survey are reported elsewhere. 33
A coding framework was developed to analyse responses to the open-ended questions. A sample of the responses to the open-ended questions was analysed by two researchers and a draft coding framework was developed. The coding framework was reviewed by the research team before being finalised. The final coding framework can be found in the supplemental online material.
Discussion
To the best of our knowledge, this is the first study that describes the development of a scenario-based/vignette survey to examine public opinions on the collection, use and sharing of health information. One advantage of vignette surveys is they allow for uniform comparison across responses, as all respondents are presented with the same scenarios. The scenarios in the current survey were developed with the aim of being accessible to respondents. This was achieved through undertaking steps to ensure that scenarios were relevant and easily understood by the target population.
This research demonstrates the value of research and public engagement to instigate and encourage implementation of new health information systems. The introduction of a successful digital technologies system for health information relies on public, governmental, and clinical support, which demonstrates an obligation to consider and represent individuals’ rights and preferences. 2
This survey will identify concerns among the public in relation to the collection, use and sharing of health information. In turn, solutions and approaches to overcome public apprehensions may be proposed. It is important to emphasise the need for public awareness and further engagement initiatives, to inform and support new initiatives with respect to collection, use and sharing of personal health information.9,11 The response rate of the current survey indicates that national surveys are a suitable way to engage with a representative sample of the public to elicit their opinions on the collection, use and sharing of health information.
The research to develop the current survey indicates that issues identified with regard to public opinions on the collection, use and sharing of health information overlap across contexts. For example, core concepts included in the present survey, for example trust and comfort sharing information, have also been identified in other contexts. 2 As developments in relation to data sharing and eHealth initiatives continue, further study is undoubtedly justified in order to advance digital technologies, and to enable progression of initiatives to reduce concerns and hesitation surrounding the sharing of health information. 34 Further investigation into the potential benefits of vignette research to understand public attitudes and perceptions on health information, particularly to understand decision-making relating to health information among the public on a broad scale, is warranted. Additional studies could be conducted to test the generalisability of the scenarios developed in the current research to other contexts. Though this survey was developed specifically for use within an Irish context, minor adaptations may ensure its suitability for use within other jurisdictions and health systems.
A strength of this survey is that it addresses uses of health information for both direct care and uses beyond direct care, such as to research, service planning and quality improvement. Therefore, it provides a comprehensive perspective on the public’s view on the collection, use and sharing of their health and social care information. Moreover, the methodology enables engagement with the public on a broad scale. A further strength of this research is that it seeks to determine the broad public perspective on health information rather than focusing solely on people who use health services. Recent evidence highlights the need to engage with the public in order to allay concerns relating to consent and foster trust as new initiatives are implemented. 35
Limitations
A possible limitation to the validity of this research is that due to the public health restrictions in place during survey development, it was not possible to recruit members of the public to participate in focus groups carried out to inform the survey development process. Focus group participants were staff employed at the organisation of the author team. While participants worked in a health-related organisation, they were not recruited from teams working in health information and were instead sourced from other areas of the organisation. Consequently, the sample that participated in this phase may represent a homogenous group. Additionally, as the participants were employed by a health-related organisation, they may have been more aware of issues pertaining to health information than the general population. Therefore, the generalisability of the findings from this phase of the survey development may be limited. However, the rigorous methodology, comprising multiple stages, attempts to mitigate this potential limitation. A further limitation posed by the scenario-based survey methodology is that results are based on responses to hypothetical scenarios rather than direct observed behaviour.
Conclusion
The findings from this survey will be used together with findings from other phases of a wider project to understand public views on how health information is collected, used and shared in Ireland. Moreover, the learnings from this work may inform similar projects in other jurisdictions with the aim of measuring public opinions and attitudes towards new initiatives and policies regarding health information. This is of crucial importance given that jurisdictions report common issues regarding the introduction of eHealth technologies such as trust and concerns surrounding sensitive information. With minor adaptations to account for local contexts, the survey tool should be suitable for administration in jurisdictions outside of Ireland. The conduct of this survey is timely and will inform the introduction of new technologies to support the health system. Additionally, the findings will help to identify where progress is needed to drive transformational change in relation to the collection, use and sharing of health and social care information and will inform policy in this area. Finally, the development of a methodology to engage with the public on the uses of the health and social care information will ensure that future initiatives will meet the needs of those who avail of health and social care services.
Supplemental Material
Supplemental Material - Engaging with the public on their views on the collection, use and sharing of personal health information: Development of a scenario-based survey instrument
Supplemental Material for Engaging with the public on their views on the collection, use and sharing of personal health information: Development of a scenario-based survey instrument by Ella Tuohy, Alana Murphy-Dooley, Sarah Jane Flaherty, Catherine Duggan, Barbara Foley and Rachel Flynn in Journal of Research Methods in Medicine & Health Sciences.
Supplemental Material
Supplemental Material - Engaging with the public on their views on the collection, use and sharing of personal health information: Development of a scenario-based survey instrument
Supplemental Material for Engaging with the public on their views on the collection, use and sharing of personal health information: Development of a scenario-based survey instrument by Ella Tuohy, Alana Murphy-Dooley, Sarah Jane Flaherty, Catherine Duggan, Barbara Foley and Rachel Flynn in Journal of Research Methods in Medicine & Health Sciences.
Footnotes
Acknowledgements
The National Public Engagement on Health Information was undertaken by the Health Information and Quality Authority (HIQA) in partnership with the Department of Health and the Health Service Executive (HSE) in Ireland. Thank you to all the people who took part in the focus groups and interviews to inform the survey development; your time and contribution was greatly appreciated. Thank you to the members of the Advisory Group, Steering Group, and the Partnership Project Team; your direction and guidance was invaluable. Thank you, finally, to other members of the HIQA team who provided support at various stages of the study.
Author contributions
ET, SJF, CD, BF and RF are current employees of the Health Information and Quality Authority (HIQA), Ireland. HIQA is an independent government agency with statutory responsibilities for health information and for the regulation of health and social care services in Ireland. AMD contributed to the manuscript while on a work placement completed as part of the requirements of the BSc Public Health programme at University College Cork. Conceptualization: RF, BF; Methodology: RF, BF, CD; Investigation: BF, CD; Project administration: CD; Writing – Original Draft: ET, AMD; Writing – Review & Editing: ET, AMD, SJF, CD, BF, RF; Supervision: RF; Funding acquisition: RF.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The National Public Engagement on Health Information was funded by the Health Information and Quality Authority and the Health Service Executive in Ireland.
Ethical statement
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
