Use and impact of virtual primary care on quality and safety: The public's perspectives during the COVID-19 pandemic

Abstract

Introduction

With the onset of Coronavirus disease (COVID-19), primary care has swiftly transitioned from face-to-face to virtual care, yet it remains largely unknown how this has impacted the quality and safety of care. We aim to evaluate patient use of virtual primary care models during COVID-19, including change in uptake, perceived impact on the quality and safety of care and willingness of future use.

Methodology

An online cross-sectional survey was administered to the public across the United Kingdom, Sweden, Italy and Germany. McNemar tests were conducted to test pre- and post-pandemic differences in uptake for each technology. One-way analysis of variance was conducted to examine patient experience ratings and perceived impacts on healthcare quality and safety across demographic characteristics.

Results

Respondents (n = 6326) reported an increased use of telephone consultations ( + 6.3%, p < .001), patient-initiated services ( + 1.5%, n = 98, p < 0.001), video consultations ( + 1.4%, p < .001), remote triage ( + 1.3, p < 0.001) and secure messaging systems ( + 0.9%, p = .019). Experience rates using virtual care technologies were higher for men (2.4 ± 1.0 vs. 2.3 ± 0.9, p < .001), those with higher literacy (2.8 ± 1.0 vs. 2.3 ± 0.9, p < .001), and participants from Germany (2.5 ± 0.9, p < .001). Healthcare timeliness and efficiency were the dimensions most often reported as being positively impacted by virtual technologies (60.2%, n = 2793 and 55.7%, n = 2,401, respectively), followed by effectiveness (46.5%, n = 1802), safety (45.5%, n = 1822), patient-centredness (45.2%, n = 45.2) and equity (42.9%, n = 1726). Interest in future use was highest for telephone consultations (55.9%), patient-initiated digital services (56.1%), secure messaging systems (43.4%), online triage (35.1%), video consultations (37.0%) and chat consultations (30.1%), although significant variation was observed between countries and patient characteristics.

Discussion

Future work must examine the drivers and determinants of positive experiences using remote care to co-create a supportive environment that ensures equitable adoption and use. Comparative analysis between countries and health systems offers the opportunity for policymakers to learn from best practices internationally.

Keywords

Telemedicine primary care COVID-19

Introduction

The delivery of health and care services has evolved rapidly following the onset of the Coronavirus disease (COVID-19) pandemic. To mitigate the transmission of COVID-19, and respond to the increased burden of care, the pandemic has advanced the use of virtual care technologies in facilitating care for patients globally, disrupting traditional face-to-face care models. Virtual remote care may be delivered via telephone or text, or video and other online tools which require Internet connection. In England, based on updated policy guidance, primary care practitioners have adapted their models of care to offer over 85% of consultations remotely via phone or video link, where previously 95% of consultations took place face to face.¹ Similarly, in Italy, the Digital Solidarity initiative launched by the Minister for Technological Innovation and Digitization, offers online public or specialist services in response to the pandemic, including health services such as ‘Video Visita’ (Video Visit), which offers patients free video consultations with family doctors and paediatricians signed up to the service.²

While the use of virtual care accelerated rapidly in response to the COVID-19 pandemic, the digitisation of healthcare has been advancing over decades,³ as has the utilisation of digital technology in healthcare delivery as a policy ambition at the global level.⁴ For example, in the UK, the NHS Five Year Forward Review, for example, outlined plans for England to utilise digital technologies to offer patients more convenience in accessing care.⁵ However, uptake at scale prior to the pandemic was slow and challenging.

While part of an emergency response, the rapid adoption of virtual care solutions in General Practice will likely have long-term effects on how services are delivered, there is little evidence on its impact on the quality, safety and equity of care.⁶ There is also sparse evidence on the lessons learned through the experience of delivering large-scale virtual services during the pandemic, which could be utilised to lay the foundations for high-quality virtual care delivery in the post-pandemic future. To facilitate safe, sustainable adoption of virtual models for health and care delivery, it is critical to listen to the perspectives of patients as end users. Targeted public research must seek to understand their perspectives on the impact of the care received and the risk of widening health inequities between different groups of patients.

This study aimed to evaluate the impact of the adoption of virtual technologies by patients for primary care access during the COVID-19 pandemic. The primary aims were to evaluate the use of virtual care solutions by patients during the COVID-19 crisis, explore different rates of adoption and perceptions across different patient groups, to understand the perceived impact on quality and safety of care during the crisis and to examine how they would like to continue using virtual primary care, in the post-pandemic future.

Methodology

Study design

A cross-sectional online survey was performed, adhering to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline for cross-sectional studies.⁷

Participants

Eligible study participants were adults aged over 18 years who were able to speak, read and write in their resident country's language. Recruitment was carried out with YouGov, an international research data and analytics group. Participants were recruited by YouGov from a host of different sources, including via standard advertising, and strategic partnerships with a broad range of websites. Stratified sampling was used to recruit responses from four countries (Germany, Italy, Sweden, United Kingdom) and, to ensure nationally representative samples, YouGov drew a subsample representative of national adults in terms of age, gender, social class and education level. Data collection took place in December 2020. Ethical approval was granted by Imperial College London's Ethics Research Committee (Approval number: 20IC5956).

Survey development

The questions of the survey were developed based on a scoping review and expert consultation. A draft was piloted with general practitioners (GPs)/foundation doctors (FDs) across 20 countries, and feedback was incorporated in the final version. The final survey included 4 sections with a total of 13 items.

The first section examined patients’ characteristics (i.e. gender, age, ethnicity, overall health status, digital health literacy). Digital health literacy was assessed using the eHealth Literacy Scale (eHEALS), a validated tool that reflects the individuals’ own perception of their knowledge and skills at using eHealth information.⁸ The validity and reliability of eHEALS have been demonstrated in several health conditions,^9,10 and have been translated into various languages.^11,12 As in previous studies, the eHEALS score was categorised into high eHealth (eHEALS≥26 out of 40) and low eHealth literacy levels (eHEALS<26).^9,11,12

The second section explored patients’ use of virtual care before and during the pandemic, including the use of telephone consultations, video consultations, chat consultations, remote triage, patient-initiated digital services (i.e. scheduling, prescription renewal, test requests, sick notes/medical certificates) or secure messaging systems (i.e. to receive test results or other information). Patients’ experiences of these models of care were assessed using a free-text question ‘In general, how was your overall experience using these technologies?’ and a 5-point Likert scale (‘1 – very bad’ to ‘5 – very good’).

The third section assessed patients’ perceptions on the impact of these technologies on several dimensions of quality and safety of care, as defined by the Institute of Medicine, including patient-centredness, effectiveness, efficiency, safety, timeliness and equity.¹³

The fourth section explored respondents’ willingness of future use of each investigated virtual care technology.

The online survey was hosted on the Qualtrics platform. A copy of the Qualtrics survey is provided in Supplementary Material.

Data analysis

Quantitative data were analysed using Statistical Package for the Social Sciences (SPSS) v.27. Descriptive statistics were calculated, including absolute (n) and relative frequencies (%) for categorical variables, and mean (μ) and SD for continuous variables. McNemar tests were used to examine differences in adoption of each respective technology across countries before versus during the COVID-19 pandemic. One-way analysis of variance (ANOVA) was used to test group-based differences in the rated experience of using virtual technologies. Chi-square tests of independence were generated to test associations between demographic characteristics and perceived impacts of virtual care technologies on the quality and safety of care, as well as willingness to use virtual care technologies in the future. The significance level for all statistical tests was set at p-value (p) <0.05, two-tailed.

Role of the funding sources

The funders of the study were not involved in the development of the study design; in the collection, analysis and interpretation of data; in the writing of the report or in the decision to submit the paper for publication.

Results

Participants’ characteristics

A total of 6325 participants replied to the survey. A detailed description of the characteristics of the sample can be found in Table 1, including a breakdown per country.

Table 1.

Characteristics of the respondent population (n = 6326), by country.

	UK(n = 2019)	Germany (n = 2161)	Italy(n = 1131)	Sweden(n = 1015)	Total(n = 6326)
Age, n (%)
18–24	214 (10.6)	196 (9.1)	92 (8.1)	84 (8.3)	586 (9.3)
25–34	326 (16.1)	330 (15.3)	152 (13.4)	205 (20.2)	1013 (16.0)
35–44	320 (15.8)	314 (14.5)	193 (17.0)	143 (14.1)	970 (15.3)
45–54	327 (16.2)	388 (18.0)	221 (19.5)	155 (15.3)	1090 (17.2)
55 +	831 (41.2)	933 (43.2)	474 (41.9)	428 (42.2)	2668 (42.2)
Gender, n (%)
Female	1038 (51.4)	1103 (51.0)	593 (52.4)	507 (50.0)	3241 (51.2)
Male	981 (48.6)	1058 (49.0)	538 (47.6)	508 (50.0)	3085 (48.8)
Ethnicity, n (%)
Asian	59 (2.9)	0 (0.0)	4 (0.4)	21 (2.1)	84 (1.3)
Black/African/Caribbean	21 (1.0)	0 (0.0)	5 (0.4)	6 (0.6)	32 (0.5)
Mixed/Multiple ethnic groups	48 (2.4)	0 (0.0)	65 (5.7)	45 (4.4)	158 (2.5)
Other ethnic group	25 (1.2)	0 (0.0)	19 (1.7)	11 (1.1)	55 (0.9)
White	1866 (92.4)	0 (0.0)	1019 (90.0)	914 (90.0)	3798 (60.0)
Prefer not to say	0 (0.0)	2161 (100.0)	20 (1.8)	18 (1.8)	2199 (34.8)
eHealth Literacy Scale (eHEALS), μ (SD)	17.8 (6.4)	20.0 (7.0)	20.1 (6.5)	17.0 (6.7)	18.8 (6.8)

No significant differences were found in the composition of the female versus male respondents in the UK (51.4% vs. 48.6%), Germany (51.0% vs. 49.0%), Italy (52.4% vs. 47.6%) and Sweden (50.0% vs. 50.0%) (χ² = 1.376, p = 0.711). The proportion of respondents aged 25–34 years was slightly higher in the UK (16.1%) and Sweden (20.2%) compared to Germany (15.3%) and Italy (13.4%), (χ² = 36.277, p < 0.001).

The proportion of White respondents was higher in the UK (92.4%) than in Sweden (90.0%) and Italy (90.0%) p < 0.001). The UK had a lower proportion of multiple ethnic respondents compared to Sweden and Italy (2.4% vs. 4.4% and 5.7%) (p < 0.001). In Germany, it was not possible to collect ethnicity data due to GDPR restrictions.

Digital literacy was lowest in Sweden (17.0 ± 6.7) compared to the UK (17.8 ± 6.4) (p = 0.009), Germany (20.0 ± 7.0) (p < 0.001) and Italy (20.1 ± 6.5) (p < 0.001).

Use of virtual primary care before and during the COVID-19 pandemic

Before the COVID-19 pandemic, the highest usage was observed for remote patient-initiated services (19.4%, n = 1228) and secure messaging systems (10.0%, n = 632). Use of telephone, video, chat consultations and remote triage were 24.9% (n = 1572), 5.4% (n = 341) 6.7% (n = 425) and 6.2% (n = 392), respectively. During the COVID-19 pandemic, significant increases were observed for telephone consultations ( + 6.3%, p < 0.001), patient-initiated services (+1.5%, n = 98, p < 0.001), video consultations ( + 1.4%, P < .001), remote triage ( + 1.3, p < 0.001) and secure messaging systems ( + 0.9%, p = 0.019) Figure 1(a)).

Figure 1.

Percentage of usage of each respective technology before and after COVID-19, by country (in %).

In the UK, there was a significant increase for all virtual care technologies, including telephone consultations ( + 18.4%, p < 0.001), remote triage ( + 3.9%, p < 0.001), video consultations ( + 3.8%, p < 0.001), patient-initiated services ( + 2.7%, p< 0.001), secure messaging ( + 1.4%, p = 0.024) and chat consultations ( + 1.2%, p = 0.013) (Figure 1(b)). In Germany, there was no significant variation in any of the virtual care technologies (Figure 1(c)). In Italy, there was a significant increase in patient-initiated services (4.5%, P < .001), telephone consultations (3.8%, p = 0.004), chat consultations (2.7%, p = 0.010) and remote triage (2.6%, p < 0.001) (Figure 1(d)), but not in secure messaging services or video consultations (1.9%, p = 0.123 and 1.4%, p = 0.072, respectively). In Sweden, telephone consultations decreased significantly (−3.7%, p = 0.007), and there was no significant difference in the use of other technologies (Figure 1(e)).

Patient experience of virtual primary care before and during the COVID-19 pandemic

Of the 3976 (62.9%) responses to the experience item, the majority reported having had either a good or very good experience (59.8%, n = 2379). Only a small minority reported a bad or very bad experience (8.4%, n = 333). Lower rated experience was observed in women (2.4 ± 1.0 vs. 2.3 ± 0.9, p < 0.001) and those with low literacy respondents (eHEALS≥26) (2.3 ± 0.9 vs. 2.8 ± 1.0, p < 0.001). Patient experience (expressed as the average Likert scale value) significantly varied between countries (p < 0.001), with participants from Germany reporting a higher average Likert scale value (2.5 ± 0.9) compared to those from the UK (2.2 ± 1.0) (p < 0.001), Italy (2.3 ± 0.8) (p < 0.001) and Sweden (2.3 ± 1.0) (p < 0.001).

Impact of digital remote care on quality and safety of care during the COVID-19 pandemic

Globally, digital patient-initiated services were most often reported as having a positive impact on the quality of care received across respondents (61.0%, n = 2171), followed by telephone consultations (56.5%, n = 2102), and video consultations (44.1%, n = 1246). Online triage (42.5%, n = 1294), chat consultations (40.3%, n = 1177) and remote monitoring (36.6%, n = 968) were least often reported as impacting positively on the quality of care. A breakdown per country for each technology and proportions of missing answers are provided in Supplementary Tables 1–6.

Virtual care technologies were most often reported as having positively impacted on timeliness (60.2%, n = 2793) and efficiency (‘avoiding waste’) (55.7%, n = 2401) of care, followed by effectiveness (46.5%, n = 1802), safety (‘avoiding harm’) (45.5%, n = 1822) and patient-centredness (45.2%, n = 45.2). Equity was least often reported as having been positively impacted (42.9%, n = 1726). The proportion of missing answers for each domain of quality is reported in Supplementary Table 7.

Significant variation was observed in the perceived impact of virtual care technologies on all dimensions of quality between countries (Figure 2) (all p < 0.001). Respondents from Italy were most likely to report a positive impact on each of the six dimensions of care quality. Conversely, respondents in the UK were most likely to report a negative impact on each of the six dimensions of care quality, although all <17.0%.

Figure 2.

Respondents’ perceptions of how virtual care technologies have impacted on each aspect of care received, by country (in %).

Willingness to use virtual care technologies after the COVID-19 pandemic

Amongst those that answered the questions about intended future use, a slight majority of participants reported wanting to continue using telephone consultations (55.9%, n = 3196) and patient-initiated digital services (56.1%, n = 3206) (Figure 3). Less than half of the participants expressed willingness to use secure messaging systems (43.4%, n = 2480), online triage (35.1%, n = 2005), video consultations (37.0%, n = 2114) or chat consultations (30.1%, n = 1718) in the future.

Figure 3.

Respondents' future willingness to use of each respective technology after COVID-19, by country (in %).

A significant variation was observed between countries (p < 0.001), with German participants (24.8%, n = 477) being least likely to want to use any of the listed virtual care technologies, followed by the UK (14.6%, n = 267), Sweden (7.4%, n = 50) and Italy (4.8%, n = 50). UK participants reported the highest support for future use of telephone consultations (67.1%, n = 1226), video consultations (47.0%, n = 858) and online triage (46.3%, n = 845). In Sweden, most participants supported the future use of patient-initiated digital services (66.1%, n = 606), secure messaging systems (52.2%, n = 479) and chat consultations (37.8%, n = 347). In Italy, participants were most frequently interested in using patient-initiated digital services (64.4%, n = 679), secure messaging systems (54.2%, n = 570) and telephone consultations (52.3%, n = 550) in the future. German participants showed the least frequent willingness in future use of all the technologies evaluated, although nearly half of respondents expressed interest in the future use of telephone consultations (44.5%, n = 857) and patient-initiated services (46.3%, n = 891).

Willingness to use telephone consultations increased with increasing age; for video, chat consultations and online triage, interest in future use reached a peak at 35–54 years, progressively declining in older groups (all p < 0.001). Not wanting to use any of these remote care solutions was higher in men (14.7 vs. 12.6, p = 0.017), those with higher digital literacy (24.4 vs. 12.0, p < 0.001), and those in higher age bands (p < 0.001). Interestingly, participants with higher eHEALS score consistently reported a lower intention to use each one of the remote care technologies evaluated (all p < .001). A detailed overview of these results is provided in Supplementary Tables 8–12.

Discussion

Our findings indicate a stark increase in the use of remote care technologies in primary care during the COVID-19 pandemic in the UK and Italy. This is in line with recently published data in the UK, which suggests that, by April 2020, almost half of the appointments were reported as taking place remotely.¹⁴ This sharp rise contrasts with usage in Germany and Sweden, which were found to remain more conservative in the triaging patients to virtual care (despite German participants providing the most positive experience ratings). These differences may reflect the respective health policy landscape in each country, underscoring the potential of lessons to be gained from inter-country comparisons. For example, in contrast to the UK, where commercial products such as Skype, WhatsApp or Facetime are used in urgent cases (if no alternatives available), digital health applications in Germany require approval by the responsible government agencies based on a demonstration of evidence of positive effects to be accredited and reimbursed.¹⁵

Most of our participants reported having had a (very) good experience using remote care. In the UK, Imlach et al. (2020) also found that patients reported high satisfaction with virtual care, describing it as a convenient mode of care that allowed them to safely access health care without having to weigh up the fear of COVID-19 infection against the need to be observed.¹⁶ Yet, it is important to note that patients’ experiences vary across patients’ characteristics; our results suggest a better experience by those with higher digital health literacy and, in Italy, by some ethnic groups. Timeliness and efficiency were reported as the quality of care dimensions most positively impacted by the introduction of virtual care. Graetz et al. (2020) examined patient-initiated primary care visits in a large integrated health care delivery system with over 4 million members and observed that fewer than half of in-person visits were scheduled within 1 day of making the appointment, compared to over half of video visits, and over two-thirds of telephone visits.¹⁷ On average, telephone visits were scheduled 50% sooner than office visits. This is in line with findings of previous studies highlighting that remote care can reduce wait times.^18,19 Previous evidence has also highlighted a potential benefit of remote care regarding the efficacy of physicians, reduced medical costs and improved the access to health care services,^20–23 although evidence is somewhat inconsistent.^24,25

Across countries, more than half of the participants wanted to continue using telephone consultations and patient-initiated digital services. Interestingly, participants with higher eHealth literacy consistently reported a lower willingness to use each one of the remote care technologies evaluated – despite reporting higher experience rates. Interest in future use differed significantly by country, with German participants being least likely to want to use any of the listed virtual care technologies, potentially due to the relative novelty of these applications. Our results also suggest that patients with higher eHealth literacy may report a lower willingness to use virtual care in the future – this finding was consistently observed for all technologies. As per the technology acceptance model (TAM), the adoption of technology can be forecasted by perceived usefulness and perceived ease of use.²⁶ It might be hypothesised that a more in-depth understanding of the risks associated with virtual care, as well as some potential digital fatigue by respondents with high digital literacy, may have a negative impact on their interest in future use. Other variables, such as social influence, facilitating conditions, trust, privacy, perceived risk, technological anxiety and resistance towards technology may also play an important role and could contribute to the variations observed.²⁷

Finally, it is important to note that, amongst all dimensions of quality of care evaluated in this study, impact on healthcare equity was considered as least positive. These findings echo recent results by Chang et al. (2021) that recognise important differences based on community characteristics in both the primary mode of remote care used and the types of barriers experienced.²⁸ Our findings suggest that some patient groups may be particularly less engaged in the future use of remote care, including older persons. Reed et al. (2020) observed that, for example, patients aged 65 years and over were less likely than patients aged 18–44 years to choose telemedicine.²⁹ As mobile devices are used in most video visits, strategies to increase availability and access in these groups, as well as the co-design of user-centred solutions, may represent valuable opportunities to engage these patients.³⁰ These differences in preferences by type of technology may indicate that access is not uniform.^29,31 Despite the trend of a lower willingness in future use in older persons for most technologies, it is important to note that there appears to be a nuance in the choice of telephone visits and other types. For example, the proportion of participants wanting to keep using telephone consultations progressively increases with age. These results suggest important aspects of the impact of age and intended future use of different remote care choices and highlight that some remote care options may better meet specific patient groups’ needs than others.

Strengths and limitations

A major strength of the study is the large sample size which gives more reliable results through greater statistical power, covering four different healthcare settings (UK, Germany, Italy, Sweden). The international scope of this work offers a wider lens in which to consider the perceived impact on quality, safety and equity on primary care, and compare across a variety of settings. The sample populations were nationally representative, ensuring results accurately reflect the population in each national context. Furthermore, this study performed a detailed characterisation of the sample, including through the eHEALS health literacy tool.

The survey was conducted online meaning there is a risk of selection bias, as individuals with poor access to digital technology and the Internet are less likely to have participated in the survey. As participants were only included if they were able to speak the resident country's language, a proportion of the population was excluded from participation, likely including some vulnerable groups such as migrant workers and refugees. The non-randomised sample used in the study is a further limitation as its use makes the results less generalisable than the results of a randomised sample. However, this limitation was partly mitigated by ensuring the samples analysed were nationally representative. Third, reporting bias is also widely recognised as a limitation of self-reported surveys assessing self-assessed behaviour.³² While the anonymity of the survey may have helped to reduce reporting bias, the possibility of this bias persisting should still be acknowledged as the study measured reported use rather than actual use.

Implications for research and policy

This study is a starting point in considering the perspectives of the public across different health systems, as end users of virtual care solutions in primary care through the COVID-19 pandemic and beyond.

Discrepancies in experience rates between different groups of patients (particularly older persons or those with low digital health literacy), require further exploration to determine the risk of virtual care services amplifying existing inequities in healthcare. Creating effective digital health solutions is an opportunity that can be easily missed if digital design and implementation does not consider the needs and preferences of lower health literacy groups.³³ There is, therefore, a need to develop digital health solutions that specifically meet the needs of lower health-literate individuals, in order to increase overall acceptability and experience.³⁴

Equity was the dimension that participants least often reported a positive impact – proactive efforts to address remote care barriers associated with specific groups of patients are therefore critical to ensure the wide-scale implementation of virtual care does not entrench existing health disparities in health access in already underserved communities.²⁸ After the emergency phase of the pandemic, we need to understand what worked well and for which patients, incorporating the lessons learned into new strategies to improve equity and digital inclusion.

Timeliness and efficiency were the most reported positive aspects of virtual care solutions, which offer the opportunity for further research to understand how these impact other domains of quality of care, particularly patient experience, and healthcare system savings. Until now, evidence on the cost-cutting potential of remote care has not been met with the same vigour as evidence generation on their role in improving health outcomes.³⁵

Conclusion

The study provides new evidence on the topic of the perceived impact of using virtual care technologies in primary care during COVID-19, evaluating the use and perceptions on a range of virtual care tools. Future work must examine the drivers and determinants of positive experiences using remote care to co-create a supportive environment that ensures equitable adoption and use across different patient groups. Comparative analysis between countries and health systems offers the opportunity for policymakers to learn from best practices internationally.

Supplemental Material

sj-docx-1-jtt-10.1177_1357633X211066235 - Supplemental material for Use and impact of virtual primary care on quality and safety: The public's perspectives during the COVID-19 pandemic

Supplemental material, sj-docx-1-jtt-10.1177_1357633X211066235 for Use and impact of virtual primary care on quality and safety: The public's perspectives during the COVID-19 pandemic by Ana Luisa Neves, Jackie van Dael, Niki O’Brien, Kelsey Flott, Saira Ghafur, Ara Darzi and Erik Mayer in Journal of Telemedicine and Telecare

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work is supported by the National Institute for Health Research (NIHR) Imperial Patient Safety Translational Research Centre and Imperial College COVID-19 Support Fund. Infrastructure support was provided by the NIHR Imperial Biomedical Research Centre.

Supplemental material

Supplemental material for this article is available online.

ORCID iD

Niki O’Brien

References

Health Foundation. How might COVID-19 have affected people's ability to see their GP? [Internet]. health.org.uk. 2021. Available from: https://www.health.org.uk/news-and-comment/charts-and-infographics/how-might-covid-19-have-affected-peoples-ability-to-see-GP (accessed Jan 16, 2021).

Ministro per l’innovazione tecnologica e la digitalizzazione. Coronavirus, la solidarietà digitale si estende in tutta Italia [Internet]. innovazione.gov.it. 2020. Available from: https://innovazione.gov.it/notizie/articoli/coronavirus-la-solidarieta-digitale-si-estende-in-tutta-italia/ (accessed Jan 23, 2021).

NHS Management Executive. An Information Management and Technology Strategy for the NHS in England: A view for hospital doctors [Internet]. medconfidential.org. 1992. Available from: https://medconfidential.org/wp-content/uploads/2018/04/1992-IT-NHS-Strategy-Hospdoct.pdf (accessed Jan 23, 2021).

Mitchell

Kan

. Digital technology and the future of health systems. Health Systems & Reform 2019; 5: 113–120.

NHS. Next steps on the NHS five year forward view [Internet]. england.nhs.uk. 2017. Available from: https://www.england.nhs.uk/wp-content/uploads/2017/03/NEXT-STEPS-ON-THE-NHS-FIVE-YEAR-FORWARD-VIEW.pdf (accessed March 3, 2018).

Guo

Ashrafian

Ghafur

, et al. Challenges for the evaluation of digital health solutions—A call for innovative evidence generation approaches. NPJ Digital Medicine 2020; 3: 1–14.

Von Elm

Altman

Egger

, et al. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Bull W H O 2007; 85: 867–872.

Norman

Skinner

. eHEALS: the eHealth literacy scale. J Med Internet Res 2006; 8: e27.

Milne

Puts

Papadakos

, et al. Predictors of high eHealth literacy in primary lung cancer survivors. J Cancer Educ 2015; 30: 685–692.

10.

Robinson

Graham

. Perceived internet health literacy of HIV-positive people through the provision of a computer and internet health education intervention. Health Information & Libraries Journal 2010; 27: 295–303.

11.

Van der Vaart

Van Deursen

Drossaert

, et al. Does the eHealth literacy scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations. J Med Internet Res 2011; 13: e86.

12.

Mitsutake

Shibata

Ishii

, et al. Developing Japanese version of the eHealth literacy scale (eHEALS). [Nihon koshu eisei zasshi] Japanese journal of public health 2011; 58: 361–371.

13.

Institute of Medicine. Crossing the quality chasm: a new health care system for the 21st century. Washington, DC: National Academy Press, 2001.

14.

NHS Digital. Appointments in General Practice - April 2020 [Internet]. digital.nhs.uk. 2020. Available from: https://digital.nhs.uk/data-and-information/publications/statistical/appointments-in-general-practice/april-2020 (accessed June 5, 2020).

15.

Gerke

Stern

Minssen

. Germany's digital health reforms in the COVID-19 era: lessons and opportunities for other countries. NPJ digital medicine 2020; 3: 1–6.

16.

Imlach

McKinlay

Middleton

, et al. Telehealth consultations in general practice during a pandemic lockdown: survey and interviews on patient experiences and preferences. BMC Fam Pract 2020; 21: 1–14.

17.

Graetz

Huang

Muelly

, et al. Patient choice of telemedicine increases timeliness of primary care visits. American Medical Informatics Association 2020; 502–503.

18.

AlAzab

Khader

. Telenephrology application in rural and remote areas of Jordan: benefits and impact on quality of life. Rural Remote Health 2016; 16: 1–9.

19.

Breen

Murphy

Browne

, et al. Formative evaluation of a telemedicine model for delivering clinical neurophysiology services part I: utility, technical performance and service provider perspective. BMC Med Inform Decis Mak 2010; 10: 1–12.

20.

Tsopra

Gimenez

, et al. Benefits and challenges of using virtual primary care during the COVID-19 pandemic: from Key lessons to a framework for implementation. medRxiv 2021; 2021.07.28.21261021. [Preprint]

21.

Snoswell

Stringer

Taylor

, et al. An overview of the effect of telehealth on mortality: a systematic review of meta-analyses. J Telemed Telecare 2021; 0(0): 1–10.

22.

Snoswell

Chelberg

De Guzman

, et al. The clinical effectiveness of telehealth: a systematic review of meta-analyses from 2010 to 2019. J Telemed Telecare 2021; 0(0): 1–16.

23.

Langabeer

Champagne-Langabeer

Alqusairi

, et al. Cost–benefit analysis of telehealth in pre-hospital care. J Telemed Telecare 2017; 23: 747–751.

24.

Ekeland

Bowes

Flottorp

. Effectiveness of telemedicine: a systematic review of reviews. Int J Med Inf 2010; 79: 736–771.

25.

Snoswell

Taylor

Comans

, et al. Determining if telehealth can reduce health system costs: scoping review. J Med Internet Res 2020; 22: e17298.

26.

Chau

Sheng

ORL

, et al. Examining the technology acceptance model using physician acceptance of telemedicine technology. Journal of management information systems 1999; 16: 91–112.

27.

Kamal

Shafiq

Kakria

. Investigating acceptance of telemedicine services through an extended technology acceptance model (TAM). Technol Soc 2020; 60: 101212.

28.

Chang

Lai

Gupta

, et al. Rapid transition to telehealth and the digital divide: implications for primary care access and equity in a post-COVID Era. Milbank Q 2021; 99(2): 340–368.

29.

Reed

Huang

Graetz

, et al. Patient characteristics associated with choosing a telemedicine visit vs office visit with the same primary care clinicians. JAMA network open 2020; 3: e205873–e205873.

30.

Graetz

Huang

Brand

, et al. Bridging the digital divide: mobile access to personal health records among patients with diabetes. Am J Manag Care 2018; 24: 43.

31.

Drake

Zhang

Chaiyachati

, et al. The limitations of poor broadband internet access for telemedicine use in rural america: an observational study. Ann Intern Med 2019; 171: 382–384.

32.

Rosenman

Tennekoon

Hill

. Measuring bias in self-reported data. International Journal of Behavioural and Healthcare Research 2011; 2: 320–332.

33.

Mackert

Mabry-Flynn

Champlin

, et al. Health literacy and health information technology adoption: the potential for a new digital divide. J Med Internet Res 2016; 18: e264.

34.

Mackert

Whitten

Garcia

. Interventions for low health literate audiences. J Comput Mediat Commun 2008; 13: 504–515.

35.

Rahimi

. Digital health and the elusive quest for cost savings. The Lancet Digital Health 2019; 1: e108–e109.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.12 MB