Patients’ and caregivers’ experiences of virtual care in a primary care setting during the COVID-19 pandemic: A patient-oriented research study

Abstract

Objective

This study explored patient and caregiver expectations and experiences of virtual primary care in Manitoba, Canada. This study focused on accessibility of care, acceptability and perceptions of quality from ‘users’ of primary healthcare services. Due to the rapid implementation of virtual primary care during the COVID-19 pandemic in Canada, patient/public input was largely bypassed.

Methods

A mixed method was conducted in collaboration with Patient and Caregiver Community Advisors. Data was obtained from 696 surveys and 9 focus groups (n = 41 patients and caregivers).

Results

Data suggest good acceptance of virtual visits, although considered a new experience despite almost exclusive use of the telephone. Participants preferred more input for choosing the type of visit but experienced less stress, time and inconvenience by using virtual care. There were mixed opinions of quality. More complex visits were associated with incomplete consultations and serve as one exemplar of the limitations due to lack of physical presence or contact. Unique communication skills were required to convey health concerns adequately and accurately. A more transactional approach was perceived from the lack of visual cues and the awkwardness associated with pauses during the phone conversation. Virtual care may be better used for certain circumstances but should encompass patient-centred decision making for when and how. Many expressed interests in video options; technology access and user ability are additional considerations for advancing virtual care.

Conclusions

The experiences and recommendations from patients and caregivers provide an important contribution to decision-making and integrating and sustaining quality virtual care for patient-centered healthcare service delivery. Keywords: Virtual care experiences, primary care, patient-oriented research, mixed methods, COVID-19.

Keywords

Primary health care COVID-19 virtual care patient-oriented patient experience patient care telemedicine < General telehealth < General

Introduction

Patients and caregivers experienced significant changes in the delivery of primary health care services during the COVID-19 pandemic. In adjusting to requirements for physical distancing, there was a rapid adoption of virtual care in Canadian primary care settings to minimize the need for in-person visits unless absolutely necessary.^1–3 Virtual care is defined as “any interaction between patients and/or members of their circle of care, occurring remotely, using any forms of communication or information technologies.”⁴ (p. 609). In Canada, virtual visits have generally been in the form of telephone (and some video) calls, with remuneration introduced during COVID to facilitate access or consultation with primary care providers. The pandemic has highlighted many gaps in the health care system, with insights based on patient and caregiver knowledge and direct experience.⁵ Virtual visits appear to be one example of a sustained change resulting from the pandemic; however, rapid responses to facilitate delivery of health services resulted in patient and public stakeholders’ input being largely bypassed.

In Canada, investigation into patient and caregivers’ experiences of virtual care has been increasing,^6–9 employing both quantitative and qualitative methodologies. Research has begun to examine barriers for accessing virtual care,⁷ the type of healthcare provider and individual preferences and satisfaction using virtual technology in primary care,⁸ and considerations around the quality and future use of virtual care in hospital settings.^6,9 Regarding patient's experiences, previous research found that patients appreciate virtual care accessibility,^8,9 they found it convenient, and are willing to continue to use virtual care after the COVID-19 pandemic.⁸ Patients have reported that the effectiveness of virtual care depends on the complexity of the medical visit and is a factor influencing an effective patient–provider relationship. Different cultural backgrounds and limited access to the internet in rural areas have been identified as barriers to virtual care use.⁷ Ethnically or racially diverse individuals were found to be less likely to recommend virtual visits compared to caucasian patients.^6,9 There have been few patient-oriented mixed-methods studies benefiting from the input of Patient and Caregiver Community Advisors who collaborate with the research team across all phases of the research, particularly to investigate virtual interactions between patients, caregivers and community-based primary care providers.

This patient-oriented research focused on the perspectives of patients and caregivers as the predominant ‘users’ of primary healthcare services who experienced virtual care visits during the COVID-19 pandemic. This research aimed to obtain patients’ and caregivers’ insights into accessibility, acceptability (highlighting both the benefits and challenges) and perceptions of virtual care, while exploring whether these outcomes were associated with specific demographics or types of visits. Given the importance of patient and caregiver expectations, experiences, equitable access, safety and a supportive environment for virtual care delivery,¹⁰ our goal was to ensure their voices are placed in the forefront of virtual care discussions in order to learn from their perspectives and inform recommendations that will promote sustainable, patient-centred virtual care.

Methods

Study design and setting

This study was part of a larger research initiative using an exploratory sequential mixed method across two study phases (Figure 1).¹¹ This approach optimized combining clinical health information, survey and qualitative data from providers, patients and caregivers for a multilevel perspective of virtual care from key ‘user’ groups, drawing from their experiences during the COVID-19 pandemic.¹² The first phase, reported elsewhere, characterized the use of virtual visits in primary care using data generated by the electronic medical records.¹³ Perspectives of health care providers about the drivers, barriers, and nuances of virtual care are currently being analyzed. In this paper, we focus on findings regarding the outcome, quality and future use of virtual visits as well as an in-depth understanding of the reasons for patients’ use of virtual care.

Figure 1.

Sequential mixed methods design for virtual visits and management of primary care in a pandemic environment.

The advancement of patient engagement (PE) in research, driven by the need to include patients as experts who have experiences and knowledge of living with a health condition and navigating the healthcare system, incorporates “meaningful and active collaboration in the governance, priority setting, and conduct of research and knowledge translation.”¹⁴(p. 5) Through this more participatory approach and actively engaging patients and caregivers, there is opportunity to formulate relevant research questions and improve study design.¹⁵ Additionally, engaging patients and caregivers as part of the team can have a substantial role in improving care delivery, and contributing to leadership decisions on policy and practice at the organizational and governance levels.¹⁶

The full study received approval from the University of Manitoba's Health Research Ethics Board (HS24197/H2020:377) and all participants provided consent prior to their participation. The GRIPP (Guidance for Reporting Involvement of Patients and Public) checklist was utilized in the production of this manuscript.¹⁷

Patient and caregiver community advisors

A Patient and Caregiver Community Advisory Committee, comprised of 4 members (two patients and two with additional responsibilities as caregivers) with lived experience of virtual care, was initiated early in the research to achieve collaborative and co-produced patient-informed research. In this study, lived experience means patients or caregivers who have attended one or more virtual care visits previous to the study. All advisory committee members had active roles throughout the study and participated in various research activities, including framing research questions, providing input on survey and interview questions, attending research team meetings, reviewing data, and translating results. One member of the Committee analyzed focus group (qualitative) transcripts. Feedback from Advisory members was facilitated by setting the frame for the discussion, creating an atmosphere to encourage open dialogue, and ensuring all members felt comfortable sharing their thoughts, perceptions, views, and ideas with all members of the research team.¹⁸ Advisory members were compensated for their time dedicated to the study.¹⁹

Participants

Adult patients and caregivers (18 years and older) across the province of Manitoba (with a population slightly over 1.3 million)²⁰ who had received at least one virtual visit either by telephone or video from a primary care provider (i.e., family physician, nurse practitioner, or pediatrician) between March 14, 2020 and June 30, 2020 were eligible to participate in the survey.¹⁶ A caregiver was an individual who self-identified as caring for another person who had a virtual visit.

We included patients and caregivers who had experienced one or more visits with a primary care provider between March 14, 2020 and June 30, 2020 and answered the survey. Patients and/or caregivers who completed the survey were invited to attend a focus group for more in-depth exploration of virtual care. Those who agreed to participate in focus groups were included in the focus group.

Data collection

Patient/caregiver survey

A brief survey consisting of 18 questions took approximately 10 minutes to complete and was designed with input from Patient and Caregiver Community Advisors to ensure relevance and appropriateness while minimizing participant burden. Survey content included constructs identified within the virtual care literature, and based on items obtained from several well-established instruments^21,22 including practical aspects of virtual care such as how the visit was conducted, reason for the visit, quality of communication, confidentiality, and other options considered as an alternative to virtual care. Additional questions inquired about transactional use of virtual care (i.e., perceived quality and impact on care), the outcomes of the visit, i.e., whether the visit was complete and if further follow-up was needed and perceptions of virtual care as an option for future visits (Supplementary Material, Appendix I). A complete visit occurs when the patient's concerns can be resolved during the visit, while an incomplete visit implies that further steps are needed to solve the patient's concern.

Patients/caregivers may have heard about the study in two ways: 1) After they had a visit (virtual or in-person) with a primary care provider participating in this study. The participating primary care providers were recruited through a newsletter and a local primary care research network.²³ In this case, the distribution of the survey was facilitated by using a secure web-based link to an online survey platform²⁴ or via an integrated module within the Electronic Medical Record (EMR) provided by Ocean (CognisantMD^TM).²⁵ All communication between the clinic EMR and patients and caregivers was encrypted. 2) Through social media and advertisements in community and broadsheet newspapers. In this case, a web-based link to the online survey platform was also publicly available through messaging on social media and advertisements. Importantly, all participants had access to the survey in the same way: a web-based link to the online survey platform.

Patient and caregiver focus groups

Patients and/or caregivers who completed the survey were invited to attend a focus group for more in-depth exploration of virtual care. Those who agreed to participate in focus groups were asked to complete a brief demographic questionnaire and a signed consent form. Focus groups were conducted using Zoom videoconferencing.²⁶ It has been suggested that the online environment could make it difficult for the moderator(s) to observe non-verbal cues or to manage the number and speed of overlapping discussions,²⁷ therefore we limited the session attendance to 5 or fewer participants.²⁸ The interview guide was co-designed by the Patient and Caregiver Community Advisors (Supplementary Material, Appendix II) with topics of discussion focusing on accessibility, acceptability, and perceptions of quality care. Participants were asked to expand on the benefits and challenges of using technology, the impact of a virtual visit on provider-patient/caregiver interactions and considerations for virtual patient care in the future. Focus group sessions sought to elicit examples from participants of what elements of virtual care worked and/or what needed to be changed.

Focus group sessions were facilitated by GH and AB, lasted approximately 60 min and participants were provided with a $50 gift card honorarium. Discussions were audio-recorded and transcribed to provide verbatim data and preserve the authenticity of the feedback and reduce recall bias when conducting the analysis. Participants were given an opportunity to debrief with the focus group facilitators afterwards if they felt it necessary to do so. This provided a brief opportunity for the participants to reflect at the conclusion of the focus group, ask questions of the researchers, and provide any additional data outside of the recording.

Analysis

Data collected from surveys and focus groups were anonymized and aggregated prior to analysis and therefore not associated with a specific patient or patient record. Patient surveys and focus group data were first analyzed independently and then merged to gain a comprehensive understanding of virtual care and its transactional use in primary care practices in Manitoba, Canada.

Quantitative analysis

A total of 700 surveys were collected from the combined data sources (Ocean, SurveyMonkey). Of the total collected, 4 surveys were blank, yielding a total sample of 696 surveys eligible for analysis.

The data (survey and focus groups) were analyzed for the primary purpose of describing participants’ perspectives regarding the outcomes, quality and use of virtual visits with few inferential goals. For the quantitative analyses, we consider Weisberg & Bowen (1977)²⁹ guidelines suggesting 400 observations are needed from an e-survey, accepting an error level of 5%. An online survey calculator²³suggests a sample of 385 considering the total population of Manitoba (1,342,000 as per Statistics Canada data),²⁰ 95% confidence and 5% margin of error. For the inferential statistics, findings from a large study conducted by Neves et al.³⁰ informed the proportions (.06–.51) used to calculate the sample size needed. Our sample far exceeded the minimum size needed to explore the specified associations; however, the larger sample size improved statistical power.

Statistical analysis

Descriptive statistics were calculated for rank order and multiple-choice responses to the questions about experiences with their most recent virtual care visit and input about future virtual care visits. Bivariate analysis including cross-tabulations with chi-square and Fisher's exact tests of independence was used to examine the seven reasons for the virtual visit (the independent variable) and whether each was associated with a specific visit outcome (the dependent variable). The target outcomes included whether the reason for the visit was associated with participants’ reporting the visit as complete or incomplete as well as associations with the quality of the visit reported as better, same or worse. Additional chi-square tests explored associations between the quality of the visit and the future use of virtual care.

Qualitative analysis

Two members of the research team (GH, AB) reviewed transcripts from nine audiotaped focus group sessions, consisting of a total of 41 patients/caregivers. All focus group transcripts were imported into the NVivo 12.0 software program for coding. A content analysis²⁸ approach was used by the researchers and patient partner (KM) to review transcripts first independently, and then altogether. The researchers analyzing the data first read the transcripts and then coded statements and segments. Initial or open coding was completed iteratively resulting in an approved code list created from input of the team members. The codes were then structured into categories by grouping the data, and finally collapsing categories into higher order themes. Each step was accompanied by regular discussions between those analyzing the data, drawing on each other's perspectives and insights to promote collaborative reflexivity³¹ as we sought consensus regarding the approach and findings. The themes represent the main categories to describe virtual primary care visits from the patients’ and caregivers’ experiences and were labeled to reflect the content of the sub-categories and further triangulated with findings in current literature.^32,33 Direct quotes used to exemplify summarized findings are listed in Table 6.

Results

Sociodemographic characteristics

The sociodemographic characteristics of survey and focus group participants are presented in Table 1. Survey participants were mostly between the ages of 61–70 (n = 200, 28.7%) and 51–60 (n = 131, 18.8%). The majority of survey participants identified as female (n = 535, 76.9%), declared English as their preferred language (n = 688, 98.8%) and reported good (n = 297, 42.7%) or excellent (n = 282, 40.5%) ability to use computers. Regarding community size, 54.6% (n = 380) of the survey participants lived in urban areas, and 56% (n = 390) spent less than 15 minutes traveling to their primary care clinic.

Table 1.

Socio-demographic characteristics of the survey and focus group participants.

Variable	Survey, n (%)	Focus groups, n (%)
	n = 696	n = 41
Age
0–30	54 (7.8)	1 (2.4)
31–40	97 (13.9)	2 (4.9)
41–50	98 (14.1)	2 (4.9)
51–60	131 (18.8)	6 (14.6)
61–70	200 (28.7)	16 (39.0)
71–80	85 (12.2)	12 (29.3)
Over 80	29 (4.2)	2 (4.9)
No response	2 (0.3)	–
Gender
Female	535 (76.9)	27 (65.9)
Male	156 (22.4)	14 (34.1)
Other response (non-binary, none of the above, prefer not to answer)	5 (0.7)	–
No response	–	–
English as the Preferred Language
Yes	688 (98.8)	41 (100.0)
No	6 (0.9)	–
No response	2 (0.3)
Ability to Use Computers
Excellent	282 (40.5)	34 (82.9)
Good	297 (42.7)	2 (4.9)
Fair	87 (12.5)	1 (2.4)
Poor	29 (4.2)	3 (7.3)
No response	1 (0.1)	1 (2.4)
Size of Community
Urban – 100,000 residents and over	380 (54.6)	36 (87.8)
Mid-sized – 10,000 to 99,999 residents	153 (22.0)	1 (2.4)
Rural – Under 10,000 residents	127 (18.2)	3 (7.3)
Don’t know	30 (4.3)	–
Prefer not to answer	4 (0.6)	–
No response	2 (0.3)	1 (2.4)
Travel Time to Clinic
Less than 5 min	64 (9.2)	16 (39.0)
5 to 15 min	326 (46.8)	20 (48.8)
16 to 30 min	188 (27.0)	4 (9.8)
31 to 60 min	96 (13.8)	–
61 to 120 min	16 (2.3)	–
More than 120 min	3 (0.4)	–
No response	3 (0.4)	1 (2.4)

Similar to the survey respondents’ demographic characteristics, most participants in the focus group were females in the 61–70 age group, declaring English as their preferred language, living in urban communities, and with good or excellent ability to use computers.

Patient survey

Survey completion rates and responses to each of the survey questions are summarized in Table 2. The telephone was reported as the most widely used method of communication (n = 633, 91%), compared to video (n = 4, 0.6%), and the use of video and telephone visits (n = 8, 1.1%). The most commonly reported reasons for having the virtual visit were for follow-up of test results (n = 245, 35.2%), follow-up to a previous appointment (n = 228, 32.8%), consultation about an ongoing or chronic health concern (n = 198, 28.4%), and to seek medical care or advice for a new health concern (n = 196, 28.2%). Saving time (n = 528, 75.9%) and more convenient access to care or services (n = 444, 63.8%) characterized the experience of virtual care.

Table 2.

Summary of patient survey responses to virtual care visits.

Survey response	Respondents, n (%)
How was your virtual visit conducted?	n = 696
Telephone	633 (91.0)
Video	4 (0.6)
Both	8 (1.1)
No response	51(7.3)
What was the reason for your visit? (select all that apply)	n = 696
Follow-up for test results	245 (35.2)
Follow-up to a previous appointment	228 (32.8)
Consult health care provider about an ongoing (chronic) health concern	198 (28.4)
Seek medical care or advice for a new health concern	196 (28.2)
Request for medication (prescription)	156 (22.4)
Discuss medication	107 (15.4)
Follow up after discharge from hospital	12 (1.7)
Other	46 (6.6)
In your experience, did your last virtual visit (select all that apply)	n = 696
Save you time (e.g., by avoiding travel or arranging care for dependents)	528 (75.9)
Make accessing care or services more convenient	444 (63.8)
Save you money (e.g., by not having to pay for transportation/parking, care for dependents, or having to take time off work)	274 (39.4)
None of the above	80 (11.5)
When you think about quality of care, how did having a virtual visit compare to having an in-person visit with your primary care provider?	n = 696
Better than having an in-person visit	79 (11.3)
Same as having an in-person visit	406 (58.3)
Worse than having an in-person visit	120 (17.2)
Not sure	78 (11.2)
No response	13 (1.9)
Did the quality of the sound or video negatively affect the virtual visit?	n = 696
Yes	52 (7.5)
No	602 (86.5)
Not sure	21 (3.0)
No response	21 (3.0)
What was the outcome of your virtual visit?	n = 696
The visit was completed, but I will follow-up if needed	398 (57.2)
The visit was completed, and no follow up was needed	223 (32.0)
The visit was incomplete, and I was advised to go in-person to the clinic	25 (3.6)
The visit was incomplete due to other reasons	26 (3.7)
The visit was incomplete due to technical issues	3 (0.4)
The visit was incomplete, and I was advised to go to emergency department	1 (0.1)
The visit was incomplete, and I was advised to go to urgent care clinic	0 (0.0)
No response	20 (2.9)
Did you find the virtual visit helpful?	n = 696
Yes	602 (86.5)
No	37 (5.3)
Not Sure	42 (6.0)
No response	15 (2.2)
Thinking about your last virtual visit with a primary care provider, what would you have done if you had not been able to have a virtual visit?	n = 696
Make an appointment to see my primary care provider in-person	538 (77.3)
Go to a walk-in clinic	67 (9.6)
Nothing, I would not have sought care at the time	35 (5.0)
Go to an Emergency Room (ER)	12 (1.7)
Called Health Links	3 (0.4)
Other	24 (3.4)
No response	17 (2.4)
Did you feel confident that your personal information would be kept secure and confidential when having the virtual visit?	n = 696
Yes	646 (92.8)
No	5 (0.7)
Not Sure	31 (4.4)
No response	14 (2.0)
In the future, would you like to have the option of consulting with a primary care provider using a phone or computer?	n = 696
Yes	449 (64.5)
Open to considering	172 (24.7)
No	41 (5.9)
No response	34 (4.9)
For what purposes would you use a virtual visit? (select all that apply)	n = 696
Receiving test results	604 (86.8)
Prescription renewal	603 (86.6)
Follow-up of a health problems	515 (74.0)
An urgent but minor health problem	278 (40.0)
New health problem	226 (32.5)
Annual or routine visit	143 (20.5)
Many issues to discuss	140 (20.1)
Pregnancy follow-up	22 (3.2)
Other	25 (3.6)
Which of the following options would you prefer to have in the future? (select all that apply)	n = 696
In person with my doctor or another health care provider in my usual clinic	572 (82.2)
Phone call with my doctor or another health care provider in my usual clinic	527 (75.7)
Videoconference with my doctor or another health care provider in my usual clinic	283 (40.7)
Text/email with my doctor or another health care provider in my usual clinic	262 (37.6)
Virtual service/health provider	205 (29.4)
Walk-in clinic	197 (28.3)
Telehealth	113 (16.2)
Health Links-Info Santé	79 (11.3)
None of these	4 (0.6)

Most virtual visits were considered completed (n = 621, 89.2%) and helpful (n = 602, 86.5%). A dichotomous dependent variable for incomplete visits was created based on five of the seven survey responses which identified a visit as incomplete. Among the respondents who had experienced at least one of the seven reasons for a virtual visit, only two reasons resulted in a statistically significant difference in the outcome of the visit. First, 12.78% of visits in which individuals were seeking medical care or advice for ‘a new health concern’ were reported as incomplete (χ2 = 7.850; df = 1; p = 0.005). Second, there is a statistically significant difference in the outcome among individuals who reported on a virtual visit for ‘follow up after discharge from hospital’ (χ2 = 4.823; df = 1; p = 0.028), (Table 3).

Table 3.

Bivariate association analysis: outcome of virtual visits.

Reason for medical visit	Outcome of virtual visit		Chi-square		Fisher’s Exact test
Reason for medical visit	Complete	Incomplete	χ²/df = 1	p-value	p-value
	n (%)	n (%)
Seek medical care or advice for a new health concern	157 (87.22)	23 (12.78)	7.850	0.005
Consult about an ongoing (chronic) health concern	168 (91.30)	16 (8.70)	0.179	0.672
Follow-up to a previous appointment	193 (91.04)	19 (8.96)	0.415	0.519
Follow up for test results	215 (93.07)	16 (6.93)	0.548	0.459
Request for medication (prescription)	137 (93.84)	9 (6.16)	0.850	0.356
Discuss medication	91 (91.00)	9 (9.00)	0.167	0.682
Follow up after discharge from hospital	9 (75.00)	3 (25.00)	n.a	n.a	0.063
Other	35 (87.5)	5 (12.5)	n.a	n.a	0.239

Note:

χ2 = Chi-square, df = degrees of freedom, p-value of χ2

n.a. For very small sample sizes (N < 5) we report the Fisher’s exact test p-value and not the chi-square test

When asked to compare a virtual visit to an in-person visit, respondents felt the quality of the primary care visit was the same (n = 406, 58.3%) or better (n = 79, 11.3%), however, 17.2% (n = 120) reported the quality as worse. A large number (n = 91, 13.1%) were not sure or did not respond to the question regarding quality of the visit. Chi-square tests were used to explore whether the reasons for the visit were related to the quality of the visit. A categorical variable for the quality of the visit was created based on three of the four survey responses which identified a virtual visit as being better, same, or worse compared to an in-person visit. There were no statistically significant differences between any of the reasons for the virtual visit and the quality of the visit (Table 4). A sensitivity analysis was conducted by considering different options for constructing the quality variable (i.e., worse and non-worse, better and worse). The same result was obtained after the sensitivity analysis (Tables 4a and 4b). These results suggest that the quality of the visit does not depend on the reasons for the visit.

Table 4.

Bivariate association analysis: quality of virtual visits.

Reason for medical visit	Quality of visit			Chi-square test		Fisher’s exact test
Reason for medical visit	Better	Same	Worse	χ² / df = 2	p-value	p-value
	n (%)	n (%)	n (%)
Seek medical care or advice for a new health concern	21 (13.29)	104 (65.82)	33 (20.89)	0.557	0.757
Consult about an ongoing (chronic) health concern	18 (11.11)	112 (69.14)	32 (19.75)	0.889	0.641
Follow-up to a previous appointment	24 (12.18)	130 (65.99)	43 (21.83)	1.694	0.429
Follow up for test results	29 (13.55)	144 (67.29)	41 (19.16)	0.050	0.975
Request for medication (prescription)	20 (16.00)	87 (69.60)	18 (14.40)	2.759	0.252
Discuss medication	10 (11.63)	61 (70.93)	15 (17.44)	0.459	0.795
Follow up after discharge from hospital	1 (10.00)	7 (70.00)	2 (20.00)	n.a.	n.a.	1.000
Other	7 (22.58)	21 (67.74)	3 (9.68)	n.a.	n.a.	0.165

Note:

χ2 = Chi-square, df = degrees of freedom, p-value of χ2

n.a. For very small sample sizes (N < 5) we report the Fisher’s exact test p-value and not the chi-square test

Table 4a.

Sensitivity analysis: Quality of virtual visits (worse, non-worse).

Reason for medical visit	Quality of visit		Chi-square test		Fisher’s exact test
Reason for medical visit	Worse	Non-worse	χ² / df = 1	p-value	p-value
	n (%)	n (%)
Seek medical care or advice for a new health concern	33 (18.23)	148 (81.77)	0.318	0.573
Consult about an ongoing (chronic) health concern	32 (17.39)	152 (82.61)	0.044	0.833
Follow-up to a previous appointment	43 (19.91)	173 (80.09)	2.092	0.148
Follow up for test results	41 (17.52)	193 (82.48)	0.101	0.750
Request for medication (prescription)	18 (12.24)	129 (87.76)	2.936	0.087
Discuss medication	15 (14.85)	86 (85.15)	0.357	0.550
Follow up after discharge from hospital	2 (16.67)	10 (83.33)	n.a.	n.a.	1.000
Other	3 (7.69)	36 (92.31)	n.a.	n.a.	0.127

Note:

χ2 = Chi-square, df = degrees of freedom, p-value of χ2

n.a. For very small sample sizes (N < 5) we report the Fisher’s exact test p-value and not the chi-square test

Table 4b.

Sensitivity analysis: Quality of virtual visits (better, worse)

Reason for medical visit	Quality of visit		Chi-square test		Fisher’s exact test
Reason for medical visit	Better	Worse	χ² / df = 1	p-value	p-value
	n (%)	n (%)
Seek medical care or advice for a new health concern	21 (38.89)	33 (61.11)	0.151	0.697
Consult about an ongoing (chronic) health concern	18 (36.00)	32 (64.00)	0.730	0.393
Follow-up to a previous appointment	24 (35.82)	43 (64.18)	1.200	0.273
Follow up for test results	29 (41.43)	41 (58.57)	0.005	0.940
Request for medication (prescription)	20 (52.63)	18 (47.37)	2.636	0.104
Discuss medication	10 (40.00)	15 (60.00)	0.014	0.906
Follow up after discharge from hospital	1 (33.33)	2 (66.67)	n.a.	n.a.	1.000
Other	7 (70.00)	3 (30.00)	n.a.	n.a.	0.094

Note:

χ2 = Chi-square, df = degrees of freedom, p-value of χ2

n.a. For very small sample sizes (N < 5) we report the Fisher’s exact test p-value and not the chi-square test

Finally, the majority of respondents were either in favour of virtual visits in the future (n = 449, 64.5%) or reported being open to considering the possibility (n = 172, 24.7%). When asked about the purpose for which the respondent would use virtual visits, the most frequently selected options were to receive test results (n = 604, 86.8%), prescription renewal (n = 603, 86.6%), and follow-up for a health problem (n = 515, 74%). Demographic variables were not found to predict willingness to use virtual visits in the future (logistic regression not presented here). Furthermore, we noted the skewed distribution of the outcome variable, with 95% of the sample willing to consider using virtual visits in the future. We then used a chi-square test of independence to explore the association between the quality of the visit (the independent variable) and willingness to use virtual visits in the future (the dependent variable). A categorical variable for the future use of virtual care was created based on the three survey responses (i.e., yes, open to considering, no). There was a statistically significant difference in the willingness to use virtual visits in the future among individuals who reported different outcomes for the quality of the visit (χ2 = 130.5; df = 2; p < 0.0001). Among the respondents who are willing (vs not willing) to use virtual visits in the future, 91.67% also reported the quality of the visit was either the same or better. Among those who are not willing to use virtual visits, 81.25% reported the quality of the visit as worse. Thus, the result suggests an association between the quality of the visit and the future use of virtual care (Table 5).

Table 5.

Bivariate association analysis: Future use of virtual visits.

Quality of visit	Future visits			Chi-square test
Quality of visit	No	Yes	Open to	χ² / df = 2	p-value
	n (%)	n (%)	n (%)	130.56	P < 0.0001
Not-worse	6 (18.75)	396 (91.67)	113 (71.07)
Worse	26 (81.25)	36 (8.33)	46 (28.93)

Notes:

χ2 = Chi-square, df = degrees of freedom, p-value of χ2.

Focus group interviews

A total of 5 themes and 16 subthemes with illustrative quotes were identified and presented in Table 6. The key themes are: 1) virtual care was a new experience, 2) mixed opinions on quality of the interaction, 3) virtual visits mitigated the stress of a trip to the clinic, 4) challenges of virtual visits were compounded by no physical presence or contact, and 5) virtual care is best utilized for particular instances.

Table 6.

Themes, subthemes, and additional supporting quotes from focus group sessions.

Theme #1: Virtual care was a new experience encountered by patients during the early phase of the COVID-19 pandemic
Subthemes	Quotes (Focus Group = FG Participant = P)
1a: Patients were limited in decision-making about the type of visit they would receive	“I called my doctor's office to make an appointment and the receptionist gave me such and such a date. And I was under the assumption that I had to go to the clinic like I've always gone to the clinic. And it just somehow came up. The receptionist said to make sure I’m at my phone within that hour time period. And she kind of threw me for a loop because I wasn't expecting that.” (FG 4, P5) “I wasn't given a choice. I was more or less told that the clinic was going virtual and that, appointments, until all of this blows over, are going to be virtual”. (FG 8, P1) “They called me a week before to tell me that the doctor reviews and decides which appointments will be in-person, and which will be on the phone. Ours was selected to be over the phone.” (FG 5, P5)
1b: Telephone visits were predominantly used to deliver virtual care	“I've only had by phone. I tried to send images. I like, I asked to be able to do that and was denied.” (FG 9, P2) “Strictly telephone. I have a follow up every month with my GP. I'll take my blood pressure at home. My GP will ask, well, what was your blood pressure, but what was your blood pressure like today? There was one other virtual visit where I had lab work done a couple of weeks earlier and my GP would pull up the results and give them to me over the phone.” (FG 3, P4)) “It was during the first stages of the pandemic in March of last year. And basically, the agreement was for both sides to be able to be on the safe side in terms of the risks of the exposure to the COVID. And my understanding at that time, she was also doing the consults at home over the phone. I don't think she's the technical person doing the Zoom meeting. But basically, we were just on the phone doing the consultation. And I think it was agreeable, basically. It was actually a reasonable request for both of us to be able to be on the safe side.” (FG 4, P4) “And I said, well, I would like to keep the appointment with my doctor, even if it is going to be over the telephone.” (FG 8, P1)
Theme #2: Experiences of the virtual visit were given mixed reviews about the quality of the interactions
2a: Telephone visits were conducted on time	“I don't think there was any wait time as long as you've set the appointment time and she would actually call me on the spot, like on the dot, like literally on the dot and like, when you're going there at the clinic, to the clinic, sometimes I would have to wait 30 minutes to an hour, but I think this is better because I don't have to wait for long.” (FG 4, P4) “One of the other fronts is that the appointments that I've had and that my wife has had generally take place right on time. It saves an awful lot of wait time in a waiting room, crowded with other people that you might be exposing yourself to other things, not necessarily COVID, but you know, colds and flus and that type of thing. I think that's sort of a spinoff than just they're more prompt, the doctors seemed to be on time and they're succinct. When you go through a description of the things you're trying to talk about, it's really on topic and things are resolved relatively quickly.” (FG 5, P1) “In terms of waiting, my doctor he has a lot of patients and I understand completely when I get there five minutes early or on time quite often, it's 20, 25 minutes before he actually comes into the room. I get put into the room fairly quickly, but then I sit there for 10, 15, 20 minutes. On the virtual, maybe it was five minutes after the specified time. I enjoyed that part of it, not sitting around. I said, okay, good. Okay, he’s having a good walk-in and he helped me right away.” (FG 6, P5)
2b: Less wait times to see a provider	“I think for me, it was easier to make an appointment and it was easier to book an appointment with my doctor. I never had any challenge with it, as compared to booking an appointment in person. It was easier for me to get an appointment with the doctor.” (FG 4, P4) “The virtual visit I got in within an hour of making contact where like, typically with my primary caregiver, I would say it's like a week booked out in advance.” (FG 5, P5) “The availability is just like when I was experiencing those symptoms and I was finding out, it's like, okay, I have talked to the doctor, and I went to book an appointment on a Friday afternoon. That's when I went to book it and I got an appointment for Monday morning and I was like, okay, I really like this. Yeah.” (FG 3, P3)
2c: Telephone visits felt rushed and impersonal	“Talking on the phone to your doctor, that's just, to me, it's just very impersonal.” (FG 1, P1) “I also feel that telephone calls, they feel rushed and it's like, there's a feeling that you need to fill in every moment with talking or else the doctor may end the call. So there doesn't seem to be any room for reflection on what you’re taking in, the information you're taking in or giving.” (FG 1, P4) “I do find that the telephone appointments are rushed. I'm met with indifference and the human contact is lost.” (FG 2 P2)
2d: Effective telephone communication skills were required for the patient and the provider	“I need to find the right vocabulary to describe my issues to the doctor, as well as the doctor needs to use more effective questioning because the telephone, it's not the same as a personal visit.” (FG1, P1) “You know, I found that in preparing myself for the telephone call the first time, I really had to think and put my thoughts together maybe a little better than I would have had I'd been in the office because I wanted to make sure that I expressed myself well and she understood what I was getting at.” (FG 8, P2) “I just want to make a comment that on these phone calls I find that I have to be much more prepared than for an in-clinic visit, simply because you don't have the ability to communicate visually and the doctor can't really see, respond to your reactions, which there's a big hole I find in communication that way.” (FG 1, P4)
2e: Privacy was not an issue with telephone visits	“As long as a doctor is using whatever is approved by, I guess the college, whoever needs to approve it, as long as they're doing what they need to do by the book, I'm not concerned. Same thing with sending an image, as long as it’s encrypted email or it’s encrypted portal I can use, then that's completely fine with me.” (FG 1, P4) “Well, you know, when you're talking about to security and privacy and that kind of thing, first of all, you never know who's standing on the side of the door when you're in with your doctor. I mean, there could be all sorts of security issues there. And also, I think back to times when I've gone in for a visit that's been upsetting and I've been crying and then it just, which is like, how do you get out of the clinic without everybody seeing you, right? So, there’s a, a huge advantage there in terms of privacy. I can talk to my doctor; I can cry my eyes out and nobody's going to know.” (FG 3, P3) “I mean, I think so. If it was, you know, so-called secure, why not?” (FG 2, P1)
Theme #3: Virtual Care helped to mitigate the stressors associated with a trip to the clinic
3a: Virtual Care is a convenient option for patients and caregivers	“I agree, the travel time, the economic aspect of it, the gas, the parking and your time in terms of going in person is, it's really, really very tasking in terms of traveling to the doctor in person. So it saves you a lot of time. It saves you a lot of effort in terms of going in person.” (FG 4. P4) “And quite frankly, from my perspective it was convenient to have that option. It saved me the time and trouble of having to go there. I guess, specific to the COVID situation, it eliminated the need for me to be exposed to the public for whatever I would have been exposed in my travel to the clinic.” (FG 8, P3) “I could see myself taking the virtual option more just because it is more convenient. (FG 9, P3) My doctor sent me a lab rec by email, and I printed it on my printer, and I took it to the lab. I didn't even have to, and then he phoned me with the results. So that's how that virtual circle went. And that was pretty convenient.” (FG 5, P4)
3b: Virtual Care eliminates the need to incur costs for parking	“You don't have to drive there; you don't have to pay for parking. In some places it's very difficult to find parking. You don't have to worry about getting a parking ticket because you went in and now, you're just waiting in the office for them, and your meters run out and you can’t just run out and pay for it. So, I mean, that's what I like about it.” (FG 1, P1) “I like the fact that I don't have to pay those outrageous prices for parking.” (FG 1, P3) “And that's $6 bus fare that I save. I appreciate that a little bit too. That's a coffee.” (FG 5, P2) “You don’t have to pay parking. You don't have to hunt and pay for parking spots.” (FG 5, P6) “I liked not having to drive down there and find parking and pay for. Now, I don't have to pay for parking.” (FG 7, P1) “I guess you could factor in the cost of driving there, the use of the car and all of that.” (FG 8, P3)
3c: Virtual Care is time saving and efficient	“Yeah, I really like it too. I think it's much more efficient, just like she said. I’ve had good experiences; I can explain to him exactly what the issue is. He listens, we kind of dialogue back and forth.” (FG 3, P2) “They're much faster obviously. They're suitable for pre-existing conditions. In other words, something flares up again and you need attention to that. Things that the doctor is familiar with, they're very useful and timesaving.” (FG 9, P3) “I think it's more efficient just by going through the phone, like going through you via the telephone conversation, consultation rather than in-person.” (FG 4, P4)
Theme #4. Challenges of Virtual Care were compounded by a lack of patient-to-provider contact
4a: Patients appreciate the physical contact with their providers	“I'll concur with the participant who spoke to her doctor regarding the physical. Mine was also a physical and I would have liked to have had that one-to-one situation where he would take a look at me and say, hey, you can, stand to lose a couple pounds or the blood pressure is a little high. Those tactile situations were missing.” (FG 6, P2) “I still like to reserve the point about the actual, the person in-person visit with a doctor, especially if I am seriously ill or in great pain and so on. I feel that I could explain better.” (FG 2, P4) “I thought it was really good that he said, no, I can't solve this problem. I think I have to see you. And I got an appointment within a couple of days. I was quite pleased with it. I thought well, that’ really good service.” (FG 3, P2)
4b: Patients cannot see the provider’s face or body language	“I just found that there would be some things that I would personally prefer a face-to-face with my doctor. I'm not saying that my experience was horrible and I'm not one to give perfect scores. That's just me.” (FG 3, P4) “Well, and also just to add to that, I would imagine that a doctor listens to their patient, but then there’s also looking at them for like visual clues that might add as to what’s going on, like, maybe it has to involve mental health there. And you might not get that through a phone call, maybe a little bit more through a video chat, but I would imagine that you’re obviously not picking up body language through a phone call and maybe that would add to a better diagnosis.” (FG 5, P5) “I think, and that what I see is about it’s the communication thing. And sometimes it’s by what people don’t say. And yet you see the body language, you see their demeanor, you see the care that they’re giving themselves, you get a lot from that, which if you’re doing just virtual, especially just telephone, you’re not going to see that.” (FG 1, P2)
4c: Technology is the gateway to modernized health care although barriers exist	“I'm just thinking there has to be a way younger people are more adept with all technology and as you age, I know the only thing constant in our life is change, but accessibility to technology is one thing, and we certainly don't have it in all the rural and Northern parts of Manitoba. And then there's the cost factor.” (FG 9, P1) “I think the medical community needs to be aware of their own privilege as people who are typically paid quite well and that there's a lot of … And I'm not speaking from my personal experience, but just from being aware in the work that I do, that I've had people who've had to borrow a phone to call me, or they don't have internet at home and the digital divide is a big issue.” (FG 1, P3) “I think we have to keep in mind that there's a pretty large population that even though they have a cell phone, they don't have any data or any minutes. And so they have to be in a Wi-Fi zone in order to take that call and to, so they could be hanging out in front of 7-Eleven for an hour waiting for that call using the free Wi-Fi there.” (FG 5, P4)
Theme #5: Virtual Care was best utilized for particular conditions (more that others)
5a: Prescription renewal and the delivery of lab results are well suited for Virtual Care	“Yeah, for routine things like in my situation, I don't need to go to my doctor's office. He knows that I need a requisition for the lab to get the blood work done. He gets the results. He phones me and he phones the pharmacy and changes the prescription. I don't need to go to his office twice to get that done. I could very easily get all that done over the phone.” (FG 9, P5) “I'm thinking of a very simple example of a quick virtual visit is you need your prescription renewed and you don't have to have blood pressure taken or anything. You just need a prescription renewed and that's it.” (FG 9, P1) “I just wanted to talk to my doctor about changing my medication. I called the office, and they informed me that, well, you don't have to come in for that. We can do a virtual visit. And I said, okay, great. I did the virtual visit, and it was all good.” (FG 6, P5)
5b: Virtual Care is not suitable when a physical assessment is needed	“I still like to reserve the point about the actual, the person in-person visit with a doctor, especially if I am seriously ill or in great pain and so on. I feel that I could explain better. The virtual visit is good when I'm healthy, I'm okay. Nothing much to report on and so on. That part of it. So it could be a combination, the in-person plus the virtual.” (FG 2, P4) “You don't have to go into the office when you just have to speak. You have to go into the office if it's something they have to see.” (FG 8, P2)
5c: Virtual Care is particularly challenging for new patients	“I think that rapport has to be built first in-person and the trust has to be there for the patient to be honest on the phone or a video call, as well as for the doctor to take the time to explain, make sure you understand what they call active listening so that you speak back what you think you heard. And it's the same thing doctor was trying to tell you, so there's no misunderstanding.” (FG 2, P3) “I wouldn't want my first visits to my doctor to be virtual. Only after you built up the relationship with the doctor, does it seem appropriate to then rely to certain extent on virtual care.” (FG 2, P1) “The reasons these interviews work is because he knows me. It's like if I had a bump on my arm and he seen it, and then we discussed it virtually, he would know what I'm talking about. I felt comfortable, but I don't think virtual visits can be done if you've never met or they haven't had an opportunity to view your wound or what it is.” (FG 4, P1)

Virtual care was a new experience

Virtual care was described as a new experience encountered by participants during the COVID-19 pandemic. The rapid response to COVID-19 required alternative options for healthcare consults; despite the growing awareness and use of various communication technologies, virtual visits occurred predominantly by telephone. Participants had few privacy or safety concerns when exchanging information by telephone. However, it was commonly reported that patients had limited choice about the type of visit (in-person or virtual) being scheduled: “…the receptionist gave me such and such a date…[and] said to make sure I’m at my phone within that hour time period.” (FG 4, P5) Another added: “…the doctor reviews and decides which appointments will be in-person, and which will be on the phone.” (FG 5, P5) Participants felt strongly that patients should be given the opportunity to choose the type of visit based on their preferences.

Mixed opinions on quality of the interaction

Although virtual visits were generally rated as acceptable, they were given mixed reviews when discussing more specific elements of the visit. While telephone visits were conducted on time and considered more accessible, with substantially less wait times to see providers, virtual visits at times felt rushed and impersonal. Many focus group participants commented that calls were on time, “…she would actually call me on the spot, like on the dot, like literally on the dot…” (FG 4, P4). Several also felt they could schedule an appointment time sooner with a virtual visit: “I went to book an appointment on a Friday afternoon… and I got an appointment for Monday morning, and I was like, okay, I really like this. Yeah.” (FG 3, P3)

Virtual visits mitigated the stress of a trip to the clinic

Virtual visits were thought to mitigate stressors participants commonly experienced with an in-person visit to the clinic, for example, the costs associated with travel: “the gas, the parking and your time in terms of going in person is, it's really, really very tasking in terms of traveling to the doctor in person.” (FG 4. P4). Also, participants noted the reduced health risk(s): “It saves an awful lot of wait time in a waiting room, crowded with other people that you might be exposing yourself to other things, not necessarily COVID, but you know, colds and flus” (FG 5, P1).

Challenges of virtual visits were compounded by no physical presence or contact

The timeliness and swift response noted as positives of virtual care were also met with some negatives. For example, participants mentioned feeling rushed and/or an impersonal undercurrent of virtual visits: “there's a feeling that you need to fill in every moment with talking or else the doctor may end the call. So there doesn't seem to be any room for reflection on what you’re taking in, the information you're taking in or giving.” (FG 1, P4) Participants also commented on how good telephone communication skills were necessary for themselves and their providers; finding the right vocabulary to describe their issues and responding more effectively to questions from the provider were essential and not as easily achieved on the telephone when compared to a face-to-face visit. “I have to be much more prepared than for an in-clinic visit, simply because you don't have the ability to communicate visually and the doctor can't really see, respond to your reactions, which there's a big hole I find in communication that way.” (FG 1, P4).

The challenges associated with virtual care appeared to intensify by the lack of patient-to-provider contact participants experienced with telephone visits. While there were situations in which virtual care seemed more suited, there were also a number of instances where it was considered not appropriate. “I still like to reserve the point about the actual, the person in-person visit with a doctor, especially if I am seriously ill or in great pain and so on. I feel that I could explain better.” (FG 2, P4). Similar notions were expressed regarding the quality of virtual care visits: “I would imagine that you’re obviously not picking up body language through a phone call and maybe that would add to a better diagnosis.” (FG 5, P5)

Access to technology, although seen as the gateway to modernized health care was often challenging and not equitable for all. Focus group participants recognized the overall limitations of technology, not specific to any particular age group, where access to virtual services related more to its affordability and geographical connectivity: “there's a pretty large population that even though they have a cell phone, they don't have any data or any minutes. And so they have to be in a Wi-Fi zone in order to take that call and so they could be hanging out in front of 7-Eleven for an hour waiting for that call using the free Wi-Fi there.” (FG 5, P4)

Virtual care is best utilized for particular instances

Overall, virtual care was thought to be best utilized for particular conditions or instances such as receiving laboratory results or prescription renewals. Preferences for in-person visits still prevailed over virtual care as participants felt there are a range of circumstances in which a physical exam or in-person conversation is most suitable. In addition, these circumstances needed to respond to patients’ needs, and therefore a prescriptive template or approach for determining appointment type would not be the most suitable. Virtual care was also thought to be less suited for patients who were new to providers since establishing a relationship is best achieved in-person as opposed to over the telephone, or virtually. “I wouldn't want my first visit to my doctor to be virtual. Only after you built up the relationship with the doctor, does it seem appropriate to then rely to certain extent on virtual care.” (FG 2, P1) A second participant added: “The reasons these interviews work is because [the physician] knows me. (FG 4, P1)

Discussion

The experiences shared by the participants through focus group interviews and patient surveys enabled an in-depth investigation of virtual care. The reasons for having a virtual visit were described, as were the benefits and challenges experienced by the participants. The findings converged on three topic areas: 1) the use or basic logistics around virtual visits, 2) the quality of the virtual visit experience, and 3) considerations for future use.

Logistics

A key theme emerging from the focus group discussion was that virtual care was a new experience even though the telephone was predominately used for a virtual visit with primary care providers. It is likely the rapid introduction of virtual care required using the most accessible option for the largest number of people, and hence the use of telephone.

All participants similarly identified the most common reasons for the virtual visit; however, the focus group discussions allowed a more in-depth exploration of the process for obtaining an appointment, decision making around the type of visit and patient preferences. In fact, a key message from the participants was their limited opportunity to decide on the type of visit and felt strongly that patients should have a role in choosing the type of visit based on their preferences. While most preferred in-person visits with their regular health care provider, the option to use virtual care in future should be tailored to patients’ needs, preferences and expectations. Further, study participants recommended providers be equipped with agreed upon criteria or practice standards for what constitutes an appropriate virtual visit or necessitates an in-person visit. In turn, they articulated that patients should be educated on general criteria, so they have a better sense of what to expect and can be better equipped to request a particular kind of visit. The participants conveyed a bit of a learning curve occurring during the initial use of virtual care and suggested sustained use may require more discussion with patients to promote inclusive decision-making regarding the type of visit (virtual or in-person). Initially, more time may also be needed to foster an empathic connection, guide patients through a virtual interaction and ensure they are satisfied with the visit.³⁴

Quality

A large number of participants in both the survey and focus groups responded positively to the concept of virtual care. It was clear that virtual visits were convenient, saved the user (and/or a caregiver) the time and stress of having to travel to a clinic, or having to deal with parking challenges, to name a few. In the experience of the participants, there was less wait time to see a provider and the visit was generally conducted on time. The cost savings, convenience, and accessibility of virtual visits that patients reported as beneficial have also been found elsewhere in the literature.^8,9,35–37

Most virtual visits were considered completed and helpful, and participants indicated their capacity to determine follow up. The two scenarios most associated with an incomplete visit were for ‘a new health concern’ or for ‘follow up after discharge from hospital.’ Intuitively, these two reasons for virtual visits demand more personal connection, in-person conversation, and physical examination. In further exploring the limitations of the focus group participants, it was evident that they felt complex situations demanded more personal connection, in-person conversation, or physical examination.

When asked about the overall of quality of the primary care visit when conducted virtually, it was generally reported as the same or better however there was also a notable number of respondents either reporting the quality as worse or simply unsure or not responding to this question. There were no statistically significant differences between any of the reasons for the virtual visit and the quality of the visit as reported in the surveys. The focus group participants concluded that no matter the reason for the consultation, a virtual interaction with the primary care provider was simply not the same as having an in-person visit. From this more in-depth exploration of quality, more nuanced descriptions of expectations and the quality of communication emerged, resulting in mixed reviews of quality. Participants articulated the lack of patient-provider contact as being a factor when trying to address more complex health issues; the missing physical assessment, whether visual or tactile, left some unease among the participants. Further, they felt a sense of responsibility for adequately and accurately conveying all aspects of their health concern. This was also accompanied by uncertainty and the visual cues or facial expressions that gave them a sense they were understood along with the provider's reaction to the information provided. At the same time, many participants felt the phone did not allow the ‘space’ for silence and, as per usual telephone conversations, if there was a pause then it was an indication of the conversation drawing to a close. In other words, there was little opportunity to pause, reflect or think through how an issue was being presented. This was one example of how a telephone visit felt rushed and took a more transactional approach, and thus feeling more impersonal. Similar to other studies, the barriers of virtual care included a loss of physical interactions with providers,^8,38,39 the inability to use visual cues,^35,37,40 and lack of personal connection.⁷

Participants believed that being more prepared for future virtual visits would improve their overall experiences of virtual care. To them, keeping better track of their health history, creating a list of discussion items before a telephone visit, and using vocabulary that best describes their issues were recommended. They recommended the provider have more education and access to resources to help them implement more user-friendly language and better communication with patients. These drawbacks of virtual care need to be considered in the decision about what kind of visit is most appropriate. Improving communication skills is a responsibility for both providers and patients.

Future use

Participants would either like the option or were open to considering phone or computer consultations in the future, however positive uptake of future virtual care was associated with the perceptions of quality in past visits. The participants suggested several indications for when virtual care may be useful in future, such as for prescription renewal, follow-up of a health problem or for test results. Respondents were also very aware of how frequently a consultation would be limited by lack of physical assessment. They also felt virtual care would not be suitable when seeing a health care provider for the first time as a relationship needs to be developed to support the communication and consultation processes.

Overall, participants acknowledged the increasing role of technology mediating health care interactions but still maintained there needs to be a patient-centred approach for deciding when and how it should be used. In essence, there was a strong preference for a hybrid model, with both virtual and in-person care options being offered in the future. On the one hand, virtual visits were limited to some people who did not have access due to unreliable internet connectivity, cell service or even limited telephone options. The health equity barriers associated with the use of technology have been pointed out in other studies.^7,9,41–44 On the other hand, and for those who had greater capacity to engage through technology, many participants expressed willingness to have more video visits. While prior research has shown video consultations to be acceptable, safe, and effective for a variety of health conditions,^45–52 a very limited number of participants in this study were given the option of video visits. Video visits could enable participants to have more visual interactions with providers⁶ and perhaps satisfy the need for more human contact, which was thought to be currently missing. Of note, the recommendation was made by study participants with greater representation from an older age category. In fact, 83.2% of the participants declared to have excellent or good ability using computers, thus dispelling the notion that video visits are preferred by younger patients.^6,53 These findings suggest careful consideration of access to technology and ability to use computers.

Strengths and limitations of the study

A highlight for the research team and a strength of this research was the collaboration with Patient and Caregiver Community Advisors. Despite the inability to attend in-person meetings due to COVID-19 pandemic restrictions, members participated using videoconferencing to provide important feedback throughout the study. This was pivotal to determining ease of use/readability of the survey instrument and enhanced its face validity. However, the survey was not pilot tested further, and this may be a limitation.

The community advisors’ engagement and facilitation in disseminating study findings were some of the highlights of this collaboration, which resulted in authorship on abstracts as well as attendance and presentations at virtual conferences. Importantly, our research corroborates a similar and important message as others^10,54 in that, patients should be the focal point of health system decision-making and more engaged in consultations about the future application of virtual care.

The novel use of the electronic medical record to cue automated distribution of patient surveys in Manitoba helped minimize the burden of research participation on clinical practices. However, the reliance on technology for distribution of the surveys may limit the findings by missing input from those who do not have adequate access to the internet or technology, have difficulty navigating an online survey or do not have a regular primary care provider. The broader invitation to participate in the survey using community newspapers and a mixed method approach are strengths of this study.

Certain groups, including patients with low income, limited English proficiency, or sensory impairments are under-represented in the patient survey responses, and may also be a limitation given that the focus groups were conducted using Zoom. Although the survey and focus group samples were mixed, there were more female than male participants. Also, the number of responses in some categories of the quantitative analysis was quite low and require further investigation. Inclusion of more male-affiliated experiences and those of a younger population may have resulted in different views based on different access and interactional experiences. Inclusion of more regional representation, including more participation of Northern and remotely located communities and participants would have made the results more generalizable to the overall population of patients receiving virtual care.

Finally, we recognize the rapid pace with which technology has been adopted for communicating during and since the pandemic environment; these results represent a unique point in time but have imparted important points about patient-centred use of technology for virtual primary care visits they develop and advance toward the future.

Conclusion

Virtual care enabled patients to access primary health care services while maintaining physical distance; however, the rapid up-scale of virtual care during the COVID-19 pandemic left many questions about the impact on patients and their interactions with primary care providers. Drawing from the experiences of patients and caregivers, as the predominant ‘users’ of the health care system, we gain much needed insight into accessibility of care, acceptability (highlighting both benefits and challenges) and perceptions of quality of virtual care. Findings indicated that virtual care did have a place in the health system, virtual visits were better suited to address some issues over others, and patients should be given a choice based on their preferences together in consultation with their provider. The recommendations from study participants are an important contribution to decision-making, integrating and sustaining quality virtual care for long-term use. This patient-oriented research highlights the importance of ensuring that the voices of patients and caregivers are at the forefront of virtual care discussions and guideline development for patient-centered healthcare service delivery.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241232949 - Supplemental material for Patients’ and caregivers’ experiences of virtual care in a primary care setting during the COVID-19 pandemic: A patient-oriented research study

Supplemental material, sj-docx-1-dhj-10.1177_20552076241232949 for Patients’ and caregivers’ experiences of virtual care in a primary care setting during the COVID-19 pandemic: A patient-oriented research study by Gayle Halas, Alanna Baldwin, Kerri Mackay, Ernesto Cardenas, Lisa LaBine, Phyllis Cherrett, Linda Abraham, Vivianne Fogarty, Alexander Singer, Alan Katz and Sarah Kirby in DIGITAL HEALTH

Supplemental Material

sj-docx-2-dhj-10.1177_20552076241232949 - Supplemental material for Patients’ and caregivers’ experiences of virtual care in a primary care setting during the COVID-19 pandemic: A patient-oriented research study

Supplemental material, sj-docx-2-dhj-10.1177_20552076241232949 for Patients’ and caregivers’ experiences of virtual care in a primary care setting during the COVID-19 pandemic: A patient-oriented research study by Gayle Halas, Alanna Baldwin, Kerri Mackay, Ernesto Cardenas, Lisa LaBine, Phyllis Cherrett, Linda Abraham, Vivianne Fogarty, Alexander Singer, Alan Katz and Sarah Kirby in DIGITAL HEALTH

Footnotes

Acknowledgements

The authors would like to acknowledge the support of Sabrina T. Wong, Siddhesh Talpade, Elissa M Abrams, Jose Francois and Eric Bohm.

Contributorship

GH and AS conceptualized the study. GH, AS, AK, AB, LL, SK were involved in protocol development, and gaining ethical approval. AB and GH were involved in patient recruitment, data collection and analyses with additional assistance from KM, and LL. AB wrote the first draft of the manuscript. GH, KM, EC and LL completed subsequent revisions and additional analyses. All authors reviewed and edited the manuscript and approved the final version.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

University of Manitoba's Health Research Ethics Board (HS24197/H2020:377)

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a Manitoba COVID-19 Rapid Response Research Grant awarded to Dr Gayle Halas by Research Manitoba.

Guarantor

ORCID iD

Ernesto Cardenas

Supplemental material

Supplemental material for this article is available online.

References

Bhatia

Chu

Pang

, et al. Virtual care use before and during the COVID-19 pandemic: a repeated cross-sectional study. CMAJ Open 2021; 9: E107–E114.

Alexander

Tajanlangit

Heyward

, et al. Use and content of primary care office-based vs telemedicine care visits during the COVID-19 pandemic in the US. JAMA Netw Open 2020; 3: e2021476.

Webster

. Virtual health care in the era of COVID-19. Lancet 2020; 395: 1180–1181.

Shaw

Jamieson

Agarwal

, et al. Virtual care policy recommendations for patient-centred primary care: findings of a consensus policy dialogue using a nominal group technique. J Telemed Telecare 2018; 24: 608–615.

Immonen

. The views of patients and the public should be included in policy responses to COVID-19. BMJ Opinion 2020; 30: 2020.

Chu

Nayyar

Bhattacharyya

, et al. Patient and provider experiences with virtual care in a large, ambulatory care hospital in Ontario, Canada during the COVID-19 pandemic: observational study. J Med Internet Res 2022; 24: e38604.

Chan-Nguyen

O’Riordan

Morin

, et al. Patient and caregiver perspectives on virtual care: a patient-oriented qualitative study. CMAJ Open 2022; 10: E165–E172.

Zhao

Elshoni

O’Brien

, et al. Patient and provider experiences with virtual care during the COVID-19 pandemic: a mixed methods study. Patient Exp J 2022; 9: 62–70.

Nayyar

Pendrith

Kishimoto

, et al. Quality of virtual care for ambulatory care sensitive conditions: patient and provider experiences. Int J Med Inform 2022; 165: 104812.

10.

Canadian Medical Association. Virtual Care: Recommendations for scaling up virtual medical services. Report of the virtual care task force. Ottawa, ON. Published 2020. https://www.cma.ca/

11.

Ivankova

Creswell

Stick

. Using mixed-methods sequential explanatory design: from theory to practice. Field Methods 2006; 18: 3–20.

12.

Halas

Singer

Katz

, et al. Examining virtual visit use during a pandemic and perspectives of primary care providers, patients and caregivers: a mixed-methods research protocol. BMJ Open 2022; 12: e062807.

13.

Singer

Kosowan

LaBine

, et al. Characterizing the use of virtual care in primary care settings during the COVID-19 pandemic: a retrospective cohort study. BMC Primary Care 2022; 23: 320.

14.

Canadian Institutes of Health Research. Strategy for Patient-Oriented Research -Patient Engagement Framework. (2014). Retrieved From.; 2014.

15.

Haywood

Lyddiatt

Brace-McDonnell

, et al. Establishing the values for patient engagement (PE) in health-related quality of life (HRQoL) research: an international, multiple-stakeholder perspective. Qual Life Res 2017; 26: 1393–1404.

16.

Baker

GR.

Evidence boost: a review of research highlighting how patient engagement contributes to improved care. Ottawa: Canadian Foundation for Healthcare Improvement. Published online 2014.

17.

Staniszewska

Brett

Mockford

,et al. The GRIPP checklist: strengthening the quality of patient and public involvement reporting in research. Int J Technol Assess Health Care 2011; 27: 391–399.

18.

Creswell

Qualitative Inquiry and Research Design: Choosing among Five Approaches. Second. Thousand Oaks, CA: Sage Publications Sage CA; 2007.

19.

Snow

Tweedie

Pederson

. Heard and valued: the development of a model to meaningfully engage marginalized populations in health services planning. BMC Health Serv Res 2018; 18: 181.

20.

Women’s College Hospital Institute for Health Systems Solutions and Virtual Care. Enhanced Access to Primary Care: Project Evaluation Final Report . Toronto, Ontario: Ontario Telemedicine Network, Ministry of Health and Long-Term Care; 2019. https://otn.ca/wp-content/uploads/2019/08/eapc-evaluation-report.pdf

21.

Canadian Medical Association. What Canadians Think About Virtual Care. June 1, 2020. Published online. https://www.cma.ca/sites/default/files/pdf/virtual-care/cma-virtual-care-public-poll-june-2020-e.pdf

22.

MaPCReN. Manitoba Primary Care Research Network. https://mapcren.ca/

23.

SurveyMonkey. Virtual visits and management of primary care in a pandemic environment. https://www.surveymonkey.com/

24.

Cognisant MD. Available at https://www.cognisantmd.com/ocean-studies/.

25.

Zoom Video Communications Inc. Available at: https://zoom.us/

26.

Moore

McKee

McCoughlin

. Online focus groups and qualitative research in the social sciences: their merits and limitations in a study of housing and youth. People, Place and Policy Onlin 2015; 9: 17–28.

27.

Lobe

Morgan

. Assessing the effectiveness of video-based interviewing: a systematic comparison of video-conferencing based dyadic interviews and focus groups. Int J Soc Res Methodol 2021; 24: 301–312.

28.

Weisberg

Krosnick

Bowen

. An Introduction to Survey Research and Data Analysis. Scott: Foresman & Co, 1989.

29.

Statistics Canada. Census profile, 2021 census of population. Statistics Canada Catalogue no 98-316-X2021001. Published online 2022.

30.

Neves

van Dael

O’Brien

, et al. Use and impact of virtual primary care on quality and safety: The public’s perspectives during the COVID-19 pandemic. J Telemed Telecare 2024; 30: 393–401.

31.

Olmos-Vega

Stalmeijer

Varpio

, et al. A practical guide to reflexivity in qualitative research: AMEE guide No. 149. Med Teach 2023; 45: 241–251.

32.

Hsieh

Shannon

. Three approaches to qualitative content analysis. Qual Health Res 2005; 15: 1277–1288.

33.

Elo

Kyngäs

. The qualitative content analysis process. J Adv Nurs 2008; 62: 107–115.

34.

Wong

Bhyat

Srivastava

, et al. Patient care during the COVID-19 pandemic: use of virtual care. J Med Internet Res 2021; 23: e20621.

35.

Hammersley

Donaghy

Parker

, et al. Comparing the content and quality of video, telephone, and face-to-face consultations: a non-randomised, quasi-experimental, exploratory study in UK primary care. Br J Gener Pract 2019; 69: e595–e604.

36.

Donaghy

Atherton

Hammersley

, et al. Acceptability, benefits, and challenges of video consulting: a qualitative study in primary care. Br J Gener Pract 2019; 69: e586–e594.

37.

McKinstry

Watson

Pinnock

, et al. Telephone consulting in primary care: a triangulated qualitative study of patients and providers. Br J Gener Pract 2009; 59: e209–e218.

38.

Hyman

. The disappearance of the primary care physical examination—losing touch. JAMA Intern Med 2020; 180: 1417.

39.

Seuren

Wherton

Greenhalgh

, et al. Physical examinations via video for patients with heart failure: qualitative study using conversation analysis. J Med Internet Res 2020; 22: e16694.

40.

McKinstry

Hammersley

Burton

, et al. The quality, safety and content of telephone and face-to-face consultations: a comparative study. Qual Saf Health Care 2010; 19: 298–303.

41.

Hardcastle

Ogbogu

. Virtual care: enhancing access or harming care? Healthc Manage Forum 2020; 33: 288–292.

42.

Crawford

Serhal

. Digital health equity and COVID-19: the innovation curve cannot reinforce the social gradient of health. J Med Internet Res 2020; 22: e19361.

43.

Cukier

‘Left out of this equation’: Why virtual health care is leaving some ontarians behind. TVO Org. 2020 Nov 17. https://www.tvo.org/article/left-out-of-this-equation-why-virtual-health-care-is-leaving-some-ontarians-behind

44.

Kilvert

Wilmot

Davies

, et al.

Virtual consultations: are we missing anything?

Pract Diabetes 2020; 37: 143–146.

45.

Freeman

Duke

Harris

. Behavioral health care for adolescents with poorly controlled diabetes via skype: does working alliance remain intact? J Diabetes Sci Technol 2013; 7: 727–735.

46.

Hansen

Perrild

Koefoed

, et al. Video consultations as add-on to standard care among patients with type 2 diabetes not responding to standard regimens: a randomized controlled trial. Eur J Endocrinol 2017; 176: 727–736.

47.

Katz

Pirabhahar

Williamson

, et al. Iconnect CKD - virtual medical consulting: a web-based chronic kidney disease, hypertension and diabetes integrated care program. Nephrology 2018; 23: 646–652.

48.

Nield

Hoo

GWS

. Real-Time telehealth for COPD self-management using Skype^TM. COPD: J Chronic Obstruct Pulmon Dis 2012; 9: 611–619.

49.

Bennell

Nelligan

Dobson

, et al. Effectiveness of an internet-delivered exercise and pain-coping skills training intervention for persons with chronic knee pain. Ann Intern Med 2017; 166: 453.

50.

Müller

Alstadhaug

Bekkelund

. Acceptability, feasibility, and cost of telemedicine for nonacute headaches: a randomized study comparing video and traditional consultations. J Med Internet Res 2016; 18: e140.

51.

Westra

Niessen

. Implementing real-time video consultation in plastic surgery. Aesthetic Plast Surg 2015; 39: 783–790.

52.

Viers

Lightner

Rivera

, et al. Efficiency, satisfaction, and costs for remote video visits following radical prostatectomy: a randomized controlled trial. Eur Urol 2015; 68: 729–735.

53.

Schifeling

Shanbhag

Johnson

, et al. Disparities in video and telephone visits among older adults during the COVID-19 pandemic: cross-sectional analysis. JMIR Aging 2020; 3: e23176.

54.

Bhatia

Jamieson

Shaw

, et al. Canada’s Virtual Care Revolution: A Framework for Success. Ontario, Canada: CD Howe Institute Toronto, 2020.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

0.02 MB