Sage Journals: Discover world-class research

Abstract

Objective

The COVID-19 pandemic accelerated the use of virtual health care in Canada. Nova Scotia launched VirtualCareNS—a hybrid model integrating virtual and in-person primary care—to address access gaps. This rapid evaluation assesses its feasibility, preliminary economic outcomes, and stakeholder experiences.

Methods

A mixed-methods rapid evaluation design was employed, incorporating utilization and economic analyses. Surveys were completed by key informants, including users (N = 74,159), non-users (N = 3130), and implementation team members (N = 31). Interviews were conducted with providers (N = 8), implementation team members (N = 11), and platform users (N = 28).

Results

Over 101,000 virtual primary care visits were completed, with 76,054 unique profiles created. The cost per consultation was $123 (95% CI: $99–$149), and the net cost-savings per consultation was $85 (95% CI: $62–$111), primarily driven by reduced travel time and avoidance of emergency department and walk-in clinic visits. Patient satisfaction was high (91%), and providers reported improved access, especially for patients without a regular primary care provider.

Conclusion

Our rapid evaluation suggests that a hybrid model of virtual and in-person care can effectively address non-urgent care needs, generate cost savings, and improve access for both unattached and attached patients. VirtualCareNS benefits from dedicated leadership structures and integration with Nova Scotia's broader primary care services, positioning it as a scalable and sustainable approach to primary care delivery. Ongoing refinement—guided by user, provider, and implementation feedback—will be critical to realizing its full potential within the publicly funded health system.

Keywords

Virtual care primary care digital care platform rapid evaluation mixed-methods

Background and rationale

The onset of the COVID-19 pandemic marked a pivotal moment in the evolution of healthcare delivery, with public health measures prompting the rapid adoption and expansion of virtual care services for health systems around the globe,^1–3 including impacts on the delivery of primary care services. Healthcare policies in high-income countries transformed primary care—a model of care “that supports first contact, accessible, continuous, comprehensive, and coordinated first-person care”⁴—through the integration of virtual care services.⁵ Although many high-income countries share certain policy drivers, substantial variations in digital infrastructure, reimbursement mechanisms, and regulatory frameworks can significantly affect the pace of virtual care adoption. Virtual care (also known as telemedicine, telehealth, or virtual healthcare) is “any interaction between patients and/or members of their circle of care, occurring remotely, using any forms of communication or information technologies, with the aim of facilitating or maximizing the quality and effectiveness of patient care.”⁶ This shift in policy and care delivery was integral in maintaining care continuity and adapting primary care services to meet patients’ needs during crisis conditions that defined the COVID-19 pandemic.^1,2

Canada has historically trailed behind its international counterparts in adopting digital health technologies including virtual care. However, the expansion of virtual care models necessitated during the COVID-19 pandemic resulted in improvements that have been maintained past the immediate COVID-19 crisis.^1–3,5,7 As of March 2022, approximately half of Canadians reported being offered virtual visits alongside traditional non-virtual modalities.⁷ Despite a decrease in the proportion of virtual visits compared to 2020, virtual care use exceeds pre-pandemic levels.⁷ This momentum presents a valuable opportunity to explore virtual care as a modality to address the primary care gaps being experienced across Canada.

Existing evidence and knowledge base

A growing body of evidence supports positive outcomes attributed to virtual primary care services. Virtual primary care can help reduce reliance on emergency departments (EDs),⁸ improve timely access to care, and contribute to shorter wait times.^9,10 A systematic review by Campbell et al.¹¹ exploring the impact of virtual care on the quality of adult primary care services found that primary care provided virtually is equally, if not more, effective for treating psychosocial or behavioral conditions than face-to-face primary care and may positively impact the timeliness and efficiency of care. However, research has also highlighted the potential negative effects of virtual care on care delivery and equity. Veinot et al.¹² noted that informatics interventions, including virtual care, can unintentionally exacerbate inequalities, particularly when access to technology or digital literacy is limited. Similarly, Latulippe et al.¹³ reviewed the relationship between eHealth and social health inequalities, identifying that disparities in digital access can deepen existing inequities. More recently, scholars examined digitalized primary care in the United Kingdom and found that minoritized ethnic communities often face barriers to accessing virtual care, further underscoring the importance of addressing equity in the adoption and scaling of virtual health services.¹⁴ Evidence indicates that virtual primary care provision, compared to traditional models, may be challenged by reduced thoroughness in physical examination, difficulties in patient-provider communication and relationship-building, as well as increased distraction and privacy concerns for both patients and providers, potentially compromising patient-centered and quality care delivery.^15,16 The effectiveness of virtual care depends heavily on reliable internet access, digital literacy, and clinicians’ proficiency with technology.¹⁷ Cherabuddi and colleagues¹⁸ highlighted disparities in virtual care usage, noting that younger patients (under 65 years of age), white patients, and female-identifying individuals are more likely to access or utilize virtual care. To promote equitable access, virtual care must be thoughtfully designed and implemented, addressing barriers faced by target populations and prioritizing inclusivity across all demographics.¹⁹

Emerging findings on hybrid models, which integrate both virtual and in-person care, offer a promising strategy for alleviating these challenges. Hybrid approaches can minimize the risks of missed physical exam details while retaining the accessibility benefits of telehealth.^20–22 By redirecting patients to in-person services when virtual care is inadequate or inappropriate, hybrid models can strengthen diagnostic accuracy and continuity of care, particularly in regions with longstanding health human resource constraints.

Implementation context

Within Nova Scotia, Nova Scotia Health (NSH) serves as the provincial health authority and primary care provider. Prior to the pandemic, NSH had begun introducing innovative solutions such as urgent treatment centers and mobile primary care clinics to assist patients on the Need a Family Practice Registry (NFPR) who do not have a dedicated primary care provider, referred to as unattached patients. The pandemic accelerated the readiness for digital health solutions, prompting NSH's Primary Health Care (PHC) program to introduce a novel, system-wide hybrid virtual care model: VirtualCareNS. This large-scale implementation went beyond typical local quality improvement initiatives, aiming to integrate telehealth technology into routine primary care across a diverse set of rural and urban communities. As such, it addressed not only immediate pandemic-era needs but also broader, persistent obstacles to primary care access in the province.

An essential feature of VirtualCareNS has been the establishment of a provincial virtual care program office and dedicated leadership roles within NSH. These structures were created to coordinate stakeholder engagement, oversee technological integration, and guide the province-wide rollout. This governance model ensured that VirtualCareNS had clear leadership channels and aligned strategies with other NSH initiatives in primary care, urgent treatment centres, and mobile clinics.

The Canada Health Act²³ governs Canada's healthcare system, mandating universal coverage for medically necessary hospital and physician services and establishing federal conditions, through the Canada Health Transfer, that provinces and territories must meet. In Nova Scotia, this ensures that patients with valid health cards are not charged for VirtualCareNS services. Apart from vendor-related costs, the healthcare costs reported in this study therefore closely approximate actual costs, rather than prices or charges. Within this framework, provincial fee codes for virtual services, licensing flexibilities for out-of-province providers, and dedicated funding allocations were pivotal in enabling the large-scale rollout of VirtualCareNS.

Intervention description: VirtualCareNS

VirtualCareNS was developed through a collaborative effort involving NSH patient advisors, primary care providers, and key health system partners, led by the NSH PHC program. The co-design process ensured that the service was well integrated into NSH's suite of primary care offerings, fostered rapid learning during implementation, and established a foundation for a sustainable, long-term virtual care program. The initiative prioritized addressing barriers to virtual care access while supporting patients requiring in-person services. Figure 1 outlines important considerations to ensure alignment with existing services and processes within PHC including referral processes for in-person follow-up care to primary care clinics and mobile clinics located across the province. The secure Maple platform²⁴ was selected after an extensive review of virtual solutions, meeting requirements for data security, clinical care standards, and stakeholder preferences.

Figure 1.

VirtualCareNS implementation model.

The rollout occurred in phases, initially targeting two communities in two of NSH's four health management zones, then expanding to all unattached patients registered on the NFPR. Within one year, the service was open to any unattached Nova Scotian. VirtualCareNS addresses low-acuity, non-urgent issues, such as minor infections, prescription refills, diagnostic imaging (DI), and specialist referrals. When physical examination or hands-on care is necessary, patients are directed to in-person primary care services (e.g. mobile clinics and primary care clinics). By September 2023, VirtualCareNS was supported by a roster of 79 providers, including 44 Nova Scotian physicians, 26 Nova Scotian nurse practitioners, and nine Canadian physicians practicing virtually from other provinces. Administrative staff and ongoing training underpin the service, ensuring operational continuity and consistent quality.

Purpose and objectives

We conducted a mixed methods evaluation of VirtualCareNS to:

Assess the feasibility, acceptability, and appropriateness of integrating VirtualCareNS into provincial primary care.

Examine the impact of VirtualCareNS on patient and provider experiences, service use (including ED and walk-in clinic diversions), and estimated cost savings.

Investigate scalability and sustainability by identifying barriers, facilitators, and opportunities for long-term integration into Nova Scotia's PHC system.

Methods

Study design and framework

The initial pilot of VirtualCareNS and its evaluation took place during the early stages of the COVID-19 pandemic. Given the need for timely insights to guide decision-making, the evaluation was designed as a rapid assessment to identify necessary adaptations to the program. This type of rapid evaluation approach has been described in the literature as a method for generating timely evidence to support healthcare decision-making.²⁵ Rapid evaluation approaches typically employ mixed-methods designs, iterative data collection, stakeholder engagement, and administrative data to capture real-time insights and are particularly useful in dynamic healthcare environments. They have become more widely used in healthcare, particularly for assessing the implementation of new digital health technologies, virtual care models, and pandemic response initiatives.^26–29

We utilized a mixed-methods design, guided by qualitative descriptive methodology.³⁰ Our evaluation was guided by the Standards for Reporting Implementation Studies (STaRI)³¹ and the Consolidated Health Economic Evaluation Reporting Standards (CHEERS)³² checklists. These frameworks were chosen to align with the mixed-methods design and the emphasis on implementation and economic evaluation central to our study.

The scope was deliberately focused on key areas where challenges were anticipated, including uptake and utilization, patient accessibility and acceptability, provider acceptability, perceived barriers and facilitators identified by the implementation team, and economic analysis. This evaluation was classified as a quality improvement initiative by the NSH Research Ethics Board (REB) (REB file #1027001).

Data collection

We collected data from four main sources: administrative (utilization) data, surveys, qualitative interviews, and economic data. A table with all pertinent data information can be found in Appendix A (Table A1).

Utilization data

Utilization data were obtained from the Maple platform (the technology host for VirtualCareNS), including the number of registered patients, number of appointments, repeat usage, demographics, and reasons for in-person referrals. We also identified “non-users” who did not register or who registered but did not use the service.

Surveys

User experience surveys were administered on an ongoing basis through the Maple/VirtualCareNS platform. Following their virtual visit, users were invited to complete a voluntary survey that asked them to report their satisfaction on a 5-point scale (5 being “very satisfied,” 1 being “very unsatisfied”) and provide open-ended feedback. The included survey data spans May 2021 through June 2022. A one-time email survey was also sent to individuals eligible for VirtualCareNS who never used it (e.g. those invited to use VirtualCareNS but did not register and those who registered but had not used the service at the time of receiving the survey) to better understand barriers to using virtual care and reasons for non-use. Survey data from non-users were collected between May 2021 and June 2022.

The implementation team (providers, managers, directors, and support personnel) was surveyed in January 2024, approximately two years after implementation was considered complete as the focus was on perceived impacts and lessons learned. A snowball sampling approach ensured broad coverage of PHC program staff, medical affairs, and other collaborating teams.

All the surveys were developed collaboratively with stakeholders to reflect the program's core objectives and participant experiences. While the tools were not based on previously validated questionnaires, they were iteratively refined with input from stakeholders to ensure face and content validity, in line with the goals of a rapid, pragmatic evaluation.

Copies of user, non-user, and implementation team surveys can be found in Appendix B. REDCap was used for survey distribution³³ and consent was implied upon survey completion.

Interviews

We conducted one-on-one virtual (phone or Zoom) interviews with three stakeholder groups: providers (physicians, nurse practitioners), implementation team members (policy and decision-makers, clinical leaders), and patients/users of VirtualCareNS. The interviews took place in two main phases: July–August 2021 and March–April 2022, with additional sessions taking place in October–November 2021. We used a semi-structured interview guide to explore experiences with VirtualCareNS, implementation barriers/facilitators, and sustainability considerations. Written consent was obtained from participants before each interview. In keeping with the principles of rapid evaluation and qualitative descriptive methodology, we aimed for data sufficiency rather than theoretical saturation; key themes were consistently reinforced across participants, indicating adequate depth to meet the evaluation objectives.

Data analysis

Quantitative data, made up of utilization, and survey responses, were analyzed using descriptive statistics. Qualitative data, made up of open-text survey responses and interview transcripts were analyzed using a rapid analysis approach³⁴ with team-generated summaries. Emergent themes were coded, with particular attention to how various stakeholder groups perceived barriers and facilitators. Summaries were then compared across groups to highlight consistent patterns and key differences.

Economic analysis

The economic analysis aimed to quantify the cost per VirtualCareNS visit and the overall potential short-term net cost-savings associated with using VirtualCareNS compared to what would have happened without it. In addition to the cost per VirtualCareNS visit, the analysis quantified the savings associated with diverted ED and walk-in clinic visits, the economic value of travel time saved, and the value of ED and walk-in clinic wait time saved from VirtualCareNS instead of in-person visits. A cost-benefit analysis framework, adhering closely to the CHEERS framework, was employed for the economic analysis.^32,35,36 See Table A2 in the Appendix for the relevant data.

Implementation and ongoing costs, such as setup and technology costs, provider compensation, and vendor-related costs like licensing and patient intake services, were combined with VirtualCareNS utilization data to estimate the cost per visit. Costs were derived using a cost-to-charge ratio applied to vendor expenses. VirtualCareNS utilization data came from the Maple platform.

To estimate overall cost savings, survey data from VirtualCareNS participants were analyzed. Patients who completed visits were asked where they would have otherwise sought care, with responses including ED or walk-in clinic visits. After adjusting for VirtualCareNS visits flagged as emergencies, the proportion of ED and walk-in visits diverted was calculated. The net ED costs avoided were derived from the difference between ED and VirtualCareNS costs per visit. Walk-in clinic costs, including physician fees, were proxied using Nova Scotia clinic fees, and the cost savings associated with VirtualCareNS visits were determined by comparing these costs.

The economic value of travel time saved was calculated using survey responses and Statistics Canada's 2022 hourly wage rates for unionized employees in Nova Scotia.³⁷ Patients reported travel time savings categorized as one to three hours or more than three hours, with average savings of two and three hours assumed for these groups, respectively. Additionally, the economic value of ED wait time avoided was estimated for Level 4 and Level 5 CTAS patients, assuming they spent four hours at the ED compared to the approximately two hours required for a VirtualCareNS visit (including 70 min of wait time and 30 min for the visit itself).

Walk-in clinic wait times in Nova Scotia, estimated at 70 min,³⁸ were also incorporated, assuming a total visit time equivalent to that of VirtualCareNS, with additional savings realized only through reduced travel time.

Uncertainty in these estimates was quantified using a probabilistic sensitivity analysis (PSA). Parameters, including costs and visit counts, were fitted to gamma distributions, while survey response proportions were fitted to beta distributions. The model was run 1000 times, generating a distribution of outputs to construct 95% confidence intervals for all reported results.³⁹

Results

Utilization of VirtualCareNS

Between 17 May 2021 and 1 November 2023, 76,054 unique patient profiles were created in VirtualCareNS, with 61,782 (81%) account holders and 14,272 (19%) dependents (Table 1). Over this period, a total of 101,416 virtual visits were completed, with an average user age of 42 (standard deviation [SD] = 18) years and a mean wait time of approximately 70 min, defined as the time from initial consult request to provider acceptance.

Table 1.

Utilization of VirtualCareNS from 17 May 2021 to 1 November 2023.

		n (%)
Total number of VirtualCareNS user profiles created during this period		N = 76,054
	Account holders	61,782 (81%)
	Dependents	14,272 (19%)
Total number of VirtualCareNS visits during this period		N = 101,416
	Visits resulting in prescriptions	52,369 (52%)
	Visits resulting in primary care lab requisitions	26,034 (26%)
	Visits resulting in referrals to in-person care	14,198 (14%)
	Visits resulting in diagnostic imaging requisitions	3146 (3%)
Number of VirtualCareNS users who had visits		N = 37,496
	1 visit	16,208 (43%)
	2 visits	7934 (21%)
	3 visits	4760 (13%)
	4 visits	2974 (8%)
	5 visits	1847 (5%)
	6 visits	1167 (3%)
	7 visits	750 (2%)
	8 visits	533 (1%)
	9 visits	375 (1%)
	10 + visits	948 (3%)

Note: Percentages are rounded and do not sum to 100%. 37,496 is the total number of unique individuals who have used the service whereas 101,416 accounts for all visits including repeat visits from the 37,496 unique users.

Source: Maple data.

More than half of the users (57%) used the service more than once, though the majority had only one (43%) or two visits (21%), while 3% had more than 10 visits (Table 1). Visit outcomes included prescriptions (52%), primary care lab requisitions (26%), referrals to in-person care (14%), and diagnostic imaging requisitions (3%) (Table 1).

Economic impact

Among survey respondents (users), 24% indicated they would have sought care at an ED, and 36% would have used a walk-in clinic had VirtualCareNS not been available (Table 2). Using these data, the estimated ED and walk-in clinic diversions were 24,385 (95% CI: 18,317–31,596) and 36,514 (95% CI: 26,924–47,520), respectively.

Table 2.

Avoidance of unnecessary visits and travel time due to VirtualCareNS (17 May 2021—1 November 2023).

		n (%)
If not for VirtualCareNS, where would you have otherwise sought care?		N = 74,103
	Walk-in clinic	26,994 (36%)
	Not sure	18,334 (25%)
	Emergency room	14,272 (24%)
	Nowhere	8773 (12%)
	Research symptoms online	2426 (3%)
How much travel time did this virtual visit save you?		N = 63,322
	Less than one hour	27,408 (43%)
	One to three hours	18,385 (29%)
	More than three hours	9460 (15%)
	Not applicable	8069 (13%)

Note: Percentages are rounded and do not sum to 100%.

Source: Maple data.

The total estimated cost of the VirtualCareNS program ranged from $10.0 M to $14.8 M CAD. The mean cost per consult was $123 (95% CI: $99–$149) (Table 3). After excluding 548 emergency-flagged visits, the effective number of completed VirtualCareNS consults was 100,868.

Table 3.

Economic analysis results.

Outcome	Mean (95% CI)
Total cost	$12,421,443 ($10,011,876—$14,848,578)
Cost per VirtualCareNS visit	$123 ($99—$149
Mean savings per virtual consult	$85 ($62—$111)
Economic value of travel time saved per virtual consult	$31 ($25—$38)
The economic value of ED and walk-in clinic time saved per virtual consult	$15 ($11—$19)
ED and walk-in savings per virtual consult	$39 ($18—$62)
Total savings	$8,575,770 ($5,936,787—$11,713,783)
Emergency department: Net total savings	$5,202,155 ($3,296,194—$7,714,008)
Economic value of ED wait time savings: Net total	$1,521,841 ($1,069,538—$2,041,810)
Walk-in clinics: Net total	-$1,288,335 (-$2,545,862—-$194,273)
Economic value of travel time saved	$3,140,109 ($2,345,882—$4,049,855)

Note: 95% CI was constructed from results from a probabilistic sensitivity analysis.

The estimated average cost-savings per consult was $85 (95% CI: $62–$111), comprising savings from travel time ($31), avoided wait times ($15), and reduced use of ED and walk-in services ($39). Total estimated net cost savings were $8.6 M (95% CI: $5.9M–$11.7 M), largely attributed to ED diversions ($5.2 M) and travel time savings ($3.1 M) (Table 3).

Acceptability

User satisfaction with VirtualCareNS was high. Of the 74,159 users who responded to the user experience survey, 91% rated their experience as satisfactory or very satisfactory (Table 4). Qualitative interviews supported these findings, with participants citing positive benefits related to the intuitive technology, convenience, and the ability to access care when other options were limited.

Table 4.

User satisfaction with VirtualCareNS (17 May 2021–1 November 2023).

		n (%)
How would you rate your level of satisfaction with VirtualCareNS?		N = 74,159
	5—Very Satisfied	54,380 (73%)
	4—Satisfied	13,053 (18%)
	3—Neutral	4031 (5%)
	2—Unsatisfied	1081 (1%)
	1—Very Unsatisfied	1604 (2%)

Note: Percentages are rounded and do not sum to 100%.

Source: Maple data.

Most respondents (43%) reported saving less than one hour of travel time, while 29% saved one to three hours, and 15% saved more than three hours, suggesting strong value in the service for those living in rural or remote geographies (Table 2).

Feasibility and appropriateness

From the provider and implementation team perspectives, VirtualCareNS was largely feasible and appropriate within the provincial primary care landscape. Providers (N = 8) highlighted ease of use, ability to prescribe and refer, and patient satisfaction. They also pointed out opportunities for improvement, such as integration with the provincial electronic medical record (EMR) system and enhanced referral workflows.

Similarly, the implementation team (N = 31) emphasized the platform's contributions to improved care access (30%), modernized service delivery (22%), and enhanced patient outcomes (18%). However, operational challenges related to understanding roles and responsibilities (13.2%), communication (13.2%), coordination (11.8%), and integration with existing systems (11.0%) were identified (Table 5).

Table 5.

Implementation team perspectives on VirtualCareNS (January 2024).

		Percentage
What was VirtualCareNS’ impact?		N = 31
	Improved access to care	30%
	Modernizing health service delivery	22%
	Enhancing patient outcomes	18%
	Enhanced provider, team-based, practice-based work	12%
	Coordinated care	12%
	Strengthening partnerships (internal or external)	8%
	Other	4%
Which factors served as barriers for VirtualCareNS?		N = 31
	Roles and responsibilities	13.2%
	Communication	13.2%
	Coordination and collaboration	11.8%
	Compatibility with existing workflows, structures, and processes	11.0%
	Staffing	10.3%
	IT infrastructure and/or technology	10.3%
	Leadership support	8.8%
	Access to knowledge and information	8.1%
	Partnerships	7.4%
	Resources to implement	6.6%
	Relative priority of VirtualCareNS	4.4%
	Other	0.7%

Note: Respondents could select multiple responses, and therefore percentages do not sum to 100%.

Source: REDCap data.

The implementation team strongly emphasized the importance of embedding VirtualCareNS within the broader health system infrastructure to ensure continuity and sustainability.

Barriers and challenges

While overall reception was positive, both users and providers raised concerns about continuity of care. Some users with chronic conditions noted difficulties with care consistency, particularly when interacting with multiple providers unfamiliar with their medical history. Confusion regarding the distinction between VirtualCareNS and the Maple platform was also noted.

Technological and digital literacy challenges were mentioned, particularly among users unfamiliar with virtual platforms. Non-users (N = 3130) cited reasons for non-participation such as not needing care (58%), a preference for in-person care (11%), and difficulties accessing the service via phone or computer (7%) (Table 6). These point to issues of access, equity, and system navigation.

Table 6.

Reasons non-users did not register for VirtualCareNS (May 2021–June 2022).

		n (%)
Why did you not register for VirtualCareNS?		N = 3130
	Not currently in need of an appointment or primary care	1815 (58%)
	Other	532 (17%)
	Prefer care in person	344 (11%)
	Difficulty joining by computer or phone	219 (7%)
	I forgot	188 (6%)
	Already have a primary care provider	125 (4%)
	Poor/No access to the internet	125 (4%)
	Current hours of service are not convenient for me	125 (4%)
	Did not receive an email to register	94 (3%)

Note: Respondents could select all applicable reasons for not registering for VirtualCareNS, and therefore percentages do not sum to 100%.

Source: REDCap data.

Discussion

Main findings and implementation outcomes

The implementation of VirtualCareNS was initiated to address barriers to primary care access in Nova Scotia, accelerated by the immediate impacts of in-person primary care during the COVID-19 pandemic. Similar to other high-income settings with evolving telehealth infrastructures, VirtualCareNS highlights the need to align patient, community, provider, and system considerations for optimal virtual primary care.

VirtualCareNS follows emerging practices by offering a hybrid model that links virtual care with in-person primary care services (as required) within a user's health management zone, integrating diagnostic and laboratory services, and providing pathways to specialist referrals. This innovative approach has garnered widespread recognition, including local and national media attention, presentations at over 50 conferences, and the prestigious 3M National Award for Quality.⁴⁰ This hybrid model is particularly vital in communities with geographical barriers, high rates of older adults requiring chronic disease management, and varied broadband capacities. Its rapid expansion from two health management zones to a province-wide initiative reflects its role in addressing current needs and gaps in primary care for those without a primary care provider.

Contextual factors and adaptations

The early findings from the implementation research reveal promising trends for the potential for VirtualCareNS to reduce pressure on EDs and walk-in clinics.⁴¹ An estimated diversion of 24,385 ED visits and 36,514 walk-in clinic visits were documented. These outcomes demonstrate the platform's potential to alleviate strain on healthcare facilities while enhancing access. Most visits through VirtualCareNS address low-acuity or episodic care needs, aligning with its design. Supporting the needs of patients with complex chronic conditions through this modality will require further exploration and consideration. Digital inequities, such as limited internet or device access and technology literacy, remain barriers to broader adoption. While over half of non-users reported no immediate need for primary care at the time of invitation, usage trends indicate lower adoption among older adults, a finding significant within the context of Nova Scotia's aging population.^12,42

Program enhancements since 2021 include the establishment of a provincial virtual care program office, and data-driven tools like a daily utilization dashboard. These improvements have supported the continued integration of VirtualCareNS into the PHC program while also informing new virtual care initiatives, such as virtual urgent care and early chronic disease intervention programs. Nova Scotia's evolving virtual care landscape positions the province to contribute valuable insights on integration, equity, and the sustainable implementation of digital health solutions.

Innovations and program improvements

A distinguishing feature of VirtualCareNS is its deliberate integration within the broader landscape of PHC services in Nova Scotia, including in-person clinics, urgent treatment centres, and pharmacy-based care. Rather than operating as a stand-alone service, VirtualCareNS was intentionally designed as a component of a larger, interconnected PHC ecosystem. This systems-level orientation facilitated seamless transitions between virtual and in-person care, improved referral processes, and fostered sustained adoption of virtual care as a long-term modality rather than a temporary solution.

Our evaluation revealed that iterative program improvements were driven by ongoing feedback from patients, providers, and system partners. Notable enhancements included streamlining the user registration experience and refining internal referral workflows. While formal care audits were not conducted, qualitative interviews and user surveys played a central role in guiding these adaptations. Furthermore, insights generated from the implementation and operation of VirtualCareNS directly informed the development of related programs such as Virtual Urgent Nova Scotia (VUNS) and Virtual Emergency Nova Scotia (VENS), particularly in improving user interface design and integrating laboratory requisition functionalities.

Evidence from the broader literature further supports the value of integrated virtual care models. Ward and colleagues⁴³ reported that telehealth was widely utilized during the COVID-19 pandemic for a range of primary care functions, including chronic disease and medication management for existing patients. More recently, systematic reviews by Ezeamii et al.⁴⁴ and Ravi et al.⁴⁵ emphasized the transformative potential of telehealth in managing chronic conditions such as type 2 diabetes. With an estimated 3.7 million Canadians living with diagnosed diabetes in 2020–2021,⁴⁶ the expansion of telehealth could play a critical role in enhancing chronic disease outcomes through more accessible and sustained care delivery.⁴⁵

Healthcare Excellence Canada⁴⁷ noted that over 90% of physicians were providing virtual care during the pandemic. As virtual care becomes increasingly embedded within routine practice, key principles must continue to guide its evolution, namely equity, clinical appropriateness, shared decision-making around modality, care quality and safety, continuity, and team-based coordination.

A rapid synthesis by the McMaster Health Forum⁴⁸ found that most hybrid care models focus on primary care. Synchronous formats, such as video consultations, were more common than asynchronous ones, though both were often combined. Asynchronous messaging supported follow-up and information exchange. While integration with in-person care was frequently noted, few studies detailed how this was operationalized.

Taken together, these findings position VirtualCareNS as a case study in scalable, adaptive virtual care design. Its evolution offers practical insights for jurisdictions seeking to embed virtual modalities within broader health systems.

Strengths and limitations

A major strength of VirtualCareNS is its integration into Nova Scotia's broader primary care system,^49,50 enabled by collaboration and engagement with primary care managers, providers, and patient advisors from planning through to implementation. This collaborative model remains a driving force as the program continues to evolve within a broader primary healthcare strategy to improve access. The VirtualCareNS hybrid model has allowed patients to transition smoothly between virtual and in-person services as needed. The evaluation design—incorporating mixed methods and rapid feedback loops—also aligns well with implementation science principles and learning health system approaches.

The findings reported in this evaluation are limited by sampling bias (respondents were mostly users of VirtualCareNS) and may not be generalizable to all Nova Scotians (either those with or without a primary care provider) regarding the perspectives on and use of the virtual care service. Further, feedback was only captured by persons who used the service; apart from non-user feedback that was captured up to the end of Phase 1 (June 2022), we lacked feedback from non-users once VirtualCareNS was made widely available to all Nova Scotians. The utilization data were also not disaggregated into further sociodemographic information that would be useful to support the interpretation of the use of VirtualCareNS. Additionally, user experience ratings were not linked with further information concerning what constituted a satisfactory or an unsatisfactory visit.

This evaluation utilized multiple data sources retrieved from the vendor, leading to some challenges in comparing data. There was a discrepancy between the total number of completed consults reported by the vendor on the summary report (N = 101,416) and within the raw data (N = 99,701) for the same date range (17 May 2021 to 1 November 2023). This is due to data linkage errors that occur when merging the Maple data with the NAFPR due to issues de-coding encrypted health card numbers. This was the case for 1715 consults, which represents only 1.7% of the total consults completed, therefore this discrepancy is not expected to change the interpretations of findings. Despite the minimal variance, ongoing efforts to standardize data-linkage processes and reduce encryption errors will further enhance the reliability of reported metrics, enabling more precise outcome assessments.

The economic analysis offered only an initial snapshot of cost impacts. Although VirtualCareNS yielded measurable ED diversions, there was a net negative offset for walk-in clinic diversions due to relatively lower walk-in clinic costs. The limited scope of the evaluation also does not fully account for indirect or downstream costs—potential increases in ED use, broader societal costs, or long-term program sustainability. Future analyses should consider a more comprehensive approach to both costs and benefits, including caregiver burden, patient-reported outcomes, and a more granular examination of equity-related outcomes.

Conclusion

By providing an additional point of access, VirtualCareNS has demonstrated measurable success in diverting low-acuity cases away from EDs and walk-in clinics, contributing to estimated net cost-savings and alleviating pressure on the broader healthcare system. These results underscore the feasibility and value of a hybrid care model that combines virtual and in-person services, particularly for populations without regular primary care access or residing in geographically dispersed areas.

Beyond its functionality during the COVID-19 pandemic, the distinctiveness of VirtualCareNS lies in its governance structure and its integration into a coordinated suite of primary care services. These structural features have enabled iterative improvements and laid the groundwork for related initiatives in Nova Scotia (e.g. VUNS, VENS).

Our findings suggest that well-designed hybrid virtual care programs have the potential to be a strategic lever for primary care system transformation. Future research should explore long-term impacts on access, equity, provider engagement, and sustainability—key elements for scaling and optimizing virtual care within publicly funded systems.

Footnotes

Acknowledgements

Representatives from Nova Scotia Health that supported the conceptualization, implementation and evaluation of VirtualCareNS include Grayson Fulmer, Graeme Kohler, Sarah O’Brien, Lindsay Sutherland, Nelson Ventura, and Jo-Anne Wentzell. These individuals played a vital role in laying the groundwork for this research, and we would like to thank them for their contributions to the primary healthcare innovation landscape in the province. Our thanks to patient family partners, clinicians and team members who provided their valuable expert insights that informed the design, planning and implementation of VirtualCareNS. Thank you to Dr Nicole Boutilier for contributing her expansive knowledge of primary care in Nova Scotia during the manuscript review process. The team would also like to acknowledge Vishal Sahijwala for extending his support and unique expertise in the VirtualCareNS implementation to the authors.

ORCID iDs

Tara Sampalli

Prosper Koto

Marta MacInnis

Ethical considerations

This evaluation was delegated as a quality improvement project by NSH's Research Ethics Board (REB) (REB file #1027001).

Author contributions

TS acted as the submitting author and supported conceptualization, investigation, methodology, resources, supervision, and writing and review of the original draft. GTM led conceptualization, investigation, resources, supervision, and writing and reviewing. JM and MS supported conceptualization, formal analysis, investigation, methodology, validation, and writing and review of the original draft. PK and CK supported and oversaw data curation, formal analysis, investigation, methodology, validation, and writing and review of the original draft. MM and SS provided support for data curation, project administration, investigation, validation, and writing and reviewing the original draft. AB supported data curation, formal analysis, and validation. Both NW and CJ played a role in the conceptualization of the work, investigation, project administration, and writing and reviewing the original and revised drafts.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

All data supporting the results reported in this manuscript are stored at Nova Scotia Health on secure electronic platforms. These data will be retained in accordance with the data retention policies of the province. Requests for access to this data should be directed to Nova Scotia Health, Research, and Innovation. All data sharing is subject to Nova Scotia Health's policies and procedures to ensure data security and confidentiality. If there are any limitations or restrictions on data sharing, these will be communicated to the requestor.

Guarantor

GTM

Informed consent

All participants provided informed consent to participate in the evaluation. Written consent was obtained for the interviews, and consent was implied for the surveys.

Appendix

References

Haldane

Zhang

Abbas

, et al. National primary care responses to COVID-19: a rapid review of the literature. BMJ Open 2020; 10: e041622.

Singer

Kosowan

LaBine

, et al. Characterizing the use of virtual care in primary care settings during the COVID-19 pandemic: a retrospective cohort study. BMC Prim Care 2022; 23: 320.

Kristiansson

Gronsbell

, et al. Changes in primary care visits arising from the COVID-19 pandemic: an international comparative study by the International Consortium of Primary Care Big Data Researchers (INTRePID). BMJ Open 2022; 12: e059130.

World Health Organization. Primary care, https://www.who.int/teams/integrated-health-services/clinical-services-and-systems/primary-care (accessed 8 January 2024).

Neves

Gupta

, et al. Virtual primary care in high-income countries during the COVID-19 pandemic: policy responses and lessons for the future. Eur J Gen Pract. 2021; 27: 241–247.

Shaw

Jamieson

Agarwal

, et al. Virtual care policy recommendations for patient-centred primary care: findings of a consensus policy dialogue using a nominal group technique. J Telemed Telecare 2018; 24: 608–615.

Canadian Institute for Health Information. The Expansion of Virtual Care in Canada: New Data and Information. Ottawa, ON.

Belzile

M-N

Lam

Chan

, et al. Evaluating the impact of a virtual outpatient care programme in preventing hospitalizations, emergency department visits and mortality for patients with COVID-19: a matched cohort study. Clin Microbiol Infect Off Publ Eur Soc Clin Microbiol Infect Dis 2023; 29: 933–939.

Lindenfeld

Berry

Albert

, et al. Synchronous home-based telemedicine for primary care: a review. Med Care Res Rev MCRR 2023; 80: 3–15.

10.

Caffery

Farjian

Smith

. Telehealth interventions for reducing waiting lists and waiting times for specialist outpatient services: a scoping review. J Telemed Telecare 2016; 22: 504–512.

11.

Campbell

Greenfield

, et al. The impact of virtual consultations on the quality of primary care. Systematic review. J Med Internet Res 2023; 25: e48920.

12.

Veinot

Mitchell

Ancker

. Good intentions are not enough: how informatics interventions can worsen inequality. J Am Med Inform Assoc JAMIA 2018; 25: 1080–1088.

13.

Latulippe

Hamel

Giroux

. Social health inequalities and eHealth: a literature review with qualitative synthesis of theoretical and empirical studies. J Med Internet Res 2017; 19: e136.

14.

Islam

Bailey

Netto

. Digitalised primary care in the UK: a qualitative study of the experiences of minoritised ethnic communities. Br J Gen Pract J R Coll Gen Pract 2024; 74: e823–e831.

15.

Raffan

Anderson

Sinclair

, et al. The virtual care experience of patients diagnosed with COVID-19. J Patient Exp 2021; 8: 23743735211008310.

16.

Andreadis

Muellers

Ancker

, et al. Telemedicine impact on the patient-provider relationship in primary care during the COVID-19 pandemic. Med Care 2023; 61: S83–S88.

17.

Zachrison

Yan

Sequist

, et al. Patient characteristics associated with the successful transition to virtual care: lessons learned from the first million patients. J Telemed Telecare 2023; 29: 621–631.

18.

Cherabuddi

Heidemann

Gwinn

, et al. Disparities in use of virtual primary care during the early COVID-19 pandemic. Telemed E-Health 2023; 29: 1127–1133.

19.

Govier

Cohen-Cline

Marsi

, et al. Differences in access to virtual and in-person primary care by race/ethnicity and community social vulnerability among adults diagnosed with COVID-19 in a large, multi-state health system. BMC Health Serv Res 2022; 22: 511.

20.

Madhusudhan

Watts

Lord

, et al. Employer-sponsored health centers provide access to integrated care via a hybrid of virtual and in-person visits. Telemed Rep 2021; 2: 247–257.

21.

Fitzsimon

Belanger

Glazier

, et al. Clinical and economic impact of a community-based, hybrid model of in-person and virtual care in a Canadian rural setting: a cross-sectional population-based comparative study. BMJ Open 2023; 13: e069699.

22.

Aghajafari

Abboud

Claussen

, et al. Impact of COVID-19 on primary care: addressing health concerns and patient experience of virtual care. Adv Prev Med Health Care 2022.

23.

Government Of

Canada

. Canada Health Act; 1985.

24.

Maple

Home

. Maple, https://www.getmaple.ca/ (accessed 4 January 2024).

25.

Norman

Mason

Dumville

. Approaches to enabling rapid evaluation of innovations in health and social care: a scoping review of evidence from high-income countries. BMJ Open 2022; 12.

26.

Vindrola-Padros

. Can we re-imagine research so it is timely, relevant and responsive? comment on “experience of health leadership in partnering with university-based researchers in Canada: a call to 're-imagine' research”. Int J Health Policy Manag 2021; 10: 172–175.

27.

Vindrola-Padros

Johnson

. Rapid techniques in qualitative research: a critical review of the literature. Qual Health Res 2020; 30: 1596–1604.

28.

Sreedhara

Suvada

Bostic

, et al. Rapid evaluations of telehealth strategies to address hypertension: a mixed-methods exploration at two US health systems during the COVID-19 pandemic. Prev Chronic Dis 2022; 19: E81.

29.

Hassani

Mcelroy

Coop

. Rapid implementation and evaluation of virtual health training in a subspecialty hospital in British Columbia, in response to the COVID-19 pandemic. Front Pediatr 2021; 9.

30.

Kim

Sefcik

Bradway

. Characteristics of qualitative descriptive studies: a systematic review. Res Nurs Health 2017; 40: 23–42.

31.

Pinnock

Barwick

Carpenter

. Standards for reporting implementation studies (StaRI) statement. BMJ 2017; 356.

32.

Husereau

Drummond

Augustovski

. Consolidated health economic evaluation reporting standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations. BMC Med 2022; 20.

33.

Harris

Taylor

Minor

. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019; 95.

34.

Ritchie

Spencer

.The Qualitative Researcher's Companion. SAGE Publications, Inc., pp.305–329.

35.

Hoomans

Severens

. Economic evaluation of implementation strategies in health care. Implement Sci 2014; 9.

36.

O’leary

Lich

Frerichs

. Extending analytic methods for economic evaluation in implementation science. Implement Sci IS 2022; 17.

37.

Statistics Canada.

Average weekly earnings, average hourly wage rate and average usual weekly hours by union status, annual, https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1410013401 (2023, accessed 4 January 2024).

38.

Medimap. The walk-in clinic wait time index, https://medimap.ca/en/wait-time-index-2024/ (2024, accessed 4 June 2024).

39.

Hatswell

Bullement

Briggs

. Probabilistic sensitivity analysis in cost-effectiveness models: determining model convergence in cohort models. PharmacoEconomics 2018; 36: 1421–1426.

40.

Canadian College of Health Leaders. Congratulations to the 2023 3M health care quality team awards recipients https://cchl-ccls.ca/news_article/congratulations-to-the-2023-3m-health-care-quality-team-awards-recipients/ (2023, accessed 4 June 2024).

41.

Palinkas

Aarons

Horwitz

. Mixed method designs in implementation research. Adm Policy Ment Health Ment Health Serv Res 2011; 38: 44–53.

42.

Statistics Canada.

A portrait of Canada’s growing population aged 85 and older from the 2021 Census, https://www12.statcan.gc.ca/census-recensement/2021/as-sa/98-200-X/2021004/98-200-X2021004-eng.cfm (2022, accessed 5 June 2024).

43.

Ward

Vagholkar

Sakur

. Visit types in primary care with telehealth use during the COVID-19 pandemic: systematic review. JMIR Med Inform 2022; 10.

44.

Ezeamii

Okobi

Wambai-Sani

. Revolutionizing healthcare: how telemedicine is improving patient outcomes and expanding access to care. Cureus 2024; 16.

45.

Ravi

Meyerowitz-Katz

Yung

. Effect of virtual care in type 2 diabetes management - a systematic umbrella review of systematic reviews and meta-analysis. BMC Health Serv Res 2025; 25.

46.

Public Health Agency of Canada.

Snapshot of diabetes in Canada, 2023, https://www.canada.ca/en/public-health/services/publications/diseases-conditions/snapshot-diabetes-canada-2023.html (2023, accessed 8 April 2025).

47.

Healthcare Excellence Canada.

What we heard: results of a policy lab on the appropriate use of virtual care in a primary care setting, https://www.healthcareexcellence.ca/en/resources/supporting-health-leaders-to-further-the-delivery-of-appropriate-and-safe-virtual-care/ (2022, accessed 8 April 2025).

48.

Vélez

Demaio

Bhuiya

. Rapid synthesis: optimizing the use of hybrid-care models for delivery of healthcare services. McMaster Health Forum. https://www.mcmasterforum.org/docs/default-source/product-documents/rapid-responses/optimizing-the-use-of-hybrid-care-models-for-delivery-of-healthcare-services.pdf?sfvrsn=236190ed_5 (2023).

49.

Government of Nova Scotia.

Primary care available in more pharmacies in May, https://news.novascotia.ca/en/2023/04/26/primary-care-available-more-pharmacies-may-0 (2023, accessed 4 June 2024).

50.

Government of Nova Scotia.

Mobile primary healthcare clinic to visit more communities, https://news.novascotia.ca/en/2022/11/03/mobile-primary-healthcare-clinic-visit-more-communities (2022, accessed 4 June 2024).

Virtual care Nova Scotia: An evaluation of a hybrid model of virtual primary care to inform scaling and sustainability considerations

Abstract

Objective

Methods

Results

Conclusion

Keywords

Background and rationale

Existing evidence and knowledge base

Implementation context

Intervention description: VirtualCareNS

Purpose and objectives

Methods

Study design and framework

Data collection

Utilization data

Surveys

Interviews

Data analysis

Economic analysis

Results

Utilization of VirtualCareNS

Economic impact

Acceptability

Feasibility and appropriateness

Barriers and challenges

Discussion

Main findings and implementation outcomes

Contextual factors and adaptations

Innovations and program improvements

Strengths and limitations

Conclusion

Footnotes

Acknowledgements

ORCID iDs

Ethical considerations

Author contributions

Funding

Declaration of conflicting interests

Data availability statement

Guarantor

Informed consent

Appendix

References