Virtual Versus In-Person Follow-up After a Psychiatric Emergency Visit: A Population-Based Cohort Study: Suivi virtuel opposé à en personne après une visite à l’urgence psychiatrique : une étude de cohorte dans la population

Abstract

French

Objective

With increased utilization of virtual care in mental health, examining its appropriateness in various clinical scenarios is warranted. This study aimed to compare the risk of adverse psychiatric outcomes following virtual versus in-person mental health follow-up care after a psychiatric emergency department (ED) visit.

Methods

Using population-based health administrative data in Ontario (2021), we identified 28,232 adults discharged from a psychiatric ED visit who had a follow-up mental health visit within 14 days postdischarge. We compared those whose first follow-up visit was virtual (telephone or video) versus in-person on their risk for experiencing either a repeat psychiatric ED visit, psychiatric hospitalization, intentional self-injury, or suicide in the 15–90 days post-ED visit. Cox proportional hazard models generated adjusted hazard ratios (aHRs) and 95% confidence intervals (CIs), adjusted for age, income quintile, psychiatric hospitalization, and intentional self-injury in the 2 years prior to ED visit. We stratified by sex and diagnosis at index ED visits based on the International Classification of Diseases and Related Health Problems, 10th Revision, Canada (ICD-10-CA) coding.

Results

About 65% (n = 18,354) of first follow-up visits were virtual, while 35% (n = 9,878) were in-person. About 13.9% and 14.6% of the virtual and in-person groups, respectively, experienced the composite outcome, corresponding to incidence rates of 60.9 versus 74.2 per 1000 person-years (aHR 0.95, 95% CI 0.89 to 1.01). Results were similar for individual elements of the composite outcome, when stratifying by sex and index psychiatric diagnosis, when varying exposure (7 days) and outcome periods (60 and 30 days), and comparing “only” virtual versus “any” in-person follow-up during the 14-day follow-up.

Conclusions and Relevance

These results support virtual care as a modality to increase access to follow-up after an acute care psychiatric encounter across a wide range of diagnoses. Prospective trials to discern whether this is due to the comparable efficacy of virtual and in-person care, or due solely to appropriate patient selection may be warranted.

Keywords

virtual care ED visits psychiatric care

While the emergency department (ED) is a significant point of entry for individuals experiencing acute mental health crises, up to two-thirds of those with an ED visit for a psychiatric reason are in fact not admitted to hospital at the time of their visit.¹ For these individuals, timely outpatient follow-up mental health care, usually within 7, 14, or 30 days is recommended to promote continuity of management plans initiated in the ED.^2,3 However, there is evidence that fewer than half of individuals with a psychiatric ED visit receive mental health follow-up care in a timely manner postdischarge.⁴ Barriers to care may include individual-level factors such as psychosocial instability or homelessness, or system factors such as lack of primary care connection, distance from health-care options or lack of availability of timely service.⁵ This presents a significant opportunity for improvement of mental health system performance.

Virtual care has been occurring for some time in many jurisdictions to improve access to specialized care at a distance.^6,7 During the COVID-19 pandemic, virtual care, either via telephone or video, exploded into widespread use as a health-care delivery modality in alignment with COVID-19 containment efforts.⁸ This trend persisted, due to the convenience of virtual care and its potential to enhance access to mental health care, especially in rural and remote regions,⁹ and has opened up many opportunities to improve access to care across diverse populations.¹⁰ Virtual care could improve access to follow-up after a psychiatric ED visit. However, the extent to which virtual care is used in follow-up in this circumstance, and whether virtual care is a safe and effective alternative to in-person care for a highly acute population such as this one, is unknown.

The overall goal of this research was to add to knowledge about virtual care as a visit modality after a psychiatric ED visit. In a population-based sample in Ontario, Canada's largest province (population ∼15 million), where universal health care covers all inpatient, ED and outpatient medical care, our specific research objectives were to: (1) examine the extent to which virtual care is being used for follow-up mental health care after an ED visit for a psychiatric reason and (2) compare those receiving virtual versus in-person follow-up on their risk for a composite adverse psychiatric outcome (repeat ED visit for a psychiatric reason, psychiatric hospitalization, or completed suicide) within 90 days after discharge from the ED.

Methods

Study Design and Data Sources

This population-based cohort study used health administrative databases accessed at ICES (formerly the Institute for Clinical Evaluative Sciences) in Ontario, Canada. ICES is an independent, nonprofit research institute whose legal status under Ontario's health information privacy law allows it to collect and analyse health care and demographic data, without consent, for health system evaluation and improvement. This allows for access to a secure and accessible array of Ontario's health-related datasets to be linked using unique encoded identifiers and analysed at ICES (see ices.on.ca for data access details and procedures).

For the current study, we used: (1) the National Ambulatory Care Reporting Service (NACRS) data to capture ED visits occurring for a psychiatric reason; (2) the Ontario Health Insurance Plan (OHIP) to capture outpatient physician services, such as mental health follow-up care; and (3) the Canadian Institutes for Health Information Discharge Abstract Database (CIHI-DAD) and Ontario Mental Health Reporting System (OMHRS) to capture psychiatric hospitalizations. We also used ICES's Registered Persons Database (RPDB) to capture sex, age, and postal code, allowing for identification of neighbourhood income level and region of residence). These datasets were linked using unique encoded identifiers and analyzed at ICES. See Supplemental Table S1 for additional details of the datasets.

Participants

We included all Ontario residents (aged 18 years or older) who were (1) discharged to the community after an ED visit for a psychiatric reason between January 1 and December 31, 2021, in Ontario and (2) had at least 1 outpatient mental health-care visit to a physician in the 14 days postdischarge from the ED visit. Fourteen days was chosen because it represents the typical follow-up time in urgent psychiatric care programs.⁴ ED visits were considered to be for a psychiatric reason when there was a psychiatric diagnosis in the primary diagnostic field (International Classification of Diseases and Related Health Problems, 10th Revision, Canada [ICD-10-CA] F06–F99) and/or when intentional self-injury was coded in any diagnostic field (ICD-10-CA X60–X84, Y10–Y19, Y28 when DX10CODE1 F06–F99; intentional self-injury is not recorded in primary diagnostic fields in the NACRS dataset¹¹). A person's first ED visit during the year was considered the index visit, so as to be able to capture subsequent ED visits in the outcome.

Most ambulatory physician-based mental health care in Ontario is provided either in primary care or by psychiatrists.¹² As such, outpatient mental health follow-up visits were either to (1) a psychiatrist identified using OHIP specialty code 19 (any diagnosis) or (2) a primary care clinician using either a mental health “service” code or mental health “diagnostic” code submitted by the physician to the OHIP (sensitivity 80.7%, specificity 97% when validated against primary care chart review in the ability to differentiate between mental health and nonmental health-related visits.^13,14

Individuals with improbable age (>105 years), invalid sex, invalid ICES Key Number/non-Ontario residents (precluding linkage across datasets), and no OHIP coverage for at least 1 year prior to the index ED visit (precluding baseline covariate assessment) were excluded, as were those who were discharged to a non-home setting such as a long-term care facility or group home, or who revisited the ED or were readmitted to hospital within the 14-day window. Those with no outpatient mental health physician visits in the 14 days post-ED visit were also excluded from the cohort but flagged for additional analyses.

Follow-up Visit Modality

The primary exposure of interest was the modality of the first follow-up physician mental health visit (virtual vs. in-person) within 14 days of the index ED visit, as defined by the OHIP fee codes that accompanied the physician billing submission when virtual care modalities were utilized (Supplemental Table S1). A secondary exposure compared those having exclusively virtual visits versus “any” in-person visits during the 14-day postdischarge period. Virtual visits were conducted by video or by telephone; the OHIP billing codes did not differentiate between telephone and video visits during the time period under study. Physician remuneration was equivalent to virtual (video or telephone) and in-person care during the study period. For video visits, any videoconferencing platform that met government-regulated privacy criteria could be used, including well-known platforms such as Zoom and Webex, as well as the Ontario Telemedicine Network.¹⁵ While no specific protocols were utilized, physicians were expected to conform to policies on virtual care laid out by the provincial regulatory body. These policies include consideration of patient privacy, patient choice and appropriateness of a virtual care encounter.¹⁶

Outcomes

Repeat psychiatric ED visits, psychiatric hospitalizations, intentional self-injury, and suicide deaths are clinically important and widely considered to be key indicators of mental health system performance, including in Ontario.^17–20 The primary outcome was a composite outcome comprising each of these, between 15 and 90 days postdischarge.²¹ Repeat ED visits and hospitalizations were those with a psychiatric diagnosis in the primary diagnostic field (ICD-10-CA F06–F99 for NACRS and CIHI, or any in OMHRS) and/or with intentional self-injury (ICD-10-CA X60–X84, Y10–Y19, Y28 when DX10CODE1 F06–F99) in any diagnostic field. Suicides were identified by combining ED records for intentional self-injury, ICD-10-CA X60–X84, Y10–Y19, and Y28 with death reported as the discharge disposition from the repeat ED visit or hospitalization (Supplemental Table S2).

Covariates

Covariates were selected based on known associations between the outcomes and sociodemographic and clinical factors.²² Age, sex, rurality, and neighbourhood income quintile may all be baseline variables that could be associated with both the exposure and outcomes.^23–26 These were derived from the RPDB, the latter 2 using the postal code conversion file to align with the results of the Canadian census. Past psychiatric history may also play a role. Using the NACRS, CIHI-DAD, and OMHRS datasets, we included the number of psychiatric ED visits (with or without intentional self-injury), psychiatric hospitalizations, and ED visits with intentional self-injury in the 2 years prior to the index ED visit.²⁷ We also determined whether the individual had a usual primary care or psychiatrist provider (3+ visits to a primary care physician or psychiatrist with >50% of those visits to the same provider) in the 2 years prior to the index ED visit, as this was expected to be a major driver of the ability to obtain timely follow-up.²⁸ Diagnosis is also likely to be strongly associated with outcome risk—and could also be associated with the use of virtual care—so we extracted the primary diagnostic category at the index ED visit based on ICD-10-CA coding, for stratifying analyses into anxiety and related disorders, bipolar disorders, depressive disorders, personality disorders, psychotic disorders, and substance and alcohol use disorders (Supplemental Table S2).

Statistical Analysis

Covariates were described for the overall cohort, and also for those excluded because they did not have a follow-up appointment in the 14 days postdischarge. We then reported on the study covariates by exposure group (virtual vs. in-person outpatient visit modality), evaluating the balance of covariates using standardized differences (SD).²⁹ Cox proportional hazard models were used to compare the primary composite outcome between exposure groups, adjusting for sex, age, neighbourhood income quintile and rurality, psychiatric hospitalization and intentional self-injury within 2 years prior to index ED visit, and index ED visit diagnosis. The analysis was repeated for those with “only” virtual outpatient visits (vs. “any” in-person visit) during the 14-day exposure period. In preplanned additional analyses, we stratified the main analysis by sex(male or female) and psychiatric diagnostic category at the index presentation and varied the exposure period for outpatient follow-up visits to 7 days, and the outcome period to 60 and 30 days.

Analyses were conducted using SAS, version 9.4 (SAS Institute). Use of the data in this project is authorized under section 45 of Ontario's Personal Health Information Protection Act and does not require review by a Research Ethics Board.

Results

Cohort

Of the 270,197 Ontario residents (aged 18 or older) discharged from a psychiatric ED visit in Ontario in 2021, 90,547 unique individuals remained after excluding those with improbable age (age >105 years), invalid sex, those unable to be linked across datasets (an invalid ICES Key Number, non-Ontario residents), and those not eligible for provincial coverage for at least 1 year prior to the index ED visit. Of these, 28,232 (31.1%) individuals had an outpatient mental health care follow-up visit within 14 days of ED discharge (Figure 1). Median age was 35 years (interquartile range [IQR] 26–49), 53.2% were female, 49.8% were low-income (two lowest income quintiles), and 88.2% resided in an urban area. Almost all (97.3%) had a usual primary care provider; 21.1% had a consistent psychiatric care provider (Table 1). Anxiety and related disorders were the most common diagnoses at the index ED visit (46.1% of the cohort), followed by substance and/or alcohol use disorders (22.7%). Only 4.7% presented with schizophrenia or another primary psychotic disorder.

Figure 1.

Flowchart listing exclusion criteria and number of individuals excluded for each criterion to form the final cohort.

Table 1.

Characteristics of n = 28,232 with a Follow-up Visit in the 14 Days Post-ED Discharge, by Modality of the First Visit Type (Virtual or In-Person), Presented as N (%), Unless Otherwise Specified.

	Visit modality
	Virtual (n = 18,354)	In-person (n = 9,878)	Total (n = 28,232)	Standardized difference
Sociodemographic characteristics
Mean (SD) age, years	38.6 (15.8)	38.8 (16.0)	38.7 (15,9)	0.02
Sex, female	10,338 (56.3)	4673 (47.3)	15,011 (53.2)	0.18
Neighbourhood income quintile (Q)*
Q1 lowest	4892 (26.7)	2876 (29.1)	7,768 (27.5)	0.05
Q2	3942 (21.5)	2104 (21.3)	6,046 (21.4)	0
Q3	3420 (18.6)	1754 (17.8)	5,174 (18.3)	0.02
Q4	3115 (17.0)	1632 (16.5)	4,747 (16.8)	0.01
Q5 highest	2882 (15.7)	1390 (14.1)	4,272 (15.1)	0.05
Missing	103 (0.6)	122 (1.2)	225 (0.8)	0.07
Region of residence (urban)**	16,195 (88.2)	8705 (88.1)	24,900 (88.2)	0
Health service use (past 2 years)
Median (IQR) psychiatric ED visits	0 (0–1)	0 (0–1)	0 (0–1)	0.04
Median (IQR) psychiatric admissions	0 (0–0)	0 (0–0)	0 (0–0)	0.05
ED visits with intentional self-injury	757 (4.3)	399 (4.0)	1156 (4.1)	0.01
Usual primary care provider***	17,943 (97.8)	9524 (96.4)	27,467 (97.3)	0.08
Usual psychiatrist provider***	4067 (22.2)	1893 (19.2)	5960 (21.1)	0.07
Primary diagnosis at index ED visit
Anxiety and related disorders	8965 (48.8)	4043 (40.9)	13,008 (46.1)	0.16
Bipolar disorders	399 (2.2)	237 (2.4)	636 (2.3)	0.02
Depressive disorders	2645 (14.4)	1454 (14.7)	4099 (14.5)	0.01
Personality disorders	331 (1.8)	227 (2.3)	558 (2.0)	0.03
Psychotic Disorders	705 (3.8)	627 (6.3)	1332 (4.7)	0.11
substance and Alcohol use disorders	3955 (21.5)	2467 (25.0)	6422 (22.7)	0.08
Other categories:
Self-injury (no psychiatric diagnosis)	850 (4.6)	535 (5.4)	1385 (4.9)	0.04
Other disorders	504 (2.7)	288 (2.9)	792 (2.8)	0.01

Note. ED = emergency department; SD = standard deviation; IQR = interquartile range.

*Neighbourhood income quintile is determined by the individual's geographic information and is divided into 5 quintiles, with 1 being the lowest and 5 being the highest.

**Rurality was defined using the patient's geography-derived information and comparing that information to the Statistics Canada community size information leading to categorizing it as urban or rural.

***Consistent primary care or psychiatrist provider ≥3 visits, and >50% of these visits to the same provider.

The cohort characteristics were fairly similar to those of the n = 62,315 who did not have a follow-up visit within 14 days of ED discharge, with some exceptions. Those with no follow-up were less likely to live in an urban area (82.2% vs. 88.2%), and to have a usual primary care provider (91.3% vs. 97.3%) or psychiatrist (10.4% vs. 21.1%). They were less likely to have had an index ED visit diagnosis of depression (8.1 vs. 14.5%) and more likely to have had an index ED visit diagnosis of substance or alcohol use disorder (33.9% vs. 22.7%) (Supplemental Table S3).

Main Outcomes

About 65.0% of the cohort (n = 18,354) had a virtual first outpatient follow-up visit within 14 days of ED discharge, with 4436 visits being to a psychiatrist (24.1%). Among those whose first visit was in-person (n = 9878, 35.0% of the cohort), 3339 (33.8%) of the visits were to a psychiatrist. The virtual follow-up group was 56.5% female, while only 47.3% of those in the in-person follow-up group were female (Table 1). Close to half (48.8%) of the virtual follow-up group had presented with an anxiety or related disorder, compared to only 40.9% of those receiving in-person follow-up. Schizophrenia and other psychotic disorder diagnoses were the reason for presentation in 6.3% of the in-person follow-up group, and only 3.8% of the virtual follow-up group. However, other characteristics were very similar between groups (SD < 0.10).

The primary outcome was experienced by about 13.9% (n = 2552) of those with virtual care for their first outpatient mental health follow-up visit and about 14.6% (n = 1443) of those with the first visit in-person. This corresponded to incidence rates of 60.9 versus 64.2 per 1000 person years (py), adjusted hazard ratio (aHR) of 1.00 (95% CI, 0.94 to 1.07) (Table 2). Results were also not statistically different between groups when comparing those with only virtual versus any in-person visits during the 14 days post-ED discharge (aHR 0.99, 95% CI, 0.93 to 1.06), for repeat psychiatric ED visits, repeat psychiatric ED visits with intentional self-injury, psychiatric hospitalizations, or completed suicide.

Table 2.

Adverse Composite Psychiatric Outcome of Psychiatric Hospitalization, Repeat Psychiatric ED Visit or Suicide in the 90-Days Post-ED Discharge, by Visit Modality.

	Modality	N (%)	Rate per 1000 py	aHR* (95% CI)
Modality of first mental health follow-up visit
Composite outcome	In-person (n = 9878)	1443 (14.6)	64.2	1.00 (referent)
Composite outcome	Virtual (n = 18,354)	2552 (13.9)	60.9	1.00 (0.94–1.07)
Individual outcomes
Psychiatric hospitalization	In-person (n = 9878)	497 (5.0)	21.0	1.00 (referent)
Psychiatric hospitalization	Virtual (n = 18,354)	898 (4.9)	20.4	1.06 (0.95–1.18)
Repeat ED visit	In-person (n = 9878)	1346 (13.6)	59.5	1.00 (referent)
Repeat ED visit	Virtual (n = 18,354)	2385 (13)	56.6	1.00 (0.94–1.07)
Repeat ED visit with intentional self-injury	In-person (n = 9878)	139 (1.4)	5.8	1.00 (referent)
Repeat ED visit with intentional self-injury	Virtual (n = 18,354)	250 (1.4)	5.6	1.02 (0.83–1.26)
Suicide	In-person (n = 9878)	NR	<0.05	1.00 (referent)
Suicide	Virtual (n = 18,354)	NR	<0.10	2.15 (0.22–21.25)
Only virtual versus any in-person
Composite outcome	Any in-person (n = 11,410)	1676 (10)	64.5	1.00 (referent)
Composite outcome	Only virtual (n = 16,822)	2319 (20.3)	60.3	0.99 (0.93–1.06)
Individual outcomes
Psychiatric hospitalization	Any in-person (n = 11,410)	579 (3.4)	21.2	1.00 (referent)
Psychiatric hospitalization	Only virtual (n = 16,822)	816 (7.2)	20.2	1.04 (0.93–1.15)
Repeat ED visit	Any in-person (n = 11,410)	1568 (9.3)	60.1	1.00 (referent)
Repeat ED visit	Only virtual (n = 16,822)	2163 (19.0)	55.9	0.99 (0.92–1.05)
Repeat ED visit with intentional self-injury	Any in-person (n = 11,410)	166 (1.0)	6.0	1.00 (referent)
Repeat ED visit with intentional self-injury	Only virtual (n = 16,822)	223 (2.0)	5.4	0.96 (0.78–1.17)
Suicide	Any in-person (n = 11,410)	NR	<0.05	1.00 (referent)
Suicide	Only virtual (n = 16,822)	NR	<0.10	2.77 (0.28–27.2)

Note. aHRs and 95% CIs are reported. Adjusted for age, sex, neighbourhood income quintile, rurality, index ED visit diagnosis, psychiatric hospitalization in 2 years prior to the index ED visit and intentional self-injury in 2 years prior to the index ED visit. If any cell is ≤5, these cannot be reported due to privacy concerns. ED = emergency department; aHR = adjusted hazard ratio; CI = confidence interval; py = person years; NR = not reported.

Additional Analyses

There were no statistically significant differences in outcome risks between virtual and in-person follow-up in sex-stratified analyses either for the composite outcome or its individual elements (Table 3). Stratification by index ED visit diagnosis revealed that the overall incidence of the composite outcome was highest for those with psychotic disorders, bipolar disorders and substance and alcohol use disorders, and lowest for those with anxiety and depressive disorders. However, within diagnoses, there were no statistically significant differences in composite outcome risk between those with virtual versus in-person follow-up (Figure 2). Results were similar for individual elements of the primary outcome (Supplemental Table S4).

Figure 2.

Adverse psychiatric composite outcome of psychiatric hospitalization, repeat psychiatric emergency department (ED) visit or suicide in the 90 days post-ED discharge comparing first virtual versus first in-person follow-up visit, stratified by mental health diagnosis at index ED visit. Hazard ratios are adjusted for age, sex, neighbourhood income quintile, rurality, index ED visit diagnosis, psychiatric hospitalization in 2 years prior to the index ED visit and intentional self-injury in 2 years prior to the index ED visit.

Table 3.

Adverse Composite Psychiatric Outcome of Psychiatric Hospitalization, Repeat Psychiatric ED Visit or Suicide in the 90 Days Post-ED Discharge.

	Modality	N (%)	Rate per 1000 py	aHR (95% CI)
Males
Composite outcome	In-person (n = 5205)	820 (15.8)	69.7	1.00 (referent)
Composite outcome	Virtual (n = 8016)	1238 (15.4)	68.1	1.03 (0.94 to 1.12)
Individual outcomes
Psychiatric hospitalization	In-person (n = 5205)	280 (5.4)	22.5	1.00 (referent)
Psychiatric hospitalization	Virtual (n = 8016)	431 (5.4)	22.5	1.08 (0.93–1.26)
Repeat ED visit	In-person (n = 5205)	760 (14.6)	64.1	1.00 (referent)
Repeat ED visit	Virtual (n = 8016)	1145 (14.3)	62.6	1.03 (0.94 to 1.13)
Repeat ED visit with intentional self-injury	In-person (n = 5205)	66 (1.3)	5.2	1.00 (referent)
Repeat ED visit with intentional self-injury	Virtual (n = 8016)	96 (1.2)	4.9	1.02 (0.74 to 1.40)
Suicide	In-person (n = 5205)	NR	≤0.1	1.00 (referent)
Suicide	Virtual (n = 8016)	NR	≤0.2	2.15 (0.22 to 21.3)
Females
Composite outcome	In-person (n = 4673)	623 (13.3)	58.2	1.00 (referent)
Composite outcome	Virtual (n = 10,338)	1314 (12.7)	55.3	0.98 (0.89 to 1.08)
Individual outcomes
Psychiatric hospitalization	In-person (n = 4673)	217 (4.6)	19.3	1.00 (referent)
Psychiatric hospitalization	Virtual (n = 10,338)	467 (4.5)	18.8	1.04 (0.88 to 1.22)
Repeat ED visit	In-person (n = 4673)	586 (12.5)	54.5	1.00 (referent)
Repeat ED visit	Virtual (n = 10,338)	1240 (12.0)	51.9	0.99 (0.89 to 1.09)
Repeat ED visit with intentional self-injury	In-person (n = 4673)	73 (1.6)	6.4	1.00 (referent)
Repeat ED visit with intentional self-injury	Virtual (n = 10,338)	154 (1.5)	6.1	1.03 (0.78 to 1.37)
Suicide	In-person (n = 4673)	0	0	Not applicable
Suicide	Virtual (n = 10,338)	0	0	Not applicable

Note. Results are presented as aHRs and 95% CIs, comparing the first follow-up virtual versus in-person visit modality, stratified by sex (male or female based on health card). Adjustment is for age, sex, neighbourhood income quintile, rurality, index ED visit diagnosis, psychiatric hospitalization in 2 years prior to the index ED visit and intentional self-injury in 2 years prior to the index ED visit. If any cell is ≤5, these cannot be reported due to privacy concerns. ED = emergency department; aHR = adjusted hazard ratio; CI = confidence interval; py = person years; NR = not reported.

Neither varying the exposure period to 7 days (Supplemental Tables S5 and S6) nor the outcome length to 30 or 60 days (Supplemental Table S7) materially altered the findings.

Discussion

In this population-based study of physician visits following psychiatric ED visits in Ontario, Canada in 2021, two-thirds of the first post-ED mental health outpatient follow-up visits were conducted virtually. There were few demographic differences between those who had virtual versus in-person visits. Notably, both groups contained a similar percentage of people who were low-income and who lived rurally. The proportion of women and people with anxiety disorders was higher in the virtual care group, and the proportion of people with psychotic disorders was lower. The risk of a serious adverse psychiatric outcome within 90 days post-ED discharge was not significantly different whether the outpatient follow-up visit occurred virtually or in-person. Findings were robust by sex and psychiatric diagnosis, and when varying exposure and outcome time periods.

The results of this study are consistent with local and global literature about the use of virtual mental health care. The high rate of virtual care follow-up is consistent with research in the Ontario jurisdiction on the uptake of virtual care more generally after the onset of the COVID-19 pandemic,^30–32 as well as in other jurisdictions.³³ While it is often recognized that virtual care may not be accessible to everyone, particularly those of lower socioeconomic status or who live rurally, our virtual care and in-person groups were quite balanced on these variables. This may speak to the widespread availability of mechanisms for virtual care; about 96% of Canadians aged 15–44 had a cellular telephone in 2020.³⁴ Further, in our study, virtual visits could be by telephone, so may have been possible even if there was no internet capacity for videoconferencing.

Although our study was observational, the results of this study are also consistent with the growing body of clinical trials studying the efficacy of virtual mental health care. Results of recent systematic reviews of videoconferencing for psychotherapy for depression and anxiety found similar efficacy for in-person and virtual care delivery,³⁵ and there is ample evidence that virtual mental health care more generally performs similarly to in-person care in diverse ambulatory populations.³⁶ Much less research has focused on how virtual care performs in follow-up after an acute care health encounter, such as in an ED, where the patient is likely more acutely ill than in ambulatory encounters more generally. In a large U.S. observational study (n = 12,848), virtual follow-up care (telephone or video) after a general medical ED visit (i.e., not for a psychiatric condition) was associated with an increased rate of return to the ED and hospitalization within 30 days of the ED visit, compared to in-person follow-up.³⁷ The differences between this study and ours may be related to the nature of the conditions that were being treated. General medical follow-up may more often require a physical examination, compared to mental health care which relies more heavily on patient-reported symptoms and a mental status exam that does not always require physical examination.

There are several potential explanations for our findings of similar outcome risk between virtual and in-person care. As indicated above, the nature of the visit (i.e., not requiring a physical examination in most cases) may make virtual care an appropriate option in this case. Further, almost all patients in our cohort had a primary care clinician. Virtual care when delivered by a patient's usual primary care clinician versus a walk-in clinic, is associated with better outcomes, including reduced ED visits.³⁸ The main question that remains is whether the modalities are truly equivalent in this acutely ill post-ED population, as this was not a clinical trial, such that the modality of the visit was not randomly assigned. In Ontario, physicians provide either in-person or virtual care at their discretion, in accordance with clinical appropriateness and patient preference.³⁹ As such, our results could reflect that clinicians are appropriately selecting patients for virtual and in-person care—or that patients are appropriately self-selecting. A greater proportion of the in-person visits were to psychiatrists than were the virtual care visits. It is plausible that more severely ill patients were directed to psychiatrists, and that these were also the patients who needed to be seen in person (e.g., for physical examinations or due to psychosocial issues).

While a major strength of this study was its large population-based sample and ability to examine comparisons not only overall, but also within key subgroups (e.g., by sex and by diagnosis), there are limitations to the analysis. As indicated, this was an observational study and not a clinical trial, and we were somewhat limited in the clinical variables that we could measure to determine what was driving the outcome equivalence observed (i.e., subtle differences in severity or comorbidities that we were unable to measure might have led to the selection of one modality over the other for a given patient). We also did not have access to race and ethnicity data, we were unable to differentiate between virtual follow-up visits conducted by telephone versus video, and we could not capture follow-up visits to nonphysicians such as case managers or psychologists. Since almost the entire cohort had primary care providers, it is unclear whether the findings would generalize to populations without primary care access. Having a primary care provider was more common in our cohort than among those who were excluded from the study because they did not have a follow-up visit (97.3% vs. 91.3%), highlighting that issues of primary care access are crucial to consider in any planning for postacute care interventions. The findings may also not be fully generalizable to health jurisdictions where virtual care and/or outpatient follow-up care and remuneration models differ from those in our setting.

Future randomized trials in this area may not be practical, given the extent to which virtual care has become the standard service delivery modality in many jurisdictions. However, future research to further explore the role of virtual care delivery in mental health crisis intervention and how to optimize it is warranted, particularly given that the clinical acuity is likely to be higher than in much of the virtual care literature. Additional research to understand to whom these results generalize, and how clinicians select patients or patients self-select for virtual versus in-person follow-up after an ED visit, would be helpful in guiding future best practice clinical recommendations in this area. Future research can also continue to explore the quality of postpsychiatric ED discharge care delivered, and how virtual care might contribute to improving this care, and by extension the outcomes of individuals in psychiatric crisis after discharge from an acute care encounter.

In summary, these results from across a large health jurisdiction have key implications for patients, clinicians, policymakers, and researchers in the field of mental health care. The integration of virtual care into mental health services can potentially alleviate strain on health-care resources or improve access to care for underserved populations. As such, our study to determine its level of appropriateness in a highly acute psychiatric population has great potential for impact. We found that regardless of clinical diagnosis, virtual and in-person follow-up in the 14 days after a psychiatric ED visit were associated with a similar rate of repeat ED visits, and psychiatric hospitalizations. This lends support to the emerging literature around the safety and effectiveness of virtual mental health care and provides strong reassurance about extending this form of access even in more acute psychiatric clinical scenarios.

Supplemental Material

sj-docx-1-cpa-10.1177_07067437241281068 - Supplemental material for Virtual Versus In-Person Follow-up After a Psychiatric Emergency Visit: A Population-Based Cohort Study

Supplemental material, sj-docx-1-cpa-10.1177_07067437241281068 for Virtual Versus In-Person Follow-up After a Psychiatric Emergency Visit: A Population-Based Cohort Study by Matthew Crocker, MPH, Anjie Huang, MSc, Kinwah Fung, PhD, Therese A. Stukel, PhD, Alene Toulany, MD, MSc, Natasha Saunders, MD, MSc, Paul Kurdyak, MD, PhD, Lucy C. Barker, MD, PhD, Tanya S. Hauck, MD, PhD, Martin Rotenberg, MD, MSc, Emily Hamovitch, MSc and Simone N. Vigod, MD, MSc in The Canadian Journal of Psychiatry

Footnotes

Acknowledgments

This document used data adapted from the Statistics Canada Postal CodeOM Conversion File, which is based on data licensed from Canada Post Corporation, and/or data adapted from the Ontario Ministry of Health Postal Code Conversion File, which contains data copied under license from ^©Canada Post Corporation and Statistics Canada. Parts of this material are based on data and/or information compiled and provided by CIHI and the Ontario Ministry of Health. The dataset from this study is held securely in coded form at ICES. While legal data-sharing agreements between ICES and data providers (e.g., health-care organizations and government) prohibit ICES from making the dataset publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at (email: das@ices.on.ca). The full dataset creation plan and underlying analytic code are available from the authors upon request, understanding that the computer programs may rely upon coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification. The analyses, conclusions, opinions, and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH) and the Ministry of Long-Term Care (MLTC).

ORCID iDs

Matthew Crocker

Anjie Huang

Alene Toulany

Natasha Saunders

Paul Kurdyak

Lucy C. Barker

Tanya S. Hauck

Supplemental Material

Supplemental material for this article is available online.

References

Marzola

Duranti

De-Bacco

Lupia

Villari

Abbate-Daga

. Psychiatric patients at the emergency department: factors associated with length of stay and likelihood of hospitalization. Intern Emerg Med. 2022;17(3):845‐855. doi: https://doi.org/10.1007/s11739-021-02820-x

Bardach

Doupnik

Rodean

, et al. ED visits and readmissions after follow-up for mental health hospitalization. Pediatrics. 2020;145(6):1-9. doi: https://doi.org/10.1542/peds.2019-2872

Mental Health and Addictions System Performance in Ontario 2021 Scorecard. ICES; 2021 [accessed 2024 May 13]. https://www.ices.on.ca/Publications/ Atlases-and-Reports/2021/Mental-Health-and-Addictions-System-Performance-in-Ontario-2021-Scorecard.

Barker

Sunderji

Kurdyak

, et al. Urgent outpatient care following mental health ED visits: a population-based study. Psychiatr Serv. 2020;71(6):616‐619. doi: https://doi.org/10.1176/appi.ps.201900466

Schrader

Robinson

Blair

, et al. Common step-wise interventions improved primary care clinic visits and reduced emergency department discharge failures: a large-scale retrospective observational study. BMC Health Serv Res. 2019;19(1):451-462. doi: https://doi.org/10.1186/s12913-019-4300-1

Serhal

Crawford

Cheng

Kurdyak

. Implementation and utilisation of telepsychiatry in Ontario: a population-based study. Can J Psychiatry. 2017;62(10):716‐725. doi: https://doi.org/10.1177/0706743717711171

Fortney

Pyne

Turner

, et al. Telepsychiatry integration of mental health services into rural primary care settings. Int Rev Psychiatry. 2015;27(6):525‐539. doi: https://doi.org/10.3109/09540261.2015.1085838

Wong

Bhyat

Srivastava

Lomax

Appireddy

. Patient care during the COVID-19 pandemic: use of virtual care. JMIR. 2021;23(1):e20621. doi: https://doi.org/10.2196/20621

Mental Health and Addictions Dashboard. ICES; 2023 [accessed 2024 May 13]. https://www.ices.on.ca/dashboards/mental-health-dashboard/.

10.

Gratzer

Torous

Lam

, et al. Our digital moment: innovations and opportunities in digital mental health care. Can J Psychiatry. 2020;66(1):5‐8. doi: https://doi.org/10.1177/0706743720937833

11.

Bethell

Rhodes

. Identifying deliberate self-harm in emergency department data. Health Rep. 2009;20(2):35‐42.

12.

ICES. Mental health and addictions dashboard—individuals seen by primary care provider, psychiatrist or paediatrician for mental health and addictions-related care: Monthly trends of individuals seen by a psychiatrist, primary care provider or paediatrician for mental health and addictions care per 100 population aged 0 to 105 years, by provider specialty, in Ontario. 2024 [accessed 2024 May 13]. https://public.tableau.com/shared/M4QB7BBSG?:display_count=n&:origin=viz_share_link&:embed=y.

13.

Steele

Glazier

Lin

Evans

. Using administrative data to measure ambulatory mental health service provision in primary care. Med Care. 2004;42(10):960‐965. doi: https://doi.org/10.1097/00005650-200410000-00004

14.

Kurdyak

Lin

Green

Vigod

. Validation of a population-based algorithm to detect chronic psychotic illness. Can J Psychiatry. 2015;60(8):362‐368. doi: https://doi.org/10.1177/070674371506000805

15.

Verified Solutions List for Virtual Visits. Ontario Health [accessed 2024 May 13]. https://www.ontariohealth.ca/system-planning/digital-standards/virtual-visits-verification/verified-solutions-list.

16.

Advice to the Profession: Virtual Care. College of Physicians and Surgeons of Ontario [accessed 2024 May 13]. https://www.cpso.on.ca/en/Physicians/Policies-Guidance/Policies/Virtual-Care/Advice-to-the-Profession-Virtual-Care?utm_source=sfmc&utm_medium=email&utm_campaign=OMA+News+June+30+2022&utm_term=https%3a%2f%2fwww.cpso.on.ca%2fen%2fPhysicians%2fPolicies-Guidance%2fPolicies%2fVirtual-Care%2fAdvice-to-the-Profession-Virtual-Care&utm_id=251722&sfmc_id=7573485.

17.

Ontario Health Indicator Library—Repeat emergency visits for mental health [accessed 2024 May 13]. https://indicatorlibrary.hqontario.ca/Indicator/Summary/Repeat-emergency-visits-mental-health/EN.

18.

Ontario Health Indicator Library—Rate of MHA-related hospitalizations [accessed 2024 May 13]. https://indicatorlibrary.hqontario.ca/Indicator/Summary/Rate-MHA-related-hospitalizations-EN/EN.

19.

Ontario Health Indicator Library. Emergency department visits self-harm [accessed 2024 May 13]. https://indicatorlibrary.hqontario.ca/Indicator/Summary/DSH/EN.

20.

Ontario Health Indicator Library. Deaths by suicide [accessed 2024 May 13]. https://indicatorlibrary.hqontario.ca/Indicator/Summary/suicide/EN.

21.

Vigod

Taylor

Fung

Kurdyak

. Within-hospital readmission: an indicator of readmission after discharge from psychiatric hospitalization. Can J Psychiatry. 2013;58(8):476‐481.

22.

Moe

Wang

McGregor

, et al. Subgroups of people who make frequent emergency department visits in Ontario and Alberta: a retrospective cohort study. CMAJ Open. 2022;10(1):E232‐E246. doi: https://doi.org/10.9778/cmajo.20210132

23.

Shah

. The relationship between suicide rates and age: an analysis of multinational data from the World Health Organization. Int Psychogeriatr. 2007;19(6):1141‐1152. doi: https://doi.org/10.1017/S1041610207005285

24.

Barrigon

Cegla-Schvartzman

. Sex, gender, and suicidal behavior. Curr Top Behav Neurosci. 2020;46:89‐115. doi: https://doi.org/10.1007/7854_2020_165

25.

Barry

Rehm

de Oliveira

Gozdyra

Kurdyak

. Rurality and risk of suicide attempts and death by suicide among people living in four English-speaking high-income countries: a systematic review and meta-analysis. Can J Psychiatry. 2020;65(7):441‐447. doi: https://doi.org/10.1177/0706743720902655.

26.

Näher

Rummel-Kluge

Hegerl

. Associations of suicide rates with socioeconomic status and social isolation: findings from longitudinal register and census data. Front Psychiatry. 2020;10:898. doi: https://doi.org/10.3389/fpsyt.2019.00898

27.

Montoy

JCC

Tamayo-Sarver

Miller

Baer

Peabody

. Predicting emergency department “bouncebacks”: a retrospective cohort analysis. West J Emerg Med. 2019;20(6):865‐874. doi: https://doi.org/10.5811/westjem.2019.8.43221

28.

Fontanella

Hiance-Steelesmith

Bridge

, et al. Factors associated with timely follow-up care after psychiatric hospitalization for youths with mood disorders. Psychiatr Serv. 2016;67(3):324‐331. doi: https://doi.org/10.1176/appi.ps.201500104

29.

Austin

. Using the standardized difference to compare the prevalence of a binary variable between two groups in observational research. Commun Stat Simul Comput. 2009;38(6):1228‐1234.

30.

Saunders

Stukel

Strauss

, et al. Association between physician characteristics and practice-level uptake of paediatric virtual mental healthcare: a population-based study. BMJ Open. 2023;13(7):e070172. doi: https://doi.org/10.1136/bmjopen-2022-070172

31.

Vigod

Dennis

. Advances in virtual care for perinatal mental disorders. World Psychiatry. 2020;19(3):328‐329. doi: https://doi.org/10.1002/wps.20775

32.

Mental Health and Addictions Dashboard—Mental health and addictions-related outpatient visits: monthly virtual visits in mental health and addictions-related outpatient visits aged 0 to 105 years, by age group and provider specialty, in Ontario. 2024 [accessed 2024 May 13]. https://public.tableau.com/shared/C5R2PKDZZ?:display_count=n&:origin=viz_share_link&:embed=y.

33.

Canadian Institute for Health Information—Virtual care: use of physician mental health services in Canada—Total number of physician services for a diagnosis of anxiety or depression, by modality, selected provinces, April 2019 to March 2021. 2022 [accessed 2024 May 13]. https://www.cihi.ca/en/virtual-care-use-of-physician-mental-health-services-in-canada.

34.

Statistics Canada—So long landline, hello smartphone. 2023 [accessed 2024 May 13]. https://www.statcan.gc.ca/o1/en/plus/3582-so-long-landline-hello-smartphone.

35.

Berryhill

Culmer

Williams

, et al. Videoconferencing psychotherapy and depression: a systematic review. Telemed J E Health. 2019;25(6):435‐446. doi: https://doi.org/10.1089/tmj.2018.0058

36.

Hilty

Ferrer

Parish

Johnston

Callahan

Yellowlees

. The effectiveness of telemental health: a 2013 review. Telemed J E Health. 2013;19:444‐454.

37.

Shah

Villaflores

Chuong

, et al. Association between in-person vs telehealth follow-up and rates of repeated hospital visits among patients seen in the emergency department. JAMA Netw Open. 2022;5(10):e2237783. doi: https://doi.org/10.1001/jamanetworkopen.2022.37783

38.

Lapointe-Shaw

Salahub

Austin

, et al. Virtual visits with only family physician vs outside family physician and emergency department use. JAMA Netw Open. 2023;6(12):e2349452.

39.

College of Physicians and Surgeons of Ontario—Virtual Care. 2022 [accessed 2023 October 24]. https://www.cpso.on.ca/en/Physicians/Policies-Guidance/Policies/Virtual-Care.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB