Sage Journals: Discover world-class research

Abstract

Introduction

Telehealth may address healthcare disparities for rural populations. This systematic review assesses the use, effectiveness, and implementation of telehealth-supported provider-to-provider collaboration to improve rural healthcare.

Methods

We searched Ovid MEDLINE®, CINAHL®, EMBASE, and Cochrane CENTRAL from 1 January 2010 to 12 October 2021 for trials and observational studies of rural provider-to-provider telehealth. Abstracts and full text were dual-reviewed. We assessed the risk of bias for individual studies and strength of evidence for studies with similar outcomes.

Results

Seven studies of rural uptake of provider-to-provider telehealth documented increases over time but variability across geographic regions. In 97 effectiveness studies, outcomes were similar with rural provider-to-provider telehealth versus without for inpatient consultations, neonatal care, outpatient depression and diabetes, and emergency care. Better or similar results were reported for changes in rural clinician behavior, knowledge, confidence, and self-efficacy. Evidence was insufficient for other clinical uses and outcomes. Sixty-seven (67) evaluation and qualitative studies identified barriers and facilitators to implementing rural provider-to-provider telehealth. Success was linked to well-functioning technology, sufficient resources, and adequate payment. Barriers included lack of understanding of rural context and resources. Methodologic weaknesses of studies included less rigorous study designs and small samples.

Discussion

Rural provider-to-provider telehealth produces similar or better results versus care without telehealth. Barriers to rural provider-to-provider telehealth implementation are common to practice change but include some specific to rural adaptation and adoption. Evidence gaps are partially due to studies that do not address differences in the groups compared or do not include sufficient sample sizes.

Keywords

Rural health remote consultations telehealth telemedicine Extension for Community Healthcare Outcomes (ECHO)systematic review video education

Numerous studies have documented health disparities for people living in rural areas of the United States.^1–4 Rural-urban differences in access and quality of care contribute to higher mortality^4–7 and morbidity from conditions including substance/opioid misuse,^8,9 chronic illnesses,^10–13 and human immunodeficiency virus/human papillomavirus and other infectious diseases.^14,15 Causes of rural-urban differences are varied and related to complex macro and micro sociologic-demographic forces¹⁶ and economic trends;^17,18 nevertheless, the need for solutions is urgent. Both the coronavirus disease 2019 (COVID-19) pandemic and acknowledgement of structural racism have focused attention on broad inequities in healthcare in the United States, disproportionate burden on rural populations, and the resulting harms to underserved populations.

Telehealth utilizes technology to provide health care across time and/or distance¹⁹, encompassing a wide range of interventions, modes, and clinical functions. Telehealth is frequently proposed to address limited access and health disparities^20,21 and there is a sizable body of research, including systematic reviews and reviews of reviews,^22–29 suggesting telehealth can be effective. However, implementation and spread have been slow,^30–32 with steady, but small increases in telehealth use before the COVID-19 pandemic.³³ Rapid increases in the use of telehealth to limit the risk of exposure and transmission during the COVID-19 pandemic may continue to fuel wider adoption and more long-term use.^34,35 Growth in telehealth has focused attention on the need to address the digital divide, promote widespread access to telehealth,³⁶ and assure telehealth promotes equitable healthcare.³⁷

Rural provider-to-provider telehealth (RPPT) is a subset of telehealth interventions focused on supporting health care providers that treat rural patients and populations through consultations and collaborative patient care as well as mentoring and education.^38–40 We conducted this systematic review to identify and synthesize the available research on whether telehealth-supported provider-to-provider communication and collaboration can contribute to improving the health and well-being of rural residents and communities. This review was conducted to inform a National Institutes of Health (NIH) Office of Disease Prevention (ODP) Pathways to Prevention (P2P) workshop, Improving Rural Health through Telehealth-Guided Provider-to-Provider Communication, held on October 12–14, 2021). The panel report and recommendations from this workshop are also published in this journal.⁴¹

Methods

We conducted this review using methods outlined in the Agency for Healthcare Research and Quality (AHRQ) Methods Guide for Effectiveness and Comparative Effectiveness Reviews (hereafter “AHRQ Methods Guide”).⁴² Our protocol is available on the AHRQ website (https://effectivehealthcare.ahrq.gov/products/rural-telehealth/protocol) and registered with PROSPERO (registration no. CRD42021233545). Detailed methods and additional information in the full report are available at https://effectivehealthcare.ahrq.gov/products/rural-telehealth/research.

Key questions

This review was designed to answer the following four questions:
Key Question 1:
What is the uptake of different types of provider-to-provider telehealth in rural areas?
Key Question 2:
What is the effectiveness of provider-to-provider telehealth for rural patients?
Key Question 3:
What strategies are effective and what are the barriers and facilitators to implementation and sustainability of provider-to-provider telehealth in rural areas?
Key Question 4:
What are the methodological weaknesses of the included studies of provider-to-provider telehealth for rural patients and what improvements in study design (e.g. focus on relevant comparisons and outcomes) might increase the impact of future research?

Data sources and searches

We developed strategies with an expert librarian, had these peer-reviewed by a second librarian, and searched Ovid MEDLINE®, CINAHL®, EMBASE, and Cochrane CENTRAL for studies published between 1 January 2010 and 13 October 2021 (Search strategies included in online Supplement 1). Studies on telehealth have been published before January 2010. However, this start date was selected in consultation with the key informants and NIH ODP stakeholders and corresponded to their request to focus the review on studies using current technology that would be most relevant to future policy and decision-making.

Study selection

Two team members reviewed abstracts and full-text articles to identify studies that met our inclusion criteria (Supplemental Table 1), and any disagreements were resolved through discussion and consensus. To be included, studies had to be of rural patients or populations, or report results separately for rural patients or populations. Trials and observational studies that provided data on use, effectiveness, and facilitators or barriers to implementation of RPPT for inpatient, outpatient, or emergency care and studies of telehealth used for training health care providers who care for rural populations were included. Descriptive articles and studies that reported only data after telehealth implementation (no comparison group) were excluded. Studies had to be conducted in the United States or countries identified as having very high or high human development levels by the United Nations Human Development Report.⁴³

We defined RPPT as any form of interactive support using telecommunications technology provided to health care professionals who care for rural patients and populations. This included: video, audio, or digital remote consultations across space (e.g. video) or time (e.g. store and forward) as well as remote mentoring or education including rounds or case reviews. We excluded (a) telehealth for patient encounters, (b) remote patient monitoring if the data were transmitted only from a patient to a single provider and were not used as part of provider-to-provider consultations, and (c) communications that were limited to referring a patient to another provider for care and with no other interaction.

To assess use of RPPT we included studies with indicators or measures of uptake such as rates of use as outcomes. For effectiveness we included studies reporting clinical outcomes (e.g. mortality and morbidity) and intermediate outcomes (e.g. treatment, satisfaction, health care services utilization, and economic outcomes) We excluded studies if the outcomes were projections or the results of simulations.

Data extraction and risk of bias assessment

One person extracted data on the study design, setting, patient population, telehealth intervention, participating providers, and outcomes from the included studies. If a study was reported in multiple publications, the study was included once in our counts, but information about the study and data on outcomes were abstracted from all available publications. When extracting data from effectiveness studies, we also extracted methodological weaknesses listed by the study authors. Extracted data were verified for completeness and accuracy by a second person. Two team members independently assessed the risk of bias for individual studies using criteria appropriate for the study design and consistent with the approach in the AHRQ Methods Guide.⁴² Studies were rated as low, moderate, or high risk of bias.

Data synthesis and analysis

We summarized surveys and analyses that provided counts or estimates to assess use of RPPT. To address the effectiveness of RPPT, we grouped studies by health care setting, then clinical indication, and summarized findings based on the direction and magnitude of effect. We also assessed the strength of evidence (SOE) for outcomes prioritized by the ODP working group, following the approach described in the AHRQ Methods Guide.⁴² Our assessment of implementation summarized the few identified studies that compared approaches, then summarized barriers and facilitators cited in included studies. We classified barriers and facilitators using an existing framework, the Consolidated Framework for Implementation Research (CFIR).⁴⁴ We then summarized how frequently barriers and facilitators were reported across all identified studies. Our synthesis of the methodologic weaknesses of the effectiveness studies consisted of summarizing the weaknesses identified by the study authors as well as those identified by the review team.

Results

Our search produced 6329 citations, after triage of abstracts we reviewed 1024 full-text articles and included 166 studies reported in 179 publications (Figure 1). Seven studies met the inclusion criterial for Key Question 1 and provided data on use of provider-to-provider telehealth is rural areas (Key Question 1), 97 studies in 106 publications provided data on effectiveness of RPPT(Key Question 2) and were also used to identify methodological weaknesses in the body of evidence (Key Question 4), and 67 studies in 71 publications were used to summarize barriers and facilitators to implementation of RPPT (Key Question 3). Detailed descriptions of the included studies, as well as the data extracted, are included in the appendix to the full report at https://effectivehealthcare.ahrq.gov/products/rural-telehealth/research.

Figure 1.
Literature flow diagram. Key Question 1: What is the uptake of different types of provider-to-provider telehealth in rural areas? Key Question 2: What is the effectiveness of provider-to-provider telehealth for rural patients? How does provider-to-provider telehealth affect outcomes for patients and populations? How does provider-to-provider telehealth affect outcomes for healthcare providers? How does provider-to-provider telehealth affect outcomes for private and public payers? Key Question 3: What strategies are effective and what are the barriers and facilitators to implementation and sustainability of provider-to-provider telehealth in rural areas? Key Question 4: What are the methodological weaknesses of the included studies of provider-to-provider telehealth for rural patients and what improvements in study design (e.g. focus on relevant comparisons and outcomes) might increase the impact of future research

Use of provider-to-provider telehealth for rural populations

We did not identify comprehensive data on the use of RPPT in the United States. However, we identified seven studies that reported on national or regional use of provider-to-provider telehealth for emergency or inpatient care and included data specific to rural areas.^45–51 These studies were published since 2015 but included data collected prior to the COVID-19 pandemic.

Most studies reported lower use in rural than in urban areas, but studies that examined changes over time reported increases in both rural and urban hospitals. A study conducted in 2018 found that urban hospitals were twice as likely as rural hospitals to use telehealth for intensive care unit (ICU) and stroke care.⁴⁶ A study of telehealth use for heart attack and stroke assessment reported that rural hospital use increased from 6% in 2012 to 16% in 2017, though this was still lower than urban hospitals.⁴⁵ One study that reported higher use in rural than in urban areas focused on rates of telestroke use (8.6 vs. 2.3 per 1000), using Medicare fee-for-service claims to compare rates at the patient, rather than hospital, level.⁵¹ Two studies of telehealth use for any reason by emergency departments (EDs) in New England in 2014⁵⁰ and nationwide in 2016⁴⁹ reported use by 49% and 54% of rural EDs, though this differed by region, with less use in the southern United States. Additionally, studies of telepsychiatry use in rural EDs in 2016⁴⁷ and by rural mental health facilities from 2010 to 2107⁴⁸ reported less than 30% of those surveyed used telehealth.

Effectiveness of provider-to-provider telehealth for rural patients

We assessed effectiveness and SOE after grouping studies by setting and clinical topic. Table 1 presents outcomes for patients, providers, and payers. Findings are summarized in the text below by setting and in terms of SOE by outcome in Supplemental Table 2.

Table 1.
Effectiveness in inpatient, outpatient and emergency care.

Clinical topic (number of studies) Patient outcomes (reference)^a Provider outcomes^a Payer outcomes^a

Inpatient care

Multiple conditions (4) ∼ Mortality in hospital ^52,53
∼ Transfers ⁵²
∼ Length of stay ^52,53
∼ Readmission ⁵³ ∼ Drug prescribing ⁵⁴
+ Communication ratings ⁵² + Hospital revenue ⁵⁵
+ Professional billing revenue ⁵⁵

Infectious disease (3) + Mortality ⁵⁶
∼ Transfers ⁵⁶
– Length of stay ⁵⁶^b
∼ Length of stay ⁵⁷
∼ 30-day readmission ⁵⁶ + Provider satisfaction ⁵⁶
+ Improved antimicrobial use or infection rate ^57,58
∼ Antibiotic use ⁵⁷
+ Appropriate prescribing and adherence to guidelines ⁵⁷ None reported

Stroke (1) ∼ Length of stay ⁵⁹ None reported + Cost ⁵⁹

Spinal fracture (1) + Length of stay ⁶⁰ + Knowledge, skills, confidence ⁶⁰ None reported

Mental and behavioral disorders (1) + Transfers ⁶¹ None reported None reported

Neonates (5) + Transfers ^62,63
∼ Length of stay ^64,65
∼ Enteral feeding ^64,65
∼ Ventilation/oxygen ^64,65
∼ Proportion of deliveries at community hospitals ⁶⁶
+ Statewide infant mortality to 1 year and telehealth hospital death before discharge ⁶²
∼ Morbidity ⁶² None reported None reported

ICU (2) ∼ Mortality in high dependency unit ⁶⁷
∼ Mortality in hospital ⁶⁷
∼ Mortality total ⁶⁷
+ Mortality 90-day ⁶⁸
+ Transfers ⁶⁷ None reported None reported

Outpatient Care

Depression (3) M Response (varies by time point [6, 12, 18 months]) ^69–75
M Remission (varies by time point [6, 12, 18 months]) ^69–75
∼ Depression-free days ^69–75
∼ Depression symptom score (BDI) 3 and 6 months ^69–75
M Quality of Life (varies by time point [6, 12, 18 months] and scale [Quality of Well-being Scale and SF-36 Mental]) ^69–75
∼ Treatment adherence 3 months,+ 6 months ^69–75
+ Medication adherence 6 and 12 months ^69–75 None reported ∼ Utilization: overall,+ outpatient only^69,70,73,74
∼ Total costs ^70,73
– Adjusted total cost ⁷²
+ Incremental cost/ depression-free days ⁷²
– Depression-related primary and mental health costs ^70,73

PTSD (1) + Prescribed any medication for PTSD ⁷⁶
∼ Adherence to medication regimen >80% ⁷⁶
∼ Any psychiatric encounter at 12 months ⁷⁶
+ Mean number of CPT sessions at 12 months ⁷⁶
M Quality of Life (varies by scale [Quality of Well-being Scale and SF-36 Mental and Physical scales]) ⁷⁶ None reported – Total outpatient costs ^76,77

ADHD (1) ∼ Teacher ratings (VADRS) ⁷⁸
∼ Caregiver ratings (VADRS-role performance and CIS-P scale) ⁷⁸ None reported None reported

Diabetes (4) + A1c ^79–82 and self-monitoring of blood glucose ⁸²
+ Diabetes Self-Management Education score ^79,82
∼ Total cholesterol ^79,82 and systolic blood pressure ⁸¹ None reported None reported

Diabetic neuropathy (1) + Screening rate ⁸³ + Screening referral or reminder ⁸³ None reported

Hepatitis C (1) ∼ Sustained virologic response ⁸⁴
+ Completion of therapy ⁸⁴
∼ Mean weeks of therapy ⁸⁴
+ Mean face-to-face visits ⁸⁴
+ Anemia and withdrawal due to adverse events ⁸⁴ None reported None reported

Rheumatology (2) + Distance to visit ⁸⁵
∼ Quality of life, disease-related function and visit satisfaction ⁸⁶ None reported + Cost of visit ⁸⁵

Oncology (2) ∼ Receipt of radiotherapy ⁸⁷ None reported + Total costs ⁸⁸

Echocardiography (1) + Total process time ⁸⁹
+ Time to specialist consultation ⁸⁹ None reported None reported

Hemodialysis (1) + Amount of urea removed ⁹⁰
∼ Other clinical measures ⁹⁰ None reported None reported

Fracture clinic (1) None reported None reported + Cost compared to transfer or sending a specialist ⁹¹

Dementia care (1) ∼ Yearly change in MMSE: all patients ⁹²
+ Yearly change in MMSE: patients with baseline MMSE 15–30 ⁹² None reported None reported

Ulcer care (1) + Time from referral to appointment ⁹³
+ Leg ulcer healing time ⁹³ None reported ∼ Cost per patient ⁹³

Pharmacy (2) ∼ A1c and systolic BP: patients with diabetes or HTN ⁹⁴
+ A1c and systolic BP: patients with diabetes and HTN ⁹⁴ + Guideline adherence ⁹⁵ None reported

Remote consultation (3) + Hospital visits ⁹⁶
+ Patient function
+ Medication adherence ⁹⁷
+ Health-related quality of life ⁹⁷ ∼ Overall referrals or to eConsult specialties ⁹⁸ None reported

Dermatology (2) ∼ Clinical course ⁹⁹ None reported M Cost per participant ¹⁰⁰

Endoscopy (1) None reported None reported ∼ Costs ¹⁰¹

Blood pressure (1) + Blood pressure ¹⁰²
+ Stroke recurrence ¹⁰² None reported None reported

Palliative care (1) ∼ Patient symptoms and function
+ Nurse or general practitioner visits, hospital admissions None reported None reported

Ultrasound for pregnancy (1) + Increase in comprehensive ultrasound for all and high-risk pregnancies ¹⁰³ None reported None reported

Emergency care

Stroke
(10)
TeleED
Video-conference ∼ In-hospital, 30-day and 90-day mortality ^51,104–107
+ 30-day all-cause mortality, super rural patients ⁵¹
∼ Symptom onset to tPA time ^51,104–111
+ tPA within 3 h of symptom onset ^{51,104–106,108–111}
∼ 90-day modified Rankin Scale ^104,106
∼ Post-tPA intracranial hemorrhage ^104,106,107
M tPA use ^{51,104–106,108–111}
∼ Length of stay, overall and rural patients ^{51,104–106,108–111}
+ Length of stay, super rural patients ⁵¹
+ Appropriate transfer to high-volume center or higher level of care ¹⁰⁹
+ Discharge to home or rehab ^107,112 + Correctness of decision-making/accurate triage ^104,112 – Total medical expenditures per event ⁵¹

Heart attack/STEMI
(2)
TeleECG + On time and time to treatment ^113,114 None reported None reported

Heart attack/STEMI
(1)
Text triage + Time to hospital arrival ¹¹⁵
M Arrival-to-balloon time ¹¹⁵
∼ False-positive STEMI ¹¹⁵ None reported None reported

Chest pain/ MI
(2)
TeleED videoconference M Time from ED arrival to ECG ^108,116
∼ Time from ED arrival to fibrinolytic ^108,116
+ Likelihood of receiving fibrinolytic when eligible ¹¹⁶ None reported None reported

Multiple
(1)
TeleED robot + Transfer ¹¹⁷
+ Length of stay (robot/transfer vs. no robot/no transfer) ¹¹⁷ None reported None reported

Multiple
(4)
TeleED video-conference ∼ Patient deaths in ER ¹¹⁸
M Transfer to another facility ^118,119
+ Admit to provider time ¹²⁰
– ED length of stay, non-transferred patients ¹²⁰
+ ED length of stay, transferred patients ¹²⁰
– Likelihood of discharge from ED ¹¹⁸
– Rural hospital admission ¹¹⁸
– Discharged AMA from ED ¹¹⁸ None reported ∼ Total ED patient volume ¹¹⁸
+ Lower ED costs and operating expenses ¹²¹

Critical care
(3)
TeleED video-conference + Transfer avoided ¹²²
+ Transfer to lower level of care ¹²²
+ Parent/guardian satisfaction ¹²³
+ Accuracy of clinical picture prior to arrival ¹²² + Physician-related ED medication errors ¹²⁴
+ Quality of care scores ¹²³
+ Referring physician satisfaction ¹²³ None reported

Trauma
(1)
TeleED video-conference ∼ Mortality
+ Chest tube, intubation ¹²⁵
+ ED length of stay ¹²⁵
M Increased diagnostic imaging ¹²⁵
+ Time to arrival at final hospital ¹²⁵
∼ Transfer ¹²⁵ None reported None reported

Hand trauma
(1)
TeleED video-conference ∼ Length of stay at first hospital ¹²⁶
∼ Type of transfer, air or ground ¹²⁶
+ Admitted from ED ¹²⁶ None reported ∼ Transport cost

Sepsis/septic shock
(1)
TeleED video-conference None reported + Adherence to sepsis treatment bundle ¹²⁷ None reported

Behavioral health; suicidal ideation or attempt
(2)
TeleED video-conference + ED wait time ¹²⁸
– ED length of stay ¹²⁸^b
+ Hospital admission ¹²⁹
+ Involuntary hold placement ¹²⁹
∼ 30-day readmission ¹²⁹ None reported None reported

Sexual abuse
(1)
TeleED video-conference None reported + Quality, completeness, accuracy of abuse examination forms¹³⁰ None reported

AMA: against medical advice; BP: blood pressure; CIS-P: Columbia impairment scale-parent version; ECG: electrocardiogram; ED: emergency department; HTN: hypertension; ICER: incremental cost-effectiveness ratio; ICU: intensive care unit; MMSE: mini-mental state examination; PTSD: post-traumatic stress disorder; SF-36: 36 item short form survey; STEMI: ST-elevated myocardial infarction; tPA: tissue plasminogen activator; VADRS: Vanderbilt Attention-Deficit/Hyperactivity Disorder Rating Scale.

^a
Symbol meaning: +: Improved outcome with telehealth; ∼: Similar outcome with telehealth; –: Worse outcome with telehealth; M: Outcomes were not consistent across studies.

^b
Length of stay was longer with telehealth, and longer stays are usually considered a negative outcome. However mortality decreased which may indicate that more care was appropriate. The authors note that the consultations tended to occur later in the hospital stay in this study and a study in which the consultation happened sooner would provide clearer evidence about the impact on length of stay.

Effectiveness for inpatient care

Seventeen studies, reported in 18 articles, evaluated RPPT for inpatient care (Table 1). The majority (n = 12) were studies that compared outcomes before and after the implementation of telehealth,^{52,54–62,66,67} two (reported in three articles) were prospective cohort studies,^63,64,131 and three were retrospective cohort studies.^53,65,68 Eleven were conducted in the United States,^{52,53,55,56,58,62–66,68} five in Australia,^{54,57,60,61,67} and one in Scotland.⁵⁹ Risk of bias was rated as high for six studies^{52,57,58,60,62,66} and medium for 11 studies.^{53–56,59,61,63–65,67,68}

Table 2.
Summary of evidence of effectiveness for provider education and mentoring.

Modality Clinical topic (number of studies) Patient outcomes^a Provider outcomes^a

ECHO video-conference Antibiotic therapy (1) ∼ In-hospital mortality and length of stay ¹³² + Antibiotic prescribing ¹³²

COVID-19 in long-term care (1) None reported + Self-efficacy and satisfaction ¹³³

Dementia (1) None reported + Comfort with assessment and treatment¹³⁴

Diabetes (2) + A1c ¹³⁵ + Self-efficacy in patient coaching/education; identification of psychosocial treatment barriers ¹³⁶

Liver disease (3) ∼ Sustained viral response ¹³⁷
∼ Serious adverse events ¹³⁷
∼ Access to direct acting antiviral treatment in rural areas ¹³⁸ + Hepatitis C virus awareness, knowledge, abilities, and intention to screen at-risk patients ¹³⁹

Mental health (5) None reported + Opioid use disorder diagnosis/prescribing ¹⁴⁰
+ Reduction in patients prescribed ≥3 psychotropic medications ¹⁴¹
+ Provider knowledge and self-efficacy ¹⁴²
+ General development and Autism-specific screening ¹⁴³
+ Pediatric behavioral health knowledge and patient management ¹⁴¹
+ Satisfaction with sessions ^141,142,144

Non-ECHO video-conference Childhood obesity (1) ∼ Child nutrition and physical activity ¹⁴⁵ + Documentation and counseling ¹⁴⁵
∼ Family centered care ¹⁴⁵

Dermatology (1) None reported + Knowledge of and ability to provide dermatology procedures ¹⁴⁶
∼ Ability to provide liquid nitrogen ¹⁴⁶

Mental health (1) None reported + Confidence in managing behavioral and psychological symptoms of dementia ¹⁴⁷
+ Satisfaction with educational program ¹⁴⁷

Pediatric burns (1) None reported + Knowledge of burn prevention, first aid, airway/inhalation injury, and circulation and fluid resuscitation ¹⁴⁸
M Chemical and electrical burns, burn wound, pain and itch management ¹⁴⁸

Perioperative care (1) None reported ∼ Perioperative training scores ¹⁴⁹
∼ Rating program as a success ¹⁴⁹

Online education course Mental health (3) None reported + Role adequacy, legitimacy, and support ¹⁵⁰
M Role motivation, work satisfaction, and task-specific self-esteem ¹⁵⁰
+ Education completion ¹⁵¹
+ Knowledge, skills, confidence, and utilization of CBT ¹⁵¹
+ Computer and internet-related skills ¹⁵²
+ Knowledge about mental health service, confidence in responding to mental health problems ¹⁵²

Low vision screening (1) None reported + Knowledge of low vision screening and treatment ¹³⁸

Short messaging service Multiple conditions (1) None reported ∼ Medical knowledge ¹⁵³
+ Satisfaction with intervention ¹⁵³

A1c: glycated hemoglobin; CBT: cognitive-behavioral therapy; ECHO: Extension for Community Healthcare Outcomes.^44,154

^a
Symbol meaning: +: Improved outcome with telehealth; ∼: Similar outcome with telehealth; –: Worse outcome with telehealth; M: Outcomes were not consistent across studies.

Summarizing these studies, we found evidence that telehealth consultations in rural hospitals resulted in no difference in length of hospital stay (six studies; low SOE) or transfers (three studies; low SOE) compared with usual care, which was in-person or phone consultations. Additionally, telehealth-supported care for neonates at rural hospitals resulted in no difference in clinical outcomes when compared to transfer and care at a hospital with a Level 4 neonatal intensive care unit (two studies; Low SOE). Also, mortality rates were not different when patients were treated in rural hospitals with remote ICUs rather than transported to more distant hospitals (two studies; low SOE).

Effectiveness for outpatient care

Thirty-two studies (in 35 publications) evaluated the use of RPPT interventions to support outpatient care.^{69–74,76–82,84–103,155} Sixteen studies (reported in 17 publications) were randomized controlled trials (RCTs),^{69–73,76–79,81,86,89,95,98–100,102} six were prospective cohort studies,^{74,83,85,92,96,155} six were retrospective cohort studies,^{80,84,87,88,91,101} four were pre-post study designs (same group measured before and after implementation),^82,90,94,97 and two were before-after studies (different groups/systems measured before and after implementation).^93,103 Eleven of the studies were conducted in the United States;^{69–71,76,78,82,84,85,94,95,99,100,103} the remainder were conducted in Canada,^86,90,98 Australia,^83,88,91,155 Korea,^79,92,97 China,¹⁰² Denmark,⁸⁰ New Zealand,⁸⁷ Spain,⁹⁶ United Kingdom,^93,101 Chile,⁷⁴ Taiwan,⁸¹ and Sweden.⁸⁹ Risk of bias was rated as high for eight studies,^{80,82,89,91,93,97,101,103} medium for 21 studies,^{71,74,76,78,79,81,83–88,90,92,94–96,99,100,155,156} and low for two studies.^98,102

RPPT for outpatient care varied widely across interventions and outcomes (Table 1). Outpatient telehealth consultations with specialists in various clinical specialties (e.g. diabetes, depression, outpatient medication management) resulted in improvements in clinical outcomes compared with care without specialist involvement. For patients with diabetes, RPPT resulted in improvements in A1c and self-management in patients with diabetes but had no effect on blood pressure or cholesterol levels (4 studies; Low SOE). In patients with depression, RPPT was associated with some improvement in treatment response, medication adherence, and satisfaction (three studies; Low SOE), and higher utilization and corresponding costs for outpatient consultations due to increased access, resulting in overall benefit based on cost-effectiveness analyses (two studies; Low SOE). Studies found that outpatient telepharmacy consultations improved guideline adherence and patient outcomes but only for patients with both diabetes and hypertension (two studies; Low SOE).

Effectiveness for emergency care

We included 28 studies of RPPT for emergency care, either by emergency medical services (EMS) or EDs.^51,104–130 Two of these studies were RCTs,^104,112 10 were prospective cohort studies,^{106,108,110,111,113–117,127} eight were retrospective cohort studies,^{51,121–125,129,130} and eight were studies that compared outcomes before and after telehealth initiation.^{105,107,109,118–120,126,128} Eighteen of these studies were conducted in the United States,^{51,104,108,111,116,118–130} two were conducted in Italy,^113,114 two in Australia,^105,107 and one each in Canada,¹¹⁷ Spain,¹¹⁰ Finland,¹⁰⁶ Turkey,¹¹⁵ and Japan.¹⁰⁹ Risk of bias was rated as high for one study,¹⁰⁸ medium for 26 studies,^{51,105–107,109–130} and low for one RCT.¹⁰⁴

Telehealth consultations for emergency assessment and initial care of stroke, heart attack, or chest pain at rural hospitals resulted in similar rates of mortality (five studies; Low SOE) and similar time to treatment when patients were treated locally as opposed to transferred, suggesting telehealth did not cause delays (eight studies; Low SOE). Telehealth consultations by specialists for critical care and trauma patients in rural EDs had a generally positive impact on transfers, with results reporting fewer unnecessary transfers, more appropriate transfers, or similar rates compared with care without RPPT (five studies; Low SOE).

Effectiveness for education and mentoring

Twenty-three studies evaluated RPPT for education and mentoring (Table 2),^{132–153,157} including three RCTs^144,151,153 and 20 observational studies. Observational study designs included pre-post,^{132–136,138,139,141–143,146,148,150,152} prospective cohort,^137,149 and retrospective cohort studies.^140,156 All studies were conducted across multiple clinical sites or health care organizations. Sixteen studies were performed in the United States,^{132,134–141,143–146,149,150,157} four in Australia,^{147,148,151,152} one in Canada,¹⁴² and one in Vietnam.¹⁵³ One study was rated low risk of bias,¹⁴² 11 were rated medium risk of bias,^{132,133,137,138,140,144,145,149,151,153,157} and 11 were rated high risk of bias.^{134–136,139,141,143,146–148,150,152}

Extension for Community Healthcare Outcomes (ECHO) programs is a model that uses video for remote instruction and case-based learning and was designed to promote best practices in rural healthcare^158,159 and has been use to expand access to specialty care.¹⁶⁰ ECHO programs were associated with better or equivalent patient outcomes (two studies; Low SOE), including reduction in A1c in patients of trainees after ECHO compared with before participation (one study), and similar Hepatitis C viral response and serious adverse events rates at the “spoke” site with ECHO participation to those at an academic medical center (one study). ECHO and non-ECHO video training programs (a) resulted in desired changes in provider behavior (e.g. increased appropriate prescribing practices, screening, and patient counseling) (eight studies; Low SOE) and (b) were associated with increased provider confidence, efficacy, and scores on knowledge tests (ten studies; Low SOE).

Implementation of provider-to-provider telehealth for rural patients

Sixty-seven studies in 71 publications addressed the implementation of RPPT.^{49,59,118,161–227} Most of these studies were program evaluations that combined data from several sources, such as site visits, observations, surveys, and interviews. A smaller number were qualitative research studies that analyzed interviews, focus groups, or documents and then categorized or cataloged specific barriers and facilitators to initial implementation, ongoing operations, longer-term sustainment, or spread of the use of telehealth.

Information on the barriers and facilitators was first recorded as described by the study authors. Then, because studies used different terms, we mapped these using CFIR constructs (mapping tables are provided in the full report) to standardize our description and allow us to summarize and compare these across studies. Identified barriers and facilitators mapped to 19 of the 39 constructs that make up CFIR.

Figure 2 illustrates how the 219 facilitators and 192 barriers we identified map to the CFIR constructs. Each construct and the number of times it occurred in our data is included in the label, and the width of the line represents the relative frequency. Short definitions of each construct are included as notes to the figure; comprehensive descriptions are available in CFIR articles and tools.^44,154 The figure illustrates the point that most constructs can either facilitate or impede implementation, depending both on context and their presence or absence. The two most frequently cited constructs in our analysis were Available Resources and Access to Knowledge & Information. Available Resources is a broad concept including the wide range of investments an organization dedicates to implementing or sustaining an innovation or program. This was cited 60 times, and almost equally split between mention as a facilitator and as a barrier. Access to Knowledge & Information was cited 57 times, most frequently, though not exclusively, as a facilitator. We repeated the analysis by health care setting (available in the full report). Barriers and facilitators were similar across different settings. Assessed and summarized using CFIR constructs, the included implementation studies and program evaluations provided information about what is needed to translate knowledge about effectiveness of RPPT into actual practice.

Figure 2.
Facilitators and barriers to implementation*.

Many facilitators and barriers were not specific to rural settings. The studies highlight that telehealth needs to alleviate burden on providers, the technology needs to work, staff resources and reimbursement need to be allocated to provide both start-up infrastructure and ongoing support, and training of both support staff and clinicians is needed. Engagement from a range of stakeholders from patients to health system leadership and governments was cited in studies as essential. Some issues were raised in identified studies that are specific to rural programs. In the United States, insufficient internet in rural areas remains a persistent barrier. Lack of understanding of the rural environment by urban-based consultants and educators risks reducing the utility of teleconsultations and remote training programs. Remote consultants for the care of rural patients can be used for either frequent events or serve as a resource for rare events in rural healthcare, but needed technology and procedures are different in these two cases and technology and operations needed to be tailored to frequency of use. Most RPPT programs are local or in a single health system, yet sustainment requires long-term commitment and resources, on a scale that may not be feasible for smaller rural organizations.

Methodologic challenges in studying provider-to-provider telehealth for rural patients

We abstracted limitations cited by the authors and combined these with our risk of bias and applicability assessments to identify and categorize the methodological weaknesses of the available evidence. Studies did not routinely control for confounders related to patients, providers, facilities, and differences in telehealth implementation across study sites, and have been hampered by data limitations, such as missing data or inaccuracies in data collected for reasons other than research. In our assessment, it is often difficult to attribute impact to telehealth because less rigorous study designs, rather than RCTs or prospective, well-designed observational studies, were used in more than two-thirds of the included studies (Table 3).

Table 3.
Study designs and number of sites by setting for effectiveness studies.

Inpatient Outpatient EMS/ED Education/Mentoring Total

Study design

RCT 0 15 2 3 20

Prospective cohort 2 6 10 2 20

Retrospective cohort 3 6 8 2 19

Pre/Post^a 0 4 0 14 18

Before/after^b 13 3 8 2 26

Number of sites

Multisite 13 27 22 23 85

Single center 5 7 6 0 18

Total 18 34 28 23 103

ED: emergency department; EMS: emergency medical services; RCT: randomized controlled trial.

^a
Same patients or providers evaluated pre and post-intervention.

^b
Do not have the same patients or providers in both time points.

Discussion

The research on RPPT for collaboration in the delivery of rural health care provided an evidence base that addressed questions about use, effectiveness, implementation, and methodological weaknesses, but the evidence was uneven and could not support definitive, universally applicable conclusions.

To answer Key Question 1, we identified seven studies that reported national or regional trends of increasing telehealth use, with differences across specialties and geographic locations. Increasing use was evident even before the onset of the COVID-19 pandemic. For effectiveness (Key Question 2), we identified and synthesized studies that assessed the impact of RPPT and found evidence that outcomes were better or similar with RPPT for applications in inpatient, outpatient, and emergency care. We also summarized studies demonstrating that telehealth for rural provider education and mentoring, including ECHO programs, may improve patient outcomes, change provider behavior, and increase provider knowledge and confidence in treating specific conditions. To address Key Question 3, we categorized barriers and facilitators, finding most were common to practice change initiatives, though some were specific to the rural context. We identified important methodological weaknesses (Key Question 4) in the RPPT effectiveness studies, including less rigorous study designs and small sample sizes.

Our synthesis of the available evidence was qualitative because the modes, functions, and outcomes studied and how they are measured were heterogeneous. We grouped studies of similar topics that assessed similar outcomes allowing these conclusions. However, there were several instances where only one study was identified, or two or more studies reported conflicting results leading to an assessment of “unclear effect” or “insufficient evidence.” Classification of barriers and facilitators to implementation in 67 studies confirmed that common barriers for change in practice, including inadequate provider time, technology, and other resources, are limiting the spread of RPPT. However, there are also specific barriers, such as incomplete understanding of rural context and lack of long-term commitments to maintaining the infrastructure and staffing needed. Our assessment of the methodological weaknesses of the effectiveness studies found these frequently employed less rigorous designs, had small sample sizes, and often did not minimize possible bias through design or analytic approach.

Our review had limitations associated with our methods. We only included studies published in English about research conducted in developed countries. Searching for studies of RPPT was challenging as telehealth is a broad term and studies do not consistently include “rural” in titles or abstracts. We were unable to conduct quantitative synthesis (i.e. meta-analyses) and our qualitative synthesis combined studies with similar, but not identical, outcomes.

There were also limitations due to the nature of the evidence base. Research on telehealth in general is often not based on a clear model of how telehealth is expected to affect outcomes and whether telehealth needs to produce outcomes that are better than standard care or if equivalence with in-person alternatives is sufficient. It is often unclear if the goal of telehealth is to provide care that is as good as care provided without telehealth or if the investment in telehealth requires outcomes to be better. While telehealth should increase patient and provider satisfaction, there is no agreement on how to prioritize clinical outcomes, resource use, costs, and potential harms.

Our assessment of the methodological weaknesses of the included studies suggests directions for future research. Additional research is needed to measure outcomes at multiple time points or over longer periods of time, as short-term outcomes may differ from longer-term outcomes (e.g. provider retention of knowledge acquired through remote education and mentoring). Additional trials would strengthen the evidence base, as would observational studies that include contemporaneous comparison groups and multiple sites. The former would allow the impact of telehealth to be separated from historical change or the potentially unique characteristics of specific sites or providers/consultants. More complete descriptions are needed of both telehealth interventions and comparators. Clear statements of the intended impact of telehealth would help inform assessments of fit and help clarify whether telehealth was designed to replace in-person services or add additional services.

The existing evidence base was insufficient to allow us to unequivocally endorse all potential uses of RPPT as tools for improving health care for rural patients. Nevertheless, the studies we identified and summarized do not report harm or negative consequences. More importantly, they suggest it is likely that the application of telehealth can improve patient outcomes such as access to and quality of care, provider outcomes such as knowledge and self-efficacy, and payer outcomes such as reduced costs or maintenance of payments to rural providers.

Supplemental Material

sj-docx-1-jtt-10.1177_1357633X221139892 - Supplemental material for Telehealth-guided provider-to-provider communication to improve rural health: A systematic review

Supplemental material, sj-docx-1-jtt-10.1177_1357633X221139892 for Telehealth-guided provider-to-provider communication to improve rural health: A systematic review by Annette M Totten, Dana M Womack, Jessica C Griffin, Marian S McDonagh, Cynthia Davis-O'Reilly, Ian Blazina, Sara Grusing, and Nancy Elder in Journal of Telemedicine and Telecare

Clinical topic (number of studies)	Patient outcomes (reference)^a	Provider outcomes^a	Payer outcomes^a
Inpatient care
Multiple conditions (4)	∼ Mortality in hospital ^52,53 ∼ Transfers ⁵² ∼ Length of stay ^52,53 ∼ Readmission ⁵³	∼ Drug prescribing ⁵⁴ + Communication ratings ⁵²	+ Hospital revenue ⁵⁵ + Professional billing revenue ⁵⁵
Infectious disease (3)	+ Mortality ⁵⁶ ∼ Transfers ⁵⁶ – Length of stay ⁵⁶^b ∼ Length of stay ⁵⁷ ∼ 30-day readmission ⁵⁶	+ Provider satisfaction ⁵⁶ + Improved antimicrobial use or infection rate ^57,58 ∼ Antibiotic use ⁵⁷ + Appropriate prescribing and adherence to guidelines ⁵⁷	None reported
Stroke (1)	∼ Length of stay ⁵⁹	None reported	+ Cost ⁵⁹
Spinal fracture (1)	+ Length of stay ⁶⁰	+ Knowledge, skills, confidence ⁶⁰	None reported
Mental and behavioral disorders (1)	+ Transfers ⁶¹	None reported	None reported
Neonates (5)	+ Transfers ^62,63 ∼ Length of stay ^64,65 ∼ Enteral feeding ^64,65 ∼ Ventilation/oxygen ^64,65 ∼ Proportion of deliveries at community hospitals ⁶⁶ + Statewide infant mortality to 1 year and telehealth hospital death before discharge ⁶² ∼ Morbidity ⁶²	None reported	None reported
ICU (2)	∼ Mortality in high dependency unit ⁶⁷ ∼ Mortality in hospital ⁶⁷ ∼ Mortality total ⁶⁷ + Mortality 90-day ⁶⁸ + Transfers ⁶⁷	None reported	None reported
Outpatient Care
Depression (3)	M Response (varies by time point [6, 12, 18 months]) ^69–75 M Remission (varies by time point [6, 12, 18 months]) ^69–75 ∼ Depression-free days ^69–75 ∼ Depression symptom score (BDI) 3 and 6 months ^69–75 M Quality of Life (varies by time point [6, 12, 18 months] and scale [Quality of Well-being Scale and SF-36 Mental]) ^69–75 ∼ Treatment adherence 3 months,+ 6 months ^69–75 + Medication adherence 6 and 12 months ^69–75	None reported	∼ Utilization: overall,+ outpatient only^69,70,73,74 ∼ Total costs ^70,73 – Adjusted total cost ⁷² + Incremental cost/ depression-free days ⁷² – Depression-related primary and mental health costs ^70,73
PTSD (1)	+ Prescribed any medication for PTSD ⁷⁶ ∼ Adherence to medication regimen >80% ⁷⁶ ∼ Any psychiatric encounter at 12 months ⁷⁶ + Mean number of CPT sessions at 12 months ⁷⁶ M Quality of Life (varies by scale [Quality of Well-being Scale and SF-36 Mental and Physical scales]) ⁷⁶	None reported	– Total outpatient costs ^76,77
ADHD (1)	∼ Teacher ratings (VADRS) ⁷⁸ ∼ Caregiver ratings (VADRS-role performance and CIS-P scale) ⁷⁸	None reported	None reported
Diabetes (4)	+ A1c ^79–82 and self-monitoring of blood glucose ⁸² + Diabetes Self-Management Education score ^79,82 ∼ Total cholesterol ^79,82 and systolic blood pressure ⁸¹	None reported	None reported
Diabetic neuropathy (1)	+ Screening rate ⁸³	+ Screening referral or reminder ⁸³	None reported
Hepatitis C (1)	∼ Sustained virologic response ⁸⁴ + Completion of therapy ⁸⁴ ∼ Mean weeks of therapy ⁸⁴ + Mean face-to-face visits ⁸⁴ + Anemia and withdrawal due to adverse events ⁸⁴	None reported	None reported
Rheumatology (2)	+ Distance to visit ⁸⁵ ∼ Quality of life, disease-related function and visit satisfaction ⁸⁶	None reported	+ Cost of visit ⁸⁵
Oncology (2)	∼ Receipt of radiotherapy ⁸⁷	None reported	+ Total costs ⁸⁸
Echocardiography (1)	+ Total process time ⁸⁹ + Time to specialist consultation ⁸⁹	None reported	None reported
Hemodialysis (1)	+ Amount of urea removed ⁹⁰ ∼ Other clinical measures ⁹⁰	None reported	None reported
Fracture clinic (1)	None reported	None reported	+ Cost compared to transfer or sending a specialist ⁹¹
Dementia care (1)	∼ Yearly change in MMSE: all patients ⁹² + Yearly change in MMSE: patients with baseline MMSE 15–30 ⁹²	None reported	None reported
Ulcer care (1)	+ Time from referral to appointment ⁹³ + Leg ulcer healing time ⁹³	None reported	∼ Cost per patient ⁹³
Pharmacy (2)	∼ A1c and systolic BP: patients with diabetes or HTN ⁹⁴ + A1c and systolic BP: patients with diabetes and HTN ⁹⁴	+ Guideline adherence ⁹⁵	None reported
Remote consultation (3)	+ Hospital visits ⁹⁶ + Patient function + Medication adherence ⁹⁷ + Health-related quality of life ⁹⁷	∼ Overall referrals or to eConsult specialties ⁹⁸	None reported
Dermatology (2)	∼ Clinical course ⁹⁹	None reported	M Cost per participant ¹⁰⁰
Endoscopy (1)	None reported	None reported	∼ Costs ¹⁰¹
Blood pressure (1)	+ Blood pressure ¹⁰² + Stroke recurrence ¹⁰²	None reported	None reported
Palliative care (1)	∼ Patient symptoms and function + Nurse or general practitioner visits, hospital admissions	None reported	None reported
Ultrasound for pregnancy (1)	+ Increase in comprehensive ultrasound for all and high-risk pregnancies ¹⁰³	None reported	None reported
Emergency care
Stroke (10) TeleED Video-conference	∼ In-hospital, 30-day and 90-day mortality ^51,104–107 + 30-day all-cause mortality, super rural patients ⁵¹ ∼ Symptom onset to tPA time ^51,104–111 + tPA within 3 h of symptom onset ^{51,104–106,108–111} ∼ 90-day modified Rankin Scale ^104,106 ∼ Post-tPA intracranial hemorrhage ^104,106,107 M tPA use ^{51,104–106,108–111} ∼ Length of stay, overall and rural patients ^{51,104–106,108–111} + Length of stay, super rural patients ⁵¹ + Appropriate transfer to high-volume center or higher level of care ¹⁰⁹ + Discharge to home or rehab ^107,112	+ Correctness of decision-making/accurate triage ^104,112	– Total medical expenditures per event ⁵¹
Heart attack/STEMI (2) TeleECG	+ On time and time to treatment ^113,114	None reported	None reported
Heart attack/STEMI (1) Text triage	+ Time to hospital arrival ¹¹⁵ M Arrival-to-balloon time ¹¹⁵ ∼ False-positive STEMI ¹¹⁵	None reported	None reported
Chest pain/ MI (2) TeleED videoconference	M Time from ED arrival to ECG ^108,116 ∼ Time from ED arrival to fibrinolytic ^108,116 + Likelihood of receiving fibrinolytic when eligible ¹¹⁶	None reported	None reported
Multiple (1) TeleED robot	+ Transfer ¹¹⁷ + Length of stay (robot/transfer vs. no robot/no transfer) ¹¹⁷	None reported	None reported
Multiple (4) TeleED video-conference	∼ Patient deaths in ER ¹¹⁸ M Transfer to another facility ^118,119 + Admit to provider time ¹²⁰ – ED length of stay, non-transferred patients ¹²⁰ + ED length of stay, transferred patients ¹²⁰ – Likelihood of discharge from ED ¹¹⁸ – Rural hospital admission ¹¹⁸ – Discharged AMA from ED ¹¹⁸	None reported	∼ Total ED patient volume ¹¹⁸ + Lower ED costs and operating expenses ¹²¹
Critical care (3) TeleED video-conference	+ Transfer avoided ¹²² + Transfer to lower level of care ¹²² + Parent/guardian satisfaction ¹²³ + Accuracy of clinical picture prior to arrival ¹²²	+ Physician-related ED medication errors ¹²⁴ + Quality of care scores ¹²³ + Referring physician satisfaction ¹²³	None reported
Trauma (1) TeleED video-conference	∼ Mortality + Chest tube, intubation ¹²⁵ + ED length of stay ¹²⁵ M Increased diagnostic imaging ¹²⁵ + Time to arrival at final hospital ¹²⁵ ∼ Transfer ¹²⁵	None reported	None reported
Hand trauma (1) TeleED video-conference	∼ Length of stay at first hospital ¹²⁶ ∼ Type of transfer, air or ground ¹²⁶ + Admitted from ED ¹²⁶	None reported	∼ Transport cost
Sepsis/septic shock (1) TeleED video-conference	None reported	+ Adherence to sepsis treatment bundle ¹²⁷	None reported
Behavioral health; suicidal ideation or attempt (2) TeleED video-conference	+ ED wait time ¹²⁸ – ED length of stay ¹²⁸^b + Hospital admission ¹²⁹ + Involuntary hold placement ¹²⁹ ∼ 30-day readmission ¹²⁹	None reported	None reported
Sexual abuse (1) TeleED video-conference	None reported	+ Quality, completeness, accuracy of abuse examination forms¹³⁰	None reported

Modality	Clinical topic (number of studies)	Patient outcomes^a	Provider outcomes^a
ECHO video-conference	Antibiotic therapy (1)	∼ In-hospital mortality and length of stay ¹³²	+ Antibiotic prescribing ¹³²
COVID-19 in long-term care (1)	None reported	+ Self-efficacy and satisfaction ¹³³
Dementia (1)	None reported	+ Comfort with assessment and treatment¹³⁴
Diabetes (2)	+ A1c ¹³⁵	+ Self-efficacy in patient coaching/education; identification of psychosocial treatment barriers ¹³⁶
Liver disease (3)	∼ Sustained viral response ¹³⁷ ∼ Serious adverse events ¹³⁷ ∼ Access to direct acting antiviral treatment in rural areas ¹³⁸	+ Hepatitis C virus awareness, knowledge, abilities, and intention to screen at-risk patients ¹³⁹
Mental health (5)	None reported	+ Opioid use disorder diagnosis/prescribing ¹⁴⁰ + Reduction in patients prescribed ≥3 psychotropic medications ¹⁴¹ + Provider knowledge and self-efficacy ¹⁴² + General development and Autism-specific screening ¹⁴³ + Pediatric behavioral health knowledge and patient management ¹⁴¹ + Satisfaction with sessions ^141,142,144
Non-ECHO video-conference	Childhood obesity (1)	∼ Child nutrition and physical activity ¹⁴⁵	+ Documentation and counseling ¹⁴⁵ ∼ Family centered care ¹⁴⁵
Dermatology (1)	None reported	+ Knowledge of and ability to provide dermatology procedures ¹⁴⁶ ∼ Ability to provide liquid nitrogen ¹⁴⁶
Mental health (1)	None reported	+ Confidence in managing behavioral and psychological symptoms of dementia ¹⁴⁷ + Satisfaction with educational program ¹⁴⁷
Pediatric burns (1)	None reported	+ Knowledge of burn prevention, first aid, airway/inhalation injury, and circulation and fluid resuscitation ¹⁴⁸ M Chemical and electrical burns, burn wound, pain and itch management ¹⁴⁸
Perioperative care (1)	None reported	∼ Perioperative training scores ¹⁴⁹ ∼ Rating program as a success ¹⁴⁹
Online education course	Mental health (3)	None reported	+ Role adequacy, legitimacy, and support ¹⁵⁰ M Role motivation, work satisfaction, and task-specific self-esteem ¹⁵⁰ + Education completion ¹⁵¹ + Knowledge, skills, confidence, and utilization of CBT ¹⁵¹ + Computer and internet-related skills ¹⁵² + Knowledge about mental health service, confidence in responding to mental health problems ¹⁵²
Low vision screening (1)	None reported	+ Knowledge of low vision screening and treatment ¹³⁸
Short messaging service	Multiple conditions (1)	None reported	∼ Medical knowledge ¹⁵³ + Satisfaction with intervention ¹⁵³

	Inpatient	Outpatient	EMS/ED	Education/Mentoring	Total
Study design
RCT	0	15	2	3	20
Prospective cohort	2	6	10	2	20
Retrospective cohort	3	6	8	2	19
Pre/Post^a	0	4	0	14	18
Before/after^b	13	3	8	2	26
Number of sites
Multisite	13	27	22	23	85
Single center	5	7	6	0	18
Total	18	34	28	23	103

Footnotes

Acknowledgements

AHRQ staff, an NIH ODP working group, an NIH-convened content area expert group, and a technical expert panel helped refine the project scope. The draft report was presented at an NIH ODP P2P workshop. Experts in the field, AHRQ and NIH partners, and the public reviewed earlier drafts of the full report. The investigators are solely responsible for the contents of this manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This project was funded by the National Institutes of Health (NIH) through an interagency agreement with the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services (HHS) under Contract No. 75Q80120D00006/Task Order 75Q80120F32001. The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ or NIH. Therefore, no statement in this report should be construed as an official position of AHRQ, NIH, or HHS.

ORCID iD

Annette M Totten

Supplemental material

Supplemental material for this article is available online.

References

Rural health. Health Aff (Millwood) 2019; 38: 1964–1965. DOI: 10.1377/hlthaff.2019.01365.

Galambos

. Health care disparities among rural populations: A neglected frontier. Health Soc Work 2005; 30: 179–181.

Garcia

Faul

Massetti

, et al. Reducing potentially excess deaths from the five leading causes of death in the rural United States. MMWR Surveill Summ 2017; 66: 1–7.

Singh

Siahpush

. Widening rural-urban disparities in all-cause mortality and mortality from major causes of death in the USA, 1969-2009. J Urban Health 2014; 91: 272–292.

Garcia

Rossen

Bastian

, et al. Potentially excess deaths from the five leading causes of death in metropolitan and nonmetropolitan counties - United States, 2010-2017. MMWR Surveill Summ 2019; 68: 1–11.

Hall

Moonesinghe

Bouye

, et al. Racial/ethnic disparities in mortality: contributions and variations by rurality in the United States, 2012-2015. Int J Environ Res Public Health 2019; 16: 436.

Villapiano

Iwashyna

Davis

. Worsening rural-urban gap in hospital mortality. J Am Board Fam Med 2017; 30: 816–823.

Keyes

Cerdá

Brady

, et al. Understanding the rural-urban differences in nonmedical prescription opioid use and abuse in the United States. Am J Public Health 2014; 104: e52–e59.

Monnat

Rigg

. Examining rural/urban differences in prescription opioid misuse among US adolescents. J Rural Health 2016; 32: 204–218.

10.

Boring

Hootman

Liu

, et al. Prevalence of arthritis and arthritis-attributable activity limitation by urban-rural county classification - United States, 2015. MMWR Morb Mortal Wkly Rep 2017; 66: 527–532.

11.

Croft

Wheaton

Liu

, et al. Urban-rural county and state differences in chronic obstructive pulmonary disease - United States, 2015. MMWR Morb Mortal Wkly Rep 2018; 67: 205–211.

12.

Krishna

Gillespie

McBride

. Diabetes burden and access to preventive care in the rural United States. J Rural Health 2010; 26: 3–11.

13.

Primm

Ferdinand

Callaghan

, et al. Congestive heart failure-related hospital deaths across the urban-rural continuum in the United States. Prev Med Rep 2019; 16: 101007.

14.

Nelson

Kinder

Johnson

, et al. Differences in selected HIV care continuum outcomes among people residing in rural, urban, and metropolitan areas-28 US jurisdictions. J Rural Health 2018; 34: 63–70.

15.

Zahnd

Rodriguez

Jenkins

. Rural-urban differences in human papillomavirus-associated cancer trends and rates. J Rural Health 2019; 35: 208–215.

16.

Probst

Moore

Glover

, et al. Person and place: the compounding effects of race/ethnicity and rurality on health. Am J Public Health 2004; 94: 1695–1703.

17.

Austin

Glaeser

Summers

. Saving the heartland: place-based policies in 21st century America. In: BPEA conference draft, Spring 2018.

18.

Porter

. The hard truths of trying to “save” the rural economy, https://www.nytimes.com/interactive/2018/12/14/opinion/rural-america-trump-decline.html (2018, accessed 9 September 2020).

19.

Tuckson

Edmunds

Hodgkins

. Telehealth. N Engl J Med 2017; 377: 1585–1592.

20.

Castro

Miller

Nager

. Unlocking the potential of physician-to-patient telehealth services, https://itif.org/publications/2014/05/12/unlocking-potential-physician-patient-telehealth-services (2014, accessed 5 September 2020).

21.

Lustig

. The role of telehealth in an evolving health care environment - workshop summary. Washington DC: The National Academies Press. 2012.

22.

Bashshur

Shannon

Smith

, et al. The empirical foundations of telemedicine interventions for chronic disease management. Telemed e-Health 2014; 20: 769–800.

23.

Ekeland

Bowes

Flottorp

. Effectiveness of telemedicine: a systematic review of reviews. Int J Med Inf 2010; 79: 736–771.

24.

Hersh

Wallace

Patterson

, et al. Telemedicine for the medicare population-evidence reports/technology assessments, No. 24. Rockville, MD. July 2001.

25.

Hersh

Hickman

Severance

, et al. Telemedicine for the Medicare population: update - evidence reports/technology assessments, No. 131. Rockville, MD. February 2006.

26.

Hersh

Wallace

Patterson

, et al. Telemedicine for the medicare population: pediatric, obstetric, and clinician-indirect home interventions. Evid Rep Technol Assess (Summ) 2001; 24 Suppl: 1–32.

27.

Totten

Womack

Eden

, et al. Telehealth: mapping the evidence for patient outcomes from systematic reviews - technical brief No. 26. Rockville, MD: Pacific Northwest Evidence-based Practice Center, 2016.

28.

Snoswell

Chelberg

De Guzman

, et al. The clinical effectiveness of telehealth: a systematic review of meta-analyses from 2010 to 2019. J Telemed Telecare 2021: 1357633x211022907. DOI: 10.1177/1357633x211022907.

29.

Snoswell

Stringer

Taylor

, et al. An overview of the effect of telehealth on mortality: a systematic review of meta-analyses. J Telemed Telecare 2021: 1357633x211023700. DOI: 10.1177/1357633x211023700.

30.

Scott Kruse

Karem

Shifflett

, et al. Evaluating barriers to adopting telemedicine worldwide: a systematic review. J Telemed Telecare 2018; 24: 4–12.

31.

Adler-Milstein

Kvedar

Bates

. Telehealth among US hospitals: several factors, including state reimbursement and licensure policies, influence adoption. Health Aff (Millwood) 2014; 33: 207–215.

32.

Broderick

Lindeman

. Scaling telehealth programs: lessons from early adopters. New York, NY: Commonwealth Fund, http://www.commonwealthfund.org/Publications/Case-Studies/2013/Jan/Telehealth-Synthesis.aspx (2013, accessed 8 July 2016).

33.

Rae

Cox

Claxton

. Coverage and utilization of telemedicine services by enrollees in large employer plans, https://www.healthsystemtracker.org/brief/coverage-and-utilization-of-telemedicine-services-by-enrollees-in-large-employer-plans/ (2020, accessed 5 September 2020).

34.

Smith

Thomas

Snoswell

, et al. Telehealth for global emergencies: implications for coronavirus disease 2019 (COVID-19). J Telemed Telecare 2020; 26: 309–313.

35.

Thomas

Haydon

Mehrotra

, et al. Building on the momentum: sustaining telehealth beyond COVID-19. J Telemed Telecare 2022; 28: 301–308.

36.

Gallegos-Rejas

Thomas

Kelly

, et al. A multi-stakeholder approach is needed to reduce the digital divide and encourage equitable access to telehealth. J Telemed Telecare 2022: 1357633x221107995. DOI: 10.1177/1357633x221107995.

37.

Shah

Sall

Peng

, et al. Exploring the role of telehealth in providing equitable healthcare to the vulnerable patient population during COVID-19. J Telemed Telecare 2022: 1357633x221113711. DOI: 10.1177/1357633x221113711.

38.

Cameron

Ray

Sabesan

. Physicians’ perceptions of clinical supervision and educational support via videoconference: a systematic review. J Telemed Telecare 2014; 20: 272–281.

39.

McBain

Sousa

Rose

, et al. Impact of project ECHO models of medical tele-education: a systematic review. J Gen Intern Med 2019; 34: 2842–2857.

40.

Totten

Hansen

Wagner

, et al. Telehealth for acute and chronic care consultations. Rockville, MD: Agency for Healthcare Research and Quality (US), 2019.

41.

Mary W, Joanne MC, Sara M, et al. National institutes of health pathways to prevention workshop: Improving rural health through telehealth-guided provider-to-provider communication. J Telemedicine Telecare 2022. DOI: 10.1177/1357633X221139630.

42.

Berkman

Lohr

Ansari

, et al. Grading the strength of a body of evidence when assessing health care interventions: an EPC update. J Clin Epidemiol 2015; 68: 1312–1324.

43.

United Nations Development Programme. Human development report 2020: the next frontier: human development and the Anthropocene. New York, NY: United Nations Development Programme, 2020.

44.

Damschroder

Aron

Keith

, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009; 4: 50.

45.

Alishahi Tabriz

Turner

Williams

, et al. Association of financial factors and telemedicine adoption for heart attack and stroke care among rural and urban hospitals: a longitudinal study. Telemed J E Health 2021; 24: 24.

46.

Chen

Amaize

Barath

. Evaluating telehealth adoption and related barriers among hospitals located in rural and urban areas. J Rural Health 2021; 37: 801–811.

47.

Freeman

Boggs

Zachrison

, et al. National study of telepsychiatry use in U.S. Emergency departments. Psychiatr Serv 2020; 71: 540–546.

48.

Spivak

Cullen

, et al. Telepsychiatry use in U.S. Mental health facilities, 2010-2017. Psychiatr Serv 2020; 71: 121–127.

49.

Zachrison

Boggs

Hayden

, et al. Understanding barriers to telemedicine implementation in rural emergency departments. Ann Emerg Med 2020; 75: 392–399.

50.

Zachrison

Hayden

Schwamm

, et al. Characterizing New England emergency departments by telemedicine use. West J Emerg Med 2017; 18: 1055–1060.

51.

Zhang

Wang

Zhu

, et al. Expansion of telestroke services improves quality of care provided in super rural areas. Health Aff (Millwood) 2018; 37: 2005–2013.

52.

Kuperman

Linson

Klefstad

, et al. The virtual hospitalist: a single-site implementation bringing hospitalist coverage to critical access hospitals. J Hosp Med 2018; 13: 759–763.

53.

Boltz

Cuellar

Cole

, et al. Comparing an on-site nurse practitioner with telemedicine physician support hospitalist programme with a traditional physician hospitalist programme. J Telemed Telecare 2019; 25: 213–220.

54.

Poulson

Nissen

Coombes

. Pharmaceutical review using telemedicine--a before and after feasibility study. J Telemed Telecare 2010; 16: 95–99.

55.

Dharmar

Sadorra

Leigh

, et al. The financial impact of a pediatric telemedicine program: a children’s hospital’s perspective. Telemed e-Health 2013; 19: 502–508.

56.

Tande

Berbari

Ramar

, et al. Association of a remotely offered infectious diseases eConsult service with improved clinical outcomes. Open Forum Infect Dis 2020; 7: 1–4.

57.

Avent

Walker

Yarwood

, et al. Implementation of a novel antimicrobial stewardship strategy for rural facilities utilising telehealth. Int J Antimicrob Agents 2021; 57: 106346.

58.

Yam

Fales

Jemison

, et al. Implementation of an antimicrobial stewardship program in a rural hospital. Am J Health-Syst Pharm 2012; 69: 1142–1148.

59.

Barber

Frieslick

Maclean

, et al. The western isles stroke telerehabilitation (specialist medical consultation) service–implementation and evaluation. Eur Res Telemed 2015; 4: 19–24.

60.

Gallagher

Giles

Morison

, et al. Telehealth-based model of care redesign to facilitate local fitting and management of patients with a spinal fracture requiring a thoracic lumbar sacral orthosis in rural hospitals in New South Wales. Aust J Rural Health 2018; 26: 181–187.

61.

Buckley

Weisser

. Videoconferencing could reduce the number of mental health patients transferred from outlying facilities to a regional mental health unit. Aust N Z J Public Health 2012; 36: 478–482.

62.

Kim

Teague-Ross

Greenfield

, et al. Telemedicine collaboration improves perinatal regionalization and lowers statewide infant mortality. J Perinatol 2013; 33: 725–730.

63.

Haynes

Dharmar

Hill

, et al. The impact of telemedicine on transfer rates of newborns at rural community hospitals. Acad Pediatr 2020; 20: 636–641.

64.

Makkar

McCoy

Hallford

, et al. Evaluation of neonatal services provided in a level II NICU utilizing hybrid telemedicine: a prospective study. Telemed e-Health 2020; 26: 176–183.

65.

Makkar

McCoy

Hallford

, et al. A hybrid form of telemedicine: a unique way to extend intensive care service to neonates in medically underserved areas. Telemed e-Health 2018; 24: 717–721.

66.

Hall

Hall-Barrow

Garcia-Rill

. Neonatal regionalization through telemedicine using a community-based research and education core facility. Ethn Dis 2010; 20: S1–136–140.

67.

Panlaqui

Broadfield

Champion

, et al. Outcomes of telemedicine intervention in a regional intensive care unit: A before and after study. Anaesth Intensive Care 2017; 45: 605–610.

68.

Kahn

Barnato

, et al. ICU Telemedicine and critical care mortality: a national effectiveness study. Med Care 2016; 54: 319–325.

69.

Davis

Deen

Bryant-Bedell

, et al.

Does minority racial-ethnic status moderate outcomes of collaborative care for depression?

Psychiatr Serv 2011; 62: 1282–1288.

70.

Fortney

Maciejewski

Tripathi

, et al. A budget impact analysis of telemedicine-based collaborative care for depression. Med Care 2011; 49: 872–880.

71.

Fortney

Pyne

Mouden

, et al. Practice-based versus telemedicine-based collaborative care for depression in rural federally qualified health centers: a pragmatic randomized comparative effectiveness trial. Focus 2017; 15: 361–372.

72.

Pyne

Fortney

Mouden

, et al. Cost-effectiveness of on-site versus off-site collaborative care for depression in rural FQHCs. Psychiatr Serv 2015; 66: 491–499.

73.

Pyne

Fortney

Tripathi

, et al. Cost-effectiveness analysis of a rural telemedicine collaborative care intervention for depression. Arch Gen Psychiatry 2010; 67: 812–821.

74.

Rojas

Guajardo

Martinez

, et al. A remote collaborative care program for patients with depression living in rural areas: open-label trial. J Med Internet Res 2018; 20: e158.

75.

Fortney

Pyne

Edlund

, et al. A randomized trial of telemedicine-based collaborative care for depression. J Gen Intern Med 2007; 22: 1086–1093.

76.

Fortney

Pyne

Kimbrell

, et al. Telemedicine-based collaborative care for posttraumatic stress disorder: a randomized clinical trial. JAMA Psychiatry 2015; 72: 58–67.

77.

Painter

Fortney

Austen

, et al. Cost-effectiveness of telemedicine-based collaborative care for posttraumatic stress disorder. Psychiatr Serv 2017; 68: 1157–1163.

78.

Myers

Vander Stoep

Zhou

, et al. Effectiveness of a telehealth service delivery model for treating attention-deficit/hyperactivity disorder: a community-based randomized controlled trial. J Am Acad Child Adolesc Psychiatry 2015; 54: 263–274.

79.

Cho

Kwon

Kim

, et al. Effects on diabetes management of a health-care provider mediated, remote coaching system via a PDA-type glucometer and the internet. J Telemed Telecare 2011; 17: 365–370.

80.

Levin

Madsen

Petersen

, et al. Telemedicine diabetes consultations are cost-effective, and effects on essential diabetes treatment parameters are similar to conventional treatment: 7-year results from the Svendborg Telemedicine Diabetes Project. J Diabetes Sci Technol 2013; 7: 587–595.

81.

Liou

Soon

Chen

, et al. Shared care combined with telecare improves glycemic control of diabetic patients in a rural underserved community. Telemed e-Health 2014; 20: 175–178.

82.

McLendon

Wood

Stanley

. Enhancing diabetes care through care coordination, telemedicine, and education: evaluation of a rural pilot program. Public Health Nurs 2019; 36: 310–320.

83.

Crossland

Askew

Ware

, et al. Diabetic retinopathy screening and monitoring of early stage disease in Australian general practice: tackling preventable blindness within a chronic care model. J Diabetes Res 2016; 2016: 8405395.

84.

Rossaro

Torruellas

Dhaliwal

, et al. Clinical outcomes of hepatitis C treated with pegylated interferon and ribavirin via telemedicine consultation in Northern California. Dig Dis Sci 2013; 58: 3620–3625.

85.

Wood

Caplan

. Outcomes, satisfaction, and costs of a rheumatology telemedicine program: a longitudinal evaluation. J Clin Rheumatol 2019; 25: 41–44.

86.

Taylor-Gjevre

Nair

Bath

, et al. Addressing rural and remote access disparities for patients with inflammatory arthritis through video-conferencing and innovative inter-professional care models. Musculoskelet 2018; 16: 90–95.

87.

Stevens

Loh

Kolbe

, et al. Comparison of recommendations for radiotherapy from two contemporaneous thoracic multidisciplinary meeting formats: co-located and video conference. Intern Med J 2012; 42: 1213–1218.

88.

Thaker

Monypenny

Olver

, et al. Cost savings from a telemedicine model of care in northern Queensland, Australia. Med J Aust 2013; 199: 414–417.

89.

Boman

Olofsson

Berggren

, et al. Robot-assisted remote echocardiographic examination and teleconsultation: a randomized comparison of time to diagnosis with standard of care referral approach. JACC Cardiovasc Imaging 2014; 7: 799–803.

90.

Sicotte

Moqadem

Vasilevsky

, et al. Use of telemedicine for haemodialysis in very remote areas: the Canadian first nations. J Telemed Telecare 2011; 17: 146–149.

91.

McGill

North

. Teleconference fracture clinics: a trial for rural hospitals. ANZ J Surg 2012; 82: 2–3.

92.

Kim

Jhoo

Jang

. The effect of telemedicine on cognitive decline in patients with dementia. J Telemed Telecare 2017; 23: 149–154.

93.

Summerhayes

McGee

Cooper

, et al. Introducing leg ulcer telemedicine into rural general practice. Wounds UK 2012; 8: 28–36.

94.

Anderson

Axon

Taylor

, et al. Impact evaluation of a four-year academic-community partnership in provision of medication management and tertiary prevention services for rural patients with diabetes and/or hypertension. Prev Med Rep 2020; 17: 101038.

95.

Carter

Levy

Gryzlak

, et al. Cluster-randomized trial to evaluate a centralized clinical pharmacy service in private family medicine offices. Circ Cardiovasc Qual Outcomes 2018; 11: e004188.

96.

De Batlle

Massip

Vargiu

, et al. Implementing mobile health-enabled integrated care for complex chronic patients: intervention effectiveness and cost-effectiveness study. JMIR Mhealth Uhealth 2021; 9: e22135.

97.

Kwak

Hwang

Lee

. Effects of the physician-primary-healthcare nurse telemedicine model (P-NTM) on medication adherence and health-related quality of life (HRQoL) of patients with chronic disease at remote rural areas. Int J Environ Res Public Health 2021; 18: 2502.

98.

Liddy

Moroz

Keely

, et al. Understanding the impact of a multispecialty electronic consultation service on family physician referral rates to specialists: a randomized controlled trial using health administrative data. Trials 2019; 20: 348.

99.

Whited

Warshaw

Kapur

, et al. Clinical course outcomes for store and forward teledermatology versus conventional consultation: a randomized trial. J Telemed Telecare 2013; 19: 197–204.

100.

Datta

Warshaw

Edison

, et al. Cost and utility analysis of a store-and-forward teledermatology referral system: a randomized clinical trial. JAMA Dermatol 2015; 151: 1323–1329.

101.

Van der Pol

McKenzie

. Costs and benefits of tele-endoscopy clinics in a remote location. J Telemed Telecare 2010; 16: 89–94.

102.

Yan

Gong

, et al. Effectiveness of a primary care-based integrated mobile health intervention for stroke management in rural China (SINEMA): a cluster-randomized controlled trial. PLoS Med 2021; 18: e1003582.

103.

Long

Angtuaco

Lowery

. Ultrasound in telemedicine: its impact in high-risk obstetric health care delivery. Ultrasound Q 2014; 30: 167–172.

104.

Demaerschalk

Raman

Ernstrom

, et al. Efficacy of telemedicine for stroke: pooled analysis of the stroke team remote evaluation using a digital observation camera (STRokE DOC) and STRokE DOC Arizona telestroke trials. Telemed e-Health 2012; 18: 230–237.

105.

Nagao

Koschel

Haines

, et al. Rural Victorian telestroke project. Intern Med J 2012; 42: 1088–1095.

106.

Sairanen

Soinila

Nikkanen

, et al. Two years of Finnish Telestroke: thrombolysis at spokes equal to that at the hub. Neurology 2011; 76: 1145–1152.

107.

Bladin

Moloczij

Ermel

, et al. Victorian Stroke Telemedicine Project: implementation of a new model of translational stroke care for Australia. Intern Med J 2015; 45: 951–956.

108.

Mohr

Young

Harland

, et al. Telemedicine is associated with faster diagnostic imaging in stroke patients: a cohort study. Telemed e-Health 2019; 25: 93–100.

109.

Nagayoshi

Oshima

Ogawa

. Clinical impact of telemedicine network system at rural hospitals without on-site cardiac surgery backup. Telemed e-Health 2016; 22: 960–964.

110.

Pedragosa

Alvarez-Sabin

Rubiera

, et al. Impact of telemedicine on acute management of stroke patients undergoing endovascular procedures. Cerebrovasc Dis 2012; 34: 436–442.

111.

Swanson

Miller

Ward

, et al. Emergency department telemedicine consults decrease time to interpret computed tomography of the head in a multi-network cohort. J Telemed Telecare 2021; 27: 343–352.

112.

Helwig

Ragoschke-Schumm

Schwindling

, et al. Prehospital stroke management optimized by use of clinical scoring vs mobile stroke unit for triage of patients with stroke: a randomized clinical trial. JAMA Neurol 2019; 76: 1484–1492.

113.

Brunetti

Dell’Anno

Martone

, et al. Prehospital ECG transmission results in shorter door-to-wire time for STEMI patients in a remote mountainous region. Am J Emerg Med 2020; 38: 252–257.

114.

Brunetti

Di Pietro

Aquilino

, et al. Pre-hospital electrocardiogram triage with tele-cardiology support is associated with shorter time-to-balloon and higher rates of timely reperfusion even in rural areas: data from the Bari- Barletta/Andria/Trani public emergency medical service 118 registry on primary angioplasty in ST-elevation myocardial infarction. Europ Heart J Acute Cardiovasc Care 2014; 3: 204–213.

115.

Astarcioglu

Sen

Kilit

, et al. Time-to-reperfusion in STEMI undergoing interhospital transfer using smartphone and WhatsApp messenger. Am J Emerg Med 2015; 33: 1382–1384.

116.

Miller

Ward

Ullrich

, et al. Emergency department telemedicine consults are associated with faster time-to-electrocardiogram and time-to-fibrinolysis for myocardial infarction patients. Telemed e-Health 2020; 26: 1440–1448.

117.

Holt

Sari

Hansen

, et al. Remote presence robotic technology reduces need for pediatric interfacility transportation from an isolated northern community. Telemed e-Health 2018; 24: 927–933.

118.

Sterling

Seals

Jones

, et al. The impact of the TelEmergency program on rural emergency care: an implementation study. J Telemed Telecare 2017; 23: 588–594.

119.

Natafgi

Mohr

Wittrock

, et al. The association between telemedicine and emergency department (ED) disposition: a stepped wedge design of an ED-based telemedicine program in critical access hospitals. J Rural Health 2020; 36: 360–370.

120.

Mohr

Young

Harland

, et al. Emergency department telemedicine shortens rural time-to-provider and emergency department transfer times. Telemed e-Health 2018; 24: 582–593.

121.

Williams

Simpson

King

, et al.

Do hospitals providing telehealth in emergency departments have lower emergency department costs?

Telemed e-Health 2020; 27: 1011–1020.

122.

Harvey

Yeager

Cramer

, et al. The impact of telemedicine on pediatric critical care triage. Pediatr Crit Care Med 2017; 18: e555–e560.

123.

Dharmar

Romano

Kuppermann

, et al. Impact of critical care telemedicine consultations on children in rural emergency departments. Crit Care Med 2013; 41: 2388–2395.

124.

Dharmar

Kuppermann

Romano

, et al. Telemedicine consultations and medication errors in rural emergency departments. Pediatrics 2013; 132: 1090–1097.

125.

Mohr

Vakkalanka

Harland

, et al. Telemedicine use decreases rural emergency department length of stay for transferred North Dakota trauma patients. Telemed e-Health 2018; 24: 194–202.

126.

Tripod

Tait

Bracey

, et al. The use of telemedicine decreases unnecessary hand trauma transfers. Hand 2020; 15: 422–427.

127.

Mohr

Campbell

Swanson

, et al. Provider-o-provider telemedicine improves adherence to sepsis bundle care in community emergency departments. J Telemed Telecare 2021; 27: 518–526.

128.

Fairchild

Ferng-Kuo

Laws

, et al. Telehealth decreases rural emergency department wait times for behavioral health patients in a group of critical access hospitals. Telemed e-Health 2019; 25: 1154–1164.

129.

Vakkalanka

Harland

Wittrock

, et al. Telemedicine is associated with rapid transfer and fewer involuntary holds among patients presenting with suicidal ideation in rural hospitals: a propensity matched cohort study. J Epidemiol Community Health 2019; 73: 1033–1039.

130.

Miyamoto

Dharmar

Boyle

, et al. Impact of telemedicine on the quality of forensic sexual abuse examinations in rural communities. Child Abuse Negl 2014; 38: 1533–1539.

131.

Haynes

Hoffman

Patel

, et al. The use of telemedicine for stabilization of neonates transferred from rural community hospitals. Telemed e-Health 2021; 27: 1393–1398.

132.

Wilson

Banks

Crnich

, et al. Changes in antibiotic use following implementation of a telehealth stewardship pilot program. Infect Control Hosp Epidemiol 2019; 40: 810–814.

133.

Lingum

Sokoloff

Meyer

, et al. Building long-term care staff capacity during COVID-19 through just-in-time learning: evaluation of a modified ECHO model. J Am Med Dir Assoc 2021; 22: 238–244.e1.

134.

Lindauer

Wild

Natonson

, et al. Dementia 360 ECHO: using technology to facilitate diagnosis and treatment. Gerontol Geriatr Educ 2022; 43: 202–208.

135.

Watts

Roush

Julius

, et al. Improved glycemic control in veterans with poorly controlled diabetes mellitus using a specialty care access network-extension for community healthcare outcomes model at primary care clinics. J Telemed Telecare 2016; 22: 221–224.

136.

Bouchonville

Hager

Kirk

, et al. Endo ECHO improves primary care provider and community health worker self-efficacy in complex diabetes management in medically underserved communities. Endocr Pract 2018; 24: 40–46.

137.

Arora

Thornton

Murata

, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. N Engl J Med 2011; 364: 2199–2207.

138.

Nipp

Vogtle

Warren

. Clinical application of low vision rehabilitation strategies after completion of a computer-based training module. Occup Ther Health Care 2014; 28: 296–305.

139.

Momin

Mera

Essex

, et al. Implementation of liver cancer education among health care providers and community coalitions in the Cherokee nation. Prev Chronic Dis 2019; 16: 1–7.

140.

Gadomski

Anderson

Chung

, et al. Full agonist opioid prescribing by primary care clinicians after buprenorphine training. Subst Abus 2022; 43: 69–75.

141.

Hostutler

Valleru

Maciejewski

, et al. Improving pediatrician’s behavioral health competencies through the project ECHO teleconsultation model. Clin Pediatr (Phila) 2020; 59: 1049–1057.

142.

Sockalingam

Arena

Serhal

, et al. Building provincial mental health capacity in primary care: an evaluation of a project ECHO mental health program. Acad Psychiatry 2018; 42: 451–457.

143.

Bellesheim

Kizzee

Curran

, et al. ECHO Autism: integrating maintenance of certification with extension for community healthcare outcomes improves developmental screening. J Dev Behav Pediatr 2020; 41: 420–427.

144.

Zittleman

Curcija

Sutter

, et al. Building capacity for medication assisted treatment in rural primary care practices: the IT MATTTRs practice team training. J Prim Care Community Health 2020; 11: 2150132720953723.

145.

Shaikh

Nettiksimmons

Joseph

, et al. Collaborative practice improvement for childhood obesity in rural clinics: the healthy eating active living telehealth community of practice (HEALTH COP). Am J Med Qual 2014; 29: 467–475.

146.

McFarland

Raugi

Taylor

, et al. Implementation of an education and skills programme in a teledermatology project for rural veterans. J Telemed Telecare 2012; 18: 66–71.

147.

Doyle

Jackson

Loi

, et al. Videoconferencing and telementoring about dementia care: evaluation of a pilot model for sharing scarce old age psychiatry resources. Int Psychogeriatr 2016; 28: 1567–1574.

148.

McWilliams

Hendricks

Twigg

, et al. Burns education for non-burn specialist clinicians in Western Australia. Burns 2015; 41: 301–307.

149.

Seibert

Reddy

Whitmore

, et al. The use of telemedicine to train perioperative nurses in rural settings. J Telemed Telecare 2013; 19: 311–314.

150.

Puskar

Heeyoung

Mitchell

, et al. Interprofessional collaborative education for substance use screening: rural areas and challenges. Online J Rural Nurs Health Care 2016; 16: 76–96.

151.

Bennett-Levy

Hawkins

Perry

, et al. Online cognitive behavioural therapy training for therapists: outcomes, acceptability, and impact of support. Aust Psychol 2012; 47: 174–182.

152.

Robinson

Hills

Kelly

. The evaluation of an online orientation to rural mental health practice in Australia. J Psychiatr Ment Health Nurs 2011; 18: 629–636.

153.

Gill

Le Ngoc

Halim

, et al. The mCME project: a randomized controlled trial of an SMS-based continuing medical education intervention for improving medical knowledge among Vietnamese community based physicians’ assistants. PLoS One 2016; 11: e0166293.

154.

CFIR Research Team-Center for Clinical Management Research. Consolidated Framework for Implementation Research. (2021, accessed 12 July 2021).

155.

Jiang

Bills

Poon

. Integrated telehealth-assisted home-based specialist palliative care in rural Australia: a feasibility study. J Telemed Telecare 2020: 1357633X20966466. DOI: 10.1177/1357633X20966466.

156.

Fortney

Pyne

Mouden

157.

Tran

Feldman

Riley

III , et al. Association of the extension for community healthcare outcomes project with use of direct-acting antiviral treatment among US adults with hepatitis C. JAMA Netw Open 2021; 4: e2115523.

158.

Arora

Geppert

Kalishman

, et al. Academic health center management of chronic diseases through knowledge networks: project ECHO. Acad Med 2007; 82: 154–160.

159.

Arora

Thornton

Jenkusky

, et al. Project ECHO: Linking university specialists with rural and prison-based clinicians to improve care for people with chronic hepatitis C in New Mexico. Public Health Rep 2007; 122: 74–77.

160.

Arora

Kalishman

Dion

, et al. Partnering urban academic medical centers and rural primary care clinicians to provide complex chronic disease care. Health Aff (Millwood) 2011; 30: 1176–1184.

161.

Katzman

Gygi

Swift

, et al. How hands-on pain skills intensive trainings complement ECHO pain and opioid management programs: a program evaluation with the Indian Health Service. Pain Med 2020; 21: 1769–1778.

162.

Zhu

Merchant

KAS

Mohr

, et al. Real-time learning through telemedicine enhances professional training in rural emergency departments. Telemed e-Health 2021; 27: 441–447.

163.

Weigel

Bhagianadh

Merchant

, et al. Tele-emergency behavioural health in rural and underserved areas. J Telemed Telecare 2021; 27: 453–462.

164.

Weigel

Merchant

Wittrock

, et al. Paediatric tele-emergency care: a study of two delivery models. J Telemed Telecare 2021; 27: 23–31.

165.

Weaver

Neumann

Navaneethan

, et al. Human touch via touchscreen: rural nurses’ experiential perspectives on telehealth use in pediatric hospice care. J Pain Symptom Manage 2020; 60: 1027–1033.

166.

Shea

Gertner

Green

. Barriers and perceived usefulness of an ECHO intervention for office-based buprenorphine treatment for opioid use disorder in North Carolina: a qualitative study. Subst Abus 2021; 42: 54–64.

167.

Peracca

Jackson

Lamkin

, et al. Implementing teledermatology for rural veterans: an evaluation using the RE-AIM framework. Telemed e-Health 2021; 27: 218–226.

168.

Mahmoud

Vogt

Dahdouh

, et al. Using continuous quality improvement to design and implement a telepsychiatry program in rural Illinois. Psychiatr Serv 2020; 71: 860–863.

169.

Brunet

Moore

Lendvai Wischik

, et al. Increasing buprenorphine access for veterans with opioid use disorder in rural clinics using telemedicine. Subst Abus 2022; 43: 39–46.

170.

Simpson

Harvey

DiLembo

, et al. Population health indicators associated with a statewide telestroke program. Telemed e-Health 2020; 26: 1126–1133.

171.

Tynan

Deeth

McKenzie

, et al. Integrated approach to oral health in aged care facilities using oral health practitioners and teledentistry in rural Queensland. Aust J Rural Health 2018; 26: 290–294.

172.

Adcock

Choi

Alvi

, et al. Expanding acute stroke care in rural America: a model for statewide success. Telemed e-Health 2020; 26: 865–871.

173.

Lesher

Fakhry

DuBose-Morris

, et al. Development and evolution of a statewide outpatient consultation service: leveraging telemedicine to improve access to specialty care. Popul Health Manag 2020; 23: 20–28.

174.

Thies

Anderson

Beals-Reid

. Project ECHO chronic pain: a qualitative analysis of recommendations by expert faculty. Pain Med 2019; 20: 1450–1452.

175.

Jewer

Parsons

Dunne

, et al. Evaluation of a mobile telesimulation unit to train rural and remote practitioners on high-acuity low-occurrence procedures: pilot randomized controlled trial. J Med Internet Res 2019; 21: e14587.

176.

White

Crowther

Lee

. Supporting rural midwifery practice using a mobile health (mHealth) intervention: a qualitative descriptive study. Rural Remote Health 2019; 19: 5294.

177.

Cronin

. Implementing a stroke program using telemedicine. J Emerg Nurs 2013; 39: 613–618.

178.

Marsh-Feiley

Eadie

Wilson

. Paramedic and physician perspectives on the potential use of remotely supported prehospital ultrasound. Rural Remote Health 2018; 18: 4574.

179.

Sabesan

Senko

Schmidt

, et al. Enhancing chemotherapy capabilities in rural hospitals: implementation of a telechemotherapy model (QReCS) in North Queensland, Australia. J Oncol Pract 2018; 14: e429–e437.

180.

Tynan

Deeth

McKenzie

. An integrated oral health program for rural residential aged care facilities: a mixed methods comparative study. BMC Health Serv Res 2018; 18: 515.

181.

Hensel

Yang

Rai

, et al. Optimizing electronic consultation between primary care providers and psychiatrists: mixed-methods study. J Med Internet Res 2018; 20: e124.

182.

Trondsen

Tjora

Broom

, et al. The symbolic affordances of a video-mediated gaze in emergency psychiatry. Soc Sci Med 2018; 197: 87–94.

183.

Ness

Annese

Martinez-Paz

, et al. Using an innovative telehealth model to support community providers who deliver perinatal HIV care. AIDS Educ Prev 2017; 29: 516–526.

184.

Bagot

Cadilhac

Kim

, et al. Transitioning from a single-site pilot project to a state-wide regional telehealth service: the experience from the Victorian Stroke Telemedicine programme. J Telemed Telecare 2017; 23: 850–855.

185.

Newell

Strauss

Freier

, et al. Design and initial results of the Minneapolis Heart Institute TeleHeart program. Circ Cardiovasc Qual Outcomes 2017; 10: 1–4.

186.

Narva

Romancito

Faber

, et al. Managing CKD by telemedicine: the Zuni telenephrology clinic. Adv Chronic Kidney Dis 2017; 24: 6–11.

187.

Williams

Kirsh

Aron

, et al. Evaluation of the Veterans Health Administration’s specialty care transformational initiatives to promote patient-centered delivery of specialty care: a mixed-methods approach. Telemed e-Health 2017; 23: 577–589.

188.

Al Kasab

Adams

Debenham

, et al. Medical university of South Carolina telestroke: a telemedicine facilitated network for stroke treatment in South Carolina-a progress report. Telemed e-Health 2017; 23: 674–677.

189.

Ray

Felmet

Hamilton

, et al. Clinician attitudes toward adoption of pediatric emergency telemedicine in rural hospitals. Pediatr Emerg Care 2017; 33: 250–257.

190.

Evans

Lerch

Boyce

, et al. An innovative approach to enhancing access to medical screening for miners using a mobile clinic with telemedicine capability. J Health Care Poor Underserved 2016; 27: 62–72.

191.

Hofmeyer

Leider

Satorius

, et al. Implementation of telemedicine consultation to assess unplanned transfers in rural long-term care facilities, 2012-2015: a pilot study. J Am Med Dir Assoc 2016; 17: 1006–1010.

192.

Ward

Ullrich

MacKinney

, et al.

Tele-emergency utilization: In what clinical situations is tele-emergency activated?

J Telemed Telecare 2016; 22: 25–31.

193.

Jhaveri

Larkins

Kelly

, et al. Remote chemotherapy supervision model for rural cancer care: perspectives of health professionals. Eur J Cancer Care (Engl) 2016; 25: 93–98.

194.

Pimentel

Gately

Barczi

, et al. GRECC Connect: geriatrics telehealth to empower health care providers and improve management of older veterans in rural communities. Fed Pract 2019; 36: 464–470.

195.

Habashi

Bouchard

Nguyen

. Transforming access to specialist care for inflammatory bowel disease: the PACE telemedicine program. J Can Assoc Gastroenterol 2019; 2: 186–194.

196.

Alschuler

Stobbe

Hertz

, et al. Impact of multiple sclerosis project ECHO (extension for community healthcare outcomes) on provider confidence and clinical practice. Int J MS Care 2019; 21: 143–150.

197.

Pindyck

Kalishman

Flatow-Trujillo

, et al. Treating hepatitis C in American Indians/Alaskan Natives: a survey of project ECHO(R) (extension for community healthcare outcomes) utilization by Indian Health Service providers. Sage Open Med 2015; 3: 2050312115612805.

198.

Singh

Mathiassen

Switzer

, et al. Assimilation of web-based urgent stroke evaluation: a qualitative study of two networks. JMIR Med Inform 2014; 2: e6.

199.

MacKinney

Ward

Ullrich

, et al. The business case for tele-emergency. Telemed e-Health 2015; 21: 1005–1011.

200.

Hilt

Barclay

Bush

, et al. A statewide child telepsychiatry consult system yields desired health system changes and savings. Telemed e-Health 2015; 21: 533–537.

201.

Cadilhac

Bladin

. Experience with scaling up the Victorian Stroke Telemedicine programme. J Telemed Telecare 2014; 20: 413–418.

202.

Trondsen

Bolle

Stensland

, et al. Video-confidence: a qualitative exploration of videoconferencing for psychiatric emergencies. BMC Health Serv Res 2014; 14: 544.

203.

Ramnath

Khazeni

. Centralized monitoring and virtual consultant models of tele-ICU care: a side-by-side review. Telemed e-Health 2014; 20: 962–971.

204.

Kulcsar

Gilchrist

George

. Improving stroke outcomes in rural areas through telestroke programs: an examination of barriers, facilitators, and state policies. Telemed e-Health 2014; 20: 3–10.

205.

Khalil

Cullen

Chambers

, et al. Implementation of a successful electronic wound documentation system in rural Victoria, Australia: a subject of collaboration and community engagement. Int Wound J 2014; 11: 314–318.

206.

Johansson

Lindberg

Soderberg

. The views of health-care personnel about video consultation prior to implementation in primary health care in rural areas. Prim Health Care Res Dev 2014; 15: 170–179.

207.

Conn

Madan

Lam

, et al. Program evaluation of a telepsychiatry service for older adults connecting a university-affiliated geriatric center to a rural psychogeriatric outreach service in Northwest Ontario, Canada. Int Psychogeriatr 2013; 25: 1795–1800.

208.

Haozous

Doorenbos

Demiris

, et al. Role of telehealth/videoconferencing in managing cancer pain in rural American Indian communities. Psycho-Oncol 2012; 21: 219–223.

209.

Moffatt

Eley

. Barriers to the up-take of telemedicine in Australia--a view from providers. Rural Remote Health 2011; 11: 1581.

210.

Lucas

JAM

Day

Honey

MLL

. Clinician’s perceptions of telehealth for emergency care on the west coast of New Zealand: findings of a descriptive study. Emergency Nurse New Zealand 2016: 6–10.

211.

Nqala

Rout

Aldous

. Remote clinical support by telephone for rural district hospital medical officers in the Eastern Cape. S Afr Fam Pract 2015; 57: 286–290.

212.

Olenik

Lehr

. Counteracting brain drain of health professionals from rural areas via teleconsultation: analysis of the barriers and success factors of teleconsultation. J Public Health 2013; 21: 357–364.

213.

Ritter

Robinette

Cofano

. Evaluation of a statewide telemedicine program. Calif J Health Promot 2010; 8: 1–9.

214.

Pandit

Ray

Sabesan

. Managing emergencies in rural North Queensland: the feasibility of teletraining. Int J Telemed Appl 2018; 2018: 1–7.

215.

Rogove

McArthur

Demaerschalk

, et al. Barriers to telemedicine: survey of current users in acute care units. Telemed e-Health 2012; 18: 48–53.

216.

Wood

Freeman

Banner

, et al. Exploring user perspectives of factors associated with use of teletrauma in rural areas. Australas Emerg Care 2022; 25: 106–114.

217.

Banbury

Smith

Mehrotra

, et al. A comparison study between metropolitan and rural hospital-based telehealth activity to inform adoption and expansion. J Telemed Telecare 2021: 1357633X21998201. DOI: 10.1177/1357633X21998201.

218.

Morrissette

Pearlman

Kovar

, et al. Attitudes and perceived barriers toward store-and-forward teledermatology among primary care providers of the rural Mississippi. Arch Dermatol Res 2022; 314: 37–40.

219.

Howland

Tennant

Bowen

, et al. Psychiatrist and psychologist experiences with telehealth and remote collaborative care in primary care: a qualitative study. J Rural Health 2021; 37: 780–787.

220.

Haque

DeStefano

Banger

, et al. Factors influencing telehealth implementation and use in frontier critical access hospitals: qualitative study. JMIR Form Res 2021; 5: e24118.

221.

Parks

Hunter

Taylor

, et al. Design, development and implementation of the virtual, coordination, access, referral and escalation service in western New South Wales. Aust J Rural Health 2021; 29: 794–800.

222.

Luscombe

Hawthorn

, et al. ‘Empowering clinicians in smaller sites’: a qualitative study of clinician’s experiences with a rural virtual paediatric feeding clinic. Aust J Rural Health 2021; 29: 742–752.

223.

Lim

Liberali

SAC

Calache

, et al. Specialist networks influence clinician willingness to treat individuals with special needs. JDR Clin Trans Res 2022; 7: 267–276.

224.

Powell

Alexander

. Consequences of rapid telehealth expansion in nursing homes: promise and pitfalls. Appl Clin Inform 2021; 12: 933–943.

225.

Mundt

Irarrazaval

Martinez

, et al. Telepsychiatry consultation for primary care treatment of children and adolescents receiving child protective services in Chile: mixed methods feasibility study. JMIR Public Health Surveill 2021; 7: e25836.

226.

Becevic

Smith

Golzy

, et al. Melanoma extension for community healthcare outcomes: a feasibility study of melanoma screening implementation in primary care settings. Cureus 2021; 13: e15322.

227.

May

Jonas

Fehler

, et al. Challenges in current nursing home care in rural Germany and how they can be reduced by telehealth - an exploratory qualitative pre-post study. BMC Health Serv Res 2021; 21: 925.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.23 MB