Abstract
Background:
The COVID-19 pandemic demanded rapid development of telemedicine services for pediatric care and highlighted disparities for marginalized communities.
Objective:
To understand the demographic characteristics of patients with completed and incomplete telemedicine visits at Ann and Robert H. Lurie Children's Hospital of Chicago.
Methods:
This was a cross-sectional retrospective analysis of telemedicine visits for patients <25 years old scheduled between March 21, 2020, and March 17, 2021. We examined visit outcomes and compared outcomes by race/ethnicity, language, and payer using logistic regression. Geographic information system mapping and linear regression were used to examine the relationship between incomplete visits and broadband access within Cook County.
Results:
A total of 13,655 eligible video visits were scheduled for children within 147 ZIP codes during the study time frame. Patient characteristics included median age 9 years, 53% female, 42% non-Latinx White, 31% Latinx, 13% non-Latinx Black, 11% non-Latinx other, and 3% declined/unknown. Preferred language was 89% English, 10% Spanish, and 1% other. Payer was 56% private, 43% public, and <1% other/self-pay. Overall, 86% video visits were completed, 7% cancelled, and 7% no-show with significant variation by patient demographic. Odds of incomplete visits were higher for Latinx patients (odds ratio [OR] 1.93) and non-Latinx Black patients (OR 2.33) than for non-Latinx White patients, patients with preferred language other than English (OR 1.53), and patients not privately insured (OR 1.89). Incomplete visit rates and broadband access were inversely related.
Conclusion:
System and policy solutions are needed to ensure equitable access and address disparities in incomplete telemedicine visits for marginalized populations in urban areas with lower broadband.
Introduction
Telemedicine has long been promoted to increase access to health care for patients living in locations remote from medical services and providers. 1 The initial phase of the COVID-19 pandemic came with rapidly changing health care delivery methods and utilization patterns across the U.S. health care system. 2 In-person health care visits experienced a sharp decline across pediatric and adult primary care. 3 Policy changes in response to the COVID-19 pandemic facilitated the rapid expansion of telemedicine services, even when distance was not the cited barrier.
Telemedicine utilization rates increased, and the health care delivery landscape shifted toward telephone and videoconferencing. Telemedicine provided both patients and providers a way to minimize exposure to and transmission of the virus. It also allowed for the provision of routine care that would have otherwise not been allowable during stay-at-home orders.
It is unclear whether these changes benefitted all patients equally or whether they widened the health care access gap for some. Although telemedicine promised to bring medical attention to people and communities with the greatest need in the safest way possible, the question remained whether vulnerable populations were able to participate in kind. 3,4 Prepandemic attitudes regarding telemedicine's effectiveness for increasing access varied. 5 Some groups purported telemedicine as the great access equalizer emphasizing successes in rural health care. 6 Others saw it as a mechanism to further exclude vulnerable populations without access to technology, thus imposing additional social determinants of health that result in barriers to health care access. 7
In this study, we sought to elucidate the telehealth utilization patterns among pediatric patients during the 1st year of the pandemic. Specifically, we describe patient and visit-level variables and how these differed between patients who had completed telemedicine visits versus those with cancelled appointments or no shows (i.e., incomplete visits). We hypothesized that there would be demographic differences among patients in these two groups with disproportionate incomplete visit rates in patients from minoritized groups (Latinx, non-Latinx, Black, and non-Latinx Asian, non-Latinx, other, or multiple races), with preferred language other than English, and public payer status.
Methods
STUDY DESIGN
We conducted a retrospective cross-sectional analysis of the first scheduled outpatient telemedicine video visit for patients <25 years between March 21, 2020, and March 17, 2021.
SETTING
Ann and Robert H. Lurie Children's Hospital is an independent quaternary academic children's hospital located in Chicago, IL. It serves >220,000 children and their families, many of whom are publicly insured. Before the COVID-19 pandemic, ambulatory services were delivered largely in-person, with few departments utilizing telehealth resources in limited ways. There was widespread institutional adoption of telehealth services in response to local stay-at-home orders.
DATA SOURCES
Data were obtained from a registry of scheduled telemedicine appointments that draws from the hospital electronic medical record (EMR) (Epic, Verona, WI). We linked patient ZIP code to data from the 2015 to 2019 American Community Survey (ACS) extracted from Metopio (Chicago, IL) to understand community characteristics at the ZIP code level. We also obtained counts of the total number of visits to Lurie Children's (in-person and virtual) in each ZIP code within Cook County from 2019 and 2020 to serve as the denominator of patient visits over time.
INCLUSION/EXCLUSION CRITERIA
We included the first scheduled outpatient video visit between March 21, 2020, and March 17, 2021, for patients <25 years old with an outpatient status and a home address ZIP code within Cook County. We excluded telephone visits, open encounters, and patients with an unspecified/missing payer designation. We focused our analyses on patients with a Cook County ZIP code to ensure sufficient visit numbers at the ZIP code level. Within Cook County, we excluded ZIP codes with <10 telemedicine visits in the ZIP code over the study period from geospatial analysis. This study was deemed not human subjects research by Lurie Children's Institutional Review Board.
OUTCOMES
Our primary outcome was the number and proportion of first scheduled video visits that were unplanned incomplete (based on a visit status of same day cancellation or no show). Our secondary outcome was any incomplete visit (based on a visit status of canceled, same day cancellation, or no show). We examined these outcomes at the patient and ZIP code levels.
VARIABLES
Demographic characteristics in the telemedicine registry included age, race/ethnicity, language preference, payer, patient being new to the clinical department, and date of service. Age was a continuous variable and categorized into four groups (<4, 5–9, 10–14, and 15 years and older). We created a composite variable from the race and ethnicity data in the EMR: non-Latinx White, Latinx any race, non-Latinx Black, non-Latinx other or multiple races, and declined/not recorded. We consolidated non-Latinx other or multiple race and declined/not recorded into a single group for logistic regression analyses. Language preference was categorized as English, Spanish, and other.
Language was dichotomized (English or non-English) for logistic regression analysis. Payer was categorized as private, public, or other/self-pay. Payer was dichotomized (private or other) in logistic regression analysis. Data from the ACS were utilized for community-level variables, with the main predictor variable of interest being the proportion of households without high-speed internet within each ZIP code in Cook County. This variable was categorized in quartiles (Q1 < 11%, Q2 11–16%, Q3 17–21%, and Q4 > 21%) for ease of interpretation. We also examined predominant racial/ethnic group and language of the ZIP code.
STATISTICAL ANALYSIS
Chi-square statistics were calculated for visit outcomes across race/ethnicity, language, and payer groups. We performed unadjusted and adjusted logistic regression to identify predictors of unplanned incomplete visits and incomplete visits overall. We explored predictors of visits that were rescheduled.
We employed linear regression to examine the ZIP code level relationship between the proportion of incomplete video visits per ZIP code with at least 10 visits and the proportion of households without high-speed internet adjusted for predominant race/ethnicity and language. We also mapped unplanned incomplete visits by the proportion of households without high-speed internet in quartiles. Analyses were conducted in Stata 15.1 (Stata Corp., College Station, TX) and Maptitude 13 (Caliper Corporation, Newton, MA).
Results
There were 102,326 closed telemedicine visits from March 21, 2020, to March 17, 2021, among 33,902 patients <25 years old in the registry. Among those, 13,655 (40%) were for patients with home ZIP codes within Cook County. This population was included in analyses. At the community level, 14 out of 161 zip codes in Cook County were excluded due to having <10 telemedicine visits during the study period.
Patients scheduled for telemedicine services had a median age of 9 years (interquartile range 3, 14) and gender recorded in the EMR was female for 53%. The patient population was racially and ethnically diverse, 42% non-Latinx White, 31% Latinx, 13% non-Latinx Black, 11% other races, and 3% declined/unknown. English was the language preference listed for 89% of patients, Spanish was listed for 10%. Public payer was associated with 43% of visits and <1% were other/self-pay status.
Telemedicine visit outcomes differed by demographic characteristics (Table 1). Compared with non-Latinx White patients, higher percentages of incomplete visits were observed for Latinx patients, non-Latinx Black patients, and patients with other/declined/unknown race/ethnicity. In addition, higher percentages of incomplete visits were observed for patients with Spanish or other language preference, and for patients who were publicly insured or were listed as other/self-pay. All comparisons were statistically significant (p < 0.001).
First Scheduled Telemedicine Visit Among Patients with Home Address in Cook County
Q, quartile.
In logistic regression analysis, we found the odds of unplanned incomplete visits were higher for Latinx patients (odds ratio [OR] 2.63; 95% confidence interval [CI] 2.28–3.03), non-Latinx Black patients (OR 3.51; 95% CI 2.98–4.13), and non-Latinx other race patients (OR 1.25; 95% CI 1.02–1.53) than for non-Latinx White patients (Table 2). Similarly, patients with preferred language other than English and patients who were publicly insured or other/self-pay had higher odds of unplanned incomplete visits than patients with English language preference and those privately insured (OR 1.72; 95% CI 1.48–2.01 and OR 2.63; 95% CI 2.34–2.96, respectively).
Same Day Cancellation or No-Show Among All Scheduled Visits (n = 13,655)
Multivariable model was adjusted for patient age, if patient was new to department with their first scheduled telemedicine encounter, month of year, and all other variables presented in this table.
CI, confidence interval; OR, odds ratio.
Patients from ZIP codes with higher proportions of households without broadband access (quartiles 2–4) had higher odds of unplanned incomplete visits than patients from ZIP codes in the lowest quartile for households without broadband internet. Findings for Latinx and non-Latinx Black race/ethnicity, payer, and the ZIP codes with the highest quartile of households without high-speed internet persisted as statistically significant in analyses adjusted for patient age, patient new to the clinical department, and month of scheduled visit (Table 2). Findings were similar when we examined any incomplete visit as the outcome (Table 3). Payer was the only demographic characteristic that was significantly associated with rescheduling an incomplete visit in bivariate logistic regression analysis (Table 4).
Incomplete Visits (Including Cancelled, Same Day Cancellation or No Show) Among All Scheduled Visits (n = 13,655)
Multivariable model was adjusted for patient age, whether patients were new to department with their first scheduled telemedicine encounter, month of year, and all other variables presented in this table.
Rescheduled Appointments Among Incomplete Visits (n = 1,923)
Multivariable model was adjusted for patient age, whether patient was new to department with their first scheduled telemedicine encounter, month of year, and all other variables presented in this table.
Lastly, we compared the percentage of incomplete visits per ZIP code in quartiles and the proportion of households without high-speed internet in quartiles per ZIP code (Fig. 1). The percentage of incomplete video visits in each ZIP code was positively associated with the percentage of households without high-speed internet. This relationship was statistically significant in both unadjusted and adjusted regression models (R-squared = 0.1776, p < 0.0001 and adj R-squared = 0.2287, p ≤ 0.0001).

Incomplete telemedicine visits and broadband availability. This figure shows the percentage of incomplete visits in quartiles per ZIP code
Discussion
In this 1-year study of pediatric telemedicine visits, we found differences in telemedicine video visit completion among marginalized racial/ethnic groups, those with language other than English, public or other/self-pay, and community-level high-speed internet access. Our findings raise concerns about social and structural barriers that may limit telemedicine services for urban populations where broadband access is limited. Similar concerns with telemedicine uptake have been highlighted in other vulnerable groups, that is, rural populations, older adults, those with limited health literacy, those with intellectual disabilities, and those with low socioeconomic status. 8 –11
These challenges are further exacerbated by digital gaps wherein issues of device ownership, internet speed heterogeneity, and digital literacy hinder telemedicine engagement. 12 –15 Differential telemedicine utilization observed among marginalized groups provides yet another example of the profound influence of social determinants of health care. 16 –18 In addition, it underscores some of the long-standing inequities within our health care systems.
Patients with language other than English constitute a cohort of patients with known disparities in telemedicine uptake and provide an example of the need for deliberate efforts to achieve equity for all. Our study supports existing literature associating language discordance with decreased participation in telemedicine. 19 –21 These findings did not persist in multiple variable analyses adjusting for race, payer, and ZIP code-level broadband access, suggesting an interplay of social determinants on telehealth visit completion. However, it is known that patients with limited English proficiency have additive hurdles to overcome including navigating digital material in a different language.
Ready interpreter access and proper integration of this role into the telemedicine workflows become an essential way to bridge care for these patients. 10,21 –23 In addition, consideration of alternative telemedicine modalities, that is, telephone versus videoconferencing, may be required to reach this and other disadvantaged groups. 15,24
The COVID-19 pandemic created a seminal moment for telemedicine as a primary mechanism of health care delivery. With a significant uptick in utilization during its initial phase, concern about the ways that telemedicine could exacerbate existing health care equity gaps quickly arose. This was compounded by the disproportionate number of infections among disadvantaged groups who were potentially at risk of being marginalized by technology, too.
Our study contributes to the growing body of literature that describes the telemedicine experience among different demographic groups in pediatrics. Our results are consistent with other's that show that for telemedicine to provide sustained access to health care beyond the pandemic, it needs to proactively address barriers through formal evaluation, monitoring, and deliberate action. 18,25,26
Taken together, our results support the need to monitor and practice early mitigation of potential barriers so that telehealth care can be an equitable resource for all. COVID-19 infection spread rapidly and disproportionately across communities of color and resource-poor areas. Hesitancy in seeking care during this time kept the highest risk individuals in the highest risk neighborhoods away from health care. 27,28 Looking at conditions such as COVID-19 infections through a health equity lens allows us to understand how vulnerable populations carry not only the burden of disease but also the social and economic ramifications of high infection rates. 10
In this sense, telemedicine may offer substantial benefits to economically disadvantaged patient groups by limiting travel time, time waiting for care, missed work and school, 4,27 and the need to obtain child care if there are multiple children in the household. In addition, use of telemedicine has been shown to increase chronic disease management and increase care adherence. 27
LIMITATIONS
The first limitation of our study of a single-center experience is that our findings may not be generalizable to other settings, though it likely represents that of other urban independent children's hospitals that expanded telemedicine services in response to the COVID-19 pandemic. Second, there are inherent limitations to working with administrative and EMR data. Data elements in our telemedicine registry are of limited granularity and there is no opportunity to resolve missing information or verify demographic characteristics. There is potential for a mismatch between a patients' self-identified race/ethnicity and what is recorded in the EMR.
In addition, mismatches of patients' language proficiency versus documented language preference may be present. We cannot estimate the directionality of this misclassification bias on the interpretation of our results. Likewise, attribution bias is inherent when individual patients are assigned community-level characteristics. ACS estimates may not reflect the current circumstances of a given area or a given household.
Conclusions
Although telemedicine did provide access to care to our pediatric patients during the COVID-19 pandemic, efforts are still needed to address higher incomplete visit rates for Latinx, non-Latinx, Black, and patients of other race categories, patients with non-English language preference, and those without private insurance. Telemedicine is a health care resource that will continue beyond the pandemic. To that end, it is important to ensure the telemedicine infrastructure is equitable through assuring seamless integration of interpreter services, telemedicine parity, and broadband accessibility.
Footnotes
Acknowledgments
The authors thank Emily Golbeck and Patricia Labellarte, both from the Smith Child Health Outcomes, Research, and Evaluation Center within Stanley Manne Children's Research Institute at Lurie Children's Hospital of Chicago, for their support and expert assistance throughout the execution of this study.
Authors' Contributions
K.K. contributed to conceptualization, writing—original draft, visualization, funding acquisition, validation, investigation, and project administration. L.B. carried out formal analysis, resources, and writing—review and editing. M.C. was involved in software, validation, resources, and data curation. T.L.S. took charge of software, validation, resources, data curation, formal analysis, and visualization. D.S. was in charge of validation, and writing—review and editing. M.L.M. was in charge of conceptualization, methodology, validation, formal analysis, investigation, resources, writing—review and editing, and supervision.
Disclosure Statement
The authors of this article have no conflicts of interest to disclose.
Funding Information
This study was supported through a COVID-19 Springboard Award from Stanley Manne Children's Research Institute.
