Abstract
The COVID-19 pandemic challenged school health professionals to navigate a dynamic public health emergency and to stay aware of changing recommendations. This study aimed to determine the value of the COVID Just-in-Time ECHO (Extension for Community Health Outcomes) Learning Series for Schools among participating school health personnel and to describe their ongoing information needs during the pandemic. School health, public health, and education professionals across Colorado participated in this ECHO series. Participants attended 1-hour sessions every 2 to 4 weeks from March 2020 through December 2021 for a total of 34 sessions. Data collection included postsession and postseries surveys assessing what participants found most valuable and what additional information they needed. School nurses represented 113 of 224 participants (50%). ECHO content noted as being the most valuable included epidemiology updates and special topics as chosen by participant input. Postseries surveys identified the value of experiencing shared knowledge among Just-in-Time ECHO participants. Participants identified ongoing needs for information about COVID-19 guidance and risk mitigation in schools throughout the sessions. In postseries surveys, participants reported additional ongoing information needs related to COVID-19 outside schools. This ECHO series delivered reliable and time-sensitive information for school health personnel and school leaders and may provide a useful model for information sharing among education and public health professionals.
The first 6 months of the COVID-19 pandemic were challenging for school personnel, who faced myriad unknowns, with advice and policies varying by district and with inconsistent guidance amid ever-changing information about COVID-19.1-3 Clear national public health guidance for schools was lacking, and local approaches to wearing face masks, screening for COVID-19, vaccinating against COVID-19, closing schools, and safely reopening schools varied substantially. 4 In Colorado, the state health department’s first official school guidance was not released until July 2020, leaving a void of information about factors associated with COVID-19 in schools early in the pandemic. The pandemic highlighted what many in education have long known: schools are responsible for more than educating students.5,6
Schools needed support, accurate information, and guidance to safely educate children during the pandemic. Social distancing required changes in how schooling and any information sharing to support schools would take place. Virtual platforms grew to facilitate exchanging knowledge, building community, and sharing resources. 7 In response, the Children’s Hospital Colorado School Health Program sought to support diverse school and childcare settings with pediatric health needs and mobilized the existing infrastructure of ECHO (Extension for Community Health Outcomes) Colorado to launch a COVID Just-in-Time (JiT) ECHO Learning Series for Schools. This initiative sought to provide participants with timely COVID-19 information from physicians and public health leaders and advice on safely navigating the pandemic and addressing student and family concerns.
Adaptation of JiT ECHO: COVID JiT ECHO Learning Series for Schools
The ECHO model was first developed in New Mexico as a virtual capacity-building education program that provided real-time dissemination of best practices through strengthening participant skills and sharing processes and resources. 8 An interdisciplinary faculty, an “all teach, all learn” approach (ie, participants and facilitators exchange knowledge together), and case-based learning characterize ECHO programs. 9 ECHO Colorado, a replication site of Project ECHO, offers various programs and resides within the University of Colorado Anschutz Medical Campus.
In early 2020, various community-based organizations recognized that the ECHO model could be used to disseminate information to clinicians across Colorado, requested a statewide series to help primary care providers understand the novel virus, and initiated the COVID JiT ECHO series that ran from March 2020 through December 2021. This series adapted the ECHO model, using rolling enrollment to accommodate increased attendance and meet the just-in-time educational needs of participants with new information. The COVID JiT ECHO Learning Series for Schools was an offshoot of this series. 10
Purpose
This study aimed to determine the value of the COVID JiT ECHO Learning Series for Schools among participating school health personnel and to describe their ongoing information needs during the COVID-19 pandemic. These findings may inform the potential for similar initiatives to promote collaboration among education, public health, and health care professionals during a future public health emergency.
Methods
ECHO Series
The COVID JiT ECHO Learning Series for Schools curriculum was based on just-in-time participant needs assessments and through task force discussions. The task force included representatives from the Colorado Department of Public Health and Environment, Colorado Association of School-Based Health Care, Colorado Department of Education, Colorado Alliance for School Health, Colorado Education Association, Colorado Association of School Boards, Colorado Association of School Executives, and Colorado Chapter of the American Academy of Pediatrics. These groups came together virtually in summer 2020 to build on their existing efforts to communicate with school partners using the ECHO Colorado infrastructure. They organized this COVID JiT ECHO Learning Series for Schools with funding support from the University of Colorado School of Medicine, Colorado Department of Public Health and Environment, and private philanthropy. One-hour video conference sessions took place every 2 to 4 weeks for a total of 34 sessions. Each session included an epidemiology update, a focus topic (Table 1), an interactive question-and-answer portion, and live polling to obtain participant feedback.
Focus topics for COVID JiT ECHO Learning Series for Schools, Colorado, March 2020 through December 2021 a
Abbreviations: app, application; CDC, Centers for Disease Control and Prevention; CDPHE, Colorado Department of Public Health and Environment; CHCO, Children’s Hospital of Colorado; ECHO, Extension for Community Health Outcomes; JiT, Just-in-Time; Q&A, question and answer.
COVID JiT ECHO Learning Series for Schools brought together education, public health, and health care professionals in Colorado for shared learning through regular virtual conferences from March 2020 through December 2021. Each session included a focus topic that was selected by participant feedback.
Epidemiology updates included COVID-19 data presented by a local pediatric infectious disease physician focusing on national and local case counts, test positivity, mortality rates, school and childcare outbreaks, and vaccination rates. This segment included public health and school guidance from the Centers for Disease Control and Prevention and the Colorado Department of Public Health and Environment. The task force used participant feedback to select focus topics, and local and regional subject matter experts presented information on these topics. Information on preferences for future focus topics was gathered by using live virtual polling during sessions, followed by an email survey soliciting information on topic preferences and session feedback.
A COVID JiT ECHO Learning Series for Schools website included scheduling and registration information, session topics, presenters’ slides, links to resources, and written session summaries.
Participants and Organization
The target audience for this series included individuals making and implementing school health policies and those sharing health-related information with students and their families. This group encompassed district and school personnel, childcare consultants, school nurses and health aides, and public health professionals. Participants were recruited through various avenues, including the Children’s Hospital Colorado School Health Program Listserv, ECHO Colorado’s website, and personal communication from series task force members.
Data Collection
We collected data on participant organization and job role through a survey form in the conference log-in process. Response options for job role included behavioral health provider, district administrator, health aide, school-based health provider, school nurse or other school clinician, teacher, school leader, and other. After each session, from March through December 2021, a 2-item email survey was distributed via Qualtrics (Qualtrics International Inc). Questions with free-text responses were “What was most helpful from today’s session?” and “What topics related to COVID-19 would you like to discuss or learn about in future sessions of this series?” Participants attending a session were eligible to take the postsession survey. After series conclusion in December 2021, an email survey was distributed via Qualtrics to participants, who were asked, “What was most valuable about the series?” and “Now that the COVID-19 JiT ECHO Learning Series for Schools ended, what COVID-19 information needs do you anticipate having?” Participants who attended at least 1 session and registered with an email address were eligible to respond to the postseries survey.
Data Analysis
The evaluation included quantitative and qualitative methods. We analyzed participant characteristics using descriptive statistics with SPSS version 25 (IBM Corp).
We created coding schemes to analyze postsession and postseries responses using an inductive approach. First, 2 authors (A.C., S.T.) independently reviewed all postsession and postseries survey responses in Microsoft Excel, added notes about meaningful aspects of responses, and developed categories for responses. Next, the authors reviewed initial coding findings and developed a shared list of topics and subtopics. We used 1 coding scheme to capture series aspects that participants found most valuable based on responses to the questions “What was most helpful from today’s session?” and “What was most valuable about the series?” (most valuable). Another coding scheme was developed for information that participants felt they needed based on responses to the questions “What topics related to COVID-19 would you like to discuss or learn about in future sessions of this series?” and “Now that the COVID-19 JiT ECHO Learning Series for Schools has concluded, what COVID-19 information needs do you anticipate having?” (additional information).
The 2 authors (A.C., S.T.) then independently coded all responses using the preliminary coding scheme, met to discuss the adequacy of the coding scheme, and added or refined topics as necessary. More than 1 code could be used for each response. Using a consensus coding approach, both authors reviewed all code selections and articulated their rationale for the selections, including concordant and discrepant coding. 11 In 1 example, the coders discussed what constitutes “general COVID-19 guidance and updates” as compared with more specific subtopics in the coding scheme. Initial coding was concordant for 90% of the most valuable responses and for 89% of the additional information responses. The authors discussed differences in assigned codes until they reached a final consensus.
We applied coding schemes for most valuable and additional information to postsession and postseries survey responses due to shared thematic patterns. We analyzed coding of postsession and postseries survey responses separately using descriptive statistics in Microsoft Excel.
The Colorado Multiple Institutional Review Board reviewed ECHO Colorado’s project evaluation approach and determined it to be not human subjects research.
Outcomes
Participant Characteristics
A total of 224 people participated in the series, with an average of 63 participants per session (range, 34-101). Most participants were school nurses or other school clinicians (n = 113, 50%), followed by “other” job roles (n = 72, 32%; predominantly epidemiologic, public health, or clinical roles), school leaders (eg, principal, vice principal; n = 13, 6%), and district administrators (eg, superintendent; n = 13, 6%), health aides (not clinicians; n = 9, 4%), school-based health providers (n = 3, 1%), and 1 behavioral health provider (<1%). Participants were from 105 organizations, mainly consisting of school districts, childcare centers, and public health agencies. From March through December 2021, we recorded 161 postsession survey responses across 18 sessions. Of 201 participants with email addresses, 48 responded to the postseries survey.
Postseries and Postsession Surveys
The final coding schemes for participant survey data included separate codes for information that participants considered most valuable and for additional information that they desired (Table 2). Some codes were shared across these 2 major categories (eg, COVID-19 epidemiologic information, information on the science of COVID-19, COVID-19 vaccines, testing, medication); other codes were unique to a single category (eg, most valuable: learning from content experts, access to up-to-date information, experiencing shared knowledge; additional information desired: risk mitigation in schools, vaccines and testing in schools, mental health).
Coding schemes and sample quotes from a qualitative analysis of postsession and postseries participant surveys from a COVID JiT ECHO Learning Series for Schools, Colorado, March 2020 through December 2021 a
Abbreviations: ECHO, Extension for Community Health Outcomes; JiT, Just-in-Time.
COVID JiT ECHO Learning Series for Schools brought together education, public health, and health care professionals in Colorado for shared learning through regular virtual conferences from March 2020 through December 2021. Participants completed surveys after each session and at the end of the series.
Most valuable
ECHO series content emerged as highly valued, with 77% (n = 124) of postsession responses and 69% (n = 33) of postseries responses commenting on content (Table 3). ECHO series attributes were mentioned more frequently in postseries responses (67%; n = 32) than in postsession responses (25%; n = 41), with many postseries responses identifying the value of experiencing shared knowledge and having access to up-to-date information. In postsession surveys, the content most frequently mentioned as being valuable was COVID-19 epidemiologic data, followed closely by ECHO series on special topics and information on vaccine, testing, and medication. In postseries responses about ECHO content, general COVID-19 guidance and epidemiologic information were most frequently identified as being valuable.
Results of postsession and postseries surveys among participants (N = 224) across 105 organizations in a COVID JiT ECHO Learning Series for Schools, Colorado, March 2020 through December 2021 a
Abbreviations: ECHO, Extension for Community Health Outcomes; JiT, Just-in-Time.
COVID JiT ECHO Learning Series for Schools brought together education, public health, and health care professionals in Colorado for shared learning through regular virtual conferences from March 2020 through December 2021. Participants completed surveys after each session and at the end of the series.
More than 1 code could be applied for each response; thus, percentages may not total 100%.
After each session, a 2-item email survey was distributed to participants. Questions with free-text responses were “What was most helpful from today’s session?” and “What topics related to COVID-19 would you like to discuss or learn about in future sessions of this series?”
After series conclusion in December 2021, an email survey was distributed to participants. Participants were asked, “What was most valuable about the series?” and “Now that the COVID-19 JiT ECHO Learning Series for Schools ended, what COVID-19 information needs do you anticipate having?”
Additional information
Postsession responses (46%; n = 71) more often expressed a need for additional information about COVID-19 within school settings than postseries responses (26%; n = 9) (Table 3). Postsession needs for school COVID-19 information focused on general school guidance and risk mitigation in schools. Postseries responses (62%; n = 21) more so than postsession responses (43%; n = 66) anticipated needs for additional information about COVID-19 outside school settings. Postsession and postseries responses about out-of-school COVID-19 information needs focused on general COVID-19 guidance updates and information on the science of COVID-19. Postseries responses also frequently identified a need for information about mental health (21%; n = 7) and perinatal and early childhood populations (18%; n = 6).
Lessons Learned
Children and adolescents faced substantial disruptions to education and to physical and mental health during the COVID-19 pandemic.12-14 Understanding how we can better support child health and education during the next public health crisis is critical. ECHO has been used to support school nurses in managing medical conditions such as seizures and diabetes.15,16 The COVID JiT ECHO Learning Series for Schools series included school nurses and other professionals and focused on communicable disease management. This ECHO series was unique in that it allowed medical and public health experts to share the latest COVID-19 information with participants from childcare and school settings, identify ongoing information gaps in real time, and promote a collaborative learning community.
The series addressed the needs of educators, school administrators, and school health professionals as COVID-19–related protocols and system practices changed frequently and amplified the need for just-in-time knowledge sharing.4,10 Survey respondents frequently identified epidemiologic updates as being valuable and emphasized the value of timely information, underscoring the relevance of a JiT ECHO approach for a situation such as a pandemic. While we often think of pandemics as rare events, outbreaks of infectious diseases such as H1N1 influenza or mpox, the emergence of new diseases such as COVID-19, the reemergence of diseases such as measles, and natural disasters all affect children and schools and would lend themselves to a JiT ECHO model. This JiT series was an evolution of the traditional ECHO model and can serve as a potential tool in other crises–response situations in which an event creates ever-changing information needs.
Education theory suggests that communities of practice (ie, groups of people with a shared interest or concern in something that they do, who work together toward improvement of that shared activity) are a key feature of just-in-time learning, which aims to provide knowledge to learners when and how it is needed.17,18 Postsession and postseries surveys confirmed that participants valued access to timely information about COVID-19 school guidance and focus topics tailored to the group’s needs.
This JiT ECHO model supported collaboration among education, public health, and health care professionals, with the common goal of supporting children’s cognitive, physical, social, and emotional development, as described by the Whole School, Whole Community, Whole Child model. 19 Particularly in postseries surveys, participants expressed their appreciation for experiencing shared knowledge. The pandemic posed substantial challenges to school staff, and this project showed that ECHO can overcome many barriers (eg, financial costs, travel difficulties) for various professionals to enable an exchange of up-to-date information and to promote shared experiences. Focus topics related to mental health, mindfulness, and vaccine hesitancy (Table 1) allowed participants to discuss shared challenges and strategies to address those challenges. Although not every focus topic was highlighted in survey responses, experiencing shared learning was a frequent theme among postseries surveys, demonstrating that while specific topical information itself is important, belonging to a community is also valued. Promoting connectedness was important during the COVID-19 pandemic and may help build capacity for school health and education professionals in future crises.
In postsession and postseries surveys, participants identified ongoing information needs related to COVID-19 guidance inside and outside of schools. An ongoing need for school-related COVID-19 guidance illustrates the importance of regular dialogue between school communities and medical and public health professionals. When this ECHO series concluded in late 2021, COVID-19 vaccinations were available for adolescents and had just recently become available to children aged ≥5 years. 20 Adults had been eligible for COVID-19 vaccination for most of 2021, and many COVID-19 prevention measures had begun to fade in the United States; however, COVID-19 continued to have a great effect on children in 2021 and 2022. 21 While our JiT ECHO series addressed the need for guidance updates on COVID-19 while it was ongoing, sustained channels of communication with regularly updated COVID-19 and non–COVID-19 health information may be needed beyond an ECHO series or through a longer-running learning collaborative. Although a desire for COVID-19 guidance outside of schools may not have been uniquely relevant to participants’ professional roles, this expressed need suggests that this ECHO series filled an information gap even when other resources were available (eg, information from national or local media, public health authorities).
This study had several limitations. First, recall bias may have occurred in survey responses. Second, although the overall number of ECHO participants was relatively large, a smaller number participated in all aspects of the evaluation, possibly because of competing work priorities during the pandemic. Third, collection of demographic information was limited to organization and job role. Fourth, data were limited to free-text responses in brief participant surveys and did not include observations from within the virtual meetings. Future research using interviews or focus groups could enrich our understanding of participant experiences with similar ECHO series and explore the potential for collaborative learning approaches to benefit school health communities.
We conducted a 20-month COVID-19 JiT ECHO Learning Series for Schools, connecting school health professionals and leaders across Colorado with local public health and medical professionals. Participants valued timely COVID-19 epidemiology and guidance updates and experiencing shared knowledge as a community. They identified ongoing information needs about COVID-19 guidance and risk mitigation in schools during the series and expressed a need for more information about COVID-19 outside of schools at series conclusion. The JiT ECHO model has the potential to reach more learners and bring quality, timely knowledge to a wide audience that may not otherwise receive the tailored information that it needs during a health crisis. Further research is needed on the application of this model to educational settings, including application outside of a pandemic.
Footnotes
Disclaimer
The views and conclusions in this article are those of the authors and do not represent the official views of the project funders.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding support for this project was provided by the University of Colorado School of Medicine, the Colorado Department of Public Health and Environment, and a private southwestern Colorado nonprofit foundation.
