Abstract
COVID-19 disrupted early educational experiences for a generation of children, but research on how missing school impacted academic development for young children is limited. This paper investigated the longitudinal impact of COVID-19 on the language and literacy development of children in a Florida school district, specifically following children who were in pre-K in March 2020 through the end of first grade. This COVID-19 cohort (n = 1,167) was compared with a pre-COVID-19 cohort (n = 1,211) using propensity score matching, multilevel regression models, and linear growth modeling. Results indicated that the negative impacts of COVID-19 on language and literacy emerged late and increased over time for young children but were not driven primarily by remote learning. Young children showed especially severe learning loss in unconstrained literacy skills (e.g., comprehension) versus constrained skills (e.g., phonics). COVID-19 impacts were not moderated by child characteristics (e.g., poverty or race/ethnicity).
Keywords
In early 2020, coronavirus disease 2019 (COVID-19) arrived in the United States and began to spread rapidly. In an unprecedented moment in U.S. history, schools were closed nationwide to slow virus transmission, and teachers were asked to quickly pivot to remote learning with little to no preparation. Many students continued to learn remotely during the 2020–21 school year, and others experienced additional disruptions to learning due to quarantines, teacher and staff shortages, and physical and mental health issues related to the pandemic well into the 2021–22 school year (Zviedrite et al., 2023).
Currently, we are still reckoning the impacts of these COVID-19-related disruptions to multiple years of school on children’s development and academic achievement. A relatively large number of reports have addressed the shorter-term impacts of COVID-19-related school closures on children’s learning in the United States, examining impacts in the fall of 2020 and through the first full year of pandemic-affected school (2020–21). However, few studies have followed the same group of children longitudinally from the pre-COVID-19 period and across subsequent school years to trace how COVID-19 disruptions have impacted learning over time. Even fewer studies have focused on an early childhood population because standardized testing data are less available for young children compared with those in upper elementary school and beyond. Understanding the impact of COVID-19 on this population is essential because early learning experiences lay the foundation for later school success, and young children are particularly vulnerable to long-lasting effects from environmental adversities that occur during their early years (Shonkoff, 2016). It is possible that missed instructional opportunities due to COVID-19 may become more apparent and even be compounded over time for this group of children, who have spent their entire school careers learning within a COVID-19-affected landscape. In this paper, we address important gaps in the COVID-19 literature through our longitudinal investigation of how school disruptions impacted the early language and literacy development of children who were in pre-K in March 2020 (n = 1,123), following their progress through kindergarten (2020–21) and through the end of their first grade year (2021–22). We also explore the implications of these findings by investigating which early language and literacy skills are most severely impacted when children’s typical school instruction is disrupted. This examination allows for a unique examination of which language and literacy skills are supported by in-person early childhood instruction and how children’s development is impacted when they miss out on this instruction.
This study offers several advantages in terms of its data quality, research design, and statistical methods in comparison with previous reports. Drawing on administrative data from a large school district in west-central Florida, our dataset included detailed child-level demographic variables, child-level information about learning modes (i.e., remote or in-person), and baseline data on children’s language and literacy scores prior to COVID-19. In estimating the impacts of COVID-19, we compared children from the COVID-19 cohort with those from a pre-COVID-19 historical cohort using propensity score matching to mimic an experimental design and facilitate comparisons between the two cohorts. Therefore, our analysis allowed us to make causal inferences and attribute differences in learning between the two cohorts to COVID-19 school disruptions.
Language and Literacy Development in Early Childhood
This paper specifically focuses on children’s language and literacy development during the early childhood years (pre-K through first grade). This time period is often considered a critical window for building the skills that enable children to read and learn from texts, use and understand language, and communicate orally and in written form. Not only are early language and literacy skills highly predictive of later reading (Duncan et al., 2007; National Early Literacy Panel, 2008) and writing proficiency (Manfra et al., 2017), but they are also strong predictors of later math achievement (Claessens et al., 2009). Early language and literacy therefore can be considered to be a cornerstone of future academic achievement. In particular, children’s early language and literacy competencies shape their later reading abilities in profound and long-lasting ways. Longitudinal studies have shown that most of the changes in English-speaking children’s reading trajectories occur during first or second grade (Little et al., 2021; Pianta et al., 2008), suggesting that many children’s reading trajectories are largely “locked in” by early elementary school.
Children acquire many literacy and language practices at home (e.g., Heath, 1983), but when they enter pre-K, their teachers begin more formalized instruction of early literacy skills such as concepts of print, letter names, and phonemic awareness. The foundational competencies that children acquire in the early school years can be grouped into two categories: constrained and unconstrained skills (Paris, 2005). According to models such as the simple view of reading (SVR; Hoover & Gough, 1990), both sets of skills are necessary for reading but differ in their scope and time to mastery. Constrained skills have a finite number of items and can be taught and mastered over a relatively brief time span. For example, letter knowledge is considered a constrained skill because children typically master the names and sounds of all 26 letters in the early childhood period. Other constrained skills include writing one’s name, letter–sound correspondences, phonics, and concepts about print. Unconstrained skills are those with a broader scope (i.e., they contain a nearly limitless number of items) that continue to be improved and refined over a lifetime. Skills such as comprehension, vocabulary, and using language for a variety of purposes are considered unconstrained because even adults continue to learn new words and deepen their ability to understand a range of texts over time.
Different kinds of instruction are also used for constrained versus unconstrained skills. It is recommended that constrained skills be taught explicitly and systematically using an intentional scope and sequence in which one skill builds on the next (Ehri, 2020). Assessment of constrained skills is also relatively straightforward because teachers can, for example, quickly assess children’s knowledge of short vowel sounds using a brief checklist. In contrast, unconstrained skills are open ended in nature and therefore less concrete to teach and more difficult to assess. Instruction in unconstrained skills such as oral language development often involves highly interactive, responsive methods such as reading and discussing books, scaffolding children’s language during play, acting out stories, and engaging in back-and-forth conversations (Hadley & Dickinson, 2019; Nicolopoulou et al., 2015; Snow & Matthews, 2016). Assessment of such skills also can be imprecise and often measures only relatively surface-level aspects of these complex skills (Hadley & Dickinson, 2020; Stahl, 2014). Because of the difficulty of assessing unconstrained skills, it may not be immediately apparent when young children’s development of unconstrained skills such as comprehension and vocabulary lags. Instead, difficulties in these areas typically emerge in the later elementary years when children are asked to make meaning from more complex texts and struggle to do so (Reardon et al., 2012). Weaknesses in unconstrained skills are therefore often less visible in the earlier years, and by the time these difficulties do manifest, they can be difficult to ameliorate (Snow & Matthews, 2016).
In addition to the importance of these cognitive skills for reading comprehension, models such as the componential model of reading (CMR) expand on the SVR to suggest that psychological and ecological domains further explain variation in children’s reading achievement (Aaron et al., 2008). For example, psychological factors such as motivation, trauma, and self-efficacy can impact the development of reading comprehension, as can disruptions to the nested set of ecological contexts in which children learn and grow (e.g., home, neighborhood, and school settings).
Children’s School Experiences During Pandemic-Affected Years
The first case of COVID-19 occurred in the United States in late January 2020, and by the end of March, all but one public school district was closed (Zviedrite et al., 2021). Schools were fully closed for an average of eight instructional days before they began offering remote learning (Zviedrite et al., 2021). However, teachers had very little time to prepare and few structural supports in place for providing remote learning, presenting challenges for even the most experienced educators (Darling-Hammond & Hyler, 2020). Many teachers reported difficulties with attendance and engaging children in online sessions (Ford et al., 2021). Real-time instruction was relatively infrequent: A survey of early childhood educators in the spring of 2020 found that most teachers held virtual whole-class meetings once a week and posted lesson materials daily (McKenna et al., 2021). Opportunities to engage in conversations, discussions about books, and other interactive and hands-on activities were almost nonexistent during this time period (Ford et al., 2021; Steed et al., 2022).
As the fall 2020 semester began, most U.S. children went back to school as remote learners (National Center for Education Statistics, 2022), with ~40% of school districts also offering full in-person instruction as an option (Panaggio et al., 2023). As the school year progressed, more districts began to offer an in-person instruction option, with most districts offering both in-person and remote instruction by the end of the year (National Center for Education Statistics, 2022). Remote learning in 2020–21 continued to be a challenge for young children, who often could not access online materials themselves and needed a caregiver to facilitate their learning, resulting in issues with engagement and participation (Jalongo, 2021; McKenna et al., 2023). Early childhood educators often selected digital apps and other online resources that were focused on supporting constrained skills, such as letter knowledge, rather than unconstrained skills, such as comprehension (Hadley et al., 2024). The 2020–21 school year also involved other disruptions to learning, including school closures due to quarantines and lower student attendance rates (Carminucci et al., 2021; Zviedrite et al., 2023). Children’s in-person school experiences also were potentially impacted by social distancing and disease-mitigation measures such as the use of face masks, but research on the effects of these measures on language and literacy development is mixed (Lalonde et al., 2022; Surrain et al., 2023). Finally, studies reported high rates of teacher burnout and sharp declines in teacher mental health (Hutchison et al., 2022; Pressley, 2021), which may have further affected student outcomes (Madigan & Kim, 2021).
The following academic year (2021–22) in many ways represented a return to normal, in that 98% of school districts offered full-time in-person learning as children returned to school in the fall (National Center for Education Statistics, 2022). However, COVID-19-related school closures and quarantines still occurred with some regularity (Institute of Education Sciences, 2022; Zviedrite et al., 2023). School district personnel also reported that chronic absenteeism, both from teachers and from students, had greatly increased since pre-pandemic times and did not improve in 2021–22 (Institute of Education Sciences, 2022).
In all, the picture of learning that emerged during this time period suggests that the nature and quality of young children’s learning experiences were profoundly impacted both in the period immediately following COVID-19 shutdowns and to some extent through the 2021–22 school year. In particular, many children who were in pre-K when COVID-19 arrived attended a substantial portion of kindergarten entirely online, limiting their opportunities to use language in the context of hands-on activities, play, and conversations with teachers and peers (e.g., Vaillancourt et al., 2021; Watts & Pattnaik, 2023).
Impacts of COVID-19 School Disruptions on Learning
A number of studies to date have examined the impacts on learning caused by COVID-19 school disruptions. It is important to note that nearly all COVID-19-era studies have had to contend with data-quality issues that potentially bias results, such as higher rates of missing data for vulnerable student demographic groups and remote modes of assessment (Ho, 2021). Therefore, there is a critical need for studies, such as the present one, that account for factors such as missing data and differences in comparison groups to more accurately estimate COVID-19 impacts.
Short-Term COVID-19 Impacts
To date, the literature on the short-term impacts of COVID-19, or those measured in the immediate aftermath of COVID-19 school closures (i.e., during the 2020–21 school year), is somewhat mixed. In our own work, we reported on the short-term impacts of COVID-19 with the same sample used here. We examined differences in the fall of 2020 for two cohorts of kindergarteners: a COVID-19 cohort, whose pre-K year was disrupted due to COVID-19 school closures, and a historical pre-COVID-19 cohort, who had a typical pre-K year (Hadley et al., 2023). We found no significant differences between the two cohorts, with the COVID-19 cohort even outperforming the pre-COVID-19 cohort in some domains. Our prior findings aligned with those of other studies, one of which found no impact of COVID-19 on reading scores in the early elementary grades (Renaissance Learning, 2020) and another of which found that Head Start children’s learning gains during the 2020–21 school year were within a normal range (Lynch et al., 2023). However, other studies, both with early childhood populations (Amplify, 2021; McGinty et al., 2021) and with elementary-aged populations (Domingue et al., 2022; Dorn et al., 2021; Furjanic et al., 2024; Kuhfeld et al., 2022; Pier et al., 2021; Relyea et al., 2023), have found that COVID-19 had significant negative impacts on students’ reading scores in the short term. The magnitude of these negative impacts ranged from effect sizes of 0.06 (Kuhfeld et al., 2022) to 0.54 (Relyea et al., 2023). Of great concern are findings that COVID-19 had larger negative impacts on the reading scores of children who are Black, Latinx, English language learners, of low socioeconomic status, and/or have disabilities (Dorn et al., 2021; Pier et al., 2021; Relyea et al., 2023).
Longer-Term COVID-19 Impacts
Longer-term studies that investigated the academic impacts of COVID-19 into the 2021–22 school year are more scant, and findings are equivocal, with some results suggesting that the learning of COVID-19-era children is returning to normal, and others indicate that children are falling further behind. One study across 30 states found that the average student lost 0.31 grade levels in reading from 2019 to 2022 but that reading scores began to improve rapidly in 2022–23 (Fahle et al., 2024). Two large-scale national studies (Kuhfeld & Lewis, 2022b; Renaissance Learning, 2022) and one focused on Washington, DC, schools (EmpowerK12, 2022) found that growth rates in reading returned to nearly normal in the 2021–22 school year, although students’ overall scores still lagged behind historical averages. Children’s recovery rates were highly dependent on their local contexts, with learning rebounding in districts where leaders invested in resources such as intensive tutoring, whereas student learning in other districts declined further by the end of 2023 (Fahle et al., 2024). These variable results suggest that nationwide averages at this point in time may be less helpful in understanding COVID-19 impacts than careful evaluations of individual school districts and local areas (e.g., Relyea et al., 2023).
An additional complexity to measuring the impacts of COVID-19 on learning is the variation in learning modalities during the post-COVID-19 years. Some children attended school remotely, others attended in person, and children often switched settings within a school year. To date, studies have indicated that children who attended school in person appeared to learn more than those who attended remotely; these effects were particularly strong for younger children (Halloran et al., 2021; Tomasik et al., 2021; Uthappa et al., 2023). One Switzerland-based study found that elementary-aged children learned more than twice as fast as children learning remotely (Tomasik et al., 2021).
This Study
To date, the literature on COVID-19 impacts on learning has several important gaps: Few studies have investigated an early childhood population, followed the same children longitudinally, or investigated specific subcomponent skills beyond the broad categorizations of math and reading. This study addressed these gaps by examining the impact of COVID-19 on children’s language and literacy skills longitudinally through the essential early childhood years. This study complements existing larger-scale national studies that reported broad trends and patterns, but that were typically cross-sectional rather than longitudinal, and therefore could not track individual children’s development over time or trace the impact of child-level factors such as whether a child attended school remotely or was diagnosed with a disability.
In this study, we addressed the following research questions:
What are the longer-term impacts of COVID-19 school disruptions on language and literacy development for young children? Are there differences in the growth trajectories of children impacted by COVID-19 versus those in a pre-COVID-19 cohort?
How did the long-term impacts of COVID-19 differ for children who attended kindergarten remotely versus their peers who attended school in person?
Are specific kinds of language and literacy skills more affected by COVID-19 school disruptions than others?
Are the longer-term impacts of COVID-19 moderated by children’s background characteristics?
Method
This paper draws on data collected by a large school district (Sunnyside) in west-central Florida with schools in both urban and suburban areas. Florida offers half-day pre-K to all 4-year-olds through its Voluntary Pre-K (VPK) Program, which operates in both public schools and community-based settings. Our project examined children who attended VPK programs located in public elementary schools and then continued on to kindergarten and first grade within the same school district. We followed children from the beginning of their VPK year to the end of their first grade year. Longitudinal research that begins in pre-K is relatively rare because there is often no way to track children from their preschool educational setting into elementary school. Our data source presented an advantage because a child attending VPK in a public elementary school is assigned an ID number at pre-K entry that is used throughout their enrollment in subsequent grade levels within that district. Therefore, we were able to follow and compare two cohorts of children over several crucial early childhood years.
Sample
Our dataset included two cohorts of children from 89 schools total. The pre-COVID-19 cohort attended VPK in the Sunnyside school district in 2016–17 (n = 1,211), and the COVID-19 cohort attended VPK in Sunnyside in 2019–20 (n = 1,167). After VPK, children continued kindergarten and first grade in the Sunnyside school district. All children who attended VPK in the Sunnyside district during the targeted years and who took at least one Measure of Academic Progress Growth Reading (MAP Reading) assessment during the kindergarten or first grade year were included in our sample. Table 1 provides information about each cohort’s demographic characteristics at the starting point of the analysis for this paper (winter of the kindergarten year).
Demographic information and child characteristics by cohort
Proportional difference in sample size between cohorts.
Primary home language of parent.
Based on family income.
Data collected in kindergarten.
Children with disabilities were defined as those with an individualized education plan.
Child is eligible under Section 504 of the Rehabilitation Act to receive accommodations to ensure their academic success.
p < .05; **p < .01.
Language and Literacy Instruction in Sunnyside Schools
Throughout our study time period (2017–22), district curricula remained consistent across cohorts. Sunnyside primarily used a custom district-designed reading curriculum that was supplemented by the Houghton Mifflin Harcourt Journeys reading program. The Sunnyside curriculum was designed to be aligned with evidence and address all five pillars of reading (i.e., phonics, phonemic awareness, vocabulary, fluency, and comprehension; National Reading Panel, 2000). Schools used a range of Tier 2 and Tier 3 interventions that remained essentially the same over study years.
The Sunnyside district also was subject to changes in Florida state law related to reading instruction. Over the past decade, the State of Florida enacted reading-related legislation that prioritized the identification and support of children with reading difficulties alongside a renewed focus on instruction aligned with the “science of reading.” Key reading-related laws were passed in 2017 and 2023. Both our pre-COVID-19 and COVID-19 cohorts were subject to laws passed in 2017 that required the use of explicit, systematic, and sequential reading interventions. These laws also required professional development and teacher training programs to teach explicit and systematic instructional approaches. The 2023 changes, which mandated phonics as the primary instructional strategy for teaching word-reading (and banned use of the three-cueing method), occurred after the study period ended.
Instructional Modes During COVID-19
VPK Instruction
Sunnyside schools closed on March 16, 2020, due to COVID-19 and remained physically closed for the remainder of the school year. Therefore, children in the COVID-19 cohort missed 47 of 180 face-to-face instructional days in pre-K, ~26% of the school year. Remote instruction began at the end of March 2020 and was conducted for the remainder of the VPK school year. Remote instruction included optional live online sessions with teachers, recorded lessons, and worksheets/learning resources sent home.
Kindergarten Instruction
During the COVID-19 cohort’s kindergarten year (2020–21), children had the option of attending school remotely or in person. Remote instruction was provided via a separate online learning system (i.e., Canvas) and a videoconferencing app. Online classes were developed and taught by district teachers and used the same curriculum as the in-person option. Children and teachers followed a specific daily schedule that included time for independent work, lessons taught synchronously by the teacher, and small-group instruction.
Once enrolled in remote instruction, students were required to complete the entire quarter as an online student. After that, families could request that their child move from a remote learning mode to an in-person classroom. Families also could request that children move from in-person instruction to remote instruction. Less than half the children began the kindergarten year as remote learners (42.7%); by the end of the school year, only 9.7% of children were learning remotely (Table 2). Children who attended school remotely during the kindergarten year also took assessments remotely.
Learning modes in kindergarten for the COVID-19 cohort
Children’s learning mode at the beginning of the fall semester in kindergarten.
Children’s learning mode at the beginning of the spring semester in kindergarten.
An indicator of learning mode change between the kindergarten beginning of the year and end of the year.
First Grade Instruction
In the first grade year for the COVID-19 cohort (2021–22), the school district did not permit children to attend school remotely or take assessments remotely—all children enrolled in Sunnyside elementary schools attended school and took assessments in person.
Measures
Our measures included several assessments used to measure language and literacy skills in pre-K, kindergarten, and first grade. Table 3 provides information about when each assessment was conducted and the language and literacy skills assessed by each.
Language and literacy assessments administered in Pre-K, kindergarten, and first grade
VPK, Voluntary Pre-K
Note. The spring VPK Assessment was not administered to the COVID-19 cohort because school was canceled.
Pre-COVID-19 Literacy and Language Skills: VPK Assessment
Florida VPK providers administered the VPK Assessment, a universal screening measure for early academic skills, at the fall, winter, and spring of the pre-K year (Academic Period, or AP, 1, 2, and 3, respectively). The VPK Assessment measures four domains: oral language, phonologic awareness, print knowledge, and mathematics. In this analysis, the VPK Assessment scores from fall and winter allowed us to compare our two cohorts’ language and literacy skills at baseline. VPK Assessment scores as well as child demographic variables therefore were used as covariates for estimating the propensity scores and matching students between cohorts. The VPK Assessment has moderate to high internal consistency and test–retest reliability (for our study sample, McDonald’s omega = .74 for AP1 and .78 for AP2; Cronbach’s alpha = .71 for AP1 and .77 for AP2). Although these alpha values are within an acceptable range (i.e., >.70; Tavakol & Dennick, 2011), readers should be aware of the moderate reliability of the VPK Assessment.
Kindergarten and First Grade Language and Literacy Skills: MAP Reading
Children in Sunnyside elementary schools take the MAP Reading assessment twice in kindergarten (winter and spring) and three times in the first grade (fall, winter, and spring). MAP Reading is a computer-adaptive assessment that measures children’s academic achievement in literacy and language skills and tracks their growth in literacy and language skills over time. MAP Reading uses the Rasch Unit (RIT) scale for scores to facilitate comparisons across children and at multiple timepoints. The assessment is untimed but typically takes <30 minutes to complete. MAP Reading assesses children’s knowledge of language and literacy across four subdomains, including phonics, vocabulary, writing conventions, and comprehension (Table 3). It contains audio support for items where decoding is not the skill being assessed (e.g., phonologic awareness items) because many children in kindergarten and first grade are not yet reading text independently. Children learning remotely during the kindergarten year in Sunnyside also took the MAP Reading assessment remotely. MAP Reading assessment procedures include several safeguards for accuracy and security during remote administration, including a remote test proctor at the school or district level who must start the test administration for children and a test engagement capability that detects and eventually invalidates the results if students are rapidly responding (i.e., guessing) on a test.
The developer of the MAP Reading assessment provides evidence of reliability, validity, and fairness in its technical manual (Northwest Evaluation Association [NWEA], 2019). MAP Reading assessment shows good test–retest reliability (ranging from .69 to .86 over kindergarten and first grade time points) and strong internal consistency (ρθ = .96 in kindergarten and .97 in first grade). For our study sample, McDonald’s omega = .92, .87, .87, .86, and .86 for RIT and Subdomains 1–4, respectively, and Cronbach’s alpha = .92, .88, .86, .86, and .86 for RIT and Subdomains 1–4, respectively. Although evidence of concurrent validity was not available for MAP Reading in kindergarten or first grade, the MAP Reading assessment for third grade showed good concurrent validity with state reading tests administered to the same students (r = .79). An analysis of differential item functioning in the MAP Reading item pool found that most of the items exhibited negligible levels of differential item functioning, suggesting that items were generally fair across student subgroups (i.e., different genders and racial/ethnic backgrounds).
Child and School Demographic Characteristics
We included several child-level variables in our analysis, such as gender, race/ethnicity, the primary home language of the child’s parent(s), free or reduced-price lunch status, disability status, 504 Plan eligibility (i.e., qualifying for academic accommodations), and children designated as dual language learners by their school. We also included the school-level demographic factor of Title I funding status, which is based on the percentage of children at the school from low-income families. Demographic variables in our dataset were reported during the VPK year and updated in kindergarten and first grade if they changed. All VPK child variables were used as covariates in the propensity score analysis. The updated child variables were used as covariates in the models we built to address the research questions.
Analysis Plan
Before we estimated the effects of COVID-19 school disruptions on each child’s growth trajectories, we conducted missing-data analysis by cohort. We also conducted a series of preliminary analyses (e.g., examining demographic differences using χ2 tests and MAP score differences using t tests) to investigate whether COVID-19 had any impact on the number and demographic makeup of students taking the MAP Reading assessments. After examining whether the missing mechanism is missing at random (i.e., missingness as a function of the observed data but not the missing data; Rubin, 1976), multiple imputation by chained equations was performed 30 times (m = 30) using the mice package (Van Buuren, 2021) in R to impute the missing data in the covariates and the MAP Reading scores.
To answer the research questions, which investigated the long-term impact of COVID-19 on language and literacy development for pre-K children compared with their pre-COVID-19 counterparts, we used propensity score matching (PSM; Rosenbaum & Rubin, 1983). PSM allowed us to match and compare the COVID-19 cohort with a counterfactual cohort that had not experienced school closures (the pre-COVID-19 cohort). We matched the COVID-19 cohort and pre-COVID-19 cohort children on demographics and all available pre-K literacy assessment (VPK) scores and then compared matched children with respect to their longitudinal MAP Reading scores at five time points. Because the repeated measures of MAP Reading scores are nested in children and children are nested in schools, we accounted for the nested structure in data analysis, which we explain further for each analytic method we used in this study. The intraclass correlation values for the dependency of children nested in schools are .07, .11, .07, .07, and .09 for the five RIT scores for the pre-COVID-19 cohort, respectively, whereas the intraclass correlation values are .06, .08, .09, .09, and .12 for the COVID-19 cohort, respectively.
We first estimated the propensity scores (the probability that a child belonged to the treatment condition) using logistic regression with the cohort membership as a binary outcome and VPK covariates and their interactions as predictors (Green & Stuart, 2014). Furthermore, we included dummy-coded school variables as fixed effects to account for the clustered structure of the data (Arpino & Cannas, 2015). We then implemented the propensity scores through full matching with a caliper of .03 (Hansen, 2004), which has been shown previously to be effective at reducing bias due to confounding (Stuart & Green, 2008). We further specified the average treatment effect on the treated as the target estimand because we were interested in the impact of school disruption on the random COVID-19 cohort student. We diagnosed the covariate balance through a visual evaluation of the area of common support using a kernel density plot (see online Supplemental Figure S1) and absolute standardized mean differences between cohorts with a cutoff value of .25 SD (Stuart, 2010; Thoemmes & Kim, 2011).
To address the first research question (RQ1), we compared learning between COVID-19 and pre-COVID-19 children at each time point using a multilevel linear regression model with the matched data. We used school ID to identify the clusters and added cohort membership as the only predictor. For RQ1a and RQ2, we estimated the impact of school disruptions on students’ growth trajectories with the matched data using multilevel unconditional latent growth models (LGMs; Duncan et al., 2006). We used Mplus 8.3 (Muthén & Muthén, 1998-2017) to model the five MAP Reading scores in a multivariate format and simultaneously accounted for the nesting structure of children within schools in a two-level model. Combining PSM (i.e., matched data) with a multilevel latent growth model allowed us to estimate the growth trajectories of students impacted by the school disruptions while accounting for nesting. Specifically, for RQ2, we isolated the effect of the learning mode and allowed comparisons of results with and without remote learners by conducting two separate comparisons: all students in the COVID-19 cohort (i.e., remote and face-to-face learners) versus the pre-COVID-19 cohort and then only the face-to-face learners in the COVID-19 cohort versus the pre-COVID-19 cohort. With the 30 imputed datasets, we used the Mplus option type = imputation in the data command to run each dataset and pool the results.
To answer RQ3, we tested the cohort difference on the specific kinds of language and literacy skills with matched datasets using a multilevel linear regression model. Cohort membership was the only predictor in the random slopes and intercepts model. Note that we conducted PSM with each of the 30 imputed datasets and pooled the results using the MatchThem package (Pishgar et al., 2021) in R to arrive at a single set of coefficient and standard error estimates from the imputed datasets.
Finally, to investigate whether preschool disruptions made differential impacts on subgroups of children (e.g., those who received free or reduced-price lunch [RQ 4]), we included demographic variables in a multilevel conditional latent growth model to explain variance around the mean values of the intercept and slope (Kline, 2016). We tested all the demographic variables at the within level with the school’s Title I status as the between-level predictor. We again used Mplus 8.3 with the imputed datasets to answer this research question.
Results
Preliminary Analysis
No substantial differences were found in the patterns of missingness between pre-COVID-19 and COVID-19 cohorts (see online Supplemental Table S1). Child demographics generally also were comparable between cohorts (see Table 1), although the number of children with disabilities with an individualized education plan showed substantial variation between cohorts, with 89% of students in the COVID-19 cohort categorized as “no disability” in kindergarten compared with 64% in the pre-COVID-19 cohort. As shown in Tables 4 and 5, students’ MAP Reading scores were then compared between cohorts without any adjustment (no covariates controlled for, using unmatched data). The independent-sample t tests suggested that the mean differences were not statistically significant in the winter and spring of kindergarten, but there were statistically significant mean differences between the pre-COVID-19 cohort and COVID-19 cohort starting in the fall of first grade (i.e., the third time point) favoring the pre-COVID-19 cohort. In addition, several two-way analyses of variance were performed to evaluate the interaction effect between cohort membership and each demographic variable on the MAP Reading and subdomain scores without controlling for any other covariates. The results indicated significant interactions between cohort membership and some of the demographic variables (e.g., ethnicity and gender); see online Supplemental Table S3 for details. In other words, the mean differences between cohorts in students’ MAP Reading and subdomain scores depended on students’ demographics. However, it should be kept in mind that we used the unmatched data for preliminary analysis and the cohort differences should be evaluated with the matched data in which we controlled for students’ baseline differences, as presented next.
Descriptive information for MAP Reading assessments by cohort
RIT = Rasch Unit score.
Note. Mean score difference between cohorts was tested by running t tests. Cohen’s d also was calculated to represent the effect size. Text in bold represents statistically significant score differences between cohorts.
p < .05; **p < .01.
MAP Rasch Unit (RIT) score differences between cohorts at each time point using matched datasets
Note. RIT difference was estimated by taking the mean score difference between the COVID-19 cohort and the pre-COVID-19 cohort; Cohen’s d was calculated for the multilevel models to represent the effect size (i.e., the unstandardized regression coefficient b divided by the pooled within-group standard deviation SDpooled of the RIT scores (Feingold, 2015).
p < .01.
Analyses Using PSM Sample
After propensity score estimation, all the VPK covariates were balanced between cohorts. For detailed results of covariate balance and common support, see online Supplemental Table S2 and online Supplemental Figure S1. We answered the research questions using the PSM sample (pre-COVID-19: n = 834; COVID-19: n = 1,059).
Long-Term Impacts of COVID-19 School Disruptions (RQ1 & RQ1a)
To estimate the long-term impacts of COVID-19 school disruptions on children’s language and literacy development, we compared RIT score differences between cohorts at each time point. In Table 5, we see that the performance of the COVID-19 cohort declined over time compared with the pre-COVID-19 cohort. There were no significant differences in MAP scores between the COVID-19 and pre-COVID-19 cohorts at the kindergarten time points. However, the difference between cohorts became statistically significant at the beginning of first grade and remained significant for all three first grade time points. By the end of first grade, the COVID-19 children’s language and literacy MAP scores were 4.20 points lower than those of the pre-COVID-19 children, with an effect size of d = −.50.
To answer RQ1a, which evaluates differences in growth trajectories between the two cohorts, we adopted unconditional multilevel LGMs to examine overall change patterns and the degree to which students differed at the initial measurement and slopes of their trajectories. Table 6 shows that the pre-COVID-19 cohort (Mslope = 6.6; p < .01) and the COVID-19 cohort (Mslope = 5.6; p < .01) both demonstrated significant overall growth patterns across the five time points. Both cohorts also demonstrated significant variances in the intercept and slope factors at the student level, indicating significant individual differences in the starting points and growth rates. Furthermore, the significant negative correlation between the slope and intercept factors for the COVID-19 cohort (r = −.32; p < .01) suggests that students with higher initial performance showed slower growth over time. Such a pattern was not observed for the pre-COVID-19 cohort. Finally, the Wald test results indicated that only the slope is significantly different between cohorts (W(1) = 48.3; p < .01) but not the intercept (W(1) = 1.1; p > .05), suggesting that the pre-COVID-19 cohort had a significantly higher rate of growth than the COVID-19 cohort, even though they were not different at baseline. At the school level, only the COVID-19 cohort showed significant variations across schools in the growth rate (i.e., between-school variance for the slope factor = .54; p < .05), whereas the schools in the pre-COVID-19 cohort showed more homogeneous growth rates (between-school variance for the slope factor = .08; p > .05).
Unconditional latent growth model results for both cohorts using matched sample (unstandardized)
SE, standard error; n, sample size after propensity score matching; df, degree of freedom; CFI, comparative fit index; RMSEA, root-mean-square error of approximation; SRMR, standardized root-mean-squareresidual
Note. All information is based on the average estimates of 30 replications.
p < .05; **p < .01.
Differential Impacts of COVID-19 for Remote Learners (RQ2)
Next, we addressed RQ2, which investigates whether the long-term impacts of COVID-19 differed for students who attended kindergarten remotely. Nearly half the students attended kindergarten remotely (48.4%) at some point in the school year; t tests and one-way analysis of variance did not show significant mean differences for the MAP scores of these remote learners versus in-person learners on any of the five measurement occasions in kindergarten and first grade. Note that children learning remotely in kindergarten also took MAP kindergarten assessments online, so any impacts seen in kindergarten could be related to either factor. Next, using the matched data, we conducted two separate analyses comparing all students (i.e., remote and face-to-face learners) with the pre-COVID-19 cohort and then comparing only the face-to-face learners with the pre-COVID-19 cohort. Based on visual inspection of Figure 1, the two sets of comparisons showed similar trends across time, indicating that differences in scores between the pre-COVID-19 and COVID-19 cohorts were maintained even when remote learners were removed from the analysis. However, score differences between cohorts were larger when remote learners were included in the comparisons, based on Figure 1.
MAP Rasch Unit (RIT) score differences based on matched data between pre-COVID-19 and COVID-19 cohorts over time.
To further test the impact of remote learning on students’ growth trajectories, we performed multilevel unconditional LGMs only on the face-to-face learners in both cohorts. This approach isolated the effect of learning mode differences by only comparing face-to-face learners in both cohorts. We compared the results of these LGMs with those performed earlier that included both face-to-face and remote learners (RQ1a). As shown in Table 7, the slope factor was still significantly different between cohorts, with the pre-COVID-19 cohort having a slightly higher rate of growth (W(1) = 9.82; p < .01). Because the difference between cohorts in the growth rate remained without the remote learners, we do not have evidence that remote learning was the reason for the declined performance of the entire COVID-19 cohort. However, remote learning in the COVID-19 cohort might have some negative impact on remote learners whose initial MAP scores were higher, according to the correlation coefficients between the intercept and slope factors (rremote+f2f students = −.26; p < .01; rf2f students = −.05; p > .05).
Unconditional latent growth model results for face-to-face learners using matched samples (unstandardized)
SE, standard error; n, sample size after propensity score matching; df, degree of freedom; CFI, comparative fit index; RMSEA, root-mean-squared error of approximation; SRMR, standardized root-mean-squared residual
Note. All information is based on the average estimates of 30 replications.
p < .05; **p < .0.
Impact of COVID-19 on Language and Literacy Subdomains (RQ3)
To answer RQ3, we ran a multilevel linear regression model on the matched sample with the cohort membership as a predictor and school ID as cluster ID. The results showed statistically larger differences on Subdomains 3 (comprehension) and 4 (vocabulary), starting in the fall of first grade, with significantly higher means for the pre-COVID-19 cohort. Figure 2 presents the cohort difference for all the subdomains at each point in time. The positive difference value indicates that the pre-COVID-19 cohort outperformed the COVID-19 cohort. This result indicates that the difference between cohorts generally increased over time across subdomains, but the specific unconstrained language and literacy skills in Subdomains 3 (comprehension) and 4 (vocabulary) were more impacted by COVID-19 school disruptions than the constrained skills measured by Subdomains 1 (phonics) and 2 (grammar and writing).
MAP Rasch Unit (RIT) score differences based on matched data between cohorts by subdomains.
Impact of COVID-19 on Language and Literacy by Children’s Background Characteristics (RQ4)
To examine whether children’s background characteristics moderated the long-term impacts of COVID-19, we used a multilevel conditional LGM (Bollen & Curran, 2006) that included covariates measured at the beginning of kindergarten to predict intercepts and slopes over time for both cohorts. When looking at each cohort individually, there were several background characteristics that influenced students’ initial MAP Reading scores and their growth rates over time (e.g., Black children’s initial performance was significantly lower than the initial performance of White children; students who were eligible for free meals showed a slower growth rate over time). However, when the intercept and slope factors in the conditional LGMs were compared across cohorts, the Wald tests only indicated three significant differences. The first two differences suggested that Asian students in the pre-COVID-19 cohort had higher initial performance (W(1) = 4.49; p < .05) and also that students with the limited English proficiency status in the COVID-19 cohort had higher initial performance (W(1) = 8.33; p < .01). The slope difference suggested that students in the COVID-19 cohort with language disabilities had significantly slower rates of growth than students with language disabilities in the pre-COVID-19 cohort (W(1) = 7.38; p < .01). However, these findings should be interpreted with great caution because only a very small number of children were diagnosed with language disabilities in the COVID-19 cohort (n = 25; 2%) compared with children with language disabilities in the pre-COVID-19 cohort (n = 153; 13%).
Discussion
This paper reports on the long-term impacts of COVID-19 on children’s language and literacy development. We focused specifically on an understudied but particularly vulnerable group of children—those attending pre-K at the onset of COVID-19—and followed them through the end of their first grade year. Our analysis adds to the existing literature not only by reporting on longitudinal COVID-19 impacts during the critical early childhood window but also by examining COVID-19’s varying impacts on different kinds of foundational language and literacy skills. Three key findings emerged from this study. First, the negative impacts of COVID-19 on language and literacy development emerged later in our early childhood sample, but the magnitude of those impacts continued to increase over time. Second, we found that specific language and literacy skills were more impacted by COVID-19 than others, with children falling further behind in their development of unconstrained skills such as comprehension compared with constrained skills such as phonics and grammar. Finally, our results suggested that the negative impacts of COVID-19 were not solely driven by remote learning, although the language and literacy skills of children who attended kindergarten face-to-face were impacted less severely than children who attended remotely.
Accumulating Effects of COVID-19 on Young Children’s Language and Literacy Skills
Our detailed longitudinal data allowed for a nuanced look at the impact of COVID-19 on young children’s language and literacy development over time. Overall, the COVID-19 cohort showed a significantly slower rate of growth in language and literacy skills during kindergarten and first grade compared with the pre-COVID-19 cohort. A closer look at children’s point-by-point MAP Reading scores over time showed that the differences in scores between the COVID-19 and pre-COVID-19 cohorts gradually widened over time. There were no significant differences in the COVID-19 and pre-COVID-19 cohorts’ scores in the winter of kindergarten. By the spring of kindergarten, only 0.54 points on the MAP assessment divided the two groups. However, by the time children returned for their first grade year, the gap in scores between COVID-19 and pre-COVID-19 children had widened to a statistically significant 2.81 points. In other words, statistically significant impacts of COVID-19 on language and literacy emerged only by the fall of first grade for our sample, approximately 18 months after COVID-19 closed schools in March 2020. This gap continued to widen at the winter testing time point, and by the spring of first grade, 4.20 points separated the reading scores of the COVID-19 and pre-COVID-19 cohorts (d = −.50). In terms of practical significance, average growth on the MAP Reading score in first grade should be about 15.47 points from fall to spring (9.92 points in the fall semester and 5.55 points in the spring semester; Northwest Evaluation Association [NWEA], 2020). Therefore, a difference of 4.20 points suggests that the COVID-19 cohort was “behind” the pre-COVID-19 cohort by roughly 2.5 months by the end of first grade.
Findings in this study therefore reveal a pattern in which COVID-19 impacts appear to be worsening or accumulating over time, with the largest differences between COVID-19 and pre-COVID-19 children appearing at the end of first grade in the spring of 2022. Young children seemed to weather initial school closures in pre-K fairly well (e.g., Hadley et al., 2023), but the continuing instabilities in learning and children’s lives in the kindergarten and first grade years appeared to take a larger and ongoing toll on language and literacy development. These findings, which represent a negative developmental cascade, can be interpreted from a theoretical perspective as a natural and perhaps even predictable outcome of major disruptions to children’s lives inside and outside of school. According to the componential model of reading (Aaron et al., 2008), reading achievement is influenced by three main domains: cognitive (i.e., constrained and unconstrained reading skills), psychological, and ecological. COVID-19 and its associated disruptions to learning and development in and out of school likely had negative impacts across all three domains. In particular, young children’s learning and development are driven by the regular, reciprocal interactions that occur within a series of nested ecological contexts such as family, school, and community (Bronfenbrenner & Morris, 1998). Disruptions to these interactions and instability in the settings in which they occur are predicted to threaten not only academic achievement but also psychological and social development (Bronfenbrenner & Morris, 1998). During the COVID-19 years, research has suggested that young children experienced profound shifts in the way they interacted with others and spent their time across settings as caregivers and teachers experienced large increases in stress that potentially shaped interactions with children, children had fewer face-to-face interactions with teachers and peers, and there were large increases in the amount of screen time for young children (Dore et al., 2021; McKenna et al., 2021; Oppermann et al., 2021). That this continued instability in children’s learning environments consequently influenced their language and literacy trajectories in ways that are visible on standardized assessments is therefore not altogether surprising.
However, such impacts are of great concern for children’s long-term success as readers and learners. Kindergarten and first grade play a major role in the building of foundational language and literacy skills, with most children learning to read at least simple texts during their kindergarten year (e.g., Bassok et al., 2016). Other essential competencies developed in kindergarten and first grade include learning emergent academic language, gaining an awareness of and ability to manipulate sounds in language (i.e., phonemic awareness), decoding words automatically, and reading short texts accurately and fluently (Foorman et al., 2016; National Early Learning Panel, 2008). Additionally, children also develop self-regulation skills and knowledge about how to “do school” in the early grades that are equally important for academic success (Graue, 2006; Ursache et al., 2012). Deficits in these early foundational skills can have potentially serious impacts unless remediated. For example, Mol and Bus (2011) described an upward spiral of causality throughout the grades in which children with better decoding and comprehension skills read more, and this increased exposure to text, in turn, further strengthened and developed children’s language and literacy skills. The converse is also true, in that children with weak early reading skills read less and consequently gradually continue to fall further behind in literacy development, showing a downward spiral of reading achievement (Morgan & Fuchs, 2007). This is consistent with research that has shown that little movement occurs in children’s reading trajectories after third grade without intervention (Little et al., 2021; Pianta et al., 2008).
The finding of accumulating impacts of COVID-19 in early childhood contrasts with recent results for older students, which have found that growth rates in the 2021–22 school year returned to pre-pandemic norms and that COVID-19-related gaps in test scores were beginning to narrow (EmpowerK12, 2022; Renaissance Learning, 2022). Interestingly, however, two large-scale national studies found that first graders were the only grade level that, rather than returning to normal rates of learning in 2021–22, continued to fall further behind in early language and literacy (Kuhfeld & Lewis, 2022a; Renaissance Learning, 2022). These findings suggest that young children also potentially may be experiencing a somewhat different pattern of COVID-19 impacts on language and literacy skills than their older peers. Although older students impacted by COVID-19 may have been able to catch up or fill in some missed content, young children are suffering from potentially long-lasting gaps in their early foundational skills that may continue to be felt for years to come.
Compared with other studies that examined COVID-19 impacts in the spring of 2022, the differences we observed at the end of first grade are relatively large in magnitude. For example, a national study of MAP Reading scores of first graders at the same time point differed by 2.22 points from a pre-COVID-19 sample (d = −.15; Kuhfeld & Lewis, 2022a). It is possible that differences in magnitude are related to differences in methods because few other studies used a PSM approach as we did here. However, the large impacts on language and literacy at the end of first grade also may reflect the developmental factors discussed earlier, in which older children had less severe learning loss by the end of 2022 (Curriculum Associates, 2023; Kuhfeld & Lewis, 2022a). For example, the i-Ready reading scores of sixth to eighth graders showed no COVID-19 learning loss in the spring of 2022 (Curriculum Associates, 2022). Another potential explanation for the magnitude of effects seen in this study is the local context and the possibility that the district studied here may have experienced larger negative impacts than the national average (e.g., Kuhfeld & Lewis, 2022b). Recent research has indicated that there was large variability in COVID-19 impacts among U.S. school districts, with impacts ranging from zero to nearly a full school year (Fahle et al., 2023). Larger declines in test scores were reported in districts that were low income, had large percentages of minoritized students, or remained remote or hybrid for long periods in 2020–21. Two of these three factors apply to Sunnyside, the large school district in west-central Florida reported on in this paper. Our sample included very high proportions of children living in poverty (70.4%) and from minoritized backgrounds (43.3%) compared with the averages reported by Fahle et al. (2023): 47.9% poor and 26.6% minoritized. Although we did not test district- or community-level factors, the severity of the learning losses reported here at the end of first grade may have been influenced by structural factors that relate to poverty and the marginalization of racialized groups at the district level, such as the resources available to schools, disruptions to the teacher workforce, and community-level impacts of COVID-19 on health and employment.
COVID’s Differential Impact on Unconstrained vs Constrained Skills
To better understand the specific language and literacy skills that were impacted by COVID-19, we examined differences in children’s MAP Reading scores by subdomain. By the end of first grade, children’s scores across subdomains were significantly lower for the COVID-19 cohort than for the pre-COVID-19 cohort. However, the unconstrained literacy skills captured by Subdomains 3 (comprehension of narrative and informational texts) and 4 (vocabulary use and functions) were substantially more impacted by COVID-19 than constrained skills in Subdomains 1 (phonics, phonologic awareness, and print concepts) and 2 (writing mechanics, grammar, and spelling). In fact, the impact of COVID-19 on unconstrained skills was roughly double that of constrained skills at the end of first grade (e.g., 6.11 and 4.79 point differences between cohorts for comprehension and vocabulary, respectively, and differences of 2.77 and 2.72 points for phonics and spelling, respectively; see Figure 2).
A closer look at young children’s learning experiences during the pandemic reveals several potential reasons for these findings. For young children, unconstrained skills are fostered through interactive, responsive instruction such as conversations with more advanced language users, shared book reading, and the use of open-ended questions (e.g., Zucker et al., 2013). Children who cannot yet read independently have few opportunities to access and understand the complex, rich language contained in texts without the mediation of an adult such as a caregiver or teacher. However, the population we examined in this study had limited opportunities to engage in these teacher-mediated interactive learning activities at school. Remote instruction for pre-K children during the spring of 2020 appears to have been chiefly comprised of activity packets, online activities including apps and videos of YouTube read-alouds, and occasional brief synchronous sessions (McKenna et al., 2021). In the following kindergarten year, some children in our sample attended school remotely, whereas others went to school in person. However, social distancing, absenteeism, quarantines, and other interruptions to learning meant that activities such as shared book reading, hands-on science experiments, face-to-face conversations, and collaborative play were more difficult to implement across instructional modalities. Unfortunately, these are also the very types of activities that research has repeatedly shown to foster oral language, vocabulary, and listening comprehension (Cabell et al., 2015; Hadley et al., 2019; Wright & Gotwals, 2017). Some evidence suggests that instruction during the pandemic, especially remote instruction, may have emphasized discrete, constrained skills such as letter knowledge that could be more easily taught through resources such as apps and worksheets and required scaffolding from adults (Hadley et al., 2024).
An additional concern revealed by our findings is that the widening gap between COVID-19 and pre-COVID-19 cohorts was particularly apparent in children’s comprehension scores. Although impacts on other language and literacy skills appeared to be leveling off by the end of first grade (see Figure 1), the negative impacts of COVID-19 on comprehension skills increased at every time point measured. This is particularly concerning because difficulties in comprehension often only become apparent in later elementary grades (i.e., the fourth grade slump) when students are expected to understand and learn from more complex texts (Chall et al., 1990). To see such large and widening gaps in comprehension at the end of first grade, when comprehension difficulties are often not yet visible, suggests that pandemic-era students will need systematic and intensive support to face the challenge of learning from texts in later elementary grades. Understanding the source of these difficulties in comprehension is also important. It is likely that the negative impacts of COVID-19 on other literacy and language skills (e.g., vocabulary, phonics, and grammar) are closely tied to children’s difficulties with comprehension. According to the SVR, comprehension is the product of both well-developed decoding and language skills (e.g., Hoover & Gough, 1990). If children struggle to decode text, they will be unable to understand it; similarly, if children do not understand the meanings of the words on the page, comprehension will be impaired. Declining comprehension scores may serve as a warning signal, then, that many of children’s underlying language and literacy skills, including those not explicitly measured by standardized tests, are compromised due to COVID-19-related factors.
Role of Remote Learning and Child Background Characteristics
Children in our COVID-19 cohort had the opportunity to attend kindergarten remotely during the 2020–21 school year. However, we did not find strong evidence that remote learning was the main driver of learning loss during the post-COVID-19 period. Within the COVID-19 cohort, the MAP Reading scores of remote and face-to-face learners were not significantly different in either kindergarten or first grade. When we compared pre-COVID-19 and COVID-19 cohorts, children who attended kindergarten face-to-face all year did have slightly smaller COVID-19-related differences in test scores in first grade than the full sample (see Figure 2). In other words, children who attended kindergarten face-to-face all year experienced some buffering of COVID-19 impacts but still suffered COVID-19-related learning losses in language and literacy. We also found that even after removing remote learners from the analytic sample, face-to-face learners in the COVID-19 cohort had overall slower growth rates in language and literacy than their pre-COVID-19 peers.
In contrast, a national study of the impact of remote instruction found that student achievement lagged significantly when schools switched to remote or hybrid instruction (Goldhaber et al., 2022). These analyses, however, examined learning mode at the school level: Remote instruction meant that a school was fully remote. Our study instead examined learning mode at the child level: Children in our district could attend either in person or remotely. Indeed, according to the categorization used in the study by Goldhaber et al. (2022), Sunnyside school district would have been considered an in-person district rather than hybrid or remote because children had the option to attend face-to-face 5 days per week (American Enterprise Institute, 2023). Therefore, we see our results as complementary to this previous national study by examining the variation in learning modes within a school district considered “in person” and the resulting impacts on children’s learning. It is possible that for our sample, the dosage of remote learning was so low that the potentially negative impacts on learning were blunted: In our sample, 43% of children were learning remotely at the beginning of the year, but by midyear, 90% of children were attending kindergarten in person. The local context likely contributed heavily to these high rates of in-person instruction: There was intense political pressure in Florida for public institutions to operate in person as soon as possible, leading Florida to become one of the first states to resume in-person instruction in schools.
Our last key finding was that the negative impacts of COVID-19 on language and literacy were not moderated by child-level factors such as race, ethnicity, English language learner status, or poverty. This finding differs from the findings of a number of other COVID-19 studies, which have found that Black and Latinx children, children who are dual language learners, and children from lower-income families suffered more negative academic impacts from COVID-19 than their peers (Kuhfeld & Lewis, 2022b; Pier et al., 2021; Relyea et al., 2023). Some of these discrepancies in results may be related to differences in methods (e.g., our use of PSM) or the context studied. For example, most of the children in Sunnyside qualified for free or reduced-price lunch, so there was little variation in our sample in terms of socioeconomic status. Another potential explanation can be found in the work of Fahle et al. (2023), who found in their national analysis that within higher-poverty districts and those with high percentages of minoritized students, COVID-19 impacts were similar for different subgroups of children (e.g., for both White and Black children). Our results here are consistent with these findings: Sunnyside is a high-poverty and racially diverse district, and COVID-19 impacts were relatively large but similar across subgroups. In discussing their results, Fahle et al. (2023) suggested that COVID-19-related learning loss may be more strongly impacted by district- and community-level factors than by the resources of individual families.
Our analysis of child background characteristics also suggests the need to further investigate the impact of COVID-19 on children with disabilities. In this study, children with language disabilities in the COVID-19 cohort had slower growth rates than their pre-COVID-19 counterparts, but given the small sample size of the COVID-19 cohort (n = 25), these results should be interpreted with caution. However, this result points to an important underlying issue: We observed large and striking differences in the number of children diagnosed with any disability in the pre-COVID-19 (n = 416) vrsus COVID-19 cohorts (n = 125). Other studies during this time period have found that services for children with disabilities were reduced or of lower quality during COVID-19, especially during the spring of 2020 and into 2020–21 (Sonnenschein et al., 2022; Steed et al., 2022). Our results further suggest that the diagnosis of children with disabilities also was compromised during COVID-19, perhaps because teachers and other professionals were unable to properly identify and assess children with learning differences when the children were learning remotely. Without a diagnosis and individualized education plan in place, many young children may have failed to receive needed services during the crucial pre-K to first grade years, meaning that a large group of children is likely at increased risk of experiencing difficulties in their later school years and beyond.
Limitations
One of the limitations of this research is that we did not have detailed school-level information about the instruction provided during the 2017–22 period our study spans, although we did have information about district-wide curricula and instructional materials. Therefore, it is important to note the possibility that some of the differences between cohorts reported here could be attributable to instructional changes not captured at the district level. Florida, like many other states, has been influenced by the nationwide science of reading (SOR) movement and its push for explicit phonics instruction and passed laws in 2017 and 2023 that mandated more SOR-aligned instruction (see details in “Method” section). Therefore, it is possible that COVID-19 had a less negative impact on children’s phonics skills than other skills because teachers may have used more explicit phonics instruction after COVID-19, thereby mitigating the impact of COVID-19 for this skill only. However, this hypothesis is unlikely for several reasons. First, the Sunnyside district used the same curricula across both cohorts. Second, the SOR movement has been active in Florida since before COVID-19, and both cohorts would have been subject to the 2017 changes in state law. The 2023 law took place after the time period captured in this study. However, we cannot completely rule out the possibility that instructional changes not captured by curricular or legislative shifts may have contributed to the pattern of findings for constrained versus unconstrained skills shown here.
Next, it is essential not to generalize the results of this study to all U.S. children nationwide because there was large variation nationwide in responses to COVID-19 and children’s experiences in returning to school. In particular, children in Florida during COVID-19 likely experienced relatively more in-person instruction than their peers elsewhere. It is also important to note that children who learned remotely during the kindergarten year also took assessments remotely; it is possible that such assessments were biased due to caregiver assistance. All children took first grade assessments in person.
Finally, our primary instrument, the MAP Reading assessment, is a valid and reliable measure of constrained and unconstrained skills. However, no single assessment or data source can fully capture all facets of children’s language and literacy development. There is a need for additional nuanced descriptive and/or qualitative studies that can further contextualize COVID-19 impacts by exploring how specific aspects of children’s language and literacy development may have been affected.
Implications and Conclusion
This study makes an important contribution to the existing literature on COVID-19 impacts in its longitudinal examination of the understudied early childhood population and its investigation of how specific language and literacy skills were affected during the most acute period of COVID-19-related disruptions to schooling. Our results suggest that COVID-19 may have had a particularly profound impact on the foundational language and literacy skills of young children and that these negative effects appear to be worsening over time. Many young children in this cohort would benefit from intensive research-based reading interventions or tutoring programs, and investments in learning for this group may be particularly crucial given research on the importance of early literacy to overall learning trajectories. Fortunately, there is ample evidence that well-designed reading interventions can move the needle on children’s comprehension outcomes in elementary school and beyond. Meta-analyses of K–3 reading interventions have consistently reported effect sizes of around .37–.38 on standardized language and reading comprehension outcomes (Denton et al., 2022; Wanzek et al., 2016), which, if applied to our sample, could come close to eliminating the gaps seen between the pre-COVID-19 and the COVID-19 cohorts. In particular, studies point to the promise of multicomponent reading interventions for the upper elementary grades (Vaughn et al., 2022), which provide instruction on both code-focused constrained skills and language-focused competencies such as background knowledge, vocabulary, and inferencing. Addressing the language and literacy challenges identified in this study will require a coordinated effort to implement interventions tailored to young children’s needs. By prioritizing early and sustained support, we can help mitigate the long-term impacts of COVID-19-related disruptions and set children on a path toward improved learning outcomes.
Supplemental Material
sj-docx-1-ero-10.1177_23328584251338816 – Supplemental material for Tracing the Impact of COVID-19 on Early Language and Literacy Development from Pre-K Through First Grade
Supplemental material, sj-docx-1-ero-10.1177_23328584251338816 for Tracing the Impact of COVID-19 on Early Language and Literacy Development from Pre-K Through First Grade by Elizabeth Burke Hadley, Siyu Liu, Eunsook Kim, Meaghan McKenna and Katharine Hull in AERA Open
Footnotes
Declaration of Conflicting Interests
The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This research was supported by a grant from the Spencer Foundation (#202200049) and the American Educational Research Association Division E Doctoral Student and Early Career Scholar Seed Grant Program. The views expressed are those of the authors and do not necessarily reflect the views of the Spencer Foundation.
Note: This manuscript was accepted under the editorship of Dr. Kara Finnigan.
Authors
ELIZABETH BURKE HADLEY is an associate professor at the University of South Florida College of Education. Her research focuses on oral language and literacy development in pre-K, with a focus on helping teachers create equitable and enriching classroom language environments for children.
SIYU LIU is a doctoral student at the University of South Florida College of Education. Her research focuses on applying propensity score methods in latent class/latent profile analysis.
EUNSOOK KIM is a professor at the University of South Florida College of Education. Her research focuses on quantitative research methodology and psychometrics, including structural equation modeling, multilevel modeling, mixture modeling, and measurement invariance testing.
MEAGHAN MCKENNA is an assistant research professor at the University of Connecticut in the Department of Educational Psychology in the Neag School of Education. Her research focuses on engaging in collaborative partnerships to solve educational problems of practice and enhancing how early literacy is defined, assessed, and supported within a multitiered system of supports.
KATHARINE HULL is a doctoral candidate at the University of South Florida College of Education. Her research interests include critical literacy, disciplinary literacy, and qualitative methodologies.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
