Abstract
Given the inequitable distribution of resources across school, neighborhood, and home contexts in the United States, lower resourced students may have had fewer opportunities to learn during the coronavirus disease 2019 pandemic, which may have caused previous disadvantages to accumulate during the pandemic. Nevertheless, research has yet to comprehensively explore how school, neighborhood, and home contexts—together—relate to perceptions of school quality and, ultimately, learning outcomes during the pandemic. To fill this gap, the authors leverage a unique multiwave survey of households across 47 states. Using multinomial modeling, the authors find that previous disadvantages were not always accumulated during the pandemic and that in some cases, perceptions of school quality and student learning improved for students who had struggled before the pandemic. The results also suggest stratification across race/ethnicity, parental education levels, school types, learning modes, and a range of learning resources.
Given the inequitable distribution of resources across school, neighborhood, and home contexts in the United States, lower resourced students may have had fewer opportunities to learn during the coronavirus disease 2019 (COVID-19) pandemic, which could exacerbate preexisting inequalities. For example, low-income students and students of color often attend lower resourced schools, which may have been less likely to offer effective distance learning programs during COVID-19 (Dorn et al. 2020; Herold 2020; Goudeau et al. 2021). Many of these students also tend to live in lower resourced neighborhoods, which may be less likely to have effective Internet connections, a necessary condition for effective distance learning (Ong 2020). As COVID-19 disproportionately affects lower resourced communities (Muñoz-Price et al. 2020), students from these communities may have been unable to access other community resources, such as libraries and civic centers during the pandemic. Furthermore, as these students often live in homes with economic resource constraints, they may be less likely to have the technology tools needed to access distance learning programs (Bacher-Hicks, Goodman, and Mulhern 2021). Moreover, students from lower resourced homes may be more likely to have working parents with less flexible jobs (Cerullo 2020), which could limit parents’ ability to provide academic assistance to their children.
Moreover, as noted by Blagg et al. (2020), these inequalities were often layered during the pandemic. For example, many students who live in high poverty areas not only live in areas that lack technology resources, but also live in (1) crowded areas, (2) areas with vulnerable economic sectors, (3) areas with high rates of single parenthood, (4) areas with high disability rates, and (5) areas with high levels of linguistic isolation (Blagg et al. 2020). Together these “layers” can accumulate disadvantages for vulnerable students during COVID-19.
Nevertheless, current studies on COVID-19 often consider school, neighborhood, and home contexts separately when exploring their relationships to learning, which can make these studies vulnerable to omitted variable bias. Furthermore, current studies on COVID-19 often focus on either school quality or student learning, which can make it difficult to understand how these contexts are and are not related. Moreover, with limited academic data in the 2020–2021 school year and, at times, conflicting messages regarding school safety, parents often made decisions on the basis of how they perceived school quality and student learning during the pandemic, which highlights the importance of often understudied parent perceptions. Beyond family-level decision making, Jabbari et al. (2022) demonstrated that instructional modalities during COVID-19 often reflected parents’ perceptions of school safety, suggesting that schools may also be responsive to these concerns. Policy makers can also be seen as responding to the perceptions and concerns of parents, who represent a core voting constituency, capable of political organization and mobilization, as evident in recent school board elections (Granitz 2022).
To fill this gap, we have conducted the first—and to our knowledge only—representative survey during COVID-19 that captures school, neighborhood, and home resources, as well as parents’ perceptions of school quality and student learning. By considering if historically and contemporaneously marginalized students, as well as students who were previously performing lower academically, are exposed to worse perceptions of school quality during COVID-19 and experience lower rates of perceived learning, we will be able to better understand if and how the pandemic accumulated disadvantages for particularly vulnerable students. Moreover, by exploring multiple student contexts, as well as their relationship to parents’ perceptions of school quality and learning, we will be able to identify areas for resource investment across school, neighborhood, and home contexts. These will be important considerations for policy makers and other stakeholders who are committed to curbing the accumulation of disadvantages endured by vulnerable students in the wake of the pandemic. We ask the following research questions:
How do demographic, student, and family characteristics, school enrollment and learning mode, and home and school resources relate to changes in perceptions of school quality during the pandemic?
How do demographic, student, and family characteristics, school enrollment and learning mode, and home and school resources relate to changes in perceptions of student learning during the pandemic?
Conceptual Framework
The “summer slide” refers to learning loss due to an extended break from instruction over the summer months (Kuhfeld and Tarasawa 2020). Although the actual size of the gaps has recently been questioned (Von Hippel 2019), historically, research on the summer slide has demonstrated that the learning loss associated with the summer months can often exacerbate achievement gaps across social class and race/ethnicity (Alexander, Entwisle, and Olson 2007; Downey, Von Hippel, and Broh 2004; Johnson and Wagner 2017; Quinn et al. 2016). These results align with core premises of the Coleman report (Coleman et al. 1966), which suggested that students’ family backgrounds were stronger predictors of educational achievement than school quality indicators. In the context of COVID-19, when many students are learning from home, the inequitable distribution of resources across family and neighborhood contexts can be seen as compounding the effects stemming from the inequitable distribution of resources across school contexts. In addition to directly providing resources, higher resourced families may also be better equipped to outsource education assistance and, by doing so, further widen opportunity and achievement gaps. This trend has been widely observed with high stakes tutoring (Buchmann, Condron, and Roscigno 2010; Smyth 2009).
In considering the accumulation of disadvantages during COVID-19, Kuhfeld, Soland, Tarasawa, Johnson, Ruzek, and Liu (2020) used learning patterns during extended breaks for more than 5 million students to create a series of projections of learning outcomes during COVID-19. These projection models not only predicted significant learning losses in math and reading for the upcoming school year overall, but also heterogeneous effects. Here, Kuhfeld, Soland, Tarasawa, Johnson, Ruzek, and Liu (2020) predicted that students in the bottom 25th percentile of current math and reading scores will demonstrate the steepest declines during COVID-19, whereas students in the top 25th percentile of current math and reading scores will demonstrate flat trajectories, and in some cases, sizable gains.
In the context of COVID-19, student learning disadvantages may not only arise out of instructional offerings, but also from students’ ability to access these offerings. Thus, COVID-19 may act less like an extended summer for all students (in which instructional offerings are uniform across students) and more like an extended “absence” for some students who are not able to access instructional offerings. In addition to families, schools and neighborhoods also play a pivotal role in students’ ability to access instructional offerings during COVID-19. These contexts structure the manner in which instruction is offered and accessed, whether it be through schools’ mode of learning (e.g., in person vs. online), the types of tools offered to access these modes of learning (e.g., laptops, tablets), or the neighborhood infrastructure needed to use these tools (e.g., broadband Internet).
Taken together, we hypothesize that COVID-19 will lead to steep and widespread inequities and that the COVID-19 “slide” will not only vary across family contexts, but school and neighborhood contexts as well. Moreover, in addition to observed learning losses, we hypothesize that the slide will be reflected in perceived learning losses as well. For example, we hypothesize that lower income families and families with additional needs (e.g., parents of children with learning disabilities, parents of children with previous academic difficulties) and fewer resources (e.g., the tools and experiences needed to support learning at home) will experience steeper declines in perceived learning during the pandemic (i.e., accumulation effects). In the context of COVID-19, perceptions of learning are uniquely important, as they help fill the void of observed learning metrics that were largely absent during the 2020–2021 school year, as well as help inform family decisions regarding their children and the schools that they attend. In alignment with the “summer slide” framework, parents’ perceptions of learning may guide future decisions regarding their children’s education (e.g., tutoring)
Literature Review
Although there is limited research on parents’ perceptions of school quality and student learning during COVID-19, initial research has demonstrated that the implications of COVID-19 on student learning outcomes tends to vary across student populations. This variation is often a result of school, neighborhood, and student characteristics, as well as family resources and experiences. We review the previous research on COVID-19 and learning outcomes in the following sections.
School, Neighborhood, and Student Characteristics
Previous research has demonstrated that learning opportunities and outcomes significantly varied across school and student characteristics during the pandemic, which can ultimately affect parents’ perceptions of school quality and student learning during the pandemic. One of the primary school characteristics related to learning has been instructional modality. Unlike spring 2020, where virtually all schools were completely remote, schools varied in their instructional offerings during fall 2020 and spring 2021. In a systematic review of 63 studies, data from 46 states showed that 50 percent of public elementary schools were fully in person, 36 percent were hybrid, and 12 percent were solely remote as of March 2021 (Weiland et al. 2021). As research has found that private schools were often more likely to offer in-person instruction (Fuchs-Schündeln et al. 2021) and districts with stronger teacher unions were more likely to offer remote instruction (Hartney and Finger 2022), some scholars, such as Hartney and Finger (2022), suggest that school governance models (i.e., charter, private, etc.) may be differentially related to instructional modes during the pandemic through varying degrees of responsiveness to institutional and environmental demands. However, recent research by Jabbari et al. (2022) suggests that the relationship between school governance models and school quality is not simply explained by instructional modalities, but rather moderated by modality, such that these relationships vary across the type of instruction offered.
When considering the relationship between instructional mode and learning, Halloran et al. (2021:1) leveraged testing data across 12 states, finding that compared with prior years, pass rates declined overall in spring 2021 and that these declines were largest in districts with less in-person instruction. When considering race/ethnicity, Oster et al. (2021) used data from the Centers for Disease Control and Prevention and the National Center for Education Statistics containing information from more than 1,200 districts across all 50 states, finding that from September 2020 to April 2021, White students experienced higher rates of in-person instruction than Black and Hispanic students. Even as most schools implemented some form of distance learning in the wake of the pandemic (Lake and Dusseault 2020), and all schools offered core instruction in the 2020–2021 school year, many schools were unable to provide students with technology devices and Internet access during these times (see Malkus and Christensen, 2020). Research has also noted difficulties in providing instruction to English-language learners (Catalano, Torff, and Anderson 2021; Chafouleas and Iovino 2021) and students with individualized education plans (Garbe et al. 2020). When considering neighborhood contexts, recent research by Rome and Lay (2022) revealed that usage rates and time on task across a large digital platform was lowest among urban areas and higher poverty ZIP codes; rates were also low for Hispanic students.
Given these trends in learning opportunities, learning outcomes have also been shown to vary across both school and student demographic characteristics. Using data from the 2021–2022 school year, Dawson (2022) demonstrated that urban schools, schools with lower percentages of White students, and schools with higher percentages of students in poverty experienced the steepest declines in learning compared with previous cohorts. When considering student outcomes in light of the disparities in learning opportunities, data suggesting that Black and Hispanic students were falling further behind in reading during the pandemic (Goldberg 2021; Kuhfeld, Tarasawa, et al. 2020) were unsurprising. Indeed, leveraging data from more than 2.1 million students across 10,000 schools in 49 states, Goldhaber et al. (2022) found that remote instruction was a primary driver in widening achievement gaps across race/ethnicity and school poverty. Lending support to our hypothesis on accumulation effects, Dawson (2022) used interim assessment data from 2021 to demonstrate that students who were furthest behind before the pandemic experienced the largest declines in growth compared with prior cohorts.
Family Resources and Experiences
Beyond school, neighborhood, and student characteristics, family resources and experiences can also be important predictors of student learning during the pandemic, especially in the context of distance education, in which not all families are equally equipped to provide the tools and guidance necessary for distance education. According to Ong (2020), two in five households with less than $50,000 in annual income have experienced limited access to technology and/or Internet during the pandemic. Conversely, most households with incomes greater than $100,000 reported that they had the access to the resources needed for their students to continue learning at home during the pandemic (Garbe et al. 2020). Furthermore, on the basis of data from the U.S. Census Bureau’s Household Pulse Survey, conducted in May 2020, households with higher incomes used online resources at higher rates than those with lower incomes, which used paper materials at higher rates (McElrath 2020). Moreover, using a robust sample of nearly 10,000 elementary school parents, Domina et al. (2021:1) found that parents’ education level predicted engagement with remote learning and that even after controlling for family socioeconomic resources, students with Internet-enabled devices and access to high-speed Internet had higher levels of engagement with remote learning.
It is also important to note that many parents faced additional barriers associated with distance learning and that these barriers often differed across family contexts. For example, as low-income parents were more likely to be essential workers, they may have been less likely to be able to assist their children with remote learning (Gandolfi, Ferdig, and Kratcoski 2021). Additionally, as these families may have faced greater exposure to COVID-19 (Wasfy et al. 2021), their children may have been more likely to experience infections and subsequent quarantines, as well as psychological trauma from the illness or death of a family member. Finally, women, Black men, Hispanic men, and Asian men (Dias 2021), as well as less educated workers (Kesler and Bash 2021), experienced a disproportionate share of layoffs during the early part of the pandemic, while women and Black renters faced a disproportionate share of eviction notices (Hepburn et al. 2021); together these trends may contribute to disparities in household instability, which can further disrupt child learning.
Data
While the previous section has summarized the important role of student, family, school, and neighborhood characteristics in learning during the pandemic, research has yet to simultaneously explore these characteristics in relation to perceptions of school quality and student learning during the pandemic. One reason for this shortcoming is the availability of data. To expand upon previous research we rely on the Socioeconomic Impacts of COVID-19 Survey, administered by the Social Policy Institute (2021) at Washington University in St. Louis. The Socioeconomic Impacts of COVID-19 Survey is a five-wave, online survey collected at quarterly intervals between April 2020 and June 2021. The survey sample was constructed using a quota-based sampling procedure that ensured the sample would reflect the U.S. population in terms of gender, age, race/ethnicity, and income. The survey was distributed to approximately 5,000 respondents from all 50 U.S. states and the District of Columbia. Roughly 50 percent of survey takers completed subsequent waves. Although research has demonstrated that online, nonprobability samples using Qualtrics panels tend to generate samples that closely approximate those of the General Social Survey (Zack et al. 2019), it is likely that our sample is not exactly representative of the U.S. population, as we focus on a subset of survey takers who have school-aged children. Nevertheless, our sample does collect a robust set of information for households across 47 states, the District of Columbia, and Puerto Rico.
This study is based on the last three waves of the survey administered in November and December 2020 (wave 3), February and March 2021 (wave 4), and May and June 2021 (wave 5), as these waves best capture the 2020–2021 school year. Of the survey participants, we focused on respondents with children younger than 18 years who responded to demographic and employment-related questions as well as child education questions. 1 If participants responded to multiple waves of the survey, we kept their most recent response to better capture experiences and perspectives throughout the 2020–2021 school year. Respondents who did not provide a response to the items used in this analysis were excluded using listwise deletion. The final analytical sample includes responses from 967 participants.
Variables
To construct the education outcomes of children during the pandemic, we use two education-related questions: perceptions of school quality and learning more or less during the pandemic. Starting with school quality, we asked participants to rate their perceptions of school quality both before the pandemic and currently (at the time of their survey period): “How would you rate the quality of your child’s school’s instruction [before COVID-19 (before March 13, 2020)/currently]?” Then the survey provided a five-level, Likert-type scale of “very poor,” “poor,” “average,” “very good,” and “excellent.” On the basis of the two questions, we reconstructed our dependent question into three categories—(1) school quality got worse, (2) school quality was the same, and (3) school quality got better—during the pandemic. Moving on to student learning, we asked, “Do you think your child is learning more or less than they would have if the COVID-19 pandemic had not occurred?” Then the survey provided a five-level, Likert-type scale: “My child is learning a lot more/somewhat more/the same amount/somewhat less/a lot less.” Because of small cell sizes on the extremes ends of this scale, we transformed the five-level scale into a three-level scale: (1) learning more, (2) learning the same amount, and (3) learning less.
Our empirical models consider five sets of explanatory variables—baseline and child attributes, demographic and family characteristics, school enrollment and instructional mode, home resources, and school resources—in addition to the prepandemic school quality measures.
Baseline and child variables include survey period; urbanicity (population density at the county level, thousands per square mile); COVID-19 cases (cumulative at the county level, logged); and child’s age, disability status, and school performance and proficiency levels. 2
Demographic and family variables include respondent’s gender 3 and race/ethnicity, number of children younger than 18 years, home language, parent’s (or parents’) educational attainment, employment status, and household income (area median income at the county level, adjusted for family size 4 ).
Enrollment and instructional mode variables include school type (public, public charter, or private and parochial) and learning mode (in person, online, or hybrid).
Home resources variables includes a set of binary variables regarding online learning tools, broadband Internet, a quiet space to study, and adult supervision availabilities at home.
School resource variables include a set of binary variables indicating whether schools and teachers provide daily grades or feedback on assigned work, instructional videos from the Internet (e.g., YouTube), homework, one-on-one instruction, and video conference meetings with the whole class.
Table 1 reports summary statistics for the variables in use.
Descriptive Statistics.
Categorical variables (percentage).
Categorical variables (percentage); categories are not mutually exclusive.
Continuous variables.
Empirical Model Design
As our conceptual framework does not restrict the direction of the impacts of COVID-19 on perceptions of school quality and child learning patterns, we employ a multinomial logit (MNL) modeling strategy assuming “no change” as the base outcome and simultaneously testing multiple outcomes against the base category. For each of the two dependent variables, we conducted a series of MNL models, adding demographic and family, enrollment and instructional mode, home resource, and school resource variable sets to the baseline and child variable sets in a cumulative manner. While the school quality change model controls for the prepandemic school quality across all MNL models, the learning more or less model considers the prepandemic school quality measure as the final explanatory variable set. The statistical representation of each final model (i.e., containing all explanatory variable sets 5 ) is as follows:
where Pr(·) represents the logistic probability function. Yi = −1 for those who reported that school quality got worse (the school quality change model) or their children learning less (learning more or less model) during or because of the pandemic. Yi = +1 for those who reported that school quality got better (the school quality change model) or their children learning more (learning more or less model) during or because of the pandemic. Yi = 0 for those who reported that the pandemic does not affect school quality (the school quality change model) or children’s learning (the learning more or less model). Because MNL models are nonlinear, we report relative risk ratios (RRR) 6 instead of coefficients. The RRR ranges from zero to positive infinity, and an association is assumed positive if its ratio is significantly greater than 1. If the ratio is significantly less than 1, the association is negative.
Findings
In the following sections, we first describe all statistically and marginally significant relationships when variable sets are consecutively added to our model (i.e., main effects), and then describe how some of these relationships alter with the addition of subsequent variable sets (i.e., potential confounding or mediating effects). 7 For ease of interpretation we provide both full regression tables, as well as RRR plots for both marginally and statistically significant relationships in our final models.
School Quality: Main Effects
All school quality models account for parents’ prepandemic perceptions of school quality, wave, population density, and COVID-19 cases (Figure 1, Table 2). In models 1a and 1b we explore child characteristics. Having a learning disability was significantly associated with increased chances of better perceived school quality (RRR = 2.518, p < .001), which could signify more individualized attention during the pandemic. Concerning grades, it appears that parents of students with “average” levels of performance were less likely to perceive their children’s school quality as getting worse, whereas parents of students with “below average” levels of performance had mixed perceptions, as their school quality got worse for some and better for others.
Multinomial logit results: school quality change (base outcome: not changed). (A) Demographic attributes and (B) educational attributes.
Multinomial Logit Results: School Quality Change (Base Outcome: Not Changed).
Note: Data are exponentiated coefficients; values in parentheses are standard errors. Reference categories: school quality (average), gender (male), race/ethnicity (White), number of children (one), employment status (not working), income (low), school type (traditional public), and learning mode (in person). AIC = Akaike information criterion; BIC = Bayesian information criterion; COVID-19 = coronavirus disease 2019.
p < .10. *p < .05. **p < .01. ***p < .001.
In models 2a and 2b we explore family characteristics. Survey takers’ gender—identifying as female—was significantly associated with increased chances of worse perceived school quality (RRR = 1.610, p < .01) and marginally associated with decreased chances of better perceptions of school quality (RRR = 0.637, p < .10). Given that many of the historical gender norms around parenting appear to be reinforced during the pandemic (see Mize, Kaufman, and Petts 2021), this finding could signify women’s greater awareness of their children’s schooling. When considering working conditions, working outside the home or working full-time was often associated with worse perceived school quality, while having a spouse work full-time was associated with better perceived school quality. Here, working could signal increased levels of stress and frustration from having to balance one’s own employment work and supervision of their children’s school work. Conversely, not working may allow or, in the case of having a spouse work full-time, compel parents to help their children better engage with instruction. Future research is needed to further disentangle these relationships and understand the potential moderating role of gender. Unsurprisingly, experiencing an employment or income shock in the past three months was significantly associated with increased chances of worse perceived school quality.
In models 3a and 3b, we add school enrollment characteristics. Compared with traditional public schools, attending a private school is significantly related to decreased chances of worse perceived school quality (RRR = 0.532, p < .01), while attending a public charter school (RRR = 1.833, p < .10) and a private school (RRR = 1.698, p < .10) was marginally associated with increased chances of better perceived school quality. In line with Hartney and Finger’s (2022) findings, schools with fewer institutional demands (e.g., teacher unions) may have been able to adapt to the pandemic in a way that matched parents’ preferences. Compared with in-person learning, online (RRR = 1.626, p < .05) and hybrid (RRR = 1.581, p < .05) learning modes were significantly related to increased chances of worse perceived school quality, which affirms previous research demonstrating the positive impact of in-person learning.
Additionally, in models 4a and 4b, we add home resources. Having an online learning tool was significantly associated with decreased chances of better perceived school quality (RRR = 0.613, p < .05). Here, the use of online learning tools may act as a substitute for what parents perceive as better learning opportunities. Finally, in models 5a and 5b, we add school resources. Having daily videos (RRR = 0.669, p < .10) and one-on-one interactions with teachers (RRR = 0.549, p < .05) were marginally and significantly related to decreased chances of worse perceived school quality; having daily Zoom meetings was marginally associated with increased chances of worse perceived school quality (RRR = 1.407, p < .10) and significantly associated with increased chances of better perceived school quality (RRR = 1.922, p < .05). Multimedia and one-on-one interactions could represent active learning and engagement, while the heterogeneity with daily Zoom meetings could represent the differences in instructional quality offered during the pandemic.
School Quality: Potential Confounding or Mediating Effects
Finally, there were a few instances for which the relationships from initial predictors of school quality altered when additional sets of predictors were added in subsequent models. For example, when home resources were added in model 4, having a D or below in math or reading was no longer significantly associated with increased chances of better school quality. As home resources appear to partially explain the relationship between previous performance and school quality, it may be the case that additional home resources allow some families to provide their lower performing children with more school support. Furthermore, when home resources were added in model 4, attending a charter school was no longer marginally associated with increased chances of better school quality, yet when school resources were added in model 5, attending a private school was no longer marginally associated with increased chances of better school quality. Here, the relationships between charter schools and increased school quality may be partially explained by home resources (e.g., higher resourced families sending their children to charter schools), while the relationship between private schools and increased school quality may be partially explained by school resources (e.g., private schools having more school resources).
Student Learning: Main Effects
Similarly, all student learning models account for parents’ prepandemic perceived school quality, wave, population density, and COVID-19 cases (Figure 2, Table 3). In models 1a and 1b we explore child characteristics. Increased child age was marginally associated with increased chances of perceived learning losses (RRR = 1.044, p < .10) and decreased chances of perceived learning gains (RRR = 0.938, p < .01), suggesting that older students fared worse during the pandemic. Similar to the school quality model, having a learning disability was significantly associated increased chances of perceived learning gains (RRR = 2.046, p < .001). Being below proficient in math or reading was marginally associated with increased chances of perceived learning losses, while having a C (RRR = 2.199, p < .001) or a D or an F (RRR = 2.353, p < .001) in math or reading in the past two years was significantly associated with increased chances of perceived learning gains. Here, it appears that although those with lower prior performances on standardized tests lost more ground during the pandemic, those with lower performances on grades gained ground during the pandemic.
Multinomial logit results: learning more or less (base outcome: not changed). (A) Demographic attributes and (B) educational attributes.
Multinomial Logit Results: Learning More or Less (Base Outcome: Not Changed).
Note: Data are exponentiated coefficients; values in parentheses are standard errors. Reference categories: school quality (average), gender (male), race/ethnicity (White), number of children (one), employment status (not working), income (low), school type (traditional public), and learning mode (in person). AIC = Akaike information criterion; BIC = Bayesian information criterion; COVID-19 = coronavirus disease 2019.
p < .10. *p < .05. **p < .01. ***p < .001.
In models 2a and 2b we explore family characteristics. Similar to the school quality model, survey takers’ gender—identifying as female—was significantly associated with increased chances of perceived learning losses (RRR = 1.516, p < .05) and decreased chances of perceived learning gains (RRR = 0.513, p < .001). Having either parent with a bachelor’s degree or higher was marginally associated with increased chances of perceived learning gains (RRR = 1.515, p < .10), which could reflect their ability to offer their children more learning support. Compared with low-income households, being from a middle-income household income was significantly related to decreased chances of perceived learning gains (RRR = 0.578, p < .05). Middle-income parents could be at a unique disadvantage, as they may be ineligible for public benefits that could help aid their children’s learning, while not earning enough to provide these supports themselves.
In models 3a and 3b, we add school enrollment characteristics. Similar to the school quality model, attending a charter school was significantly related to decreased chances of perceived learning losses (RRR = 0.461, p < .05), while attending a private school was significantly associated with decreased chances of perceived learning losses (RRR = 0.312, p < .001) and marginally associated with increased chances of perceived learning gains (RRR = 1.540, p < .10). Compared with in-person learning, online (RRR = 1.655, p < .05) and hybrid (RRR = 1.948, p < .05) learning were significantly related to increased chances of perceived learning losses.
In models 4a and 4b, we add home resources. Having an online learning tool was significantly associated with increased chances of perceived learning gains (RRR = 1.494, p < .05), while having broadband Internet (RRR = 0.630, p < .05) and a quiet place to study (RRR = 0.582, p < .01) were significantly associated with decreased chances of perceived learning gains. Having adult supervision was significantly associated with increased chances of perceived learning losses (RRR = 1.892, p < .01). Here, online tools may aid students learning during the pandemic, while broadband Internet may offer additional opportunities to engage online (e.g., gaming, browsing) that do not increase learning. At the same time, having a quiet place to study and adult supervision may reflect parents’ deeper knowledge of their children’s actual learning during the pandemic.
In models 5a and 5b, we add school resources. Having daily feedback was significantly associated with decreased chances of perceived learning losses (RRR = 0.518, p < .01); having homework every day was significantly associated with increased chances of perceived learning losses (RRR = 1.829, p < .01); and having Zoom meetings every day was both significantly associated with increased chances of perceived learning losses (RRR = 1.546, p < .01) and perceived learning gains (RRR = 1.680, p < .01). Here, having daily feedback may help improve learning, while daily homework may reflect “busywork” that does not substantially affect learning.
In models 6a and 6b, we add prepandemic school quality. Compared with average prepandemic perceptions of school quality, those with poor (RRR = 3.339, p < .01), very good (RRR = 1.961, p < .01), and excellent (RRR = 2.096, p < .01) perceptions of school quality were significantly associated with increased chances of perceived learning gains. Here, those with poor prepandemic perceptions of school quality may have learned better outside of normal school operations, while those with good prepandemic perceptions of school quality may attend schools that are better equipped to promote learning outside of normal school operations.
Student Learning: Potential Confounding or Mediating Effects
There were also instances in which the relationships from initial predictors of student learning altered when additional sets of predictors were added in subsequent models. For example, when family characteristics were added in model 2, scoring below proficiency in math or reading was no longer marginally associated with increased chances of perceived learning losses. Furthermore, when school enrollment was added in model 3, having a D or lower in math or reading was marginally (and later significantly) associated with increased chances of perceived learning losses. Here, family characteristics appear to partially explain the relationship between proficiency and learning, while school enrollment and learning mode appear to partially explain the relationship between performance and learning. In the former case, family resources, such as parents’ education or income may negate the negative impact of scoring below proficiency; in the latter case, lower performance may be negatively related to certain school types, which, when not accounted for in the model, could suppress the effect of performance. Furthermore, increased child age was no longer significantly associated with increased chances of perceived learning losses when school enrollment and learning mode was added in model 3. Here, learning mode, which tends to vary across age during the pandemic (e.g., older children receiving virtual instruction), appears to partially explain the relationship between child age and perceived learning, which may signal the negative relationship between virtual instruction and learning.
Additionally, when home resources were added in model 4, being from wave 5 was no longer significantly associated with increased chances of perceived learning gains, and when school resources were added in model 5, being from wave 5 was no longer marginally associated with decreased chances of perceived learning losses. Here, home and school resources may partially explain the relationship between waves and learning, such that passages in time (e.g., responding in a later wave) no longer influences learning. This could signal that higher resourced homes and schools were better able to learn earlier in the pandemic. Furthermore, when home resources were added in model 4, survey takers’ gender was no longer marginally associated with increased chances of learning less, while being Black was marginally associated with decreased chances of learning less; however, the association between Black and learning dissipated when school resources were added in model 5. Although home resources appear to partially explain the relationship between gender and perceived learning losses, the opposite is true for survey takers’ race. Here, being Black may be negatively associated with certain home resources, which, when not included in the model, could suppress the effect of being Black. Yet these effects also appear to be partially explained by school resources, such that when school resources are accounted for, being Black is no longer related to perceived learning. This could signify lower home resources, yet greater school resources experienced by Black respondents during the pandemic. Moreover, when school resources were added in model 5, online learning mode was no longer significantly associated with learning less. As school resources appear to partially explain the relationship between learning mode and perceived learning, it could be the case that school resources alleviate some of the negative effects associated online learning. Finally, when prepandemic school quality was added in Model six, having a parent with a bachelor’s degree (or higher) was no longer significantly associated with perceived learning gains. As prepandemic school quality appears to partially explain the relationship between parental education and perceived learning, this finding could signal the advantages afforded to students with higher educated parents, especially in lower (perceived) quality schools. Conversely, this finding could signal the fact that higher educated parents may have been more likely to attend higher quality schools before the pandemic.
Discussion
Research on COVID-19 has yet to comprehensively explore how demographic, student, and family characteristics, school enrollment and learning mode, and home and school resources—together—relate to perceptions of school quality and, ultimately, learning outcomes during the pandemic. To fill this gap, we leverage a novel data set and a series of multinomial regression models across three waves of representative surveys during the 2020–2021 school year. Our research not only sheds light on who has experienced disadvantages during the pandemic, but also provides important policy and program insights that can be rapidly deployed across home, school, and neighborhood contexts.
When considering notable student-level influences, we found that disadvantages were not always accumulated during the pandemic and that in some cases perceptions of school quality and learning actually improved for previously struggling students. For example, perceptions of school quality increased for C students, but results for D or F students were mixed. Moreover, there were differences in learning outcomes across performance and proficiency levels. C and D or F students were perceived as learning more during the pandemic, whereas students who were below proficient in math or reading were perceived as learning less. There are multiple plausible explanations for these differences, including grade inflation during the pandemic. However, given the differences across performance and proficiency levels, it could also be the case that even though some schools struggled to teach core skills during the pandemic, students who were previously disengaged received more individualized attention. Individualized instruction may also explain why students with disabilities were perceived as learning more during the pandemic when considering academic performance. Indeed, distance learning may provide certain levels of flexibility and adaptability that may be beneficial for learning in this context. The use of effective interventions (Kim and Fienup 2022) could also explain part of this effect. Additionally, as Moreland-Russell and her colleagues (2021) recently found that parents of children with disabilities experienced improved mental health during the pandemic, it could be the case that parents feel more in control and better able to support their children’s learning in this context, which could translate into perceptions of learning.
At the family level, students whose parents had a bachelor’s degree or higher were perceived as learning more, potentially suggesting intergenerational transmission of educational achievement during the pandemic. Here, higher educated parents may have been better able to support their children’s learning in the absence of standard school instruction. Additionally, as prepandemic school quality appears to partially explain the relationship between parental education and perceived learning, high-quality schools can be seen as alleviating some of the advantages afforded to students with higher educated parents. Conversely, parents’ hardships also appeared to trickle down into their children’s educational experiences: students whose parents experienced economic hardships were perceived as having worse school quality during the pandemic. This suggests that the perceived quality of educational experiences during the pandemic may be related to parents’ ability to emotionally and economically support their children’s learning.
Given the positive relationships between children who attended charter or private schools during the pandemic and perceptions of school quality and student learning–when accounting for instructional mode, our findings might suggest that more autonomous school structures may have allowed teachers and school leaders to better adapt to the pandemic. Although this is in line with market theories of education (Chubb and Moe 1990), the literature connecting charter schools to innovation is often lacking, and in surveys such as ours it can be difficult to disentangle selection effects. For example, these findings might signal greater parental involvement (Preston et al. 2012) or how the preferences for schools that are “chosen” by “customers” are related to perceptions of school quality and student learning. In addition to school types, learning modes were also related to perceptions of school quality and student learning. Confirming prior research, students who engaged in online and hybrid learning were perceived as having worse school quality; unsurprisingly, these students were also perceived as having learned less. Moreover, even after accounting for school type and learning mode, students who had online learning tools were perceived as learning more during the pandemic, suggesting that that the digital divide may continue to play an important role in student learning even after the pandemic. In addition to home resources, school resources were also instrumental to school quality and student learning. Highlighting the importance of human resources, students who received one-on-one attention were perceived as having better school quality, whereas students who received daily feedback were perceived as having learned more.
However, our findings also suggest that it is not only enrollment decisions that matter but also the resources that underlie these decisions, which again, emphasizes the importance of considering these factors simultaneously. These trends are most evident in findings suggesting confounding or mediating effects. For example, the relationship between charter schools and increased perceptions of school quality appear to be partially explained by home resources, while the relationship between private schools and increased perceptions of school quality appear to be partially explained by school resources. Furthermore, as school resources appear to partially explain the relationship between learning mode and perceived learning, additional school resources could be seen as alleviating some of the negative effects previously associated online learning. Moreover, when considering racial dynamics across these contexts our findings suggest that children of Black parents have better perceptions of learning outcomes when we account for home resources, yet these outcomes dissipate when we account for school resources. Here, lack of home resources may hinder perceptions of learning among Black families, while school resources may improve these perceptions.
Finally, when considering that COVID-19 was associated with better perceptions of school quality and increased learning when accounting for population density, our findings could signal a trade-off between pandemic precautions and perceived learning outcomes. Future research that includes student and family health metrics will be needed to better understand these trade-offs. Although parents’ perceptions provide important insights into their experiences and perspectives of the pandemic, future research should also consider linking parents’ perceptions of school quality and student learning with administrative data from schools.
Conclusion
With limited academic data in the 2020–2021 school year, parents often made decisions on the basis of how they perceived school quality and student learning during the pandemic. These perceptions affected and will continue to affect school leaders and policy makers. In particular, our findings highlight some of the ways in which disadvantages were accumulated, while demonstrating stratification trends across race/ethnicity, parental education levels, school types, learning modes, and resources. In doing so, these findings have implications for school officials, as well as policy makers interested in curbing educational and societal equities.
In later waves of the survey, parents’ perceptions of school quality and student learning improved, suggesting that schools did adapt to the pandemic and learning improved. Although our findings highlight some of the difficulties with virtual and hybrid instruction, most schools have returned to in-person instruction. Nevertheless, certain aspects of remote learning, such as flipped classrooms, online discussion boards, and digital communication with parents, may be here to stay (see Klein 2022). Thus, as our research suggests the importance of online learning tools, policy makers and other stakeholders should consider these types of investments as long-term strategies, rather than COVID-19 stopgaps. If additional investments in resources across school, neighborhood, and home contexts are not made in the future, disadvantages may continue to accumulate, leading to rising inequalities that can last a lifetime.
Furthermore, the fact that students with both poor and good prepandemic parental perceptions of school quality were learning more in a disrupted context suggests that learning, to some extent, did occur for both the “haves” and the “have-nots.” However, this trend can be interpreted as maintaining, rather than reducing, inequalities. Moreover, learning expectations may have been lower among the “have-nots.” Beyond these explanations, these findings might suggest that students from lower (perceived) quality schools may have learned better outside of normal school operations, whereas students from higher (perceived) quality schools may have attended schools that were better equipped to promote learning outside of normal school operations, ultimately suggesting inadequacies and inequalities in the current system. Of course, it is important to recognize that descriptively, only 37 percent of respondents perceived poor school quality, while 39 percent reported that their children learned more, refuting notions that schools, overall, did not serve students well during the pandemic.
More generally, our research demonstrates that social contexts consistently matter, perhaps even more so, during times of social disruptions, like the COVID-19 pandemic. Indeed, the consistency across our school quality and student learning models demonstrate the salience of these contextual factors, as well as well as the links between school quality and student learning. When considering these contexts, the role of families appear especially important. For example, home resources and family characteristics appear to partially explain the relationships between poor performance and proficiency and perceptions of school quality and learning. Thus, policies that invest in families may allow them to provide their children, especially those who have previously struggled academically, with more support. Ultimately, these strategies can help improve learning for those who need it the most, as opposed to allowing these disadvantages to accumulate in their current state. Additionally, when home resources are accounted for, Black families experience perceived learning gains, suggesting that these strategies can also improve racial equity in education. Finally, it is important to note that the percent of variance in learning less or more during the pandemic nearly doubled (R2 increased from .078 to .143) when family characteristics were added to the model. Given that the majority of students spent at least some time learning at home during the pandemic, these results lend support to Coleman et al.’s (1966) original findings, which suggests that family resources can play an important role in generating gaps in learning.
Thus, although more must be done to shore up resources across schools and neighborhoods, policy solutions that consider family resources should be considered as well. The child tax credit, which was created in response to the COVID-19 pandemic, has clear implications for families’ investments in their children’s education (see Jabbari, Anglum et al., 2021). However, for families to make these types of investments, policies that invest in families–like the child tax credit–must also be pursued. Nevertheless, this is not meant to diminish the role of school contexts, home resources, and school resources, which collectively increase the percentage of variance explained by one third (from .143 to .213), but rather to suggest the importance of shoring up families in times of crisis.
Footnotes
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: JP Morgan Chase Foundation, Mastercard Center for Inclusive Growth, and Annie E. Casey Foundation.
1
For parents with two children or more, we randomly assigned one of the children and asked them to respond regarding the assigned reference child.
2
Our empirical models use two types of academic measures. First, we measure school performance: “my child received C (or D/F) in Math or Reading in the past 2 years”; second, we measure academic proficiency: “my child achieved below proficiency in math or reading in the past two years.” While the prior measure can be seen as capturing effort and other factors relating to grades, the latter measure can be seen as capturing a child’s academic mastery.
3
Because of survey limitations, we are able to observe only respondent’s gender; as such, we are unable to determine the gendered effects of the pandemic on children.
4
We adopt the U.S. Department of Housing and Urban Development’s income criteria as follows: low income (0 percent to 80 percent area median income), moderate income (80 percent to 120 percent area median income), middle income (120 percent to 170 percent area median income), and high income (≥170 percent area median income).
5
For consistency, school quality (as a variable set) is not included in the equation, as it was only used in the learning more or less models.
6
The RRR is equivalent, but not identical, to the odds ratio in a nonlinear regression model with binary outcomes (e.g., logistic or probit regression). Distinctively, RRRs compare the risk (or chance) of an event in one group versus the risk in the reference group, whereas odds ratios compare the odds of an event’s occurring in one group compared with another.
7
Although we do not formally test mediation through structural models, our “block-adding” approach allows to understand how accounting for new sets of variables may explain part of the effects of previously accounted for variables.