Abstract
Commonly regulated structural quality features, like educator education levels and group size, are thought to be foundational to the quality of children’s everyday experiences in early education and care settings. Yet little is known about how these features relate to the day-to-day interactions and activities that occur in these settings—or process quality features—across the landscape of early education and care. In this study, we examine the association between structural quality features and process quality features in a diverse sample of classrooms (n = 672) participating in a statewide study of early education and care. Using a permutation test approach, we found that group size and child-to-adult ratio were most consistently linked to children’s experiences but educator education, experience, and curriculum usage were largely unrelated. Implications of these findings for quality improvement initiatives are discussed.
High-quality early education and care programs are seen as a key policy approach for preparing children for kindergarten and beyond. Yet few programs today meet the quality standards considered necessary for supporting young children’s development (Chaudry et al., 2017). In response, the number of systemwide quality improvement initiatives—or Quality Rating and Improvement Systems (QRIS)—has nearly tripled in the United States between 2007 and 2017 (Build Initiative & Child Trends, 2019). Despite a great deal of investment in such initiatives, we still have little evidence on the critical, regulable features of early education and care programs that foster high-quality, developmentally supportive experiences for young children in these settings. This article addresses this broad issue by applying a novel empirical approach in a large, statewide data set to examine whether certain structural quality features consistently predict the everyday interactions and activities in early education and care settings.
Structural quality features include the readily quantifiable and regulable features of settings, like staff qualifications, group size, and classroom materials, hypothesized to underlie process quality features (Phillipsen et al., 1997). Process quality has been variously defined in the literature but generally reflects what actually happens on a day-to-day and moment-to-moment basis in settings, including activities and interactions between and among adults and children (Cryer et al., 1999; Howes et al., 2008; Pianta et al., 2005). Process features are considered the most proximal determinants of children’s learning and have been proposed as a key mechanism linking structural features to children’s outcomes (Hamre, 2014; Markowitz et al., 2017). Specifically, structural features are theorized to set the stage for process quality by creating the conditions in which high-quality, developmentally supportive processes can occur (NICHD Early Child Care Research Network [ECCRN], 2002; Slot, 2018; Slot et al., 2015). This hypothesis largely motivates the widespread regulation of structural features: In 2016, 100% of QRIS included stipulations for minimum educator qualifications, approximately 80% set curricular standards, and more than 50% considered group size and adult-to-child ratios as quality indicators (National Center for Early Childhood Quality Assurance, 2017). Yet there exists mixed empirical evidence in support of the hypothesis that structural features drive process quality (e.g., Burchinal et al., 2002; Cryer et al., 1999; Hanno et al., 2020; Hu et al., 2016; Mashburn et al., 2008; NICHD ECCRN, 2002; Phillips et al., 2000; Pianta et al., 2005; Slot et al., 2015).
In this study, we extend the literature on the associations between structural and process features in three principal ways. First, whereas most studies have focused on a limited range of settings in one district or region and thus often observe limited variability in quality features, we examine quality across the full landscape of group-based education and care settings in one state. Second, whereas most researchers rely on broad, global measures of process quality, which may obscure true associations between structures and processes, we consider more specific process features reflecting the moment-to-moment experiences of children and adults in early education and care settings. Third, whereas traditional analyses tend to examine numerous bivariate associations between individual structures and processes, introducing the risk of overinterpreting associations that occur by chance, we use permutation testing to understand which structures are consistently associated with the collection of process features under consideration.
After reviewing the literature on the link between structural and process features, we document quality features in community-based child care (CCC), family child care (FCC), Head Start (HS), and public school prekindergarten (PSP) programs across Massachusetts. Although quality improvement efforts tend to operate at the systems level by regulating quality across this range of group-based settings, there is little empirical work simultaneously documenting quality features in all of these settings. Next, we examine the links between structural and process quality features in this unique sample. In doing so, this work advances our understanding of quality in early education and care, providing more rigorous and precise information on which to draw in policy and practice discussions tied to quality improvement.
Associations Between Structural and Process Quality Features
Structural quality includes a large and diverse set of features at the systems, center, and classroom levels, all believed to influence what children and adults do every day in early education and care settings (Cryer et al., 1999; Slot, 2018). In this article, we focus on six classroom-level structures, reflecting staff qualifications (years of experience and education), group size, child-to-adult ratio, and curricular materials (use of a formal or social-emotional curriculum), that are commonly regulated and considered directly proximal to process features, which, as defined above, include the everyday activities and interactions in settings thought to drive children’s development. Classroom structures are theorized to influence high-quality processes through two primary mechanisms: (1) educator knowledge and (2) educator capacity (Cryer et al., 1999; Phillipsen et al., 1997). Greater staff qualifications, including higher education and more years of experience, are thought to afford educators increased knowledge of what works best to support children’s development, in turn increasing the quality of processes in their settings (Burchinal et al., 2002; Early et al., 2007; Lin & Magnuson, 2018). Although the majority of early education settings have multiple adults present, most studies, including ours, focus on the qualifications of a single primary educator, with the understanding that they frequently guide day-to-day operations in settings. As with staff qualifications, having a curriculum is hypothesized to provide educators with an understanding of what to do with children, shaping the nature and content of the activities they plan and how they engage and interact with children during those activities (Jenkins et al., 2019). Finally, group size and child-to-adult ratio are thought to influence educators’ capacity to engage in rigorous, dynamic activities and emotionally supportive and responsive interactions with children, as those in settings with relatively few children may have more time and headspace to plan and execute these types of processes than those in settings with more children (NICHD ECCRN, 2002).
A large body of empirical research has explored whether these structural features are indeed linked to a wide range of process features (e.g., Burchinal et al., 2002; Cryer et al., 1999; Early et al., 2006; Early et al., 2007; Hanno et al., 2020; Hu et al., 2016; Lin & Magnuson, 2018; Mashburn et al., 2008; NICHD ECCRN, 2002; Phillips et al., 2000; Phillipsen et al., 1997; Pianta et al., 2005; Slot et al., 2015; Slot et al., 2018). To address this question, most studies have tended to apply bivariate (e.g., correlations) and regression-based approaches to document associations between a set of structures and a set of processes. For example, in a multistate study of 238 PSP classrooms, Pianta and colleagues (2005) examined the associations between nine teacher and program features and seven process quality features, including the nature of teacher–child interactions and time spent in different activity formats (e.g., whole group, free-choice centers), for a total of more than 60 structure-process comparisons. For the most part, studies applying these types of methods have been inconclusive, with only some finding statistically significant associations between educator education level, educator experience, child-to-adult ratios, or group size, on one hand, and process quality features, on the other (see Appendix Table A1 for a summary of findings from the cited literature). Even within studies, associations between a given structure and the range of process features considered are often inconsistent. Pianta and colleagues, for example, found that educator experience was related with two of the seven process features they considered.
There are fewer studies examining whether curricular materials are associated with process features, although a number of studies evaluate the impact of a specific curriculum on educator practices, which, like process features, reflect how educators engage and interact with children (e.g., Barnett et al., 2008; Clements & Sarama, 2008; Domitrovich et al., 2009). For example, Domitrovich and colleagues (2009) found that HS educators randomly assigned to receive a curriculum and associated training supports focused on language, literacy, and social-emotional development spoke more with children and engaged in deeper conversations with them than those who did not receive the supports. Importantly, educators in these types of evaluation studies typically receive extensive training and ongoing mentorship (e.g., in the Domitrovich et al., 2009, study, the educators had 4 days of workshops focused on the curriculum and regular support from a mentor teacher). As such, these studies provide little insight into how curricula might influence educator practices or process features in more typical conditions. Moreover, they tend to evaluate curricula focused on learning in a specific domain, like math or literacy, offering little evidence on the role of whole-child curricula focused on supporting children’s development across a number of domains. To address this gap, Jenkins and colleagues (2019) examined the variation in processes between classrooms where teachers were or were not using a comprehensive curriculum. They found that 8 of 13 bivariate comparisons between process features reflecting the quality of teacher-child interactions in classrooms with and without curricula were statistically significant, although few were robust to more rigorous regression-based approaches. Taken together, there is little consistent evidence that process features are more prevalent in classrooms where curricula are being used than in those where they are absent.
In general, the number of associations typically tested in studies linking structures and processes makes it challenging to know whether significant associations are the product of random statistical chance or whether children in settings defined by certain structures indeed have meaningfully different day-to-day experiences than those in other settings. Although seldom used in this literature, validation tests for multiple hypotheses based on resampling techniques—like permutation testing—consider structures’ associations with multiple processes simultaneously and offer an opportunity to learn whether a given structural feature is generally associated with a set of process quality features above and beyond what is expected by random chance (Sherman & Funder, 2009). Such approaches advance our knowledge base by moving beyond a singular focus on individual associations between specific structures and specific processes to rigorously evaluate the full set of associations holistically.
Moving From Molar to Molecular Features of Process Quality
Beyond the preponderance of bivariate comparisons, researchers have largely relied on global, or molar, process measures that may obscure the associations between structural features and the elemental, or molecular, processes constituting children’s moment-to-moment experiences in early education and care settings. 1 Specifically, the Classroom Assessment Scoring System (CLASS; Pianta et al., 2008) and the Early Childhood Environment Rating Scales (ECERS; Harms & Clifford, 1980) are used in the majority of relevant studies, and these are both global observational measures in the sense that their scores reflect observers’ summative assessment across multiple theoretically motivated, abstract properties of processes (Pianta et al., 2020). For example, the CLASS’ three domains represent composites of multiple processes. More concretely, the instructional support domain includes how well educators engage in rigorous instruction and model language, among other practices (Hamre, 2014). This means two settings may have identical instructional support scores but one might have high levels of rigorous instruction and low language supports, whereas the other might lack rigorous instruction and have rich language supports. Blending these more fine-grained processes is problematic if they each have unique associations with structures. For example, it may be that group size is related to educators’ capacity to engage in rigorous instruction but not to how they model language. In this example, when the two processes are combined, the association between group size and instructional support would likely be muted.
Novel approaches to operationalizing process quality in early education and care settings offer insight into the everyday by capturing more precise, molecular features of process quality reflecting discrete behaviors. Specifically, the Teacher Observation in Preschool (TOP; Bilbrey et al., 2007) protocol takes repeated, seconds-long snapshots—or sweeps—of individual educators’ behaviors to precisely characterize how they spend their time with children. Over the course of 3 to 5 seconds, observers watch a specific educator and note what they are doing in that moment along six dimensions: language use (e.g., talking, listening), schedule (e.g., whole group, centers), task (e.g., instruction, behavior approving/disapproving), level of instruction, focus (e.g., math, English language arts [ELA]), and tone. For example, while observing an educator for the dimension of language use, observers note whether they are talking, listening to another person, or neither. If the educator is talking or listening, the observer also records to whom (e.g., a child or another adult). At the end of the observation, these snapshots are averaged to yield the proportion of time adults in a classroom are engaged in language-rich interactions with children.
In partnership with a PSP program, the creators of the TOP identified what they believed to be the most salient processes for children’s positive developmental outcomes recorded by the TOP 2 (Farran et al., 2017). They argued that high-quality settings were marked by (1) high levels of instruction, (2) more positive emotional climates (i.e., positive tone, little behavior disapproving and much behavior approving), and (3) a high frequency of educators listening to children. Beyond these processes, the TOP captures many additional elemental processes that are likely important to children’s learning and development. For instance, some research suggests that how frequently educators talk with and model language for children may relate to children’s own language development (Justice et al., 2008; Wasik et al., 2006). How educators spend their time with children may also matter for a broader range of child skills (Camilli et al., 2010; Fuhs et al., 2013). Specifically, the TOP captures the schedule of settings, meaning whether educators are engaged in whole-group activities, in center-based activities, or in transition. It also records the tasks educators are engaged in, such as instructional activities (i.e., any learning activity during which an educator is engaging with a child) or supporting the personal care needs of children (e.g., tying shoes, blowing noses). Finally, observers note the content area focus of educators’ time, including ELA, math, and science. Importantly, while the amount of time spent in any given activity type or content area may be a regulated structure (e.g., a mandated 20-minute math block or an hour in free-choice centers), we consider the amount of time actually spent in that content area as a dynamic process reflecting the activities educators and children engage in (i.e., even with a mandated block, time actually spent on the relevant content area is likely to vary).
To date, little research has examined the associations between structures and these more molecular aspects of process quality. As hypothesized in prior studies, we might expect educators with more years of education or experience, in settings with fewer students or lower child-to-adult ratios, or with curricula to enact more desirable processes (e.g., higher instructional levels, more positive tone) and with greater frequency (e.g., more time on instruction and focused on academic content, more talking and listening, and more behavior approving) than educators with less experience or education, in contexts with more students or higher ratios, or without curricula. We might also expect less desirable process quality features (e.g., more time in personal care activities, transitions, or behavior disapproving) to occur with less frequency in settings led by educators with relatively more education or experience, with fewer students or lower ratios, or with curricula. There are other processes, such as time spent in whole groups or centers, whose hypothesized relations to structural features are less clear. It is also possible that there exist differential associations between specific structures and specific processes. For example, curricular materials might be most closely related to time spent in instruction and on academic content given that such materials often include explicit guidance about time usage. However, in the absence of extensive research on molecular process features, these kinds of differential associations are poorly understood.
The Current Study
Building on the robust literature on quality in early education and care, our first goal is to describe the structural and process features of settings across the contemporary landscape of early education and care. To address this aim, we collected detailed information about quality features in CCC, HS, FCC, and PSP programs across the state of Massachusetts. Our second objective is to examine the associations between structural and process quality features. In addressing these two aims, we extend the literature on quality in early education and care in three principal ways. First, we consider the full range of group-based settings where young children spend their time. This allows us to observe greater variation in structural features than is typical in studies that focus on only one type of early education and care setting. Second, we consider more molecular features of process quality than are captured by commonly used global measures, shedding light on the precise moment-to-moment processes that children experience in early education and care settings. Third, in addition to applying traditional bivariate approaches, we use a methodological approach rarely applied in this area of research to generate insight into which structural features are most consistently associated with process quality. In doing so, we identify structures that are associated with specific facets of children’s early learning experiences, as well as with meaningful differences in the full range of process features.
Method
Sample
Data for the current study come from the first year of the Early Learning Study at Harvard (ELS@H; Jones et al., 2020), which was designed to be representative of 3- and 4-year-old children living in Massachusetts. Using three primary recruitment methods (i.e., a household survey, network sampling, and random sampling of licensed early education and care settings), the sample in the first year of the study included 3,222 children (see Jones et al., 2020, for additional details on recruitment). The early education and care providers of these children, the focus of the present study, were also recruited to participate in the study.
The present study’s analytic sample comprised all 672 classrooms in the 451 group-based programs that were observed during the first year of the study. This sample includes classrooms in all four types of group-based programs (i.e., CCC, HS, FCC, and PSP) located in 247 distinct zip codes, representing almost half the zip codes in the state. The majority of educators leading these classrooms were female (98.49%), spoke English as a primary language (89.94%), and were White (78.07%). Relatively fewer educators in the sample were Latinx (13.23%), Asian (6.43%), or Black (3.78%). On average, the educators in the sample were 45.11 years old (SD = 11.93). There was wide variation in the household incomes of the educators. (See Table 1 and Appendix Table A2 for additional descriptive information about the educators in the analytic sample.) Given the study’s sampling approach, demographic information for most of the children in these settings was not available, although the programs were in communities reflecting the socio-demographic diversity of Massachusetts (see Appendix Table A3).
Descriptive Characteristics of Educators (N = 672)
Procedure
Data were collected through educator surveys and in-person observations. Parents of ELS@H participants were asked to provide information on their child’s educator. 3 These educators were then asked to complete an online survey between April and August 2018 with a wide variety of items, including about their demographic characteristics, qualifications (e.g., education and experience levels), and practices (e.g., use of a curricula). In a few cases (n = 26), the educators completed the survey on paper. During the same period (April to August 2018), trained observers conducted in-person visits in early education and care settings. The visits typically lasted 4 hours, beginning in the morning and continuing until nap time in the afternoon.
Measures
Features of Structural Quality
On the survey, the educators reported their years of experience and education level. Specifically, they were asked to report the number of years they had been taking care of or teaching children (excluding their own). They were also asked to report the highest grade or year of schooling completed from a list of options (i.e., 12th grade or below, high school diploma/GED, vocational/technical program, some college, associate’s degree [2-year degree], bachelor’s degree [4-year degree], some graduate or professional school, master’s degree, or doctoral degree). For the analyses, education level was converted into a continuous variable (e.g., those with a high school diploma were treated as having 12 years of education).
The educators also reported on the survey whether they used curricula in their programs. First, they were asked whether a published curriculum was a source of learning activities in their program. From this item, a binary indicator was created to distinguish between programs using a formal curriculum and those that were not. Second, the educators were asked whether they or their program used a curriculum to “help children learn about their own emotions and other children’s emotions and about managing their own behavior.” Educators who responded affirmatively to this item were regarded as using a social-emotional curriculum.
Finally, group size and child-to-adult ratio were obtained during the in-person observations. Group size was the total number of children who were observed in the classroom, and child-to-adult ratio was the number of children observed per educator observed in the classroom. Child-to-adult ratio was calculated by dividing the number of children in the setting by the number of educators.
Features of Process Quality
Information on 14 features of process quality was captured using the TOP (Bilbrey et al., 2007) protocol. During the in-person visits, trained observers conducted repeated sweeps, each lasting typically 3–5 seconds, for the duration of the observation (approximately 4 hours across all settings). As part of each sweep, the observers noted the specific behaviors of an educator in that moment. In case there were multiple educators in a setting, one educator was observed for several seconds, then the next educator was observed for the next few seconds, and so on. A subset of observed behaviors served as our 14 process quality measures. These observed behaviors included those related to the educators’ use of language—that is, whether (1) listening to children or (2) talking with children; the format of the current activity or schedule—that is, whether in (3) whole group, (4) transitions, or (5) centers; the type of task they were completing—that is, whether (6) approving or (7) disapproving of children’s behavior, (8) engaging in instruction, or (9) supporting the personal care needs of children; and the domain focus of the current activity or task—that is, whether focused on (10) ELA, (11) math, or (12) science. The observers also noted the (13) average tone of the adult’s voice in that moment, using a 5-point scale ranging from 1 = extremely negative to 5 = vibrant. If educators were engaged in instruction, observers also rated the (14) level of instruction, using a 4-point scale ranging from 1 = low-level instruction to 4 = high inferential learning. Across the 672 classrooms in the analytic sample, these features were observed for 1,413 unique adults for an average of 15.28 sweeps each (SD = 3.82).
After receiving training on the TOP codebook and observation protocol, all observers were required to demonstrate high levels of agreement with master observers during in-person practice observations. Agreement between the trainees’ codes and those of the master observers was calculated to establish reliability. Across all TOP items, average agreement was 89%.
Classroom-level scores for each feature were constructed by averaging the counts or ratings from all educator sweeps in the classroom. In the case of features that were marked during sweeps as being present or not (e.g., whether or not the adult was talking to children), the average of counts across all sweeps represents the proportion of sweeps during which the behavior was present. In the case of features rated using a scale (i.e., tone and instructional level), the average of sweep-level ratings represents the average level across all sweeps.
Covariates
Information on educator demographics (i.e., race/ethnicity, primary language, and household income) came from the online survey. Information on provider type (i.e., whether a program was a CCC, FCC, HS, or PSP program) came from administrative records.
Analytic Plan
First, to understand the structural and process features across the landscape of early education and care in Massachusetts, we documented the levels of structural and process features in the sample. Second, to assess the associations between the structural and process features, we initially applied traditional bivariate approaches consistent with those employed in previous studies. Specifically, we examined the correlations between each structural and process feature. We then used multilevel models with program-level random intercepts to predict each of the 14 features of process quality separately as a function of educator years of experience, educator years of education, group size, child-to-adult ratio, an indicator for whether a formal curriculum was used, and an indicator for whether a social-emotional learning curriculum was used (a total of 14 models). Because structural features were likely to differ systematically across provider types (see Appendix Table A4), all models included provider type controls. Additional covariates included the set of educator demographics outlined above. We adopted program-level random intercepts to account for the nesting of classrooms within settings, as more than one third of the programs (162 of the 451 programs; 35.92%) had multiple classrooms observed.
Recognizing that the number of individual comparisons made using traditional approaches increases the risk of interpreting spurious associations, we then used permutation testing to determine whether the identified associations were more than what was expected by chance. In doing so, we also identified which structures were consistently associated with the range of process features considered. Following the procedures outlined by Sherman and Funder (2009), the six structural characteristics were randomly redistributed across the classrooms in the sample, and then multilevel models predicting z-scored process quality features from the structural features with covariates were run within the pseudosamples. Redistributing the structural features across classrooms generates a null association between the predictors (structures) and outcomes (processes) while maintaining the observed associations between covariates and outcomes. This randomization and reestimation procedure was repeated 1,000 times. Then, we calculated the average absolute associations between each structural quality feature and the 14 process quality features across these 1,000 permutations. For each structure, the distribution of the average absolute associations from the permutations—which approximates the sampling distribution of the average absolute association expected by chance—was then charted and compared with the observed average absolute association in the analytic sample. The proportion of absolute associations from the permutations greater than the absolute observed association was then calculated for each structural feature. Smaller proportions indicated that the absolute association between that structure and the 14 processes in the observed sample was greater than expected by chance, whereas larger proportions indicated that the observed absolute association could have been expected by chance.
Missing Data
Given our focus on settings with observation data, there was no missingness on process quality features, with the exception of instructional level. In 7 classrooms (1.04%), the educators did not engage in any instruction during the entire observational period, and therefore the level of instruction in those settings was never assessed. These missing instructional quality observations were not imputed for analysis. There was relatively more missingness on the structural features (ranging from 0.00% to 23.96%) primarily because not every classroom had an educator complete the survey. Of the 672 classrooms in the analytic sample, 537 (79.91%) had survey data. As educators were not required to complete all items of the survey, there was also some missingness on particular items among those educators who did take the survey (see Tables 1 and 2 for details). For the multilevel regression analyses, missing covariates and structural features were imputed using multiple imputation by chained equations in Stata 17 (StataCorp, 2021), a process that yielded 20 complete data sets. The first complete data set from this procedure was used in the permutation testing given its computational intensiveness. Complete-case analysis indicated that the findings from the multilevel modeling and permutation testing were not sensitive to our missing data approach (see Appendix Table A5 and Appendix Figure A1).
Descriptive Statistics of Features of Structural and Process Quality
Results
Features of Structural and Process Quality
Structural Quality
The top panel of Table 2 presents descriptive information on features of structural quality. Educators across Massachusetts had an average of nearly two decades of experience working with young children (M = 18.51 years, SD = 10.14). They also had an average of 15.76 years of education (SD = 1.88), equivalent to slightly less than a bachelor’s degree (16 years). The classrooms in the sample had an average of 10.82 children (SD = 5.00) and an average child-to-adult ratio of 5.28 (SD = 2.37), indicating that many had multiple adults present. The histograms in Figure 1 illustrate variability across these four continuous structural features (i.e., educator experience, educator education, group size, and child-to-adult ratio). Finally, almost half the educators (45.07%) reported using a formal published curriculum to guide learning activities, and far more educators reported using a social-emotional curriculum (80.04%).
Distribution of continuous structural features.
Process Quality
Table 2 also presents descriptive information on the 14 process quality features we considered in this study. In terms of the language environment, educators tended to speak to children in the majority of sweeps (M = 57.21, SD = 15.58). Educators tended to listen to children far less than they spoke to children (M = 6.27, SD = 6.54).
Features captured by the schedule, task, and focus variables suggest that the educators engaged in a diverse range of activities throughout the day. Educators spent approximately 30% of sweeps in whole groups and centers and an average of 14.87% of sweeps in transition (SD = 9.98). In terms of the types of tasks the educators engaged in, instructional tasks took place in approximately 30% of the sweeps (M = 31.47, SD = 16.11), although a considerable proportion of sweeps were also spent attending to the personal care needs of children (M = 12.48, SD = 10.19). Educators spent less time approving or disapproving of children’s behaviors than they did in instruction or on personal care. Relatively more sweeps were spent disapproving of children’s behaviors (M = 7.82, SD = 7.81) than approving of children’s behaviors (M = 2.91, SD = 3.81). In terms of focus, educators spent little time focused on traditional academic domains (i.e., ELA, math, and science). They spent approximately one tenth of sweeps focused on ELA (M = 8.84, SD = 8.07). Educators tended to spend less time focused on math (M = 3.63, SD = 5.37) or science (M = 5.32, SD = 7.13) than they did on ELA.
The average instructional level across the sample was low (M = 1.65, SD = 0.33), indicating that instruction was most often either low-level instruction (e.g., cutting with scissors, singing songs) or focused on basic skills (e.g., learning letters and numbers). Average tone in the sample was 3.41 (SD = 0.32), suggesting that the adults’ tone tended to be somewhere between flat and pleasant.
Linking Structural and Process Quality Features
Table 3 presents bivariate correlations between structural and process quality features, and Table 4 presents multilevel models predicting each process feature from the structural quality indicators, accounting for covariates. In general, results from these two approaches suggest that group size and child-to-adult ratio were most consistently and robustly associated with the array of process features. Specifically, group size was a significant predictor of 6 of the 14 process features, accounting for covariates, and child-to-adult ratio was a significant predictor of 5. As compared with group size or child-to-adult ratio, the remaining structural features were less consistently linked to process quality features. We noted that even in the presence of some statistically significant associations between structural and process features, structural features appeared to account for little variation in process features across classrooms (see Appendix Table A6).
Bivariate Correlations Between Features of Structural and Process Quality
p < .05. **p < .01. ***p < .001.
Multilevel Models Predicting Process Quality From Structural Quality
Note. All models include a program-level random intercept. All models include indicator variables for whether the program was a public school prekindergarten, Head Start, or family child care, making community-based child care the reference group. Additional covariates in all models include educator household income and indicators for whether they were White or spoke English as a primary language. Standard errors are in parentheses.
p < .05. **p < .01. ***p < .001.
Group size was associated with process features capturing language use, schedule, task, focus, and tone. In large groups, educators tended to talk less to children. Each additional child in the classroom was associated with 1.68 percentage points (SE = 0.19, p < .001) fewer sweeps with adults talking with children, accounting for the other structural features, educator covariates, and provider type. Educators in larger groups also tended to spend more time in transition and less time engaged in instructional tasks than those in smaller groups. More concretely, each additional child was associated with 0.28 percentage point (SE = 0.13, p < .05) more sweeps spent in transition and 1.15 percentage points (SE = 0.19, p < .001) fewer sweeps spent in instructional activities. Group size additionally appeared to be inversely associated with the proportion of time educators spent focused on ELA or math, as well as with average educator tone, although the association between group size and educator tone was substantively small (b = −0.01, SE = 0.00, p < .05). Figure 2 illustrates the predicted moment-to-moment experiences in smaller groups (i.e., 8 or fewer children; 25th percentile for group size) as compared with larger groups (i.e., 15 or more children; 75th percentile for group size). For example, educators in classrooms with 8 or fewer children were estimated to speak to children almost two thirds of the time, whereas those in classrooms with 15 or more children were estimated to speak to children about half of the time.
Predicted process quality features by classroom group size.
Although child-to-adult ratio was also related to a number of process features, it was not always associated in expected ways. In line with our hypotheses, behavioral disapproval was more common in settings with larger child-to-adult ratios than in settings with smaller ratios. Adjusting for covariates, each one-unit increment in the child-to-adult ratio was associated with a 0.48 percentage point (SE = 0.18, p < .01) difference in the proportion of sweeps spent disapproving of children’s behaviors. However, contrary to our hypotheses, we found that child-to-adult ratio was positively and strongly associated with (1) more talk with children, (2) more time spent focused on ELA, and (3) more time in instruction. In these instances, the unadjusted bivariate associations between child-to-adult ratio and the process features were indistinguishable from 0 or only weakly correlated. When accounting for other structural features and provider type, however, the relations were substantively large and positive. Each one-unit difference in child-to-adult ratio was associated with a 2.94 percentage point difference in sweeps talking to children (SE = 0.33, p < .001), a 1.03 percentage point difference in sweeps focused on ELA (SE = 0.18, p < .001), and a 1.87 percentage point difference in sweeps on instruction (SE = 0.35, p < .001).
The results of the permutation tests illustrated in Figure 3 confirmed that associations of both group size and child-to-teacher ratio with process features were greater than expected by random chance. In both cases, the observed average absolute associations between process features and child-to-adult ratio and group size were far larger than the associations yielded by chance. This is illustrated in Figure 3 by the black vertical lines representing the observed absolute average association positioned far to the right of the distribution of associations expected by chance. Permutation tests confirmed that educator education level, experience, and use of curricula were not systematically related to process quality features (i.e., the observed average absolute associations could have been observed by random chance). In the absence of this exercise, we may have been tempted to evaluate the significant bivariate associations observed between these four structural features and a minority of processes in Table 3. Permutation testing offers information about the likelihood that associations across the full set of processes were observed by random chance, providing clarity about which bivariate associations to give weight to and, more broadly, which structures were most associated with the range of experiences and activities that constitute process quality.
Approximate sampling distribution for the average association between each structural quality feature and process features.
Discussion
A large body of research emphasizes the importance of quality for children’s learning in early education and care programs (e.g., Hamre, 2014; Hanno et al., 2021; Markowitz et al., 2017; Mashburn et al., 2008), but little work considers the finer-grained, molecular features of process quality across multiple types of early education and care programs, nor has it examined them in relation to oft-regulated structural features. Using unique data from a statewide study of early education and care in Massachusetts, we first documented quality features across the landscape of group-based early education and care in the state. We then examined the associations between structural and process quality features in these settings to inform the conversation on whether certain structures are likely to underlie the day-to-day realities in early education and care programs.
Features of Quality in Early Education and Care
Structural features in our sample mirror broader trends in early education and care. In comparison with educators working with children in the nationally representative Early Childhood Longitudinal Birth Cohort, the educators in our sample had similar average education levels but more years of experience (Bassok et al., 2016). The finding that many educators have worked for several decades with young children is notable given prior work documenting high levels of staff turnover in the early education and care field (Whitebook et al., 2014). It may be that educator turnover in Massachusetts is less of a challenge than in other locations or that educators move between programs within the field. Findings also show that, as hypothesized, early education and care programs tended to have group sizes and child-to-adult ratios lower than are typical in K–12 settings. Last, nearly twice as many educators in our sample reported using a curriculum to guide social-emotional learning than a formal curriculum to support children’s learning across a wider variety of domains. This stands in contrast to the findings of Jenkins and colleagues (2019), which showed that between 60% and 100% of the primarily HS and PSP settings in their sample used a whole-child curriculum. The difference between our findings is likely driven by the presence in our sample of settings in which such curricula are not frequently mandated (e.g., FCC).
The observed process quality features in our sample showed that educators split their time among a variety of activities. First, educators spent far more time talking to children than listening to them. In their analysis of 26 public prekindergarten classrooms, Farran and colleagues (2017) similarly found that educator talk outpaced listening. Second, educators spent the majority of time (58%) engaged in either whole-group or center-based activities, although transitions still accounted for a considerable proportion of time (15%). The combined proportion of time spent in whole groups or centers was nearly identical to that observed by Pianta and colleagues (2005) in their study of PSP programs (59%). Third, in terms of the types of tasks educators completed, educators spent more than twice the proportion of sweeps disapproving of children’s behaviors than they did approving of children’s behaviors, although the amount of time educators spent doing either was low (<10%). They spent relatively more time in instruction and supporting the personal care needs of children than in behavior approving or disapproving. Finally, less than one fifth of the total time was spent on activities focused on core academic subjects (i.e., ELA, math, or science).
Whereas the aforementioned process features shed light on the moment-to-moment activities of educators (the what), the remaining two features—instructional level and tone—provided insight into the qualitative nature of how educators operated in their classrooms. The average instructional level in the sample indicated that educators tended to engage mostly in low-level or basic skills instruction, and the average tone indicated that educators tended to use generally pleasant tones. A low average instructional level and a positive average tone were similarly observed in the Farran and colleagues (2017) study. These patterns also align with those found using the CLASS, which show that emotional supports tend to be high across settings but instructional supports tend to be low (Hamre et al., 2007; Mashburn et al., 2008).
Linking Structural and Process Quality Features
We found that group size and child-to-adult ratio were significantly and consistently associated with process quality. This implies that children and adults have different moment-to-moment experiences in settings with a large number of children than in those with a small number of children. It also implies that children have different experiences in contexts where individual adults are responsible for many children than in those where adults are responsible for a few children. Despite the general associations between both group size and child-to-adult ratio with process quality features, we noted that group size appeared more consistently related to process quality than child-to-adult ratio. Although group size and child-to-adult ratio are likely both proxies for the load educators face, group size likely represents the overall administrative burden on educators. This burden may be relevant for how educators spend their time if certain activities are challenging to facilitate in a large group regardless of the number of adults present. For example, if children must wash their hands before eating a snack (a personal care task), it is almost certain to take longer with a group of 20 children than with a group of 10 children.
We also observed several unexpected associations suggesting that higher child-to-adult ratios were positively associated with process quality after accounting for the other structural features. Specifically, we found that educators tended to speak more to children, spend more time in instruction, and focus more on ELA in contexts with higher child-to-adult ratios. These results imply, for example, that if we were to compare two classrooms both with 20 students, but one had three educators and the other had two, we would expect the classroom with two educators to focus more on ELA and have more talk with children than the one with three educators. It could be that classrooms with relatively more educators adopt a “divide and conquer” approach, with particular educators devoting more of their time to administrative tasks or children’s personal care needs and less time to direct interaction and instruction. More research is needed to understand how educators in settings with multiple adults share and distribute job demands (Sheridan et al., 2014).
Although group size and child-to-adult ratio were associated with process features capturing time allocation, neither was strongly associated with the two qualitative process features (i.e., instructional level and educator tone). Group size was inversely associated with average tone, but the association was substantively small. This suggests that these two structural features might shape what activities educators do but may not necessarily influence how they engage in those activities. For example, an educator in a large classroom may not be able to talk much with children due to many competing demands but may be able to maintain a pleasant tone while speaking with children.
In contrast to our findings that group size and child-to-adult ratio were significantly related to many process quality features, there was little evidence that educator experience, education level, or curriculum use were consistently related to process quality. As hypothesized in prior work (e.g., Hanno et al., 2020; Slot et al., 2015), the absence of associations between these structural and process quality features could be attributable to several factors. First, it may be that the structural features as measured here obscure the wide variation within like levels. For example, two educators may both have a bachelor’s degree, but they could have attended programs that prepared them in unique ways to work with young children. In line with this hypothesis, some evidence suggests that settings led by educators who do not receive any early childhood–specific credits have lower process quality than those with educators who do have specialized training (Lin & Magnuson, 2018). Relatedly, two settings may both have a social-emotional curriculum, but in one it may be used more extensively than in the other or the content of the two curricula could vary greatly.
Second, structural features could have interactive associations with each other (Slot, 2018). For example, it may be that educators are only able to implement high-quality processes if they have a certain level of education and use a curriculum. In line with this hypothesis, the “strongest hope” model for improving quality is thought to combine curricula with ongoing, intensive supports for educators, like coaching (Weiland et al., 2018; Yoshikawa et al., 2013). Third, it could be that educator education and experience, in particular, have nonlinear associations with process features. Some recent research conducted in China on the association between structural and process features found that educator experience was nonlinearly related to global processes (Hu et al., 2017). That is, differences in educator experience were associated with positive differences in process quality but only up to a point. Although not exhaustive, these explanations highlight several avenues for future research on educator education, experience, and curriculum use.
Limitations and Future Directions
Despite the numerous strengths of this study (e.g., a large and diverse contemporary sample, novel measurement of process features), there are several important limitations. First, our design does not allow us to make causal assertions about the associations between structural and process features. For example, although we found that group size was linked to a number of process features, we cannot say that group size caused differences in these processes. It may be that other factors correlated with both group size and process features, varying within provider types, like program budgets or workplace stress, may explain the associations.
Second, the structural and process measures employed in this study likely obscure important variation in these features within settings. Regarding the structural measures, staff qualifications come from only one educator per setting and are therefore unlikely to reflect the experiences of all educators in each setting. Similarly, group size and child-to-teacher ratio were observed during classroom visits and may therefore not represent stable conditions over time. Regarding the process measures, it is likely there is variation in what individual educators and children experience within the same setting (e.g., some teachers spoke more or less than others in the same classroom). Research is needed to document this variation and understand its implications for children’s learning.
Relatedly, our study does not provide evidence on the relevance of the considered process features for children’s positive development, which is an important limitation insofar as quality features primarily matter if they support children’s learning. Some previous work offers preliminary evidence on the relevance of these types of molecular features for children’s development in a number of domains (Farran et al., 2017; Fuhs et al., 2013; Jones et al., 2020). However, as the measurement of molecular features of process quality is relatively novel, little is known about the precise levels of these features necessary to nurture children’s healthy development. Whereas it is commonly assumed that higher scores indicate better quality on global measures than lower scores, the same is unlikely to be true of more molecular measures. For example, although more listening to children is likely to be a desirable process feature, a setting where educators spend 100% of their time listening to children, and no time talking, is unlikely to benefit children’s development. Similarly, instruction could be inaccessible and cognitively exhausting if educators were to only engage in high levels of inferential learning. Future research is therefore needed to understand the ideal levels and combinations of various molecular features for promoting children’s development across learning domains.
Finally, although this sample includes educators from education and care programs across Massachusetts, the results are not representative at the state level. Participation in our study likely varied systematically by educator characteristics as well as by program features. Relatedly, the early education and care system of Massachusetts may be distinct from those of other states.
Conclusion
As policymakers continue to pursue quality improvement across the early education and care system, our study underscores the importance of taking into account a wider array of quality features and, in particular, considering more precise, molecular features of process quality that capture the moment-to-moment realities of settings. Applying a synthetic methodological approach with unique data on these features from a statewide study of early education and care, we found that group size and child-to-adult ratio were significantly and consistently associated with a range of process quality features. In contrast, we found little evidence that educator education or experience level or the use of a formal or social-emotional curriculum were associated with these process features. As such, blanket requirements for minimum education requirements or curriculum use are unlikely to yield meaningful improvements in children’s day-to-day experiences in early education and care programs. These findings do, however, suggest that group size and child-to-adult ratio are potentially important structural features to consider in developing policies that make a difference for those experiences.
Footnotes
Appendix
Proportion of Variance in Process Features Explained by Covariates and Structural Features
| R 2 | Language use |
Schedule |
Task |
Instructional level |
Focus |
Tone |
||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| (1) |
(2) |
(3) |
(4) |
(5) |
(6) |
(7) |
(8) |
(9) |
(10) |
(11) |
(12) |
(13) |
(14) |
|
| R2 from model including covariates (educator characteristics + provider type) | .02 | .15 | .04 | .06 | .03 | .07 | .06 | .14 | .03 | .04 | 0.07 | 0.08 | 0.08 | 0.12 |
| R2 from model with covariates + structural features | .03 | .31 | .07 | .07 | .06 | .08 | .08 | .20 | .05 | .07 | 0.12 | 0.09 | 0.10 | 0.13 |
Note. R2 comes from regression analyses with complete cases and standard errors clustered at the provider level. An alternative approach to understanding the variance explained by covariates and structural features would be to calculate pseudo-R2 using variance components yielded from multilevel models, as explained by Singer and Willett (2003). However, using this approach sometimes yields pseudo-R2 values less than 0, which are not interpretable (Singer & Willet, 2003). This was the case with several outcomes in our data, and as such, we report only R2.
Acknowledgements
We thank Abt Associates, in particular Amy Checkoway, Barbara Goodson, and Kerry Hofer, for their expertise, ongoing collaboration, and support of this work. We are also deeply grateful to all the educators who have so generously shared their time and perspectives with us. The research activities were funded by the Saul Zaentz Charitable Foundation through its generous support of the Saul Zaentz Early Education Initiative at Harvard Graduate School of Education.
Notes
Authors
EMILY C. HANNO is a postdoctoral researcher at the Harvard Graduate School of Education.
KATHRYN E. GONZALEZ is a researcher at Mathematica Policy Research and former graduate student at the Harvard Graduate School of Education.
STEPHANIE M. JONES is the Gerald S. Lesser Professor in Early Childhood Development and codirector of the Saul Zaentz Early Education Initiative at the Harvard Graduate School of Education.
NONIE K. LESAUX is the Juliana W. and William Foss Thompson Professor of Education and Society and codirector of the Saul Zaentz Early Education Initiative at the Harvard Graduate School of Education.