Abstract
Intervention dosage is foundational to realizing intended impacts but is often variable, particularly when interventions are implemented under real-world conditions. In this study, we examined dosage of small-group emergent literacy intervention experienced by preschool children (n = 154) identified as at risk for later reading difficulties in authentic classroom settings. We documented considerable variability in dosage that was largely due to when instructors stopped offering lessons. Drawing from extant literature and an ecological orientation, we found that instructor factors (i.e., instructor self-efficacy for teaching language and literacy, instructor perception of lesson acceptability, average small-group size) and classroom factors (i.e., classroom teachers’ self-efficacy for decision-making), but not child factors, significantly predicted children’s intervention dosage. Moreover, most variance could be attributed to differences between small groups/instructors rather than individual differences among children. We discuss implications for preschool teachers, administrators, researchers, and intervention developers seeking to better support successful small-group intervention implementation.
Keywords
Research has demonstrated positive effects of small-group intervention for promoting the emergent literacy skills of preschool children, particularly those at risk for later reading difficulties (e.g., Bailet et al., 2009, 2013; Goldstein et al., 2017; Lonigan & Phillips, 2016; Phillips et al., 2021; Zettler-Greeley et al., 2018). Early instruction increases the chances that children enter kindergarten with the skills they need to become successful readers (National Early Literacy Panel [NELP], 2008). However, implementation of small-group literacy instruction is variable when used in authentic preschool classrooms (Farley et al., 2017; Piasta et al., 2021). One key aspect of implementation is dosage, which can be operationalized as the number of lessons a child experiences and is positively associated with children’s outcomes (Bailet et al., 2009; Zettler-Greeley et al., 2018). In the current study, we considered dosage from an ecological orientation that positions the child within a larger system, as we posit that child, instructor, and classroom factors might influence intervention dosage (Durlak & DuPre, 2008; Fixsen et al., 2005). Identifying ecological factors that predict the dosage of small-group literacy interventions in preschool classrooms may help researchers, curriculum developers, and teachers better design and support effective instruction.
Implementation Fidelity and Dosage
Attention to implementation fidelity in intervention research has increased over the past 30 years, especially as funding agencies and peer-reviewed publications require reporting of fidelity information (Capin et al., 2018). A widely adopted framework of implementation fidelity includes five dimensions: (a) adherence, the degree to which critical components of the intervention are delivered as intended; (b) dosage, the number or duration of lessons provided; (c) quality of delivery, how well the intervention is carried out; (d) participant responsiveness, the extent to which participants respond to and are engaged in the instruction; and (e) program differentiation, how the intervention treatment differs from the comparison condition (O’Donnell, 2008). Consideration of these dimensions can help inform intervention scale-up and may be related to improved learning outcomes (Capin et al., 2018; O’Donnell, 2008).
In the current study, we focused specifically on dosage, one of the most commonly documented aspects of implementation fidelity in early childhood studies (Darrow, 2013) and a foundational aspect of implementation. To demonstrate adherence, quality of delivery, and responsiveness, dosage must first occur (i.e., children must be exposed to intervention lessons). Thus, although dosage alone is insufficient, we posit that it is a minimally necessary component of implementation. Furthermore, children who receive more lessons likely have more interactions with curricular objectives, more instances to hear explanations of complex content, and more opportunities to receive feedback and practice skills. This increased exposure to intervention content may be critical for children identified as at risk for reading difficulties, as more opportunities to practice under expert guidance are often needed for these children to achieve mastery (Gersten et al., 2008). Correspondingly, empirical evidence indicates that dosage is positively related to emergent literacy gains (Bailet et al., 2009; Marti et al., 2018; Piasta et al., 2015).
However, dosage can vary widely (Goldstein et al., 2017; Marti et al., 2018; Piasta et al., 2015; Varghese et al., 2021). For example, Goldstein and colleagues found that preschool children’s dosage of a small-group early literacy intervention ranged between 19 and 36 lessons throughout their study. Consequently, questions arise about factors that lead to some children experiencing more or less dosage when receiving intervention in authentic preschool settings.
Dosage Within an Ecological Orientation
An ecological orientation provides a way to consider a child’s intervention dosage within the context of a broader system to which they belong. We hypothesize that factors pertaining to the child receiving the intervention, the instructor providing the intervention, and the classroom where the intervention is taking place can influence the successful implementation of small-group emergent literacy intervention. Examining dosage at the child, teacher, and classroom levels allows us to consider how factors at each level can either promote or hinder the number of intervention lessons a child receives. We selected teacher and classroom factors included in ecologically oriented implementation fidelity frameworks (Durlak & DuPre, 2008; Fixsen et al., 2005) and child factors that could be theoretically or empirically linked to dosage. Whereas we acknowledge that many ecological factors may influence dosage, we were limited in the number of factors that could be examined given statistical considerations and also limited to factors that were available in the analysis dataset. However, in our review of the dosage literature, few studies have considered these ecological factors detailed below, highlighting the potentially important contribution for researchers, curriculum developers, and classroom teachers.
Child and Family Factors
Several child- and family-related factors theoretically could be associated with intervention dosage. A child’s age may be positively associated with dosage, given that increased self-regulation, attention, and inhibitory control (e.g., Milburn et al., 2019) may better afford participation in small groups. Likewise, given that self-regulation increases as children have opportunities to engage in small-group and play contexts (Timmons et al., 2016), children with more preschool experience may be better able to participate in small-group lessons. Conversely, more years of preschool is also associated with increased behavior problems (Ansari et al., 2019) which might make small-group literacy intervention less feasible.
Furthermore, absenteeism likely affects whether children receive intervention lessons. Thus, factors influencing a preschool child’s attendance may also affect intervention dosage. Klein et al. (2020) found that absenteeism was negatively related to parental education level, and McIntosh et al. (2016) found that children who attended schools in high-poverty communities were less likely to receive sustained intervention over time, suggesting that small-group lesson dosage could be related to family income. Given that more adults in the home may increase social support and resources that facilitate child attendance and, conversely, reduce chronic absenteeism (e.g., Lim et al., 2019), the number of adults in a child’s home might also relate to whether children are present and able to experience intervention lessons.
Finally, children with lower initial emergent literacy skills are often recognized as needing immediate supplemental support (Milburn et al., 2019) and thus be more likely to experience emergent literacy intervention. We suggest that this may be true even within a sample of children who are all at risk for later reading difficulties, given heterogeneity within such samples (e.g., Cabell et al., 2011).
Instructor Factors
At the interpersonal level of the ecological model, characteristics of the instructor—the adult responsible for implementing lessons (e.g., classroom teacher, interventionist, volunteer)—may be associated with intervention dosage. Research has shown a positive relation between dosage and instructors’ years of teaching experience (Thierry et al., 2020), suggesting that instructors with greater teaching experience may be more likely to teach small-group lessons regularly. In addition to years of experience, instructors’ education level and content knowledge for teaching language and literacy could relate to small-group instruction dosage. Mendive et al. (2016) found that preschool instructors who received professional development to build content knowledge provided greater intervention dosage than those who did not receive professional development. Furthermore, research has shown positive associations between instructors’ prior experience with a structured curriculum and dosage (Marti et al., 2018; Varghese et al., 2021). Instructors having greater familiarity with a structured curriculum may be more comfortable teaching scripted small-group lessons and more likely to implement them.
In addition to experience and content knowledge, instructors’ beliefs might predict intervention dosage (Ransford et al., 2009). For example, given connections between instructor self-efficacy and engagement (Skaalvik & Skaalvik, 2014), instructors with higher self-efficacy for teaching language and literacy may be more likely to engage in emergent literacy intervention than instructors with lower efficacy and thus more likely to continue intervention when challenges arise. Relatedly, instructors’ beliefs about new practices, or openness to change, relate to whether they see value in engaging in and sustaining new activities and routines (e.g., Vannatta & Fordham, 2004). Therefore, instructors who are open to new practices may provide more small-group lessons. Instructors’ acceptability of the lessons may follow this same pattern. Those who believe the intervention lessons to be of instructional value may be more dedicated to ensuring that the lessons occur. Similarly, instructors’ perception of small groups as a barrier may be negatively associated with dosage, as those who do not believe that small-group instruction is effective may be more likely to give up when challenges arise. Similarly, as instructors’ workload and burnout often impact intervention dosage negatively (Ransford et al., 2009), instructors who spend more time preparing lessons may be less likely to implement them. Yet, to our knowledge, these instructor characteristics have not been investigated in relation to small-group intervention dosage in preschool.
Small-group size is another factor that may be associated with instructors’ provision of intervention lessons. Groups with fewer children may afford greater relationship building, be more manageable for instructors, and reduce behavior challenges (Wanzek & Vaughn, 2008), thus making it more likely that an instructor routinely implements small-group intervention.
Classroom Factors
Small-group instruction is situated in the larger ecology of the classroom environment. Thus, we identified several classroom factors that could promote or hinder intervention dosage. The length of the instructional day may influence intervention dosage, with classrooms that offer full-day programming (as opposed to half-day) affording more time and flexibility for teachers or other instructors to fit in small-group lessons. Likewise, well-organized classrooms, as exemplified by established classroom routines, clear expectations, and effective classroom management, likely afford more time to conduct intervention lessons along with facilitating instructors’ abilities to work with small groups while other children are otherwise engaged; indeed, research suggests that small-group instruction is positively associated with the quality of classroom organization (Farley et al., 2017). Moreover, small-group lessons might be easier to integrate when classroom routines already include time spent in small groups or time spent in learning centers, as children in these classrooms are already familiar with expectations for working in groups or on their own while the instructor works with other children.
Previous studies found that the child-to-teacher ratio negatively predicted the amount of small-group instruction taking place in the classroom (Farley et al., 2017; Marti et al., 2018). Marti and colleagues postulated that finding time for small-group intervention may be challenging in classrooms with high child-to-teacher ratios; providing teachers with additional support may be necessary for increasing intervention dosage. Due to younger children’s tendency to have less developed self-regulation and inhibitory control (e.g., Diamond, 2002; Montroy et al., 2016), classrooms enrolling children below 4 years of age (i.e., 3 years and younger) might also make small-group instruction—and the classroom management that it requires—challenging for instructors. The availability of additional support, such as another adult, could be related to the number of classrooms at the preschool center, with larger centers possibly having more personnel and resources available.
In addition, classroom schedules and curricula may either facilitate or serve as a barrier to small-group intervention. Thus, the extent to which the lead teacher in the classroom is responsible for instructional planning may determine whether they incorporate time in their schedules and curriculum planning for themselves or others to complete lessons, consequently predicting intervention dosage. Moreover, research suggests increased intervention implementation when teachers perceive themselves as having input into decisions about their classrooms and instruction (Durlak & DuPre, 2008). Whereas lead teachers with higher self-efficacy for decision-making may believe they can prioritize intervention and arrange classroom schedules to allow instructors and children to engage in small-group lessons, those with lower self-efficacy for decision-making may believe they do not have the authority to do so.
The Current Study
In the current study, we considered the intervention dosage experienced by children identified as at risk for later reading difficulties in authentic preschool settings. We specifically examined dosage of the Nemours BrightStart! (NBS!) intervention, as one example of a growing number of small-group emergent literacy interventions that are commercially available for use in preschool classrooms. NBS! is a supplemental intervention comprising 20 lessons that provide explicit, systematic instruction in code- and meaning-focused emergent literacy skills. NBS! has been used in 27 states with an estimated one quarter of a million children.
NBS! has demonstrated positive intent-to-treat effects on children’s emergent literacy learning in highly controlled efficacy trials when implemented by trained interventionists supervised by the research and development team (Bailet et al., 2009, 2013; Zettler-Greeley et al., 2018). NBS! also exhibits dosage effects, such that children who completed more lessons exhibited more positive literacy outcomes than those who completed fewer lessons (Bailet et al., 2009; Zettler-Greeley et al., 2018). Given impending scale up of NBS! and similar interventions, additional research has investigated its implementation and impacts when used under more routine conditions. Most recently, Piasta et al. (2022) assessed the impact of NBS! when delivered by classroom teachers or community aides (i.e., adult volunteers from a kindergarten readiness initiative who partnered with local early childhood centers for a variety of purposes, including providing NBS! intervention) in authentic preschool settings. In this work, intervention dosage was noticeably lower than those reported in earlier efficacy studies, and both types of instructors noted challenges related to child behavior and absences, classroom management, and small-group functioning that affected their implementation (Piasta et al., 2021). Consequently, NBS! showed minimal effects. As Piasta et al. (2021) suggest, small-group literacy interventions like NBS! may be more likely to realize intended impacts if research can identify—and subsequently address—factors related to intervention dosage. We investigated this in the current study as guided by our overarching research question: To what extent do selected child, instructor, and classroom factors predict intervention dosage?
Method
We capitalized on data collected during a recent randomized controlled trial of the NBS! intervention. Full details of the randomized controlled trial and its results are reported in Piasta et al. (2022). The trial involved randomly assigning classrooms (and eligible children enrolled in these classrooms) to one of two intervention conditions, which differed in terms of who served as the intervention instructor, or to a control condition. Instructors for the intervention conditions were either the classroom teacher or a community aide; the latter visited early childhood centers to provide intervention. Notably, the randomized controlled trial involved implementing the NBS! intervention under real-world, authentic conditions. Teachers and community aides who voluntarily participated in the trial received the NBS! intervention materials plus the professional development provided by the publisher and were asked to provide the intervention to identified children for one academic year. In the current study, we focus exclusively on these two intervention conditions.
Participants
Children enrolled in preschool classrooms participating in the randomized controlled trial were eligible to receive the NBS! intervention if they met study inclusion and exclusion criteria. First, children had to (a) be between 3 and 5 years of age; (b) have parental consent to participate; (c) not have severe attendance issues or behavioral problems as reported by their classroom teacher; (d) speak English as their primary home language or, if not the primary language, understand and speak English with at least basic fluency as reported by their parent; and (e) not have disabilities that were greatly affecting their ability to learn or participate in classroom activities, as reported by their parent. We applied criteria (d) and (e) to ensure that children could appropriately participate in intervention and assessments, which were conducted in English. Second, children who met these initial criteria completed the Get Ready to Read!—Revised screener (GRTR-R; Whitehurst & Lonigan, 2010). Children who scored in the below-average/at-risk range, per the manualized instructions, were identified as being at risk for later reading difficulties and thus likely to benefit from NBS! intervention. Of these children, we randomly selected up to four per classroom to participate in the randomized controlled trial (n = 195 children in 67 classrooms assigned to receive NBS! intervention). Finally, for purposes of the current study, we excluded children whose teachers withdrew from the trial prior to lesson implementation (n = 15 children in five classrooms) and children who left their early childhood programs before intervention completion (n =26; four who comprised the only eligible children in three classrooms plus 22 children in various classrooms), as these situations precluded the opportunity to experience full intervention dosage. The final analytic sample was thus 154 children in 59 classrooms.
Fifty percent of children were boys, and the average age was 4.3 years (SD = 0.5). As reported by parents, 57% were Black, 21% were White, 10% were multiracial, 3% were Asian, and 5% were of races other than the categories listed on the census (5% unreported); 14% were Hispanic or Latinx. Most children were in their first year of preschool (56%), with some in their second (27%), third (11%), or additional (5%) years. Two children had individual education plans. Most children lived in homes with one (35%) or two (49%) adults, although this ranged up to five (3%). Parents’ education levels were distributed across nine categories ranging from completing eighth grade (3%) through holding a doctoral degree (2%); 63% had a high school diploma as their highest degree, 10% had an associate’s degree, 8% had a bachelor’s degree, and 4% had a master’s degree. Parents reported annual family income in $5,000 increments, ranging from $5,000 or less to $165,001 or more, with the average between $25,001 and $35,001.
Most of the 59 classrooms were located in public schools or early childhood centers (93%) in urban areas (76%), with an average of 3.9 classrooms at the school/center (range = 1–14). Sixty-eight percent were full-day programs, and 32% enrolled only children 4 years or older; other classrooms included younger children. Classroom teachers were predominantly female (95%) and not Hispanic/Latinx (92%). Forty-six percent of teachers identified as Black, 48% identified as White, and 5% identified as multiracial or of other races. Teachers’ education levels were distributed across six categories ranging from a high school diploma (5%) through a master’s degree (7%); 31% had an associate’s degree as the highest degree earned, and 34% had a bachelor’s degree. Preschool teaching experience averaged 11.4 years (SD = 8.5). Fifteen percent held teaching licenses. Most teachers reported using Creative Curriculum (73%), with various other curricula used by less than 10% of teachers.
Seven community aides served as NBS! instructors for 32 of the 59 classrooms. Although nine community aides were hired by the local kindergarten readiness initiative, two resigned their positions after providing five to six lessons, and the remaining community aides assumed their intervention responsibilities. All community aides were female, and all had prior experience working with children (e.g., classroom, substitute, Sunday School teacher; reading tutor; summer camp aide). Fifty-seven percent identified as Black, 29% identified as White, 14% identified as multiracial, and none identified as Hispanic/Latina. Education levels ranged from a high school diploma plus some college courses (14%) through a doctoral degree (14%); 29% had a bachelor’s degree as the highest degree earned, and 43% had a master’s degree.
Procedures
Participating children in each classroom comprised the small group who received the NBS! intervention provided by either their classroom teacher or a community aide; community aides served multiple classrooms/small groups. As noted, the NBS! intervention is a small-group, supplemental intervention that includes instruction on both code-focused and meaning-focused emergent literacy skills. Detailed information about the 20 intervention lessons, including the instructional routine and activities, is provided in the Supplemental Material; additional information can be found on the publisher’s site (www.kaplanco.com/product/19721/nemours-reading-brightstart-the-complete-program-for-early-literacy-success-level-one?c=25%7CCU1045) and in Bailet et al. (2009, 2013), Piasta et al. (2022), and Zettler-Greeley et al. (2018).
To support implementation, teachers and community aides received all intervention materials (i.e., instructor guide, lesson plans, books, and manipulatives) and completed the standard 2-day in-person professional development training offered by the publisher. The training includes an overview of the NBS! intervention, step-by-step instructions for implementing lessons, live and video demonstrations, and opportunities to practice implementation. For the current study, teachers and community aides were asked to implement the NBS! intervention with their small group(s) at a rate of one lesson per week, with each lesson split into two 20- to 30-min sessions. Teachers and community aides videotaped each lesson administration and also completed lesson logs, noting whether each lesson was completed, when, and whether participating children were present or absent; these were submitted to the research team. When children missed lessons due to absences, teachers and community aides were asked to provide makeup lessons when possible; these were also recorded and logged. We used lesson videos and logs to document implementation fidelity, including dosage, for purposes of the randomized controlled trial. Additional details regarding fidelity measures and findings are presented in the work by Piasta et al. (2021, 2022). In brief, teachers and community aides averaged 77% adherence (SD = 0.13) on a checklist measuring implementation of key lesson components, 2.13 out of 3 (SD = 0.33) on a quality of delivery scale in which 3 represented expert implementation, and 2.47 out of 3 (SD = 0.33) on a participant responsiveness/child engagement scale in which 3 indicated that all children were responding throughout the lesson.
Parents, classroom teachers, and community aides completed questionnaires as part of data collection. Parents reported information about participating children via a background questionnaire returned with consent forms. Children’s classroom teachers completed a questionnaire at the start of the preschool year to report classroom and demographic information, and community aides also completed an initial questionnaire to report demographic information. Instructors (i.e., teachers and community aides who provided intervention) completed another questionnaire after intervention ended and reported on additional factors related to their experience implementing the intervention.
In addition, at the start of the academic year, trained assessors conducted the GRTR-R screener (Whitehurst & Lonigan, 2010) one-on-one with participating children in quiet locations at their respective preschools. Children also completed other emergent literacy assessments at pretest and posttest not relevant to the current study (fully described in Piasta et al., 2022).
Measures
Intervention Dosage
We operationalized intervention dosage as the number of lessons experienced by each child. We measured dosage based on instructors’ lesson logs, which we verified against submitted lesson videos. Because each lesson was split into two sessions, we counted each session as 0.5 of a lesson. We then summed the number of sessions each child experienced to compute their intervention dosage (maximum = 20 lessons), which served as the dependent variable of interest.
Child and Family Factors
We used responses from the parent questionnaire as well as GRTR-R data to measure child and family factors hypothesized to be related to intervention dosage. Parents reported children’s age and number of years of preschool plus their highest education level (treated as a continuous, ordinal-level variable), annual family income (treated as a continuous variable with $5,000 increments), and the number of adults in the home. Children’s raw scores on the GRTR-R served as a measure of initial emergent literacy skill. The GRTR-R consists of 25 items (α = .80) focused on print knowledge and phonological awareness as key emergent literacy skills. Children are shown sets of four pictures for each item and asked to point to the one that represents the correct response to a question (e.g., “These pictures are: mouse, cloud, cow, moon. Find what you get when you put /m/ and /oon/ together”). Correct answers are tallied to derive raw scores. When used as a screener, raw scores are compared with age-based benchmarks provided in the manual; children who score below benchmark are identified as at risk for later reading difficulties. GRTR-R exhibits concurrent and predictive validity with other literacy measures (Missall & McConnell, 2004; Phillips et al., 2009) and high sensitivity and specificity (Wilson & Lonigan, 2010).
Instructor Factors
We used teachers’ and community aides’ responses on questionnaires to measure instructor factors hypothesized to be related to intervention dosage. On the start-of-year questionnaire, instructors directly reported their education levels (treated as a continuous, ordinal-level variable), years of preschool teaching experience, and curricula/intervention use. For the latter, teachers reported all curricula/interventions used in their classrooms, and community aides reported any curricula/interventions with which they had prior experience. Drawing on Chambers et al. (2016), we classified these as structured (referred to by Chambers et al. as comprehensive or supplemental curricula) or developmental-constructivist; any experience with a structured curriculum was dummy coded as 1 for use in analysis.
We included three scales on the questionnaires to measure instructors’ content knowledge, self-efficacy, and openness to new instructional practices. Instructors completed Cunningham et al.’s (2009) 19-item multiple-choice and short-answer measure of content knowledge related to early literacy; we tallied the number of correct responses for use in analyses (α = .75). Instructors reported their feelings of self-efficacy with respect to promoting language and literacy on a scale of 0 = no feelings of efficacy to 4 = very strong feelings of efficacy (see Ottley et al., 2015); we computed the average across these nine self-efficacy items for use in analyses (α = .91). Instructors also reported their openness to new instructional practices using a 7-item measure adapted from and validated in previous work (Neuman & Cunningham, 2009; Vannatta & Fordham, 2004; see Ottley et al., 2015). Instructors indicated their extent of agreement with statements (e.g., “I enjoy learning about new ways to teach early reading and writing skills.”) on a scale of 1 = strongly disagree to 5 = strongly agree; we computed the average across these 9 items, such that higher scores reflected greater openness to new instructional practices (α = .76).
We measured three additional factors related to instructors’ experience implementing the intervention via their responses to the end-of-year questionnaire. Instructors reported the amount of time per week that they spent preparing NBS! lessons on a scale of 0 = none to 5 = 60+ min. Instructors rated the acceptability of the NBS! intervention with respect to the relevant 9 items on the Usage Rating Profile-Intervention (Briesch et al., 2013). Items are rated on a scale of 0 = strongly disagree to 5 = strongly agree, with scores averaged such that higher ratings indicate greater acceptability (α = .78). Using the same scale, instructors also rated their agreement with the statement, “The use of small groups is a barrier to using the NBS! program.” As a final instructor factor, we used child attendance as reported on lesson logs to measure the size of the small group to which the instructor provided intervention. We calculated this as the average number of children in the small group across all lessons.
Classroom Factors
We used classroom teachers’ responses on a start-of-year questionnaire to measure classroom factors hypothesized to be related to intervention dosage. Teachers reported school/center size (operationalized as the number of preschool classrooms), length of the instructional day (dummy coded with full day = 1), ages served (dummy coded as enrollment of children below 4 years old = 1), and child:teacher ratio. Teachers also reported on four additional classroom factors. Teachers reported how much time children spent in small-group activities and centers on a typical day, using a scale of 0 = none, 1 = half hr or less, 2 = about 1 hr, 3 = about 2 hr, or 4 = 3 hr or more. Very few teachers engaged children in small-group activities; thus, due to significant positive skew, we dichotomized this variable into a dummy code (more than 30 min in small groups = 1), although we left the amount of time in centers as a continuous variable. Teachers reported the extent to which they assumed responsibility for instructional planning within their classrooms on a scale of 0 = 0% of instructional planning to 5 = 100% of instructional planning. Teachers also reported their feelings of self-efficacy with respect to making classroom instructional decisions. These two items were from the NICHD Study of Early Child Care and Youth Development and rated on a scale of 0 = no feelings of efficacy to 4 = very strong feelings of efficacy; we computed the average for use in analyses (α = .88). Finally, we conducted a 1-day observation of typical instruction in each classroom during the winter months. Research staff used the Classroom Assessment Scoring System (Pianta et al., 2006) to code the quality of classroom organization (1 = low quality to 7 = high quality; interrater agreement = 93%); see Supplementary Material for more information.
General Analytic Strategy
Prior to addressing our research question, we used descriptive statistics and histograms to document children’s intervention dosage in terms of the number of NBS! lessons that each child experienced and whether this differed among children. We then examined patterns characterizing variability in intervention dosage using an exploratory, person-centered approach similar to Logan et al. (2019) as well as a multilevel approach to determine the extent to which intervention dosage differed among children or was shared among small groups or instructors. We used the results of these preliminary analyses to inform our modeling approach for the main predictive analyses. Details of each analysis and analytic decision are presented along with results below.
Results
Preliminary Analyses
Descriptive statistics confirmed that children exhibited considerable variability in the number of lessons that they experienced (range of 0–20; M = 10.56; SD = 6.40). These data are depicted graphically in Figure 1.
Histograms (Full Sample and by Condition) Indicating Variability in the Number of Lessons Experienced by Children.
To examine patterns characterizing this variability, we first took a person-centered approach and used latent class analysis to determine whether there were qualitatively distinct groups of children based on their intervention dosage. We conducted this analysis using Mplus (Muthén & Muthén, 1998–2012) and the Mplus LCA helper (Uanhoro & Logan, 2017). In this analysis, whether or not each child experienced each lesson session served as the indicator variables. Because we did not have a priori expectations about dosage groups (e.g., how many groups/classes might emerge and patterns distinguishing these), we initially conducted an exploratory analysis to identify the number of classes evident in the data (Nylund et al., 2007). We estimated models in which we specified two to seven classes (see Figure 2) and followed the model-comparison methods used by Nylund et al. (2007) to determine which model best represented the data. We examined the Akaike information criterion (AIC) and Bayesian information criterion (BIC) as goodness of fit indices; lower values indicate better fit. We also used the adjusted Lo–Mendell–Rubin test (LMR; Lo et al., 2001) to test whether a given model fits statistically better than the model with one fewer class. We selected the five-class model as demonstrating the best fit, as it had the lowest AIC and BIC along with a statistically significant LMR. Notably, all models had entropy >.99, indicating that the models identified classes that uniquely grouped children into classes (i.e., children were identified in one and only one class).
Latent Class Analysis Results.
The five-class model is depicted in Figure 2, with each line representing a group of children, the x-axis indicating the lesson sessions in sequential order, and the y-axis indicating the probability that a child in that group experienced that lesson session. These findings show that groups were defined by when children stopped receiving lessons. For example, children in Class 1 had high probabilities of experiencing all 20 lessons, whereas children in Class 5 had an almost 0 probability of experiencing any lessons beyond Lesson 4; likewise, children in Classes 2, 3, and 4 also appeared to stop experiencing lessons over time. Given these findings, the overall number of lessons experienced seems sufficient for characterizing variability in dosage, and we thus used the total number of lessons experienced as the dependent variable in subsequent analyses.
The second way that we examined variability patterns in children’s intervention dosage involved calculating intraclass correlations (ICCs). This allowed us to understand the extent of shared variance at the child and other levels. For children whose teachers served as their intervention instructors, we estimated a two-level (children nested within small group/classroom) unconditional model with dosage as the dependent variable; for these children, small group was redundant with their classroom and teacher (i.e., only one small group per classroom and instructor). The ICC was .91, indicating that most of the variance was between small groups/classrooms. For children who received intervention from community aides, we estimated a three-level (children nested within small group/classroom nested within community aide) unconditional model; in these cases, small group was redundant with classroom/teacher, but the same community aide could serve multiple small groups. The ICCs were .51 at Level 2 and <.01 at Level 3, indicating little shared variance due to community aide. Given this finding, plus the fact that we had few community aides, we removed this third level of nesting and estimated a two-level model (children nested within small group/instructor/classroom) for the full analytic sample. The ICC was .83, suggesting that variability in intervention dosage largely may be due to differences across classrooms, small groups, and/or instructors, with less variability attributed to individual differences among children. We continued to use this two-level model in predictive analyses.
Predictive Analyses
Finally, we addressed our research question concerning whether child, instructor, and classroom factors predicted intervention dosage. Based on the results above, we estimated two-level (children nested within small group/instructor/classroom) models to predict intervention dosage from identified independent variables. Given the number of predictors, along with a priori grouping into child, instructor, and classroom factors, we estimated three separate models. We estimated these in Mplus using full information maximum likelihood estimation to preserve the full sample size and applied the Benjamini–Hochberg correction to each model to control the false discovery rate (Benjamini & Hochberg, 1995). All models controlled for the original intervention condition to which small groups were assigned (teacher-implemented condition = 1) as, in the larger project, we anticipated that small-group literacy intervention might be particularly challenging for teachers (see Farley et al., 2017; Zucker et al., 2021), and that community aides, as extra adults visiting the classroom, might be more likely to implement lessons. This was confirmed in prior work showing that community aides implemented significantly more NBS! lessons than teachers (Piasta et al., 2021; see also Figure 1).
Results, along with descriptive statistics for each predictor, are provided in Table 1. None of the child factors significantly predicted intervention dosage. Three instructor factors significantly predicted intervention dosage after controlling for the false discovery rate: instructor self-efficacy for teaching language and literacy, instructor perception of lesson acceptability, and average small-group size. Children whose instructors felt more efficacious in promoting language and literacy skills and whose instructors rated the intervention lessons as more acceptable tended to experience more lessons. Children also tended to experience more lessons when small groups included fewer children. Of the classroom factors, only the classroom teacher’s self-efficacy for decision-making significantly predicted intervention dosage; children tended to experience more lessons when their classroom teachers had greater self-efficacy for decision-making.
Descriptive Statistics for Independent Variables and Results of Multilevel Analyses Predicting Number of Lessons Children Experienced.
Note. Scales for all variables provided in the Method section. Mean for dichotomous variables represents proportion coded = 1. Multilevel models estimated using full information maximum likelihood to preserve the full sample size (154 children in 59 small groups). All continuous variables grand-mean centered. Independent variables that remained statistically significant predictors after applying the Benjamini–Hochberg procedure are in bold; all but time preparing lessons remained statistically significant after correction. Coeff. = coefficient; GRTR-R = Get Ready to Read—Revised.
Discussion
As small-group emergent literacy interventions are scaled up and used more widely in authentic preschool settings, understanding how these are implemented and how implementation might be better supported is critical (Piasta et al., 2021; Zucker et al., 2021). In this study, we examined intervention dosage within the context of the NBS! intervention, as one example of a small-group intervention for children identified as at risk for reading difficulties that is commercially available and used across the country. We were guided by an ecological orientation in which dosage might be influenced by child, instructor, and classroom factors (Durlak & DuPre, 2008; Fixsen et al., 2005). We documented considerable variability in the number of lessons that preschool children experienced, along with patterns and predictors of this variability. These findings make important contributions to the literature and have implications for preschool teachers and administrators seeking to successfully implement small-group interventions, as well as for researchers studying such interventions in authentic settings.
Variability in intervention dosage was not unexpected based on prior work that tracked the number of small-group emergent literacy lessons completed by instructors (Goldstein et al., 2017; Piasta et al., 2021). However, the disparity in dosage across children was striking, with some children experiencing very few lessons and others experiencing all intervention lessons. This finding highlights a need to better understand who is likely to experience high (or low) intervention dosage and why, which we have started to unpack in this study. Furthermore, our consideration of individual child dosage within the larger ecology of the classroom afforded use of both person-centered and multilevel approaches to examining patterns in this variability. Our results indicated that intervention dosage was sufficiently characterized by the number of lessons that children experienced, and that variability largely was attributed to differences between classrooms or instructors rather than differences among children. Conceivably, more complex dosage patterns could have existed (e.g., sporadic dosage due to intermittent child absences or inconsistent implementation). Yet, the number of lessons that children experienced was largely driven by when their instructors stopped offering lessons altogether. This finding requires replication in other samples when using other small-group interventions but suggests that parsimonious measures of dosage may be adequate. This finding also suggests the importance of monitoring whether intervention is ongoing and supports attending to ecological factors that may lead instructors to prematurely stop implementation.
Our results seemingly contradict suggestions that child factors, including attendance, are primary drivers of children’s intervention dosage. In addition to finding that very little of the variance in dosage was attributable to individual differences between children, none of the child factors significantly predicted the number of lessons experienced. Absence of evidence is not evidence of absence, and there could be alternative explanations for the lack of associations with child-related factors. For example, our inclusion criteria excluded children whom teachers reported as having severe attendance issues in the fall (n = 4), and although our logs captured lesson attendance, we did not directly measure children’s preschool attendance. Other, unmeasured factors (e.g., children’s social and behavioral skills; Ansari et al., 2019; Musci et al., 2019) also could predict individual differences in intervention dosage. However, there was very little variability at the child level, limiting the likelihood that child-level factors would be strong predictors. Thus, classroom and instructor factors may be more important than child factors when considering small-group emergent literacy intervention implementation in a preschool context. Emerging research has documented specific challenges to small-group intervention in preschool classrooms, including maintaining required child:adult ratios, unfamiliarity with use of small groups, and need for additional support in class/behavior management strategies (Piasta et al., 2021; Wyatt & Chapman-DeSousa, 2017; Zucker et al., 2021).
We identified four instructor and classroom factors that predicted intervention dosage: instructors’ self-efficacy for supporting language and literacy development, instructors’ perceived acceptability of the NBS! intervention, small-group size, and classroom teachers’ self-efficacy for decision-making. Each of these factors can be linked theoretically to dosage; however, to our knowledge, this study is the first to empirically demonstrate associations with dosage of small-group emergent literacy intervention. Notably, and as aligned with an ecological orientation, beyond individually predicting intervention dosage, these factors are likely interrelated. According to expectancy-value theory, beliefs about success and subjective value of activities contribute to motivation for engaging in activities (Wigfield & Eccles, 2000). For example, instructors with higher self-efficacy for supporting language and literacy development who perceived the NBS! intervention as acceptable likely believed they would successfully implement an intervention they saw as valuable, thus providing motivation for implementing lessons; this also emphasizes the central role of instructor feelings toward interventions. Similarly, fewer children in small groups might enhance instructors’ perceived acceptability, and thus perceived value, of the intervention.
Importantly, each of these factors is malleable, which leads to implications for increasing intervention dosage. Small-group size is perhaps most readily adjusted; instructors might consider limiting group sizes, particularly when new to implementing an intervention. Instructors’ self-efficacy for teaching language and literacy can be supported through follow-up coaching and availability of resources such as classroom books (Tschannen-Moran & Johnson, 2011; Tschannen-Moran & McMaster, 2009). Additional factors associated with instructor self-efficacy include classroom management skills (Emmer & Stough, 2001), opportunities for collaboration and administrative support (Ransford et al., 2009), and improvement in student performance (Guskey, 2002). Thus, using coaching models and encouraging collaboration among instructors, targeting management skills in addition to intervention content during professional development, integrating opportunities for instructors to notice children’s learning and attribute this to intervention, and increasing administrative supports to proactively address common challenges to small-group intervention (e.g., planning time, staff, and financial support) are all potential means for increasing dosage (see also Zucker et al., 2021). These components might be addressed at the school/center level but also might be integrated by researchers and intervention developers into the professional development accompanying small-group emergent literacy interventions, in addition to teaching the “nuts and bolts” of lesson implementation.
Our results also suggest that affording classroom teachers greater authority over curricular, time allocation, and other decisions may increase intervention dosage, and this is likewise malleable. Classroom teachers’ self-efficacy has been linked to their sense of school community, the extent to which they perceive sharing educational goals, having supportive professional relationships, and collaboratively influencing decisions within their school/center (Guo et al., 2011). Improving these aspects of teachers’ experiences, along with the general workplace climate (Chang, 2009), may increase the likelihood that they—or other instructors in their classrooms—prioritize and implement intervention. In this way, our findings echo calls to attend to the larger ecology involved in intervention implementation, particularly the need to attend to school/center (and broader) contextual factors that influence teachers, their instructional choices, and implementation (Durlak & DuPre, 2008; Zucker et al., 2021).
Our findings require further examination in other samples with other interventions beyond NBS! to determine the extent to which the current findings are replicable. Moreover, although we explored a large number of child, instructor, and classroom factors, as derived from theory and extant literature, we could not examine all possible factors in this single study and were also limited by the data and sample available. Future intervention studies might include measures of those factors that we found to be predictive but also direct measures of other factors theoretically linked to small-group intervention dosage (e.g., child attendance, social and behavior skills; instructors’ experience and skill in small-group management) including those at the school/center-level (e.g., administrative support). Furthermore, we examined child, instructor, and classroom variables as discrete and separate factors but recognize that there may be considerable interplay among these ecological systems. Future research with larger samples should consider interactions among child, teacher, and classroom factors as well as how constellations of factors might relate or interact in predicting dosage. Finally, the current findings are correlational in nature. This initial identification of predictors is important in that we have found several malleable factors that may influence successful implementation of small-group emergent literacy intervention, all of which can be facilitated through better intervention design, professional development, and other supports. Making such adjustments, and testing their efficacy, is a critical next step in enhancing positive intervention outcomes for preschool instructors and children.
Supplemental Material
sj-docx-1-jei-10.1177_10538151231155411 – Supplemental material for Small-Group Emergent Literacy Intervention Dosage in Preschool: Patterns and Predictors
Supplemental material, sj-docx-1-jei-10.1177_10538151231155411 for Small-Group Emergent Literacy Intervention Dosage in Preschool: Patterns and Predictors by Shayne B. Piasta, Alida Hudson, Robin Sayers, Jessica A. R. Logan, Kandia Lewis, Cynthia M. Zettler-Greeley and Laura L. Bailet in Journal of Early Intervention
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Nemours receives royalties through the sale of the commercially available curriculum involved in this research. The potential for bias in reporting study results was minimized through the adoption of the following precautions, as outlined in the investigators’ signed Memorandum of Understanding: (a) Nemours’s institutional responsibilities for this grant were limited to instructor professional development curriculum training and implementation fidelity monitoring, including the development and maintenance of the implementation fidelity database for this study; (b) Nemours investigators Zettler-Greeley, Bailet (affiliated with Nemours until April 2018; now retired from Kaplan Early Learning Company as of June 2021), and Lewis had no role in participant/site recruitment or data collection for the study, were blind to classroom and participant study assignment, and did not participate in data analyses concerning program impacts; (c) The Ohio State University (OSU) investigators conducted all data analyses concerning impacts of the Nemours BrightStart! program; and (d) OSU investigators retained the final decision as to the findings and interpretation that are reported.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A160261 awarded to The Ohio State University (Piasta). The opinions expressed are those of the authors and do not represent views of the Institute or U.S. Department of Education. The authors gratefully acknowledge the support of the research project staff, the cooperation of the Ready4Success initiative as led by Shelby Dowdy, and the early childhood professionals, children, and families without whom this research would not be possible.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
