Sage Journals: Discover world-class research

Abstract

Turnaround interventions often require or encourage low-performing schools to replace teachers, assuming that schools will recruit high-performing teachers who remain effective after transferring. However, teacher effectiveness may change after transferring, which could explain why some teacher replacement efforts do not improve student achievement. This paper contributes new information on the stability of teacher effectiveness by examining teachers who transfer into turnaround schools relative to teachers who transfer into low-performing but non-turnaround schools. I examine this issue by using two turnaround models in Tennessee, one with documented positive effects and one producing no effects on student achievement. I find that teacher effectiveness increases after transferring into the model that produced positive effects and either decreases or stays the same after transferring into the model that did not improve student achievement. These findings suggest that heterogeneity in turnaround effects may be partly explained by changes in teacher effectiveness after they transfer into turnaround schools.

Keywords

difference in differences school reform school turnaround teacher effectiveness

Introduction

School turnaround initiatives promising to dramatically improve student achievement often rely on interventions that help low-performing schools recruit effective teachers (Dragoset et al., 2017; Redding & Nguyen, 2020; Rice & Malen, 2010; Schueler et al., 2021). These staff replacement policies in turnaround schools have received substantial resource investments and policy attention, but evaluations report mixed results on student achievement, even when the reforms successfully recruit high-performing teachers (Carlson & Lavertu, 2018; Glazerman et al., 2013; Henry et al., 2020). Mixed results in some reform models, despite an influx of effective teachers, call into question the assumption that effective teachers will perform equally well after transferring into a turnaround school.

To test this assumption, I examine the stability of teacher effectiveness after they transfer into Tennessee’s turnaround schools. The Tennessee context is highly informative for this study because the state has been implementing two active turnaround models that have both recruited effective teachers into their schools (Henry et al., 2020): the Achievement School District (ASD) and local Innovation Zones (iZones). I describe the ASD and iZone reforms in detail below, but although both models recruited effective teachers, prior research has found that ASD schools did not improve student achievement, while iZone schools produced significantly positive effects (Pham et al., 2020; Zimmer et al., 2017). In this paper, I compare changes in the effectiveness of teachers who transfer into ASD schools and teachers who transfer into iZone schools, both relative to changes in the effectiveness of teachers who transfer into low-performing but non-turnaround schools. This analysis provides important new insights to explain whether the mixed impact of turnaround reforms can be at least partially explained by changes in teacher effectiveness after they transfer.

Additionally, this study makes important contributions to our understanding of contextual influences on teacher effectiveness. Although a few studies have examined differences in teacher effectiveness after transferring across schools serving different poverty levels or different levels of academic performance (Xu et al., 2012; Xu et al., 2015), researchers have generally focused on comparing low-performing schools with their higher-performing counterparts, without differentiating turnaround schools from other low-performing schools. Specifically focusing on teacher effectiveness in the turnaround context is important because these schools have the most resources to recruit teachers, so a large proportion of teachers transferring into low-performing schools are likely moving into turnaround settings (Papay & Hannon, 2018; Springer et al., 2015; Strunk et al., 2016; Zimmer et al., 2017). Moreover, teacher effectiveness in turnaround schools will likely differ from that in other low-performing schools because of the high-pressure environment and expectation for rapid change that are unique to turnaround schools. For example, after transferring to a turnaround school, teachers may improve their instructional practice because of the increased resources and unifying goal of dramatic improvement, or they may have difficulty maintaining a consistent level of effectiveness, given the strong pressure to quickly improve test scores. Also, turnaround interventions can affect the culture and climate in schools, and reforms that create a more positive learning environment or provide more opportunities for collaboration may help teachers improve, whereas disruptive reforms that leave teachers without consistent administrative support could result in lower teacher effectiveness (Meyers & Hitt, 2017).

Using statewide administrative data from Tennessee, this study contributes more nuanced evidence at the intersection of research on school reform and teacher effectiveness by examining teachers before and after they transfer into turnaround schools. I use a difference-in-differences (DID) model with teachers and school-by-year fixed effects to estimate within-teacher changes in effectiveness after transferring to a turnaround school relative to teachers transferring into low-performing but non-turnaround schools (which I call comparison schools). Given these goals, this study answers this question: To what extent does transferring into a turnaround school change teachers’ effectiveness?

My results suggest that teachers who move into iZone schools perform significantly better in reading and math relative to teachers who move into comparison schools. These increases in teacher effectiveness align with previous research finding that iZone reforms improved the schools’ professional environment and suggest that turnaround reforms should focus on recruiting teachers and creating professional environments where they can be effective (Strunk et al., 2016). In contrast, I find that after moving into ASD schools, teachers perform worse in reading relative to teachers who move into comparison schools. In math, the difference is not statistically significant. These results suggest that ASD reforms may have failed to improve student achievement partly because teacher effectiveness declined after they moved into ASD schools. When examining the heterogeneity of effects, I find that positive effects in reading for teachers who transfer into iZone schools are larger for Black teachers. The same effect for Black teachers transferring into ASD schools is not significant. Overall, these results indicate that teachers who transfer into turnaround schools face a different environment from teachers in other low-performing school settings, and successful reforms should attend to helping teachers adjust after they move.

Literature Review

Many models for school reform, including the most prominent turnaround initiatives, rely to some extent on recruiting effective teachers (Schueler et al., 2021). The focus on teacher recruitment has been largely influenced by two consistent findings in the research literature. First, teachers are the single-most-important school-based contributor to student achievement (Aaronson et al., 2007; Chetty et al., 2011; Rivkin et al., 2005; Rothstein, 2014; Sanders & Rivers, 1996). Second, high-performing teachers are unevenly distributed across schools because more effective and more experienced teachers tend to transfer into higher-performing schools in lower-poverty communities (Clotfelter, 2001; Feng & Sass, 2017; Hanushek et al., 2004; Imazeki, 2005; Lankford et al., 2002; Scafidi et al., 2007). Together, these two findings support a course of action where low-performing schools are given resources and either encouraged or required to recruit effective teachers. Under No Child Left Behind, this theory of action was operationalized through the four turnaround models federally mandated for schools receiving School Improvement Grants (SIGs): restart, turnaround, transformation, or closure. To avoid closure, schools could opt to restart, which involves closing and reopening under new management. Besides restarting, schools could also choose turnaround or transformation. Turnaround requires replacement of the school leader and at least 50% of the instructional staff, while transformation requires replacement of the school leader in addition to other interventions. Collectively referred to as turnaround, these four models have been heavily influential in shaping school reform efforts in recent years.

Under the current Every Student Succeeds Act (ESSA), efforts to support low-performing schools have become more diverse because states now have more autonomy over how to support their lowest-performing schools. Although states are no longer limited to one of the four SIG-prescribed turnaround models, current school reform initiatives continue to prioritize attracting high-performing teachers into low-performing schools by using such strategies as recruitment bonuses (Glazerman et al., 2013; Springer et al., 2015), involuntary transfer policies (Grissom et al., 2014), academies taught by the highest-performing teachers in the district (Schueler et al., 2017), and residency programs that place promising new teachers into high-need schools. The practice of recruiting effective teachers appears to be well supported by existing research on the impact of teachers, but these staffing interventions all assume that teacher effectiveness is largely stable across different school settings, while existing literature provides evidence that contextual factors can influence teachers (Jackson, 2013).

Organizational theory and empirical evidence suggest that school climate, culture, and context all shape teachers’ motivation, sense of self-efficacy, and satisfaction (Johnson et al., 2012; Kraft & Papay, 2014; Ladd, 2011; Loeb et al., 2005). For example, researchers find evidence that peer effects can alter teacher performance through productivity norms (Mas & Moretti, 2009) and informal learning networks (Jackson & Bruegmann, 2009). In addition to peers, multiple factors could potentially influence teacher effectiveness across schools, including level of administrative support (Boyd et al., 2011), availability of instructional resources (Horng, 2010), opportunities for in-service professional learning (Kraft et al., 2016), and orientation activities that help new teachers acclimate to the school (Ingersoll & Strong, 2011). These contextual features are especially influential in the turnaround context because reforms may affect a school’s professional environment. For example, turnaround interventions that invest in recruiting and supporting effective school leaders will likely give rise to a professional environment where teachers receive reliable administrative support (Meyers & Hitt, 2017). Thus, the effect of reforms on school culture and climate are distinct features that can affect teachers’ performance once they transfer.

Given evidence that school context matters, a few studies have investigated the stability of teacher effectiveness across schools (Jackson, 2013; Xu et al., 2012; Xu et al., 2015). However, the thin literature on this topic generally examines teacher effectiveness when they move from higher-poverty (or lower-performing) to lower-poverty (or higher-performing) schools, without attention to the distinct environment in turnaround schools. For example, Xu and colleagues (2012) find that teacher value-added scores are not affected when teachers transfer from schools serving fewer economically disadvantaged students to schools with more disadvantaged students. However, Jackson (2013) finds that teacher effectiveness increases after teachers move to a new school. These mixed findings from the few studies on this topic suggest that teacher effectiveness can change when switching schools, but not always, and that the direction of that change remains unclear, depending perhaps on the teacher’s new school, their individual characteristics, or some combination of both. Thus, by focusing on a policy-relevant but unexamined school context (turnaround schools), this paper adds nuance to research on the stability of teacher effectiveness by studying the distinct environment of schools undergoing mandated reforms.

Within the school turnaround literature, research on the stability of teacher effectiveness is especially important, given mixed results from impact evaluations that investigate the relationship between teacher recruitment and student achievement. Some researchers have found that the positive effects of turnaround can be partly attributed to the recruitment of high-performing teachers (Henry et al., 2020). Other researchers have found that even when turnaround schools bring in effective teachers, student test scores do not improve (Heissel & Ladd, 2017). This study provides a potential explanation for these disparate findings by examining whether teacher effectiveness changes after teachers transfer into a turnaround school.

Situated at the intersection of research on school turnaround and teacher effectiveness, this paper makes several contributions. First, this study examines an important theoretical assumption that has received very little attention in the school reform literature: that teachers recruited into turnaround schools will remain effective after they transfer. Second, it contributes to literature on the stability of teacher effectiveness by examining turnaround schools as a unique context among low-performing schools. Third, as states continue to evaluate and refine school improvement plans under ESSA, this research makes a policy contribution by helping educational authorities better understand the extent to which teacher effectiveness changes across schools. Finally, this study contributes new information to help reconcile mixed findings from the turnaround literature over the effect of teacher recruitment on student achievement because simply recruiting effective teachers may not be enough to improve low-performing schools. Rather, improving school performance also requires attention to how teacher performance changes after they transfer.

Tennessee Context

Tennessee’s turnaround approach required its Department of Education (TDOE) to identify the lowest-performing 5% of schools in the state, called priority schools. Priority schools are eligible for one of two turnaround interventions: the ASD and local iZones. As Tennessee’s boldest school reform model, the ASD is a statewide school district that removes priority schools from their local district to either directly manage them or convert them into charter schools managed by charter management organizations (CMOs). The ASD approach is similar to that of many restart models across the country that use state takeover in partnership with external management organizations to improve school performance (Gill et al., 2007; Papay et al., 2021; Schueler et al., 2017). In contrast, the iZones are a less dramatic model wherein schools remain part of their local district but are managed as part of a network that is supported by full-time district staff. Under ESSA, the iZone model has grown in popularity across the country as a way for states to give schools and districts more flexibility in implementing improvement strategies without the burden of administrative regulations (Patrick et al., 2018).

From 2012–2013 (the first year of Tennessee’s turnaround efforts) through 2017–2018 (the last year of data available for this study), Tennessee designated 116 schools as priority, most of which serve low-income and Black students in the Memphis metropolitan area. Because priority schools could be placed into either the ASD or an iZone, the decision of which new schools would be targeted for turnaround was made annually in meetings between TDOE and district leaders. My communications with TDOE and district leadership suggest that priority schools were not systematically chosen for ASD or iZone reforms. The only commonly used criterion was a desire to select schools within the same feeder pattern. Because of this desire to select schools in the same feeder pattern, many ASD and iZone schools were located relatively close to each other. By 2017–2018, 25 of the 116 priority schools had joined the ASD, 40 had joined an iZone, 25 were closed, and 26 were still operating without any turnaround interventions (i.e., business as usual). Although I cannot fully rule out school selection, I show in Appendix Table 1 that ASD, iZone, and non-turnaround priority schools had similar achievement levels and demographic characteristics in the years before turnaround. Thus, I use teachers who transferred into non-turnaround priority schools as a comparison group for teachers who transferred into ASD and iZone schools.

Although the ASD and iZone governance and management models differ markedly (i.e., the state-led ASD versus district-led iZones), the two models share a few similarities. First, both models focus on replacing teachers in the first year of turnaround as part of a bold push to recruit effective educators. To aid in recruitment, TDOE implemented a bonus pay program that provided turnaround schools with US$7,000 bonuses to recruit teachers with effectiveness ratings of 5, the highest possible score on Tennessee’s value-added assessment system (TVAAS). Thus, in the first year after beginning turnaround reforms, ASD and iZone schools recruited more teachers with high value-added scores than did non-turnaround priority schools (Henry et al., 2014). On average, the proportions of teachers new to ASD, iZone, and comparison schools were 0.51, 0.36, and 0.29, respectively. Also, both models required schools to continue enrolling students from the schools’ local catchment areas, including ASD schools managed by CMOs. This meant that neither the ASD nor iZones could choose which students attended their schools.

After initial efforts to replace the principal and teachers, the ASD gave school leaders and operators wide autonomy over day-to-day management, while the ASD central leadership mostly monitored progress. This approach reflects a theory of action that relies on recruiting effective principals and teachers and then removing bureaucratic obstacles so these educators can focus on improving student achievement. In contrast, iZone schools were given ongoing attention and resources from their districts. Thus, iZone schools hired additional instructional coaches, expanded professional learning communities, and offered performance bonuses for effective teachers (Iyengar et al., 2017). Additionally, the different ASD and iZone management structures (state versus district) led to differing levels of support from local communities around these turnaround schools (Glazer & Egan, 2018).

Research evaluating the overall impact of the ASD and iZones found that ASD schools did not perform better or worse than priority schools receiving no turnaround interventions, while iZone schools produced positive and significant student achievement gains (Pham et al., 2020; Zimmer et al., 2017). Also, researchers found that both ASD and iZone schools hired effective teachers (Henry et al., 2020). Therefore, Tennessee’s turnaround schools are a highly informative setting for examining the stability of teacher effectiveness, because differences in the overall effect of reforms may be partly explained by changes in teacher effectiveness after they transfer into ASD versus iZone schools.

Methods

Data, Measures, and Sample

Data for this analysis are provided by TDOE and managed by the Tennessee Education Research Alliance. These administrative data contain characteristics of students, teachers, and schools for all public schools throughout the state, between 2006–2007 and 2017–2018.¹ Student test scores in reading and math are standardized at the year-subject-grade level and used as the primary outcome of interest. The student data also include demographic characteristics, such as gender, race/ethnicity, English language learner (ELL) status, free or reduced-price meals (FRPM) eligibility, and special education (SPED) eligibility. Additionally, I create an indicator for whether the student is new to the school after making a nonstructural move. This new-to-school indicator does not count students as mobile in years that they make structural moves due to changing school levels (e.g., moving to a middle school after completing the final grade offered at an elementary school).

The teacher-level data sets contain demographic variables (e.g., gender, race/ethnicity, age), professional characteristics (e.g., salary, highest degree earned, years of experience), and performance ratings (e.g., observation scores). Tennessee’s teacher evaluation system provides teachers with observation scores that range from 1 (Not Effective) to 5 (Highly Effective). Additionally, the data include TVAAS scores for teachers in tested grades and subjects, which I standardize statewide within each year.

Finally, the school-level data include total enrollment, school level (elementary, middle, high), and school performance ratings, and all student and teacher characteristics can be aggregated to the school level. Teachers can be linked to individual schools in each academic year, so when teachers transfer, school addresses can be used to calculate the geodetic distance (as the crow flies), travel distance, and travel time between teachers’ sending and receiving schools. The travel distance and travel time are calculated by using the Here application program interface, which estimates driving distance and time based on Here’s proprietary database of average traffic conditions. I calculate these distances by using the geodist (Picard, 2012) and georoute (Weber & Péclat, 2017) packages in Stata.

The analytic sample is composed of all teachers in tested grades and subjects in Tennessee public schools who transferred into an ASD, iZone, or comparison priority school. Schools are considered part of the ASD or iZone only in years after they undergo reform.² My focus is on comparing the effectiveness of teachers who moved into ASD or iZone schools with that of teachers who moved into comparison priority schools, so the sample does not include teachers who transferred into non-priority schools or teachers who never transferred schools. Using only teachers who transferred between schools allows me to compare teacher effectiveness, net the influence of moving itself. Also, I use only teachers who transferred to non-turnaround priority schools as a comparison group to align with the theory that turnaround schools are a unique environment relative to other low-performing schools.

In addition to examining only teachers who moved into priority schools, I make two additional restrictions. First, I restrict the sample to only the observations in the sending and receiving schools before and after the first time a teacher moved from a non-priority school into a priority school. That is, for teachers with multiple moves, I drop observations after they moved into a new school for the second time. This restriction provides a cleaner interpretation because it does not include any instances where teachers made multiple moves from one priority school to another. However, in practical terms, this restriction is unlikely to affect my results because less than 1% of the total teacher-year observations were cases where teachers moved into multiple priority schools. Also, there were some cases of teachers who moved into a priority school and then stayed as that school transitioned to being part of the ASD or an iZone, and I do not include these teachers in the sample. Although these teachers did experience some changes in school environment when the school began turnaround, they differed from teachers who moved between schools because they continued to work with the same population of students even after turnaround interventions were put into place. This restriction removes about 4% of all teachers who ever moved into priority schools.

Thus, my sample of teachers who moved into priority schools can be divided into three mutually exclusive groups: (1) those who ever moved into an ASD priority school, (2) those who ever moved into an iZone priority school, and (3) those who ever moved into a non-ASD, non-iZone priority school. For each of these three groups, I examine teachers’ effects on student test scores in the sending school before they moved with those of the receiving school after they moved.³ Appendix Table 2 provides counts for the number of unique teachers in each category who can be linked with student test scores in reading and math. The table shows that my sample consists of 1,963 unique teachers in reading and 1,904 teachers in math.

Analytic Strategy

To examine the stability of teacher effectiveness, I use a DID model with teacher and school-by-year fixed effects, similar to the approach used in prior research to examine teacher effectiveness across schools (Jackson, 2013). The DID model uses within-teacher differences in effectiveness before and after transferring into turnaround schools relative to the same before-after differences for teachers who transferred into comparison schools, all while controlling for direct effects of the schools on student achievement in each year. The DID model relies on the assumption that in the absence of turnaround reforms, the teachers who transferred into turnaround schools would have similar changes in effectiveness as those who transferred into comparison schools. Below, I show evidence that the assumption of parallel trends holds in my data. I also outline a series of alternative specifications and robustness checks that provide additional evidence supporting estimates from the DID model as a plausibly causal effect of transferring into turnaround schools.

Thus, I estimate the following model, where y is the test score for student i in grade g with teacher j in school s and year t:

\begin{matrix} y_{i g j s t} = β_{0} + β_{1} P o s t m o v e_{j t} + β_{2} P o s t M o v e_{j t} * M o v e t o A S D_{j} \\ + β_{3} P o s t M o v e_{j t} * M o v e t o i Z o n e_{j} + β_{4} y_{i g s t - 1} \\ + X_{i g j s t}^{'} α + J_{j t}^{'} π + θ_{j} + δ_{s t} + ϕ_{g} + ε_{i g j s t} \end{matrix}

(1)

Equation 1 includes PostMove, an indicator that equals 1 in all years after teachers moved into a priority school. $β_{1}$ estimates the average difference in student achievement for teachers before and after moving into a comparison priority school. MovetoASD and MovetoiZone are indicators for teachers who ever moved into either ASD or iZone schools, respectively.⁴ The interaction terms allow me to estimate the coefficients of interest $β_{2}$ and $β_{3},$ which represent the difference in student achievement before and after moving for teachers who moved into ASD or iZone schools relative to the same difference for teachers who moved into comparison priority schools. Equation 1 also includes prior year test scores ( $y_{i g s t - 1})$ . In Appendix Table 4, I also test models that include quadratic prior-year achievement terms and models that include prior-year achievement in math and reading at the same time. The results are robust to these various specifications. $X_{i g j s t}$ is a vector of student characteristics used as control variables: gender, race, ELL, FRPM, SPED, and new-to-school. $J_{j t}$ is a vector of time-varying teacher characteristics, including whether the teacher had a graduate degree and years of experience. Because teacher, year, and experience effects cannot be estimated simultaneously, I follow recommendations in previous literature to include indicators for experience bins (1–3, 4–9, 10–24, and 25 or more years), with 25 or more as the reference category (Papay & Kraft, 2015). Equation 1 also includes teacher ( $θ_{j}$ ), school-by-year ( $δ_{s t}$ ), and grade ( $ϕ_{g}$ ) fixed effects. I estimate Equation 1 on reading and math test scores separately, with robust standard errors clustered at the teacher level. In Appendix Table 5, I test and find similar results when using standard errors clustered at the school level and bootstrap standard errors from randomly sampling whole schools 1,000 times with replacement.

The teacher and school-by-year fixed effects in Equation 1 are important because they allow me to disentangle school-specific teacher effects from the overall effect of the school and the general effect of the teacher.⁵ The school-by-year effect includes any features that affect all teachers and students equally at a school in any given year (e.g., a strong leadership team or characteristics of the local neighborhoods served by the school). Notably, estimates could be biased if the model attributes direct effects of ASD or iZone interventions on student test scores to changes in teacher effectiveness after they moved into the turnaround school. However, any direct effects of turnaround reforms are captured by the school-by-year fixed effect because the ASD and iZones are designed to be schoolwide turnaround models. Below, I also show that my results are robust to alternative specifications that use a school fixed effect and time-varying school characteristics.

Similarly, any teacher-specific factors that affected all their students (across all their schools) are part of the general teacher effect (e.g., content expertise) and are captured by the teacher fixed effect. The teacher fixed effect is particularly important to control for nonrandom selection of teachers into turnaround schools. Prior research has shown that the types of teachers who choose to work in the high-pressure environment of a turnaround school are likely different from the overall teacher workforce (Viano et al., 2020). For example, teachers with more experience working in high-need schools are probably more likely to choose an ASD or iZone school precisely because they are attracted to the turnaround environment. Thus, the teacher fixed effect allows me to control for these general teacher characteristics that are likely to affect the probability they would move into ASD or iZone schools and their effectiveness after they transferred. Controlling for the school and general teacher effect allows me to leverage variation in teacher effectiveness across different school settings (i.e., complementarities between specific teachers in specific schools). Heterogeneity across different teacher-school combinations is the variation I leverage to examine the stability of teacher effectiveness.

In addition to estimating to the overall effect of moving, I also estimate trends in teacher effectiveness over time by using an event history model:

\begin{matrix} y_{i g j s t} = β_{0} + \sum_{κ = - 5}^{κ = 3} β_{1 κ} M o v e Y e a r_{j κ} \\ + \sum_{κ = - 5}^{κ = 3} β_{2 κ} M o v e Y e a r_{j κ} * M o v e t o A S D_{j} \\ + \sum_{κ = - 5}^{κ = 3} β_{3 κ} M o v e Y e a r_{j κ} * M o v e t o i Z o n e_{j} + β_{4} y_{i g s t - 1} \\ + X_{i g j s t}^{'} α + J_{j t}^{'} π + θ_{j} + δ_{s t} + ϕ_{g} + ε_{i g j s t} \end{matrix}

(2)

Equation 2 replaces the $P o s t M o v e$ indicator with a vector of indicators for the number of years before and after teachers transferred into the priority school ( $M o v e Y e a r$ ), where 0 is the year immediately prior to moving and 1 is the first year after teachers moved into the priority school. Because turnaround began in 2012–2013, it is possible for teachers to have moved into a priority school in 2012–2013 and have 6 years of post-move outcomes through 2017–2018; however, the number of teachers who have more than 3 years of post-move data is extremely limited, especially after separating into ASD, iZone, and comparison priority schools (see Appendix Table 2). Therefore, I estimate Equation 2 by using indicators for 1, 2, and 3 or more years after moving.⁶ Interactions between the year indicators and $M o v e t o A S D$ and $M o v e t o i Z o n e$ allow me to estimate the cumulative effect for each year before and after moving, with Year 0 as the reference category. For example, $β_{2 κ = 2}$ is the cumulative effect for a teacher who was in their second year after moving into an ASD school relative to the year just before they moved, all compared to the same second year minus pre-move year change for a teacher who transferred to a non-turnaround priority school. Equation 2 is also estimated for reading and math separately, with standard errors clustered at the teacher level.

Finally, to better understand whether teacher characteristics have a moderating influence on the effect of transferring into a turnaround school, I estimate Equation 1 with the addition of individual teacher characteristics interacted with the post-move indicators and the indicators for moving into ASD and iZone schools. Specifically, I examine whether the teacher (a) was female, (b) was Black,⁷ (c) had a graduate degree, (d) had a TVAAS score of 4 or greater in the year before moving, (e) had more years of experience than the median for all movers in the year before moving, and (f) had tenure in the sending school that is above median in the year before moving.⁸ I choose these characteristics because previous literature has shown that teachers’ gender, effectiveness, experience, and tenure in their current school all predict how likely they are to turnover and their effectiveness (Nguyen et al., 2020). I also examine Black teachers specifically because the vast majority of students in Tennessee’s priority schools are Black (see Table 2 below), and previous research has shown that having a same-race teacher can significantly increase student achievement (Joshi et al., 2018). The three-way interactions for each characteristic ( $T e a c h e r C h a r a c t e r i s t i c * P o s t M o v e * M o v e t o A S D$ and $T e a c h e r C h a r a c t e r i s t i c * P o s t M o v e * M o v e t o i Z o n e$ ) allow me to estimate heterogeneous effects of moving into ASD or iZone schools for different groups of teachers.

Results

Descriptive Results

Table 1 shows descriptive characteristics of all teachers who moved into priority schools, in the baseline year before they moved. Most teachers who moved to a priority school were female (75%–81%) and Black (60%–68%), and most non-Black teachers were white (31%–38%), with fewer than 2% who were a different race/ethnicity. ASD and iZone schools hired teachers with higher TVAAS and observation scores than did comparison schools, with the most effective teachers moving into iZone schools. For example, teachers moving into iZone schools had an average observation score of 3.74, relative to 3.67 and 3.57 for teachers moving into ASD and comparison schools, respectively.⁹ Teachers moving into ASD schools also tended to have fewer years of experience than did teachers moving into iZone and comparison schools. The modal group of teachers moving into ASD schools had 1–3 years of experience (about 47%), whereas the modal group of teachers moving into iZone (36%) and comparison schools (40%) had 10–24 years of experience. Previous research has shown that these differences in the effectiveness and experience of teachers moving into ASD and iZone schools help partly explain positive iZone effects (Henry et al., 2020).

Table 1

Descriptive Characteristics of Movers in the Year Prior to Moving

	(1)	(2)	(3)
	Teachers who move to comparison schools	Teachers who move to ASD schools	Teachers who move to iZone schools
Characteristics of teachers
Female	0.75	0.81***	0.81***
Age (years)	42.43	35.78***	40.67***
White	0.31	0.34	0.38***
Black	0.68	0.65	0.60***
Other race	0.01	0.02	0.02
Standardized TVAAS index	−0.34	−0.19	−0.12**
Observation score (1–5)	3.57	3.67*	3.74***
Teacher salary ($1,000)	52.31	47.27***	50.74*
Tenure	3.34	2.66***	3.56*
Experience: 1–3 years	0.25	0.47***	0.29*
Experience: 4–9 years	0.24	0.29**	0.26
Experience: 10–24 years	0.40	0.20***	0.36*
Experience: 25-plus years	0.11	0.04***	0.09
MA degree or higher	0.64	0.56***	0.66
Characteristics of move
Within-district move	0.78	0.73*	0.80
Geodetic distance (miles)	16.90	18.81	17.56
Travel distance (miles)	21.26	23.43	22.08
Travel time (minutes)	28.83	30.42	29.66

Note. Only teachers who move into ASD, iZone, or comparison priority schools are included in the sample. Significance stars in column 2 are from t-tests comparing teachers moving into ASD versus comparison schools. Significance stars in column 3 are from t-tests comparing teachers moving into iZone versus comparison schools. ASD = Achievement School District; iZone = Innovation Zone; MA = master’s degree; TVAAS = Tennessee’s value-added assessment system.

p < 0.10. * p < 0.05. ** p < 0.01. *** p < 0.001.

Table 1 also shows that depending on whether they move to ASD, iZone, or comparison schools, about 73%–80% of teachers who moved into priority schools were coming from other schools in the same district, with an average geodetic distance of 16.90 to 18.81 miles between schools. Travel distances were similar, translating to an average travel time of about 30 minutes between sending and receiving schools. These times suggest that teachers who moved into priority schools were coming from nearby sending schools.

Table 2 shows average characteristics of sending and receiving schools for teachers who moved into ASD, iZone, and comparison schools. The table shows that, on average, teachers in the sample were moving between low-performing schools with below-average standardized test scores. However, teachers who moved into priority schools tended to be coming from moderately higher-performing sending schools, reflecting the status of priority schools as the lowest-performing schools in the state. Moreover, sending schools tended to serve larger proportions of white students, fewer FRPM eligible students, and fewer new-to-school students. Also, students in sending schools tended to have somewhat higher attendance rates. For example, teachers moved from sending schools where 77% of students were FRPM-eligible on average to receiving iZone schools where 89% of students were FRPM-eligible. Overall, these results show that teachers who transferred into priority schools were coming from sending schools that were modestly higher performing and serving somewhat fewer disadvantaged students.

Table 2

Descriptive Characteristics of Sending and Receiving Schools

	Comparison		ASD		iZone
	Sending	Receiving	Sending	Receiving	Sending	Receiving
Student test scores
Average reading	−0.62	−0.95***	−0.64	−0.98***	−0.50	−0.89***
	(0.60)	(0.57)	(0.60)	(0.46)	(0.62)	(0.51)
Average math	−0.56	−0.90***	−0.59	−0.97***	−0.44	−0.74***
	(0.61)	(0.56)	(0.59)	(0.54)	(0.61)	(0.51)
Student body characteristics
Proportion female	0.49	0.49	0.49	0.48**	0.49	0.48***
	(0.05)	(0.09)	(0.04)	(0.03)	(0.04)	(0.02)
Proportion white	0.13	0.03***	0.09	0.02***	0.21	0.05***
	(0.23)	(0.05)	(0.17)	(0.03)	(0.28)	(0.06)
Proportion Black	0.80	0.87***	0.81	0.93***	0.70	0.87***
	(0.27)	(0.15)	(0.23)	(0.10)	(0.32)	(0.14)
Proportion other race	0.07	0.10***	0.10	0.05***	0.09	0.08**
	(0.10)	(0.12)	(0.13)	(0.07)	(0.11)	(0.11)
Proportion ELL	0.03	0.05***	0.05	0.03***	0.04	0.04*
	(0.06)	(0.08)	(0.08)	(0.04)	(0.07)	(0.07)
Proportion FRPM	0.81	0.87***	0.82	0.84*	0.77	0.89***
	(0.18)	(0.13)	(0.18)	(0.14)	(0.23)	(0.08)
Proportion SPED	0.15	0.16***	0.13	0.16***	0.15	0.18***
	(0.05)	(0.05)	(0.05)	(0.04)	(0.06)	(0.04)
Proportion new-to-school	0.31	0.36***	0.32	0.35***	0.29	0.36***
	(0.15)	(0.15)	(0.16)	(0.14)	(0.13)	(0.12)
Average attendance rate	93.54	91.92***	93.58	91.73***	93.58	91.71***
	(2.97)	(4.89)	(3.37)	(3.34)	(2.73)	(2.44)
Total enrollment	647.16	523.00***	618.06	424.90***	604.31	486.88***
	(416.75)	(219.12)	(363.20)	(169.68)	(329.47)	(145.35)
Teacher characteristics
Proportion female	0.77	0.75**	0.77	0.79***	0.78	0.77***
	(0.14)	(0.12)	(0.12)	(0.11)	(0.13)	(0.11)
Proportion white	0.40	0.32***	0.39	0.34***	0.51	0.35***
	(0.26)	(0.19)	(0.21)	(0.15)	(0.29)	(0.20)
Proportion Black	0.59	0.66***	0.60	0.64***	0.48	0.62***
	(0.26)	(0.19)	(0.22)	(0.16)	(0.30)	(0.22)
Proportion other race	0.01	0.02***	0.01	0.02***	0.01	0.03***
	(0.02)	(0.03)	(0.03)	(0.04)	(0.02)	(0.05)
Average experience (years)	12.47	10.84***	10.39	5.72***	12.06	9.48***
	(3.60)	(3.22)	(4.14)	(1.66)	(3.25)	(2.39)
Proportion MA degree or higher	0.57	0.57	0.56	0.49***	0.59	0.61***
	(0.12)	(0.11)	(0.13)	(0.13)	(0.11)	(0.13)
Average standardized TVAAS index	−0.23	−0.41***	−0.18	−0.48***	−0.09	−0.05
	(0.58)	(0.66)	(0.65)	(0.66)	(0.51)	(0.58)
Average observation score (1–5)	3.78	3.72***	3.83	3.41***	3.73	3.84***
	(0.37)	(0.33)	(0.35)	(0.42)	(0.43)	(0.40)

Note. Standard deviations are in parentheses. Stars show significance levels from t-tests comparing characteristics of sending and receiving schools. New-to-school includes only students who make a non-structural move. ASD = Achievement School District; ELL = English language learners; iZone = Innovation Zone; FRPM = free and reduced-price meals; MA = master’s degree; SPED = special education; TVAAS = Tennessee’s value-added assessment system.

p < 0.10. * p < 0.05. ** p < 0.01. *** p < 0.001.

Although all teachers moving into priority schools tended to come from sending schools with higher proportions of white teachers and more experienced teachers, descriptive teacher characteristics between sending and receiving schools reveal different patterns between ASD and iZone schools. For example, teachers who moved into ASD schools were coming from sending schools that averaged 10.39 years of teacher experience to arrive in ASD schools where average teacher experience was 5.72 years. The parallel values for teachers moving to iZone schools were 12.06 years to 9.48 years. Also, teachers who moved into ASD schools were coming from sending schools with average standardized TVAAS scores of −0.18 to an average of −0.48 in the receiving ASD school (a decrease in average effectiveness). However, teachers were moving from sending schools with average teacher TVAAS scores of −0.09 to iZone schools with average teacher TVAAS scores of −0.05 (an increase in average effectiveness). Teacher observation scores follow a similar pattern—decreasing for teachers who moved into ASD schools and increasing for teachers who moved into iZone schools.

Parallel Trends

To examine trends in teacher effectiveness, I plot average standardized TVAAS scores for teachers in each year before and after they transferred into a priority school in Figure 1. I plot TVAAS scores in this figure instead of directly plotting student test scores because my goal is to show trends in teacher effectiveness, and plotting descriptive test scores would capture a host of effects on student achievement outside the teacher (e.g., factors outside the school). Figure 1 shows that, prior to moving, teachers who moved into ASD, iZone, and comparison schools had effectiveness trends that were reasonably parallel, suggesting that teachers moving into comparison priority schools are a valid counterfactual for teachers moving into turnaround schools. Providing statistical tests of parallel trends, estimates from the event history model show that the pre-move estimates in each year (and jointly for all pre-move years) were not significantly different between teachers transferring to ASD versus comparison schools, nor were they significantly different for teachers transferring to iZone versus comparison schools (see Figure 2 and Appendix Table 6).

Figure 1.

Average TVAAS Scores in the Years Before and After Transferring Schools for Teachers Moving Into ASD, iZone, and Comparison Priority Schools.

Figure 2.

Event History Model for the Years Before and After Teachers Transferred Into a Priority School.

Figure 1 shows noticeable drops in average teacher TVAAS for ASD teachers after moving, whereas teachers who moved into iZone schools had increased TVAAS scores. Appendix Figure 1 shows similarly parallel trends for observation scores in the years before teachers transferred into ASD, iZone, or comparison scores. Like the TVAAS scores, teacher observation scores increased for teachers moving into iZone and comparison schools. After an initial dip in the first year, teachers moving into ASD schools also experienced an increase in observation scores, suggesting that teachers were potentially learning to meet the expectations of observers in ASD schools, even if their effects on student test scores did not improve over time.

DID Results

Table 3 shows results from estimating Equation 1. Column 1 shows results for reading with a teacher fixed effect. Column 2 adds a school fixed effect, and column 3 shows the preferred model, including the teacher and school-by-year fixed effects. Columns 4–6 show the parallel results for math. The coefficients on $P o s t M o v e$ show that the average change in student test scores after teachers moved into comparison priority schools is not statistically significant, except a marginally significant difference of 0.053 standard deviation units (SDUs) in reading when including the teacher and school fixed effects. Focusing on the preferred results, for teachers who moved into ASD schools, the average post-move–pre-move difference in reading is 0.053 SDUs lower than the same difference for teachers who moved into comparison schools, but this difference is marginally significant at the 10% level. The effect is not statistically significant in math. In contrast, the effect for teachers who moved into iZone schools is positive and significant for reading (0.101 SDU) and math (0.212 SDU), suggesting improvements in effectiveness for iZone teachers. The effect estimates for teachers who moved into ASD schools versus those who moved into iZone schools are significantly different from each other across all models and subjects.

Table 3

DID Effects Before and After Teachers Moved Into ASD and iZone Schools Relative to Non-turnaround Priority Schools

	(1)	(2)	(3)	(4)	(5)	(6)
	Reading	Reading	Reading	Math	Math	Math
Post-move	0.007	0.053+	0.010	−0.033	0.016	0.015
	(0.015)	(0.030)	(0.017)	(0.025)	(0.028)	(0.037)
Ever moved to ASD*post-move	−0.051*	−0.072*	−0.053+	0.000	0.026	0.150
	(0.024)	(0.030)	(0.030)	(0.041)	(0.053)	(0.105)
Ever moved to iZone*post-move	0.068***	0.097***	0.101***	0.164***	0.208***	0.212*
	(0.020)	(0.028)	(0.027)	(0.031)	(0.054)	(0.096)
Teacher FE	Yes	Yes	Yes	Yes	Yes	Yes
School FE	No	Yes	No	No	Yes	No
School × Year FE	No	No	Yes	No	No	Yes
p_{Moveto ASD=MovetoiZone}	0.00	0.00	0.00	0.00	0.01	0.04
R-squared	0.60	0.60	0.60	0.54	0.54	0.57
Observations	291,291	291,288	291,288	264,102	264,097	264,046

Note. Robust standard errors in parentheses are clustered at the teacher level. Sample includes only teachers who transferred schools and not teachers who stayed when priority school began turnaround reforms. All models include grade fixed effects and the full set of covariates. Student-level covariates include gender, race, English language learner status, free or reduced-price meal eligibility, special education eligibility, and mobility. Teacher covariates include graduate degree attainment and experience. Teacher experience is coded as indicators for 1–3 years, 4–9 years, 10–24 years, and 25+ years, with 25+ years as the reference category. ASD = Achievement School District; DID = difference-in-differences; FE = fixed effects; iZone = Innovation Zone.

p < 0.10. * p < 0.05. ** p < 0.01. *** p < 0.001.

Figure 2 shows results from the event history analysis that includes indicators for years before and after teachers moved into a priority school (Equation 2). The figure plots coefficients from the interaction between moving to the ASD or iZones and each of the year indicators. For full results, see Appendix Table 6. For teachers who moved into ASD schools, Figure 2 shows that average teacher contributions to student achievement did not differ significantly between the baseline year and each year before. The F-test of joint significance for all pre-move years was also insignificant ( $p_{p r e = 0} =$ 0.95 in reading and 0.16 in math). Then, teachers who transferred into an ASD school experienced a −0.07 SDU effect on reading scores in Year 1 after the move. The effect of moving into an ASD school was not statistically significant after Year 1 in reading. In math, the estimates were positive but not significant in any post-move year. Turning to the iZones, coefficients for the pre-move years were also not individually or jointly different from the baseline year ( $p_{p r e = 0} = 0.86$ in reading and 0.92 in math). After teachers transferred into iZone schools, Figure 2 shows positive effects in reading (0.10 SDU) and math (0.20 SDU) in the first post-move year. Also, teachers transferring into iZone schools experienced a positive effect in reading for Years 2 (0.07 SDU) and 3+ (0.14 SDU) and in math for Year 3 (0.26 SDU).

Table 4 shows heterogeneous effects for different groups of teachers in reading (panel A) and math (panel B). The first column replicates the main results from Table 3. Then, columns 2–7 each add a different teacher characteristic as a moderator, with the coefficients of interest shown in the three-way interactions between the teacher characteristic, $P o s t M o v e$ , and the indicators for teachers who moved into ASD or iZone schools. I find that most of the teacher characteristics in Table 4 did not have a significant influence on the effect of moving into either ASD or iZone schools. The only positive and statistically significant moderating effect was for Black teachers who transferred into iZone schools. Table 4 shows that the effect for teachers who transferred into an iZone school was 0.126 SDU higher in reading among Black teachers relative to non-Black teachers who transferred into iZone schools. Note that the post-move estimate in reading for non-Black teachers transferring into iZone schools was not statistically significant, suggesting that Black teachers were driving the gains in effectiveness for those who moved into iZone schools. Table 4 also shows a significant negative effect in math for teachers with a TVAAS score of 4 or greater who transferred into ASD schools (−0.289 SDU).

Table 4

Heterogeneous Results by Characteristics of Teachers Who Transferred Schools

	(1)	(2)	(3)	(4)	(5)	(6)	(7)
Teacher Characteristics:	Overall results	Female	Black	Masters and above	TVAAS ≥ 4	Experience > median	Tenure > median
Panel A: reading
Post-move	0.010	0.023	0.076**	0.004	0.005	0.011	0.008
	(0.017)	(0.026)	(0.029)	(0.026)	(0.020)	(0.021)	(0.023)
Ever moved to ASD*post-move	−0.053+	−0.045	−0.124*	−0.061	−0.038	−0.075*	−0.040
	(0.030)	(0.051)	(0.052)	(0.039)	(0.035)	(0.032)	(0.038)
Ever moved to iZone*post-move	0.101***	0.080*	−0.006	0.131***	0.111***	0.110***	0.129***
	(0.027)	(0.037)	(0.040)	(0.036)	(0.031)	(0.032)	(0.037)
Teacher characteristics
Teacher characteristic*post-move		−0.015	−0.078**	0.009	0.011	0.000	−0.000
		(0.027)	(0.028)	(0.028)	(0.024)	(0.024)	(0.026)
Teacher characteristicpost-movemoved to ASD		−0.011	0.089+	0.014	−0.029	0.039	−0.028
		(0.050)	(0.054)	(0.043)	(0.046)	(0.041)	(0.045)
Teacher characteristicpost-movemoved to iZone		0.030	0.126***	−0.043	−0.024	−0.017	−0.039
		(0.037)	(0.036)	(0.034)	(0.033)	(0.031)	(0.034)
Observations	291,288	289,420	291,123	291,288	291,288	291,288	291,288
Panel B: math
Post-move	0.015	0.105	0.015	0.037	−0.027	0.065	0.103+
	(0.037)	(0.069)	(0.091)	(0.062)	(0.047)	(0.053)	(0.054)
Ever moved to ASD*post-move	0.150	0.107	0.141	0.173	0.274*	0.184	0.110
	(0.105)	(0.162)	(0.156)	(0.129)	(0.113)	(0.128)	(0.122)
Ever moved to iZone*post-move	0.212*	0.130	0.154	0.273*	0.277**	0.170	0.127
	(0.096)	(0.118)	(0.145)	(0.117)	(0.102)	(0.112)	(0.108)
Teacher characteristics
Teacher characteristic*post-move		−0.103	0.003	−0.041	0.082	−0.085	−0.143*
		(0.073)	(0.087)	(0.063)	(0.081)	(0.064)	(0.061)
Teacher characteristicpost-movemoved to ASD		0.056	0.018	−0.024	−0.289*	−0.052	0.013
		(0.131)	(0.129)	(0.090)	(0.121)	(0.098)	(0.089)
Teacher characteristicpost-movemoved to iZone		0.085	0.074	−0.044	−0.141	0.066	0.119
		(0.086)	(0.096)	(0.075)	(0.087)	(0.078)	(0.078)
Observations	264,046	262,637	263,910	264,046	264,046	264,046	264,046

Note. Robust standard errors in parentheses are clustered at the teacher level. Sample includes only teachers who transferred schools and not teachers who stayed when priority school began turnaround reforms. All models include school-by-year and grade fixed effects and the full set of covariates. Student-level covariates include gender, race, English language learner status, free or reduced-price meal eligibility, special education eligibility, and mobility. Teacher covariates include graduate degree attainment and experience. Teacher experience is coded as indicators for 1–3 years, 4–9 years, 10–24 years, and 25+ years, with 25+ years as the reference category. ASD = Achievement School District; iZone = Innovation Zone; TVAAS = Tennessee’s value-added assessment system.

p < 0.10. * p < 0.05. ** p < 0.01. *** p < 0.001.

Robustness Checks

Figure 1 and Appendix Figure 1 show evidence of parallel trends in effectiveness among teachers who moved into ASD, iZone, and comparison priority schools in the years before they moved, suggesting that teachers who moved into comparison priority schools were a valid counterfactual. However, I also test whether my results are robust to this choice of comparison group by comparing teachers who transferred into ASD or iZone schools with teachers who transferred in low-performing schools in the bottom 6%–10% of schools in Tennessee. Appendix Table 7 shows that using this alternative comparison group results in similar estimates.

Besides parallel trends, I test whether my results are driven by omitted confounders that affected teachers’ likelihood of transferring and student outcomes (e.g., principal effectiveness). I also test for uncharacteristic dips in performance in the year before teachers transferred that could mean the results were driven by mean reversion. The appendix describes these checks, and Appendix Tables 8–10 show that results are robust to these alternative explanations. Finally, a potential alternative explanation for changes in teacher effectiveness after transferring into ASD or iZone schools may be that the DID models are capturing direct effects of the ASD or iZone interventions and not only differences in teacher effectiveness across different school settings. However, direct effects of turnaround interventions are unlikely to bias my results because any factors that affected all teachers and students in the same school and year (including turnaround reforms) would be captured by the school-by-year fixed effect. As an additional alternative specification, I test models that include time-varying school characteristics and again find that my results are robust (Appendix Table 4).

Additionally, to more directly test for changes in teacher effectiveness, I estimate DID models by using teacher observation scores and TVAAS scores as the outcome. Although observation scores and TVAAS scores are imperfect measures of effectiveness, they were directly measured for teachers. That is, although test scores may have been influenced by turnaround reforms that operated outside changes in teacher effectiveness, any changes in teacher observation and TVAAS scores were more likely the direct result of the organizational climate and culture that teachers experienced when they transferred into the turnaround school. The estimates, shown in Table 5, suggest that teacher observation and TVAAS scores increased after teachers moved into iZone schools and decreased after they moved into ASD schools relative to changes for teachers who moved into non-turnaround priority schools, but the estimates are not statistically significant.

Table 5

DID Effects on Teacher Observation and TVAAS Scores

	(1)	(2)
	Observation score(1–5)	StandardizedTVAAS
Post-move	0.002	0.039
	(0.060)	(0.138)
Ever moved to ASD*post-move	−0.261	−0.147
	(0.328)	(0.385)
Ever moved to iZone*post-move	0.105	0.216
	(0.086)	(0.226)
R-squared	0.77	0.85
Observations	5,223	2,470

Note. Robust standard errors in parentheses are clustered at the teacher level. Sample includes only teachers who transferred schools and not teachers who stayed when priority school began turnaround reforms. All models include teacher and school-by-year fixed effects, prior year observation score, and teacher covariates: graduate degree attainment and experience. Teacher experience is coded as indicators for 1–3 years, 4–9 years, 10–24 years, and 25+ years, with 25+ years as the reference category. No grade or student characteristics are included because teacher observation and TVAAS scores are at the teacher level, not the student level. ASD = Achievement School District; DID = difference-in-differences; iZone = Innovation Zone; TVAAS = Tennessee’s value-added assessment system.

p < 0.10. * p < 0.05. ** p < 0.01. *** p < 0.001.

Finally, given recent literature highlighting potential bias in DID models that pool staggered treatment times (Callaway & Sant’Anna, 2021; Goodman-Bacon, 2021), I examine results from estimating Equation 1 separately for each cohort of teachers moving into ASD, iZone, and comparison schools. My results show that effect estimates are similar and lead to the same conclusions when I examine each incoming teacher cohort separately (Appendix Table 12). I also examine whether results vary across the different ASD and iZone cohorts (Appendix Table 13) and again find similar effects. Similar results across cohorts of incoming teachers and cohorts of ASD and iZone schools are in alignment with how the ASD and iZone interventions changed very little over time. They also provide evidence that the pooled DID estimates from Equation 1 were not by biased by differential effects based on when teachers transferred or when schools began turnaround reforms.

Discussion

This study examines the stability of teachers’ effectiveness after they transfer into turnaround schools. Results suggest that teachers who transferred into iZone schools experienced significant positive increases in their students’ achievement. The effects for teachers transferring into ASD schools were inconsistent: mostly negative in reading and null in math. Separating these effects into the number of years before and after transferring, I find that the positive effects for teachers transferring into iZone schools appeared in both subjects for multiple post-move years, whereas the negative effect for ASD teachers in reading was only significant in the first year. These results are robust to alternative explanations and mostly do not differ by characteristics of the teachers themselves, except the positive effect in reading is larger for Black teachers than for non-Black teachers who moved into iZone schools. The post-move estimate in reading for non-Black teachers (the vast majority of whom were white) moving into iZone schools was quite modest in magnitude (−0.006 SDU) and not significantly different from that of non-Black teachers moving into comparison schools, suggesting that the improvement in reading for movers into iZone schools came primarily from Black teachers. In contrast, teachers with high TVAAS scores experienced negative effects in math after moving into ASD schools.

This study contributes to the school turnaround literature by providing compelling evidence to help reconcile inconsistent results from previous evaluations of turnaround reforms. For example, a meta-analysis by Schueler et al. (2021) finds that school improvement models emphasizing teacher replacement are associated with the largest positive effects relative to 13 other commonly used interventions (e.g., curricular change or principal replacement). Additionally, a meta-analysis by Redding and Nguyen (2020) finds that transformation models, which do not require teacher replacements, are associated with larger positive effects in math than are turnaround models, which do require teacher replacements. Results in this paper provide evidence that teachers’ effectiveness can change after they transfer into a turnaround school, suggesting that the inconsistencies over whether replacing teachers can improve student achievement may be explained by whether and to what extent teachers’ effectiveness changes after they move into a turnaround school. Specifically, although ASD and iZone schools recruited more effective teachers than did non-turnaround priority schools, I find that average student achievement increased significantly for teachers who moved into iZone schools, whereas teachers in ASD schools experienced declines or no changes. Improved teacher effectiveness in iZone schools helps explain positive overall effects of iZone reforms and supports an expanded theory of action for turnaround that goes beyond simply recruiting effective teachers. One implication for turnaround policy is the need to invest in strategies that help teachers adjust to their new school. Previous research suggests that robust teacher induction (Carver & Feiman-Nemser, 2009), strong collaborative instructional teams (Goddard et al., 2007), and support from more experienced mentors (Davis & Higdon, 2008) are potentially effective strategies to support teachers who transfer into a new school, and ESSA reform plans should consider these strategies to follow up on any initial recruitment efforts.

Results from this study also corroborate prior research finding that a component of teachers’ effectiveness stems from differences in the marginal effectiveness of school inputs across teachers (Jackson, 2013; Strunk et al., 2016). That is, teachers’ performance is partly influenced by how well their teaching style and personal preferences fit within the culture at their school. Because the work culture within a turnaround school is likely distinct from that of other low-performing schools that do not have the same accountability pressures, this result suggests that leaders in turnaround schools should recruit teachers based not only on their overall effectiveness but also on their ability to adapt to a turnaround environment.

Comparing contextual and organizational features in ASD and iZone schools can also help illuminate potential reasons why teacher effectiveness improved in iZone schools and not in ASD schools. One barrier to improvement in ASD schools came from resistance among local community leaders, who viewed the state-run district as an external takeover of their community schools (Glazer & Egan, 2018). Mistrust from local communities, along with the disruptive ASD interventions that required schools to close and reopen under the management of CMOs with little prior experience in the local context, meant that ASD schools had difficulty hiring teachers and principals locally. Prior research has found that these difficulties led ASD schools to hire principals who were inexperienced and unfamiliar with the schools they were hired to lead (Dixon et al., 2021). The difficulty with recruiting experienced principals led to high rates of principal turnover, which researchers have documented as a suppressor of potentially positive ASD effects (Henry et al., 2020). Without consistent leadership, the unstable environment in many ASD schools likely hindered teachers’ ability to maintain high levels of effectiveness after they transferred into an ASD school (Meyers & Hitt, 2017).

Another potential reason teacher effectiveness did not improve in ASD schools may have to do with insufficient organizational support for the school. Researchers have emphasized building a cohesive infrastructure to support school improvement at superordinate levels of governance, such as district or network offices with staff whose sole task is to meet the needs of turnaround schools (Peurach & Neumerski, 2015). However, the ASD was explicitly designed to emphasize school-level autonomy by removing schools from their local districts. Losing district-level infrastructure and support may have added burdens (e.g., unclear budgets and transportation plans) for the school leadership team that could have detracted from their ability to build and maintain consistent school operating routines. Without a focused leadership team and consistent schoolwide operating procedures, teachers may have had more difficulty focusing on classroom instruction, leading to decreases in their instructional effectiveness.

In contrast, local iZones were designed to directly increase district-level support for low-performing schools. This means that a dedicated office within local districts was tasked with ensuring that iZone schools received additional resources, such as instructional support staff and classroom supplies. District iZone offices were also responsible for ensuring that daily operations ran smoothly in iZone schools, ranging from renovating the school building to ensuring that bus schedules ran smoothly. Creating district-level infrastructure to support iZone schools likely lifted administrative burdens from school leaders, allowing them to focus on creating a positive school climate that prioritized effective classroom instruction, which in turn could have helped teachers improve their effectiveness. Prior research supports this hypothesis, finding that iZone reforms led to a more positive learning environment, greater teacher collaboration, and more opportunities for professional learning (Pham, 2022). These advances in the professional environment in iZone schools help potentially explain improvements in teacher effectiveness, suggesting that future reform efforts should invest in helping principals create school environments that are conducive to effective teaching.

This study also adds nuance to existing research on teacher effectiveness across schools by focusing on turnaround schools. The finding that teachers moving into ASD and iZone schools were coming from schools that were themselves low-performing highlights the importance of examining turnaround schools as distinct from other low-performing schools, and, indeed, researchers have found that recruiting effective teachers into turnaround schools resulted in unintended negative effects on low-performing but non-turnaround schools that lost effective teachers to a turnaround school (Kho et al., 2022).

In this paper, I examine the stability of teacher effectiveness in a turnaround context. Understanding teacher effectiveness in turnaround schools is important because a large number of teachers who transferred into low-performing schools were likely moving into turnaround schools in response to recruitment efforts in these schools (Springer et al., 2015). My communications with district leaders suggest that this was the case in the Memphis iZone, where supports for iZone schools (e.g., recruitment bonuses) helped them attract more teachers than did other low-performing schools in the district. I find that teacher effectiveness did change after teachers transferred into turnaround schools in ways that differed from teachers who transferred into low-performing but non-turnaround schools. Thus, ongoing research on teacher effectiveness should consider distinct features of schools undergoing reform and how these features differentiate them from other low-performing schools.

Comparing the iZone and ASD experience also points to complementarities between teacher effectiveness and retention, both of which appear to have helped produce positive results in iZone schools. After iZone schools recruited teachers, this study finds evidence that the schools were able to improve teachers’ effectiveness. Moreover, iZone schools have been shown to have high levels of teacher retention, higher than comparison priority schools and much higher than ASD schools (Henry et al., 2020). Taken together, these findings suggest that the supports teachers received in iZone schools may have had a dual impact on increasing their effectiveness and retention rates. In contrast, the ASD approach focused on teachers’ recruitment without clear strategies to support them after they transferred, leading to some negative effects on teacher effectiveness and significantly lower retention rates relative to comparison schools. Together, the ASD and iZone experience suggests that strategies to develop and retain teachers may need to go hand-in-hand to successfully support school improvement.

Also, the finding that Black teachers experienced greater increases in student achievement after transferring into iZone schools deserves further attention. Because iZone schools serve primarily Black students, this finding aligns with previous research that finds academic benefits for students who are assigned to racially congruent teachers (Joshi et al., 2018). Although an exploration of why Black teachers tended to do better after transferring into iZone schools is beyond the scope of this study, it will be important in future research to further examine the experiences of non-white teachers in turnaround schools serving mainly non-white students.

This study finds convincing evidence that successful school reforms must go beyond recruiting effective teachers by investing in efforts to develop them. However, this paper examines teacher effectiveness based only on student test scores, which could differ from teacher effectiveness on other outcomes. Also, this study leaves open the question of what reform mechanisms help teachers improve. One important route for future research is to examine features of reforms that tend to influence teacher effectiveness. For example, what role does the leader of a turnaround school play in helping teachers adjust after they move so that they do not experience declines in effectiveness? Further illuminating strategies to help build teachers’ capacity will be important next steps in the ongoing effort to support our lowest-performing schools.

Supplemental Material

sj-docx-1-ero-10.1177_23328584221139763 – Supplemental material for Is Teacher Effectiveness Stable Across School Contexts? An Examination of Teachers Who Transfer Into Turnaround Schools

Supplemental material, sj-docx-1-ero-10.1177_23328584221139763 for Is Teacher Effectiveness Stable Across School Contexts? An Examination of Teachers Who Transfer Into Turnaround Schools by Lam D. Pham in AERA Open

Footnotes

Open Practices

Instructions to access data and analysis files for this article can be found at

ORCID iD

Lam D. Pham

Notes

Author

LAM D. PHAM is an assistant professor of educational evaluation and policy analysis at North Carolina State University. His research interests include school improvement and reform, with a focus on teachers and leaders in low-performing schools.

References

Aaronson

Barrow

Sander

(2007). Teachers and student achievement in the Chicago public high schools. Journal of Labor Economics, 25(1), 95–135.

Boyd

Grossman

Ing

Lankford

Loeb

Wyckoff

(2011). The influence of school administrators on teacher retention decisions. American Educational Research Journal, 48(2), 303–333.

Callaway

Sant’Anna

P. H.

(2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225(2), 200–230.

Carlson

Lavertu

(2018). School Improvement Grants in Ohio: Effects on student achievement and school administration. Educational Evaluation and Policy Analysis, 40(3), 287–315. https://doi.org/10.3102/0162373718760218

Carver

C. L.

Feiman-Nemser

(2009). Using policy to improve teacher induction critical elements and missing pieces. Educational Policy, 23(2), 295–328. https://doi.org/10.1177/0895904807310036

Chetty

Friedman

J. N.

Rockoff

J. E.

(2011). The long-term impacts of teachers: Teacher value-added and student outcomes in adulthood. National Bureau of Economic Research. http://www.nber.org/papers/w17699

Clotfelter

C. T.

(2001). Are Whites still fleeing? Racial patterns and enrollment shifts in urban public schools, 1987–1996. Journal of Policy Analysis and Management, 20(2), 199–221. https://doi.org/10.1002/pam.2022

Davis

Higdon

(2008). The effects of mentoring/induction support on beginning teachers’ practices in early elementary classrooms (K–3). Journal of Research in Childhood Education, 22(3), 261–274.

Dixon

L. L.

Pham

L. D.

Henry

G. T.

Corcoran

S. P.

Zimmer

(2021). Who leads turnaround schools? Characteristics of principals in Tennessee’s Achievement School District and Innovation Zones. Educational Administration Quarterly, 58(2). https://doi.org/10.1177/0013161X211055702

10.

Dragoset

Thomas

Herrmann

Deke

James-Burdumy

Graczewski

Boyle

Upton

Tanenbaum

Giffin

(2017). School Improvement Grants: Implementation and effectiveness. NCEE 2017-4013. National Center for Education Evaluation and Regional Assistance.

11.

Feng

Sass

T. R.

(2017). Teacher quality and teacher mobility. Education Finance and Policy, 12(3), 396–418.

12.

Gill

Zimmer

Christman

Blanc

(2007). State takeover, school restructuring, private management, and student achievement in Philadelphia. RAND Corporation.

13.

Glazer

J. L.

Egan

(2018). The ties that bind: Building civic capacity for the Tennessee Achievement School District. American Educational Research Journal, 55(5), 928–964. https://doi.org/10.3102/0002831218763088

14.

Glazerman

Protik

Teh

Bruch

Max

(2013). Transfer incentives for high-performing teachers: Final results from a multisite randomized experiment. Mathematica Policy Research. https://ideas.repec.org/p/mpr/mprres/4269bc8810414c8a8f64d3c36fde8211.html

15.

Goddard

Y. L.

Goddard

R. D.

Tschannen-Moran

(2007). A theoretical and empirical investigation of teacher collaboration for school improvement and student achievement in public elementary schools. Teachers College Record, 109(4), 877–896.

16.

Goodman-Bacon

(2021). Difference-in-differences with variation in treatment timing. Journal of Econometrics, 225(2), 254–277.

17.

Grissom

J. A.

Loeb

Nakashima

N. A.

(2014). Strategic involuntary teacher transfers and teacher performance: Examining equity and efficiency. Journal of Policy Analysis and Management, 33(1), 112–140.

18.

Hanushek

E. A.

Kain

J. F.

Rivkin

S. G.

(2004). Disruption versus Tiebout improvement: The costs and benefits of switching schools. Journal of Public Economics, 88(9), 1721–1746.

19.

Heissel

J. A.

Ladd

H. F.

(2017). School turnaround in North Carolina: A regression discontinuity analysis. Economics of Education Review, 62, 302–320.

20.

Henry

G. T.

Pham

L. D.

Kho

Zimmer

(2020). Peeking into the black box of school turnaround: A formal test of mediators and suppressors. Education Evaluation and Policy Analysis, 40(2), 232–256.

21.

Henry

G. T.

Zimmer

Attridge

Kho

Viano

(2014). Teacher and student migration in and out of Tennessee’s Achievement School District. Tennessee Consortium on Research, Evaluation & Development. https://peabody.vanderbilt.edu/TERA/files/ASD_Teacher_Student_Migration.pdf

22.

Horng

E. L.

(2010). Teacher tradeoffs: Disentangling teachers’ preferences for working conditions and student demographics. American Educational Research Journal, 46(3), 690–717. https://doi.org/10.3102/0002831208329599

23.

Imazeki

(2005). Teacher salaries and teacher attrition. Economics of Education Review, 24(4), 431–449.

24.

Ingersoll

R. M.

Strong

(2011). The impact of induction and mentoring programs for beginning teachers: A critical review of the research. Review of Educational Research, 81(2), 201–233. https://doi.org/10.3102/0034654311403323

25.

Iyengar

Lewis-LaMonica

Perigo

(2017). School district Innovation Zones: A new wave of district-led efforts to improve economic mobility. Bridgespan Group. https://www.bridgespan.org/bridgespan/Images/articles/school-district-innovation-zones/school-district-innovation-zones-a-new-wave-of-district-led-efforts-to-improve-economic-mobility.pdf

26.

Jackson

C. K.

(2013). Match quality, worker productivity, and worker mobility: Direct evidence from teachers. Review of Economics and Statistics, 95(4), 1096–1116.

27.

Jackson

C. K.

Bruegmann

(2009). Teaching students and teaching each other: The importance of peer learning for teachers. American Economic Journal: Applied Economics, 1(4), 85–108.

28.

Johnson

S. M.

Kraft

M. A.

Papay

J. P.

(2012). How context matters in high-need schools: The effects of teachers’ working conditions on their professional satisfaction and their students’ achievement. Teachers College Record, 114(10).

29.

Joshi

Doan

Springer

M. G.

(2018). Student-teacher race congruence: New evidence and insight from Tennessee. AERA Open, 4(4). https://doi.org/10.1177/2332858418817528

30.

Kho

Henry

G. T.

Pham

L. D.

Zimmer

(2022). Spillover effects of recruiting teachers for school turnaround: Evidence from Tennessee. Educational Evaluation and Policy Analysis. https://doi.org/10.3102/01623737221111807

31.

Kraft

M. A.

Papay

J. P.

(2014). Can professional environments in schools promote teacher development? Explaining heterogeneity in returns to teaching experience. Educational Evaluation and Policy Analysis, 36(4), 476–500.

32.

Kraft

Blazar

Hogan

(2016). The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence. Brown University Working Paper.

33.

Ladd

H. F.

(2011). Teachers’ perceptions of their working conditions: How predictive of planned and actual teacher movement? Educational Evaluation and Policy Analysis, 33(2), 235–261. https://doi.org/10.3102/0162373711398128

34.

Lankford

Loeb

Wyckoff

(2002). Teacher sorting and the plight of urban schools: A descriptive analysis. Educational Evaluation and Policy Analysis, 24(1), 37–62.

35.

Loeb

Darling-Hammond

Luczak

(2005). How teaching conditions predict teacher turnover in California schools. Peabody Journal of Education, 80(3), 44–70.

36.

Mas

Moretti

(2009). Peers at work. American Economic Review, 99(1), 112–145.

37.

Meyers

C. V.

Hitt

D. H.

(2017). School turnaround principals: What does initial research literature suggest they are doing to be successful? Journal of Education for Students Placed at Risk (JESPAR), 22(1), 38–56.

38.

Nguyen

T. D.

Pham

L. D.

Crouch

Springer

M. G.

(2020). The correlates of teacher turnover: An updated and expanded meta-analysis of the literature. Educational Research Review, 31, 100355.

39.

Papay

Hannon

(2018). The effects of school turnaround strategies in Massachusetts [Working Paper].

40.

Papay

J. P.

Kraft

M. A.

(2015). Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence on long-term career improvement. Journal of Public Economics, 130, 105–119.

41.

Papay

Kraft

M. A.

James

(2021). Operator versus partner: A case study of blueprint school network’s model for school turnaround [Working Paper].

42.

Patrick

Worthen

Frost

Truong

(2018). Innovation Zones: Creating policy flexibility for competency-based personalized learning. Issue Brief. INACOL.

43.

Peurach

D. J.

Neumerski

C. M.

(2015). Mixing metaphors: Building infrastructure for large scale school turnaround. Journal of Educational Change, 16(4), 379–420.

44.

Pham

L. D.

(2022). Why do we find these effects? An examination of mediating pathways explaining the effects of school turnaround. Journal of Research on Educational Effectiveness. https://doi.org/10.1080/19345747.2022.2081276

45.

Pham

L. D.

Henry

G. T.

Kho

Zimmer

(2020). Sustainability and maturation of school turnaround: A multi-year evaluation of Tennessee’s achievement school district and local innovation zones. AERA Open, 6(2). https://doi.org/10.1177/2332858420922841

46.

Picard

(2012). GEODIST: Stata module to compute geodetic distances.

47.

Redding

Nguyen

T. D.

(2020). The relationship between school turnaround and student outcomes: A meta-analysis. Educational Evaluation and Policy Analysis, 42(4), 493–519.

48.

Rice

J. K.

Malen

(2010). School reconstitution as an education reform strategy. National Education Association. http://199.223.128.58/assets/docs/School_Reconstruction_and_an_Education_Reform_Strategy.pdf

49.

Rivkin

S. G.

Hanushek

E. A.

Kain

J. F.

(2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417–458.

50.

Rothstein

(2014). Revisiting the impacts of teachers. UC-Berkeley Working Paper.

51.

Sanders

W. L.

Rivers

J. C.

(1996). Cumulative and residual effects of teachers on future student academic achievement. http://beteronderwijsnederland.net/files/cumulative%20and%20residual%20effects%20of%20teachers.pdf

52.

Scafidi

Sjoquist

D. L.

Stinebrickner

T. R.

(2007). Race, poverty, and teacher mobility. Economics of Education Review, 26(2), 145–159. https://doi.org/10.1016/j.econedurev.2005.08.006

53.

Schueler

B. E.

Asher

C. A.

Larned

K. E.

Mehrotra

Pollard

(2021). Improving low-performing schools: A meta-analysis of impact evaluation studies. American Educational Research Journal, 59(5). https://doi.org/10.3102/00028312211060855

54.

Schueler

B. E.

Goodman

J. S.

Deming

D. J.

(2017). Can states take over and turn around school districts? Evidence from Lawrence, Massachusetts. Educational Evaluation and Policy Analysis, 39(2), 311–332.

55.

Springer

M. G.

Swain

W. A.

Rodriguez

L. A.

(2015). Effective teacher retention bonuses: Evidence from Tennessee. Educational Evaluation and Policy Analysis, 38(2), 199–221. https://doi.org/10.3102/0162373715609687.

56.

Strunk

K. O.

Marsh

J. A.

Hashim

A. K.

Bush-Mecenas

(2016). Innovation and a return to the status quo: A mixed-methods study of school reconstitution. Educational Evaluation and Policy Analysis, 38(3), 549–577.

57.

Viano

Pham

L. D.

Henry

G. T.

Kho

Zimmer

(2020). What teachers want: School factors predicting teachers’ decisions to work in low-performing schools. American Educational Research Journal, 58(1). https://doi.org/10.3102/0002831220930199

58.

Weber

Péclat

(2017). A simple command to calculate travel distance and travel time. Stata Journal, 17(4), 962–971.

59.

Özek

Corritore

(2012). Portability of teacher effectiveness across school settings. Working Paper 77. National Center for Analysis of Longitudinal Data in Education Research.

60.

Özek

Hansen

(2015). Teacher performance trajectories in high-and lower-poverty schools. Educational Evaluation and Policy Analysis, 37(4), 458–477.

61.

Zimmer

Henry

G. T.

Kho

(2017). The effects of school turnaround in Tennessee’s Achievement School District and Innovation Zones. Educational Evaluation and Policy Analysis, 39(4), 670–696.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB