Abstract
For decades, charter schools have been promoted as a panacea for increasing competition in the educational marketplace. Supporters argue that increased choice forces neighboring schools to innovate, while opponents contend that charters “skim” students and funds away from traditional public schools (TPS). We test the two differing views by comparing academic achievement and school segregation in TPS in South Florida facing competition from charter schools compared to TPS with no competition. We find that when a charter school moves into the community, it fails to substantively change test scores or diversity of the nearby TPS, even 10 years after a charter is established.
Charter school supporters have long argued that with competition from charter schools, traditional school performance and racial diversity will increase. These supporters argue that the introduction of charter schools into the local educational marketplace encourages traditional public schools (TPS) to become more accountable to students and parents by improving school performance (Chubb & Moe, 1990; Mintrom & Vergari, 1997; Vergari, 2007). Traditional schools that consistently underperform will lose students and be threatened with closure. In part, supporters argue, TPS improved performance comes as innovations launched by charter schools are adopted in public schools (Dye, 1998; Zimmer et al., 2009). This traditional school improvement position is widely used by proponents and is even included in Florida education statute which states that “rigorous competition within the public-school district (will) stimulate continual improvement in all public schools.” 1 Proponents also argue that charters can give options to poor and minority students in urban areas who often attend segregated and low-performing schools, thus reducing segregation and achievement gaps (Bifulco & Ladd, 2006; Riel et al., 2018).
Charter school opponents argue that instead of improvement, a competing charter school harms traditional schools. This argument reasons that a new charter school may “skim” the diligent students (or, more likely, students with diligent parents) and leave the traditional schools with students whose parents are less engaged and who may struggle in any school (Hoxby, 2000; West et al., 2006). Specifically, the concern is that charters will attract more middle-class and higher performing students, leaving behind poor, minority, and lower-ability students in traditional schools (Fabricant & Fine, 2012; Garcia, 2008). Alternatively, a new charter school may attract many different students but “crop out” certain expensive but vital programs that serve disadvantaged students (Bergman & McFarlin, 2018; Lacireno-Paquet et al., 2002; Lubienski et al., 2009). Under cropping, traditional schools are left with students who are viewed as too resource intensive for the charter school even though these students, and families, may wish to enroll in in the charter program. Thus, opponents argue charter schools will attract more middle-class and higher performing students, leaving behind poorer and lower-ability students—resulting in schools that are more racially segregated (Fabricant & Fine, 2012; Garcia, 2008; Lacireno-Paquet et al., 2002; Miron et al., 2010).
Empirical findings of the two hypotheses are mixed (Bettinger, 2005; Bifulco & Ladd, 2007; Booker et al., 2007; Zimmer & Buddin, 2009). One reason is that there is enormous variation in state laws regulating charter schools, thus making interstate comparisons difficult. However, there is also considerable within-state variation since charter schools differ in their focus and ownership. The diversity of charter schools makes aggregate analysis suspect. On the other hand, research that focuses on only a single county or district can also be misleading since local contexts and policies may be determining the results (Mann & Bruno, 2022). Additionally, studies also vary in the level of analysis, with many focusing on student-level outcomes. Yet, claims about the effects of introducing charter schools near traditional public schools are framed at the school-level.
This paper addresses these conflicting hypotheses and adds to the literature on the impact of charter schools on the performance and diversity of traditional schools with a novel research design. We collect an original dataset of traditional public schools that faced competition versus those that did not to examine the impact on traditional schools when a charter school moves into the neighborhood. We use a difference-in-differences approach to best examine this effect, comparing traditional public schools facing competition from charters against public schools which do not. In addition, we look at the impact over time on these specific schools—an approach not utilized by other studies. Finally, even though we are digging down to the school level over time, our sample is quite large and diverse in an urban area of a state (Florida) which has been an enthusiastic supporter of charter schools.
We fail to find support for either side of the charter school divide. In short, when a charter school moves into the neighborhood of a traditional school, there is little to no effect on the performance or diversity of the traditional school in either direction. Further, this non-effect is persistent many years after the charter school has been established. Consistent with a growing body of evidence, our findings suggest that both the competition arguments for improving traditional schools’ performance and diversity and the counter arguments that competition will harm traditional schools’ performance and diversity are off the mark. The discussion might be better aimed at how to make both charter and traditional schools improve their educational programs, perhaps, as indicated by Mann and Bruno (2022) by increasing financial assistance to traditional schools experiencing enrollment losses to charter schools.
The paper proceeds as follows. First, we outline what informs this study—particularly research on the impact of charter schools. We then set forth the hypotheses, research design, and findings. We conclude with implications of this work for policymakers and practitioners.
Charter School Impact on Traditional Schools
Charter Schools and Student Achievement
Tiebout (1956) posited the notion of an educational marketplace. He argued that a system of local governments creates a “market” for local public goods in which households choose to locate based on those goods. Members of those households move to areas with desired services by “voting with their feet” (Corcoran, 2014). Tiebout sorting has often been applied to education policy where homeowners seek to buy housing in areas with the “best” schools for their children. Hoxby (2000) argues that the Tiebout process is the “most powerful force in American schooling” (p. 1209). Charter schools offer a less intrusive test of the Tiebout sorting model since parents can move their children (but not their homes) and without the additional cost of private school.
The normative argument that charter schools will improve traditional schools can be traced to Friedman (1955) who argued that competition will improve the efficiency and effectiveness of traditional schools. John Chubb and Terry Moe, in a highly influential book in 1990, argued that the failures of public-school education systems can be attributed to a lack of competition in the educational marketplace. With competition, the argument goes, the overall quality of education will improve. Charter schools are one of the most common policy solutions designed to promote the competition espoused by Chubb, Moe, and Friedman. Subsequent technical and academic evaluations of the charter school movement have identified this connection (Buckley & Schneider, 2007; Mintrom & Vergari, 1997).
Opponents of charter schools argue that charter schools siphon off or skim good students from traditional schools (Fabricant & Fine, 2012). Information is key to choosing a nontraditional school, and some parents are better able to access and use the information than others. They might be better informed as to the availability of charter schools, have the ability to successfully complete the application process which can include teacher recommendations, commitments to parental involvement, and provide transportation to and from the charter school (Frankenberg et al., 2010). Not only will traditional schools lose some of their best students through skimming, they might also be left with high-need students who were cropped by the charter schools. Financially, traditional schools may lose critical funds as students depart and enrollment numbers decline (Ozek et al., 2018). Although traditional schools are less likely to fail than charter schools, they are closed due to declining enrollment or more recently where schools have persistently low student performance (Bifulco & Schwegman, 2020).
Findings related to effects of charter schools on public schools’ performance have been decidedly mixed. Booker et al. (2007) found support for the argument that traditional school performance improved near charter schools. Two studies in Texas found a modest overall performance improvement for students enrolled in traditional public schools (Bohte, 2004; Booker et al., 2008). Holmes et al. (2003) found small traditional school achievement gains with charter school competition in North Carolina—as did Hoxby (2002) in Michigan and Ridley and Terrier (2018) in Massachusetts. In Florida, Winters (2012) finds that, as traditional schools lose more students to charter schools, they experience mild increases in math and English test scores. In New York, Cordes (2018) likewise finds that charter schools increase math and English scores in traditional schools—and this relationship heightens among more proximal charter and traditional schools. In a meta-analysis of school choice and competition, Jabbar et al. (2022) find a small positive effect of competition on student achievement.
Many studies have found neither improvement nor negative effects. Zimmer and Buddin (2009) in a study of California traditional and charter schools found no improvement. Bettinger (2005) similarly compared charter schools with traditional schools in Michigan and found no effect on test scores of traditional schools located withing 5 miles of a charter. Hoxby (2000) found that Tiebout choice had little effect on average level of achievement in a national study of private school competition.
Other studies report negative effects. Using an instrumental variable approach, Imberman (2011) finds that nearby charter schools induce modest drops in English and math test scores among traditional school students—especially those in elementary school.
Student Selection Into Charter Schools
Several studies looked specifically at skimming—also termed cream-skimming—where the highest performing students leave traditional schools for charter schools (West 2006). Cowen and Winters (2013) in a study of Florida students found no evidence of skimming—in fact the better a student was doing in math, the less likely she was to move to a charter school the following year (there was no relationship between reading scores and propensity to move). Finn et al. (2000), in an early study of charter schools, also found little support for skimming. Similarly, Buckley and Schneider (2007) found that charter school parents in the District of Columbia were similar in socioeconomic status and education to traditional school parents. On the other hand, Slungaard Mumma (2019) finds that charter school openings in North Carolina reduced the enrollment among nearby traditional schools by 5%. Barseghyan et al. (2019) develop a theoretical model and find that when peer preferences are strong, school competition can reduce the quality of the public school system in several ways. This is because choice can reduce schools’ incentives to focus on improvement, particularly in the more desired school, while also exacerbating educational inequalities through motivating skimming.
Other studies examine cropping—where certain students are selected out, excluded, or disincentivized from enrolling into charter schools because they use specialized services (like special education programs and English language assistance), also because they are associated with moderate to lower educational outcomes. Cropping is distinct from skimming, where specialized services like gifted programs are provided to entice high achievers to enroll. Lacireno-Paquet et al. (2002) found that rather than skimming the top students, market-oriented charter schools were more likely to be “cropping off” services that cater to English Language Learners and those with special education needs. In an analysis of market-oriented schools in both England and the United States, West et al. (2006) conclude that both skimming and cropping appear to occur when choice and selection are involved. Several studies have found that charter schools are significantly less likely to serve students with disabilities relative to traditional schools (Dudley-Marling & Baker, 2012; Frankenberg et al., 2010; Jessen, 2013; Miron et al., 2010). Bergman and McFarlin (2018) conducted an audit experiment of over 6,000 public schools. The authors found that charter schools were significantly less likely to respond to parents who signaled they had a special needs child relative to traditional schools.
Charter Schools and Racial Segregation
Regarding racial and ethnic segregation, the results are fairly consistent. Prior studies find that charter school students are more racially isolated than traditional school students—particularly for Black or minority students (Bifulco & Ladd, 2007; Cobb & Glass, 1999; Frankenberg & Lee, 2003; Frankenberg et al., 2010; Garcia, 2008; Ladd & Fiske, 2000). Cowen and Winters (2013) in a Florida study, found that minority students were more likely to move to charters with more minorities and Whites were more likely to move to charters with more Whites than their traditional public schools. This increased segregation is important since research has linked it to students’ academic performance (Borman et al., 2004; Hanushek et al., 2009; Mickelson, 2015; Mickelson et al., 2013; Mickelson & Bottia, 2009). For example, Mickelson et al. (2013) in a meta-analysis find a negative association between increasing student racial isolation and aptitude in mathematics. Given the increasing re-segregation of the nation’s schools in general, this linkage is particularly relevant (Orfield et al., 2019). This does not mean that racial/ethnic segregation is the only type of sorting that may occur after the introduction of charter schools. 2 However, given the salience of race in subnational policymaking (see e.g., Maltby, 2017) it remains the focus of our study.
Why do we observe a large variety of outcomes when examining the impact of charter schools? One reason is research design; prior studies utilize varying research approaches and units of analysis. Most studies simply compare charter schools to traditional schools operating in the same neighborhood (Cobb & Glass, 1999; Kamienski, 2011). Others follow students who leave charter schools which is important but does not address the question of the impact of the opening of a charter school on neighboring traditional schools (Bilfulco & Ladd, 2006; Cowen & Winters, 2013; Garcia, 2008). Some use surveys of principals and teachers (Zimmer & Buddin, 2009). Still others measure charter school introduction through the number of charter schools in a district (Zhang & Yang, 2008).
Finally, there is the possible impact of time. Charter schools that are open longer may have an advantage in attracting and retaining students and may have a reputation that encourages more students to apply (Fabricant & Fine, 2012; Riel et al., 2018). In one of the few studies to look over time (a 5-year period), Cowen and Winters (2013) found students were more likely to move to charter schools in the years after they initially open. More mature charter schools may also have more accomplished teaching and administration leading to higher test scores for the students (Sass, 2006).
In sum, the findings regarding the impact of charter schools are mixed concerning the academic effects on traditional schools but fairly consistent over racial segregation effects. Effects over time are important but not widely studied. However, the research does not address specifically what happens to a traditional school when a charter school moves in the neighborhood in performance and diversity, especially if this effect changes over time.
Hypotheses
Given the competing expectations regarding the impact of charter schools on neighboring traditional schools, we will test two hypotheses that flow from the normative literature posing different outcomes. Our goal is to carefully measure the impact of charter schools on traditional schools by evaluating traditional schools against themselves, that is, their performance before and after the opening of a neighboring charter school. One arm of that research argues that competition from the new charter school will increase the performance of traditional schools; another argues the opposite—that charter schools will skim-off desirable students from traditional schools, leading to a fall in their performance. We list these below as separate hypotheses:
H1a COMPETITON: Traditional schools will see their test scores improve after a charter school moves into their neighborhood.
H1b SKIMMING/CROPPING: Traditional schools will see their test scores fall after a charter school moves into their neighborhood.
The literature on the impact of charter schools on race and ethnicity is more consistent, but generally compares the racial composition of charter and traditional schools rather than looking at the impact of a charter school on a traditional school (Bifulco & Ladd, 2007; Cobb & Glass, 1999; Frankenberg et al., 2010; Frankenberg & Lee, 2003). They find that charter schools have increased racial isolation of both Black and White students—Black students gravitate toward black charter schools and White students toward white charter schools (Bifulco & Ladd, 2007). Studies that trace student movement such as Garcia (2008) find that charter elementary school choosers enroll in charter schools that are more racially segregated than the district schools they exited, again leading to more racially isolated charter schools. But what happens to the traditional schools left behind? In one sense, they will become less racially isolated and more racially equally distributed, but this depends on whether the neighborhood charter school is predominantly White or Black. We are unsure at this point and therefore will not predict a direction for the racial makeup of traditional schools.
H2: Traditional schools will see their racial and ethnic segregation change when a charter school moves into their neighborhood.
The final hypothesis deals with time. Hoxby (2000) argues that school choice reforms may take a decade or more to have positive effects. She points out that until many students experience an increased degree of choice, reforms are unlikely to affect public schools, either through competition pressure or through sorting. Some research has indicated that there is a lag in the performance of charter school students, perhaps after the novelty of the new schools fade. For example, Sass (2006) found that the performance of charter school students was initially lower in charters but by their fifth year of operation had reached a par with the average traditional public school in math and higher reading scores than their traditional public school counterparts. With our research focus based on the effect of charter schools on nearby traditional schools, we do not have evidence about the directional effect of time but think that the impact on both racial and ethnic diversity and performance might change as the charter school matures.
Therefore, we hypothesize:
H3: The impact of charter schools on traditional schools’ racial and ethnic diversity and performance will change over time.
Research Design
We looked at elementary schools in three urban counties in Florida: Broward, Miami-Dade, and Palm Beach from the 2000 through the 2016 school year. We chose to look at elementary schools for several reasons. First, in Florida and other states, most charter schools serve elementary students (Kamienski, 2011). Florida is home to 544 charter elementary schools, compared to 217 charter high schools (Public School Review, 2022). Second, parents are more likely to be involved in picking their child’s elementary school than their high school, since primary school is the most formative time in a child’s education (Vaughn & Witko, 2013). Finally, high school choice is often influenced by factors such as graduation rates and sports programs. These factors are not central to elementary school choice, and would therefore warrant a separate analysis (Hastings et al., 2005).
We chose Florida for several reasons. First, Florida state law encourages the formation of charter schools and the state’s charter school enrollment exceeds the national average. Nationally, some 6% of public school students are in charter schools; in Florida it is 10%. Second, in Florida, as in other states, charter schools are generally located in urban areas and Florida has several urban counties/districts with substantial penetration from charter schools. For example, in Miami-Dade, 35% of all public schools are charters; in Broward, the market share is 30%. Figure 1 shows how the elementary charter school enrollment in the three-county South Florida area compares to the rest of the state from 2000 to 2016. The prevalence of charter schools and the large student population in these three counties provides us a large sample to study. Third, Florida schools are highly racially/ethnically diverse. Black students make up 22% of Florida public school students compared to 15% nationally. Hispanic students are 31% of Florida public school students, compared to 25% nationally (Orfield & Ee, 2017).
Percentage of students in Charter schools by year for Broward, Dade, Palm Beach, and the rest of Florida from 2000 to 2016.
Our level of analysis is the school, not individual students. We made this choice given that our hypotheses are that traditional schools are affected by the opening of a charter school. Many studies have examined the impact of charter schools on students, and we report some of this work. Betts and Tang (2019) and Zimmer and Buddin (2009) argue that using school-level data to estimate competitive effects may not be able to capture the nonrandom movement of students from traditional schools to charter schools. For example, traditional school scores might rise because low-performing students disproportionately leave traditional schools for charters. However, it is this nonrandom movement that is important in understanding whether traditional schools are negatively or positively affected by the opening of a neighboring charter school. Thus, we argue schools are the appropriate level of analysis for this study.
Finally, we deal with the impact of the opening of a charter school nearby traditional schools over time. We have data on the traditional schools’ performance and racial composition before and after the opening of a charter school over a 17-year period. We are then able to assess the immediate and longer-term impact of competition from charter schools.
Key to this study is the definition of neighborhood; in other words, what is the distance from a traditional school that a charter school would be considered as competing with it. Studies examining the location of charter schools have used a variety of distances—from 1 to 20 miles (Bettinger, 2005; Koller & Welsch, 2017; Lubienski et al., 2009). Clearly the population density affects the distance that makes sense, with higher density areas offering more schools within an equivalent geographic area. Figure 2 shows the density of schools by school type in our three-county area. We chose 1 mile as the distance that would best lead to competition. In supplemental evaluations we assess the sensitivity of this decision by evaluating the influence of charter schools using a 0.5-mile and 2-mile buffer. Our findings are consistent across specifications.
The distribution of elementary schools across South Florida.
The analytical approach used here is a difference-in-differences. Intuitively, the difference-in-differences estimate represents the differential change in school performance/diversity from the time prior to the establishment of a charter school (the pre-treatment period) to after the establishment of a charter school (the post-treatment period) for a school located both close to and far away from charter schools. The difference-in-differences strategy better overcomes threats of bias from time-invariant factors—both observed and unobserved (Angrist & Pischke, 2008). The difference-in-differences design is an increasingly common approach leveraged to evaluate policy outcomes. The approach has been used to evaluate the effects of fiscal policy (Mughan, 2019), social policy (Jung et al., 2018), and—most critically for our study—education policy. Previous education policy differences-in-differences designs have evaluated the effects of school vouchers (Carlson et al., 2014) and deregulation of school management (Wang & Yeung, 2019).
To conduct a difference-in-differences design, we must do two things: (1) establish a treatment and control group within the dataset, and (2) determine a pre-treatment and post-treatment period of analysis within the data.
To accomplish the first task, we utilized the GIS software ArcMap to assign traditional public schools to nearby charter schools. All schools report their location—both as a street address and as a latitude and longitude—to the Department of Education. Latitude and longitudinal coordinates were evaluated for accuracy by the authors, and then projected into geographic space. Figure 2 illustrates the distribution of elementary schools in our sample across the three county South Florida metro area. We use dots to represent traditional public schools and “school symbol” to represent charter schools. We drew a 1-mile buffer around each charter school within our dataset. Schools that fell within the buffer were assigned as the treated schools in the analysis. Conversely, schools that are located outside of the buffer were assigned as the control schools within our analysis.
Establishing a pre-treatment versus post-treatment time period for our data is somewhat more challenging. First, the legislation establishing charter schools within Florida pre-dates No Child Left Behind and the reporting of school performance standards. Second—and of greater concern—the establishment of charter schools in South Florida was piecemeal. Charter schools established themselves gradually in the community, starting in the late 1990s and continuing through 2016—the most recent year of data in our analysis. As our unit of interest is at the school-level, we determined that the year of a charter school’s establishment is the most appropriate time at which to assign a change from the pre-treatment to post-treatment period.
Determining the pre-treatment versus post-treatment period of analysis for our treated schools is relatively simple. The year that a charter school opens determines when the traditional public school moves from the pre to the post-treatment period within our data. If a treated school is within 1 mile of multiple charter schools, we use the earliest established charter school to assign the shift from pre- to post-treatment. 3
To determine when our control schools move from the pre to the post-treatment period we rely on the following procedure. For schools more than 1 mile away from a charter, we use ArcMap to measure the nearest charter school to each untreated school. We then use the year the nearest charter school was established to determine when our control group schools migrate from the pre to the post-treatment time-period.
Data Sources and Variables
Supplemental Appendix Table A1 lists data sources and Supplemental Appendix Table A2 provides descriptive information about the variables used in the analysis. The dependent variables in our models reflect both performance and diversity measures.
The performance dependent variables are the school’s average test scores for four subjects included in the Florida Comprehensive Assessment Test: reading, math, science, and writing. The reading and math portions were administered yearly to all students starting in grade 3, writing portions in grade 4, and students in grades 5 took the science portion. 4 While math and reading scores are the most common performance indicators used in research, we also examine writing and science scores as a validity check. The variable measures the percentage of students who received a satisfactory or higher grade on the exam (a score of 3, 4, or 5 on a 1–5 scale). The writing and science scores were averages for only one grade (4th and 5th, respectively); the math and reading scores were averaged across all classes taking the test.
The diversity dependent variables reflect the mean percentage of Hispanic and Black students in each school from the National Center for Education Statistics (NCES), the primary federal organization for collecting and analyzing education data. School-level variables were obtained through the Elementary and Secondary Information Systems (ELSI). Table 1 provides a breakdown of racial/ethnic diversity for each of the three districts in our sample. Miami/Dade students are heavily Hispanic (59%), Broward County students are predominantly Black (42%), and around one third of Palm Beach Students are White and one-third are Black. Table 1 also illustrates the differences in racial/ethnic makeup between charters and traditional schools, especially evident in the Miami/Dade school district.
Racial/Ethnic Composition of Districts by Elementary School Type.
Important to our analysis is when the charter school opened, information that was obtained from the Master School Identification file from the Florida Department of Education (FLDOE). Finally, the latitude/longitude, key in assessing when a charter school moves into a neighborhood, was also obtained from the Master School Identification (MSID) file at FLDOE.
Analysis
One potential concern of the analysis is that charter school placement is driven by the dependent variables in our analysis, particularly with respect to school performance. While implementing legislation might have set expectations that charter schools would locate in poorly performing communities—evaluations of charter school placement find that this goal may not be met. Instead, research has found that, at the school-level, there are several economic and political factors which may drive charter school placement. These findings hold both in Florida-centered research (Weissert et al., 2020) and broader charter school studies (Bifulco & Buerger, 2015; Ferreyra & Kosenok, 2015; Henig & MacDonald 2002; Lubienski et al., 2009; Stoddard & Corcoran, 2007).
Potential confounders—variables that influence both the placement of charter schools and our dependent variables—could still introduce bias. While we cannot randomly assign charter school placement, we can attempt to address potential confounders and further guard against bias using matching. Matching is by no means a panacea, but it does allow us at a minimum to balance several important covariates of interest.
We leverage the cem (coarsened exact matching) package in Stata (Blackwell et al., 2009). To achieve a balance between treatment and control schools we employ coarsened matching on the following covariates that can be plausibly assigned pre-treatment and unrelated to the treatment: the opening of a nearby charter school, the percent of school on free and reduced-price lunch (low, medium, and high), and school size (small, medium, and large). We also include exact match on school district (Broward, Miami-Dade, and Palm Beach).
Functionally, this means that we are comparing a large elementary school, with a low percentage of reduced-price lunch students in Broward that is facing competition from a charter school to a large elementary school, with a low percentage of reduced-price lunch students in Broward that does not face competition from a charter school. Of the 7,406 school-years included in the sample, a total of 7,146 are included in the matched data—2,630 in the treatment group and 4,516 in the control.
After matching on covariates, we perform balance tests to evaluate if our data violate the parallel trends assumption. Functionally, the difference-in-differences design relies on the underlying assumption that each dependent variable in our analysis is not meaningfully different between our treatment and control schools in the pre-treatment period, controlling for covariates (Wang & Yeung 2019; Yang, 2019). The balance test evaluates whether there is a significant difference between the treatment and the control schools in the period prior to the charter school entering the neighborhood. We report balance tests across our six dependent variables and find no suggestive evidence of violating of the parallel trends assumption. Across tests we include measures of school racial composition—where appropriate, as well as year fixed effects. Throughout models, standard errors are clustered at the school-level.
Examining the balance tests for the racial make-up of schools we find no statistical, or substantive, differences between control and treatment schools. Our models report that treatment schools were, on average, 2.4% less Black and 1.5% more Hispanic than control schools in the pre-treatment period. Both variables are far from traditional levels of statistical significance—and are within one standard error of the point estimate (see Table 2 below and Supplemental Appendix Tables A3 and A4).
Balance Tests.
Standard errors reported in parentheses.
School performance is likewise reflected in Table 2. Our school performance variable is measured as the percentage of students in the school that score a satisfactory or higher grade across the various subject specific standardized tests. Again, we fail to find statistically or substantively significant differences between treatment or control schools in these models. Compared to control schools, on average, treatment schools have 0.4% more of their students pass the math standardized tests; 0.2% more of their students pass the reading standardized test; 0.5% fewer students pass the writing standardized test; and have 1.5% fewer students pass the science standardized test. Across performance measures, the coefficient for the treatment group is within one standard error of the mean. The results across balance tests suggest that we are not violating the parallel trends assumption. In Supplemental Appendix Tables A6 and A8 we report balance tests and a replication of the study without using matching. The findings of both balance tests and the difference-in-differences are generally consistent with the findings reported in the main body of the paper.
Results
We now turn to evaluating the difference-in-differences. Across tables, the main variable we are evaluating against the hypotheses is the interaction term Treatment Group × Post Charter. This coefficient substantively represents the effect of being a traditional public school near a charter school, after the establishment of the charter school. These coefficients are evaluated against the baseline category of traditional public schools located away from a charter school prior to the charter school’s establishment. The constituent terms represent the effect of being a traditional public school near a charter school prior to the charter school’s establishment (Treatment Group), and the effect of being a traditional public school located away from a charter school, after the charter school’s establishment (Post Charter). The same suite of covariates that were included in balance tests are included in the results tables. Standard errors are clustered by school.
Our results for school evaluation do not support the competition hypothesis articulated by Chubb and Moe (1990). At best, charter schools have no meaningful impact on traditional public school performance and, potentially, at worst, might decrease traditional public school performance. We find that, on average, a treatment school has 1.8% fewer students pass the math standardized test after the establishment of a charter school. This result is significant at the 0.1 level. Our models likewise report negative coefficients for the difference-in-difference effects for reading and science standardized tests—although results fail to reach conventional levels of significance. On average, treatment schools have 1.5% fewer students pass the reading standardized test and 0.2% fewer students pass the science standardized test after a charter school opens. The results for the writing standardized tests are essentially zero—on average, a treatment school has 0.01% increase in the number of students who pass the writing standardized test after a charter school is established. These results directly contradict Hypothesis 1a which argued that increased competition would improve school performance. Furthermore, at best we find very tentative support for hypothesis 2 regarding skimming. We find that math scores modestly decreased after charter schools entered the educational marketplace (Table 3).
Difference-in-Differences—School Performance.
Note. Two-tailed tests. Standard errors clustered on schools.
p < .10. *p < .05.
Similarly, we find no evidence that charter schools influenced the racial or ethnic makeup of traditional public schools in South Florida (see Table 4). Traditional public schools were, on average, 0.4% more Black and 1.1% less Hispanic after a charter school opened near them. The results are substantively trivial and far from traditional levels of statistical significance. These results fail to find support for Hypothesis 2: there was no real difference in the racial/ethnic makeup of traditional schools when a charter school moves to the neighborhood. It is worth noting that the standard deviation for both these variables is over 30 (see Supplemental Appendix Table A2), which indicates that the effect of charter schools is substantively trivial.
Difference-in-Differences—Racial Diversity.
Note. Two-tailed tests. Standard errors clustered on schools.
p < .05.
Overall, our models do a good job of explaining the determinants of school diversity—explaining approximately 54% of all variation in our data—as well as variation in school performance—explaining anywhere from 47% to 63% of the variation in our data. In sum, these results suggest that—at least at the school-level—charter schools have a limited impact on measures of racial/ethnic diversity or performance for traditional public schools.
Over-Time Effects
Previous literature has identified that pooled difference-in-differences evaluations prevent scholars from identifying delays in treatment effects (Blom-Hansen et al., 2016; Mughan, 2019). We address this potential limitation in Figures 3 and 4, reporting the year-over-year influence of charter school establishment across models. Figure 3 reports changes in school achievement over-time; Figure 4 reports the over-time effects of racial and ethnic school enrollment. The y-axes in the figures can be interpreted as percent change in each dependent variable. The x-axes represent the over-time trend—where the left-hand most coefficient represents the estimated influence of charter schools 1 year after establishment and the right-hand most coefficient represents the influence of charter schools a decade after establishment. Each coefficient reported in between represents an additional 1-year shift in evaluation. Tabular results are reported in Supplemental Appendix Table A5.
Overtime influence of Charter competition on school performance.
Overtime influence of Charter competition on school diversity.
Examining the school performance models—across 10 years of analysis and four subject areas—we fail to find evidence of a single instance where charter school establishment has significantly improved nearby traditional school performance. 5 This is additional evidence arguing against the hypothesis espoused in Chubb and Moe (1990). We find no evidence of a meaningful change in performance for either writing or science scores.
We do find evidence of charter school performance influencing math and reading scores, but in the opposite direction the competition hypothesis would predict (see Figure 4). From the second through the fourth year after a charter school is established, nearby traditional schools have approximately 2.5% to 3% fewer students who achieve satisfactory math scores on state standardized tests. After the fourth year, math scores again become indistinguishable from the pretreatment period at the .05 level. Yet even a decade after a charter school’s establishment, these schools do not have significantly higher levels of student achievement. Changes in reading scores are more muted only reporting marginal significance at the .1 level. In the first and second year after a charter school is established, nearby traditional public schools have approximately 2% fewer students who receive satisfactory scores on state standardized tests. In subsequent years reading scores again become indistinguishable from reading scores in the pre-treatment period. Again, we find no evidence of reading scores ever improving when evaluated against the pre-treatment period.
We also fail to find any evidence of a charter schools significantly affecting school diversity over-time. As illustrated in Figure 4, for both Black and Hispanic enrollment, the establishment of a charter school does not meaningfully impact the enrollment of nearby traditional schools—as the parameter estimates remain far from traditional thresholds of statistical significance.
Our results find little evidence that the establishment of charter schools in South Florida has meaningfully influenced traditional public school performance. The introduction of charter schools in South Florida did not induce nearby traditional schools to improve their test scores. We also find limited support that charter schools skim students, thus lowering the performance of traditional schools. We also failed to find support for the argument that charter effects on traditional schools’ performance change over time. Traditional schools located near charters experienced a slight decrease in math standardized test achievement 2 to 4 years after a charter school’s establishment. We find no evidence of charter schools establishment influencing other subjects evaluated through standardized testing.
Additionally, we find that the introduction of charter schools into the educational marketplace does not meaningfully influence racial/ethnic diversity within traditional public schools—even a decade after establishment. We must be cautious, however, in interpreting this result. South Florida is one of the most racially and ethnically diverse communities in the nation and the lack of change in school enrollment patterns may be due to the majority-minority status of the community. Future scholarship should expand this research design to other communities—particularly with a more entrenched history of segregation—and expand on the generalizability of this finding.
Conclusion
For scholars, these findings promote development of an important research agenda in education policy. Over the previous 30 years, evaluation of education policy has revolved heavily around a framework of competition (Chubb & Moe, 1990). This research agenda has provided some valuable insights permeating across public administration (Rabovsky, 2011), public policy (Fleming, 2014), political science (Moe, 2009), economics (Epple & Ramano, 1998), and sociology (Berends, 2015). While the framework of competition increasing educational performance is theoretically appealing, we find no evidence of it occurring in our Florida sample. Even over time, we find negligible impact to traditional schools as charter schools mature. While there is the possibility that Florida’s results are not generalizable, there is perhaps a greater possibility that earlier work has overstated the impact of charter schools on traditional schools by not examining the effects on traditional schools over time and not sufficiently accounting for alternative explanations through their research designs—a problem we address with a difference-in-differences approach.
For practitioners, the results are also important. We confirm what previous research has suggested—that charter schools do not adversely affect traditional school racial/ethnic diversity and performance. What does this mean for educators, parents, students, and policymakers? Perhaps instead of arguing that charter schools are skimming the best students and leaving traditional schools with low performing students, they should instead concentrate on improving both traditional and charter schools. Competition may work as theory but in fact, competition has had negligible impact on the performance and diversity of traditional schools initially and over time in South Florida.
The long-standing rhetoric about charter schools’ effect on student achievement and diversity should be shelved in the case of South Florida. However, we are not suggesting that charter schools have no impact on the local education marketplace. The introduction of charter schools still warrants continued research in several areas, including funding, programmatic considerations, individual-level student achievement, accountability, school closure rates, and wealth segregation.
Finally, we think that our analysis can be viewed as instructive outside of education policy. Competition in the policy marketplace has been a persistent and impactful component of research for decades. Our analysis questions this impact in a policy area where competition was expressly applied and argued by advocates and scholars alike. Perhaps the boundaries of competition need to be further explored and in areas outside of education policy.
Supplemental Material
sj-docx-1-epx-10.1177_08959048221142049 – Supplemental material for The Boundaries of Competition: Examining Charter Schools’ Impact on Traditional Schools
Supplemental material, sj-docx-1-epx-10.1177_08959048221142049 for The Boundaries of Competition: Examining Charter Schools’ Impact on Traditional Schools by Matthew J. Uttermark, Kenneth R. Mackie, Carol S. Weissert and Alexandra Artiles in Educational Policy
Footnotes
Acknowledgements
We wish to thank Anna Grace Lewis, a graduate of Florida State University, for her able and valuable research assistance on this paper. Previous versions of this paper were presented at the 2020 American Political Science Association Annual Conference and the 2021 Wright Symposium at the American Society for Public Administration’s Annual Conference.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
Notes
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
