Toward a Rich Data Future for School Finance Research

Abstract

This special topic is intended to advance research in school finance using two newly produced compilations of multiple longitudinal data systems, the School Finance Indicators Database (http://schoolfinancedata.org/) and the Urban Institute, Education Data Explorer (https://educationdata.urban.org/data-explorer/). This special topic contains two articles taking advantage of comprehensive longitudinal data on school finance. The first, by Victoria Sosina and Ericka Weathers, uses panel data from the School Finance Indicators Data System from 1999 to 2013 to evaluate whether and to what extent changes to Black-White and Latinx-White demographic differences among districts leads to greater resource disparity over time. The second article, by Knight and Mendoza, combines data from the Census Fiscal Survey with data from the California Department of Education to explore whether differences in data on and measures of school funding equity matter (i.e., lead to similar or different conclusions) when evaluating the effects of California’s 2013 adoption of the Local Control Funding Formula.

Introduction

The U.S. Commission on Civil Rights released a report in January of 2018 illuminating concerns over persistent inequity and declining adequacy of public school funding, exclaiming,

In 1954, the Supreme Court decreed in Brown v. Board of Education that public education “is a right which must be made available to all on equal terms,” yet all across the United States (U.S.), there are many millions of students who are unable to access a quality public education due to inequities in public education finance. With insufficient financial resources, our nation’s public schools generally struggle to provide a quality education on equal terms and evidence is concrete that “the U.S. educational system is one of the most unequal in the industrialized world, and students routinely receive dramatically different learning opportunities based on their social status.”

Meanwhile, a mounting body of rigorous empirical research indicates that equitable and adequate school funding is associated with improved student outcomes, especially for disadvantaged students (Baker, 2017). Furthermore, new evidence indicates that the substantial cuts to state school funding, which occurred during the recent recession led to reductions in student achievement and graduation rates (Jackson, Wigger, & Xiong, 2018).

New Old Data Present New Opportunities

To a significant degree, the flurry of recent, major findings surrounding the effects of state school finance reforms and effects of recessionary cuts to school funding on student outcomes, and general shifts in the equity and adequacy of state school finance systems, emanates from the growing availability of longitudinal data on school district spending and student outcomes. Increasingly, more granular data on both schooling inputs and student outcomes are becoming available, and these collections are accumulating over time into rich, longitudinal data systems. Some core elements of these data systems include:

• School district level fiscal data, primarily sourced to the U.S. Census Bureau’s Fiscal Survey of Local Governments—Public School District Finances (F-33) are consistently available at the level of local public education agency back to the early 1990s, covering two recessionary periods and the adoption of major changes to state and federal accountability policies (No Child Left Behind, Every Student Succeeds Act).

• State-level measures of student outcomes, via the National Assessment of Education Progress, have been consistently collected biennially since 2003 for all states and since the 1990s for most states.

• School and district (local public education agency) level, nationally normed student outcome data for students enrolled in Grades 3 to 8 are now available from 2009 through 2016 via the Stanford Education Data Archive (SEDA).

Longer time spans cover more significant events, including state court orders, changes to accountability systems, and external economic shocks like recessions. These provide unique opportunities to apply causal modeling to important major policy questions. To determine whether and to what extent changes to school funding equity and adequacy flow from judicial rulings, and whether those changes to funding lead to changes in student short- and long-term outcomes. They also provide the opportunity to determine how and to what extent economic shocks and other exogenous conditions affect the equity and adequacy of state school finance systems.

Many of these data come directly from consistently collected federal data sources which have now accumulated over the decades since early collections and dissemination of publicly available electronic format data (early 1990s). Nearly all national, or interstate studies of local school district finances make use of the U.S. Census Bureau’s Annual Survey of School System Finances (F-33). ¹ This single source has long been the go-to source of comparable interstate data on district revenues and expenditures and is frequently coupled with related school- and district-level data collections from the National Center for Education Statistics (NCES; Common Core of Data, Public Education Agency and Public School Universe Surveys). ²

Third-party organizations have aggregated and compiled these data for free public access and use. This special topic involved requests for manuscripts using data from two recently available warehouses. In the past few years, Rutgers University with the Education Law Center of New Jersey released panel data on state school funding equity indicators from 1993 through 2015, as well as district-level panel data used for estimating those equity indicators. Indicators include regression-based estimates of poverty progressiveness of revenues, expenditures, and staffing across districts with states and adjusted (for scale, sparsity, competitive wage) predicted values for revenues, expenditures, and staffing at varied poverty levels within states (Baker, Di Carlo, Srikanth, & Weber, 2019). This data system is now the School Finance Indicators Database, in collaboration with the Albert Shanker Institute and will soon be available through 2017 (Baker et al., 2019). ³

Concurrently, the Urban Institute produced a first-generation analysis and supporting website tracking funding levels and progressiveness by revenue source for each state (Chingos & Blagg, 2017a, 2017b). Although the methods of these two endeavors differ slightly, their findings are highly correlated. More important, these two endeavors reflect a growing recognition that it matters that states provide sufficient funding to higher poverty districts and schools.

Compilation and release of these finance-related data sets coincides with development and release of the SEDA, which provides nationally equated district- and school-level estimates of student test performance in reading and math, Grades 3 through 8 (Reardon et al., 2019). These data are now available annually from 2009 through 2016.

Meanwhile, federal agencies responsible for updating and managing these data collections continue to build on more recent collections and pursue new endeavors. The Office of Civil Rights of the U.S. Department of Education started in 2009–2010, biennial collections of school-level data, including estimates of school-level personnel and nonpersonnel spending. ⁴ They have, so far, continued to collect and study the feasibility of improving these measures, adding important granularity to data on the distribution of and access to schooling resources (Cornman et al., 2018). Over the past several years, statisticians with the Education Department have significantly built out geographic data systems on U.S. public schools and districts, increasing researchers’ ability to evaluate and understand spatial relations among schools, districts, and the economic conditions of the neighborhoods that surround them (Geverdt, 2019).

The Urban Institute is working to make all major national data sets on K–12 schools and districts—including finance measures from the F-33 survey and the Civil Rights Data Collection, along with enrollment and test-score data—widely accessible in one place through the Education Data Portal. The data portal harmonizes these data sets across years and makes them available through an Application Programming Interface, software packages for Stata and R, and a point-and-click online interface. ⁵

All of these advancements matter. But they all take time to accumulate. And that accumulation, with consistency, matters greatly to our ability to research and understand complex social systems like America’s public schools.

Recent Major Findings From Longitudinal Data Systems

Public school finance is largely a state and local endeavor. More broadly, public education policy is largely a state endeavor, and has been for decades. And thus, state data systems for annually tracking school spending and student outcomes largely predated comprehensive annual federal collections. Furthermore, the state is often the most relevant policy context within which to study the influence of policy changes or shocks. As such, through the early 2000s, as state school finance data had evolved, many of the most rigorous longitudinal studies of changes to state school finance systems, and resulting outcomes were state-specific studies, including studies in Massachusetts (Downes, Zabel, & Ansel, 2009; Guryan, 2001; Nguyen-Hoang & Yinger, 2014), Michigan (Chaudhary, 2009; Hyman, 2017; Roy, 2011; Papke, 2005), ⁶ Vermont (Downes, 2004), Kansas (Deke, 2003; Duncombe & Johnston, 2004), and elsewhere.

Earlier studies attempting national analyses lacked both sufficient time frame and granularity. The most notable of the earlier studies was by David Card and Abigail Payne (2002), who found “evidence that equalization of spending levels leads to a narrowing of test score outcomes across family background groups” (p. 49). Access to increased longitudinal data on both local district-level school finances and student outcomes has enabled this new wave of research. ⁷ A study by Kirabo Jackson, Rucker Johnson, and Claudia Persico (2015a, 2015b) looks at the long-term effects on high school graduation rates and future adult income of substantial infusions of funding to local public school districts through school finance reforms of the 1970s and 1980s. The authors used data from the Panel Study of Income Dynamics on approximately 15,000 sample members born between 1955 and 1985, who have been followed into adulthood through 2011 (Jackson et al., 2015b).

Jackson et al. (2015a) find that “the estimated effect of a 22 percent increase in per-pupil spending throughout all 12 school-age years for low-income children is large enough to eliminate the education gap between children from low-income and nonpoor families” (p. 72). This investment led to a 20 percentage point increase in graduation rates and, on average, an additional year of educational attainment for these children. Even lower levels of investment still made a sizable difference:

Increasing per-pupil spending by 10 percent in all 12 school-age years increases the probability of high school graduation by 7 percentage points for all students, by roughly 10 percentage points for low-income children, and by 2.5 percentage points for nonpoor children. (Jackson et al., 2015a, p. 74)

They also observed positive effects on adult wages (a 13% increase in hourly wages) and a substantial decrease in adult poverty rates resulting from this investment (Jackson et al., 2015a).

A similar recent study by Julien Lafortune, Jesse Rothstein, and Diane Schanzenbach (2016) evaluates the influence of adequacy-oriented school funding reforms during the 1990s and 2000s. Using NAEP data, the authors find that reforms cause gradual increases in the relative achievement of students in low-income school districts, consistent with the goal of improving educational opportunity for these students. The implied effect of school resources on educational achievement is large. In another national longitudinal analysis, Baker and Weber (2016; Baker, Farrie, & Sciarra, 2016) note that states with greater overall investment in education, resulting in more intensive staffing per pupil, tend to have higher outcomes for low-income children, higher performance in schools serving low-income children, and smaller disparities between schools serving low-income children and schools serving more advantaged populations. Christopher Candelaria and Kenneth Shores (2017) have found that there is a strong relationship between state school finance reforms and graduation rates. Seven years after the reforms, the poorest districts showed an average 12% increase in per-pupil spending and increases in graduation rates of between 6 and 12 percentage points (Candelaria & Shores, 2017).

Finally, enneth Shores and Matthew Steinberg (2017) conducted an analysis of the effects of recessionary spending cuts between 2009 and 2013 on short-run student achievement outcomes, combining data from national district-level school finance surveys with a unique collection of student level, nationally normed state assessment scores (Reardon et al., 2016). They found that the recession

significantly reduced student math and ELA achievement. Moreover, the recessionary effect on student achievement was concentrated among school districts serving more economically disadvantaged and minority students, indicating that the adverse effects of the recession were not distributed equally among the population of U.S. students. We also find that the academic impact of the recession was more severe for students who were older at the time of first exposure to the recession, compared to their younger counterparts. (Shores & Steinberg, 2017, p. 1; Shores & Steinberg, 2019)

Other studies have yielded similar findings regarding the impact of recessionary cuts on student outcomes, providing compelling new evidence of the large-scale achievement and economic benefits of substantive and sustained additional funding for schools serving higher poverty student populations and demonstrate that a reduction in funding can cause harm (Jackson et al., 2018).

In This Special Topic

This special topic contains two articles taking advantage of comprehensive longitudinal data on school finance. The first, by Victoria Sosina and Ericka Weathers, uses panel data from the School Finance Indicators Data System, or SFID (previously the School Funding Fairness Data System) from 1999 to 2013 to evaluate whether and to what extent changes to Black-White and Latinx-White demographic differences among districts leads to greater resource disparity over time. Sosina and Weathers find that changes to patterns of segregation are associated with changes to resource disparities, and importantly, find those changes to be different for Black students than for Latinx students. This article takes advantage of the fact that the Census Fiscal Survey, which the SFID draws on, includes granularity of schooling expenditure data, allowing the authors to explore whether changes to disparities in instructional spending differ from changes to disparities in capital infrastructure investment, and how these changes differ by race.

The second article, by Knight and Mendoza, combines data from the Census Fiscal Survey with data from the California Department of Education to explore whether differences in data on and measures of school funding equity matter (i.e., lead to similar or different conclusions) when evaluating the effects of California’s 2013 adoption of the Local Control Funding Formula. The authors take specific care to dissect the methods and expenditure classifications by which California Department of Education data are converted to classifications under the Census Fiscal Survey federal accounting model. Such data reconciliations are important for influencing policy for a variety of reasons. While academic researchers like to use big data, across many states and districts, and thus often turn to the Census Fiscal Surveys, state policy makers are invariably more confident in analyses and findings making use of their own state data sources, represented in state nomenclature and classifications. Most policy action on school finance occurs at the state level. Understanding the bridge between federal data collections and common state data reports is thus critically important. Knight and Mendoza make a strong case for shedding more light on the relationship between federal and state reporting.

Understanding how findings may differ across data systems and measures, on such things as characterizations of equity and adequacy is also critically important. State policy is more likely influenced for the good when messaging regarding the current state of state systems is consistent. The authors find significant consistencies, with explainable differences, in two common measures used to characterize state school finance systems—the regression-based approach underlying the SFID and the weighted average approached used by the Urban Institute to characterize progressiveness.

Opportunities and Threats

The continued growth of rich, longitudinal data systems has provided numerous research opportunities in recent years, with the two studies in this special topic providing useful examples. Significant findings regarding state school finance systems, the influence of outside factors on those systems, and the influence of those systems on student outcomes have emerged in recent years as these data have become more readily available and accessible.

Researchers have also grown dependent on these data. They are often background, context, and a given component of any rich study of public education policy. Continuation and expansion of high-quality research relying primarily on these data or even secondarily for context and backdrop requires that these data remain consistently, systematically collected, organized, and disseminated. Even short-term gaps in collections, release dates, changes to or deletion of critical measures can significantly compromise not only that research done in the immediate aftermath but research years forward which depends on consistent, accurate, and reliable over time and across context measures.

Thus, the future utility of the biennial school site data in the Civil Rights Data Collection depends on the current and all future federal administrations’ continued commitment to collecting and disseminating those data. Those data are frequently at risk of discontinuation, for political or budgetary reasons. Furthermore, many indicators in the SFID as well as in experimental geographic indicators in the NCES EDGE system depend heavily on household and individual data from the American Community Survey. These data too, are often discussed as being on the federal budget chopping block. Their continuation is critical for a variety of endeavors, including annual release of the current version of the NCES regional education cost adjustment—the American Community Survey Comparable Wage Index for Teachers (Cornman, Nixon, Spence, Taylor, & Geverdt, 2019). This is the only broadly available, updated index for regionally adjusting the value of the education dollar across settings.

Finally, because there is so much data out there, over such a long time frame, to be potentially drawn and aggregated from a variety of sources, third-party open source data aggregation and dissemination with thorough documentation is increasingly valuable to academic researchers, advocates, and policy makers. It should no longer be necessary for individual researchers to write their own code to construct 25-year panels of the census fiscal survey, to combine those data with school and district data from the NCES common core as well as a variety of geographic indicators from the EDGE data systems. And there is great value to being able to immediately combine these data with student outcome data held in the SEDA. But these data warehouses and freely accessible data collections are being managed and housed by academic institutions as well as DC beltway think tanks like the Urban Institute and Shanker Institute. Maintenance of these data systems requires funding for infrastructure and to ensure continuity of expertise, staffing, and management. The path forward will require a well-orchestrated and coordinated effort across funders, agencies, and institutions to ensure that these newly developed collections do not slip through the cracks.