Many states and school districts in the United States use standardized test scores to compute annual measures of student achievement progress and then use school-level averages of these growth measures for various reporting and diagnostic purposes. These aggregate growth measures can vary consequentially from year to year for the same school, complicating their use and interpretation. We develop a method, based on the theory of empirical best linear prediction, to improve the accuracy and stability of aggregate growth measures by pooling information across grades, years, and tested subjects for individual schools. We demonstrate the performance of the method using both simulation and application to 6 years of annual growth measures from a large, urban school district. We provide code for implementing the method in the package schoolgrowth for the R environment.
AbowdJ.CreecyR.KramarzF. (2002). Computing person and firm effects using linked longitudinal employer-employee data (Tech. Rep., TP-2002-06). U.S. Census Bureau, Center for Economic Studies.
2.
AllisonP. D. (1990). Change scores as dependent variables in regression analysis. Sociological Methodology, 20, 93–114. https://doi.org/10.2307/271083
3.
BatesD.MächlerM.BolkerB.WalkerS. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
4.
BatesD.MaechlerM. (2019). Matrix: Sparse and dense matrix classes and methods [Computer software manual]. https://CRAN.R-project.org/package=Matrix(R package version 1.2–17).
CastellanoK. E.McCaffreyD. F. (2020). Estimating the accuracy of relative growth measures using real data. Journal of Educational Measurement, 57(1), 92–123. https://doi.org/10.1111/jedm.12243
8.
CronbachL.FurbyL. (1970). How we should measure “change”: Or should we?Psychological Bulletin, 74(1), 68–80. https://doi.org/10.1037/h0029382
Data Quality Campaign. (2019, January). Growth data: It matters, and it’s complicated. Retrieved December 4, 2019, fromhttp://dataqualitycampaign.org
11.
DragosetL.BasterC.DotterD.WalshE. (2019). Measuring school performance for early elementary grades in Maryland. Regional Educational Laboratory Mid-Atlantic (ERIC Number ED601956). https://files.eric.ed.gov/fulltext/ED601956.pdf
12.
EversonK. C. (2017). Value-added modeling and educational accountability: Are we answering the real questions?Review of Educational Research, 87(1), 35–70. https://doi.org/10.3102/0034654316637199
13.
FullerW. (2009). Sampling statistics. John Wiley & Sons.
14.
GoldbergerA. (1962). Best linear unbiased prediction in the generalized linear regression model. Journal of the American Statistical Association, 57(298), 369–375. https://doi.org/10.1080/01621459.1962.10480665
15.
GoldschmidtP.ChoiK.BeaudoinJ. (2012). Growth model comparison study: Practical implications of alternative models for evaluating school performance. Council of Chief State School Officers. https://files.eric.ed.gov/fulltext/ED542761.pdf
16.
GorardS.HordosyR.SiddiquiN. (2013). How stable are “school effects” assessed by a value-added technique?International Education Studies, 6(1), 1–9. https://doi.org/10.5539/ies.v6n1p1
17.
HarrisD. (2011). Value-added measures in education: What every educator needs to know. Harvard Education Press.
18.
HarvilleD. A.JeskeD. R. (1992). Mean squared error of estimation or prediction under a general linear model. Journal of the American Statistical Association, 87(419), 724–731. https://doi.org/10.1080/01621459.1992.10475274
19.
HendersonC. R. (1975). Best linear unbiased estimation and prediction under a selection model. Biometrics, 31(2), 423–447.
20.
HighamN. J. (2002). Computing the nearest correlation matrix — A problem from finance. IMA Journal of Numerical Analysis, 22(3), 329–343.
21.
KackarR. N.HarvilleD. A. (1984). Approximations for standard errors of estimators of fixed and random effects in mixed linear models. Journal of the American Statistical Association, 79(388), 853–862. https://doi.org/10.1080/01621459.1984.10477102
22.
KaneM. (2017). Measurement error and bias in value-added models (ETS Research Rep. No. RR-17-25. ETS)
23.
KaneT.StaigerD. (2001). Improving School Accountability Measures (Tech. Rep. No. 8156). National Bureau of Economic Research.
24.
KaneT.StaigerD. (2002). The promise and pitfalls of using imprecise school accountability measures. Journal of Economic Perspectives, 16(4), 91–114. https://doi.org/10.1257/089533002320950993
25.
KaneT.StaigerD.GrissmerD.LaddH. (2002). Volatility in school test scores: Implications for test-based accountability systems. Brookings Papers on Education Policy, (5), 235–283.
LinnR. L.HaugC. (2002). Stability of school-building accountability scores and gains. Educational Evaluation and Policy Analysis, 24(1), 29–36. https://doi.org/10.3102/2F01623737024001029
28.
LittleR.RubinD. (2002). Statistical analysis with missing data (2nd ed.). John Wiley & Sons.
29.
LockwoodJ. R.McCaffreyD. F. (2014a). Correcting for test score measurement error in ANCOVA models for estimating treatment effects. Journal of Educational and Behavioral Statistics, 39(1), 22–52. https://doi.org/10.3102/1076998613509405
30.
LockwoodJ. R.McCaffreyD. F. (2014b). Should nonlinear functions of test scores be used as covariates in a regression model? In LissitzR. (Ed.), Value-added modeling and growth modeling with particular application to teacher and school effectiveness. Information Age Publishing.
31.
LockwoodJ. R.McCaffreyD. F. (2020). Using hidden information and performance-level boundaries to study student-teacher assignments: Implications for estimating teacher causal effects. Journal of the Royal Statistical Society: Series A, 183(4), 1333–1362. https://doi.org/10.1111/rssa.12533
McCaffreyD.CastellanoK.LockwoodJ. (2015). The impact of measurement error on the accuracy of individual and aggregate SGP. Educational Measurement: Issues and Practice, 34(1), 15–21.
34.
McCaffreyD.LockwoodJ.KoretzD.LouisT.HamiltonL. (2004). Models for value-added modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29(1), 67–101. https://doi.org/10.3102/10769986029001067
35.
McCaffreyD.SassT.LockwoodJ.MihalyK. (2009). The intertemporal variability of teacher effect estimates. Education Finance and Policy, 4(4), 572–606.
PinheiroJ. C.BatesD. M. (2000). Mixed-effects models in S and S-PLUS. Springer.
38.
RaudenbushS.BrykA. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Sage Publications.
39.
R Core Team. (2020). R: A language and environment for statistical computing[Computer software manual]. https://www.R-project.org/
40.
RobinsonG. (1991). That BLUP is a good thing: The estimation of random effects (with discussion). Statistical Science, 6(1), 15–51. https://doi.org/10.1214/ss/1177011926
41.
SAS Institute Inc. (2016). SAS® EVAAS® for K-12: Statistical Model. www.sas.com
42.
SchochetP.ChiangH. (2013). What are error rates for classifying teacher and school performance using value-added models?Journal of Educational and Behavioral Statistics, 38(2), 142–171. https://doi.org/10.3102/1076998611432174
43.
SearleS. (1971). Linear models. John Wiley & Sons.
44.
SearleS. (1997). The matrix handling of BLUE and BLUP in the mixed linear model. Linear Algebra and its Applications, 264, 291–311.
45.
ShenW.LouisT. (1998). Triple-goal estimates in two-stage, hierarchical models. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 60(2), 455–471. https://doi.org/10.1111/1467-9868.00135
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
