There is an increasing interest in using longitudinal measures of student achievement to estimate individual teacher effects. Current multivariate models assume each teacher has a single effect on student outcomes that persists undiminished to all future test administrations (complete persistence [CP]) or can diminish with time but remains perfectly correlated (variable persistence [VP]). However, when state assessments do not use a vertical scale or the evolution of the mix of topics present across a sequence of vertically aligned assessments changes as students advance in school, these assumptions of persistence may not be consistent with the achievement data. We develop the “generalized persistence” (GP) model, a Bayesian multivariate model for estimating teacher effects that accommodates longitudinal data that are not vertically scaled by allowing less than perfect correlation of a teacher’s effects across test administrations. We illustrate the model using mathematics assessment data.
AkaikeH. (1973). Theory and extension of the maximum likelihood principle. In PetrovB.CsakiF. (Eds.), Second International Symposium on information theory (pp. 267–281). Budapest: Akademiai Kiado.
2.
BallouD.SandersW.WrightP. (2004). Controlling for student background in value-added assessment of teachers. Journal of Educational and Behavioral Statistics, 29, 37–66.
3.
BarnardJ.McCullochR.MengX.-L. (2000). Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage. Statistica Sinica, 10, 1281–1311.
4.
BifulcoR.LaddH. (2004). The impact of charter schools on student achievement: Evidence from North Carolina. Education Finance and Policy, 1, 50–90.
5.
BraunH. (2005). Using student progress to evaluate teachers: A primer on value-added models (Technical report). Educational Testing Service, Policy Information Center.
6.
BrowneW. J.GoldsteinH.RasbashJ. (2001). Multiple membership multiple classification (MMMC) models. Statistical Modelling: An International Journal, 1, 103–124.
7.
CarlinB.LouisT. (2000). Bayes and empirical Bayes methods for data analysis. 2nd ed.Boca Raton, FL: Chapman and Hall/CRC Press.
8.
CornelissenT. (2008). The stata command felsdvreg to fit a linear model with two high-dimensional fixed effects. Stata Journal, 8, 170–189.
9.
DanielsM. J. (1999). A prior for the variance in hierarchical models. The Canadian Journal of Statistics, 27, 567–578.
10.
DanielsM. J.KassR. E. (1999). Nonconjugate Bayesian estimation of covariance matrices and its use in hierarchical models. Journal of the American Statistical Association, 94, 1254–1263.
11.
Fitz-GibbonC. T. (1997). The value added national project: Final report, feasibility studies for a national system of value added indicators. London: SCAA.
12.
GelmanA.CarlinJ.SternH.RubinD. (1995). Bayesian data analysis. London: Chapman & Hall.
13.
GelmanA.RubinD. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472.
14.
GilksW. R.RichardsonS.SpiegelhalterD. J. (1996). Markov chain Monte Carlo in practice. London: Chapman & Hall.
15.
GillB.ZimmerR.ChristmanJ.BlancS.. (2007). State takeover, school restructuring, private management, and student achievement in Philadelphia (MG-533-ANF/WPF/SDP). Santa Monica, CA: RAND. Retrieved fromhttp://www.rand.org/pubs/monographs/MG533
16.
GoldhaberA.AnthonyE..
(2004). Can teacher quality be effectively assessed?Unpublished manuscript.
17.
GordonR.KaneT.StaigerD. (2006). Identifying effective teachers using performance on the job (Technical report; White Paper 2006-01). Washington, DC: The Brookings Institution.
18.
HamiltonL. S.McCaffreyD. F.KoretzD. (2006). Longitudinal and value-added modeling of student performance. In Validating achievement gains in cohort-to-cohort and individual growth-based modeling contexts (pp. 407–434). Maple Grove, MN: JAM Press.
19.
Harcourt Brace Educational Measurement. (1997). Stanford achievement test series, ninth edition, technical data report. San Antonio, TX: Harcourt Brace and Company.
20.
HarrisD.SassT.. (2006). Value-added models and the measurement of teacher quality. Unpublished manuscript.
KaneT.RockoffJ.StaigerD.. (2006). What does certification tell us about teacher effectiveness? Evidence from New York City. Unpublished manuscript.
23.
KassR.RafteryA. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795.
24.
LeV.StecherB.LockwoodJ.HamiltonL.RobynA.WilliamsV. (2006). Improving mathematics and science education: A longitudinal investigation of the relationship between reform-oriented instruction and student achievement (MG-480-EDU). Santa Monica, CA: RAND.
25.
LissitzR. (Ed.). (2005). Value added models in education: Theory and applications. Maple Grove, MN: JAM Press.
26.
LittleR.RubinD. (1987). Statistical analysis with missing data. 2nd ed.New York: John Wiley & Sons.
27.
LockwoodJ.McCaffreyD. (2007). Controlling for individual heterogeneity in longitudinal models, with applications to student achievement. Electronic Journal of Statistics, 1, 223–252.
28.
LockwoodJ.McCaffreyD.HamiltonL.StecherB.LeV.MartinezJ. (2007). The sensitivity of value-added teacher effect estimates to different mathematics achievement measures. Journal of Educational Measurement, 44, 47–67.
29.
LockwoodJ.McCaffreyD.MarianoL.SetodjiC. (2007). Bayesian methods for scalable multivariate value-added assessment. Journal of Educational and Behavioral Statistics, 32, 125–150.
30.
LunnD.ThomasA.BestN.SpiegelhalterD. (2000). WinBUGS—A Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10, 325–337.
31.
MartineauJ. (2006). Distorting value added: The use of longitudinal, vertically scaled student achievement data for value-added accountability. Journal of Educational and Behavioral Statistics, 31, 35–62.
32.
McCaffreyD.LockwoodJ.KoretzD.LouisT.HamiltonL. (2004). Models for value-added modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29, 67–101.
33.
McCaffreyD.LockwoodJ.KoretzD.HamiltonL. (2003). Evaluating value-added models for teacher accountability (MG-158-EDU). Santa Monica, CA: RAND.
34.
McCaffreyD.LockwoodJ.MarianoL.SetodjiC. (2005). Challenges for value-added assessment of teacher effects. In Value added models in education: Theory and applications (pp. 111–144). Maple Grove, MN: JAM Press.
35.
RasbashJ.BrowneW. (2001). Modelling non-hierarchical structures. In LeylandA.GoldsteinH. (Eds.), Multilevel modelling of health statistics (pp. 93–103). Wiley:
36.
RaudenbushS. (2004). Schooling, statistics, and poverty: Can we measure school improvement? (Technical report). Princeton, NJ: Educational Testing Service.
37.
RaudenbushS.BrykA. (2002). Hierarchical linear models: Applications and data analysis methods. 2nd ed.Thousand Oaks, CA: Sage.
38.
ReckaseM. (2004). The real world is more complicated than we would like. Journal of Educational and Behavioral Statistics, 29, 117–120.
SandersW.SaxtonA.HornB. (1997). The Tennessee value-added assessment system: A quantitative outcomes-based approach to educational assessment. In MillmanJ. (Ed.), Grading teachers, grading schools: Is student achievement a valid evaluational measure? (pp. 137–162). Thousand Oaks, CA: Corwin.
41.
SchacterJ.ThumY. M. (2004). Paying for high and low-quality teaching. Economics of Education Review, 23, 411–430.
42.
SchaferJ. (1997). Analysis of incomplete multivariate data. New York: Chapman & Hall.
43.
SchmidtH.HouangR.McKnightC. (2005). Value-added research: Right idea but wrong solution?. In LissitzR. (Ed.), Value added models in education: Theory and applications (pp. 272–297). Maple Grove, MN: JAM Press.
44.
SchwarzG. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464.
45.
SearleS.CasellaG.McCullochC. (1992). Variance components. New York: John Wiley & Sons.
46.
SmithW. B.HockingR. R. (1972). Algorithm as 53: Wishart variate generator. Applied Statistics, 21, 341–345.
47.
SpiegelhalterD.BestN.CarlinB.van der LindeA. (2002). Bayesian measures of model complexity and fit (with discussion). Journal of the Royal Statistical Society Series B, 64, 583–639.
48.
TannerM.WongW. (1987). The calculation of posterior distributions by data augmentation (with discussion). Journal of the American Statistical Association, 82, 528–550.
49.
van DykD.MengX. (2001). The art of data augmentation (with discussion). The Journal of Computational and Graphical Statistics, 10, 1–111.
50.
ZimmerR.BuddinR.ChauD.DaleyG.GillB.GuarinoC. (2003). Charter school operations and performance: Evidence from California. Santa Monica, CA: RAND.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
