Despite questions about validity and reliability, the use of value-added estimation methods has moved beyond academic research into state accountability systems for teachers, schools, and teacher preparation programs (TPPs). Prior studies of value-added measurement for TPPs test the validity of researcher-designed models and find that measuring differences across programs is difficult. This study is the first to examine the reliability and usefulness of a value-added model for TPPs developed through a collaborative stakeholder process and mandated by state law for use in accountability. Based on the experience of developing a test-based metric for Texas TPPs, our results suggest that although value-added results are statistically robust, accountability status for individual programs is very sensitive to decisions about accountability criteria, the selection of teachers, and the selection of control variables.
American Institutes for Research. (2011, August1-2). Florida’s Value-Added Model: Technical assistance workshop. Presentation to the Student Growth Implementation Committee, Orlando, FL. Retrieved from http://www.fldoe.org/committees/pdf/august12tammpres.pdf
2.
Armour-GarbA. (2009). Should “value added” models be used to evaluate teachers?Journal of Policy Analysis and Management, 28, 692-712.
3.
BakerE. L.BartonP. E.Darling-HammondL.HaertelE.LaddH. F.LinnR. L.. . .ShepardL. A. (2010). Problems with the use of student test scores to evaluate teachers (Vol. 278). Washington, DC: Economic Policy Institute.
4.
BallouD.SandersW.WrightP. (2004). Controlling for student background in value-added assessment of teachers. Journal of Educational and Behavioral Statistics, 29(1), 37-65.
5.
BoydD. J.LankfordH.LoebS.RockoffJ. E.WyckoffJ. (2008). The narrowing gap in New York city teacher qualifications and its implications for student achievement in high poverty schools. Journal of Policy Analysis and Management, 27(4), 793-818.
6.
CAEP Standards for Accreditation of Educator Preparation. (2011). In Council for the Accreditation of Educator Preparation. Retrieved from http://caepnet.org/accreditation/standards/
GansleK. A.NoellG. H.KnoxR. M.SchaferM. J. (2010). Value added assessment of teacher preparation in Louisiana: 2005-2006 to 2008-2009. Baton Rouge: Louisiana State University.
9.
GoldhaberD.LiddleS. (2012, January). The gateway to the profession: Assessing teacher preparation programs based on student achievement (CALDER Working Paper 65). Washington, DC: National Center for Analysis of Longitudinal Data in Education Research.
10.
GreenbergJ.McKeeA.WalshK. (2013). Teacher prep review: A review of the Nation’s Teacher Preparation Programs 2013. Washington, DC: National Council of Teacher Quality. Retrieved from http://www.nctq.org/dmsView/Teacher_Prep_Review_2013_Report
11.
GuarinoC.ReckaseM.WooldridgeJ. (2012, June). Can value-added measures of teacher education performance be trusted (Working Paper., Vol. 18). East Lansing: Michigan Education Policy Center, Michigan State University.
12.
HarrisD. N. (2011). Value-added measures in education: What every educator needs to know. Cambridge, MA: Harvard Education Press.
13.
HenryG. T.KershawD. C.ZulliR. A.SmithA. A. (2012). Incorporating teacher effectiveness into teacher preparation program evaluation. Journal of Teacher Education, 63(5), 335-355.
14.
HenryG. T.ThompsonC. L.FortnerC. K.ZulliR. A.KershawD. C. (2010). The impact of teacher preparation on student learning in North Carolina public schools. Chapel Hill: University of North Carolina at Chapel Hill, College of Arts and Sciences, Carolina Institute for Public Policy.
15.
HillH. C. (2009). Evaluating value-added models: A validity argument approach. Journal of Policy Analysis and Management, 28(4), 700-709.
16.
HinkleD. E.WiersmaW.JursS. G. (2002). Applied statistics for the behavioral sciences (5th ed.). Stamford, CT: Cengage.
17.
KaneT. J.McCaffreyD. F.MillerT.StaigerD. O. (2013, January). Have we identified effective teachers? Validating measures of effective teaching using random assignment (Research Paper, MET Project). Seattle, WA: Bill & Melinda Gates Foundation.
18.
KoedelC.ParsonsE.PodgurskyM.EhlertM. (2012, July). Teacher preparation programs and teacher quality: Are there real difference across programs? (CALDER Working Paper No. 79). Washington, DC: National Center for Analysis of Longitudinal Data in Education Research.
19.
McCaffreyD. F.LockwoodJ. R.KoretzD. M.HamiltonL. S. (2003).Evaluating value-added models for teacher accountability (Monograph). Santa Monica, CA: RAND Corporation.
20.
MihalyK.McCaffreyD.SassT.LockwoodJ. R. (in press) Where you come from or where you go: Distinguishing between school quality and the effectiveness of teacher preparation program graduates. Journal of Education Finance and Policy.
21.
NoellG.BurnsJ. M.GansleK. A. (2011). Value-added teacher preparation programs in Louisiana: 2007-08 to 2009-10. Baton Rouge, LA: Louisiana Board of Regents.
22.
NoellG. H.BurnsJ. L. (2006). Value added assessment of teacher preparation: An illustration of emerging technology. Journal of Teacher Education, 57, 37-50.
OsborneC.LincoveJ. A.Von HippelP.MillsN.DillonA.BellowsB. (2012, June). The Texas report: Education preparation programs’ influence on student achievement (Project on Education Effectiveness and Quality). University of Texas at Austin.
27.
OsborneC.Von HippelP.LincoveJ. A.MillsN. (2013, March). The small and unreliable effects of teacher preparation programs on student test scores in Texas. Presented at the Conference of the Association of Education Finance and Management. New Orleans, LA.
28.
PleckiM. L.ElfersA. M.NakamuraY. (2012). Using evidence for teacher education program improvement and accountability: An illustrative case of the role of value-added measures. Journal of Teacher Education, 63(5), 318-334.
29.
RothsteinJ. (2009). Student sorting and bias in value-added estimation: Selection on observables and unobservables. Education, 4(4), 537-571.
30.
SchochetP. Z.ChiangH. S. (2010, July). Error rates in measuring teacher and school performance based on student test score gains (No. NCEE 2010-4004). National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
