Value-added models (VAMs) are used to measure changes in student achievement on large-scaled standardized test scores from year to year. When aggregated, VAM estimates are used to measure teacher effectiveness and hold teachers accountable for the value they purportedly add to or detract from student learning and achievement. In this study, researchers examined the extent to which purposeful (nonrandom) and random assignment of students into classrooms occurs in Arizona elementary schools (Grades 3–6). Researchers found that overwhelmingly, students are not randomly assigned and administrators, teachers, and parents play a prodigious role in the process. Findings have current implications for value-added analyses and the extent to which nonrandom assignment practices might impact or bias teachers’ value-added scores.
BakerE. L.BartonP. E.Darling-HammondL.HaertelE.LaddH. F.LinnR. L.. . . ShepardL. A. (2010). Problems with the use of student test scores to evaluate teachers. Washington, DC: Economic Policy Institute. Retrieved from http://www.epi.org/publication/bp278/
BallouD. (2012). Review of The long-term impacts of teachers: Teacher value-added and student outcomes in adulthood. Boulder, CO: National Education Policy Center. Retrieved from http://nepc.colorado.edu/thinktank/review-long-term-impacts
5.
BallouD.SandersW. L.WrightP. (2004). Controlling for student background in value-added assessment of teachers. Journal of Educational and Behavioral Statistics, 29(1), 37–66. doi:10.3102/10769986029001037
BraunH. I. (2005). Using student progress to evaluate teachers: A primer on value-added models. Princeton, NJ: Educational Testing Service (ETS). Retrieved from www.ets.org/Media/Research/pdf/PICVAM.pdf
BurnsR. B.MasonD. A. (1995). Organizational constraints on the formation of elementary school classes. American Journal of Education, 103(2), 185–212. doi:10.1086/444096
ChettyR.FriedmanJ. N.RockoffJ. E. (2011). The long-term impacts of teachers: Teacher value-added and student outcomes in adulthood. Cambridge, MA: National Bureau of Economic Research. Retrieved from http://obs.rc.fas.harvard.edu/chetty/value_added.pdf
CookT. D.CampbellD. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago, IL: Rand McNally.
14.
CorcoranS. P. (2010). Can teachers be evaluated by their students’ test scores? Should they be?Providence, RI: Annenberg Institute for School Reform.
15.
CreswellJ. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches (2nd ed.). Thousand Oaks, CA: Sage.
16.
DillsA. K.MulhollandS. E. (2010). A comparative look at private and public schools’ class size determinants. Education Economics, 18(4), 435–454. doi:10.1080/09645290903546397
17.
DunnM. C.KadaneJ. B.GarrowJ. R. (2003). Comparing harm done by mobility and class absence: Missing students and missing data. Journal of Educational and Behavioral Statistics, 28(3), 269–288. doi:10.3102/10769986028003269
18.
EhlertM.KoedelC.ParsonsE.PodgurskyM. (2012). Selecting growth measures for school and teacher evaluations. Washington, DC: National Center for Analysis of Longitudinal Data in Education Research (CALDER).
19.
ErcikanK.Wolff-MichaelR. (Eds.) (2009). Generalizing from educational research: Beyond qualitative and quantitative polarization. New York, NY: Routledge.
20.
EricksonF. (1986). Qualitative methods in research on teaching. In WittrockM. C. (Ed.), Handbook of research on teaching (pp. 119–161). New York, NY: Macmillan.
21.
GabrielR.AllingtonR. (2011). Teacher effectiveness research and the spectacle of effectiveness policy. Paper Presented at the Annual Conference of the American Educational Research Association (AERA), New Orleans, LA.
22.
GlaserB.StraussA. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine.
23.
GlassG. V.SmithM. L. (1979). Meta-analysis of research on class size and achievement. Educational Evaluation and Policy Analysis, 1(1), 2–16.
GoldhaberD.GabeleB.WalchJ. (2012). Does the model matter? Exploring the relationship between different student achievement-based teacher assessments. Seattle, WA: Center for Education Data and Research (CEDR). Retrieved from https://appam.confex.com/appam/2012/webprogram/Paper2264.html
HarrisD. N. (2009). Would accountability based on teacher value added be smart policy? An evaluation of the statistical properties and policy alternatives. Education Finance and Policy, 4, 319–350. doi:10.1162/edfp.2009.4.4.319
32.
HarrisD. N. (2011). Value-added measures in education: What every educator needs to know. Cambridge, MA: Harvard Education Press.
HillH. C.KapitulaL.UmlanK. (2011). A validity argument approach to evaluating teacher value-added scores. American Educational Research Journal, 48(3), 794–831. doi:10.3102/0002831210387916
36.
JohnsonR. B.OnwuegbuzieA. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26. doi:10.3102/0013189X033007014
37.
KaneT. J.StaigerD. O. (2008). Estimating teacher impacts on student achievement: An experimental evaluation. Cambridge, MA: The National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w14607
38.
KerstingN. B.ChenM.StiglerJ. W. (2013). Value-added added teacher estimates as part of teacher evaluations: Exploring the effects of data and model specifications on the stability of teacher value-added scores. Education Policy Analysis Archives, 21(7), 1–39. Retrieved from http://epaa.asu.edu/ojs/article/view/1167
39.
KoedelC.BettsJ. R. (2007, April). Re-examining the role of teacher quality in the educational production function. Nashville, TN: National Center on Performance Initiatives. Retrieved from http://ideas.repec.org/p/umc/wpaper/0708.html
40.
KoedelC.BettsJ. R. (2010). Does student sorting invalidate value-added models of teacher effectiveness? An extended analysis of the Rothstein critique. Education Finance and Policy6(1), 18–42. doi:10.1162/EDFP_a_00027
41.
KupermintzH. (2003). Teacher effects and teacher effectiveness: A validity investigation of the Tennessee value added assessment system. Educational Evaluation & Policy Analysis, 25(3), 287–298. doi:10.3102/01623737025003287
McCaffreyD. F.LockwoodJ.KoretzD.LouisT. A.HamiltonL. (2004). Models for value-added modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29(1), 67–101. doi:10.3102/10769986029001067
MilesM. B.HubermanA. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage.
46.
MonkD. H. (1987). Assigning elementary pupils to their teachers. Elementary School Journal, 88(2), 167–187. doi:10.1086/461531
47.
NewtonX.Darling-HammondL.HaertelE.ThomasE. (2010). Value-added modeling of teacher effectiveness: An exploration of stability across models and contexts. Educational Policy Analysis Archives, 18(23). Retrieved from http://epaa.asu.edu/ojs/article/view/810
PlayerD. (2010). Nonmonetary compensation in the public teacher labor market. Education Finance and Policy, 5(1), 82–103. doi:10.1162/edfp.2009.5.1.5105
50.
PraisnerC. (2003). Attitudes of elementary school principals toward the inclusion of students with disabilities. Exceptional Children, 69(2), 135–145.
51.
RaudenbushS. W. (2004). Schooling, statistics, and poverty: Can we measure school improvement? Princeton, NJ: Educational Testing Service (ETS). Retrieved from www.ets.org/Media/Education_Topics/pdf/angoff9.pdf
ReardonS. F.RaudenbushS. W. (2009). Assumptions of value-added for estimating school effects. Education Finance and Policy, 4(4), 492–519. doi:10.1162/edfp.2009.4.4.492
54.
RickettsC. R. (2010). End of course grades and standardized test scores: Are grades predictive of student achievement? (Doctoral dissertation). Available from ProQuest LLC. Retrieved from http://search.proquest.com/docview/755303452
55.
RivkinS. G.HanushekE. A.KainJ. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417–458. doi:10.1111/j.1468-0262.2005.00584.x
56.
RivkinS. G.IshiiJ. (2008). Impediments to the estimation of teacher value added. Paper presented at the National Conference on Value-Added Modeling. Sponsored by the Wisconsin Center for Education Research (WCER), Madison, WI.
57.
RothsteinJ. (2009, January11). Student sorting and bias in value-added estimation: Selection on observables and unobservables. Cambridge, MA: The National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w14607
58.
RothsteinJ. (2010). Teacher quality in educational production: Tracking, decay, and student achievement. Quarterly Journal of Economics, 125(1), 175–214. doi:10.1162/qjec.2010.125.1.175
RubinD. B.StuartE. A.ZanuttoE. L. (2004). A potential outcomes view of value-added assessment in education. Journal of Educational and Behavioral Statistics, 29(1), 103–116. doi:10.3102/10769986029001103
61.
SandersW. L. (1998). Value-added assessment. The School Administrator, 55(11), 24–27.
62.
SandersW. L.HornS. (1998). Research findings from the Tennessee Value-Added Assessment System (TVAAS)database: Implications for educational evaluation and research. Journal of Personnel Evaluation in Education, 12(3), 247–256.
ScherrerJ. (2011). Measuring teaching using value-added modeling: The imperfect panacea. NASSP Bulletin, 95(2), 122–140. doi:10.1177/0192636511410052
65.
StacyB.GuarinoC.RecklaseM.WooldridgeJ. (2012). Does the precision and stability of value-added estimates of teacher performance depend on the types of students they serve?East Lansing, MI: Education Policy Center. Retrieved from https://appam.confex.com/appam/2012/webprogram/Paper3327.html
66.
StakeR. E.TrumbullD. (1982). Naturalistic generalizations. Review Journal of Philosophy and Social Science, 7, 1–12.
StraussA. L.CorbinJ. (1995). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage.
69.
TeddlieC.TashakkoriA. (2006). A general typology of research designs featuring mixed methods. Research in the Schools, 13(1), 12–28.
70.
ThompsonB. (2000). The APA Task Force on Statistical Inference Report as a framework for teaching and evaluating students’ understandings of study validity. Paper presented at the annual meeting of the American Educational Research Association (AERA), New Orleans, LA.
U.S. Department of Education, National Center for Education Statistics (NCES), Common Core of Data (CCD). (1999–2000 and 2009–2010). State nonfiscal survey of public elementary/secondary education (Table 44). Retrieved from http://nces.ed.gov/programs/digest/d11/tables/dt11_044.asp
73.
U.S. Department of Education, National Center for Education Statistics (NCES), Common Core of Data (CCD). (2009–2010a). Public elementary/secondary school universe survey (Table 45). Retrieved from http://nces.ed.gov/programs/digest/d11/tables/dt11_045.asp
74.
U.S. Department of Education, National Center for Education Statistics (NCES), Common Core of Data (CCD). (2009–2010b). Public elementary/secondary school universe survey (Table 48). Retrieved from http://nces.ed.gov/programs/digest/d11/tables/dt11_048.asp
75.
U.S. Department of Education, National Center for Education Statistics (NCES), Common Core of Data (CCD). (2009–2010c). Public elementary/secondary school universe survey (Table 104). Retrieved from http://nces.ed.gov/programs/digest/d11/tables/dt11_104.asp
76.
U.S. Department of Education, National Center for Education Statistics (NCES), Common Core of Data (CCD). (2010–2011). Public elementary/secondary school universe survey (Table 2). Retrieved from http://nces.ed.gov/pubs2012/pesschools10/tables/table_02.asp
77.
U.S. Department of Education, National Center for Education Statistics (NCES), Schools and Staffing Survey. (2007–2008). Characteristics of public, private, and Bureau of Indian Education elementary and secondary school principals in the United States (Table 7). Retrieved from http://nces.ed.gov/pubs2009/2009323/tables/sass0708_2009323_p12n_07.asp
WilkinsonL., & Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54(8), 594–604. doi:10.1037//0003-066X.54.8.594
WordE.JohnstonJ.BainH. P.FultonD. B.ZahariasJ. B.AchillesC. M.. . . BredaC. (1990). The State of Tennessee’s Student/Teacher Achievement Ratio (STAR) project. Nashville, TN: Tennessee State Department of Education. Retrieved from http://d64.e2services.net/class/STARsummary.pdf
82.
WrightP.HornS.SandersW. L. (1997). Teachers and classroom heterogeneity: Their effects on educational outcomes. Journal of Personnel Evaluation in Education, 11(1), 57–67.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
