Sage Journals: Discover world-class research

Abstract

Recently, educational researchers and practitioners have turned to value-added models to evaluate teacher performance. Although value-added estimates depend on the assessment used to measure student achievement, the importance of outcome selection has received scant attention in the literature. Using data from a large, urban school district, I examine whether value-added estimates from three separate reading achievement tests provide similar answers about teacher performance. I find moderate-sized rank correlations, ranging from 0.15 to 0.58, between the estimates derived from different tests. Although the tests vary to some degree in content, scaling, and sample of students, these factors do not explain the differences in teacher effects. Instead, test timing and measurement error contribute substantially to the instability of value-added estimates across tests.

Keywords

teacher research school/teacher effectiveness teacher assessment educational policy

Get full access to this article

View all access options for this article.

References

Ballou

Sanders

Wright

(2004). Controlling for student background in value-added assessment of teachers. Journal of Educational and Behavioral Statistics, 29(1), 37–65.

Boardman

A. E.

Murnane

R. J.

(1979). Using panel data to improve estimates of the determinants of educational achievement. Sociology of Education, 52(2), 113–121.

Boyd

Grossman

Lankford

Loeb

Wycoff

(2006). How changes in entry requirements alter the teacher workforce and affect student achievement. Education Finance and Policy, 1(2), 176–216.

Boyd

Grossman

Lankford

Loeb

Wycoff

(2007). Who leaves? Teacher attrition and student achievement. Unpublished manuscript.

Briggs

D. C.

Weeks

J. P.

Wiley

(2008). The sensitivity of value-added modeling to the creation of a vertical score scale (Working paper for the National Conference on Value-Added Modeling). Retrieved from http://www.wcer.wisc.edu/news/events/natConf_papers.php

Cantrell

Fullerton

Kane

T. J.

Staiger

D. O.

(2007). National Board Certification and teacher effectiveness: Evidence from a random assignment experiment. Unpublished manuscript.

Chester

M. D.

(2003). Multiple measures and high-stakes decisions: A framework for combining measures. Educational Measurement: Issues and Practice, 22(2), 32–41.

Clotfelter

C. T.

Ladd

H. F.

Vigdor

J. L.

(2007). How and why do teacher credentials matter for student achievement? (NBER Working Paper No. 12828). Cambridge, MA: National Bureau of Economic Research.

Croninger

R. G.

Valli

(2009). “Where is the action?” Challenges to studying the teaching of reading in elementary classrooms. Educational Researcher, 38(2), 100–108.

10.

Douglas

K. M.

(2007). A general method for estimating the classification reliability of complex decisions based on configural combinations of multiple assessment scores. Unpublished doctoral dissertation, University of Maryland, College Park, MD.

11.

Gewertz

(2007). N.Y.C. district issues “value added” grades for schools. Education Week, 27(12), 6.

12.

Goldhaber

Anthony

(2007). Can teacher quality be effectively assessed? National Board Certification as a signal of effective teaching. Review of Economics and Statistics, 89(1), 134–150.

13.

Gordon

Kane

T. J.

Staiger

D. O.

(2006). Identifying effective teachers using performance on the job. Washington, DC: Brookings Institution.

14.

Hanushek

(1971). Teacher characteristics and gains in student achievement: Estimation using micro data. The American Economic Review, 61(2), 280–288.

15.

Harris

D. N.

Sass

T. R.

(2006). Value-added models and the measurement of teacher quality. Unpublished manuscript.

16.

Harris

D. N.

Sass

T. R.

(2009). The effects of NBPTS-certified teachers on student achievement. Journal of Policy Analysis and Management, 28(1), 55–80.

17.

Jacob

B. A.

Lefgren

(2005). Principals as agents: Subjective performance measurement in education (NBER Working Paper No. 11463). Cambridge, MA: National Bureau of Economic Research.

18.

Kane

T. J.

Rockoff

J. E.

Staiger

D. O.

(2008). What does certification tell us about teacher effectiveness? Evidence from New York City. Economics of Education Review, 27(6), 615–631.

19.

Kane

T. J.

Staiger

D. O.

(2002). Volatility in school test scores: Implications for test-based accountability systems (Brookings Papers on Education Policy). Washington, DC: Brookings Institution.

20.

Koretz

D. M.

(2002). Limitations in the use of achievement tests as measures of educators’ productivity. The Journal of Human Resources, 37(4), 752–777.

21.

Lockwood

J. R.

McCaffrey

D. F.

Hamilton

L. S.

Stecher

Martinez

J. F.

(2007). The sensitivity of value-added teacher effect estimates to different mathematics achievement measures. Journal of Educational Measurement, 44(1), 47–67.

22.

McCaffrey

D. F.

Lockwood

J. R.

Koretz

D. M.

Hamilton

L. S.

(2003). Evaluating value-added models for teacher accountability. Santa Monica, CA: RAND.

23.

McCaffrey

D. F.

Lockwood

J. R.

Koretz

Louis

T. A.

Hamilton

(2004). Models for value-added modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29(1), 67.

24.

McCaffrey

D. F.

Sass

T. R.

Lockwood

J. R.

(2008). The intertemporal stability of teacher effect estimates. National Center for Performance Incentives, Working Paper No. 2008-22. Retrieved from http://www.performanceincentives.org/data/files/news/PapersNews/McCaffrey_et_al_2008.pdf

25.

McNeil

(2009). Starting gun sounds for “Race to the Top.” Education Week, 29(12), 1, 18–19.

26.

Murnane

R. J.

(1984). Selection and survival in the teacher labor market. The Review of Economics and Statistics, 66(3), 513–518.

27.

Olson

(2007). Houston board OKs revamped performance-pay plan. Education Week, 27(4), 11.

28.

Raudenbush

S. W.

(2004). What are value-added models estimating and what does this imply for statistical practice? Journal of Educational and Behavioral Statistics, 29(1), 121–129.

29.

Rivkin

S. G.

Hanushek

E. A.

Kain

J. F.

(2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417–458.

30.

Rockoff

J. E.

(2004). The impact of individual teachers on student achievement: Evidence from panel data. The American Economic Review, 94(2), 247–252.

31.

Rogosa

D. R.

Willett

J. B.

(1983). Demonstrating the reliability of the difference score in the measurement of change. Journal of Educational Measurement, 20(4), 335–343.

32.

Rothstein

(2007). Do value-added models add value? Tracking, fixed effects, and causal inference. Unpublished manuscript.

33.

Rubin

D. B.

Stuart

E. A.

Zanutto

E. L.

(2004). A potential outcomes view of value-added assessment in education. Journal of Educational and Behavioral Statistics, 29(1), 103–116.

34.

Sanders

(2000). Value-added assessment from student achievement data: Opportunities and hurdles. Journal of Personnel Evaluation in Education, 14(4), 329–339.

35.

Sanders

W. L.

Horn

S. P.

(1994). The Tennessee Value-Added Assessment System (TVAAS): Mixed-model methodology in educational assessment. Journal of Personnel Evaluation in Education, 8(3), 299–311.

36.

Sass

T. R.

(2008). The stability of value-added measures of teacher quality and implications for teacher compensation policy. National Center for Analysis of Longitudinal Data in Education Research Brief No. 4. Retrieved from http://www.urban.org/UploadedPDF/1001266_stabilityofvalue.pdf

37.

Tekwe

C. D.

Carter

R. L.

Algina

Lucas

M. E.

Roth

. (2004). An empirical comparison of statistical models for value-added assessment of school performance. Journal of Educational and Behavioral Statistics, 29(1), 11–35.

38.

Todd

P. E.

Wolpin

K. I.

(2003). On the specification and estimation of the production function for cognitive achievement. Economic Journal, 113(485), F3–33.

Different Tests,Different Answers

Abstract

Keywords

Get full access to this article

References