Policymakers are revolutionizing teacher evaluation by attaching greater stakes to student test scores and observation-based teacher effectiveness measures, but relatively little is known about why they often differ so much. Quantitative analysis of thirty schools suggests that teacher value-added measures and informal principal evaluations are positively, but weakly, correlated. Qualitative analysis suggests that some principals give high value-added teachers low ratings because the teachers exert too little effort and are “lone wolves” who work in isolation and contribute little to the school community. The results suggest that the method of evaluation may not only affect which specific teachers are rewarded in the short term, but shape the qualities of teacher and teaching students experience in the long term.
AaronsonD.BarrowL.SanderW. (2007). Teachers and student achievement in Chicago Public Schools. Journal of Labor Economics, 25:95–135.
2.
AbernathyT.ForsythA.MitchellJ. (2001). The bridge from student to teacher: What principals, teacher education faculty and students value in a teaching applicant. Teacher Education Quarterly, 28, 109–119.
3.
ArmorD.Conry-OsegueraP.CoxM.KingN. J.McDonnellL. M.PascalA. H.. . . ZellmanG. L. (1976). Analysis of the school preferred reading program in selected Los Angeles minority schools. Santa Monica, CA: Rand Corporation.
4.
BakerB. D.CooperB.S. (2005). Do principals with stronger academic backgrounds hire better teachers? Policy implications for improving high poverty schools. Education Administration Quarterly, 41, 413–448.
5.
BallouD. (1996). Do public schools hire the best applicants?The Quarterly Journal of Economics, 111, 97–133.
6.
BallouD.MokherC. G.CavalluzzoL. (2012). Using value-added assessment for personnel decisions: How omitted variables and model specification influence teacher’s outcomes. Unpublished manuscript.
7.
BidwellC. E. (2001). Schools as organizations: Long-term permanence and short-term change. Sociology of Education, 74, 100–114.
8.
BorghansL.ter WeelB.WeinbergB. (2008). Interpersonal styles and labor market outcomes. Journal of Human Resources, 43, 815–858.
9.
BraunJ.WillemsA.BrownM.GreenK. (1987). A survey of hiring practices in selected school districts. Journal of Teacher Education, 38, 45–49.
10.
BrobergJ. P. (1987). Ranking criteria for hiring newly certified teachers: A delphi technique (Unpublished doctoral dissertation). Oklahoma State University, Stillwater, OK.
11.
Cain-CastonM. (1999). A survey of opinions of North Carolina school administrators regarding factors considered most important in hiring teachers for their first teaching positions. Journal of Instructional Psychology, 26, 69–73.
12.
CampbellD. T. (1976). Assessing the impact of planned social change. Retrieved from ERIC database (ED303512).
13.
CarnoyM.LoebS. (2003). Does external accountability affect student outcomes? A cross-state analysis. Educational Evaluation and Policy Analysis, 24, 305–331.
14.
ClotfelterC.LaddH.VigdorJ. (2010). Teacher credentials and student achievement in high school: A cross-subject analysis with student fixed effects. Journal of Human Resources, 45, 655–681.
15.
CrossL. H.FraryR. B. (1999). Hodgepodge grading: Endorsed by students and teachers alike. Applied Measurement in Education, 12, 53–72.
16.
CunhaF.HeckmanJ. J.SchennachS. M. (2010). Estimating the technology of cognitive and noncognitive skill formation. Econometrica, 78, 883–931.
Darling-HammondL.Amrein-BeardsleyL.HaertelE.RothsteinJ. (2012). Evaluating teacher evaluation: Popular modes of evaluating teachers are fraught with inaccuracies and inconsistencies, but the field has identified better approaches. Phi Delta Kappan, 93(6), 8–15.
19.
DenzinN. K.LincolnY. S. (1998). Collecting and interpreting qualitative materials. Thousand Oaks, CA: Sage.
20.
DuntonJ. (2001). Selection criteria used by high school principals in Virginia when hiring first year career and technical teachers (Unpublished doctoral dissertation). Virginia Polytechnic Institute and State University, Blacksburg, VA.
21.
ElmoreR. F. (2000). Building a new structure for school leadership. Washington, DC: The Albert Shanker Institute.
22.
FenstermacherG. D.RichardsonV. (2007). On making determinations of quality in teaching. Teachers College Record, 107, 186–213.
23.
Florida Department of Education. (2005). Minority representation of Florida’s public school teachers, fall 2004. Tallahassee, FL: Author.
24.
GallagherH. A. (2004). Vaughan Elementary’s innovative teacher evaluation system: Are teacher evaluation scores related to growth in student achievement. Peabody Journal of Education, 79, 79–107.
25.
GaretM. S.CronenS.EatonM.KurkiA.LudwigM.JonesW.. . . SztejnbergL. (2008). The impact of two professional development interventions on early reading instruction and achievement (NCEE 2008-4031). Washington, DC: U.S. Department of Education.
26.
GaretM.WayneA.StancavageF.TaylorJ.WaltersK.SongM.. . . DoolittleF. (2010). Middle school mathematics professional development impact study: Findings after the first year of implementation (NCEE 2010-4009). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
GoldhaberD.AnthonyE. (2007). Can teacher quality be effectively assessed? National board certification as a signal of effective teaching. Review of Economics and Statistics, 89, 134–150.
29.
GordonR.KaneT.StaigerO. (2006). Identifying effective teachers using performance on the job. Washington, DC: The Brookings Institution.
30.
GrissomJ.LoebS. (2011). Triangulating principal effectiveness: How perspectives of parents, teachers, and assistant principals identify the central importance of managerial skills. American Education Research Journal, 48, 1091–1123.
HallingerP. (2005). Instructional leadership and the school principal: A passing fancy that refuses to fade away. Leadership and Policy in Schools, 4, 221-239.
33.
HallingerP.HeckR.H. (1996). Reassessing the principal’s role in school effectiveness: A review of empirical research, 1980–1995. Educational Administration Quarterly, 32, 5–44.
34.
HanushekE. A. (2011). The economic value of higher teacher quality. Economics of Education Review, 30, 466–479.
35.
HanushekE. A.RivkinS. G. (2010). Using value-added measures of teacher quality. Washington, DC: The Urban Institute.
36.
HarrisD.AndersonA. (2012). Bias of public sector worker performance monitoring: Theory and empirical evidence from middle school teachers. Paper presented at the 2012 annual meeting of the Association for Education Finance and Policy.
37.
HarrisD.RutledgeS. (2010). Models and predictors of teacher effectiveness: A review of the evidence with lessons from (and for) other occupations. Teachers College Record, 112, 914–960.
38.
HarrisD.RutledgeS.IngleW.ThompsonC. (2010). Mix and match: What principals really look for when hiring teachers. Education Finance and Policy, 5, 228–246.
39.
HarrisD.SassT. (2006). Value-added models and the measurement of teacher productivity. Paper presented at the annual meeting of the American Education Finance Association.
40.
HarrisD.SassT. (2009a). The effects of NBPTS-certified teachers on student achievement. Journal of Policy Analysis and Management, 28, 55–80.
41.
HarrisD.SassT. (2009b). What makes for a good teacher and who can tell? (CALDER Working Paper No. 30). Washington, DC: Urban Institute.
42.
HarrisD. N.SassT. R. (2011). Teacher training, teacher quality and student achievement. Journal of Public Economics, 95, 798–812.
43.
IngleW. K.RutledgeS. A.BishopJ. L. (2011). Context matters: Principals’ sensemaking of teacher hiring and on-the-job performance. Journal of Educational Administration, 49, 579–610.
44.
JacksonC. K. (in press). Teacher quality at the high-school level: The importance of accounting for tracks. Journal of Labor Economics.
45.
JacobB. A.LefgrenL. (2004). The impact of teacher training on student achievement: Quasi-experimental evidence from school reform efforts in Chicago. Journal of Human Resources, 39, 50–79.
46.
JacobB. A.LefgrenL. (2008). Can principals identify effective teachers? Evidence on subjective performance evaluation in education. Journal of Labor Economics, 26, 101–136.
47.
KaiserH. F. (1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23, 187–200.
48.
KaneT. J.McCaffreyD. M.MillerT.StaigerD. O. (2013). Have we identified effective teachers? Validating measures of effective teaching using random assignment. Seattle, WA: Bill and Melinda Gates Foundation.
49.
KaneT.StaigerD. (2008). Estimating teacher impacts on student achievement: An experimental evaluation (NBER Working Paper No. 14607). Cambridge, MA: National Bureau of Economic Research, Inc.
KennedyM. M. (2008). Sorting out teacher quality. Phi Delta Kappan, 90(1), 59–63.
52.
KennedyM. (2010). Introduction: The uncertain relationship between teacher assessment and teacher quality. In KennedyM. (Ed.), Teacher assessment and the quest for teacher quality: A handbook (pp. 9–42). San Francisco, CA: Jossey-Bass.
53.
KimballS. M.MilanowskiA. (2004). Examining the relationship between teacher evaluation and student assessment results in Washoe County. Peabody Journal of Education, 79(4), 54–78.
54.
LittleO. (2009). Teacher evaluation systems: The window for opportunity and reform. Washington, DC: National Education Association.
55.
LiuE.JohnsonS. M. (2006). New teachers’ experiences of hiring: Late, rushed, and information-poor. Educational Administration Quarterly, 42, 324–360.
56.
LouisK. S.LeithwoodK.WahlstromK.AndersonS. (2010). Investigating the links to improved student learning: Final report of research findings. St. Paul, MN: University of Minnesota Center for Applied Research and Educational Improvement.
57.
MarzanoR.WatersT.McNultyB. (2011). Leadership that works: From research to results. Alexandria, VA: ASCD.
58.
McCaffreyD. F.LockwoodJ. R.KoretzD. M.HamiltonL. S. (2003). Evaluating value-added models for teacher qccountability. Santa Monica, CA: RAND Corporation.
59.
MedleyD. M.CokerH. (1987). The accuracy of principals’ judgments of teacher performance. Journal of Educational Research, 80, 242–247.
60.
MetzgerS. A.WuM. J. (2008). Commercial teacher selection instruments: The validity of selecting teachers through beliefs, attitudes, and values. Review of Educational Research, 78, 921–940.
61.
MilanowskiA. (2004). The relationship between teacher performance evaluation scores and student assessment: Evidence from Cincinnati. Peabody Journal of Education, 79, 33–53.
62.
MilesM. B.HubermanA. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks, CA: Sage.
63.
MonkD. H. (1994). Subject matter preparation of secondary mathematics and science teachers and student achievement. Economics of Education Review, 13, 125–145.
64.
MurnaneR. J. (1975). The impact of school resources on the learning of inner-city children. Cambridge, MA: Ballinger.
65.
NewmannF. M.SmithB.AllensworthE.BrykA. S. (2001). Instructional program coherence: What it is and why it should guide school improvement policy. Educational Evaluation and Policy Analysis, 23, 297–321.
66.
OnwuegbuzieA. J.WitcherA. E.CollinsK. M. T.FilerJ. D.WiedmaierC. D.MooreC. W. (2007). Students’ perceptions of characteristics of effective teachers: A validity study of a teaching evaluation form using mixed methods analysis. American Educational Research Journal, 44, 113–160.
67.
PainterS. R. (2000). Principals’ efficacy beliefs about teacher evaluation. Journal of Educational Administration, 38, 368–378.
68.
ParsonsT. (1960). Structure and process in modern societies. Glencoe, IL: Free Press.
69.
PetersonK. (1987). Teacher evaluation with multiple and variable lines of evidence. American Educational Research Journal, 24, 311–317.
70.
PetersonK. D. (2000). Teacher evaluation: A comprehensive guide to new directions and practices. Thousand Oaks, CA: Corwin Press.
71.
RalphE. G.KestenC.LangH.SmithD. (1998). Hiring new teachers: What do school districts look for?Journal of Teacher Education, 49, 1–10.
72.
RiceJ. K. (2003). Teacher quality: Understanding the effectiveness of teacher attributes. Washington, DC: Economic Policy Institute.
73.
RivkinS.HanushekE. A.KainJ. F. (2005). Teacher schools, and academic achievement. Econometrica, 73, 417–458.
74.
RockoffJ. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review, 94, 247–252.
75.
RockoffJ. E.StaigerD. O.KaneT. J.TaylorE. S. (2010). Information and employee evaluation: Evidence from a randomized intervention in public schools (NBER Working Paper No. 16240). Cambridge, MA: National Bureau of Economic Research.
76.
RothsteinJ. (2009). Student sorting and bias in value added estimation: Selection on observables and unobservables (NBER Working Paper No. 14666). Cambridge, MA: National Bureau of Economic Research, Inc.
77.
RutledgeS. A.HarrisD. N.ThompsonC. T.IngleW. K. (2008). Certify, blink, hire: An examination of the process and tools of teacher screening and selection. Leadership and Policy in Schools, 7, 237–263.
78.
RutledgeS. A.HarrisD. N.IngleW. K. (2010). How principals “bridge and buffer” the new demands of teacher quality and accountability. A mixed-methods analysis of teacher hiring. American Journal of Education, 116, 211–242.
79.
SchmidtF. L.HunterJ. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274.
80.
ShulmanL. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1–22.
81.
SnyderT. D.DillowS. A.HoffmanC. M. (2007). Digest of Education Statistics 2006 (NCES 2007-017). Washington, DC: U.S. Government Printing Office.
82.
StodoloskyS. (1984). Teacher evaluation: The limits of looking. Educational Researcher, 13(9), 11–18.
83.
StraussR. P.VogtW. B. (2006). Should teachers know, or know how to teach?Revised version of paper presented at the 2001 Annual Meeting of the American Educational Finance Association.
84.
TheelR. K.TallericoM. (2004). Using portfolios for teacher hiring: Insights from principals. Action in Teacher Education, 26(1), 26–33.
85.
ToddP. E.WolpinK. I. (2003). On the specification and estimation of the production function for cognitive achievement. The Economic Journal, 113, F3–F33.
86.
WayneA. J.YoungsP. (2003). Teacher characteristics and student achievement gains: A review. Review of Educational Research, 73, 89–122.
WilsonS.FlodenR. (2003). Creating effective teachers: Concise answers for hard questions. New York, NY: AACTE Publications.
89.
WilsonS.FlodenR.Ferrini-MundyJ. (2001). Teacher preparation research: Current knowledge, gaps, and recommendations. Seattle, WA: Center for the Study of Teaching and Policy, University of Washington.
90.
ZwickW. R.VelicerW. F. (1986). Comparison of five rules for determining the number of components to retain. Psychological Bulletin, 99, 432–442.
