Sage Journals: Discover world-class research

Abstract

Most education agencies have implemented new teacher evaluation systems that promise to improve teacher performance. Post-observation performance feedback is a theoretically important driver of this promise as it should ultimately develop teacher-specific weaknesses. This is the first large-scale study to use the written feedback provided to early-career teachers during formal post-observation conferences and quantitatively link critical feedback characteristics (CFCs) to measures of teacher human capital. We find that most conferences do not include CFCs, that feedback is typically unidimensional, and that less effective early-career teachers receive higher shares of CFCs. However, goal-setting is the only CFC associated with subsequent teacher performance. Beginning and less-educated teachers, for whom goal-setting may clarify performance expectations, drive this relationship.

Keywords

evaluation school/teacher effectiveness supervision educational policy mixed methods qualitative research regression analyses

Get full access to this article

View all access options for this article.

References

Alexander

(2016). TEAM evaluator training [Certification training, TEAM evaluator training 2016-17]. http://team-tn.org/wp-content/uploads/2013/08/TEAM-Teacher-Training-2016_FINAL_PDF.pdf

Archer

Cantrell

Holtzman

Joe

Tocci

Wood

(2016). Better feedback for better teaching: A practical guide to improving classroom observations (1st ed.). Jossey-Bass.

Bacher-Hicks

Chin

Kane

Staiger

(2017). An evaluation of bias in three measures of teacher quality: Value-added, classroom observations, and student surveys (No. w23478). National Bureau of Economic Research. https://doi.org/10.3386/w23478

Ballou

Springer

M. G.

(2015). Using student test scores to measure teacher performance: Some problems in the design and implementation of evaluation systems. Educational Researcher, 44(2), 77–86. https://doi.org/10.3102/0013189X15574904

Bartholomew

Henderson

A. J. Z.

Marcia

J. E.

(2000). Coded semi-structured interviews in social psychological research. In Reis

H. T.

Judd

C. M.

(Eds.), Handbook of Research Methods in Social and Personality Psychology (pp. 286–312). Cambridge University Press.

Belur

Tompson

Thornton

Simon

(2021). Interrater reliability in systematic review methodology: Exploring variation in coder decision-making. Sociological Methods & Research, 50, 837–865. https://doi.org/10.1177/0049124118799372

Bleiberg

Harbatkin

(2020). Teacher evaluation reform: A convergence of federal and local forces. Educational Policy, 34(6), 918–952.

Brinko

K. T.

(1993). The practice of giving feedback to improve teaching. Journal of Higher Education, 64(5), 574–593.

Brophy

Good

T. L.

(1986). Teacher behavior and student achievement. In Wittrock

M. C.

(Ed.), Handbook of research on teaching (3rd ed., pp. 328–375). Macmillan.

10.

Campbell

S. L.

Ronfeldt

(2018). Observational evaluation of teachers: Measuring more than we bargained for? American Educational Research Journal, 55, 1233–1267. https://doi.org/10.3102/0002831218776216

11.

Cannata

Rubin

Goldring

Grissomn

J. A.

Neumerski

C. M.

Drake

T. A.

Schuermann

. (2017). Using teacher effectiveness data for information-rich hiring. Educational Administration Quarterly, 53(2), 180–222.

12.

Cannon

M. D.

Witherspoon

(2005). Actionable feedback: Unlocking the power of learning and performance improvement. Academy of Management Perspectives, 19(2), 120–134. https://doi.org/10.5465/ame.2005.16965107

13.

Carey

J. W.

Gelaude

(2008). Systematic methods for collecting and analyzing multidisciplinary team-based qualitative data. In Guest

MacQueen

K. M.

(Eds.), Handbook for team-based qualitative research (pp. 227–274). Altamira Press.

14.

Carver

C. S.

Scheier

M. F.

(1982). Control theory: A useful conceptual framework for personality-social, clinical, and health psychology. Psychological Bulletin, 92(1), 25.

15.

Cherasaro

T. L.

Brodersen

R. M.

Reale

M. L.

Yanoski

D. C.

(2016). Teachers’ responses to feedback from evaluators: What feedback characteristics matter? (REL 2017-190; Making Connections, pp. 1–29). REL Central.

16.

Chetty

Friedman

J. N.

Rockoff

J. E.

(2014). The long term impacts of teachers: Teacher value added and student outcomes in adulthood. American Economic Review, 104(9), 2633–2679.

17.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.

18.

Cohen-Vogel

(2011). “Staffing to the test”: Are today’s school personnel practices evidence based? Educational Evaluation and Policy Analysis, 33(4), 483–505. https://doi.org/10.3102/0162373711419845

19.

Cohen-Vogel

Tichnor-Wagner

Allen

Harrison

Kainz

Socol

A. R.

Wang

(2015). Implementing educational innovations at scale: Transforming researchers into continuous improvement scientists. Educational Policy, 29(1), 257–277. https://doi.org/10.1177/0895904814560886

20.

Cook

J. R.

Stefanski

L. A.

(1994). Simulation-extrapolation estimation in parametric measurement error models. Journal of the American Statistical Association, 89(428), 1314–1328.

21.

Cornelius

K. E.

Nagro

S. A.

(2014). Evaluating the evidence base of performance feedback in preservice special education teacher training. Teacher Education and Special Education: The Journal of the Teacher Education Division of the Council for Exceptional Children, 37(2), 133–146. https://doi.org/10.1177/0888406414521837

22.

Daley

Kim

(2010). A teacher evaluation system that works [Working paper]. National Institute for Excellence in Teaching.

23.

de Barros

. (2019). Evaluating teacher evaluation: Evidence from Chile. Organization of schools and systems & education in global contexts. Society for Research in Educational Effectiveness.

24.

Dee

T. S.

Cohodes

S. R.

(2008). Out-of-field teachers and student achievement: evidence from matched-pairs comparisons. Public Finance Review, 36(1), 7–32. https://doi.org/10.1177/109114210628933032.

25.

DeNisi

A. S.

Murphy

K. R.

(2017). Performance appraisal and performance management: 100 years of progress? Journal of Applied Psychology, 102(3), 421–433. https://doi.org/10.1037/apl0000085.supp

26.

Doan

(2019). What do classroom observation scores tell us about student success? Capturing the impact of teachers using at-scale classroom observation scores. Vanderbilt University. https://ir.vanderbilt.edu/handle/1803/15448

27.

Donaldson

M. L.

(2021). Multidisciplinary perspectives on teacher evaluation: Understanding the research and theory (1st ed.). Routledge.

28.

Donaldson

M. L.

Cobb

C. D.

LeChasseur

Gabriel

Gonzales

Woulfin

Makuch

(2014). An evaluation of the pilot implementation of Connecticut’s system for educator evaluation and development. UConn Center for Education Policy Analysis.

29.

Donaldson

M. L.

Woulfin

(2018). From tinkering to going “rogue”: How principals use agency when enacting new teacher evaluation systems. Educational Evaluation and Policy Analysis, 40(4), 531–556. https://doi.org/10.3102/0162373718784205

30.

Feeney

E. J.

(2007). Quality feedback: The essential ingredient for teacher success. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 80(4), 191–198. https://doi.org/10.3200/TCHS.80.4.191-198

31.

Fesler

Dee

Baker

Evans

(2019). Text as data methods for education research. Journal of Research on Educational Effectiveness, 12(4), 707–727. https://doi.org/10.1080/19345747.2019.1634168

32.

Finster

Milanowski

(2018). Teacher perceptions of a new performance evaluation system and their influence on practice: A within- and between-school level analysis. Education Policy Analysis Archives, 26, 41. https://doi.org/10.14507/epaa.26.3500

33.

Gallo

(2011, February 7). Making sure your employees succeed. Harvard Business Review. https://hbr.org/2011/02/making-sure-your-employees-suc

34.

Garet

M. S.

Porter

A. C.

Desimone

Birman

B. F.

Yoon

K. S.

(2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915–945. https://doi.org/10.3102/00028312038004915

35.

Garrett

Steinberg

M. P.

(2015). Examining teacher effectiveness using classroom observation scores: Evidence from the randomization of teachers to students. Educational Evaluation and Policy Analysis, 37(2), 224–242. https://doi.org/10.3102/0162373714537551

36.

Gershenson

(2016). Linking teacher quality, student attendance, and student achievement. Education Finance and Policy, 11(2), 125–149. https://doi.org/10.1162/EDFP_a_00180

37.

Glickman

Gordon

Ross-Gordon

(2018). Supervision and instructional leadership (10th ed.). Pearson.

38.

Goldring

Grissom

J. A.

Rubin

Neumerski

C. M.

Cannata

Drake

Schuermann

(2015). Make room value added: Principals’ human capital decisions and the emergence of teacher observation data. Educational Researcher, 44(2), 96–104. https://doi.org/10.3102/0013189X15575031

39.

Grissom

J. A.

Bartanen

(2018). Strategic retention: Principal effectiveness and teacher turnover in multiple-measure teacher evaluation systems. American Educational Research Journal, 56(2), 514–555. https://doi.org/10.3102/0002831218797931

40.

Grissom

J. A.

Kalogrides

Loeb

(2017). Strategic staffing? How performance pressures affect the distribution of teachers within schools and resulting student achievement. American Educational Research Journal, 54(6), 1079–1116. https://doi.org/10.3102/0002831217716301

41.

Grissom

J. A.

Loeb

(2017). Assessing principals’ assessments: Subjective evaluations of teacher effectiveness in low- and high-stakes environments. Education Finance and Policy, 12, 369–395.

42.

Hagen

M. S.

(2012). Managerial coaching: A review of the literature. Performance Improvement Quarterly, 24(4), 17–39. https://doi.org/10.1002/piq

43.

Hardin

J. W.

Carroll

R. J.

(2003). Measurement error, GLMs, and notational conventions. The Stata Journal, 3(3), 329–341.

44.

Hardin

J. W.

Schmiediche

Carroll

R. J.

(2003). The simulation extrapolation method for fitting generalized linear models with additive measurement error. The Stata Journal, 3(4), 373–385.

45.

Harkins

S. G.

Lowe

M. D.

(2000). The effects of self-set goals on task performance. Journal of Applied Social Psychology, 30(1), 1–40. https://doi.org/10.1111/j.1559-1816.2000.tb02303.x

46.

Harris

D. N.

Sass

T. R.

(2011). Teacher training, teacher quality and student achievement. Journal of Public Economics, 95(7–8), 798–812. https://doi.org/10.1016/j.jpubeco.2010.11.009

47.

Harris

D. N.

Sass

T. R.

(2014). Skills, productivity and the evaluation of teacher performance. Economics of Education Review, 40, 183–204. https://doi.org/10.1016/j.econedurev.2014.03.002

48.

Hattie

Timperley

(2016). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

49.

Hemmeter

M. L.

Snyder

Kinder

Artman

(2011). Impact of performance feedback delivered via electronic mail on preschool teachers’ use of descriptive praise. Early Childhood Research Quarterly, 26(1), 96–109. https://doi.org/10.1016/j.ecresq.2010.05.004

50.

Hill

H. C.

Grossman

(2013). Learning from teacher observations: Challenges and opportunities posed by new teacher evaluation systems. Harvard Educational Review, 83(2), 371–385. https://doi.org/10.1017/CBO9781107415324.004

51.

Hitt

D. H.

Tucker

P. D.

(2016). Systematic review of key leader practices found to influence student achievement: A unified framework. Review of Educational Research, 86(2), 531–569.

52.

Huff

Preston

Goldring

(2013). Implementation of a coaching program for school principals: Evaluating coaches’ strategies and the results. Educational Management Administration & Leadership, 41(4), 504–526. https://doi.org/10.1177/1741143213485467

53.

Hunter

S. B.

(2019). The effects of more frequent observations on student achievement scores (No. 2019–04; Strengthening Tennessee’s Education Labor Market). Tennessee Education Research Alliance. https://peabody.vanderbilt.edu/TERA/files/TERA_Working_Paper_2019-04.pdf

54.

Hunter

S. B.

(2020). The unintended effects of policy-assigned teacher observations: Examining the validity of observation scores. AERA Open. 6(2). Advance online publication. https://doi.org/10.1177/2332858420929276

55.

Hunter

S. B.

(2021). Do you mean what I mean? Comparing teacher performance self-scores and evaluator-generated scores. Journal of Education Human Resources, e20200026. https://doi.org/10.3138/jehr-2020-0026

56.

Hunter

S. B.

Ege

(2021). Linking student outcomes to school administrator discretion in the implementation of teacher observations. Educational Administration Quarterly, 57, 607–640. https://doi.org/10.1177/0013161X211003134

57.

Hunter

S. B.

Rodriguez

L. A.

(2021). Examining the demands of teacher evaluation: Time use, strain and turnover among Tennessee school administrators. Journal of Educational Administration, 59, 739–758.

58.

Ilgen

D. R.

Fisher

C. D.

Taylor

M. S.

(1979). Consequences of individual feedback on behavior in organizations. Journal of Applied Psychology, 64(4), 349–371. https://doi.org/10.1037/0021-9010.64.4.349

59.

Ivancevich

J. M.

(1982). Subordinates’ reactions to performance appraisal interviews: A test of feedback and goal-setting techniques. Journal of Applied Psychology, 67(5), 581–587. https://doi.org/10.1037/0021-9010.67.5.581

60.

Jackson

C. K.

(2018). What do test scores miss? The importance of teacher effects on non–test score outcomes. Journal of Political Economy, 126(5), 36.

61.

Jawahar

I. M.

(2010). The mediating role of appraisal feedback reactions on the relationship between rater feedback-related behaviors and ratee performance. Group & Organization Management, 35(4), 494–526. https://doi.org/10.1177/1059601110378294

62.

Kane

T. J.

Taylor

E. S.

Tyler

J. H.

Wooten

A. L.

(2011). Identifying effective classroom practices using student achievement data. The Journal of Human Resources, 46(3), 587–613.

63.

Kimball

S. M.

(2003). Analysis of feedback, enabling conditions and fairness perceptions of teachers in three school districts with new standards-based evaluation systems. Journal of Personnel Evaluation in Education, 16(4), 241–268. https://doi.org/10.1023/A:1021787806189

64.

Kimball

S. M.

Milanowski

(2009). Examining teacher evaluation validity and leadership decision making within a standards-based evaluation system. Educational Administration Quarterly, 45(1), 34–70.

65.

Kinicki

A. J.

Prussia

G. E.

B. J.

McKee-Ryan

F. M.

(2004). A covariance structure analysis of employees’ response to performance feedback. Journal of Applied Psychology, 89(6), 1057–1069. https://doi.org/10.1037/0021-9010.89.6.1057

66.

Koedel

Springer

M. G.

Tan

(2019). Teacher performance ratings and professional improvement.

67.

Journal of Research on Educational Effectiveness, 12(1), 90–115.

68.

Kraft

M. A.

Christian

(2021). Can teacher evaluation systems produce high-quality feedback? An administrator training field experiment. American Educational Research Journal. Advance online publication. https://doi.org/10.3102/00028312211024603

69.

Kraft

M. A.

Gilmour

A. F.

(2016). Can principals promote teacher development as evaluators? A case study of principals’ views and experiences. Educational Administration Quarterly, 52(5), 711–753. https://doi.org/10.1177/0013161X16653445

70.

Kraft

M. A.

Papay

J. P.

Chi

(2020). Teacher skill development: Evidenced from performance ratings by principals. Journal of Policy Analysis and Management, 39(2), 315–347.

71.

Ladd

H. F.

Sorensen

L. C.

(2017). Returns to teacher experience: Student achievement and motivation in middle school. Education Finance and Policy, 12(2), 241–279. https://doi.org/10.1162/EDFP_a_00194

72.

Linderbaum

B. A.

Levy

P. E.

(2010). The development and validation of the Feedback Orientation Scale (FOS). Journal of Management, 36(6), 1372–1405.

73.

Liu

Bell

C. A.

Jones

N. D.

McCaffrey

D. F.

(2019). Classroom observation systems in context: A case for the validation of observation systems. Educational Assessment, Evaluation and Accountability, 31(1), 61–95. https://doi.org/10.1007/s11092-018-09291-3

74.

Locke

E. A.

Latham

G. P.

(2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57(9), 705–717. https://doi.org/10.1037/0003-066X.57.9.705

75.

Lockwood

J. R.

McCaffrey

D. F.

(2020). Recommendations about estimating errors-in-variables regression in Stata. The Stata Journal: Promoting Communications on Statistics and Stata, 20(1), 116–130. https://doi.org/10.1177/1536867X20909692

76.

London

Smither

J. W.

(2002). Feedback orientation, feedback culture, and the longitudinal performance management process. Human Resource Management Review, 12(1), 81–100. https://doi.org/10.1016/S1053-4822(01)00043-2

77.

Long

(2019). Teachers’ perceptions and experiences with a reformed teacher evaluation system: Conditions necessary for changing practice [Doctoral dissertation]. The Pennsylvania State University. https://etda.libraries.psu.edu/files/final_submissions/18672

78.

Marques

J. F.

McCall

(2005). The application of interrater reliability as a solidification instrument in a phenomenological study. The Qualitative Report, 10(3), 449.

79.

Marsh

J. A.

Bush-Mecenas

Strunk

K. O.

Lincove

J. A.

Huguet

(2017). Evaluating teachers in the big easy: How organizational context shapes policy responses in New Orleans. Educational Evaluation and Policy Analysis, 39(4), 539–570. https://doi.org/10.3102/0162373717698221

80.

National Council on Teacher Quality. (2019). NCTQ: Yearbook: State teacher policy database. https://www.nctq.org/yearbook/home

81.

Neumerski

C. M.

Grissom

J. A.

Goldring

Drake

T. A.

Rubin

Cannata

Schuermann

(2018). Restructuring instructional leadership: How multiple-measure teacher evaluation systems are redefining the role of the school principal. The Elementary School Journal, 119(2), 28.

82.

Odden

Kelly

J. A.

(2008). Strategic management of human capital in public education. Strategic Management of Human Capital.

83.

Papay

J. P.

(2012). Refocusing the debate: Assessing the purposes and tools of teacher evaluation. Harvard Educational Review, 82(1), 123–141. https://doi.org/10.17763/haer.82.1.v40p0833345w6384

84.

Papay

J. P.

Kraft

M. A.

(2015). Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence on long-term career improvement. Journal of Public Economics, 130, 105–119. https://doi.org/10.1016/j.jpubeco.2015.02.008

85.

Penuel

W. R.

Fishman

B. J.

Yamaguchi

Gallagher

L. P.

(2007). What makes professional strategies development effective? Strategies that foster curriculum implementation. American Educational Research Journal, 44(4), 921–958. https://doi.org/10.3102/0002831207308221

86.

Phipps

A. R.

(2018). Incentive contracts in complex environments: Theory and evidence on effective teacher performance incentives [Doctoral dissertation]. LibraETD, University of Virginia. https://libraetd.lib.virginia.edu/public_view/qn59q448d

87.

Phipps

A. R.

Wiseman

E. A.

(2021). Enacting the rubric: Teacher improvements in windows of high-stakes observation. Education Finance and Policy, 16(2), 283–312. https://doi.org/10.1162/edfp_a_00295

88.

Rigby

J. G.

(2015). Principals’ sensemaking and enactment of teacher evaluation. Journal of Educational Administration, 53(3), 374–392. https://doi.org/10.1108/JEA-04-2014-0051

89.

Rowan

Raudenbush

S. W.

(2016). Teacher evaluation in American schools. In Handbook of Research on Teaching (5th ed., pp. 1159–1216). American Educational Research Association.

90.

Sanders

W. L.

Rivers

J. C.

(1996). Cumulative and residual effects on teachers on future student academic achievement. Research Progress Report. Knoxville: University of Tennessee Value-Added Research and Assessment Center. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.474.3738&rep=rep1&type=pdf

91.

Sartain

Zou

Gutierrez

Shyjka

Hinton

Brown

E. R.

Easton

J. W.

(2020). Teacher evaluation in CPS: Perceptions of REACH implementation, five years in. Research Brief. University of Chicago Consortium on School Research.

92.

SAS Analytics Software. (2015). Technical documentation for 2015 TVAAS analyses 1.1. https://www.tn.gov/education/data/tvaas.html

93.

Scheeler

M. C.

Ruhl

K. L.

McAfee

J. K.

(2004). Providing performance feedback to teachers: A review. Teacher Education and Special Education: The Journal of the Teacher Education Division of the Council for Exceptional Children, 27(4), 396–407. https://doi.org/10.1177/088840640402700407

94.

Steinberg

M. P.

Donaldson

M. L.

(2016). The new educational accountability: Understanding the landscape of teacher evaluation in the post-NCLB era. Education Finance and Policy, 11(3), 340–359. https://doi.org/10.1162/EDFP_a_00186

95.

Steinberg

M. P.

Garrett

(2016). Classroom composition and measured teacher performance: What do teacher observation scores really measure? Educational Evaluation and Policy Analysis, 38, 293–317. https://doi.org/10.3102/0162373715616249

96.

Steinberg

M. P.

Sartain

(2015). Does teacher evaluation improve school performance? Experimental evidence from Chicago’s Excellence in Teaching Project. Education Finance and Policy, 10(4), 535–572. https://doi.org/10.1162/EDFP_a_00173

97.

Sun

Mutcheson

R. B.

Kim

(2016). Teachers’ use of evaluation for instructional improvement and school supports for such use. In Grissom

Youngs

(Eds.), Improving teacher evaluation systems: Making the most of multiple measures (1st ed., pp. 102–116). Teachers College Press.

98.

Teacher and Principal Evaluation Policy. (2013). https://web.archive.org/web/20140123232134/http://www.tn.gov/sbe/Policies/5.201_Teacher_and_Principal_Evaluation_Policy_11-5-13.pdf

99.

Tennessee Department of Education. (2016). Evaluation—TEAM-TN. https://web.archive.org/web/20161105020918/http://team-tn.org/evaluation/

100.

Thurlings

Vermeulen

Bastiaens

Stijnen

(2013). Understanding feedback: A learning theory perspective. Educational Research Review, 9, 1–15. https://doi.org/10.1016/j.edurev.2012.11.004

101.

Tuytens

Devos

(2011). Stimulating professional learning through teacher evaluation: An impossible task for the school leader? Teaching and Teacher Education, 27(5), 891–899. https://doi.org/10.1016/j.tate.2011.02.004

102.

Tuytens

Devos

(2013). How to activate teachers through teacher evaluation? School Effectiveness and School Improvement, 25(4), 509–530. https://doi.org/10.1080/09243453.2013.842601

103.

Wayne

A. J.

Youngs

(2003). Teacher characteristics and student achievement gains: A review. Review of Educational Research, 73(1), 89–122. https://doi.org/10.3102/00346543073001089

104.

Weisberg

Sexton

Mulhern

Keeling

(2009). The Widget Effect. http://tntp.org/assets/documents/TheWidgetEffect_2nd_ed.pdf

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

2.99 MB

Critical Feedback Characteristics,Teacher Human Capital,and Early-Career Teacher Performance: A Mixed-Methods Analysis

Abstract

Keywords

Get full access to this article

References

Supplementary Material