What Can Student Perception Surveys Tell Us About Teaching? Empirically Testing the Underlying Structure of the Tripod Student Perception Survey

Abstract

We conducted a theory-based analysis of the underlying structure of the Tripod student perception survey instrument using the Measures of Effective Teaching (MET) database (N = 1,049 middle school math class sections; N = 25,423 students). Multilevel item factor analyses suggested that an alternative bifactor structure best fit the Tripod items, and preliminary evidence suggests that both the general responsivity and the classroom management–specific dimensions are positively associated with teacher value-added scores. In our discussion, we consider the distinct characterizing features of adolescents as raters of teaching, the implications for teacher professional learning opportunities, and key areas for future research.

Keywords

multilevel confirmatory factor analysis student perceptions teacher evaluation

Get full access to this article

View all access options for this article.

References

Brophy

J. E.

Good

T. L.

(1974). Teacher-student relationships: Causes and consequences. Oxford, England: Holt, Rinehart & Winston.

Cai

(2010). A two-tier full-information item factor analysis model with applications. Psychometrika, 75(4), 581–612. doi:10.1007/s11336-010-9178-0

Cai

Yang

Hansen

(2011). Generalized full-information item bifactor analysis. Psychological Methods, 16, 221–248.

Carlisle

Kelcey

Berebitsky

Phelps

(2011). Embracing the complexity of instruction: A study of the effects of teachers’ instruction on students’ reading comprehension. Scientific Studies of Reading, 15, 409–439. doi:10.1080/10888438.2010.497521

Cartledge

Vincent

Robinson-Ervin

(2015). Culturally responsive classroom management. In Evertson

Weinstein

(Eds.), Handbook of classroom management (2nd Ed.). New York, New York: Routledge Publications.

Cholewa

Amatea

West-Olatunji

C. A.

Wright

(2012). Examining the relational processes of a highly successful teacher of African American children. Urban Education, 47(1), 250–279. doi:10.1177/0042085911429581

Davis

H. A.

(2003). Conceptualizing the role and influence of student-teacher relationships on children’s social and cognitive development. Educational Psychologist, 38(4), 207–234. doi:10.1207/S15326985EP3804_2

Desimone

Smith

Frisvold

(2010). Survey measures of classroom instruction: Comparing student and teacher reports. Educational Policy, 24(2), 267–329. doi:10.1177/0895904808330173

Emmer

E. T.

Strough

(2001). Classroom management: A critical part of educational psychology, with implications for teacher education. Educational Psychologist, 36(2), 103–112. doi:10.1207/S15326985EP3602_5

10.

Erickson

(1987). Transformation and school success: The politics and culture of educational achievement. Anthropology and Education Quarterly, 18(4), 335–356. doi:10.1525/aeq.1987.18.4.04x0023w

11.

Erickson

Bagrodia

Cook-Sather

Espinoza

Jurow

Shultz

J. J.

Spencer

(2007). Students’ experience of school curriculum: The everyday circumstances of granting and withholding assent to learn. In Connelly

F. M.

M. F.

Phillion

(Eds.), The Sage Handbook of Curriculum and Instruction (pp. 198–218). Thousand Oaks, CA: Sage.

12.

Fauth

Decristan

Rieser

Klieme

Büttner

(2014). Student ratings ofteaching quality in primary school: Dimensions and prediction of student outcomes. Learning and Instruction, 29, 1–9. doi:10.1016/j.learninstruc.2013 .07.001

13.

Ferguson

(2010, October14). Student perceptions of teaching effectiveness. Discussion brief from the National Center for Teacher Effectiveness and the Achievement Gap Initiative, Harvard University, Cambridge, MA.

14.

Ferguson

R. F.

Danielson

(2014). How framework for teaching and Tripod 7Cs evidence distinguish key components of effective teaching. In Kane

Kerr

Pianta

(Eds.), Designing teacher evaluation systems: New guidance from the Measures of Effective Teaching project (pp. 98–143). San Francisco: Jossey-Bass.

15.

Friedrich

Flunger

Nagengast

Jonkmann

Trautwein

(2015). Pygmalion effects in the classroom: Teacher expectancy effects on students’ math achievement. Contemporary Educational Psychology, 41, 1–12. doi:10 .1016/j.cedpsych.2014.10.006

16.

Furrer

Skinner

(2003). Sense of relatedness as a factor in children’s academic engagement and performance. Journal of Educational Psychology, 95(1), 148–162. doi:10.1037/0022-0663.95.1.148

17.

Gay

(2010). Culturally responsive teaching: Theory, research, and practice. NewYork: Teachers College Press.

18.

Gest

S. D.

Rodkin

P. C.

(2011). Teaching practices and elementary classroom peer ecologies. Journal of Applied Developmental Psychology, 32(5), 288–296. doi:10.1016/j.appdev.2011.02.004

19.

Gibbons

R. D.

Hedeker

D. R.

(1992). Full-information item bi-factor analysis. Psychometrika, 57, 423–436.

20.

Good

T. L.

(1987). Two decades of research on teacher expectations: Findings and future directions. Journal of Teacher Education, 38(4), 32–47. doi:10.1177/002248718703800406

21.

Goodenow

(1993). The psychological sense of school membership among adolescents: Scale development and educational correlates. Psychology in the Schools, 30(1), 79–90. doi:10.1002/1520–6807(199301)30:1<79::AID-PITS2310 300113>3.0.CO;2-X

22.

Greeno

(1998). The situativity of knowing, learning, and research. American Psychologist, 53(1), 5–26. doi:10.1037/0003-066X.53.1.5

23.

Gregory

Ripski

M. B.

(2008). Adolescent trust in teachers: Implications for behavior in the high school classroom. School Psychology Review, 37(3), 337.

24.

Hamre

Hatfield

Pianta

Jamil

(2014). Evidence for general and domain-specific elements of teacher-child interactions: Associations with preschool children’s development. Child Development, 85(3), 1257–1274. doi:10.1111/cdev.12184

25.

Jennings

P. A.

Greenberg

M. T.

(2009). The prosocial classroom: Teacher social and emotional competence in relation to student and classroom outcomes. Review of Educational Research, 79(1), 491–525. doi:10.3102/0034654308325693

26.

Jöreskog

K. G.

Moustaki

(2006). Factor analysis of ordinal variables with full information maximum likelihood. Unpublished manuscript.

27.

Kane

T. J.

Cantrell

(2010). Learning about teaching: Initial findings from the measures of effective teaching project. Seattle: The Bill & Melinda Gates Foundation.

28.

Kane

T. J.

Staiger

D. O.

(2012). Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains. Seattle: The Bill & Melinda Gates Foundation.

29.

Kleinfeld

(1975). Effective teachers of Eskimo and Indian students. The School Review, 83, 301–344.

30.

Kline

R. B.

(2010). Principles and practice of structural equation modeling (3rd ed.). New York: Guilford Press.

31.

Kulik

J. A.

Kulik

C. L.

(1988). Timing of feedback and verbal learning. Review of Educational Research, 58(1), 79–97. doi:10.3102/00346543058001079

32.

Kunter

Baumert

(2006). Who is the expert? Construct and criteria validity of student and teacher ratings of instruction. Learning Environments Research, 9(3), 231–251. doi:10.1007/s10984-006-9015-7

33.

Kunter

Tsai

Y.-M.

Klusmann

Brunner

Krauss

Baumert

(2008). Students’ and mathematics teachers’ perceptions of teacher enthusiasm and instruction. Learning and Instruction, 18(5), 468–482. doi:10.1016/j.learnin struc.2008.06.008

34.

Lee

C. D.

(1992). Literacy, cultural diversity, and instruction. Education and Urban Society, 24(2), 279–291. doi:10.1177/0013124592024002008

35.

Lee

C. D.

(2001). Is October Brown Chinese? A cultural modeling activity system for underachieving students. American Educational Research Journal, 38(1), 97–141. doi:10.3102/00028312038001097

36.

Lee

V. E.

Smith

J. B.

(1999). Social support and achievement for young adolescents in Chicago: The role of school academic press. American Educational Research Journal, 36(4), 907–945. doi:10.3102/00028312036004907

37.

Lee

V. E.

Smith

J. B.

Perry

T. E.

Smylie

M. A.

(1999). Social support, academic press, and student achievement: A view from the middle grades in Chicago. Improving Chicago’s schools. Chicago: Chicago Annenberg Research Project.

38.

MacCallum

R. C.

Browne

M. W.

Sugawara

H. M.

(1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130–149.

39.

Marsh

H. W.

Roche

L. A.

(1997). Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist, 52(11), 1187. doi:10.1037/0003-066X.52.11.1187

40.

Martin

A. J.

Dowson

(2009). Interpersonal relationships, motivation, engagement, and achievement: Yields for theory, current issues, and educational practice. Review of Educational Research, 79(1), 327–365. doi:10.3102/0034654308325583

41.

Maydeu-Olivares

Joe

(2005). Limited-and full-information estimation and goodness-of-fit testing in 2 n contingency tables: a unified framework. Journal of the American Statistical Association, 100(471), 1009–1020. doi:10.2139/ssrn.1019782

42.

Mayer

R. E.

(2002). Rote versus meaningful learning. Theory Into Practice, 41(4), 226–233. doi:10.1207/s15430421tip4104_4

43.

McCarthy

C. J.

Lineback

Reiser

(2014). Teacher stress, emotion, and classroom management. In Emmer

Sabornie

E. J.

(Eds.), Handbook of classroom management (2nd ed.). New York: Routledge.

44.

McCaslin

(2009). Co-regulation of student motivation and emergent identity. Educational Psychologist, 44(2), 137–146. doi:10.1080/00461520902832384

45.

McCaslin

Hickey

D. T.

(2001). Self-regulated learning and academic achievement: A Vygotskian view. In Zimmerman

B. J.

Schunk

(Eds.), Self-regulated learning and academic achievement: Theory, research, and practice (2nd ed., pp. 227–252). Mahwah, NJ: Erlbaum.

46.

McGee

E. O.

Martin

D. B.

(2011). “You would not believe what I have to go through to prove my intellectual value!” Stereotype management among academically successful black mathematics and engineering students. American Educational Research Journal, 48(6), 1347–1389. 10.3102/0002831211423972

47.

Milner

H. R.

Tenore

F. B.

(2010). Classroom management in diverse classrooms. Urban Education, 45(5), 560–603. doi: 10.1177/0042085910377290

48.

Nakkula

M. J.

Toshalis

(2006). Understanding youth: Adolescent development for educators. Cambridge: Harvard Education Publishing Group.

49.

Nelson

R. M.

DeBacker

T. K.

(2008). Achievement motivation in adolescents: The role of peer climate and best friends. Journal of Experimental Education, 76(2), 170–189. doi:10.3200/JEXE.76.2.170–190

50.

Noguera

P. A.

(2003). Schools, prisons, and social implications of punishment: Rethinking disciplinary practices. Theory Into Practice, 42, 341–350. doi:10.1207/s15430421tip4204_12

51.

Patrick

Anderman

L. H.

Ryan

A. M.

(2004). Social motivation and the classroom environment. In Midgley

(Ed.), Goals, goal structures, and patterns of adaptive learning (pp. 85–108). Mahwah, NJ: Lawrence Erlbaum Associates.

52.

Pianta

R. C.

Hamre

B. K.

(2009). Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher, 38(2), 109–119. doi:10.3102/0013189X09332374

53.

Pianta

R. C.

La Paro

Hamre

B. K.

(2008). Classroom assessment scoring system. Baltimore: Brookes.

54.

Raudenbush

S. W.

Jean

(2014). To what extent do student perceptions of classroom quality predict teacher value added? In Kane

Kerr

Pianta

(Eds.), Designing teacher evaluation systems: New guidance from the Measures of Effective Teaching project (pp. 170–202). San Francisco: Jossey-Bass.

55.

Reeve

Jang

(2006). What teachers say and do to support students’ autonomy during a learning activity. Journal of Educational Psychology, 98(1), 209–218. doi:10.1037/0022-0663.98.1.209

56.

Reise

S. P.

(2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. doi:10.1080/00273171.2012.715555

57.

Rijmen

(2013). Hierarchical factor item response theory models for PIRLS: Capturing clustering effects at multiple levels. IERI Monograph Series: Issues and Methodologies in Large-Scale Assessments, 4, 59–74.

58.

Roeser

R. W.

Skinner

Beers

Jennings

P. A.

(2012). Mindfulness training and teachers’ professional development: An emerging area of research and practice. Child Development Perspectives, 6(2), 167–173. doi:10.1111/j.1750-8606 .2012.00238.x

59.

Ruzek

E. A.

Hafen

C. A.

Allen

J. P.

Gregory

Mikami

A. Y.

Pianta

R. C.

(2016). How teacher emotional support motivates students: The mediating roles of perceived peer relatedness, autonomy support, and competence. Learning and Instruction, 42, 95–103. doi:10.1016/j.learninstruc.2016.01.004

60.

Ryan

A. M.

Patrick

(2001). The classroom social environment and changes in adolescents’ motivation and engagement during middle school. American Educational Research Journal, 38(2), 437–460. doi:10.3102/00028312038002437

61.

Schweig

(2014). Cross-level measurement invariance in school and classroom environment surveys: Implications for policy and practice. Educational Evaluation and Policy Analysis, 36(3), 259–280. doi:10.3102/0162373713509880

62.

Shernoff

D. J.

Ruzek

E. A.

Sinha

(2016). The influence of the high school classroom environment on learning as mediated by student engagement. School Psychology International. doi:10.1177/0143034316666413

63.

Sliter

K. A.

Zickar

M. J.

(2014). An IRT examination of the psychometric functioning of negatively worded personality items. Educational and Psychological Measurement, 74(2), 214–226. doi:10.1177/0013164413504584

64.

Spearman

Watt

H. M. G.

(2013). Perception shapes experience: The influence of actual and perceived classroom environment dimensions on girls’ motivations for science. Learning Environments Research, 16(2), 217–238. doi:10.1007/s10984-013-9129-7

65.

Thompson

Windschitl

Braaten

(2013). Developing a theory of ambitious early-career teacher practice. American Educational Research Journal, 50(3), 574–615. doi:10.3102/0002831213476334

66.

Toshalis

(2015). Make me: Understanding and engaging student resistance in school. Cambridge: Harvard Education Publishing Group.

67.

Turner

J. C.

Meyer

D. K.

(2000). Studying and understanding the instructional contexts of classrooms: Using our past to forge our future. Educational Psychologist, 35(2), 69–85. doi:10.1207/S15326985EP3502_2

68.

Wagner

Göllner

Helmke

Trautwein

Lüdtke

(2013). Construct validity of student perceptions of instructional quality is high, but not perfect: Dimensionality and generalizability of domain-independent assessments. Learning and Instruction, 28, 1–11. doi:10.1016/j.learninstruc.2013.03.003

69.

Wallace

T. L.

Chhuon

(2014). Proximal processes in urban classrooms: Engagement and disaffection in urban youth of color. American Educational Research Journal, 51(5), 937–973. doi:10.3102/0002831214531324

70.

Wallace

T. L.

Sung

H. C.

Williams

J. D.

(2014). The defining features of teacher talk within autonomy-supportive classroom management. Teaching and Teacher Education, (42), 34–46. doi:10.1016/j.tate.2014.04.005

71.

Wanless

S.B.

(2016). The role of psychological safety in human development. Research in Human Development, 13(1), 6–14. doi: 10.1080/15427609.2016. 1141283

72.

Ware

(2006). Warm demander pedagogy culturally responsive teaching that supports a culture of achievement for African American students. Urban Education, 41(4), 427–456. doi:10.1177/0042085906289710

73.

Weinstein

C. S.

Tomlinson-Clarke

Curran

(2004). Toward a conception of culturally responsive classroom management. Journal of Teacher Education, 55(1), 25–38. doi:10.1177/0022487103259812

74.

Wentzel

K. R.

(2005). Peer relationships, motivation, and academic performance at school. In Elliot

A. J.

Dweck

C. S.

(Eds.), Handbook of competence and motivation (pp. 279–295). New York: Guilford Press.

75.

Wentzel

K. R.

Battle

Russell

S. L.

Looney

L. B.

(2010). Social supports from teachers and peers as predictors of academic and social motivation. Contemporary Educational Psychology, 35(3), 193–202. doi:10.1016/j.cedpsych .2010.03.002

76.

Williams

J. D.

Woodson

A. N.

Wallace

T. L.

(2016). “Can we say the N-word?” Exploring psychological safety during race talk. Research in Human Development, 13(1), 15–31. doi:10.1080/15427609.2016.1141279

77.

White

Rowan

(2014). User guide to the Measures of Effective Teaching Longitudinal Database (MET LDB). Ann Arbor: Inter-University Consortium for Political and Social Research, University of Michigan.