The Factorial Invariance of Student Evaluations of College Teaching

Abstract

University instructors who taught the same course at least four times over a 4-year period were evaluated by different groups of students in each of these four classes (N = 316 instructors, 1,264 classes, 31,322 students). Factor analyses of the class average ratings clearly replicated the nine evaluation dimensions found in earlier studies. A multitrait-multimethod (MTMM) matrix was formed in which the nine dimensions were the multiple traits, and the four different offerings of the same course were the multiple methods. Across all nine dimensions the average convergent validity was r = .68. MTMM analyses with both the Campbell-Fiske MTMM criteria and confirmatory factor analysis (using LISREL) demonstrated substantial convergent and discriminant validity, but little method/halo effect. The confirmatory factor analysis also demonstrated the generalizability of the multivariate structure of the ratings across the four sets of courses. Taken together, these findings provide stronger evidence for the multidimensionality of student ratings than does any previously published research and describe a powerful analytic tool for use in student evaluation research.