Carbno (1981) used generalizability theory as a conceptual framework for analyzing student ratings data from two courses taught by a single instructor. This article argues that unwarranted conclusions were reached in the study because its design included an improper operationalization of the object of measurement, given the problems addressed, and because the sample sizes employed were too small.
Get full access to this article
View all access options for this article.
References
1.
Carbno, W. C. (1981). Student evaluation of teacher effectiveness: A case study. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 41, 937-951.
2.
Cronbach, L. J. , Gleser, G. C., Nanda, H., and Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalization for scores and profiles. New York: John Wiley and Sons.
3.
Crooks, T. J. and Kane, M. T. (1981). The generalizability of student ratings of instructors: Item specificity and section effects. Research in Higher Education, 15, 305-314.
4.
Feldman, K. A. (1977). Consistency and variability among college students in rating their teachers and courses: A review and analysis. Research in Higher Education, 6, 223-274.
5.
Gillmore, G. M. (1980). Student instructional ratings: To what universe can we dependably generalize results?Invited address to the Special Interest Group on Instructional Evaluation at the annual meeting of the American Educational Research Association Convention, Boston.
6.
Gillmore, G. M. , Kane, M. T., and Naccarato, R. W. (1976). The generalizability of student instructional ratings: General theory and application to the University of Washington Instructional Assessment System. EAC Reports, No. 76-14, Seattle: Educational Assessment Center, University of Washington.
7.
Gillmore, G. M. , Kane, M. T., and Naccarato, R. W. (1978). The generalizability of student ratings of instruction: Estimation of the teacher and course components. Journal of Educational Measurement, 15, 1-13.
8.
Kane, M. T. , Gillmore, G. M., and Crooks, T. J. (1976). Student evaluations of teaching: The generalizability of class means. Journal of Educational Measurement, 13, 171-184.
9.
Marsh, H. W. and Overall, J. U. (1979). Long-term stability of students' evaluations. Research in Higher Education, 10, 139-147.
10.
Smith, P. L. (1978). Sampling errors of variance components in small sample multifacet generalizability studies. Journal of Educational Statistics, 3, 319-346.
11.
Smith, P. L. (1979a). The stability of teacher performance in the same course over time. Research in Higher Education, 11, 153-165.
12.
Smith, P. L. (1979b). The generalizability of student ratings of courses: Asking the right questions. Journal of Educational Measurement, 16, 77-87.
