Validation studies of the Course-Faculty Evalua tion Instrument (CFI) are described. Seven dimen sions were constructed which characterize each class and predict student rating of the instructor, course, and text. Different measurement scales and methods were analyzed, using a multitrait-multi method (MTMM) strategy. The MTMM matrix for the CFI and a similar MTMM matrix for the Course-Evaluation Instrument (CEI) reported by Schwab (1974) were analyzed and compared. The same method of scaling was found to be superior in both studies. Using an analysis of variance frame work to summarize MTMM matrices, the CFI demonstrated greater discriminant validity using more dimensions (traits) and had a lower error component than the CEI. The benefits of compar ing instruments and implications for future course- faculty evaluation research are discussed.
Get full access to this article
View all access options for this article.
References
1.
Aleamoni, L., & Spencer, R.The Illinois course evaluation questionnaire: A description of its development and a report of some of its results. Educational and Psychological Measurement, 1973, 33,669-684.
2.
Barnoski, R.P., & Sockloff, A.L.A validation study of the faculty and course evaluation (FACE) instrument. Educational and Psychological Measurement, 1976, 36, 391-400.
3.
Bausell, R., & Magoon, J.Expected grades in a course, gradepoint average, and student rating of instructor . Educational and Psychological Measurement, 1972, 32, 1013-1023.
Campbell, D.T., & Fiske, D.W.Convergent and discriminant validation by the multi-trait multimethod matrix . Psychological Bulletin, 1959, 56, 81-105.
6.
Cohen, J., & Cohen, P.Applied multiple regression/correlation analysis for the behavioral sciences . Hillsdale, NJ: Lawrence Erlbaum Associates, 1975.
7.
Costin, F., Greenough, W.T., & Menges, R.J.Student ratings of college teaching: Reliability, validity, and usefulness . Review of Educational Research, 1971, 41, 511-535.
8.
DuBois, B., & Burns, J.A.An analysis of the meaning of the question mark response category in attitude scales. Educational and Psychological Measurement, 1975, 35, 869-884.
9.
Fogli, L., Hulin, C.L., & Blood, M.R.Development of first-level behavioral criteria. Journal of Applied Psychology, 1971, 55, 3-8.
10.
Freedman, R.D., & Stumpf, S.A.Course-Faculty Evaluation Instrument: Development and applications (Working Paper #76-59. New York: New York University, Graduate School of Business Administration, November 1976.
11.
Freedman, R.D., & Stumpf, S.A.What can one learn from the Learning Style Inventory. Academy of Management Journal, in press.
12.
Harari, O., & Zedeck, S.Development of behavior-ally anchored scales for the evaluation of faculty teaching. Journal of Applied Psychology, 1973 , 58. 261-265.
13.
Harman, H.Modern Factor Analysis. Chicago: University of Chicago Press, 1967.
14.
Holmes, D.The teaching assessment blank (TAB): A form for the assessment of college instructors. Journal of Experimental Education, 1971, 39, 34-38.
15.
Jenkins, G.D., & Taber, T.D.A monte carlo study of factors affecting three indices of composite scale reliability. Journal of Applied Psychology, 1977, 62, 392-398.
16.
Jöreskog, K.G.Analyzing psychological data by structural analysis of covariance matrices . In D. H. Krantz, R. C. Atkinson, R. D. Luce, & P. Suppes (Eds.), Measurement, Psychophysics, and Neural Information Processing. (Vol. 2). San Francisco: W. H. Freeman & Co., 1974, pp. 1-56.
17.
Kavanagh, M.J., MacKinney, A.C., & Wolins, L.Issues in managerial performance: Multitrait-multimethod analyses of ratings . Psychological Bulletin, 1971, 75, 34-49.
18.
Kolb, D., Rubin, I., & McIntyre, J.Organizational psychology: An experiential approach (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall, 1974.
19.
Linn, R.L., Centra, J.A., & Tucker, L.Between, within, and total group factor analyses of student ratings of instruction . Multivariate Behavioral Research, 1975, 10, 277-288.
20.
Lissitz, R.W., & Green, S.B.Effect of the number of scale points on reliability: A monte carlo approach . Journal of Applied Psychology, 1975, 60, 10-13.
21.
McKeachie, W.J., Lin, Y., & Mann, W.Student ratings of teacher effectiveness: Validity studies. American Educational Research Journal, 1971, 8, 435-455.
22.
Norton, S.D., Gustafson, D.P., & Foster, C.E.Assessment for management potential: Scale design and development, training effects, and rater/ratee sex effects. Academy of Management Journal, 1977, 20, 117-131.
23.
Rodin, M., & Rodin, B.Student evaluation of teachers. Science, 1972 , 177, 1164-1166.
24.
Schuh, A.J., & Crivelli, M.A. , Aminadversion error in student evaluations of faculty teaching effectiveness, Journal of Applied Psychology , 1973, 58, 259-260.
25.
Schwab, D.Development of the CEI: Scale construction and initial testing (CEI Report No. 1, University of Wisconsin—Madison: Graduate School of Business and Industrial Relations Research Institute, February 1974.
26.
Schwab, D.Manual for the Course Evaluation Instrument. University of Wisconsin— Madison: Graduate School of Business and Industrial Relations Research Institute, 1976.
27.
Schwab, D., & Forrest, R.Accounting for the relationship between students' evaluation of courses and expected grades. Proceedings of the 35th Annual Meeting of the Academy of Management, 1975, 10-12.
28.
Schwab, D., & Wallace, M.The validity extension of a measure to assess teaching effectiveness in business schools. Proceedings of the Midwest Academy of Management Meeting, 1975, 18, 128-135.
29.
Sharon, A.T.Eliminating bias from student ratings of college instructors. Journal of Applied Psychology, 1970, 54, 278-281.
30.
Smith, P.C., Kendall, L.M., & Hulin, C.L.The measurement of satisfaction in work and retirement: A strategy for the study of attitudes. Chicago: Rand McNally, 1969
31.
Sockloff, A.L.Instruments for student evaluation of faculty: Ideal and actual. In A. L. Sockloff (Ed.), Proceedings: The First Invitational Conference on Faculty Effectiveness as Evaluated by Students . Philadelphia: Temple University, Measurement and Research Center, 1973, 132-151.
32.
Stumpf, S.A., & Freedman, R.D.The nature of expected grade bias in course-faculty evaluations (Working Paper #77-08. New York: New York University, Graduate School of Business Administration, March 1977.
33.
Stumpf, S.A.Freedman, R.D., & Krieger, K.Validity generalization and extension of the Course-Faculty Evaluation Instrument (CFI. Unpublished Manuscript, New York University, Graduate School of Business Administration, September, 1977.
34.
Wallace, M., & Schwab, D.The validation of a teaching-effectiveness measure in two business schools . Proceedings of the 33rd Annual Meeting of the Academy of Management, 1973,33,229-235.
35.
Zedeck, S., Kafry, D., & Jacobs, R.Format and scoring variations in behavioral expectation scales. Organizational Behavior and Human Performunce.1976,17, 171-184.
