This article looks at the methodological problems created by incorrectly using between-class data when looking at effects generated by individual students. The grade/evaluation association found in the student evaluation of teaching was used as a case example. A theoretical framework to understand the contributions to the relationship by both between-class data (leniency) and within-class data (reciprocity) was created that shows how the two are confounded when looking only at between-class effects. The findings suggest that within-class data is the appropriate starting point when investigating associational problems with either within-or between-case data.
Bacon, D. R., & Novotny, J. (2002). Exploring achievement striving as a moderator of the grading leniency effect. Journal of Marketing Education, 24, 1-14.
2.
Bharadwaj, S., Futrell, C. M., & Kantak, D. M. (1993). Using student evaluations to improve learning. Marketing Education Review, 3(2), 16-21.
3.
Birnbaum, M. H. (2000). A survey of faculty opinions concerning student evaluation of teaching. Retrieved June 28, 2006, from http://psych.fullerton.edu/mbirnbaum/faculty3.htm
4.
Braskamp, L. A., & Ory, J. C. (1994). Assessing faculty work: Enhancing individual and institutional performances. San Francisco: Jossey-Bass.
5.
Cashin, W. E. (1995). Student rating of teaching: The research revisited (Idea Paper No. 32). Manhattan: Center for Faculty Evaluation and Development, Division of Continuing Education, Kansas State University.
6.
Clayson, D. E. (2004). A test of reciprocity effects in the student evaluation of instructors in marketing classes. Marketing Education Review, 14(2), 11-21.
7.
Clayson, D. E., Frost, T. F., & Sheffet, M. J. (2006). Grades and the student evaluation of instruction: A test of the reciprocity effect. Academy of Management Learning & Education, 5, 52-65.
8.
Feldman, K. A. (1976). Grades and college students; evaluations of their courses and teachers. Research in Higher Education, 4(1), 69-111.
9.
Goldberg, G., & Callahan, J. (1991). Objectivity of student evaluations of instructors. Journal of Education for Business, 66, 377-378.
10.
Goldman, L. (1985). The betrayal of the gatekeepers: Grade inflation. Journal of General Education, 37, 97-121.
11.
Greenwald, A. G., & Gillmore, G. M. (1997). No pain, no gain? The importance of measuring course workload in student ratings of instruction. Journal of Educational Psychology, 89, 783-775.
12.
(Original data published in American Psychologist, 52, 1182-1186, by A. G. Greenwald, 1991)
13.
Johnson, V. E. (2003). Grade inflation: A crisis in college education. New York: Springer.
14.
Kaplan, M., Mets, L. A., & Cook, C. E. (2000). Questions frequently asked about student ratings forms: Summary of research findings. Retrieved June 28, 2006, from http://www.crlt.umich.edu/tstrategies/studentratingfaq.html
15.
Marsh, H. W., & Dunkin, M. J. (1992). Students’ evaluations of university teaching: A multidimensional perspective. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (pp. 143-233). New York: Agathon.
16.
Marsh, H. W., & Roche, L. A. (1997). Making students’ evaluations of teaching effectiveness effective. American Psychologist, 52, 1187-1197.
17.
Marsh, H. W., & Roche, L. A. (1999). Reply upon SET research. American Psychologist, 54, 517-518.
18.
Marsh, H. W., & Roche, L. A. (2000). Effects of grading leniency and low workload on students’ evaluations of teaching: Popular myth, bias, validity, or innocent bystanders? Journal of Educational Psychology, 92, 202-228.
19.
Redding, R. E. (1998). Students’ evaluation of teaching fuel grade inflation. American Psychologist, 53, 1227-1228.
20.
Ryan, J. J., Anderson, J. A., & Birchler, A. B. (1980). Student evaluation: The faculty responds. Research in Higher Education, 12, 317-333.
21.
Seiver, D. A. (1983). Evaluations and grades: A simultaneous framework. Journal of Economic Education, 14, 32-38.
22.
Simpson, P. M., & Siguaw, J. A. (2000). Student evaluations of teaching: An exploratory study of the faculty response. Journal of Marketing Education, 22, 199-213.
23.
Stumpf, S. A., & Freedman, R. D. (1979). Expected grade covariation with student ratings of instruction: Individual versus class effects. Journal of Educational Psychology, 71, 293-302.
