How do Students and Faculty Consider Numerical Ratings and Comments About Daily Quizzing when Interpreting Student Evaluations of Teaching?

Abstract

Background

Student evaluations of teaching (SETs) are used to assess faculty performance, but prior research has identified sources of bias in the completion and interpretation of SETs.

Objective

We investigated how SET ratings and comments about quizzes are interpreted by faculty and undergraduates.

Method

Participants made judgments about teaching effectiveness after reviewing a fictional professor's SETs. SETs varied in whether the professor was rated lower or higher than the departmental average using quantitative measures, and whether qualitative comments about the professor mentioned daily quizzes.

Results

In Experiment 1, more positive evaluations were provided for SETs with higher ratings, whereas comments about quizzing minimally influenced evaluations; this pattern was similar for student and faculty responses. In Experiment 2, qualitative comments were presented in isolation. Student participants provided more positive evaluations for some measures of teaching effectiveness when regular quizzing was mentioned in student comments.

Conclusion

Quantitative SET ratings may overshadow qualitative comments about effective teaching practices yet qualitative ratings presented alone reveal some association between quizzing and evaluations of teaching.

Teaching Implications

SET interpretation should focus on effective pedagogy such as quizzing, which can aid student learning. We found no evidence that comments about quizzing lowered perceptions of teaching effectiveness.

Keywords

student evaluations of teaching student-instructor agreement quizzing teaching effectiveness

Get full access to this article

View all access options for this article.

References

Agarwal

P. K.

Nunes

L. D.

Blunt

J. R.

(2021). Retrieval practice consistently benefits student learning: A systematic review of applied research in schools and classrooms. Educational Psychology Review, 33(4), 1409–1453. https://doi.org/10.1007/s10648-021-09595-9

Anderson

K. J.

Smith

(2005). Students’ preconceptions of professors: Benefits and barriers according to ethnicity and gender. Hispanic Journal of Behavioral Sciences, 27(2), 184–201. https://doi.org/10.1177/0739986304273707

Basow

S. A.

Martin

J. L.

(2012). Bias in student evaluations. In Kite

M. E.

(Ed.), Effective evaluation of teaching: A guide for faculty and administrators (pp. 40–49). Society for the Teaching of Psychology.

Bjork

E. L.

Bjork

R. A.

(2011). Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. In Gernsbacher

M. A.

Pew

R. W.

Hough

L. M.

Pomerantz

J. R.

(Eds.), Psychology and the real world: Essays illustrating fundamental contributions to society (pp. 56–64). Worth Publishers.

Bjork

R. A.

Dunlosky

Kornell

(2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417–444. https://doi.org/10.1146/annurev-psych-113011-143823

Bosshardt

Watts

(2001). Comparing student and instructor evaluations of teaching. The Journal of Economic Education, 32(1), 3–17. https://doi.org/10.1080/00220480109595166

Boysen

G. A.

(2015a). Preventing the overinterpretation of small mean differences in student evaluations of teaching: An evaluation of warning effectiveness. Scholarship of Teaching and Learning in Psychology, 1(4), 269–282. https://doi.org/10.1080/02602938.2013.860950

Boysen

G. A.

(2015b). Significant interpretation of small mean differences in student evaluations of teaching despite explicit warning to avoid overinterpretation. Scholarship of Teaching and Learning in Psychology, 1(2), 150–162. https://doi.org/10.1037/stl0000017

Boysen

G. A.

(2015c). Uses and misuses of student evaluations of teaching: The interpretation of differences in teaching evaluation means irrespective of statistical information. Teaching of Psychology, 42(2), 109–118. https://doi.org/10.1177/0098628315569922

10.

Boysen

G. A.

(2016). Using student evaluations to improve teaching: Evidence-based recommendations. Scholarship of Teaching and Learning in Psychology, 2(4), 273–284. https://doi.org/10.1037/stl0000069

11.

Boysen

G. A.

(2017). Statistical knowledge and the over-interpretation of student evaluations of teaching. Assessment & Evaluation in Higher Education, 42(7), 1095–1102. https://doi.org/10.1080/02602938.2016.1227958

12.

Boysen

G. A.

Kelly

T. J.

Raesly

H. N.

Casner

R. W.

(2014). The (mis)interpretation of teaching evaluations by college faculty and administrators. Assessment & Evaluation in Higher Education, 39(6), 641–656. https://doi.org/10.1080/02602938.2013.860950

13.

Cain

K. M.

Wilkowski

B. M.

Barlett

C. P.

Boyle

C. D.

Meier

B. P.

(2018). Do we see eye to eye? Moderators of correspondence between student and faculty evaluations of day-to-day teaching. Teaching of Psychology, 45(2), 107–114. https://doi.org/10.1177/0098628318762862

14.

Carpenter

S. K.

Wilford

M. M.

Kornell

Mullaney

K. M.

(2013). Appearances can be deceiving: Instructor fluency increases perceptions of learning without increasing actual learning. Psychonomic Bulletin & Review, 20(6), 1350–1356. https://doi.org/10.3758/s13423-013-0442-z

15.

Carpenter

S. K.

Witherby

A. E.

Tauber

S. K.

(2020). On students’ (mis)judgments of learning and teaching effectiveness: Where we stand and how to move forward. Journal of Applied Research in Memory and Cognition, 9(2), 181–185. https://doi.org/10.1016/j.jarmac.2020.04.003

16.

Franklin

(2001). Interpreting the numbers: Using a narrative to help others read student evaluations of your teaching accurately. New Directions for Teaching and Learning, 87, 85–100. https://doi.org/10.1002/tl.10001

17.

Ghislandi

Raffaghelli

Sangrà

Ritella

(2020). The street lamp paradox: Analysing students’ evaluation of teaching through qualitative and quantitative approaches. Journal of Educational, Cultural and Psychological Studies. https://doi.org/10.7358/ecps-2020-021-ghis

18.

Gurung

R. A. R.

(2020). Call it out: Recognizing good teaching and learning. Journal of Applied Research in Memory and Cognition, 9(2), 161–164. https://doi.org/10.1016/j.jarmac.2020.02.003

19.

Heckert

T. M.

Latier

Ringwald

Silvey

(2006). Relation of course, instructor, and student characteristics to dimensions of student ratings of teaching effectiveness. College Student Journal, 40(1), 195–203. https://doi.org/10.19030/ajbe.v3i1.9024

20.

Heffernan

(2022). Sexism, racism, prejudice, and bias: A literature review and synthesis of research surrounding student evaluations of courses and teaching. Assessment & Evaluation in Higher Education, 47(1), 144–154. https://doi.org/10.1080/02602938.2021.1888075

21.

Kirk-Johnson

Galla

B. M.

Fraundorf

S. H.

(2019). Perceiving effort as poor learning: The misinterpreted-effort hypothesis of how experienced effort and perceived learning relate to study strategy choice. Cognitive Psychology, 115, 101237. https://doi.org/10.1016/j.cogpsych.2019.101237

22.

Kornell

(2020). Why and how you should read student evaluations of teaching. Journal of Applied Research in Memory and Cognition, 9(2), 165–169. https://doi.org/10.1016/j.jarmac.2020.02.006

23.

Lewis

K. G.

(2001). Making sense of student written comments. New Directions for Teaching and Learning, 87, 25–32. https://doi.org/10.1002/tl.25

24.

Lyle

K. B.

Crawford

N. A.

(2011). Retrieving essential material at the end of lectures improves performance on statistics exams. Teaching of Psychology, 38(2), 94–97. https://doi.org/10.1177/0098628311401587

25.

McDaniel

M. A.

Anderson

J. L.

Derbish

M. H.

Morrisette

(2007). Testing the testing effect in the classroom. European Journal of Cognitive Psychology, 19(4-5), 494–513. https://doi.org/10.1080/09541440701326154

26.

Miller

J. E.

Seldin

(2014). Changing practices in faculty evaluation: Can better evaluation make a difference? American Association of University Professors. https://www.aaup.org/article/changing-practices-faculty-evaluation

27.

Morehead

Rhodes

M. G.

DeLozier

(2016). Instructor and student knowledge of study strategies. Memory, 24(2), 257–271. https://doi.org/10.1080/09658211.2014.1001992

28.

Nguyen

McDaniel

M. A.

(2015). Using quizzing to assist student learning in the classroom: The good, the bad, and the ugly. Teaching of Psychology, 42(1), 87–92. https://doi.org/10.1177/0098628314562685

29.

Olivares

O. J.

(2001). Student interest, grading leniency, and teacher ratings: A conceptual analysis. Contemporary Educational Psychology, 26, 382–399. https://doi.org/1osf0.1006/ceps.2000.1070c

30.

Overson

C. E.

Hakala

C. M.

Kordonowy

L. L.

Benassi

V. A.

(Eds.). (2023). In their own words: What scholars and teachers want you to know about why and how to apply the science of learning in your academic setting. Society for the Teaching of Psychology. https://teachpsych.org/ebooks/itow

31.

Paas

Renkl

Sweller

(2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1–4. https://doi.org/10.1207/S15326985EP3801_1

32.

Rivers

M. L.

(2021). Metacognition about practice testing: A review of learners’ beliefs, monitoring, and control of test-enhanced learning. Educational Psychology Review, 33(3), 823–862. https://doi.org/10.1007/s10648-020-09578-2

33.

Rivers

M. L.

Babineau

Neely

K. P.

Tauber

(2023). How do students and faculty consider high ratings and daily quizzing when interpreting student evaluations of teaching? https://doi.org/10.17605/OSF.IO/QXS2M

34.

Roche

L. A.

Marsh

H. W.

(2000). Multiple dimensions of university teacher self-concept. Instructional Science, 28(5), 439–468. https://doi.org/10.1023/A:1026576404113

35.

Roediger

H. L.

Putnam

A. L.

Smith

M. A.

(2011). Ten benefits of testing and their applications to educational practice. Psychology of Learning and Motivation, 55, 1–36. https://doi.org/10.1016/B978-0-12-387691-1.00001-6

36.

Schrank

(2016). An assessment of student perceptions and responses to frequent low-stakes testing in introductory sociology classes. Teaching Sociology, 44(2), 118–127. https://doi.org/10.1177/0092055X1562474

37.

Shao

L. P.

Anderson

L. P.

Newsome

(2007). Evaluating teaching effectiveness: Where we are and where we should be. Assessment & Evaluation in Higher Education, 32(3), 355–371. https://doi.org/10.1080/02602930600801886

38.

Shevlin

Banyard

Davies

Griffiths

(2000). The validity of student evaluation of teaching in higher education: Love me, love my lectures? Assessment & Evaluation in Higher Education, 25(4), 397–405. https://doi.org/10.1080/713611436

39.

Spooren

Brockx

Mortelmans

(2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598–642. https://doi.org/10.3102/0034654313496870

40.

Stroebe

(2016). Why good teaching evaluations may reward bad teaching: On grade inflation and other unintended consequences of student evaluations. Perspectives on Psychological Science, 11(6), 800–816. https://doi.org/10.1177/1745691616650284

41.

Sundararajan

Adesope

(2020). Keep it coherent: A meta-analysis of the seductive details effect. Educational Psychology Review, 32(3), 707–734. https://doi.org/10.1007/s10648-020-09522-4

42.

Vialta-Cerda

McKeny

Gatlin

Sandi-Urena

(2015). Evaluation of instruction: Students’ patterns of use and contribution to RateMyProfessors.com. Assessment & Evaluation in Higher Education, 40(2), 181–198. https://doi.org/10.1080/02602938.2014.896862

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.21 MB