Student evaluations of teaching (SETs) are used to assess faculty performance, but prior research has identified sources of bias in the completion and interpretation of SETs.
Objective
We investigated how SET ratings and comments about quizzes are interpreted by faculty and undergraduates.
Method
Participants made judgments about teaching effectiveness after reviewing a fictional professor's SETs. SETs varied in whether the professor was rated lower or higher than the departmental average using quantitative measures, and whether qualitative comments about the professor mentioned daily quizzes.
Results
In Experiment 1, more positive evaluations were provided for SETs with higher ratings, whereas comments about quizzing minimally influenced evaluations; this pattern was similar for student and faculty responses. In Experiment 2, qualitative comments were presented in isolation. Student participants provided more positive evaluations for some measures of teaching effectiveness when regular quizzing was mentioned in student comments.
Conclusion
Quantitative SET ratings may overshadow qualitative comments about effective teaching practices yet qualitative ratings presented alone reveal some association between quizzing and evaluations of teaching.
Teaching Implications
SET interpretation should focus on effective pedagogy such as quizzing, which can aid student learning. We found no evidence that comments about quizzing lowered perceptions of teaching effectiveness.
AgarwalP. K.NunesL. D.BluntJ. R. (2021). Retrieval practice consistently benefits student learning: A systematic review of applied research in schools and classrooms. Educational Psychology Review, 33(4), 1409–1453. https://doi.org/10.1007/s10648-021-09595-9
2.
AndersonK. J.SmithG. (2005). Students’ preconceptions of professors: Benefits and barriers according to ethnicity and gender. Hispanic Journal of Behavioral Sciences, 27(2), 184–201. https://doi.org/10.1177/0739986304273707
3.
BasowS. A.MartinJ. L. (2012). Bias in student evaluations. In KiteM. E. (Ed.), Effective evaluation of teaching: A guide for faculty and administrators (pp. 40–49). Society for the Teaching of Psychology.
4.
BjorkE. L.BjorkR. A. (2011). Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. In GernsbacherM. A.PewR. W.HoughL. M.PomerantzJ. R. (Eds.), Psychology and the real world: Essays illustrating fundamental contributions to society (pp. 56–64). Worth Publishers.
BosshardtW.WattsM. (2001). Comparing student and instructor evaluations of teaching. The Journal of Economic Education, 32(1), 3–17. https://doi.org/10.1080/00220480109595166
7.
BoysenG. A. (2015a). Preventing the overinterpretation of small mean differences in student evaluations of teaching: An evaluation of warning effectiveness. Scholarship of Teaching and Learning in Psychology, 1(4), 269–282. https://doi.org/10.1080/02602938.2013.860950
8.
BoysenG. A. (2015b). Significant interpretation of small mean differences in student evaluations of teaching despite explicit warning to avoid overinterpretation. Scholarship of Teaching and Learning in Psychology, 1(2), 150–162. https://doi.org/10.1037/stl0000017
9.
BoysenG. A. (2015c). Uses and misuses of student evaluations of teaching: The interpretation of differences in teaching evaluation means irrespective of statistical information. Teaching of Psychology, 42(2), 109–118. https://doi.org/10.1177/0098628315569922
10.
BoysenG. A. (2016). Using student evaluations to improve teaching: Evidence-based recommendations. Scholarship of Teaching and Learning in Psychology, 2(4), 273–284. https://doi.org/10.1037/stl0000069
11.
BoysenG. A. (2017). Statistical knowledge and the over-interpretation of student evaluations of teaching. Assessment & Evaluation in Higher Education, 42(7), 1095–1102. https://doi.org/10.1080/02602938.2016.1227958
12.
BoysenG. A.KellyT. J.RaeslyH. N.CasnerR. W. (2014). The (mis)interpretation of teaching evaluations by college faculty and administrators. Assessment & Evaluation in Higher Education, 39(6), 641–656. https://doi.org/10.1080/02602938.2013.860950
13.
CainK. M.WilkowskiB. M.BarlettC. P.BoyleC. D.MeierB. P. (2018). Do we see eye to eye? Moderators of correspondence between student and faculty evaluations of day-to-day teaching. Teaching of Psychology, 45(2), 107–114. https://doi.org/10.1177/0098628318762862
14.
CarpenterS. K.WilfordM. M.KornellN.MullaneyK. M. (2013). Appearances can be deceiving: Instructor fluency increases perceptions of learning without increasing actual learning. Psychonomic Bulletin & Review, 20(6), 1350–1356. https://doi.org/10.3758/s13423-013-0442-z
15.
CarpenterS. K.WitherbyA. E.TauberS. K. (2020). On students’ (mis)judgments of learning and teaching effectiveness: Where we stand and how to move forward. Journal of Applied Research in Memory and Cognition, 9(2), 181–185. https://doi.org/10.1016/j.jarmac.2020.04.003
16.
FranklinJ. (2001). Interpreting the numbers: Using a narrative to help others read student evaluations of your teaching accurately. New Directions for Teaching and Learning, 87, 85–100. https://doi.org/10.1002/tl.10001
17.
GhislandiP.RaffaghelliJ.SangràA.RitellaG. (2020). The street lamp paradox: Analysing students’ evaluation of teaching through qualitative and quantitative approaches. Journal of Educational, Cultural and Psychological Studies. https://doi.org/10.7358/ecps-2020-021-ghis
18.
GurungR. A. R. (2020). Call it out: Recognizing good teaching and learning. Journal of Applied Research in Memory and Cognition, 9(2), 161–164. https://doi.org/10.1016/j.jarmac.2020.02.003
19.
HeckertT. M.LatierA.RingwaldA.SilveyB. (2006). Relation of course, instructor, and student characteristics to dimensions of student ratings of teaching effectiveness. College Student Journal, 40(1), 195–203. https://doi.org/10.19030/ajbe.v3i1.9024
20.
HeffernanT. (2022). Sexism, racism, prejudice, and bias: A literature review and synthesis of research surrounding student evaluations of courses and teaching. Assessment & Evaluation in Higher Education, 47(1), 144–154. https://doi.org/10.1080/02602938.2021.1888075
21.
Kirk-JohnsonA.GallaB. M.FraundorfS. H. (2019). Perceiving effort as poor learning: The misinterpreted-effort hypothesis of how experienced effort and perceived learning relate to study strategy choice. Cognitive Psychology, 115, 101237. https://doi.org/10.1016/j.cogpsych.2019.101237
22.
KornellN. (2020). Why and how you should read student evaluations of teaching. Journal of Applied Research in Memory and Cognition, 9(2), 165–169. https://doi.org/10.1016/j.jarmac.2020.02.006
23.
LewisK. G. (2001). Making sense of student written comments. New Directions for Teaching and Learning, 87, 25–32. https://doi.org/10.1002/tl.25
24.
LyleK. B.CrawfordN. A. (2011). Retrieving essential material at the end of lectures improves performance on statistics exams. Teaching of Psychology, 38(2), 94–97. https://doi.org/10.1177/0098628311401587
25.
McDanielM. A.AndersonJ. L.DerbishM. H.MorrisetteN. (2007). Testing the testing effect in the classroom. European Journal of Cognitive Psychology, 19(4-5), 494–513. https://doi.org/10.1080/09541440701326154
NguyenK.McDanielM. A. (2015). Using quizzing to assist student learning in the classroom: The good, the bad, and the ugly. Teaching of Psychology, 42(1), 87–92. https://doi.org/10.1177/0098628314562685
29.
OlivaresO. J. (2001). Student interest, grading leniency, and teacher ratings: A conceptual analysis. Contemporary Educational Psychology, 26, 382–399. https://doi.org/1osf0.1006/ceps.2000.1070c
30.
OversonC. E.HakalaC. M.KordonowyL. L.BenassiV. A. (Eds.). (2023). In their own words: What scholars and teachers want you to know about why and how to apply the science of learning in your academic setting. Society for the Teaching of Psychology. https://teachpsych.org/ebooks/itow
31.
PaasF.RenklA.SwellerJ. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1–4. https://doi.org/10.1207/S15326985EP3801_1
32.
RiversM. L. (2021). Metacognition about practice testing: A review of learners’ beliefs, monitoring, and control of test-enhanced learning. Educational Psychology Review, 33(3), 823–862. https://doi.org/10.1007/s10648-020-09578-2
33.
RiversM. L.BabineauA.NeelyK. P.TauberU. (2023). How do students and faculty consider high ratings and daily quizzing when interpreting student evaluations of teaching?https://doi.org/10.17605/OSF.IO/QXS2M
34.
RocheL. A.MarshH. W. (2000). Multiple dimensions of university teacher self-concept. Instructional Science, 28(5), 439–468. https://doi.org/10.1023/A:1026576404113
35.
RoedigerH. L.PutnamA. L.SmithM. A. (2011). Ten benefits of testing and their applications to educational practice. Psychology of Learning and Motivation, 55, 1–36. https://doi.org/10.1016/B978-0-12-387691-1.00001-6
36.
SchrankZ. (2016). An assessment of student perceptions and responses to frequent low-stakes testing in introductory sociology classes. Teaching Sociology, 44(2), 118–127. https://doi.org/10.1177/0092055X1562474
37.
ShaoL. P.AndersonL. P.NewsomeM. (2007). Evaluating teaching effectiveness: Where we are and where we should be. Assessment & Evaluation in Higher Education, 32(3), 355–371. https://doi.org/10.1080/02602930600801886
38.
ShevlinM.BanyardP.DaviesM.GriffithsM. (2000). The validity of student evaluation of teaching in higher education: Love me, love my lectures?Assessment & Evaluation in Higher Education, 25(4), 397–405. https://doi.org/10.1080/713611436
39.
SpoorenP.BrockxB.MortelmansD. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598–642. https://doi.org/10.3102/0034654313496870
40.
StroebeW. (2016). Why good teaching evaluations may reward bad teaching: On grade inflation and other unintended consequences of student evaluations. Perspectives on Psychological Science, 11(6), 800–816. https://doi.org/10.1177/1745691616650284
41.
SundararajanN.AdesopeO. (2020). Keep it coherent: A meta-analysis of the seductive details effect. Educational Psychology Review, 32(3), 707–734. https://doi.org/10.1007/s10648-020-09522-4
42.
Vialta-CerdaA.McKenyP.GatlinT.Sandi-UrenaS. (2015). Evaluation of instruction: Students’ patterns of use and contribution to RateMyProfessors.com. Assessment & Evaluation in Higher Education, 40(2), 181–198. https://doi.org/10.1080/02602938.2014.896862
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
