What Have We Learned About the Instructor’s Gender Effect on Student Evaluation of Teaching From Experimental and Quasi-Experimental Research?

Abstract

Student evaluation of teaching (SET), one of the most widely used mechanisms for evaluating teaching worldwide, is arguably gender-biased, potentially obstructing women’s access to and advancement in academia. To date, a significant number of studies have focused on interpreting a difference in SET scores by gender as evidence of gender bias. The validity of inferences from many of these studies is in jeopardy if they lack a strong research design that effectively isolates various potential confounding factors. We conducted a systematic literature review of experimental and quasi-experimental studies testing the claim that SET scores are biased against women. Our search identified 21 relevant studies published between 2000 and 2021. We qualitatively analyzed the supporting theories that predict gender-biased SET scores, the study design features used to estimate the instructor’s gender effect, and the findings across studies. The two most dominant explanations across studies are gender stereotypes and expectancy violations theory. We found ingenious methods for simulating a lecture and manipulating the instructor’s gender. SET scales across studies feature up to five types of distinctive questions about teaching performance. Findings across studies suggest that women are more likely to receive lower SET scores than men. However, this review reveals that examining gender bias is far more complex than individual studies portray. We note that the lack of strong supporting theory and tensions in research designs affect the construct and external validity of findings across studies. We discuss implications for fair evaluation of teaching that protect the development of women in academia.

Keywords

higher education faculty development gender studies equity evaluation experimental design experimental research quasi-experimental analysis faculty evaluation student evaluation of teaching gender bias fairness

Get full access to this article

View all access options for this article.

References

AAUP. (2022). The Annual Report on the economic status of the profession, 2021-22. American Association of University Professors. https://www.aaup.org/report/annual-report-economic-status-profession-2021-22

Abel

M. H.

Meltzer

A. L.

(2007). Student ratings of a male and female professors’ lecture on sex discrimination in the workforce. Sex Roles, 57(3–4), 173–78. https://doi.org/10.1007/s11199-007-9245-x

Alexander

P. A.

(2020). Methodological guidance paper: The art and science of quality systematic reviews. Review of Educational Research, 90(1), 6–23. https://doi.org/1.3102/0034654319854352

Alhija

(2017). Guest editor introduction to the special issue “Contemporary evaluation of teaching: Challenges and promises.” Studies in Educational Evaluation, 54(Supplement C), 1–3. https://doi.org/1.1016/j.stueduc.2017.02.002

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (with Joint Committee on Standards for Educational and Psychological Testing (U.S.)). (2014). Standards for educational and psychological testing. American Educational Research Association.

American Sociological Association. (2019, September 9). Reconsidering student evaluations of teaching. American Sociological Association. https://www.asanet.org/press-center/press-releases/reconsidering-student-evaluations-teaching

Andersen

Miller

E. D.

(1997). Gender and student evaluations of teaching. PS: Political Science and Politics, 30(2), 216. https://doi.org/1.2307/420499

Anderson

K. J.

(2010). Students’ stereotypes of professors: An exploration of the double violations of ethnicity and gender. Social Psychology of Education: An International Journal, 13(4), 459–472. https://doi.org/1.1007/s11218-010-9121-3

Anderson

K. J.

Smith

(2005). Students’ preconceptions of professors: Benefits and barriers according to ethnicity and gender. Hispanic Journal of Behavioral Sciences, 27(2), 184–201. https://doi.org/1.1177/0739986304273707

10.

Arbuckle

Williams

B. D.

(2003). Students’ perceptions of expressiveness: Age and gender effects on teacher evaluations. Sex Roles, 49(9), 507–516. https://doi.org/1.1023/A:1025832707002

11.

Barnet-Verzat

Wolff

(2008). Gender wage gap and the glass ceiling effect: A firm-level investigation. International Journal of Manpower, 29(6), 486–502. https://doi.org/1.1108/01437720810904185

12.

Basow

S. A.

Martin

J. L.

(2012). Bias in student evaluations. In Mary E. Kite (Ed.), Effective evaluation of teaching: A guide for faculty and administrators (pp. 40–49). Society for the Teaching of Psychology.

13.

Bertrand

(2017). The glass ceiling. SSRN Electronic Journal. https://doi.org/1.2139/ssrn.3191467

14.

Bohlscheid

J. C.

Davis

J. C.

(2012). Higher education science student perspectives on classroom instructional methods: A pilot study. Journal of Food Science Education, 11(4), 57–61.

15.

Boring

(2017). Gender biases in student evaluations of teaching. Journal of Public Economics, 145, 27–41. https://doi.org/1.1016/j.jpubeco.2016.11.006

16.

Boveda

Ford

K. S.

Frankenberg

López

(2023). Editorial vision 2022–2025. Review of Educational Research, 93(5), 635–64. https://doi.org/1.3102/00346543231170179

17.

Cabrera

N. L.

Karaman

A. K.

Turner

T. A.

Oregon

Y. G.

Gonell

E. A.

Lopez

J. D.

Deil-Amen

(2022). Race Without Gender? Trends and Limitations in the Higher Education Scholarship Regarding Men of Color. Review of Educational Research, 92(3), 331–369. https://doi.org/1.3102/00346543211054577

18.

Camilli

(2006). Test fairness. In Brennan

R. L.

(Ed.), Educational measurement (4th ed., pp. 221–256). Praeger Publishers.

19.

Centra

J. A.

(2003). Will teachers receive higher student evaluations by giving higher grades and less course work? Research in Higher Education, 44(5), 495–518. https://doi.org/1.1023/A:1025492407752

20.

Chávez

Mitchell

K. M. W.

(2020). Exploring bias in student evaluations: Gender, race, and ethnicity. PS: Political Science & Politics, 53(2), 270–274. https://doi.org/1.1017/S1049096519001744

21.

Chisadza

Nicholls

Yitbarek

(2019). Race and gender biases in student evaluations of teachers. Economics Letters, 179, 66–71.

22.

Cotter

D. A.

Hermsen

J. M.

Ovadia

Vanneman

(2001). The glass ceiling effect*. Social Forces, 80(2), 655–681. https://doi.org/1.1353/sof.2001.0091

23.

Cuddy

A. J. C.

Fiske

S. T.

Glick

(2004). When professionals become mothers, warmth doesn’t cut the ice. Journal of Social Issues, 60(4), 701–718. https://doi.org/10.1111/j.0022-4537.2004.00381.x

24.

Davison

Price

(2009). How do we rate? An evaluation of online student evaluations. Assessment & Evaluation in Higher Education, 34(1), 51–65. https://doi.org/10.1080/02602930801895695

25.

Doubleday

A. F.

Lee

L. M. J.

(2016). Dissecting the voice: Health professions students’ perceptions of instructor age and gender in an online environment and the impact on evaluations for faculty. Anatomical Sciences Education, 9(6), 537–544.

26.

Eagly

A. H.

Karau

S. J.

(2002). Role congruity theory of prejudice toward female leaders. Psychological Review, 109(3), 573–598. https://doi.org/10.1037/0033-295X.109.3.573

27.

Feldman

K. A.

(1993). College students’ views of male and female college teachers: Part II—Evidence from students’ evaluations of their classroom teachers. Research in Higher Education, 34(2), 151–211. https://doi.org/1.1007/BF00992161

28.

Gómez Cama

Larrán

M. J.

Andrades Peña

F. J

. (2016). Gender differences between faculty members in higher education: A literature review of selected higher education journals. Educational Research Review, 18, 58–69. https://doi.org/1.1016/j.edurev.2016.03.001

29.

Gotlieb

Milliman

(2005). Do student grades affect student numeric ratings of marketing professors? Applying attribution theory to help answer this question. Journal for Advancement of Marketing Education, 7, 11–26.

30.

Gwet

K. L.

(2021). Handbook of inter-rater reliability: The definitive guide to measuring the extent of agreement among raters (5th ed.). Advanced Analytics, LLC. https://www.agreestat.com/books/cac5/chapter1/chap1.pdf

31.

Haladyna

Hess

R. K.

(1994). The detection and correction of bias in student ratings of instruction. Research in Higher Education, 35(6), 669–687. https://doi.org/1.1007/BF02497081

32.

Hoorens

Dekkers

Deschrijver

(2021). Gender bias in student evaluations of teaching: Students’ self-affirmation reduces the bias by lowering evaluations of male professors. Sex Roles, 84(1/2), 34–48.

33.

Jackson

J. F. L.

O’Callaghan

E. M.

(2009). What do we know about glass ceiling effects? A taxonomy and critical review to inform higher education research. Research in Higher Education, 50(5), 460–482. https://doi.org/1.1007/s11162-009-9128-9

34.

Joint Committee on Standards for Educational Evaluation (with Gullickson, A. R., & Howard, B.). (2009). The personnel evaluation standards: How to assess systems for evaluating educators (2nd ed). Corwin Press.

35.

Jonson

J. L.

Geisinger

K. F.

(Eds.). (2022). Fairness in educational and psychological testing: Examining theoretical, research, practice, and policy implications of the 2014 standards. American Educational Research Association. https://doi.org/1.2307/j.ctv2kzv0fw

36.

Joye

S. W.

Wilson

J. H.

(2015). Professor age and gender affect student perceptions and grades. Journal of the Scholarship of Teaching and Learning, 15(4), 126–138.

37.

Kreitzer

R. J.

Sweet-Cushman

(2022). Evaluating student evaluations of teaching: A review of measurement and equity bias in SETs and recommendations for ethical reform. Journal of Academic Ethics, 20, 73–84. https://doi.org/1.1007/s10805-021-09400-w

38.

Ladha

Bharwani

McLaughlin

Stelfox

H. T.

Bass

(2017). The effect of white coats and gender on medical students perceptions of physicians. BMC Medical Education, 17(1), 93. https://doi.org/10.1186/s12909-017-0932-1

39.

Lammers

W. J.

Byrd

A. A.

(2019). Student gender and instructor gender as predictors of student-instructor rapport. Teaching of Psychology, 46(2), 127–134.

40.

Laube

Massoni

Sprague

Ferber

A. L.

(2007). The impact of gender on the evaluation of teaching: What we know and what we can do. NWSA Journal, 19(3), 87–104.

41.

Macnell

Driscoll

Hunt

A. N.

(2015). What’s in a name: Exposing gender bias in student ratings of teaching. Innovative Higher Education, 40(4), 291–303. Education Database. https://doi.org/1.1007/s10755-014-9313-4

42.

Marsh

H. W.

(1987). Students’ evaluations of University teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11(3), 253–388. https://doi.org/1.1016/0883-0355(87)90001-2

43.

Marsh

H. W.

(2007). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In Perry

R. P.

Smart

J. C.

(Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 319–383). Springer Netherlands. http://link.springer.com/1.1007/1-4020-5742-3_9

44.

Marsh

H. W.

Bornmann

Mutz

Daniel

H.-D.

O’Mara

(2009). Gender effects in the peer reviews of grant proposals: A comprehensive meta-analysis comparing traditional and multilevel approaches. Review of Educational Research, 79(3), 1290–1326. https://doi.org/1.3102/0034654309334143

45.

McPherson

Smith-Lovin

Cook

J. M.

(2001). Birds of a feather: Homophily in social networks. Annual Review of Sociology, 27, 415–444. https://doi.org/10.1146/annurev.soc.27.1.415

46.

Mendez

J. M.

Mendez

J. P.

(2018). The gender effect in student selection of professors for classes. NASPA Journal About Women in Higher Education, 11(1), 74–88.

47.

Mitchell

K. M. W.

Martin

(2018). Gender bias in student evaluations. PS: Political Science & Politics, 51(3), 648–652. https://doi.org/1.1017/S104909651800001X

48.

Nesdoly

Tulk

Mantler

(2020). The effects of perceived professor competence, warmth and gender on students’ likelihood to register for a course. Assessment & Evaluation in Higher Education, 45(5), 666–679.

49.

Onwuegbuzie

A. J.

Daniel

L. G.

Collins

K. M. T.

(2009). A meta-validation model for assessing the score-validity of student teaching evaluations. Quality & Quantity, 43(2), 197–209. https://doi.org/10.1007/s11135-007-9112-4

50.

Perez-Felkner

Erichsen

Chen

Ramirez Surmeier

Shore

(2024). Computing education interventions to increase gender equity from 2000 to 2020: A systematic literature review. Review of Educational Research, 95(3), 536–580. https://doi.org/10.3102/00346543241241536

51.

Pigott

T. D.

Polanin

J. R.

(2020). Methodological guidance paper: High-quality meta-analysis in a systematic review. Review of Educational Research, 90(1), 24–46. https://doi.org/1.3102/0034654319877153

52.

Redding

(2019). A teacher like me: A review of the effect of student–teacher racial/ethnic matching on teacher perceptions of students and student academic and behavioral outcomes. Review of Educational Research, 89(4), 499–535. https://doi.org/1.3102/0034654319853545

53.

Rivera

L. A.

Tilcsik

(2019). Scaling down inequality: Rating scales, gender bias, and the architecture of evaluation. American Sociological Review, 84(2), 248–274. Business Source Ultimate.

54.

Schönrock-Adema

Lubarsky

Chalk

Steinert

Cohen-Schotanus

(2013). “What would my classmates say?” An international study of the prediction-based method of course evaluation. Medical Education, 47(5), 453–462. https://doi.org/10.1111/medu.12126

55.

Shadish

W. R.

Cook

T. D.

Campbell

D. T.

(2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.

56.

Shen

(2013). Despite improvements, female scientists continue to face discrimination, unequal pay and funding disparities. Nature, 495(3), 22–24. https://doi.org/1.1038/495022a

57.

Smith

Anderson

K. J.

(2005). Students’ ratings of professors: The teaching style contingency for Latino/a professors. Journal of Latinos & Education, 4(2), 115–136. Academic Search Ultimate.

58.

Spoor

J. R.

Lehmiller

J. J.

(2014). The impact of course title and instructor gender on student perceptions and interest in a women’s and gender studies course. PLoS One, 9(9), 1–7. Food Science Source.

59.

Spooren

Brockx

Mortelmans

(2013). On the validity of student evaluation of teaching the state of the art. Review of Educational Research, 83(4), 598–642. https://doi.org/1.3102/0034654313496870

60.

Spooren

Vandermoere

Vanderstraeten

Pepermans

(2017). Exploring high impact scholarship in research on student’s evaluation of teaching (SET). Educational Research Review, 22, 129–141. https://doi.org/1.1016/j.edurev.2017.09.001

61.

Sprague

Massoni

(2005). Student evaluations and gendered expectations: What we can’t count can hurt us. Sex Roles, 53(11), 779–793. https://doi.org/10.1007/s11199-005-8292-4

62.

Stark

Freishtat

(2014). An evaluation of course evaluations. ScienceOpen Research. https://www.scienceopen.com/document/id/ad8a9ac9-8c60-432a-ba20-4402a2a38df4

63.

Stuber

J. M.

Watson

Carle

Staggs

. (2009). Gender expectations and on-line evaluations of teaching: Evidence from RateMyProfessors.com. Teaching in Higher Education, 14(4), 387–399.

64.

Taparia

Lenka

(2022). An integrated conceptual framework of the glass ceiling effect. Journal of Organizational Effectiveness: People and Performance, 9(3), 372–40. https://doi.org/1.1108/JOEPP-06-2020-0098

65.

Thomas

D. R.

(2006). A general inductive approach for analyzing qualitative evaluation data. American Journal of Evaluation, 27(2), 237–246. https://doi.org/1.1177/1098214005283748

66.

UNESCO International Institute for Higher Education in Latin America and the Caribbean. (2022). IESALC Annual Report 2021. https://www.iesalc.unesco.org/sites/default/files/medias/fichiers/2024/07/IESALC_230609_Memoria_Anual_2021_ENG_v01_.pdf?hub=64

67.

UNESCO. (2021). Women in higher education: Has the female advantage put an end to gender inequalities. UNESCO International Institute for Higher Education in Latin America and the Caribbean.

68.

Valencia

(2025). On the comparability of SET scores: Measurement invariance across programs, departments, and time. Assessment & Evaluation in Higher Education, 1–15. https://doi.org/10.1080/02602938.2025.2508754

69.

Wachtel

H. K.

(1998). Student evaluation of college teaching effectiveness: A brief review. Assessment & Evaluation in Higher Education, 23(2), 191–212. https://doi.org/10.1080/0260293980230207

70.

Waggoner Denton

Veloso

. (2018). Changes in syllabus tone affect warmth (but not competence) ratings of both male and female instructors. Social Psychology of Education: An International Journal, 21(1), 173–187. Education Database; Psychology Database; Research Library. https://doi.org/1.1007/s11218-017-9409-7

71.

Wallace

B. C.

Small

Brodley

C. E.

Lau

Trikalinos

T. A.

(2012). Deploying an interactive machine learning system in an evidence-based practice center: Abstrackr. Proceeding of the ACM International Health Informatics Symposium (IHI), 819–824.

72.

Weinkle

L. J.

Stratford

J. M.

Lee

L. M. J.

(2020). Voice in digital education: The impact of instructor’s perceived age and gender on student learning and evaluation. Anatomical Sciences Education, 13(1), 59–7.

73.

Weisshaar

(2017). Publish and Perish? An assessment of gender gaps in promotion to tenure in academia. Social Forces, 96(2), 529–56. https://doi.org/1.1093/sf/sox052

74.

Wilson

S. M.

Anagnostopoulos

(2021). Methodological guidance paper: The craft of conducting a qualitative review. Review of Educational Research, 91(5), 651–67. https://doi.org/1.3102/00346543211012755

75.

Wong

J. S.

Bouchard

(2021). Are students gender-neutral in their assessment of online teaching staff? Assessment & Evaluation in Higher Education, 46(5), 719–739. https://doi.org/10.1080/02602938.2020.1805410

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.22 MB