Student evaluations of teaching are among the most accepted and important indicators of college teachers’ performance. However, faculty and administrators can overinterpret small variations in mean teaching evaluations. The current research examined the effect of including statistical information on the interpretation of teaching evaluations. Study 1 (N = 121) showed that faculty members interpreted small differences between mean course evaluations even when confidence intervals and statistical tests indicated the absence of meaningful differences. Study 2 (N = 183) showed that differences labeled as nonsignificant still influenced perceptions of teaching qualifications and teaching ability. The results suggest the need for increased emphasis on the use of statistics when presenting and interpreting teaching evaluation data.
AbramiP. C. (2001). Improving judgments about teaching effectiveness using teacher rating forms. New Directions for Institutional Research, 109, 59–87. doi:10.1002/ir.4
2.
AlgozzineB.BeattieJ.BrayM.FlowersC.GretesJ.HowleyL.…SpoonerF. B. (2004). Student evaluation of college teaching: A practice in search of principles. College Teaching, 52, 134–141.
3.
BeranT.ViolatoC.KlineD.FrideresJ. (2005). The utility of student ratings of instruction for students, faculty, and administrators: A ‘consequential validity’ study. Canadian Journal of Higher Education, 35, 49–70.
4.
BeranT. N.RokoshJ. L. (2009). The consequential validity of student ratings: What do instructors really think?Alberta Journal of Educational Research, 55, 497–511.
5.
BoysenG. A.KellyT. J.RaeslyH. N.CasnerR. W. (2013). The (mis) interpretation of teaching evaluations by college faculty and administrators. Assessment & Evaluation in Higher Education, 39, 641–656. doi:10.1080/02602938.2013.860950
6.
ChenZ.KempS. (2012). Lie hard: The effect of self-assessments on academic promotion decisions. Journal of Economic Psychology, 33, 578–589. doi:10.1016/j.joep.2011.11.004
7.
CohenR. J.SwerdlikM.SturmanE. (2014). Psychological testing and assessment: An introduction to tests and measurement. (8th ed.). New York, NY: McGraw-Hill.
8.
CortinaJ. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98–104. doi:10.1037/0021-9010.78.1.98
9.
CummingG.FinchS. (2005). Inference by eye: Confidence intervals and how to read pictures of data. American Psychologist, 60, 170–180. doi:10.1037/0003-066X.60.2.170
10.
d'ApolloniaS.AbramiP. C. (1997). Navigating student ratings of instruction. American Psychologist, 52, 1198–1208. doi:10.1037/0003-066X.52.11.1198
11.
FranklinJ. (2001). Interpreting the numbers: Using a narrative to help others read student evaluations of your teaching accurately. New Directions for Teaching and Learning, 87, 59–87. doi:10.1002/tl.10001
12.
GalbraithC. S.MerrillG. B.KlineD. M. (2012). Are student evaluations of teaching effectiveness valid for measuring student learning outcomes in business related classes? A neural network and Bayesian analyses. Research in Higher Education, 53, 353–374. doi:10.1007/s11162-011-9229-0
13.
GravestockP.GreenleafE.BoggsA. M. (2009). The validity of student course evaluations: An eternal debate?Collected Essays on Learning and Teaching, 2, 152–158.
14.
GravestockP.Gregor-GreenleafE. (2008). Student course evaluations: Research, models and trends. Toronto, Canada: Higher Education Quality Council of Ontario.
15.
GreenwaldA. G. (1997). Validity concerns and usefulness of student ratings of instruction. American Psychologist, 52, 1182–1186. doi:10.1037/0003-066X.52.11.1182
16.
GreenwaldA. G.GillmoreG. M. (1997). Grading leniency is a removable contaminant of student ratings. American Psychologist, 52, 1209–1217. doi:10.1037/0003-066X.52.11.1209
17.
MarshH. W.RocheL. A. (1997). Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist, 52, 1187–1197. doi:10.1037/0003-066X.52.11.1187
18.
MussweilerT.EnglichB. (2005). Subliminal anchoring: Judgmental consequences and underlying mechanisms. Organizational Behavior and Human Decision Processes, 98, 133–143. doi:10.1016/j.obhdp.2004.12.002
19.
ShaoL. P.AndersonL. P.NewsomeM. (2007). Evaluating teaching effectiveness: Where we are and where we should be. Assessment & Evaluation in Higher Education32, 355–371. doi:10.1080/02602930600801886
20.
SmithA. R.WindschitlP. D. (2011). Biased calculations: Numeric anchors influence answers to math equations. Judgment and Decision Making, 6, 139–146.
21.
SmithD. L.CookP.BuskistW. (2011). An experimental analysis of the relation between assigned grades and instructor evaluations. Teaching of Psychology, 38, 225–228. doi:10.1177/0098628311421317
StreinerD. L. (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80, 99–103. doi:10.1207/S15327752JPA8001_18
24.
TheallM.FranklinJ. (2001). Looking for bias in all the wrong places: A search for truth or a witch hunt in student ratings of instruction?New Directions for Institutional Research, 109, 45–56. doi:10.1002/ir.3
25.
TverskyA.KahnemanD. (1971). Belief in the law of small numbers. Psychological Bulletin, 76, 105–110. doi:10.1037/h0031322
26.
TverskyA.KahnemanD. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131.
27.
WilsonT. D.HoustonC. E.EtlingK. M.BrekkeN. (1996). A new look at anchoring effects: Basic anchoring and its antecedents. Journal of Experimental Psychology: General, 125, 387–402. doi:10.1037/0096-3445.125.4.387
28.
ZuckermanM.HodginsH. S.ZuckermanA.RosenthalR. (1993). Contemporary issues in the analysis of data: A survey of 551 psychologists. Psychological Science, 4, 49–53. doi:10.1111/j.1467-9280.1993.tb00556.x
