Confirming the Factor Structure of a Research-Based Mid-Semester Evaluation of College Teaching

Abstract

End-of-semester evaluations provide scalable data for university administrators, but typically do not provide instructors with timely feedback to inform their teaching practices. Midsemester evaluations have the potential to provide instructors with beneficial formative feedback that can contribute to improved teaching practices and student engagement. However, existing research on the construction of valid, reliable midsemester tools is rare, and there are no existing midsemester evaluation scales that were constructed using education research and psychometric analysis. To address this gap, we designed and piloted a midsemester evaluation of teaching with 29 instructors and 1,350 undergraduate students. We found evidence that our Mid-Semester Evaluation of College Teaching (MSECT) is a valid and reliable measure of four constructs of effective teaching: classroom climate, content, teaching practices, and assessment. Furthermore, our factor structure remained consistent across instructor genders, providing evidence that the MSECT may be less susceptible to gender bias than prior student evaluation measures.

Keywords

student evaluations of teaching college teaching psychometrics

Get full access to this article

View all access options for this article.

References

Alcott

(2017). Does teacher encouragement influence students’ educational progress? A propensity-score matching analysis. Research in Higher Education, 58(7), 773–804. https://doi.org/10.1007/s11162-017-9446-2

Alderman

M. K.

(2013). Motivation for achievement: Possibilities for teaching and learning. Lawrence Erlbaum.

Alexander

P. A.

Winne

P. H.

(2006). Handbook of educational psychology. Lawrence Erlbaum.

Ambrose

Bridges

DiPietro

Lovett

Norman

(2010). How Learning Works: 7 Research-based Principles for Smart Teaching. Jossey-Bass.

Apodaca

Grad

(2005). The dimensionality of student ratings of teaching: Integration of uni- and multidimensional models. Studies in Higher Education, 30(6), 723–748. https://doi.org/10.1080/03075070500340101

Atkinson

Siew Leng

(2013). Improving assessment processes in higher education: Student and teacher perceptions of the effectiveness of a rubric embedded in a LMS. Australasian Journal of Educational Technology, 29(5), 651–666.

Bain

(2012). What the best college students do. The Bell Press of Harvard University Press.

Chickering

A. W.

Gamson

Z. F.

(1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 39(7), 3–7.

Cohen

P. A.

(1980). Effectiveness of student-rating feedback for improving college instruction: A meta-analysis of findings. Research in Higher Education, 13(4), 321–341. https://doi.org/10.1007/BF00976252

10.

Corkin

D. M.

Horn

Pattison

(2017). The effects of an active learning intervention in biology on college students’ classroom motivational climate perceptions, motivation, and achievement. Educational Psychology, 37(9), 1106–1124. https://doi.org/10.1080/01443410.2017.1324128

11.

Corkin

D. M.

S. L.

Wolters

C. A.

Wiesner

(2014). The role of college classroom climate on academic procrastination. Learning and Individual Differences, 32, 294–303.

12.

Costello

M. L.

Weldon

Brunner

(2002). Reaction cards as a formative evaluation tool: Students’ perceptions of how their use impacted classes. Assessment and Evaluation in Higher Education, 27(1), 23–33. https://doi.org/10.1080/02602930120105036

13.

Donlan

A. E.

Loughlin

S. M.

Byrne

V. L.

(2019). The Fearless Teaching Framework: A model to synthesize foundational education research for university instructors. To Improve the Academy: A Journal of Educational Development, 38(1), 33–49. https://doi.org/10.1002/tia2.20087

14.

Frymier

A. B.

Shulman

G. M.

(1995). “What’s in it for me?” Increasing content relevance to enhance students’ motivation. Communication Education, 44(1), 40–50.

15.

Hammer

(2016). Construct replicability calculator: A microsoft excel-based tool to calculate the Hancock and Mueller (2001) H index. http://DrJosephHammer.com/

16.

Hammonds

Mariano

G. J.

Ammons

Chambers

(2017). Student evaluations of teaching: Improving teaching quality in higher education. Perspectives: Policy and Practice in Higher Education, 21(1), 26–33. https://doi.org/10.1080/13603108.2016.1227388

17.

Hampton

S. E.

Reiser

R. A.

(2004). Effects of a theory-based feedback and consultation process on instruction and learning in college classrooms. Research in Higher Education, 45(5), 497–527.

18.

Hancock

G. R.

Mueller

R. O.

(2001). Rethinking construct reliability within latent variable systems. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.) Structural equation modelsing: Present and future (pp. 195–216). Chicago: Scientific Software International.

19.

Hattie

Timperley

(2007). The power of feedback the meaning of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

20.

Henson

R. K.

Roberts

J. K.

(2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66(3), 393–416.

21.

Howard

T. C.

(2001). Telling their side of the story: African-American students’ perceptions of culturally relevant teaching. The Urban Review: Issues and Ideas in Public Education, 33(2), 131–149. https://doi.org/10.1023/A:1010393224120

22.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.

23.

Kember

Leung

D. Y. P.

Kwan

K. P.

(2002). Does the use of student feedback questionnaires improve the overall quality of teaching? Assessment & Evaluation in Higher Education, 27(5), 411–425.

24.

Kline

R. B.

(2011). Principles and practice of structural equation modeling. Guilford Press.

25.

Knol

M. H.

Dolan

C. V.

Mellenbergh

G. J.

van der Maas

H. L. J.

(2016). Measuring the quality of university lectures: Development and validation of the Instructional Skills Questionnaire (ISQ). PLoS ONE, 11(2), 1–21.

26.

Lave

Wenger

(1991). Situated learning: Legitimate peripheral participation. Cambridge University Press.

27.

Marsh

H. W.

(1982). SEEQ: A reliable, valid, and useful instrument for collecting students’ evaluations of university teaching. British Journal of Educational Psychology, 52(1), 77–95.

28.

Marsh

H. W.

(1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11(3), 253–388. https://doi.org/10.1016/0883-0355(87)90001–2

29.

Marsh

H. W.

Roche

L. A.

(2000). Effects of grading leniency and low workload on students’ evaluations of teaching: Popular myth, bias, validity, or innocent bystanders? Journal of Educational Psychology, 92(1), 202–228. http://10.0.4.13/0022-0663.92.1.202

30.

McNeish

Stapleton

L. M.

Silverman

R. D.

(2017). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22(1), 114–140.

31.

Morin

A. J. S.

Marsh

H. W.

Nagengast

Scalas

L. F.

(2014). Doubly latent multilevel analyses of classroom climate: An illustration. The Journal of Experimental Education, 82(2), 143–167. https://doi.org/10.1080/00220973.2013.769412

32.

Murphy

P. K.

Alexander

P. A.

(2000). A motivated look at motivational terminology. (Special Issue). Contemporary Educational Psychology, 25, 3–53.

33.

Muthén

Satorra

(1995). Complex sample data in structural equation modeling. Sociological Methodology, 25, 267–316.

34.

Oon

Benson

Kam

C. C. S.

(2017). Psychometric quality of a student evaluation of teaching survey in higher education.” Assessment & Evaluation in Higher Education, 42(5), 788–800. https://doi.org/10.1080/02602938.2016.1193119

35.

Overall

J. U.

Marsh

H. W.

(1979). Midterm feedback from students: Its relationship to instructional improvement and students’ cognitive and affective outcomes. Journal of Educational Psychology, 71(6), 856–865. https://doi.org/10.1037/0022-0663.71.6.856

36.

Peugh

J. L.

Enders

C. K.

(2004). Missing data in educational research: A review of reporting practices and suggestions for improvement. Review of Educational Research, 74, 525–556.

37.

Preacher

K. J.

Zhang

Kim

Mels

(2013). Choosing the optimal number of factors in exploratory factor analysis: A model selection perspective. Multivariate Behavioral Research, 48, 28–56. https://doi.org/10.1080/00273171.2012.710386

38.

Richardson

J. T. E.

(2005). Instruments for obtaining student feedback: A review of the literature. Assessment & Evaluation in Higher Education, 30(4), 387–415.

39.

Skinner

Furrer

Marchand

Kindermann

(2008). Engagement and disaffection in the classroom: Part of a larger motivational dynamic? Journal of Educational Psychology, 100(4), 765–781. https://doi.org/10.1037/a0012840

40.

Socha

(2013). A hierarchical approach to students’ assessments of instruction. Assessment & Evaluation in Higher Education, 38(1), 94–113. https://doi.org/10.1080/02602938.2011.604713

41.

Spooren

(2010). On the credibility of the judge. Studies in Educational Evaluation, 36(4), 121–131. http://10.0.3.248/j.stueduc.2011.02.001

42.

Spooren

Brockx

Mortelmans

(2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598–642. https://doi.org/10.3102/0034654313496870

43.

Spooren

Vandermoere

Vanderstraeten

Pepermans

(2017). Exploring high impact scholarship in research on Student’s Evaluation of Teaching (SET). Educational Research Review, 22, 129–141. https://doi.org/10.1016/j.edurev.2017.09.001

44.

Surgenor

P. W. G.

(2013). Obstacles and opportunities: Addressing the growing pains of summative student evaluation of teaching. Assessment & Evaluation in Higher Education, 38(3), 363–376. https://doi.org/10.1080/02602938.2011.635247

45.

Thompson

(2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. American Psychological Association.

46.

Vygotsky

L. S.

(1978). Mind in society: The development of higher psychological processes. Harvard University Press.

47.

Wass

Timmermans

Harland

McLean

(2018). Annoyance and frustration: Emotional responses to being assessed in higher education. Active Learning in Higher Education. Advance online publication. https://doi.org/10.1177/1469787418762462

48.

Wentzel

K. R.

Brophy

J. E.

(2014). Motivating students to learn. Routledge.

49.

Wigfield

Eccles

J. S.

(2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68–81. https://doi.org/10.1006/ceps.1999.1015

50.

Zabaleta

(2007). The use and misuse of student evaluations of teaching. Teaching in Higher Education, 12(1), 55–76.

51.

Zubrunn

McKim

Buhs

Hawley

L. R.

(2014). Support, belonging, motivation, and engagement in the college classroom: A mixed method study. Instructional Science, 42, 661–684.