This study estimated the interrater reliability of medical student evaluations of clinical teaching. Data consisted of 1,570 ratings evaluating 147faculty over a 4-year period in a 3rd-year internal medicine clerkship. The number of ratings a typical faculty member receives in a year was also calculated and used to extrapolate the standard error of measurementfor data typically available to evaluatefaculty at different time intervals. The data available to evaluate afaculty member after I year was not adequate, but improved substantially at the 5-to 7-yearmark when a faculty member is typically evaluated for promotion and tenure.
Get full access to this article
View all access options for this article.
References
1.
Albanese, M. A.
, Schuldt, S. S., Case, D. E., & Brown, D. (1991). Academic Medicine, 66,26-28.
2.
Brennen, R. L.
(1983). Elements of generalizability theory. Iowa City, IA: American College Testing Program.
3.
Ebel, E. L.
(1951). Estimation of the reliability of ratings. Psychometrika, 16, 407-424.
4.
Gilmore, G. M.
, Kane, M. T., & Naccarato, R. W. (1978). The generalizability of instruction: Estimates of teacher and course components. Journal of Educational Measurement, 15,1-13.
5.
Irby, D.
, & Rakestra, P. (1981). Evaluating clinical teaching in medicine. Journal of Medical Education, 56, 181-186.
6.
Magnesson, D.
(1967). Test theory. Reading, MA: Addison-Wesley.
7.
Marsh, H. W.
(1981). The use of path analysis to estimate teacher and course effects in student ratings of instructional effectiveness. Applied Psychological Measurement, 6,47-60.
8.
Marsh, H. W.
(1984). Students' evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology, 76,707-754.
9.
Risucci, D. A.
, Lutsky, L., Rosati, R. J., & Tortolani, A. J. (1992). Reliability and accuracy of resident evaluations of surgical faculty. Evaluation & the Health Professions, 15, 313-325.
