This study explores whether a clinical teaching effectiveness (CTE) instrument provides valid scores for summative faculty assessment. The sample included all CTE instruments (n = 10,087) that learners (N = 1,194) completed to assess clinical teachers (N = 872) during 1 academic year. The authors investigated response processes (e.g., missing data, straight-line responses, level of learner), internal structure (e.g., confirmatory and exploratory factor analysis), teaching ratings by learner group (medical student or resident), and relation to other variables (e.g., correlation with global rating). Response processes identified a high prevalence of straight-line responses (same rating across all items) and differential patterns of missing data by learner group. Medical students rated their teachers higher than residents, and CTE scores had different factor structures depending on learner group. High correlation coefficients of CTE items with a single rating of overall teaching performance suggest that learners consider global performance when assessing clinical teaching performance.
Abrami, P.C., & d'Apollonia, S. (1990). The dimensionality of ratings and their use in personnel decisions. In M. Theall & J. Franklin (Eds.), Student ratings of instruction: Issues for improving practice; New directions for teaching and learning (No. 43, pp. 97-111). San Francisco: Jossey-Bass.
2.
Albanese, M.A. (1999). Challenges in using rater judgments in medical education. Journal of Evaluation in Clinical Practice, 6(3), 305-319.
3.
Albanese, M.A., Prucha, C., Barnet, J., & Gjerde, C. (1997). The effect of right or left placement of the positive response on Likert-type scales used by medical students for rating instruction . Academic Medicine, 72(7), 627-630.
4.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing . Washington, DC: American Educational Research Association.
5.
Assessing change in medical education—The road to implementation [Special issue]. (1993). Academic Medicine, 68(6).
6.
Cashin, W.E. (1990). Student ratings of teaching: Recommendations for use (Tech. Rep. No. 22). Manhattan, KS: Kansas State University, Center for Faculty Evaluation and Development.
7.
Cashin, W.E., & Downey, R.G. (1992). Using global student rating items for summative evaluation. Journal of Educational Psychology, 84(4), 563-572.
8.
Copeland, H.L., & Hewson, M.G. (2000). Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center. Academic Medicine, 75(2), 161-166.
9.
Dillman, D. (2000). Mail and internet surveys: The tailored design method. New York: Wiley.
10.
Dillman, D., Tortora, R., Conradt, J., & Bowker, D. (1998, August). Influence of plain versus fancy design on response rates for Web surveys. Paper presented at the annual meeting of the American Statistical Association, Dallas, TX.
11.
Hayward, R., Williams, B., Gruppen, L., & Rosenbaum, D. (1995). Measuring attending physician performance in a general medicine outpatient clinic. Journal of General Internal Medicine, 10, 504-509.
12.
Howell, L., Hogarth, M., & Anders, T. (2002). Creating a mission-based reporting system at an academic health center. Academic Medicine, 77(2), 130-138.
13.
Hoyle, R., & Panter, A. (1995). Writing about structural equation models. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues and applications (pp. 158-176). Thousand Oaks, CA: Sage.
14.
Irby, D.M. (1978). Clinical teacher effectiveness in medicine. Journal of Medical Education, 53, 808-815.
15.
Irby, D.M., Gillmore, G., & Ramsey, P. (1987). Factors affecting ratings of clinical teachers by medical students and residents. Journal of Medical Education , 67, 1-7.
16.
Irby, D.M., Ramsey, P., Gillmore, G., & Schaad, D. (1991). Characteristics of effective clinical teachers of ambulatory care medicine. Academic Medicine, 66(1), 54-55.
17.
Kline, R. (1998). Principles and practice of structural equation modeling. New York: Guilford.
18.
Jarrell, B., Mallot, D., Peartree, L., & Calia, F. (2002). Looking at the forest instead of counting the trees: An alternative method for measuring faculty's clinical education efforts . Academic Medicine, 77(12), 1255-1261.
19.
Johnston, M., & Gifford, R. (1996). A model for distributing teaching funds to faculty . Academic Medicine, 71(2), 138-140.
20.
Litzelman, D., Stratos, G., Marriott, D., & Skeff, K. (1998). Factorial validation of a widely disseminated educational framework for evaluating clinical teachers. Academic Medicine, 73(6), 688-695.
21.
Mallon, W., & Jones, R. (2002). How do medical schools use measurement systems to track faculty activity and productivity in teaching?Academic Medicine, 77(2), 115-123.
22.
Marsh, H.W., & Roche, L.A. (1997). Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist, 52(11), 1187-1197.
23.
McKeachie, W. (1997). Student ratings: The validity of use. American Psychologist, 52, 1218-1225.
24.
Nutter, D., Bond, J., Coller, B., D'Alessandri, R., Gewertz, B., Nora, L., et al. (2000). Measuring faculty effort and contributions in medical education. Academic Medicine, 75(2), 200-207.
25.
Parsell, G., & Bligh, J. (2001). Recent perspectives on clinical teaching. Medical Education, 35, 409-414.
26.
Paukert, J., & Richards, B. (2000). How medical students and residents describe the roles and characteristics of their influential clinical teachers. Academic Medicine, 75(8), 843-845.
27.
Roop, S., & Pangaro, L. (2001). Effect of clinical teaching on student performance during a medicine clerkship. American Journal of Medicine , 110, 205-209.
28.
Schuwirth, L.W., & van der Vleuten, C.P. (2004). Changing education, changing assessment, changing research?Medical Education , 38, 805-812.
29.
Scriven, M. (1989). The design and use of forms for the student evaluation of teaching. Instructional Evaluation, 10, 1-13.
30.
Shea, J., & Bellini, L. (2002). Evaluations of clinical faculty: The impact of level of learner and time of year. Teaching and Learning in Medicine, 14(2), 87-91.
31.
Stevens, J. (1996). Confirmatory and exploratory factor analysis. In Applied multivariate statistics for the social sciences (3rd ed., pp.362-428). Mahwah, NJ: Erlbaum.
32.
Stritter, F., Hain, J., & Grimes, D. (1975). Clinical teaching reexamined. Journal of Medical Education, 50, 876-882.
33.
Ullian, J., Bland, C., & Simpson, D. (1994). An alternative approach to defining the role of the clinical teacher. Academic Medicine, 69(10), 832-838.
34.
van der Hem-Stokroos, H., van der Vleuten, C., Daelmans, H., Haarman, H., & Scherpbier, A. (2005). Reliability of the clinical teaching effectiveness instrument. Medical Education, 39, 904-910.
35.
West, S.G., Finch, J.F., & Curran, P.J. (1995). Structural equation models with nonnormal variables: problems and remedies. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues and applications (pp. 56-75). Thousand Oaks, CA: Sage.
36.
Williams, B., Litzelman, D., Babbott, S., Lubitz, R., & Hofer, T. (2002). Validation of a global measure of faculty's clinical teaching performance. Academic Medicine, 77(2), 177-180.
37.
Williams, B., Pillsbury, M., Stern, D., & Grum, C. (2001). Comparison of resident and medical student evaluation of faculty teaching. Evaluation & the Health Professions , 24(1), 53-60.
