Confirmatory Factor Analysis of House Office Candidate Appraisals

Abstract

This article illustrates an application of confirmatory factor analysis, using the LISREL program, to retrospective analysis of rating data common in health professions education. We begin with specification of a causal model reflecting the multitraitmultimethod design of a surgical houseofficer ranking system. The model is then tested using data provided by rating system functioning over two successive years. The results suggest that although most rating measures do cluster around latent traits as hypothesized, there is more method variance in the data set than trait variance. The trait and methodfactors are also found to be highly intercorrelated. Practical implications of these results include specific modifications in the rating system, potentially leading to improvement of the system.

Get full access to this article

View all access options for this article.

References

CAMPBELL, D. T. and D. W. FISKE (1959) "Convergent and discriminant validation by the multitrait-multimethod matrix." Psych. Bull. 56: 81-105.

CLARKE, J. R. and R. S. WIGTON (1984) "Development of an objective rating system for residency applications." Presented at the meeting of the Society of University Surgeons, Chicago.

CLEMENTE, M. and W. M. MICHENER (1976) "The dean's letter of recommendation and internship performance." J. of Medical Education 51: 590-592.

FORSYTHE, G. B. , W. C. McGAGHIE , and C. P. FRIEDMAN (1986) "Construct validity of medical clinical competence measures: a multitrait-multimethod matrix study using confirmatory factor analysis." Amer. Educ. Research J. 23: 315-336.

FRIEDMAN, C. P. , W. C. TRIER , and I. HELMY (1981) "A multicomponent validity study of housestaff selection process." Annual Conference on Research in Medical Education 20: 207-212.

FRIEDMAN, R. (1983) "Fantasy land." New England J. of Medicine 308: 651-653.

GORDON, M. J. and J. A. LINCOLN (1976) "Selecting a few residents from many applicants: a new way to be fair and efficient." J. of Medical Education 51: 454-460.

GREENWALD, R. A. and S. WIENER (1976) "A standardized interviewing technique for evaluating postgraduate training applicants." J. of Medical Education 51: 912-918.

GUILFORD, J. P. (1954) Psychometric Methods. New York: McGraw-Hill.

10.

HEISE, D. R. (1975) Causal Analysis. New York: John Wiley.

11.

JÖRESKOG, K. G. and D. SORBOM (1981) LISREL User's Guide (versions V and VI). Chicago: International Educational Services.

12.

LONG, J. S. (1983) Confirmatory Factor Analysis. Beverly Hills, CA: Sage.

13.

MARSH, H. W. and D. HOCEVAR (1976) "Confirmatory factor analysis of multitraitmultimethod matrices." J. of Educ. Measurement 20: 231-248.

14.

TRIER, W. C. , C. P. FRIEDMAN , and C. G. THOMAS (1983) "Evaluation and restructuring of surgical house officer applicant ranking system." Presented at the meeting of the Association for Surgical Education, Seattle, WA.