Measuring the Instructional Validity of Clinical Simulation Problems

Abstract

Written clinical simulation problems in two formats-forced-choiced and essay were used to test junior and senior medical students at the conclusion of their pediatric rotations. A comparison was made in the performance of students with varying levels of clinical experience. There seemed to be no consistent pattern of improvement with increased instructional time on the forced-choice management problems. Junior students did show improvement over time on the essay management problems, which also seemed to reflect the increased problem-solving and organizational skills of seniors in comparison to juniors. Correlations across problems and correlations between the clinicalproblem test and other measurement techniques were very weak, partly due to the low sampling reliability of clinical simulation problems. In this study, clinical simulation problemsfailed to demonstrate responsiveness to development and maturation in the problem-solving approach to patient care.

Get full access to this article

View all access options for this article.

References

ANGOFF, W. H. (1953) "Test reliability and effective test length." Psychometrica 1-16.

BERNER, E. S. , L. A. HAMILTON , and W. R. BEST (1974) "A new approach to evaluating problem-solving in medical students." J. of Medical Education 49: 666-672.

BLUMBERG, P. (1981) "Clinical evaluation: issues of examination format." Evaluation & the Health Professions 4: 316-329.

GUILFORD, J. P. and G. FRUCHTER (1978) Fundamental Statistics in Psychology and Education. New York: McGraw-Hill.

HARASYM, P. , J. BAUMBER , H. BRYANT , D. FUNDYTUS , R. PRESHAW , M. WATANABE , and G. WYSE (1980) "An evaluation of the clinical problem-solving process using stimulation technique." Medical Education 14: 381-386.

HUBBARD, J. P. and C. F. SCHUMACHER (1971) Measuring Medical Education: The Tests and Procedures of the National Board of Examiners. Philadelphia: Lea & Febiger.

LEVINE, H. G. and J. R. NOAK (1968) The Evaluation of Complex Educational Outcomes. State of Illinois: Office of the Superintendent of Public Instruction.

LEWY, A. and C. H. McGUIRE (1966) "A study of alternative approaches in estimating the reliability of unconventional tests." Presented at the annual meeting of the American Educational Research Association, Chicago.

McCARTHY, W. H. (1966) "An assessment of the influence of cueing items in objective examinations." J. of Medical Education 41: 263-266.

10.

McGUIRE, C. H. and D. BABBOTT (1967) "Simulation technique in the measurement of problem-solving skills." J. Of Educ. Measurement 4: 1-10.

11.

National Board of Medical Examiners (1981) "Funding for development efforts related to an improved final licensing examination." Fall: 2-3.

12.

National Board of Medical Examiners (1979) Guidelines for Use of National Board Examinations. Philadelphia: Author.

13.

NATTRESS, L. W. (1969) "Orthopaedic certification examinations." J. of Medical Education 44: 890-895.

14.

NEWBLE, D. I. , A. BAXTER , and R. C. ELMSIE (1979) "Comparison of multiple choice tests and free response tests in examinations of clinical competence." Medical Education 13: 263-268.

15.

O'DONOHUE, W. J. and J. F. WERGIN (1978) "Evaluation of medical students during a clinical clerkship in internal medicine." J. of Medical Education 53: 55-58.

16.

PAGE, G. G. and D. W. FIELDING (1980) "Performance on PMPs and performance in practice: are they related? J. of Medical Education 55: 529-537.

17.

RIMOLDI, H.J.A. (1961) "The test of diagnostic skills." J. of Medical Education 36: 73-79.

18.

VAUGHAN, V. C. (1979) "The patient management problem as an evaluative instrument." Pediatrics in Review 1: 67-76.

19.

WINNE, P. H. and M. J. BELFRY (1982) "Interpretive problems when correcting for attenuation." J. of Educ. Measurement 19: 125-134.