In criterion-referenced evaluation, several procedures for setting absolute standards of performance are available. However, it is not known if a given standard-setting procedure will yield consistent evaluation outcomes. To determine this, one of three parallel certifying examinations was administered to three classes of second year medical students. The examinations contained two item types and the standards were set by two procedures. The student responses to both item types were analyzed by the Rasch latent trait model. The results indicated that the standard-setting procedure is a significant factor in the determination of the evaluation outcome.
Get full access to this article
View all access options for this article.
References
1.
Angoff, W.H.Scales, norms and equivalent scores. In R. L. Thorndike (Ed.), Educational Measurement (2nd ed.). Washington, D. C.: American Council on Education, 1971.
2.
Andrew, B.J. and Hecht, J.T.A preliminary investigation of two procedures for setting Examination standards . EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1976, 36, 45-50.
Ebel, R.L.Essentials of educational measurement. Englewood Cliffs, N. J.: Prentice-Hall, 1972 .
5.
Glass, G.V.Standards and criteria. Journal of Educational Measurement , 1978, 15, 237-261.
6.
Hambleton, R.K.Test score validity and standard-setting methods. In R. A. Berk (Ed.) Criterion-referenced measurement: The state of the art. Baltimore and London : The Johns Hopkins University Press, 1980.
7.
Hubbard, J.P.Measuring medical education: The tests and the experience of the National Board of Medical Examiners. Philadelphia: Lea & Febiger, 1978.
8.
Jaeger, R.M.Measurement consequences of selected standard-setting models. In M. A. Bunda and S. R. Sanders (Eds.) Practices and problems in competency-based measurement . Washington, D. C.: National Council on Measurement in Education, 1979.
9.
Linn, R.L.Issues of validity in measurement for competency-based programs. In M. A. Bunda and J. R. Sanders (Eds.) Practices and problems in competency-based measurement. Washington, D. C.: National Council on Measurement in Education, 1979.
10.
Millman, J.Passing scores and test lengths for domain-referenced measures. Review of Educational Research, 1973, 43, 205-216.
11.
Messick, S.The standard problem: Meaning and values in measurement and evaluation . American Psychologist, 1975, 30, 955-966.
12.
Nedelsky, L.Absolute grading standards for objective tests. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1954, 14, 3-19.
13.
Shepard, L.A.Measurements consequences of selected standard-setting models. In M. A. Bunda and S. R. Sanders (Eds.), Practices and problems in competency-based measurements . Washington, D. C.: National Council on Measurement in Education, 1979.
14.
Wright, B.D. and Mead, R.J. Bical: Calibrating items and scales with the Rasch model. Statistical Laboratory, Dept. of Education, The University of Chicago, Research memorandum 23, January of 1977.
15.
Wright, B.W. and Stone, M.H.Best test design: Rasch measurement. Chicago: Mesa Press, 1979.
