A Comparison of Two Approaches to Setting Passing Scores Based on the Nedelsky Procedure

Abstract

Two versions of the Nedelsky procedure for setting minimum passing scores are compared. Two groups of judges, one using each version, set pass ing scores for a classroom test Comparisons of the resulting sets of passing scores are made on the ba sis of (1) the raw distributions of passing scores, (2) the consistency of pass-fail decisions between the two versions, (3) the consistency of pass-fail deci sions between each version and the passing score established by the test designer, and (4) the mean pairwise agreement between judges across groups. The two versions of the procedure are found to pro duce essentially equivalent results. In addition, a significant relationship is observed between the passing score set by a judge and that judge's level of achievement in the content area of the test.

Get full access to this article

View all access options for this article.

References

Amitrano-Paiva, R.E. , & Vu, N.V. Standards for acceptable level of performance in an objectives-based medical curriculum: A case study. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, April 1979.

Andrew, B.J. , & Hecht, J.T. A preliminary investigation of two procedures for setting examination standards . Educational and Psychological Measurement, 1976, 45, 4-9.

Angoff, W.H. Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed.). Washington DC: American Council on Education, 1971.

Brennan, R.L. , & Lockwood, R.E. A comparison of two cutting scores using generalizability theory. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco , April 1979.

Ebel, R.L. Essentials of educational measurement. Englewood Cliffs NJ: Prentice-Hall, 1972.

Glass, G.V. Standards and criteria. Journal of Educational Measurement , 1978, 15, 237-261.

Hambleton, R.K. On the use of cut-off scores with criterion-referenced tests in instructional settings. Journal of Educational Measurement, 1978, 15, 277-290.

Hollander, M. , & Wolfe, D.A. Nonparametric statistical methods. New York: Wiley, 1973.

Meskauskas, J.A. Evaluation models for criterion-referenced testing: Views regarding mastery and standard-setting. Review of Educational Research, 1976, 46, 133-158.

10.

Meskauskas, J.A. , & Webster, G.W. The American Board of Internal Medicine recertification examination process and results. Annals of Internal Medicine, 1975, 82, 577-581.

11.

Nedelsky, L. Absolute grading standards for objective tests. Educational and Psychological Measurement, 1954, 14. 3-19.

12.

Shepard, L.A. Setting standards and living with them. Florida Journal of Educational Research, 1976, 18, 28-32.

13.

Smilansky, J. , & Guerin, R.O. Minimal acceptable performance levels for criterion-referenced multiple-choice examinations and their validation. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, April 1976.

14.

Zieky, M.J. , & Livingston, S.A. Manual for setting standards on the Basic Skills Assessment Tests. Princeton NJ: Educational Testing Service, 1977.