Pass/fail decisions for all two-method combinations of 10 standard-setting procedures were analyzed. The most reliable or equivalent methods were the practitioners—borderline group, Ebel—contrasting groups, and 33rd percentile—Angoff approaches. The least reliable or equivalent were the chance/ideal mean—masters group and Nedelsky—masters group approaches. The most valid of the 10 methods, as evidenced by correlations with an external criterion, were the practitioner and borderline group approaches while the least valid were the nonmasters, Nedelsky, and chance/ideal mean methods. Results of a more detailed study of the reliability and validity of the item-judgment methods are also reported.
Get full access to this article
View all access options for this article.
References
1.
Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed.). Washington, DC: American Council on Education.
2.
Berk, R. A. (1976). Determination of optimal cutting scores in criterion-referenced measurement. Journal of Experimental Education, 45, 4-9.
3.
Callis, R. and Johnson, W. (1965). Missouri College English Test. New York: Harcourt, Brace, and World.
4.
Coffman, W. E. (1971). Essay examinations. In R. L. Thorndike (Ed.), Educational measurement (2nd ed.). Washington, DC: American Council on Education.
5.
Divgi, D. R. (1982, March). The logic of standard-setting: Some issues and questions. Paper presented at the meeting of the National Council on Measurement in Education, New York.
6.
Ebel, R. L. (1979). Essentials of educational measurement (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall.
7.
Halpin, G. , Sigmon, G., and Halpin, G. (1983). Minimum competency standards set by three divergent groups of raters using three judgmental procedures: Implications for validity. Educational and Psychological Measurement, 43, 185-195.
8.
Jaeger, R. M. (1982, March). High school competency test standards and the definition of competence. Paper presented at the meeting of the National Council on Measurement in Education, New York.
9.
Koffler, S. L. (1980). A comparison of approaches for setting proficiency standards. Journal of Educational Measurement, 17, 167-178.
10.
Livingston, S. A. and Zieky, M. J. (1982). Passing scores: A manual for setting standards of performance on educational and occupational tests. Princeton, NJ: Educational Testing Service.
11.
McLean, J. E. and Halpin, G. (1984, April). Setting passing scores for the Alabama High School Graduation Examination. Paper presented at the meeting of the American Educational Research Association, New Orleans.
12.
Mills, C. N. (1983). A comparison of three methods of establishing cut-off scores on criterion-referenced tests. Journal of Educational Measurement, 20, 283-292.
13.
Nedelsky, L. (1954). Absolute grading standards for objective tests. Educational and Psychological Measurement, 14, 3-19.
14.
Poggio, J. P. , Glasnapp, D. R., and Eros, D. S. (1981, April). An empirical investigation of the Agnoff, Ebel, and Nedelsky standard setting methods. Paper presented at the meeting of the American Educational Research Association, Los Angeles.
15.
Skakun, E. N. and Kling, S. (1980). Comparability of methods for setting standards. Journal of Educational Measurement, 17, 229-235.
