Progress in the field of standard setting during the 1980s is reviewed under the headings of conceptual and theoretical developments, guides to application, and comparative and applied research studies. Two new indices of stability of contentreferenced standard-setting results are presented, relating variability ofjudges'standard-setting decisions to the variability of candidate scores and to the reliability of the test. These indices are used to indicate whether the passing scores resultingfrom a standard-setting study are ofsufficientprecision. Recommendations based on the current research literature are made. Several neglected areas for research are identified.
Get full access to this article
View all access options for this article.
References
1.
American Educational Research Association
, American Psychological Association, National Council on Measurement in Education (1985) Standards for Education and Psychological Testing. Washington, DC: American Psychological Association.
2.
ANDREW, B. J.
and J. T. HECHT (1976) "A preliminary investigation of two procedures for setting examination standards."Educ. and Psych. Measurement36: 45-50.
3.
ANGOFF, W.
(1971) "Scales, norms and equivalent scores," in R. L. Thorndike (ed.) Educational Measurement (2nd ed.). Washington, DC: American Council on Education.
4.
BRENNAN, R. L.
and R. E. LOCKWOOD (1979) A Comparison of Two Cutting Score Procedures (ACT technical bulletin 33). Iowa City, IA: ACT Publications.
5.
CONAWAY, L. E.
(1979) "Setting standards in competency-based education: some current practices and concerns," in M. A. Bunda and J. R. Sanders (eds.) Practices and Problems in Competency-Based Measurement. Washington, DC: National Council on Measurement in Education.
6.
CRONBACH, L. J.
, G. C. GLESER, H. NANDA, and N. RAJARATNAM (1972) The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. New York: John Wiley.
7.
GROSS, L. J.
(1985) "Setting cutoff scores on credentialing examinations: a refinement in the Nedelsky procedure."Evaluation & the Health Professions8: 469-493.
8.
GULLIKSEN, H.
(1950) Theory of Mental Tests. New York: John Wiley.
9.
HAMBLETON, R. K.
(1983, April) "Standard-setting: state of the art, future prospects." Presented at the meeting of the American Educational Research Association, Montreal, Canada.
10.
HAMBLETON, R. K.
and S. POWELL (1981, April) "Standards for standard setters." Presented at the annual meeting of the American Educational Research Association, Los Angeles, April.
11.
HARASYM, P. H.
(1981) "A comparison of the Nedelsky and modified Angoff standardsetting procedure on evaluation outcome."Educ. and Psych. Measurement41: 725-734.
12.
JAEGER, R. M.
(1979) "Measurement consequences of standard-setting models," in M. A. Bunda and J. R. Sanders (eds.) Practices and Problems in Competency-Based Measurement. Washington, DC: National Council on Measurement in Education.
13.
KANE, M.
and J. WILSON (1984) "Errors of measurement and standard setting in mastery testing."Applied Psych. Measurement8: 107-115.
14.
LIVINGSTON, S. A.
(1983, April) "Standard-setting: state of the art, future prospects." Presented at the American Educational Research Association, Montreal, Canada.
15.
LIVINGSTON, S. A.
and M. J. ZIEKY (1982) Passing Scores: A Manual for Setting Standards of Performance on Educational and Occupational Tests. Princeton, NJ: Educational Testing Service.
16.
MESKAUSKAS, J. A.
(1983, April) "Standard-setting: state of the art, future prospects." Presented at the American Educational Research Association, Montreal, Canada.
17.
MESKAUSKAS, J. A.
and J. J. NORCINI (1980) "Standard-setting in written and interactive (oral) specialty certification examinations: issues, models, methods, challenges."Evaluation and the Health Professions3: 321-360.
18.
MESKAUSKAS, J. A.
and G. W. WEBSTER (1975) "The American board of internal medicine recertification examination process and results."Annals of Internal Medicine82: 577-581.
19.
NEDELSKY, L.
(1954) "Absolute grading standards for objective tests."Educ. and Psych. Measurement14: 3-19.
20.
NORCINI, J. J.
, E. W. HANCOCK, G. D. WEBSTER, L. J. GROSSO, and J. A. SHEA (1986) A Criterion-referenced Performance Examination of Physician Competence. Philadelphia: American Board of Internal Medicine.
21.
POPHAM, W. J.
(1983, April) "Standard-setting: state of the art, future prospects." Presented at the American Educational Research Association, Montreal, Canada.
22.
REILLY, R. R.
, D. L. ZINK, and E. W. ISRAELSKI (1984) "Comparison of direct and indirect methods for setting minimum passing scores."Applied Psych. Measurement8, 4: 421-429.
23.
SHEPARD, L. A.
(1979) "Setting standards," in M. A. Bunda and J. R. Sanders (eds.) Practices and Problems in Competency-Based Measurement. Washington, DC: National Council on Measurement in Education.
24.
SKAKUN, E. N.
and S. KLING (1980) "Comparability of methods for setting standards."J. of Educ. Measurement17: 229-235.
25.
TRAUB, R. C.
(1983, April) "Standard-setting: state of the art, future prospects." Presented at the American Educational Research Association, Montreal, Canada.
26.
van der LINDEN, W. J.
(1982) "A latent trait method for determining intrajudge inconsistency in the Angoff and Nedelsky techniques of standard setting."J. of Educ. Measurement19: 295-308.
27.
WRIGHT, B. D.
and M. H. STONE (1979) Best Test Design: Rasch Measurement. Chicago: MESA Press.
