Performance assessments have become popular in education and credentialing, and performance standards are common for interpreting and reporting scores. However, because of the unique characteristics of these assessments compared to multiple-choice tests (such as polytomous scoring), new and validstandard-setting methods are needed. Well-known standard-setting methods are no longer applicable. A number of promising methods for setting performance standards are described and their strengths and weaknesses are discussed. Suggestions for additional research are offered.
Get full access to this article
View all access options for this article.
References
1.
Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement(2nd ed.). Washington DC: American Council on Education.
2.
Berk, R. A. (1986). A consumer’s guide to setting performance standards on criterion-referenced tests. Review of Educational Research, 56, 137–172.
3.
Cizek, G. J. (Ed.). (in press). Standard setting: Concepts, methods, and perspectives. Mahwah NJ: Erlbaum.
4.
Cohen, A. S. , Kane, M. T., & Crooks, T. J. (1999). A generalized examinee-centered method for setting standards on achievement tests. Applied Measurement in Education, 12, 343–366.
5.
Ebel, R. L. (1972). Essentials of educational measurement. Englewood Cliffs NJ: Prentice-Hall.
6.
Giraud, G. , Impara, J. C., & Buckendahl, C. W. (in press). Making the cut in public schools: Alternative methods for standard setting. Educational Assessment.
7.
Hambleton, R. K. , Jaeger, R. M., Plake, B. S., & Mills, C. N. (in press). Handbook for setting performance standards. Washington DC: Council of the Chief State School Officers.
8.
Hambleton, R. K. ,& Plake, B. S. (1995). Using an extended Angoff procedure to set standards on complex performance assessments. Applied Measurement in Education, 8, 41–55.
9.
Hambleton, R. K. , & Slater, S. (1997). Reliability of credentialing examinations and the impact of scoring models and standard-setting policies. Applied Measurement in Education, 13, 19–38.
10.
Hambleton, R. K. , Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park CA: Sage.
11.
Impara, J. C. ,& Plake, B. S. (1997). Standard-setting: An alternative approach. Journal of Educational Measurement, 34, 353–366.
12.
Jaeger, R. M. (1989). Certification of student competence. In R. L. Linn(Ed.), Educational measurement(3rd ed.). New York: American Councilon Education and Macmillan.
13.
Jaeger, R. M. (1995). Setting performance standards through two-stage judgmental policy capturing. Applied Measurement in Education, 8, 15–40.
14.
Jaeger, R. M. , & Mills, C. N. (in press). An integrated judgment procedure for setting standards on complex large-scale assessments. In G. J. Cizek (Ed.), Standard-setting: Concepts, methods, and perspectives. Hillsdale NJ: Erlbaum.
15.
Kane, M. T. (1994). Validating the performance standards associated with passing scores. Review of Educational Research, 64, 425–461.
16.
Kingston, N. M. , Kahl, S. R., Sweeney, K. P., & Bay, L. (in press). Setting performance standards using the body of work method. In G. J. Cizek(Ed.), Standard setting: Concepts, methods, and perspectives. Mahwah NJ: Erlbaum.
17.
Linn, R. L. , & Shepard, L. A. (1997, June). Itemby-item standard setting: Misinterpretations of judges’ intentions due to less than perfect item intercorrelations. Paper presented at the Large Scale Assessment Conference, Colorado Springs CO.
18.
Livingston, S. A. , & Zieky, M. J. (1982). Passing scores: A manual for setting standards of performance on educational and occupational tests. Princeton NJ: Educational Testing Service.
19.
Loomis, S. C. , & Bourque, M. L. (in press). From tradition to innovation: Standard-setting on the National Assessment of Educational Progress. In G. J. Cizek (Ed.), Standard setting: Concepts, methods, and perspectives. Mahwah NJ: Erlbaum.
20.
Mitzel, H. D. , Lewis, D. M., Patz, R. J.,& Green, D. R. (in press). The bookmark procedure: Cognitive perspectives on standard-setting. In G. J. Cizek (Ed.), Standard-setting: Concepts, methods, and perspectives. Hillsdale NJ: Erlbaum.
21.
National Academy of Education . (1993). Setting performance standards for student achievement. Stanford CA: Author.
22.
Nedelsky, L. (1954). Absolute grading standards for objective tests. Educational and Psychological Measurement, 14, 3–19.
23.
Nellhaus, J. (2000). States with NAEP-like performance standards. Washington DC: National Assessment Governing Board.
24.
Parshall, C. G. , Davey, T., & Pashley, P. J. (2000). Innovative item types for computerized testing. In W. J. van der Linden & C. Glas (Eds.), Computerized adaptive testing: Theory and practice. Boston: Kluwer.
25.
Plake, B. S. (1998). Setting performance standards for professional licensure and certification. Applied Measurement in Education, 11, 65–80.
26.
Plake, B. S. , & Hambleton, R. K. (in press-a). A standard setting method designed for complex performance assessments: Categorical assignments of student work. Educational Assessment.
27.
Plake, B. S. , & Hambleton, R. K. (in press-b). The analytic judgment method for setting standards on complex performance assessments. In G. J. Cizek (Ed.), Standard-setting: Concepts, methods, and perspectives. Hillsdale NJ: Erlbaum.
28.
Plake, B. S. , Hambleton, R. K., & Jaeger, R. M. (1997). A new standard setting method for performance assessments: The dominant profile judgment method and some field test results. Educational and Psychological Measurement, 57, 400–411.
29.
Reckase, M. D. (2000). The evolution of the NAEP achievement levels setting process: A summary of the research and development efforts conductedby ACT. Iowa City IA: ACT.
