Performance standards are a powerful way to communicate K–12 student achievement (e.g., proficiency) and are the cornerstone of standards-based reform. As education reform shifts the focus to college and career readiness, approaches for setting performance standards need to be revised. We argue that the focus on assessing student readiness can move performance standards toward an increasingly empirical grounding, leading to more meaningful and understandable standards for student achievement and better guideposts for instructional improvement. Specifically, we describe and illustrate the processes and practices associated with evidence-based standard setting, taking as our example a specific application in the American Diploma Project Algebra II End-of-Course Exam, which was developed as an indicator of college readiness.
ACT, (2006). Ready for college and ready for work: Same or different? Retrieved from http://www.act.org
2.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
3.
American Recovery and Reinvestment Act (ARRA) of 2009 (2009, Feb. 19). Pub. L. No. 111-5, 123 Stat. 115, 516.
4.
Bandeira de MelloV. (2011). Mapping state proficiency standards onto the NAEP scale: Variation and change in state standards for reading and mathematics, 2005–2009. Retrieved from http://nces.ed.gov/nationsreportcard/pdf/studies/2011458.pdf
BraunH.QianJ. (2007). An enhanced method for mapping state standards onto the NAEP scale. In DoransN. J.PommerichM.HollandP. W. (Eds.), Linking and aligning scores and scales (pp. 313–338). New York, NY: Springer.
8.
BrownW. (2000). Reporting NAEP by achievement levels: An analysis of policy and external reviews. In BourqueM. L.ByrdS. (Eds.), Student performance standards on the National Assessment of Educational Progress: Affirmations and improvements (pp. 11–39). Washington, DC: National Assessment Governing Board.
9.
CareyK. (2006). Hot air: How states inflate their educational progress under NCLB. Retrieved from http://www.educationsector.org
10.
CizekG. J. (1993). Reconsidering standards and criteria. Journal of Educational Measurement, 30(2), 93–106. doi:10.doi:10.1111/j.1745-3984.1993.tb01068.x
11.
CizekG. J. (2001). Setting performance standards: Concepts, methods, and perspectives. Mahwah, NJ: Lawrence Erlbaum.
12.
CizekG. J. (2012a). An introduction to contemporary standard setting: Concepts, characteristics, and contexts. In CizekG. J. (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 3–14). New York, NY: Routledge.
13.
CizekG. J. (2012b). Setting performance standards: Foundations, methods, and innovations (2nd ed.). New York, NY: Routledge.
14.
ConleyD. T. (2010). College and career ready. San Francisco, CA: Jossey-Bass.
15.
Council of Chief State School Officers & National Governors Association, Center for Best Practices. (2010a). Common core state standards for English language arts and literacy in history/social studies, science, and technical subjects. Retrieved from http://www.corestandards.org
16.
Council of Chief State School Officers & National Governors Association, Center for Best Practices. (2010b). Common core state standards for mathematics. Retrieved from http://www.corestandards.org
FinnC. E. (2008). Troublemaker: A personal history of school reform since Sputnik. Princeton, NJ: Princeton University Press.
19.
GlassG. V. (1978). Standards and criteria. Journal of Educational Measurement, 15, 237–261. doi:10.1111/j.1745-3984.1978.tb00072.x
20.
HaertelE. H. (2002). Standard setting as a participatory process: Implications for validation of standards-based accountability programs. Educational Measurement: Issues and Practice, 21(1), 16–22. doi:10.1111/j.1745-3992.2002.tb00081.x
21.
HaertelE. H.BeimersJ. N.MilesJ. A. (2012). The briefing book method. In CizekG. J. (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 283–299). New York, NY: Routledge.
22.
HaertelE. H.LoriéW. A. (2004). Validating standards-based test score interpretations. Measurement: Interdisciplinary Research and Perspectives, 2(2), 61–103.
23.
HambletonR. K. (1978). On the use of cut-off scores with criterion-referenced tests in instructional settings. Journal of Educational Measurement, 15, 277–290. doi:10.1111/j.1745-3984.1978.tb00075.x
24.
HambletonR. K. (1980). Test score validity and standard-setting methods. In BerkR. A. (Ed.), Criterion-referenced measurement: The state of the art (pp. 80–123). Baltimore, MD: Johns Hopkins University Press.
25.
HambletonR. K.BrennanR. L.BrownW.DoddB.ForsythR. A.MehrensW. A.. . . ZwickR. (2000). A response to “Setting Reasonable and Useful Performance Standards” in the National Academy of Sciences’Grading the Nation’s Report Card. Educational Measurement: Issues and Practice, 19(2), 5–14. doi:10.1111/j.1745-3992.2000.tb00024.x
26.
HambletonR. K.PitoniakM. J. (2006). Setting performance standards. In BrennanR. L. (Ed.), Educational measurement (4th ed., pp. 433–470). Westport, CT: American Council on Education/Praeger.
27.
HambletonR. K.PitoniakM. J.CopellaJ. M. (2012). Essential steps in setting performance standards on educational tests and strategies for assessing the reliability of results. In CizekG. J. (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 47–76). New York, NY: Routledge.
28.
JaegerR. M. (1989). Certification of student competence. In LinnR. (Ed.), Educational measurement (3rd ed., pp. 485–514). Washington, DC: American Council on Education.
29.
KaneM. (2001). So much remains the same: Conception and status of validation in setting standards. In CizekG. J. (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 53–88). Mahwah, NJ: Lawrence Erlbaum.
30.
KaneM. (2006). Validation. In LinnR. L. (Ed.), Educational measurement (4th ed., pp. 17–64). New York, NY: American Council on Education/Macmillan.
31.
LeeJ. (2012). College for all: Gaps between desirable and actual P–12 math achievement trajectories for college readiness. Educational Researcher, 41(2), 43–55. doi:10.3102/0013189X11432746.
32.
LinnR. L. (2001, October). The influence of external evaluations on the National Assessment of Educational Progress (CSE Tech. Rep. No. 548). Los Angeles, CA: Center for the Study of Evaluation/CRESST.
33.
LinnR. L.KoretzD. M.BakerE. L.BursteinL. (1991). The validity and credibility of the achievement levels for the 1990 National Assessment of Educational Progress in mathematics (CSE Tech. Rep. No. 330). Los Angeles, CA: Center for the Study of Evaluation/CRESST.
34.
LivingstonS.ZiekyM. (1982). Passing scores: A manual for setting standards of performance on educational and occupational tests. Princeton, NJ: Educational Testing Service.
35.
LoomisS. C. (2012, April). A study of “irrelevant” items: Impact on bookmark placement and implications for college and career readiness. Paper presented at the annual meeting of the National Council on Measurement in Education, Vancouver, British Columbia, Canada.
36.
McClartyK. L.DavisL. L. (2012, April). Enriched by policy: Making performance standards meaningful for educational outcomes. Paper presented at the annual meeting of the National Council on Measurement in Education, Vancouver, British Columbia, Canada.
37.
Mitzel (2005). Consistency for state achievement standards under NCLB. Washington, DC: Council of Chief State School Officers.
No Child Left Behind (NCLB) Act of 2001 (2002). Pub. L. No. 107-110, § 115, Stat. 1425.
42.
O’MalleyK.KengL.MilesJ. (2012). From Z to A: Using validity evidence to set performance standards. In CizekG. J. (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 301–322). New York, NY: Routledge.
43.
Partnership for Assessment of Readiness for College and Careers. (2012). PARCC college-ready determination policy in English and mathematics and policy and general content claims for PARCC performance levels. Retrieved from http://www.parcconline.org/
PellegrinoJ. W.JonesL. R.MitchellK. J. (Eds.) (1999). Grading the nation’s report card: Evaluating NAEP and transforming the assessment of educational progress. Washington, DC: National Academy Press.
46.
PhillipsG. W. (2010). International benchmarking: State education performance standards. Washington, DC: American Institutes for Research.
47.
PhillipsG. W. (2012). The benchmark method of standard setting. In CizekG. J. (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 323–346). New York, NY: Routledge.
48.
PlakeB. S. (2008). Standard setters: Stand up and take a stand! Educational Measurement: Issues and Practices, 27(1), 3–9. doi:10.1111/j.1745-3992.2008.00110.x
49.
PophamW. J. (1978). As always, provocative. Journal of Educational Measurement, 15, 297–300. doi:10.1111/j.1745-3984.1978.tb00077.x
50.
SackettD. L.RosenbergW. M. C.Muir GrayJ. A.HaynesR. B.RichardsonW. S. (1996). Evidence based medicine: What it is and what it isn’t. British Journal of Medicine, 312, 71. doi:10.1136/bmj.312.7023.71
ShepardL. A. (1979). Setting standards. In BundaM. A.SandersJ. R. (Eds.), Practices and problems in competency-based measurement (pp. 72–88). Washington, DC: National Council on Measurement in Education.
TwingJ. S.McClartyK. L.O’MalleyK. J.WayW. D. (2010, April). Texas comprehensive approach to college readiness. Paper presented at the annual meeting of the American Educational Research Association, Denver, CO.
55.
U. S. Department of Education. (2010). Overview information: Race to the Top Fund Assessment Program. Notice inviting applications for new awards for fiscal year (FY) 2010. Federal Register, 75(68), 18171–18185.
56.
ZiekyM. J. (2001). So much has changed: How the setting of cutscores has evolved since the 1980s. In CizekG. J. (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 19–51). Mahwah, NJ: Lawrence Erlbaum.
