The Quality of Information From NAEP: Two Examples of Work Done in Collaboration With Leigh Burstein

Abstract

This article summarizes two research efforts, both focusing on the mathematics assessments of the National Assessment of Educational Progress, that illustrate Leigh Burstein’s long-standing concern with the quality of information about the condition of education.1 The first examined nonresponse to NAEP test items; it found that omit rates were highest for difficult constructed-response items and that African American and Hispanic students had higher omit rates than Whites. The second study evaluated the validity of the 1992 achievement level descriptions as characterizations of mathematics performance; it found that the descriptions and accompanying exemplar items were misleading. In response to these findings, a variety of recommendations were offered pertaining to test construction, standards setting, routine monitoring (and reporting) of data quality, and standards-based reporting of student performance.