Standardized tests are regularly used as education system monitoring tools to compare the average performance of students living in different states or belonging to different subgroups (e.g., defined by race/ethnicity, sex, or parental income) and to track their progress over time. This article describes some uses and design features of tests in system monitoring contexts. We provide the example of the National Assessment of Educational Progress (NAEP), the only large-scale system monitoring test in the United States. The availability of NAEP data, in turn, has facilitated the construction of the Stanford Education Data Archive (SEDA), a publicly available database that can be used to describe patterns of achievement for nearly all school districts in the United States. Here, we discuss progress in and challenges to the use of standardized tests as system monitoring tools.
American Educational Research Association (AERA), American Psychological Association (APA), and National Council on Measurement in Education (NCME). 2014. Standards for educational and psychological testing. Washington, DC: AERA.
2.
Bandeira de MelloVictorRahmanT.ParkB. J.2018. Mapping state proficiency standards onto the NAEP scales: Results from the 2015 NAEP reading and mathematics assessments. Technical Report NCES 2018-159, Institute of Education Sciences, National Center for Education Statistics, U.S. Department of Education, Washington, DC.
3.
BeatonAlbert E.AllenNancy L.1992. Interpreting scales through scale anchoring. Journal of Educational Statistics17 (2): 191–204.
4.
BeatonAlbert E.RogersAlfred M.GonzalezEugenioHanlyMary BethKolstadAndrewRustKeith F.SikaliEmmanuelStokesLynneJiaYue. 2011. The NAEP Primer (NCES 2011-463). Washington, DC: U.S. Department of Education, National Center for Education Statistics.
5.
BeatonAlbert E.ZwickRebecca. 1990. The effect of changes in the national assessment: Disentangling the NAEP 1985–86 reading anomaly. Revised. Princeton, NJ: National Assessment of Educational Progress.
6.
BennettRandy E.2010. Cognitively based assessment of, for, and as learning (CBAL): A preliminary theory of action for summative and formative assessment. Measurement: Interdisciplinary Research and Perspectives8 (2–3): 70–91.
7.
BourqueMary Lyn. 2009. A History of NAEP achievement levels: Issues, implementation, and impact 1989–2009. Washington, DC: National Assessment Governing Board.
8.
BraunHenry I.QianJiahe. 2007. An enhanced method for mapping state standards onto the NAEP scale. In Linking and aligning scores and scales, Editors of this volume: DoransNeil J.PommerichMaryHollandPaul W. 313–38. New York, NY: Springer.
9.
BraunHenry I.SingerJudith D.2019. Assessment for monitoring of education systems: International comparisons. The ANNALS of the American Academy of Political and Social Science (this volume).
10.
BuckendahlChad W.DavisSusan L.PlakeBarbara S.SireciStephen G.HambletonRonald K.ZeniskyApril L.WellsCraig S.2009. Evaluation of the National Assessment of Educational Progress: Final report. Washington, DC: U.S. Department of Education, Office of Planning, Evaluation and Policy Development.
11.
BuiQuoctrungDoughertyConor. 2017. Good schools, affordable homes: Finding suburban sweet spots. New York Times, March30.
12.
BursteinLeigh. 1984. The use of existing data bases in program evaluation and school improvement. Educational Evaluation and Policy Analysis6 (3): 307–18.
13.
CampbellDonald T.1979. Assessing the impact of planned social change. Evaluation and Program Planning2 (1): 67–90.
14.
CarnoyMartinLoebSusanna. 2002. Does external accountability affect student outcomes? A cross-state analysis. Educational Evaluation and Policy Analysis24 (4): 305–31.
15.
ColemanJames S.CampbellErnest Q.HobsonCarol J.McPartlandJameMoodAlexander M.WeinfeldFrederic D.YorkRobert L.1966. Equality of educational opportunity. Report OE-38001. Washington, DC: National Center for Educational Statistics.
16.
CronbachLee J.1988. Five perspectives on validation argument. In Test validity, eds. WainerH.BraunH., 3–17. Hillsdale, NJ: Lawrence Erlbaum Associates.
17.
DeeThomas S.JacobBrian. 2011. The impact of No Child Left Behind on student achievement. Journal of Policy Analysis and Management30 (3): 418–46.
18.
DemingDavid J.2017. The growing importance of social skills in the labor market. Quarterly Journal of Economics132 (4): 1593–1640.
19.
EDFacts. 2015. State assessments in reading/language arts and mathematics: School year 2014–2015 EDFacts Data Documentation. Washington, DC: U.S. Department of Education.
20.
FabrizioLouisFriedmanAlanGarrisonShannon. 2013. General policy: Conducting and reporting the National Assessment of Educational Progress. Washington, DC: National Assessment Governing Board.
MichaelFeuerHollandPaul W.GreenBert F.BertenthalMeryl W.HemphillF. Cadell, eds. 1999. Uncommon measures: Equivalence and linkage among educational tests. Washington, DC: National Academies Press.
23.
HaertelEdward. 1989. Student achievement tests as tools of educational policy: Practices and consequences. In Test policy and test performance: Education, language, and culture, ed. GiffordBernard, 25–50. New York, NY: Springer.
24.
HaertelEdward. 2013. How is testing supposed to improve schooling?Measurement: Interdisciplinary Research and Perspectives 11 (1–2): 1–18.
HaertelEdwardBeauregardRussellConfreyJereGomezLouisGongBrianHoAndrewHorwitzPaulJunkerBrianPeaRoyRothmanRobertShepardLorrie. 2012. NAEP: Looking ahead. Leading assessment into the future. Washington, DC: National Center for Education Statistics, Initiative on the Future of NAEP.
27.
HambletonRonald K.SireciStephen G.SmithZachary R.2009. How do other countries measure up to the mathematics achievement levels on the national assessment of educational progress?Applied Measurement in Education22 (4): 376–93.
HanushekEric. 2019. Testing, accountability, and the American economy. The ANNALS of the American Academy of Political and Social Science (this volume).
30.
HeckmanJames J.RubinsteinYona. 2001. The importance of noncognitive skills: Lessons from the GED testing program. American Economic Review91 (2): 145–49.
31.
HedgesLarry V.de MelloVictor Bandeira. 2013. A validity study of the NAEP full population estimates. Washington, DC: American Institutes for Research.
32.
HedgesLarry V.VeveaJack L.1997. A study of equating in NAEP. NAEP Validity Studies. National Center for Education Statistics Working Paper, Washington, DC.
33.
JacksonC. Kirabo. 2018. What do test scores miss? The importance of teacher effects on non-test score outcomes. Journal of Political Economy126 (5): 2072–2107.
34.
JacobBrian A.2007. Test-based accountability and student achievement: An investigation of differential performance on NAEP and state assessments. Washington, DC: National Bureau of Economic Research.
35.
JaegerRichard M.1998. Reporting the results of the National Assessment of Educational Progress. Washington, DC: American Institutes for Research.
36.
KaneMichael T.2013. Validating the interpretations and uses of test scores. Journal of Educational Measurement50 (1): 1–73.
37.
KolenMichael J.BrennanRobert L.2010. Test equating, scaling, and linking. 2nd ed.New York, NY: Springer.
38.
KoretzDaniel. 1988. Arriving in Lake Woebegon: Are standardized tests exaggerating achievement and distorting instruction?American Educator: The Professional Journal of the American Federation of Teachers12 (2): 8.
39.
KoretzDaniel. 2017. The testing charade: Pretending to make schools better. Chicago, IL: University of Chicago Press.
40.
LafortuneJulienRothsteinJesseSchanzenbachDiane Whitmore. 2018. School finance reform and the distribution of student achievement. American Economic Journal: Applied Economics10 (2): 1–26.
41.
LareauAnnetteGoyetteKimberly. 2014. Choosing homes, choosing schools. New York, NY: Russell Sage Foundation.
42.
LimHwanggyuSireciStephen G.2017. Linking TIMSS and NAEP assessments to evaluate international trends in achievement. Education Policy Analysis Archives25 (11). Available from http://dx.doi.org/10.14507/epaa.25.2682.
LoebSusannaByunErika Nakamatsu. 2019. Testing, accountability, and school improvement. The ANNALS of the American Academy of Political and Social Science (this volume).
45.
MazzeoJohnLazerStephenZiekyMichael J.2006. Monitoring educational progress with group-score assessments. In Educational measurement, ed. BrennanRobert L., 681–99. Westport, CT: American Council on Education and Praeger.
46.
McFarlandHussarBillde BreyCristobalSnyderTomWangXiaoleiWilkinson-FlickerSidneyGebrekristosSemharZhangJijunRathbunAmyBarmerAmyBullock MannFarrahHinzSerena. 2017. The condition of education 2017 (NCES 2017- 144). Washington, DC: U.S. Department of Education, National Center for Education Statistics.
47.
McLaughlinDonGallagherLarryStancavageFran. 2004. Evaluation of bias correction methods for “worst-case” selective non-participation in NAEP. Washington, DC: American Institutes for Research.
48.
MessickSamuel. 1995. Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist50 (9): 741–49.
49.
MislevyRobert J.BeatonAlbert E.KaplanBruceSheehanKathleen M.1992. Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement29 (2): 133–61.
50.
MurnaneRichard J.WillettJohn B.DuhaldebordeYvesTylerJohn H.2000. How important are the cognitive skills of teenagers in predicting subsequent earnings?Journal of Policy Analysis and Management19 (4): 547–68.
51.
National Academies of Sciences, Engineering, and Medicine. 2017. Evaluation of the achievement levels for mathematics and reading on the National Assessment of Educational Progress. Washington, DC: National Academies Press.
52.
National Center for Education Statistics. 2013. U.S. states in a global context: Results from the 2011 NAEP-TIMSS Linking Study (NCES 2013460). Washington, DC: National Center for Education Statistics.
53.
National Commission on Excellence in Education. 1983. A nation at risk: The imperative for educational reform. Washington, DC: Superintendent of Documents, Government Printing.
54.
NellhausJeffreyBehuniakPeterStancavageFrances B. 2009. Guiding principles and suggested studies for determining when the introduction of a new assessment framework necessitates a break in trend in NAEP. Washington, DC: American Institutes for Research.
55.
OTA (Office of Technology Assessment). 1992. Testing in American schools: Asking the right questions. Washington, DC: U.S. Government Printing Office.
56.
PhillipsGary W.2007. Expressing international educational achievement in terms of U.S. performance standards. Washington, DC: American Institutes for Research.
57.
ReardonSean F.2018. Educational opportunity in early and middle childhood: Variation by place and age. CEPA Working Paper No. 17-12, Stanford Center for Education Policy Analysis, Stanford, CA.
58.
ReardonSean F.HoAndrew D.ShearBenjamin R.FahleErin M.KalogridesDemetraDiSalvoRichard. 2018. Stanford Education Data Archive (Version 2.1). Available from http://purl.stanford.edu/db586ns4974.
59.
ReardonSean F.KalogridesDemetraHoAndrew D.2018. Validation methods for aggregate-level test scale linking: A case study mapping school district test score distributions to a common scale. CEPA Working Paper No. 16-09, Stanford Center for Education Policy Analysis, Stanford, CA.
60.
ReardonSean F.ShearBenjamin R.CastellanoKatherine E.HoAndrew D.2017. Using heteroskedastic ordered probit models to recover moments of continuous test score distributions from coarsened data. Journal of Educational and Behavioral Statistics42 (1): 3–45.
61.
RiddleWayne C.2008. Educational testing: Implementation of ESEA Title IA requirements under the No Child Left Behind Act (RL31407). Washington, DC: Congressional Research Service.
62.
ShepardLorrie A.1993. Evaluating test validity. Review of Research in Education19 (1): 405–50.
63.
SnyderThomas D.de BreyCristobalDillowSarah A.2018. Digest of education statistics 2016 (NCES 2017-094). Washington, DC: U.S. Department of Education, National Center for Education Statistics, Institute of Education Sciences.
64.
SorensenLucy C.FoxAshley M.JungHeyjieMartinErika G.2018. Lead exposure and academic achievement: Evidence from childhood lead poisoning prevention efforts. Journal of Population Economics32 (1): 179–218.
65.
StedmanLawrence C.2009. The NAEP long-term trend assessment: A review of its transformation, use, and findings. Washington, DC: National Assessment Governing Board.
66.
TuckerMarc. 2018. Setting NAEP performance standards: How we got here. Education Week, October4. Available from http://blogs.edweek.org.
67.
VinovskisMaris A.2001. Overseeing the nation’s report card: The Creation and Evolution of the National Assessment Governing Board (NAGB). Washington, DC: NAGB.
68.
VinovskisMaris A.2019. History of testing in the United States: PK–12 education. The ANNALS of the American Academy of Political and Social Science (this volume).
69.
ZwickRebeccaErcikanKadriye. 1989. Analysis of differential item functioning in the NAEP history assessment. Journal of Educational Measurement26 (1): 55–66.
