A critically important balance in educational measurement between practical concerns and matters of technique has atrophied in recent decades, and as a result, some important issues in the field have not been adequately addressed. I start with the work of E. F. Lindquist, who exemplified the balance that is now wanting. Lindquist was arguably the most prolific developer of achievement tests in the history of the field and an accomplished statistician, but he nonetheless focused extensively on the practical limitations of testing and their implications for test development, test use, and inference. I describe the withering of this balance and discuss two pressing issues that have not been adequately addressed as a result: the lack of robustness of performance standards and score inflation. I conclude by discussing steps toward reestablishing the needed balance.
AlmehriziR. S. (2022). Reconceptualization of coefficient alpha reliability for test summed and scaled scores. Educational Measurement: Issues and Practice, 41(3), 38–47.
2.
AlsarimiA. M. (2001). New trends in assessment in the sultanate of Oman: Goals and characteristics. Educational Measurement: Issues and Practice, 20(4), 27–30.
3.
AmedaheF. K. (2001). Combining teacher-assessment scores with external-examination scores for certification: The Ghanaian experience. Educational Measurement: Issues and Practice, 20(4), 29–30.
4.
BarlevyG.NealD. (2012). Pay for percentile. American Economic Review, 102(5), 1805–1831.
5.
BrennanR. L. (2001). Some problems, pitfalls, and paradoxes in educational measurement. Educational Measurement: Issues and Practice, 20(4), 6–18.
6.
CamilliG. (2003). Comment on Cizek’s “More unintended consequences of high-stakes testing.”Educational Measurement: Issues and Practice, 22(1), 40–44.
7.
CampbellD. T. (1976). Assessing the impact of planned social change. Dartmouth Public Affairs Center, Paper #8.
8.
CampbellD. T. (1979). Assessing the impact of planned social change. Evaluation and Program Planning, 2(1), 67–90.
9.
CannellJ. J. (1987). Nationally normed elementary achievement testing in America’s public schools: How all fifty states are above the national average. Friends for Education.
10.
CannellJ. J. (1989). How public educators cheat on standardized achievement tests. Friends for Education.
11.
CizekG. (2001). More unintended consequences of high-stakes testing. Educational Measurement: Issues and Practice, 20(4), 19–27.
12.
CizekG. (2003). Rejoinder. Educational Measurement: Issues and Practice, 22(1), 40–44.
13.
CuiZ. (2022). Introduction to the special section “Lingering impact of COVID-19 on educational measurement.”Educational Measurement: Issues and Practice, 41(1), 9–63.
14.
FiglioD.GetzlerL. (2002). Accountability, ability and disability: Gaming the system. National Bureau of Economic Research Working Paper 9307. http://www.nber.org/papers/w9307
15.
FullerB.GesickiK.KangE.WrightJ. (2006). Is the No Child Left Behind Act working? The reliability of how states track achievement (Working Paper No. 061). Policy Analysis for California Education.
16.
GlassG. V. (1978). Standards and criteria. Journal of Educational Measurement, 15(1), 237–261.
17.
GreenD. R.TrimbleC. S.LewisD. M. (2003). Interpreting the results of three different standard-setting procedures. Educational Measurement: Issues and Practice, 22(1), 22–32.
18.
HaertelE. H. (2002). Standard setting as a participatory process: Implications for validation of standards-based accountability programs. Educational Measurement: Issues and Practice, 21(1), 16–22.
19.
HaertelE. H. (2013). How is testing supposed to improve schooling?Measurement, 11(1–2), 1–18.
20.
HaertelE. H.LoriéW. A. (2004). Validating standards-based test score interpretations. Measurement, 2(2), 61–103.
21.
HaladynaT. M.NolanS. B.HaasN. S. (1991). Raising standardized achievement test scores and the origins of test pollution. Educational Researcher, 20(5), 2–7.
22.
HambletonR. K. (1978). On the use of cut-off scores with criterion-referenced tests in instructional settings. Journal of Educational Measurement, 15(1), 277–290.
23.
HambletonR. K.JaegerR. M.KoretzD.LinnR. L.MillmanJ.PhillipsS. E. (1995). Review of the Measurement Quality of the Kentucky Instructional Results Information System, 1991–1994. Office of Education Accountability, Kentucky General Assembly, June.
24.
HamiltonL. S.StecherB. M.MarshJ. A.McCombsJ. S.RobynA.RussellJ.NaftelS.BarneyH. (2007). Standards-based accountability under No Child Left Behind: Experiences of teachers and administrators in three states. RAND. Retrieved August 6, 2023, fromhttps://www.rand.org/content/dam/rand/pubs/monographs/2007/RAND_MG589.pdf
25.
HamiltonL. S.StecherB. M.YuanK. (2008). Standards-based reform in the United States: History, research, and future directions. RAND. Retrieved August 6, 2023, fromhttp://www.rand.org/pubs/reprints/2009/RAND_RP1384.pdf
HarrisD. (2021). Introduction to the focal article, commentaries, and response to the commentaries. Educational Measurement: Issues and Practice, 40(4), 21–81.
28.
HoA. D. (2007). Discrepancies between score trends from NAEP and state tests: A scale-invariant perspective. Educational Measurement: Issues and Practice, 26(4), 11–20.
29.
HoA. D. (2009). A nonparametric framework for comparing trends and gaps across tests. Journal of Educational and Behavioral Statistics, 34(2), 201–228.
30.
HoA. D.HaertelE. H. (2006). Metric-Free Measures of Test Score Trends and Gaps with Policy-Relevant Examples (CSE Report 665). National Center for Research on Evaluation, Standards, and Student Testing (CRESST), Center for the Study of Evaluation, University of California, Los Angeles.
31.
HolcombeR.JenningsJ.KoretzD. (2013). The roots of score inflation: An examination of opportunities in two states’ tests. In SundermanG. (Ed.), Charting reform, achieving equity in a diverse nation (pp. 163–189). Information Age. http://dash.harvard.edu/handle/1/10880587
32.
HooverH. D. (2003). Some common misconceptions about tests and testing. Educational Measurement: Issues and Practice, 22(1), 5–24.
33.
JacobB. A. (2005). Accountability, incentives and behavior: The impact of high-stakes testing in the Chicago public schools. Journal of Public Economics, 89(5–6), 761–796. https://doi.org/10.1016/j.jpubeco.2004.08.004
34.
JacobB. A. (2007). Test-based accountability and student achievement: An investigation of differential performance on NAEP and state assessments. National Bureau of Economic Research (Working Paper 12817).
35.
JaegerR. M. (1989). Certification of student competence. In LinnR. L. (Ed.), Educational measurement (3rd ed., pp. 485–514). American Council on Education & Macmillan.
36.
JenningsJ. L.BearakJ. M. (2014). “Teaching to the test” in the NCLB era: How test predictability affects our understanding of student performance. Educational Researcher, 43(8), 381–389.
37.
JenningsJ. L.BeveridgeA. A. (2009). How does test exemption affect schools’ and students’ academic performance?Educational Evaluation and Policy Analysis, 31, 153–175.
38.
KleinS. P.HamiltonL. S.McCaffreyD. F.StecherB. M. (2000). What do test scores in Texas tell us?RAND (Issue Paper IP-202). Retrieved June 4, 2013, fromhttp://www.rand.org/publications/IP/IP202/
39.
KoretzD. M. (1988). Arriving in Lake Wobegon: Are standardized tests exaggerating achievement and distorting instruction?American Educator, 12(2), 8–15, 46–52.
40.
KoretzD. M. (2008). Measuring up: What educational testing really tells us. Harvard University Press
41.
KoretzD. M. (2015). Adapting the practice of measurement to the demands of test-based accountability. Measurement: Interdisciplinary Research and Perspectives, 13(1), 1–25. https://doi.org/10.1080/15366367.2015.1000712
42.
KoretzD. M. (2017). The testing charade: Pretending to make schools better. University of Chicago Press.
43.
KoretzD. M.BarronS. I. (1998). The validity of gains on the Kentucky instructional results information system (KIRIS). RAND.
44.
KoretzD. M.BarronS.MitchellK.StecherB. (1996). The perceived effects of the Kentucky Instructional Results Information System (KIRIS). MR-792-PCT/FF. RAND.
45.
KoretzD. M.BeguinA. (2010). Self-monitoring assessments for educational accountability systems. Measurement: Interdisciplinary Research and Perspectives, 8(2-3: special issue), 92–109. https://dash.harvard.edu/handle/1/4889562
46.
KoretzD. M.HamiltonL. S. (2006). Testing for accountability in K-12. In BrennanR. L. (Ed.), Educational measurement (4th ed., 531–578). American Council on Education/Praeger.
KoretzD. M.LinnR. L.DunbarS. B.ShepardL. A. (1991, April). The effects of high-stakes testing: Preliminary evidence about generalization across tests [Conference Presentation]. American Educational Research Association 1991 conference. http://dash.harvard.edu/handle/1/10880553.
49.
KoretzD. M.McCaffreyD.HamiltonL. (2001). Toward a framework for validating gains under high-stakes conditions. CSE Technical Report 551. National Center for Research on Evaluation, Standards, and Student Testing.
50.
KoretzD.MitchellK.BarronS.KeithS. (1996). The perceived effects of the Maryland School Performance Assessment Program. CSE Technical Report No. 409. National Center for Research on Evaluation, Standards, and Student Testing.
51.
LewisD. M.MitzelH. C.MercadoR. L.SchulzE. M. (2012). The bookmark standard setting procedure. In CizekG. J. (Eds.), Setting performance standards: Foundations, methods, and innovations. Routledge
52.
LindquistE. F. (1951). Preliminary considerations in objective test construction. In LindquistE. F. (Ed.), Educational measurement (1st ed., pp.119–158). American Council on Education.
53.
LinnR. L. (1987). Accountability: The comparison of educational systems and the quality of test results. Educational Policy1(2), 181–198.
54.
LinnR. L. (2003). Performance standards: Utility for different uses of assessment. Education Policy Analysis Archives, 11(31), n.p.
55.
LinnR. L.GraueM. E.SandersN. M. (1990). Comparing state and district results to national norms: The validity of the claim that “everyone is above average.”Educational Measurement: Issues and Practice, 9(3), 5–14.
56.
LunaC.TurnerC. L. (2001). The impact of the MCAS: Teachers talk about high-stakes testing. The English Journal, 91(1), 79–87.
57.
MadausG. (1988a). The influence of testing on the curriculum. In TannerL. (Ed.), Critical issues in curriculum (the eighty-seventh yearbook of the National Society for the Study of Education, Part I, pp. 83–121). National Society for the Study of Education.
58.
MadausG. F. (1988b). The distortion of teaching and testing: High-stakes testing and instruction. Peabody Journal of Education, 65(3), 29–46.
59.
MorleyM. E.BridgemanB.LawlessR. R. (2004). Transfer between variants of quantitative items. Educational Testing Service (GRE Board Research Report No. 00-06 R).
60.
National Research Council (2005). Measuring literacy: Performance levels for adults. National Academies Press. https://doi.org/10.17226/11267
61.
NealD. (2011). The design of performance pay in education. Handbook of the Economics of Education, 4, 495–550. Elsevier.
NiemiR. G.SmithJ. (2003). The accuracy of students’ reports of course taking in the 1994 National Assessment of Educational Progress. Educational Measurement: Issues and Practice, 22(1), 15–21.
64.
PedullaJ. J.AbramsL. M.MadausG. F.RussellM. K.RamosM. A.MiaoJ. (2003). Perceived effects of state-mandated testing programs on teaching and learning: Findings from a national survey of teachers. National Board on Educational Testing and Public Policy. Retrieved July 15, 2013, fromhttp://www.bc.edu/research/nbetpp/statements/nbr2.pdf
65.
PophamW. J. (1978). As always, provocative. Journal of Educational Measurement, 15(4), 297–300. Special issue on Standard Setting (Winter, 1978), pp. 297–300.
66.
PophamW. J. (2003). Seeking redemption for our psychometric sins. Educational Measurement: Issues and Practice, 22(1), 45–48.
RichW. (2003). Historical high-stakes policies relating to unintended consequences of high-stakes testing. Educational Measurement: Issues and Practice, 20(4), 33–35.
69.
RodamarJ. (2018). There ought to be a law! Campbell versus Goodhart. Significance, 15(6), 9–9.
70.
RombergT. A.ZariniaE.A.WilliamsS.R. (1989). The influence of mandated testing on mathematics instruction: Grade 8 teachers’ perceptions. National Center for Research in Mathematical Science Education, University of Wisconsin.
71.
RubinsteinJ. (2000). Cracking the MCAS grade 10 math. Random House.
72.
ShepardL. A. (1988, April). The harm of measurement-driven instruction[Conference presentation]. American Educational Research Association 1988 Meeting, Washington, DC.
73.
ShepardL. A.DoughertyK. C. (1991, April). Effects of high-stakes testing on instruction[conference presentation]. Annual meeting of the American Educational Research Association and National Council on Measurement in Education, Chicago.
74.
ShepardL. A.GlaserR.LinnR.BohrnstedtG. (1993). A report of the National Academy of Education Panel on the evaluation of the NAEP Trial State Assessment: An evaluation of the 1992 Achievement Levels. National Academy of Education.
75.
SmithJ. L. (1991). Meanings of test preparation. American Educational Research Journal, 28(3), 521–542.
76.
SmithM. L.RottenbergC. (1991). Unintended consequences of external testing in elementary schools. Educational Measurement: Issues and Practice, 10(4), 7–11.
77.
StecherB. M. (2002). Consequences of large-scale, high-stakes testing on school and classroom practice. In HamiltonL., et al., Test-based Accountability: A Guide for Practitioners and Policymakers. RAND. http://www.rand.org/publications/MR/MR1554/MR1554.ch4.pdf
78.
StecherB. M.BarronS. I. (2001). Unintended consequences of test-based accountability when testing in “milepost” grades. Educational Assessment, 7(4), 259–281.
79.
StudentS. R.GongB. (2022). Supporting the interpretive validity of student-level claims in science assessment with tiered claim structures. Educational Measurement: Issues and Practice, 41(4), 68–78.
80.
TaylorE. H. (2023). Teacher evaluation and training. In HanushekE. A.MachinS.WoessmannL. (Eds.). Handbook of the Economics of Education, Volume 7. Elsevier.
