In this article, the authors argue that the cultures of accountability and validity are at odds. Calls for accountability arise from the polis, or political community. Validity is the standard of quality that professionals place on tests. When the polis demands that tests serve high-stakes accountability functions, professional testing standards often are compromised. In accountability contexts, test results decide which students are retained in grade, held back from graduation, and assigned to tracks or special classifications. Yet, empirical evidence suggests that the use of flawed indicators produces unreliable and unrepresentative inferences and decisions. High-stakes testing produces teaching and testing practices that lead to inflated test scores and further disadvantage already disadvantaged students.
Get full access to this article
View all access options for this article.
References
1.
Allington, R. L. , & McGill-Franzen, A. (1992). Unintended effects of educational reform in New York. Educational Policy, 4, 397-414.
2.
American Educational Research Association, American Psychological Association and National Council on Measurement in Education . (1985). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
3.
Berk, R. A. (1988). Fifty reasons why student achievement gain does not mean teacher effectiveness. Journal of Personnel Evaluation in Education, 1, 345-363.
4.
Campbell, D. (1979). Assessing the impact of planned social change. Evaluation and Program Planning, 2, 67-90.
5.
Cook, T. , & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand-McNally.
6.
Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 443-507). Washington, DC: American Council on Education.
7.
Edelman, M. (1985). The symbolic uses of politics. Urbana: University of Illinois Press.
8.
Guskey, T. R. , & Kifer, E. W. (1989, April). Ranking school districts on the basis of statewide test results: Is it meaningful or misleading? Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco.
9.
Haladyna, T. M. , Nolen, S. B., & Haas, N. S. (1991). Raising standardized test scores and the origins of test score pollution. Educational Researcher, 20(5), 2-7.
10.
Heubert, J. P. , & Hauser, R. M. (1998). High stakes: Testing for tracking, promotion, and graduation. Washington, DC: National Academy Press.
11.
House, E. R. (1993). Professional evaluation. Newbury Park, CA: Sage.
12.
House, E. R. (1998). Schools for sale. New York: Teachers College Press.
13.
Joint Committee on Testing Practices . (1988). Code of fair testing practices. Washington, DC: National Council on Measurement in Education.
14.
Koretz, D. , Linn, R. L., Dunbar, S. B., & Shepard, L. A. (1991, April). The effects of high-stakes testing on achievement: Preliminary findings about generalizations across tests. Paper presented at the American Educational Research Association, Chicago.
15.
Kulik, J. A. , Kulik, C. C., & Bangert-Drowns, R. L. (1984). Effects of practice on aptitude and achievement test scores. American Educational Research Journal, 20(2), 435-447.
16.
Linn, R. L. (2000). Assessments and accountability. Education Researcher, 29(2), 4-16.
17.
Madaus, G. , & Clarke, M. (1998). The adverse impact of high stakes testing on minority students: Evidence from 100 years of test data [Online]. Available: http://www.law.harvard.edu/civilrights/conferences/testing98/drafts/madaus_clarke.html
18.
Madaus, G. W. , West, M. M., Harmon, M. C., Lomax, R. G., & Viator, K. A. (1992). The influence of testing on teaching math and science in grades 4-12: Executive summary. Chestnut Hill, MA: Center for the Study of Testing, Evaluation, and Educational Policy, Boston College.
19.
McDonnell, L. M. (1997). The politics of state testing: Implementing new student assessments. Los Angeles: Center for the Study of Evaluation, University of California Los Angeles.
20.
McGill-Franzen, A. , & Allington, R. L. (1993). Flunk’em or get them classified: The contamination of primary grade accountability. Educational Researcher, 22(1), 19-22.
21.
McNeil, L. M. (2000). Contradictions of school reform. New York: Routledge.
22.
McNeil, L. , & Valenzuela, A. (1999). The harmful impact of the TAAS system on testing in Texas: Beneath the accountability rhetoric [Online]. Available: http://www.law.harvard.edu/civilrights/conferences/testing98/
23.
Mehrens, W. A. (1998, April). Consequences of assessment: What is the evidence? Paper presented at the American Educational Research Association, San Diego, CA.
24.
Mehrens, W. A. , & Kaminski, J. (1988, April). Using commercial test preparation materials: Fruitful, fruitless, or fraudulent? Paper presented at the Annual Meeting of American Educational Research Association, New Orleans, LA.
25.
Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23.
26.
Messick, S. (1995). Standards of validity and the validity of standards in performance assessment. Educational Measurement: Issues and Practice, 10(1), 5-8.
27.
National Commission on Excellence in Education . (1983). A nation at risk: The imperative for educational reform. Washington, DC: U.S. Department of Education.
28.
Porter, T. M. (1995). Trust in numbers: The pursuit of objectivity in science and public life. Princeton, NJ: Princeton University Press.
29.
Sacks, P. (1999). Standardized minds. New York: HarperCollins.
30.
Schrag, P. (2000). Too good to be true. American Prospect [Online]. Available: http://www.prospect.org/archives/V11-4/schrag.html
31.
Shepard, L. A. (1991). Inflated test score gains: Is the problem old norms or teaching to the test?Educational Measurement: Issues and Practices, 9(3), 15-22.
32.
Shepard, L. A. (1993). Evaluating test validity. Review of Research in Education, 19, 405-450.
33.
Smith, M. L. (1991). Meanings of test preparation. American Educational Research Journal, 28(3), 521-542.
34.
Smith, M. L. , Fey, P., Heinecke, W., Miller-Kahn, L., & Noble, A. J. (in press). Just do something: Symbolic politics and educational policies. New York: Routledge.
35.
Smith, M. L. , Heinecke, W., & Noble, A. J. (1999). Assessment policy and political spectacle. Teachers College Record, 101(2), 157-191.
36.
Smith, M. L. , & Rottenberg, C. (1991). Unintended consequences of external testing in elementary schools. Educational Measurement: Issues and Practice, 10, 7-11.
37.
Stone, D. (1997). Policy paradox: The art of political decision making (2nd ed.). New York: Norton.
