Several school performance indices were evaluated using achievement data from multiple longitudinally matched middle school student cohorts in a large district in the southwestern United States. Measures designed to index the within-cohort achievement status and growth of students as well as the status and growth-based changes that occur over years between different student cohorts were studied. Within- and between-cohort status and growth-based estimates of school performance were generated by applying simple and relatively complex statistical models to the longitudinal time-series data obtained on students. Results indicated that within-cohort status-based estimates were closely related to student demographics and between-cohort estimates were associated with cohort enrollment size and initial performance level. These results suggest that schools may often be rewarded and penalized on the basis of factors over which school personnel have limited or no control. Implications for the measurement and evaluation of school performance are discussed.
American Evaluation Association.2006. Public statement: Educational accountability. Fairhaven, MA: American Evaluation Association. http://www.eval.org/aea.edac.flyer.pdf (accessed February 20, 2008).
2.
Ballou, D., W. Sanders, and P. Wright.2004. Controlling for student background in value-added assessment of teachers. Journal of Educational and Behavioral Statistics29 (1): 37-65.
3.
Bryk, A.S., and S.W. Raudenbush .1988. Toward a more appropriate conceptualization of research on school effects: A three-level hierarchical model. American Journal of Education97:65-108.
4.
Bryk, A.S., S.W. Raudenbush, and S. Ponisciak.2004. A value-added model for assessing improvements in school productivity: Results from the Washington, DC public schools and an analysis of their statistical conclusion validity. Chicago: Consortium on Chicago School Research.
5.
Bryk, A., Y.M. Thum, J.Q. Easton, and S. Luppescu.1998. Assessing school academic productivity: The case of Chicago school reform. Social Psychology of Education2:103-42.
6.
Carlson, D.2002. The focus on state educational accountability systems: Four methods of judging school quality and progress. In Incorporating multiple measures of student performance into state accountability systems: A compendium of resources, ed. W. J. Erpenbach , 285-97. Washington, DC: Council of Chief State School Officers .
7.
Council of Chief State School Officers.2006 . ESEA reauthorization policy statement: CCSSO policy statement on the reauthorization of the Elementary and Secondary Education Act. Washington, DC: Council of Chief State School Officers . http://www.ccsso.org/content/pdfs/ESEA_Policy_Stmnt.pdf (accessed February 20, 2008).
8.
---. 2007. Recommendations to reauthorize the elementary and secondary education act. Washington, DC: Council of Chief State School Officers. http://www.ccsso.org/content/pdfs/ESEA_rec_final.pdf (accessed February 20, 2008).
9.
Clotfelter, C.T., and H.F. Ladd.1996. Recognizing and rewarding success in public schools. In Holding schools accountable: Performance-based reform in education, ed. H. F. Ladd, 23-63. Washington, DC: The Brookings Institution.
Elmore, R.F., C.H. Abelmann, and S.H. Fuhrman.1996. The new accountability in state education reform: From process to performance. In Holding schools accountable: Performance-based reform in education, ed. H. F. Ladd, 65-98. Washington, DC: The Brookings Institution .
12.
Erpenbach, W.J., E. Forte-Fast, and A. Potts.2003. Statewide educational accountability under NCLB: Central issues arising from an examination of state accountability work-books and U.S. Department of Education reviews under the No Child Left Behind Act of 2001. Washington, DC: Council of Chief State School Officers.
13.
Flicek, M.2004. Judging school quality using longitudinal methods that are comprehensible to stakeholders. Paper presented at the annual meeting of the American Educational Research Association , April 12-16, San Diego, CA.
14.
Forte-Fast, E., and S. Hebbler.2004. A framework for examining validity in state accountability systems. Washington, DC: Council of Chief State School Officers.
15.
Goldschmidt, P., P. Roschewski, K. Choi, W. Auty, S. Hebbler, R. Blank, and A. Williams.2005. Policymakers' guide to growth models and school accountability: How do accountability models differ?Washington, DC: Council of Chief State School Officers .
16.
Gong, B.2004. Models for using student growth measures in school accountability . Paper presented at the Council of Chief State School Officers' "Brain Trust" on value-added models, November 15-16, Washington, DC.
17.
Hill, R.K., and C.A. DePascale.2003. Reliability of No Child Left Behind accountability designs. Educational Measurement: Issues and Practice22:12-20.
18.
Kane, T.J., and D.O. Staiger.2002. Volatility in school test scores: Implications for test-based accountability systems. Brookings Papers on Educational Policy1:235-83.
19.
Kim, J.S., and G.L. Sunderman.2005. Measuring academic proficiency under the No Child Left Behind Act: Implications for educational equity. Educational Researcher34 (8): 3-13.
20.
Linn, R.L.2003. Accountability: Responsibility and reasonable expectations . Educational Researcher32 (7): 3-13.
21.
---. 2004. Accountability models. In Redesigning accountability systems for education, ed. S. H. Fuhrman and R. F. Elmore, 73-95. New York: Teachers College Press.
22.
Linn, R.L., and C. Haug.2002. Stability of school-building accountability scores and gains. Educational Evaluation and Policy Analysis24 (1): 29-36.
23.
Marion, S., C. White, D. Carlson, W.J. Erpenbach, S. Rabinowitz, and J. Sheinker.2002. Making valid and reliable decisions in determining adequate yearly progress. Washington, DC : Council of Chief State School Officers.
24.
National Conference of State Legislatures.2005. Key recommendations from the NCSL task force on No Child Left Behind (NCLB) final report. http://www.ncsl.org/print/statefed/nclb/NCLBRecommendations.pdf (accessed August 9, 2005).
25.
---. 2007. Joint statement of the National Conference of State Legislatures and the American Association of School Administrators on ESEA reauthorization . http://www.ncsl.org/print/statefed/NCLB/NCSLAASAJointNCLBStatement.pdf (accessed February 20, 2008).
26.
No Child Left Behind Act of 2001, Pub. L. No. 107-110 (2002).
27.
Olson, L.2004. Value-added models gain in popularity. Education Week24 (12): 1-4.
28.
Parkes, J., & J.J. Stevens.2003. Legal issues in school accountability systems. Applied Measurement in Education16 (2): 141-58.
29.
Ponisciak, S.M., and A. Bryk.2005. Value-added analysis of the Chicago public schools: An application of hierarchical models. In Value-added modeling: Issues with theory and applications, ed. R. Lissitz, 40-81. Maple Grove, MN: JAM Press.
30.
Porter, A.C., and M.S. Polikoff.2007. NCLB: State interpretations, early effects, and suggestions for reauthorization. Social Policy Report21 (4), Society for Research in Child Development. http://www.srcd.org/documents/publications/spr/21-4_no_child_left_behind.pdf (accessed May 20, 2008).
31.
Raudenbush, S.W.2004. Schooling, statistics, and poverty: Can we measure school improvement?Paper presented at the William H. Angoff Memorial Lecture Series, April 1, Princeton, NJ.http://www.ets.org/research/ (accessed January 22, 2005).
32.
Riddle, W.C.2005. Adequate yearly progress (AYP): Might growth models be allowed under the No Child Left Behind Act?Congressional Research Service Report for Congress, Congressional Research Service: The Library of Congress.
33.
Sanders, W.L., and S.P. Horn.1994. The Tennessee value-added assessment system (TVAAS): Mixed-model methodology in educational assessment. Journal of Personnel Evaluation in Education8:299-311.
34.
Stevens, J.J.2005. The study of school effectiveness as a problem in research design. In Value-added modes in education: Theory and applications , ed. R. Lissitz, 166-208. Maple Grove, MN: JAM Press.
35.
Stevens, J.J., S. Estrada, and J. Parkes.2000. Measurement issues in the design of state accountability systems. Paper presented at the annual meeting of the American Educational Research Association, April, New Orleans, LA.
36.
Teddlie, C., and D. Reynolds.2000. The international handbook of school effectiveness research. New York: Falmer.
37.
U.S. Department of Education.2007. Secretary Spellings invites eligible states to submit innovative models for expanded growth model pilot. http://www.ed.gov/news/pressreleases/2007/12/12072007.html (accessed February 20, 2008).
38.
Willms, J.D.1992. Monitoring school performance: A guide for educators . Washington, DC: Falmer.
39.
Zvoch, K., and J.J. Stevens.2003. A multilevel, longitudinal model of middle school math and language achievement. Educational Policy Analysis Archives, 11 (20). http://epaa.asu.edu/epaa/v11n20 (accessed July 8, 2003).
