Indicators of school-level achievement, such as the percentage of students who are proficient in a particular content area, are subject to random year-to-year variation in much the same way that the results of an opinion poll will vary from one random sample to another. This random variation, which is more pronounced for a small school, should be taken into account by education officials when evaluating school progress in a policy climate of high stakes. To do otherwise is to risk the false identification of a failing school, whether for all students combined or for the subgroup of students with disabilities. In this article, I describe the application of confidence intervals to the evaluation of “adequate yearly progress” for No Child Left Behind (NCLB). Throughout, I demonstrate the particular relevance of confidence intervals for small schools in general and, more specifically, for the (smaller still) subgroup of students with disabilities.
Get full access to this article
View all access options for this article.
References
1.
Arce-Ferrer,A., Frisbie,D. A., & Kolen,M. J.(2002). Standard errors of proportions used in reporting changes in school performance with achievement levels.Educational Assessment, 8, 59–75.
2.
Borman,G. D., Hewes,G. M., Overman,L. T., & Brown,S.(2003). Comprehensive school reform and achievement: A meta-analysis.Review of Educational Research, 73, 125–230.
3.
Coladarci,T. (2003) Gallop goes to school: The Importance of confidential intervals for evaluating “Adequate Yearly Progress” in small schools.A Policy Breif of the Rural School and Community Trust.Washington, D.C.: The Rural School and Community Trust. (Available online: http://www.ruraledu.org/docs/nclb/coladarci.pdf)
4.
Cronbach,L. J., Linn,R. L., Brennan,R. L., & Haertel,E. H.(1997). Generalizability analysis for performance assessments of student achievement or school effectiveness.Educational and Psychological Measurement, 57, 373–399.
5.
Dillon,S. (2003, June 23). New Federal law may leave many rural teachers behind.New York Times, p. A1.
6.
Erpenbach,W. J., Fast,E. F., & Potts, (2003). Statewide educational accountability under NCLB.Washington, DC: Council of Chief State School Officers.
7.
Ghosh,B. K. (1979). A comparison of some approximate confidence intervals for the binomial parameter.Journal of the American Statistical Association, 74, 894–900.
8.
Glass,G. V, & Hopkins,K. D.(1996). Statistical methods in education and psychology (3rd ed.). Boston: Allyn & Bacon.
9.
Hill,R.K. (2001) Issues related to the reliability of school accountability systems. Unpublished manuscript. Portsmouth, NH: The National Center for Improvement of Educational Assessment. (Available online: http://www.nciea.org/publications/RILS2000Paper_Hill01.pdf)
10.
Hill,R. K. (2002, April). Examining the reliability of accountability systems. Paper presented at the 2002 meeting of the American Educational Research Association, New Orleans, LA. (Available online: http://www.nciea.org/publications/NCME_RHCD03.pdf)
11.
Hill,R. K., & DePascale,C. A.(2003). Reliability of No Child Left Behind accountability designs.Educational Measurement: Issues and Practices, 22(3), 12–20.
12.
Kane,T. J., & Staiger,D. O.(2001, May). Volatility in school test scores: Implications for test-based accountability systems.Accountability and its consequences for students: Are children hurt or helped by standards-based reforms? Symposium presented at the Brookings Institution, Washington, DC.
Linn,R. L., & Haug,C.(2002). Stability of school-building accountability scores and gains.Educational Evaluation and Policy Analysis, 24(1), 29–36.
17.
Marion,S. F., White,C., Carlson,D., Erpenbach,W.J., Rabinowitz,S., & Sheinker,J.(2002). Making valid and reliable decisions in determining adequate yearly progress.Washington, DC: Council of Chief State School Officers. (Available online: http://www.ccsso.org/contents/pdfs/AYPpaper.pdf)
Tompkins,R. B. (2003, March 26). Leaving rural children behind.Education Week, pp. 44, 30, 31.
22.
United States Government Accountability Office (2004, September). No Child Left Behind Act: Additional Assistance and Research on Effective Strategies Would Help Small Rural Districts. Washington, D.C.: The author.
