Sage Journals: Discover world-class research

Abstract

The evaluation of Reading First, the U.S. Department of Education’s multibillion-dollar K–3 initiative, although flawed, nevertheless offers instructive guidance for gauging the impact of future initiatives. After providing an overview of the program, its evaluation, and the historical context of federal initiatives, the authors outline limitations in applying scientific principles at scale. They argue for more nuanced approaches, including meta-analyses across projects, the use of improved statistical approaches, and the incorporation of formative designs. They conclude with four recommendations for evaluating future initiatives. Such evaluations should (a) account for fidelity systematically, (b) include outcome measures that gauge school climate and administrative support, (c) include multiple designs and aggregate the results, and (d) account for the length of implementation.

Keywords

federal programs literacy policy program evaluation reading Reading First

Get full access to this article

View all access options for this article.

References

Allington

R. L.

(Ed.). (2002). Big brother and the national reading curriculum: How ideology trumped evidence. Portsmouth, NH: Heinemann.

Bean

Draper

Turner

Zigmond

(2010). Reading First in Pennsylvania: Achievement findings after five years. Journal of Literacy Research, 42, 5–26.

Borman

G. D.

D’Agostino

J. V.

(1996). Title I and student achievement: A meta-analysis of federal evaluation results. Educational Evaluation and Policy Analysis, 18, 309–326.

Borman

G. D.

Hewes

G. M.

Overman

L. T.

Brown

(2003). Comprehensive School Reform and achievement: A meta-analysis. Review of Educational Research, 73, 125–230.

Carlisle

J. F.

Cortina

K. S.

Zeng

(2010). Reading achievement in Reading First schools in Michigan. Journal of Literacy Research, 42, 49–70.

Center on Education Policy. (2006). Keeping watch on Reading First. Washington, DC: Author. Retrieved from http://www.cep-dc.org

Center on Education Policy. (2007). Reading First: Locally appreciated, nationally troubled. Washington, DC: Author. Retrieved from http://www.cep-dc.org

Crawford

Torgesen

(2007). Teaching all students to read: Practices from Reading First schools with strong intervention outcomes. Tallahassee: Florida Center for Reading Research. Retrieved from http://www.justreadflorida.com/reading_first.asp

Darling-Hammond

(2005). Policy and change: Getting beyond bureaucracy. In Hargreaves

(Ed.), Extending educational change: International handbook of educational change (pp. 362–387). Dordrecht, The Netherlands: Springer.

10.

Darling-Hammond

(2006). Securing the right to learn: Policy and practice for powerful teaching and learning. Educational Researcher, 35(7), 13–24.

11.

Desimone

(2002). How can Comprehensive School Reform models be successfully implemented? Review of Educational Research, 72, 433–479.

12.

Dole

J. A.

Hosp

J. L.

Nelson

K. L.

Hosp

M. K.

(2010). Second opinions on the Reading First initiative: The view from Utah. Journal of Literacy Research, 42, 27–48.

13.

Duncan

(2009, June 15). Duncan offers stimulus funds for states to develop rigorous assessments linked to common standards [Press release]. Washington, DC: U.S. Department of Education. Retrieved from http://www.ed.gov/news/pressreleases/2009/06/06152009a.html

14.

Edelsky

(1990). Whose agenda is this anyway? A response to McKenna, Robinson, and Miller. Educational Researcher, 19(8), 7–11.

15.

Eisenhart

Towne

(2003). Contestation and change in national policy on “scientifically based” education research. Educational Researcher, 32(7), 31–38.

16.

Foorman

B. R.

Petscher

Lefsky

E. B.

Toste

J. R.

(2010). Reading First in Florida: Five years of improvement. Journal of Literacy Research, 42, 71–93.

17.

Gamse

B. C.

Jacob

R. T.

Horst

Boulay

Unlu

(2008). Reading First impact study: Final report (NCEE 2009-4038). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences. Retrieved from http://ies.ed.gov/ncee/pdf/20094038_1.pdf

18.

Grissmer

D. W.

Subotnik

R. F.

Orland

(2008). A guide to the use of randomized control trials (RCTs) in assessing intervention effects: The promise of multiple methods. Retrieved from http://www.apa.org/ed/schools/cpse/intervention-narrative.pdf

19.

Hamilton

L. S.

McCaffrey

D. F.

Stecher

B. M.

Klein

S. P.

Robyn

Bugliari

(2003). Studying large-scale reforms of instructional practice: An example from mathematics and science. Educational Evaluation and Policy Analysis, 25, 1–29.

20.

Higgins

J. P.

Jr. (2007, April, 20). Testimony of John P. Higgins, Jr., Inspector General, U.S. Department of Education, before the Committee on Education and Labor, U.S. House of Representatives. Washington, DC: U.S. Department of Education. Retrieved from http://www.ed.gov/about/offices/list/oig/auditrpts/stmt042007.doc

21.

Institute for Education Sciences. (n.d.). Reading for Understanding Research Initiative. Retrieved from http://ies.ed.gov/ncer/projects/program.asp?ProgID=62

22.

International Reading Association. (2006). Position statement on Reading First. Newark, DE: Author. Retrieved from http://www.reading.org/General/AboutIRA/PositionStatements/ReadingFirstPosition.aspx

23.

Jackson

McCoy

Pistorino

Wilkinson

Burghardt

Clark

Schmidt

S. R.

(2007). National evaluation of Early Reading First: Final report. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences. Retrieved from http://ies.ed.gov/ncee/pubs/20074007/index.asp

24.

Johnson

Onwuegbuzie

(2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26.

25.

Kame’enui

E. J.

Fuchs

Francis

D. J.

Good

O’Connor

R. E.

Simmons

D. C.

Torgesen

J. K.

(2006). The adequacy of tools for assessing reading competence: A framework and review. Educational Researcher, 35(4), 3–11.

26.

King

J. A.

(2003). Evaluating educational programs and projects in the USA. In Kellaghan

Stuffelbeam

D. L.

(Eds.), International handbook of educational evaluation (Part 2, pp. 721–732). Dordrecht, The Netherlands: Kluwer Academic Publishers.

27.

McDaniel

J. E.

Sims

C. H.

Miskel

C. G.

(2001). The national reading policy arena: Policy actors and perceived influence. Educational Policy, 15(1), 93–114.

28.

Miskel

Song

(2004). Passing Reading First: Prominence and processes in an elite policy network. Educational Evaluation and Policy Analysis, 26, 89–109.

29.

National Center for Education Evaluation and Regional Assistance. (n.d.). NCEE evaluation studies. Retrieved from http://ies.ed.gov/ncee/projects/

30.

National Institute of Child Health and Human Development. (2000). Report of the National Reading Panel. Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction (NIH Publication No. 00-4769). Washington, DC: U.S. Government Printing Office.

31.

Office of Inspector General. (2006). The Reading First program’s grant application process: Final inspection report. Washington, DC: U.S. Department of Education. Retrieved from http://www.ed.gov/about/offices/list/oig/aireports/i13f0017.pdf

32.

Porter

A. C.

Floden

Freeman

Schmidt

Schwille

(1988). Content determinants in elementary school mathematics. In Grows

Cooney

(Eds.), Perspectives on research on effective mathematics teaching (pp. 96–113). Reston, VA: National Council of Teachers of Mathematics.

33.

Pressley

(2005). Balanced elementary literacy instruction in the United States. In Bascia

Cumming

Datnow

Leithwood

Livingstone

(Eds.), International handbook of educational policy (Part 2, pp. 645–660). Dordrecht, The Netherlands: Springer.

34.

Reading First Federal Advisory Committee. (2008). Response to the Reading First impact study interim report. Washington, DC: U.S. Department of Education. Retrieved from http://www.ed.gov/programs/readingfirst/statement.pdf

35.

Reinking

Bradley

B. A.

(2004). Connecting research and practice using formative and design experiments. In Duke

N. K.

Mallette

M. H.

(Eds.), Literacy research methodologies (pp. 149–169). New York: Guilford.

36.

Roller

(2009). Public policy and the future of reading comprehension research. In Israel

S. E.

Duffy

G. G.

(Eds.), Handbook of research on reading comprehension (pp. 658–667). New York: Routledge.

37.

Slavin

R. E.

(2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31(7), 15–21.

38.

Slavin

R. E.

(2008). What works? Issues in synthesizing educational program evaluations. Educational Researcher, 37, 5–14.

39.

Stein

Berends

Fuchs

McMaster

Saenz

Yen

Compton

D. L.

(2008). Scaling up an early reading program: Relationships among teacher support, fidelity of implementation, and student performance across different sites and years. Educational Evaluation and Policy Analysis, 30, 368–388.

40.

Stern

(2008). Too good to last: The true story of Reading First. Washington, DC: Thomas B. Fordham Institute.

Planning and Evaluating Change at Scale

Abstract

Keywords

Get full access to this article

References