Richness or Rigour? A Discussion of Systematic Reviews and Evidence-Based Policy in Early Childhood

Abstract

In this article, the authors report on the experiences of the Early Years Review Group, one of a number of education groups contracted to carry out systematic reviews for the Evidence for Policy and Practice Information and Coordinating (EPPI) Centre in the United Kingdom. The Early Years Review Group has carried out three systematic reviews: one on the impact of integration of care and education in the early years; one on providing support to young children affected by war and armed conflict; and one on the long-term cost benefits of early childhood interventions. Using the evidence from the third review, the authors address the issue of what constitutes useful evidence for policy makers in the field of early childhood and whether certain kinds of evidence are privileged. They conclude that the systematic review process is an independent and useful tool for analysing and critiquing existing studies.

References

Aos

Phipps

Barnoski

& Lieb

(2001) The Comparative Costs and Benefits of Programs to Reduce Crime. Washington, DC: Washington State Institute for Public Policy.

Barnett

W.S.

(1993) Benefit-cost Analysis of Preschool Education: Findings from a 25-year follow-up, American Journal of Orthopsychiatry, 63(4), pp. 500–508.

Barnett

W.S.

(1996) Lives in the Balance: Age-27 benefit-cost analysis of the High/Scope Perry Preschool Program. Ypsilanti: High/Scope Educational Research Foundation.

Boaz

& Pawson

(2005) The Arduous Road from Evidence to Policy, Journal of Social Policy, 34(2), pp. 173–194. http://dx.doi.org/10.1017/S0047279404008530

Bourgois

(1998) Families and Children in Pain in the US Inner City, in Scheper-Hughes

& Sargent

(Eds) Small Wars, pp. 331–351. Los Angeles: University of California Press.

Brooks-Gunn

(2003) Do You Believe in Magic? What We Can Expect from Early Childhood Intervention Programs, Social Policy Report, XVII(1), pp. 3–7.

Campbell

F.A.

& Ramey

C.T.

(1995) Cognitive and School Outcomes for High-risk African-American Students at Middle Adolescence: Positive effects of early intervention, American Educational Research Journal, 32(4), pp. 743–772. http://dx.doi.org/10.2307/1163334

Chalmers

(2005) If Evidence-informed Policy Works in Practice, Does it Matter if it Doesn't Work in Theory? Evidence and Policy, 1(2), pp. 227–242. http://dx.doi.org/10.1332/1744264053730806

Chatterji

(2004) Evidence on ‘What Works’: An argument for extended-term mixed-method (ETMM) evaluation designs, Educational Researcher, 33(9), pp. 3–13.

10.

Clarke

S.H.

& Campbell

F.A.

(1998) Can Intervention Early Prevent Crime Later? The Abecedarian Project Compared with Other Programs, Early Childhood Research Quarterly, 13(2), pp. 319–343. http://dx.doi.org/10.1016/S0885-2006(99)80042-8

11.

Convention on the Rights of the Child (2005) Implementing Child Rights in Early Childhood. General Comment No. 7. New York: United Nations.

12.

Currie

(2000) Early Childhood Intervention Programs: What do we know? Arlington: National Science Foundation, Joint Center for Poverty Research.

13.

Currie

(2004) Viewpoint: Child research comes of age, Canadian Journal of Economics, 37(3), pp. 509–527. http://dx.doi.org/10.1111/j.0008-4085.2004.00236.x

14.

Dahlberg

& Moss

(2005) Ethics and Politics in Early Childhood. London: Routledge.

15.

Dahlberg

Moss

& Pence

(1999) Beyond Quality in Early Childhood: Post-modern perspectives. London: Falmer.

16.

Feuer

Towne

& Shavelson

R.J.

(2002) Scientific Culture and Educational Research, Educational Researcher, 31(8), pp. 4–14.

17.

Goodman

& Sianesi

(2005) Early Education and Children's Outcomes: How long do the impacts last? Fiscal Studies, 26(4), pp. 513–548. http://dx.doi.org/10.1111/j.1475-5890.2005.00022.x

18.

Greenwood

Model

& Chiesa

(1998) Diverting Children from a Life of Crime: Measuring costs and benefits. Santa Monica: RAND Corporation.

19.

Hammersley

(2001) On Systematic Reviews of Research Literature: A ‘narrative’ response to Evans and Benfield, British Educational Research Journal, 23(2), pp. 141–161.

20.

Heath

S.B.

(1983) Ways with Words: Language, life and work in communities and classrooms. Cambridge: Cambridge University Press.

21.

Heckman

(1999) Policies to Foster Human Capital. Working Paper No. 7288. Cambridge, MA: National Bureau of Economic Research.

22.

James

Pollard

Rees

& Taylor

(2005) Researching Learning Outcomes: Building confidence in our conclusions, Curriculum Journal, 16(1), pp. 109–122.

23.

Johnson

Jaeger

Randolph

Cauce

Ward

& National Institute of Child Health and Human Development Early Child Care Research Network (2003) Studying the Effects of Early Child Care Experiences on the Development of Children of Color in the United States: Towards a more inclusive research agenda, Child Development, 74(5), pp. 1227–1244. http://dx.doi.org/10.1111/1467-8624.00604

24.

Karoly

Greenwood

Everingham

Hoube

Kilburn

Rydell

Sanders

& Chiesa

(1998) Investing in Children: What we know and what we don't know about the costs and benefits of early childhood interventions. Santa Monica: RAND Corporation.

25.

Karoly

L.A.

Kilburn

M.R.

Bigelow

J.H.

Caulkins

J.P.

& Cannon

J.S.

(2001) Assessing Costs and Benefits of Early Childhood Intervention Programs. http://www.rand.org/publications/MR/MR1336

26.

Lloyd

Penn

Barreau

Burton

Davis

Potter

& Sayeed

(2005) How Effective Are Measures Taken to Mitigate the Impact of Direct Experience of Armed Conflict on the Psychosocial and Cognitive Development of Children Aged 0–8? Research Evidence in Education Library, EPPI-Centre, Social Science Research Unit, Institute of Education, University of London. http://eppi.ioe.ac.uk/cms/

27.

MacLure

(2005) ‘Clarity Bordering on Stupidity’: Where's the quality in systematic review? Journal of Educational Policy, 20(4), pp. 393–416.

28.

Masse

& Barnett

W.S.

(2003) A Benefit Cost Analysis of the Abecedarian Programme. New Brunswick: National Institute for Early Education Research.

29.

Miller

Cohen

& Wiersema

(1996) Victim Costs and Consequences: A new look. Washington, DC: US Department of Justice, Office of Justice Programs, National Institute of Justice.

30.

Morrison

(1993) Lecture and Speech of Acceptance upon the Award of the Nobel Prize for Literature, The Nobel Foundation, Stockholm, 7 December.

31.

Moseley

(2004) Little Scarlet. London: Little, Brown.

32.

Mosteller

& Boruch

(2002) Evidence Matters: Randomised trials in education research. Washington, DC: Brookings Institution Press.

33.

Oakley

(2000) Experiments in Knowing: Gender and method in the social sciences. Cambridge: Polity.

34.

Oakley

Gough

Oliver

& Thomas

(2005) The Politics of Evidence and Methodology: Lessons from the EPPI-Centre, Evidence and Policy, 1(1), pp. 5–31. http://dx.doi.org/10.1332/1744264052703168

35.

Organisation for Economic Co-operation and Development (OECD) (2001) Starting Strong: Early education and care. Paris: OECD. Organisation for Economic Co-operation and Development (OECD) (2006) Review of Early Education and Care in 20 Countries. Paris: OECD.

36.

Parish

O'Reilly

Duenas

& Wolman

(1997) State Special Education Finance Systems 1994–1995. Palo Alto: Center for Special Education Finance, American Institutes for Research.

37.

Penn

(2005) Unequal Childhoods: Young children's lives in poor countries. London: Routledge.

38.

Penn

Barreau

Butterworth

Lloyd

Moyles

Potter

& Sayeed

(2004) What Is the Impact of Out-of-home Integrated Care and Education Settings on Children Aged 0–6 and Their Parents? Research Evidence in Education Library, EPPI-Centre, Social Science Research Unit, Institute of Education, University of London. http://eppi.ioe.ac.uk/cms/

39.

Penn

Burton

Lloyd

Mugford

Potter

& Sayeed

(2006) What Is Known about the Long-term Economic Impact of Center-based Early Childhood Interventions? Research Evidence in Education Library, EPPI-Centre, Social Science Research Unit, Institute of Education, University of London. http://eppi.ioe.ac.uk/cms/

40.

Pring

& Thomas

(2004) Evidence-based Practice in Education. Maidenhead: Open University Press.

41.

Pungello

Campbell

F.A.

& Johnson

Miller S.

(2000) Benefits of High Quality Childcare for Low-income Mothers: The Abecedarian study. Paper presented at Fifth Head Start National Research Conference, Washington, DC, 28 June-1 July.

42.

Reynolds

A.J.

Temple

J.A.

Robertson

D.L.

& Mann

E.A.

(2002) Age 21 Cost-benefit Analysis of the Title I Chicago Child-Parent Centers, Educational Evaluation and Policy Analysis, 24(4), pp. 267–303.

43.

Rosaldo

(1993) Culture and Truth: The remaking of social analysis. London: Routledge.

44.

Takaki

(1995) Culture Wars in the United States: Closing reflections on the century of the colour line, in Pieterse

Nederveen J.

& Parekh

(Eds) The Decolonization of Imagination, pp. 166–176. London: Zed Books.

45.

Weikart

D.P.

(Ed.) (1967) Preschool Interventions: Preliminary results of the Perry Preschool Project. Ann Arbor: Campus.

46.

World Bank (2005) Early Child Development: Investment in human capital. Paper presented at World Bank Symposium on Early Childhood Development, Washington, 17–19 September.

47.

World Prison Brief (2005). London: International Centre for Prison Studies, King's College London.