Evaluating Complex,Area-Based Initiatives in a Context of Change

Abstract

Complex, area-based initiatives are a widely adopted policy response to problems of health inequalities and social exclusion in industrialized countries such as the UK and the US.This article explores the problems of evaluating such initiatives in a context of rapid national and local policy change by drawing on the Children's Fund, a UK government initiative that established local partnerships in each English local authority to promote children's social inclusion. It is suggested that independent local evaluations commissioned by the partnerships have had limited influence on programme design and implementation, and indeed have been highly contested in their use.The importance of dialogic engagement between evaluators and stakeholders is highlighted.

Les initiatives zonées complexes constituent une réponse courante aux problàmes d’inégalités en matiàre de santé et d’exclusion sociale dans les pays industrialisés tels que la Grande-Bretagne et les Etats-unis. Cet article explore les problàmes rencontrés pour évaluer de telles initiatives dans un contexte de changement politique rapide à partir de l’exemple du Fonds pour les enfants, une initiative du gouvernement britannique qui institue des partenariats locaux dans chaque collectivité locale britannique dans le but de promouvoir l’inclusion sociale des enfants. On suggàre que les évaluations locales indépendantes fi nancées par les partenariats ont eu une infl uence limitée sur la conception et la mise en oeuvre du programme et ont été tràs contestées dans leur utilisation. On met en évidence l’’importance d’un engagement dialogique entre les évaluateurs et les porteurs d’enjeux.

Keywords

changing policy context Children's Fund evidence-based policy local independent evaluation National Evaluation

Get full access to this article

View all access options for this article.

References

Auspos, P. and A. Kubisch (2004) Building Knowledge about Community Change. Washington, DC: Aspen Institute .

Balthasar, A. and S. Reider (2000) ` Learning from Evaluations: Effects of the Swiss Energy 2000 Programme ', Evaluation 6(3): 245—60 .

Becker, S. and A. Bryman , eds (2004) Understanding Research for Social Policy and Practice: Themes, Methods and Approaches. Bristol : Policy Press .

Boaz, A. and C. Hayden (2002) ` Pro-Active Evaluators: Enabling Research to be Useful, Usable and Used ', Evaluation 8(4): 440—53 .

Bowen, S. and A. Zwi (2005) ` Pathways to “Evidence-Informed” Policy and Practice: A Framework for Action', Public Library of Science 2(7): 600—5 .

Buse, K. , N. Mays and G. Walt ( 2005) Making Health Policy. Maidenhead: Open University Press .

Children and Young People's Unit (2001) Children's Fund Guidance. London: DfES .

Children and Young People's Unit and the Youth Justice Board (2002) Use of Children's Fund Partnership Funding for Crime Prevention Activities Jointly Agreed with Youth Offending Teams: Guidance. London: DfES .

Coote, A. , J. Allen and D. Woodhead (2004) Finding out What Works: Understanding Complex, Community-Based Initiative. London: Kings Fund .

10.

Davies, H. , S. Nutley and P. Smith , eds (2000) What Works? Evidence-Based Policy and Practice in Public Services. Bristol: Policy Press .

11.

Donaldson, S. and M. Scriven , eds (2003) Evaluating Social Programmes and Problems: Visions for the New Millennium. London: Lawrence Erlbaum Associates .

12.

Edwards, A. , M. Barnes , I. Plewis and K. Morris (2006) Working to Prevent the Social Exclusion of Children and Young People: Final Lessons from the National Evaluation of the Children's Fund. London: Department for Education and Skills .

13.

Hall, P. , H. Land , R. Parker and A. Webb (1975) Change, Choice and Conflict in Social Policy. London: Heinemann .

14.

Hope, T. (2005) ` Things Can Only Get Better ', Criminal Justice Monthly 62(2005/6): 4—6 .

15.

McDonald, G. (2000) `Social Care: Rhetoric and Reality', in H. Davies , S. Nutley and P. Smith (eds) What Works? Evidence-Based Policy and Practice in Public Services. Bristol: Policy Press .

16.

Martin, S. and I. Sanderson (1999) ` Evaluating Public Policy Experiments: Measuring Outcomes, Monitoring Processes or Managing Pilots? ', Evaluation 5(3): 245—58 .

17.

Mason, P. , K. Morris and P. Smith (2005) ` A Complex Solution to a Complicated Problem? Early Messages from the National Evaluation of the Children's Fund Prevention Programme ', Children and Society 19(2): 131—43 .

18.

Morris, K. and N. Spicer (2003) The National Evaluation of the Children's Fund: Early Messages from Developing Practice. London: Department for Education and Skills .

19.

Nutley, S. , I. Walter and N. Bland (2002) ` The Institutional Arrangements for Connecting Evidence and Policy: The Case of Drug Misuse ', Public Policy and Administration 17(3): 76—94 .

20.

Percy-Smith, J. (2000) Policy Responses to Social Exclusion: Towards Inclusion? Maidenhead: Open University Press .

21.

Pierson, J. (2002) Tackling Social Exclusion. London: Routledge .

22.

Sanderson, I. (2000) ` Evaluation of Complex Policy Systems ', Evaluation 6(4): 433—54 .

23.

Sanderson, I. (2004) ` Getting Evaluation into Practice: Perspectives on Rationality ', Evaluation 10(3): 366—79 .

24.

Short, S. (1997) `Elective Affinities: Research and Health Policy Development', in H. Gardner (ed.) Health Policy in Australia. San Francisco, CA: Jossey-Bass .

25.

Solesbury, I. (2001) Evidence Based Policy: Whence it Came and Where it is Going. UK Centre for Evidence Based Policy and Practice Working Paper 1. London: ESRC .

26.

Spencer, L. , J. Ritchie , J. Lewis and L. Dillon (2004) Quality in Qualitative Evaluation: A Framework for Assessing Research Evidence. London: Cabinet Office .

27.

Spicer, N. (2003) ` Evaluation as White-Water Rafting: The Case of the LSL Health Action Zone ', paper presented at the Third International Seminar on Finding out What Works: Evidence, Policy and Practice, Baltimore, 28—31 Jan.

28.

Spicer, N. and P. Smith (2006) Local Evaluation of Children's Services: Learning from the Children's Fund. London: Department for Education and Skills .

29.

Spicer, N. , R. Evans , P. Smith and K. Broughton (2004) Children, Young People, Parents and Carers' Participation in Children's Fund Case Study Partnerships. London: Department for Education and Skills .

30.

Taylor, D. (2005) ` Governing through Evidence: Participation and Power in Policy Evaluation', Journal of International Policy 34(4): 601—18 .

31.

Weiss, C. (1979) ` The Many Meanings of Research Utilization ', Public Administration Review 426—31 .

32.

Weiss, C. (1999) ` The Interface between Evaluation and Public Policy ', Evaluation 5(4): 468—86 .

33.

Young, K. , D. Ashby , A. Boaz and L. Grayson (2002) ` Social Science and the Evidence-Based Policy Movement ', Social Policy and Society 1(3): 215—24 .