In 2002 the UK Home Office commissioned a review of research on community involvement in area-based initiatives. This found comparatively few studies that set out to measure the impact rather than the extent and nature of involvement and hence few answers to the question of what works. This article takes that finding as its starting point and sets out to develop a more robust framework for evaluating the impact of community involvement. It notes the difficulties inherent in using a classic experimental design to evaluate processes as complex as community involvement and proposes a theory-based approach. To this end, it critically reviews the underlying theoretical claims of both community involvement and of area-based initiatives. An evaluation framework is then developed in which the potential benefits of greater involvement are considered for each stage of the process of developing an area-based initiative and positive and negative contextual factors are identified.
Attwood, C. , G. Singh , D. Prime , R. Creasyet al. (2003) 2001 Home Office Citizenship Survey: People, Families and Communities. London: Home Office/RDS .
2.
Audit Commission (2004) ‘Neighbourhood Renewal and Community Consultation’, Guidance Paper Based on Policy Focus: Neighbourhood Renewal. London: Audit Commission . Available at: www.audit.commission.gov.ak/reports/ (accessed11 Feb. 2004).
3.
Barber, B. (1984) Strong Democracy: Participatory Politics for a New Age. Berkeley, CA: University of California Press .
4.
Barnes, M. , E. Matka and H. Sullivan (2003) ‘Evidence, Understanding and Complexity’ , Evaluation9(3): 265-284 .
5.
Blunkett, D. (2003a) ‘Civil Renewal: A New Agenda’, Edith Kahn Memorial Lecture. London: Home Office .
6.
Blunkett, D. (2003b) Active Citizens, Strong Communities: Progressing Civil Renewal. Paper based on the Home Secretary’s Scarman Lecture delivered at the Citizens’ Convention, 11 Dec. London: Home Office.
7.
Boaz, A. and D. Ashby (2003) ‘Fit for Purpose? Assessing Research Quality for Evidence Based Policy and Practice’, Working Paper 11. London: ESRC UK Centre for Evidence Based Policy and Practice .
8.
Boaz, A. , D. Ashby and K. Young (2002) ‘Systematic Reviews: What Have They Got to Offer Evidence Based Policy and Practice?’, Working Paper 2. London: ESRC UK Centre for Evidence Based Policy and Practice .
9.
Bradford, M. and B. Robson (1995) ‘An Evaluation of Urban Policy’, in R. Hambelton and H. Thomas (eds) Urban Policy Evaluation: Challenges and Change, pp. 37-54. London: Paul Chapman .
10.
Burton, P. (1997) ‘Urban Policy and the Myth of Progress’ , Policy and Politics25(4): 421-436 .
11.
Burton, P. (2004) ‘Power to the People?’ , Local Economy19(3): 193-198 .
12.
Burton, P. , R. Goodlad , J. Croft , J. Abbott , A. Hastings , G. Macdonald and T. Slater (2004) ‘What Works in Community Involvement in Area-Based Initiatives? A Systematic Review of the Literature’, Home Office Online Report 53/04. London: Home Office . Available at: http://www.homeoffice.gov.uk/rds/pdfs04/rdsolr5304.pdf (accessed 25 Jan. 2005).
13.
Campbell, D. (1969) ‘Reforms as Experiments’ , American Psychologist24: 409-429 .
14.
Chanan, G. (2003) Searching for Solid Foundations: Community Involvement and Urban Policy. London: Office of the Deputy Prime Minister .
15.
Chen, H. (1990) Theory-Driven Evaluations. London: SAGE .
16.
Comptroller and Auditor General (2004) Getting Citizens Involved: Community Participation in Neighbourhood Renewal. London: Stationery Office .
17.
Connell, J. P. , A. C. Kubisch , L. B. Schorr and C. H. Weiss (eds) (1995) New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts. Washington, DC: Aspen Institute .
18.
Cook, T. D. (2000) ‘The False Choice between Theory-Based Evaluation and Experimentation’, in P. Rogers , T. Hacsi , A. Petrosino and T. Huebner (eds) Program Theory in Evaluation: Challenges and Opportunities, pp. 27-34. San Francisco, CA: Jossey-Bass .
19.
Cooke, B. and U. Kothari , eds (2001) Participation: The New Tyranny?London: Zed Books .
20.
Coulthard, M. , A. Walker and A. Morgan (2003) People’s Perceptions of Their Neighbourhood and Community Involvement: Results from the Social Capital Module of the General Household Survey 2000, Office for National Statistics. London: Stationery Office .
21.
Crow, I. , A. France , S. Hacking and M. Hart (2004) Does Communities that Care Work? An Evaluation of a Community-Based Risk Prevention Programme in Three Neighbour-hoods. York: Joseph Rowntree Foundation .
22.
Denzin, N. and Y. Lincoln (2000) Handbook of Qualitative Research. Thousand Oaks, CA: SAGE .
23.
Department of Environment, Transport and the Regions (2000) Our Towns and Cities: The Future, Delivering Urban Renaissance. London: DETR , Cm 4911.
24.
Department of Transport, Local Government and the Regions (2001) Strong Leadership: Quality Public Services. London: Stationery Office , Cm 5237.
25.
Edwards, J. (1997) ‘Urban Policy: The Triumph of Form over Content’ , Urban Studies35: 825-843 .
26.
Farrington, D. (1997) ‘Evaluating a Community Crime Prevention Program’ , Evaluation3: 157-173 .
27.
Feder, L. and R. F. Boruch (2000) ‘The Need for Experiments in Criminal Justice Settings’ , Crime and Delinquency46(3): 291-294 .
28.
Fetterman, D. (1994) ‘Empowerment Evaluation’ , Evaluation Practice15: 1-15 .
29.
Fulbright-Anderson, K. , A. C. Kubisch and J. P. Connell , eds (1998) New Approaches to Evaluating Community Initiatives, vol. 2, Theory, Measurement, and Analysis. Washington, DC: Aspen Institute .
30.
Funnell, S. (2000) ‘Developing and Using a Program Theory Matrix for Program Evaluation and Performance Monitoring’, in P. Rogers , T. Hacsi , A. Petrosino and T. Huebner (eds) Program Theory in Evaluation: Challenges and Opportunities, pp. 91-102. San Francisco, CA: Jossey-Bass .
31.
Goggin, M. , A. Bowman , J. Lester and L. O’Toole (1990) Implementation Theory and Practice: Toward a Third Generation. Glenville, IL: Little Brown .
32.
Grayson, L. and A. Gomersall (2003) ‘A Difficult Business: Finding the Evidence for Social Science Reviews’, Working Paper 19. London: ESRC UK Centre for Evidence Based Policy and Practice .
33.
Hammersley, M. and P. Atkinson (1995) Ethnography: Principle in Practice. London: Routledge .
34.
Harper, R. (2001) Social Capital: A Review of the Literature. London: Social Analysis and Reporting Division, Office for National Statistics .
35.
Hill, M. and P. Hupe (2002) Implementing Public Policy. London: SAGE .
36.
Hirst, P. and S. Khilnani , eds (1996) Reinventing Democracy. Oxford: Blackwell Publishers .
37.
HM Treasury (2003) The Green Book: Evaluation and Appraisal for Government. London: HM Treasury (website accessed 5 March 2004).
38.
Judge, K. and L. Bauld (2001) ‘Strong Theory, Flexible Methods: Evaluating Complex Community-Based Initiatives’ , Critical Public Health11(1): 19-38 .
39.
Macdonald, G. (2003) Using Systematic Reviews to Improve Social Care. London: Social Care Institute for Excellence .
40.
Nutley, S. , I. Walter and H. T. O. Davies (2003) ‘From Knowing to Doing: A Framework for Understanding the Evidence-into-Practice Agenda’ , Evaluation9(2): 125-148 .
41.
Oakley, A. (1998a) ‘Experimentation and Social Intervention: A Forgotten but Important History’ , British Medical Journal317: 1239-1242 .
42.
Oakley, A. (1998b) ‘Public Policy and Experimentation: Lessons from America’ , Policy Studies19(2): 93-114 .
43.
Oakley, A. (2000) ‘A Historical Perspective on the Use of Randomized Trials in Social Science Settings’ , Crime and Delinquency46(3): 315-329 .
44.
Pawson, R. and N. Tilley (1997) Realistic Evaluation. London: SAGE .
45.
Pawson, R. and N. Tilley (1998a) ‘Caring Communities, Paradigm Polemics, Design Debates’ , Evaluation4(1): 211-213 .
46.
Pawson, R. and N. Tilley (1998b) ‘Cook-Book Methods and Disastrous Recipes: A Rejoinder to Farrington’ , Evaluation4(2): 211-213 .
47.
Prime, D. , M. Zimmeck and A. Zurawan, A. (2002) Active Communities: Initial Findings from the 2001 Home Office Citizenship Survey. London: Home Office .
48.
Raco, M. (2003) ‘New Labour, Community and the Future of Britain’s Urban Renaissance’, in R. Imrie and M. Raco , eds, Urban Renaissance? New Labour, Community and Urban Policy, pp. 239-249. Bristol: Policy Press .
49.
Richardson, A. (1983) ‘Participation’, in V. George and P. Wilding (eds) Concepts in Social Policy, p. 151-151. London: Routledge & Kegan Paul .
50.
Ritchie, J. and J. Lewis (2003) Qualitative Research Practice: A Guide for Students and Researchers. London: SAGE .
51.
Roberts, M. and M. Roche (2001) ‘Quantifying Social Capital: Measuring the Intangible in the Local Policy Context’ , Radical Statistics76, Winter. Available at: http://radstats.org.uk/no076/robertsandroche.htm (accessed 8 March 2004).
52.
Rogers, P. , T. Hacsi , A. Petrosino and T. Huebner , eds (2000) ‘Program Theory in Evaluation: Challenges and Opportunities’, New Directions for Evaluation, 87. San Francisco, CA: Jossey-Bass .
53.
Rossi, P. , H. Freeman and M. Lipsey (1999) Evaluation: A Systematic Approach, 6th edn.London: SAGE .
54.
Sanderson, I. (2000) ‘Evaluation in Complex Policy Systems’ , Evaluation6(4): 433-454 .
55.
Social Exclusion Unit (1998) Bringing Britain Together: A National Strategy for Neighbourhood Renewal. London: Stationery Office , Cm 4045.
56.
Social Exclusion Unit (2001) A New Commitment to Neighbourhood Renewal: National Strategy Action Plan. London: Cabinet Office .
57.
Spicker, P. (2001) ‘Poor Areas and the Ecological Fallacy’ , Radical Statistics76, Winter. Available at: http://radstats.org.uk/no076/spicker.htm (accessed 8 March 2004).
58.
Stone, W. (2001) ‘Measuring Social Capital: Towards a Theoretically Informed Measurement Framework for Researching Social Capital in Family and Community Life’, Research Paper No 24. Melbourne: Australian Institute of Family Studies .
59.
Taylor, M. (2003) Public Policy in the Community. Houndmills: Palgrave Macmillan .
60.
Walker, D. (2002) ‘In Praise of Centralism: A Critique of the New Localism’, Catalyst Working Paper. London: Catalyst .
61.
Weiss, C. (2000) ‘Which Links in Which Theories Shall We Evaluate?’, in P. Rogers , T. Hacsi , A. Petrosino and T. Huebner (eds) Program Theory in Evaluation: Challenges and Opportunities, pp. 35-46. San Francisco, CA: Jossey-Bass .
