Using `evidence' to falsify rather than verify patterns in data and searching for alternative explanations enables a better understanding of the circumstances that explain why and how a social programme works or does not work. An analysis of the extent to which a programme is meeting its aims and objectives to find out if it provides a solution to the policy problem, is more rigorous.The roles researchers adopt influence the quality of an evaluation; facilitating a better understanding of the theories embodied in programmes enhances an evaluation while being a `broker of compromise' can limit access to information. Researchers have a valuable role in promoting learning. A robust evaluation framework integrates strategies for generalizing at the outset and identifying mechanisms of change or causal mechanism is a way forward. Examples are taken from recent evaluations conducted by the author and colleagues to illustrate the arguments.
Ahmad, E., R. Knight, A. Parmar, A. Sampson and S. Themelis (2003) `YIP Youth Work', proceedings from a seminar, 21 June 2003. Centre for Institutional Studies, University of East London.
2.
Brookes, N. (2002) `Monitoring Performance: A Survey of Satisfaction with Family Support Services amongst the Sure Start Target Population', Barts and The London Queen Mary's School of Medicine and Dentistry, unpublished manuscript.
3.
Brown, P. (1995) `The Role of the Evaluator in Comprehensive Community Initiatives', in J. Connell, A. Kubisch, L. Schorr and C. Weiss (eds) New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts, pp. 201—25. Washington, DC: Aspen Institute.
4.
Brown, S. (1998) `Youth Works: An Evaluation', Faculty of Law, University of Sheffield, UK, unpublished manuscript.
5.
Burgess, T. (2000) `The Logic of Learning and its Implications for High Education', Higher Education Review32(2): 53—65.
6.
Burgess, T. (2002) `Towards a Social Science: A Comment on Karl Popper's Twenty Seven Theses', paper presented at the Karl Popper Centenary Congress, University of Vienna, Austria, 3—7 July.
Chen, H.-T. and P. Rossi (1992) `Introduction: Integrating Theory into Evaluation Practice', in H.-T. Chen and P. Rossi (eds) Using Theory to Improve Program and Policy Evaluations, pp. 1—11. New York: Greenwood Press.
9.
Connell, J. and A. Kubisch (1999) Applying a Theory of Change Approach to the Evaluation of Comprehensive Community Initiatives: Progress, Prospects, and Problems . Available at http://aspenroundtable.org/vol2/connell/htm (accessed 15 June 2002).
10.
Connell, J., A. Kubisch, A. Schorr and C. Weiss (1995) New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts. Washington, DC: Aspen Institute.
Fearnley, R. and N. McInroy (2000) `Agents of Change in East London: Reflections on Conducting Urban Regeneration Research', in T. Butler (ed.) Eastern Promise: Education and Social Renewal in London's Docklands, pp. 210—25. London: Lawrence & Wishart.
13.
Foot Whyte, W., ed. (1991) Participatory Action Research. London: SAGE.
14.
Griffith, J. and A. Sampson (2002) `Using and Developing the Popperian Approach to Evaluations', paper presented at Karl Popper Centenary Congress, University of Vienna, Austria, 3—7 July.
15.
Israel, B., S. Schurman and M. Hugentobler (1992) `Conducting Action Research: Relationships between Organisation Members and Researchers', Journal of Applied Behavioural Science28(1): 74—101.
16.
Jarvie, I. (1972) Concepts and Society. London : Routledge & Kegan Paul.
17.
Judge, K. and L. Bauld (2001) `Strong Theory, Flexible Methods: Evaluating Complex Community-Based Initiatives', Critical Public Health11(1): 19—38.
18.
Kubisch, A., K. Fulbright-Anderson and J. Connell (1995) `Evaluating Community Initiatives: A Progress Report '. Available at: http://www.aspenroundtable.org/vol2/kubisch.htm (accessed June, 2002).
19.
Lincoln, L. (2001) `Engaging Sympathies: Relationships between Action Research and Social Constructivism', in P. Reason and H. Bradbury (eds) Handbook of Action Research, pp. 124—32. London: SAGE.
20.
Mackenzie, M. and A. Blamey (2005) `The Practice and the Theory: Lessons from the Application of a Theories of Change Approach', Evaluation11(2): 151—68.
21.
McQuail, L. and V. Wigfall (2001) `Euston Sure Start: A Needs Survey'. Available at: http://www.ness.bbk.ac.uk (accessed 20 February 2003).
22.
Majone, G. (1980) `Policies as Theories', Omega: The International Journal of Management Science8(2): 151—62.
23.
Marshall, J. (2001) `Self-Reflective Inquiry Practices', in P. Reason and H. Badbury (eds) Handbook of Action Research, pp. 433—9. London: SAGE.
24.
Morgan Harris Burrows (2003) `Evaluation of the Youth Inclusion Programme'. Available at http://www.youth-justice-board.gov.uk (accessed 30 November 2003).
25.
National Statistics Online (2003) ` Neighbourhood Statistics'. Available at: http://www.neighbourhood.statistics.gov.uk (accessed 3 February 2003).
26.
Nutley, S., I. Walter and H. Davies (2003) `From Knowing to Doing: A Framework for Understanding the Evidence-Into-Practice Agenda', Evaluation9(2): 125—48.
27.
Oakley, A. (2000) Experiments in Knowing. Cambridge : Polity Press.
28.
Parmar, A. and A. Sampson (2007) `Evaluating Domestic Violence Initiatives: Transferring Practice Principles Not Projects', British Journal of Criminology47(4): 671—91.
29.
Parmar, A., A. Sampson and P. Jordan (2003) `Final Site Report of Croydon Domestic Violence Advocacy Project', submitted to the Home Office, unpublished manuscript.
30.
Pawson, R. and N. Tilley (1997) RealisticEvaluation. London: SAGE.
31.
Popper, K. (1968) The Logic of Scientific Discovery, 2nd edn. London: Hutchinson .
32.
Popper, K. (1969) Conjectures and Refutations: The Growth of Scientific Knowledge, 3rd edn. London: Routledge & Kegan Paul.
33.
Reason, P. and H. Bradbury , eds (2001) Handbook of Action Research. London: SAGE.
34.
Sayer, A. (1992) Method in Social Science: A Realist Approach, 2nd edn. London: Routledge .
35.
Weiss, C. (1972) Evaluation Research. Englewood Cliffs, NJ: Prentice-Hall.
36.
Weiss, C. (1995) `Nothing as Practical as Good Theory: Exploring Theory-Based Evaluation for Comprehensive Community Initiatives for Children and Families', in J. Connell, A. Kubisch, L. Schorr and C. Weiss (eds) New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts, pp. 65—92. Washington, DC: Aspen Institute.
37.
Weiss, C. (1997a) `How can Theory-Based Evaluation Make Greater Headway?', Evaluation Review21(4): 501—524.
38.
Weiss, C. (1997b) `Theory-Based Evaluation: Past, Present and Future ', New Directions for Evaluation76: 41—55.
