Theory-based evaluation approaches are becoming increasingly popular in the evaluation of comprehensive community initiatives. Such an approach, Theory of Change (ToC), has been used in the external evaluations of two of four Scottish Health Demonstration projects. This article provides an empirically based analysis of the issues raised in the process of articulating a project’s ToC. It then considers the value of the approach in relation to sharpening planning, providing formative feedback, improving performance management, guiding internal and external evaluation, judging impact and reducing problems of attribution. A conclusion is reached that, whilst such approaches are by no means a panacea, they offer one useful framework within which to negotiate a range of evaluation practice decisions.
Accounts Commission for Scotland (1998) The Measures of Success: Developing a Balanced Scorecard. Edinburgh: Audit Scotland . http://www.audit-scotland.gov.uk.
2.
Accounts Commission for Scotland (2000) The Map to Success: Using Process Mapping to Improve Performance. Edinburgh: Audit Scotland . http://www.audit-scotland.gov.uk.
3.
Barnes, M. , E. Matka and H. Sullivan (2003) ‘Evidence, Understanding and Complexity’ , Evaluation9(3): 265-284 .
4.
Blalock, A. B. (1999) ‘Evaluation Research and the Performance Management Movement: From Estrangement to Useful Integration’ , Evaluation5(2): 117-149 .
5.
Brown, P. (1998) ‘Shaping the Evaluator’s Role in a Theory of Change Evaluation’, in A. Fulbright-Anderson , A. C. Kubisch and J. P. Connell (eds) New Approaches to Evaluating Community Initiatives: Theory, Measurement and Analysis, pp. 101-111. Washington, DC: Aspen Institute .
6.
Cole, G. E. (1999) ‘Advancing the Development and Application of Theory-based Evaluation in the Practice of Public Health’ , American Journal of Evaluation20(3): 453-470 .
7.
Connell, J. P. and A. C. Kubisch (1998) ‘Applying a Theory of Change Approach to the Evaluation of Comprehensive Community Initiatives: Progress, Prospects and Problems’, in A. Fulbright-Anderson , A. C. Kubisch and J. P. Connell (eds) New Approaches to Evaluating Community Initiatives: Theory, Measurement and Analysis, pp. 15-44. Washington, DC: Aspen Institute .
8.
Donaldson, S. I. and L. E. Gooler (2003) ‘Theory-Driven Evaluation in Action: Lessons from a $20 million Statewide Work and Health Initiative’ , Evaluation and Program Planning26(4): 355-366 .
9.
Dunn, A. L. , B. H. Marcus , J. B. Kampert , M. E. Garcia , H. W. Khol and S. N. Blair (1999) ‘Comparison of Lifestyle and Structured Interventions to Increase Physical Activity and Cardiorespiratory Fitness’ , Journal of the American Medical Association281(4): 327 .
10.
Fulbright-Anderson, A. , A. C. Kubisch and J. P. Connell , eds (1998) New Approaches to Evaluating Community Initiatives: Theory, Measurement and Analysis. Washington, DC: Aspen Institute .
11.
Herbert, S. and A. Anderson (1998) ‘Applying the ToC Approach to Two National, Multisite Comprehensive Community Initiatives’, in A. Fulbright-Anderson , A. C. Kubisch and J. P. Connell (eds) New Approaches to Evaluating Community Initiatives: Theory, Measurement and Analysis, pp. 123-148. Washington, DC: Aspen Institute .
12.
Kaplan, R. S. and D. P. Norton (1996) The Balanced Scorecard. Boston, MA: Harvard Business School Press .
13.
Lewis, J. (2001) ‘Reflections on Evaluation in Practice’ , Evaluation7(3): 387-394 .
14.
Pawson, R. and N. Tilley (1997) Realistic Evaluation. London: SAGE .
15.
Rush, B. and A. Ogborne (1991) ‘Program Logic Models: Expanding their Role and Structure for Program Planning and Evaluation’ , Canadian Journal of Program Evaluation6(2): 95-106 .
16.
Sanderson, I. (2000) ‘Evaluation in Complex Policy Systems’ , Evaluation6(4): 433-454 .
17.
Shaw, A. , A. McMunn and J. Field , eds (2000) The Scottish Health Survey, carried out on behalf of the Scottish Executive Dept of Health. London: Joint Health Surveys Unit, at the National Centre for Social Research and Dept of Epidemiology and Public Health .
18.
Sullivan, H. , M. Barnes and E. Matka (2002) ‘Building Collaborative Capacity through Theory of Change: Early Lessons from the Evaluation of Health Action Zones in England’ , Evaluation8(2): 205-266 .
19.
Susser, M. (1995) ‘Editorial: The Tribulations of Trials. Interventions in Communities’ , American Journal of Public Health85: 156-158 .
20.
Walker, R. (2001) ‘Great Expectations: Can Social Science Evaluate New Labour’s Policies?’ , Evaluation7(3): 305-330 .
21.
Weitzman, B. C. , D. Silver and K.-N. Dillman (2002) ‘Integrating a Comparison Group Design into a Theory of Change Evaluation: The Case of the Urban Health Initiative’ , American Journal of Evaluation23(4): 371-385 .
22.
Wimbush, E. and J. Watson (2000) ‘An Evaluation Framework for Health Promotion: Theory, Quality and Effectiveness’ , Evaluation6(3): 301-321 .
