This article explores the usefulness of Contribution Analysis to understand and assess the outcomes and impacts of emergent, large-scale and transformational change processes. As these take place over long periods of time and at national if not supra-national scale, they amplify some of the key challenges complex interventions pose to evaluators, most notably that of the emergence of activities and outcomes. While Contribution Analysis is discussed in the literature as useful for the evaluation of complex change processes, it is not usually recommended for experimental and emergent interventions. This article draws on the authors’ experience of using Contribution Analysis in one such case and explores the relative merits of the method for large-scale/transformational change interventions generally.
Barbrook-JohnsonP (2019) Negotiating complexity in evaluation planning: A participatory systems map of the energy trilemma. CECAN Evaluation and Policy Practice Note (EPPN) no. 12 for policy analysts and evaluators. Available at: http://epubs.surrey.ac.uk/id/eprint/850627
2.
BefaniBStedman-BryceG (2016) Process tracing and Bayesian updating for impact evaluation. Evaluation23(1): 42–60.
3.
BlameyAMackenzieM (2007) Theories of change and realistic evaluation: Peas in a pod or apples and oranges?Evaluation13(4): 439–55.
4.
BrandLQuinnCPearsonM, et al. (2019) Building programme theory to develop more adaptable and scalable complex interventions: Realist formative process evaluation prior to full trial. Evaluation25(2): 149–70.
5.
BurnsD (2014) Assessing impact in dynamic and complex environments:Systemic action research and participatory systemic inquiry. CDI practice paper 8, 1 September. Brighton: IDS.
6.
ByrneD (2013) Evaluating complex social interventions in a complex world. Evaluation19: 217–28.
DelahaisTToulemondeJ (2012) Applying contribution analysis: Lessons from five years of practice. Evaluation18(3): 281–93.
9.
DybdalLNielsenSBLemireS (2010) Contribution analysis applied: Reflections on scope and methodology. The Canadian Journal of Program Evaluation25(2): 29–57.
10.
EmeryFTristE (1963) The causal texture of organizational environments. Human Relations18(1): 21–32.
11.
FeinsteinO (2019) Dynamic evaluation for transformational change. In: van den BergRDMagroCMulderSS (eds) Evaluation for Transformational Change: Opportunities and Challenges for the Sustainable Development Goals. Exeter: International Development Evaluation Association, 17–31.
12.
GeertzC (1973) The Interpretation of Cultures. New York: Basic Books.
KemmisSMcTaggartR (2007) Participatory action research: Communicative action and the public sphere. In: DenzinNKLincolnYS (eds) Strategies of Qualitative Enquiry. Thousand Oaks: SAGE, 271–330.
15.
KolerosAMayneJ (2019) Using actor-based theories of change to conduct robust evaluation in complex settings. Canadian Journal of Program Evaluation33(3): 292–315.
16.
LarsonA (2018) Evaluation amidst complexity: Eight evaluation questions to explain how complex adaptive systems affect program impact. Evaluation24(3): 353–62.
17.
LemireSTNielsonSBDybdalL (2012) Making contribution analysis work: A practical framework for handling influencing factors and alternative explanations. Evaluation18(3): 294–308.
18.
LingT (2012) Evaluating complex and unfolding interventions in real time. Evaluation18(1): 79–91.
19.
MagroCvan den BergRD (2019) Systems evaluations for transformational change: challenges and opportunities. In: van den BergRDMagroCMulderSS (eds) Evaluation for Transformational Change: Opportunities and Challenges for the Sustainable Development Goals. Exeter: International Development Evaluation Association, 131–152.
RamalingamBJonesHToussaintR, et al. (2008) Exploring the science of complexity: Ideas and implications for development and humanitarian efforts. Working paper 285. London: Overseas Development Institute.
28.
RogersP (2008) Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation14(1): 29–48.
