The craft of evaluative practice: Negotiating legitimate methodologies within complex interventions

Abstract

Evaluations of complex interventions are likely to encounter tensions between different methodological principles, and between the inherent causal rationality of evaluation and the messy complexity of real institutional contexts. Conceptualizing evaluation as producing putatively authoritative evidence, we show how ‘legitimacy’ is a useful concept for unpacking evaluation design in practice. A case study of service integration shows how different approaches may have unpredictable levels of legitimacy, based in contrasting assessments of their methodological acceptability and actual utility. Through showing how practitioners resolved the tensions, we suggest that crafting a patchwork of different methodologies may be legitimate and effective, and can be seen as underpinned by its own pragmatic rationality. However, we also conclude that the explanatory power of theory-driven evaluation can be embedded in such an approach, both in elements of the patchwork and as an overarching guiding principle for the crafting process.