Evaluation: Drawing comparisons or achieving consensus?

Abstract

This paper looks at evaluation: what it is, how it has developed, its status in research and the questions it is expected to answer. Evaluation has had an eventful history, with evaluators from different disciplines taking different approaches and arguing strongly from the standpoint of different epistemologies. The need for evaluation continues to grow given that allocation of scarce resources to the most worthwhile high utility health and social programmes is a priority.

I draw from models described by scholars who argue from quite different positions on what the focus for evaluation should be, ranging from experimental through judgemental, pragmatic, responsive constructivist, pluralist to realistic evaluation approaches. At one extreme, responsive constructivist evaluation argues that an alternative form of disciplined inquiry to conventional experimental evaluation is needed. Others are also critical of the experimental approach but reject an exclusively naturalistic model in favour of evaluation which incorporates elements of other approaches and puts context and process as well as outcome high on the agenda.

By way of illustration I apply the models to an evaluation project currently in progress in an attempt to identify the advantages and disadvantages of each approach and to answer the question posed by the title. The models are interrogated with four questions in mind: whether, why, for whom and under what circumstances a pragramme/treatment/intervention works. The conclusion reached is that realistic evaluation has the greatest capacity to answer these questions when applied to complex health and social care organisations.