This article examines the evolution of Ray Pawson’s realist theory of evaluation, with a particular focus on his most recent book, Science of Evaluation: A Realist Manifesto. It is not a substitute for reading the original text. Reviews always say less about a book than the book in question. The goal is to provide a broader context for interpretation and an invitation to consider critically the practical import of Pawson’s grand ambitions for a new evaluation science. Like previous writings, this latest call-to-arms will appeal to some quarters of the evaluation community and dismay others. Regardless, evaluators should not remain indifferent. Pawson presents a vigorous, if at times irreverent, argument for advancing evaluation as a realist enterprise. Science of Evaluation deserves to be read and discussed widely.
AlkinM (ed.) (2013) Evaluation Roots: A Wider Perspective of Theorists’ Views and Influences, 2nd edn.Thousand Oaks, CA: SAGE.
2.
BhaskarR (1975) A Realist Theory of Science. Brighton: Harvester Press.
3.
BhaskarR (1979) The Possibility of Naturalism: A Philosophical Critique of the Contemporary Human Sciences. Brighton: Harvester Press.
4.
BlameyAMackenzieM (2007) Theories of change and realistic evaluation: peas in a pod or apples and oranges?Evaluation13(4): 439–55.
5.
BloorD (1976) Knowledge and Social Imagery. Chicago, IL: University of Chicago Press.
6.
ByngRNormanIRedfernSJonesR (2008) Exposing the key functions of a complex intervention for shared care in mental health: case study of a process evaluation. BMC Health Service Research8: 274.
7.
CampbellDT (1966) Pattern matching as an essential in distal knowing. In HammondKR (ed.), The psychology of Egon Brunswik. Austin, TX: Holt, Rinehart, and Winston, 81–106.
8.
ChelimskyE (1997) The coming transformations in evaluation. In: ChelimskyEShadishWR (eds), Evaluation for the 21st Century: A Handbook. Thousand Oaks, CA: SAGE, 1–26.
9.
ChelimskyEShadishWR (eds) (1997) Evaluation for the 21st Century: A Handbook. Thousand Oaks, CA: SAGE.
10.
ChenH-T (1990) Theory-Driven Evaluations. Newbury Park, CA: SAGE.
11.
ChenH-T (2005) Practical Program Evaluation: Assessing and Improving Planning, Implementation, and Effectiveness. Thousand Oaks, CA: SAGE.
12.
ChenH-TTurnerNC (2012) Formal theory versus stakeholder theory: new insights from a tobacco-focused prevention program evaluation. American Journal of Evaluation33(3): 395–413.
13.
CookTDCampbellDT (1979) Quasi-Experimentation: Design and Analysis for Field Settings. Chicago, IL: Rand McNally.
14.
CorynCLSNoakesLAWestineCDSchröterDC (2011) A systematic review of theory-driven evaluation practice: from 1990 to 2009. American Journal of Evaluation32(2): 199–226.
15.
CronbachLJ (1975) Beyond the two disciplines of scientific psychology. American Psychologist30(2): 116–27.
16.
CronbachLJ (1982) Designing Evaluations of Educational and Social Programs. San Francisco, CA: Jossey-Bass.
17.
CronbachLJ and Associates (1980) Toward Reform of Program Evaluation. San Francisco, CA: Jossey-Bass.
18.
DavisP (2005) The limits of realist evaluation: surfacing and exploring assumptions in assessing the best value performance regime. Evaluation11(3): 275–95.
19.
DonaldsonSI (2007) Program Theory-Driven Evaluation Science: Strategies and Applications. Mahwah, NJ: Erlbaum.
20.
DuguidSPawsonR (1998) Education, change, and transformation: the prison experience. Evaluation Review22(4): 470–95.
21.
DunnWN (1982) Reforms as arguments. Knowledge: Creation, Diffusion, Utilization3: 293–326.
22.
DunnWN (1997) Pragmatic eliminative induction: proximal range and context validation in applied social experimentation. Philosophica60: 75–112.
23.
ElsworthGR (1994) Arguing challenges to validity in field research: a realist perspective. Knowledge: Creation, Diffusion, Utilization15(3): 321–43.
GillMTurbinV (1999) Evaluating ‘Realistic Evaluation’: evidence from a study of CCTV. In: PainterKTilleyN (eds), Surveillance of Public Space: CCTV, Street Lighting and Crime Prevention. New York: Criminal Justice Press, 179–200.
26.
GreenerIMannionR (2009) A realistic evaluation of practice-based commissioning. Policy and Politics37(1): 57–73.
27.
GreenhalghTHumphreyCHughesJMacfarlaneFButlerCPawsonR (2009) How do you modernize a health service? A realist evaluation of whole-scale transformation in London. The Millbank Quarterly87(2): 391–416.
28.
GubaEGLincolnYS (1989) Fourth Generation Evaluation. Newbury Park, CA: SAGE.
29.
HarreR (1972) The Philosophies of Science. Oxford: Oxford University Press.
30.
HesseMB (1974) The Structure of Scientific Inference. London: Macmillan.
31.
HouseER (1991) Realism in research. Educational Researcher20(6): 2–9 and 25.
32.
JulnesGMarkMMHenryGT (1998) Promoting realism in evaluation: realistic evaluation and the broader context. Evaluation4(4): 483–506.
33.
LemireSTNielsenSBDybdalL (2012) Making contribution analysis work: a practical framework for handling influencing factors and alternative explanations. Evaluation18(3): 294–309.
34.
LockeJ (1996 [1690]) An Essay Concerning Human Understanding. Abridged and edited by WinklerK.Indianapolis, IN: Hackett Publishing Company.
35.
MarkMM (1990) From program theory to tests of program theory. In: BickmanL (ed.), Advances In Program Theory. New directions for program evaluation, No. 47, San Francisco, CA: Jossey-Bass, 37–51.
36.
MarkMMHenryGTJulnesG (2000) Evaluation: An Integrated Framework for Understanding, Guiding, and Improving Policies and Programs. San Francisco, CA: Jossey-Bass.
37.
MartinsonR (1974) What works? Questions and answers about prison reform. The Public Interest35: 22–54.
38.
MertonRK (1965) On the Shoulders Of Giants: A Shandean Postscript. New York: Free Press.
39.
MertonRK (1968) Social Theory and Social Structure, 3rd edn.New York: Free Press.
40.
MillerRLCampbellR (2006) Taking stock of empowerment evaluation: an empirical review. American Journal of Evaluation27(3): 296–319.
41.
MonaghanM (2012) An interview with Ray Pawson. Evidence & Policy8(4): 511–15.
42.
PawsonR (1978) Empiricist explanatory strategies: the case of causal modelling. Sociological Review26: 613–45.
43.
PawsonR (1980) Empiricist measurement strategies: a critique of the multiple indicator approach to measurement. Quality and Quantity14(5): 651–78.
44.
PawsonR (1982) Desperate measures. British Journal of Sociology33(1): 35–63.
45.
PawsonR (1983) Language and measurement: a reply to Coxon. British Journal of Sociology34(4): 491–7.
46.
PawsonR (1986) On the level: measurement scales and sociological theory. Bulletin of Sociological Methodology11(1): 49–82.
47.
PawsonR (1989) A Measure for Measures: A Manifesto for Empirical Sociology. London: Routledge.
48.
PawsonR (2006) Evidence-Based Policy: A Realist Perspective. London: SAGE.
49.
PawsonR (2008) Invisible mechanisms. Evaluation Journal of Australasia8(2): 3–13.
50.
PawsonR (2010) Middle range theory and program theory evaluation: from provenance to practice. In: VaessenJLeeuwFL (eds), Mind the Gap: Perspectives on Policy Evaluation and the Social Sciences. Comparative Policy Evaluation, Vol. 16. London: Transaction Publishers, 171–203.
51.
PawsonR (2011) From the library of…Evaluation17(2): 193–6.
52.
PawsonRManzano-SantaellaA (2012) A realist diagnostic workshop. Evaluation18(2): 176–91.
53.
PawsonRTilleyN (1982) Monstrous thoughts: weaknesses in the strong programme of the sociology of science. University of Leeds Occasional Papers in Sociology, No. 14.
54.
PawsonRTilleyN (1992) Re-evaluation: rethinking research on corrections and crime. In: DuguidS (ed.), Yearbook of Correctional Education. Burnaby, BC. Institute for the Humanities, Simon Fraser University, 19–49.
55.
PawsonRTilleyN (1994) What works in evaluation research?British Journal of Criminology34(3): 291–306.
PawsonRWongGOwenL (2011) Known knowns, known unknowns, unknown unknowns: the predicament of evidence-based policy. American Journal of Evaluation32(4): 518–46.
60.
PedersenLHRieperO (2008) Is realist evaluation a realistic approach for complex reforms?Evaluation14(3): 271–93.
61.
PopperK (1976) Unended Quest: An Intellectual Biography. London: Fontana.
62.
Rycroft-MaloneJFontenlaMBickDSeersK (2010) A realistic evaluation: the case of protocol-based care. Implementation Science5: 38.
63.
ScrivenM (1994) The fine line between evaluation and explanation. Evaluation Practice15(1): 75–7.
64.
ScrivenM (1998) Minimalist theory: the least theory that practice requires. American Journal of Evaluation19(1): 57–70.
65.
ShadishWR (1995) The logic of generalization: five principles common to experiments and ethnographies. American Journal of Community Psychology23(3): 419–28.
66.
ShadishWRCookTDCampbellDT (2002) Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin.
67.
ShadishWRCookTDLevitonLC (1991) Foundations of Program Evaluation: Theories of Practice. Newbury Park, CA: SAGE.
TilleyN (1993) Understanding car parks, crime and CCTV: evaluation lessons from Safer Cities. Crime Prevention Unit Paper 42, London: Home Office.
70.
TilleyN (2000) Experimentation and criminal justice policies in the United Kingdom. Crime & Delinquency46(2): 194–213.
71.
TrevisanMS (2007) Evaluability assessment from 1986 to 2006. American Journal of Evaluation28(3): 290–303.
72.
WeissCH (1995) Nothing as practical as good theory. In: ConnellJKubischASchorrLBWeissCH (eds), New Approaches to Evaluating Community Initiatives. New York: Aspen Institute.
73.
WeissCH (1997a) Theory-based evaluation: past, present, and future. In: RogDJFournierD (eds), Progress and Future Directions in Evaluation: Perspectives on Theory, Practice and Methods. New directions for evaluation, No. 76. San Francisco, CA: Jossey-Bass, 55–41.
74.
WeissCH (1997b) How can theory-based evaluation make greater headway?Evaluation Review21(4): 501–24.
75.
WeissCH (2004) Rooting for evaluation: a cliff notes version of my work. In: AlkinMC (ed.), Evaluation Roots: Tracing Theorists’ Views and Influences. Thousand Oaks, CA: SAGE, 153–68.
