Originated in the 1970s, decried in the 1990s, theory-based evaluation recaptured debate at the European Evaluation Society conference in 2002. The promise is that the set-up, implementation, delivery and utilization of evaluation research may be facilitated by taking the assumptions and objectives of public policy as a starting point. In addition, a theory-based approach is considered to be valuable for a government that wants to learn, whilst potentially reconciling positivist and constructivist approaches. This article explores the rationale for a theory-based approach in policy development, debate, learning and evaluation utilization. It does so by confronting the–often paradoxical–dilemmas that haunt every evaluator (and policy maker). How can we do justice to societal complexity yet still maintain focus? How can we enter apparently closed policy systems? And how do we find a balance between mere improvement and innovative learning?
Get full access to this article
View all access options for this article.
References
1.
Abma, T. A. (1996) Responsief evalueren–Discoursen, controversen en allianties in het postmoderne [Responsive Evaluation–Discussions, Controversies and Alliances in the Postmodern]. Delft: Eburon .
2.
Argyris, C. (1990) Overcoming Organizational Defenses, Facilitating Organizational Learning. Needham Heights, MA: Allyn and Bacon .
3.
Argyris, C. and D. A. Schön (1978) Organizational Learning: A Theory for Action Perspective. Reading, MA: Addison-Wesley .
4.
Festinger, L. (1957) A Theory of Cognitive Dissonance. Stanford, CA: Stanford University Press .
5.
Fischer, F. (1980) Politics, Values and Public Policy: The Problem of Methodology. Boulder, CO: Westview Press .
6.
Guba, E. G. (1990) The Paradigm Dialogue. London: Sage .
7.
Guba, E. G. and Y. S. Lincoln (1989) Fourth Generation Evaluation. Newbury Park, CA: Sage .
8.
Hoogerwerf, A. (1984) ‘Beleid berust op veronderstellingen: De beleidstheorie’ [‘Policy is Based on Assumptions: The Policy Theory’] , Acta Politica14(4): 493–531 .
9.
Hoppe, R. (1998) ‘De broosheid van debat en argumentatieve beleidsanalyse’ [‘The Fragility of Debate and Argumentative Policy Analysis’], inaugural address delivered upon his appointment as professor of policy science. Twente University, Enschede .
10.
In ‘t, Veld, R. J., L. Schaap, C. J. A. M. Termeer and M. J. W. van Twist, eds (1991) Autopoiesis and Configuration Theory: New Approaches to Societal Steering. Dordrecht: Kluwer Academic .
11.
Lam, T. C. M. (2002) ‘Theory-Based Evaluation and Objective-Based Evaluation: An Integration of the Two Approaches’ , paper presented at the European Evaluation Society conference, Seville, Spain, 12 October.
12.
Leeuw, F. L. (1983) Overheid en bevolkingsgroei: een evaluatie van beleidstheorieën [Government and Population Growth: An Evaluation of Policy Theories]. Leyden: University of Leyden .
13.
Leeuw, F. L. (2002) ‘Evaluating and the Ranking of Higher Education Studies’, in J. A. A. M. Bevers and M. J. F. Hulshof (eds) Willems en wetens. Nijmegen: IOWO .
14.
Lindblom, C. E. and E. J. Woodhouse (1993) The Policy-Making Process, 3rd edn. Englewood Cliffs, NJ: Prentice-Hall .
15.
Mabry, L. (2002) ‘Postmodern Evaluation–or Not?’ , American Journal of Evaluation23(2): 141–158 .
16.
Majone, G. (1989) Evidence, Argument and Persuasion in the Policy Process. New Haven, CT: Yale University Press .
17.
March, J. G. and J. P. Olsen (1989) Rediscovering Institutions: The Organizational Basis of Politics. New York: Free Press .
18.
Miles, M. B. and A. M. Huberman (1988) ‘Drawing Valid Meaning from Qualitative Data: Toward a Shared Craft’, in D. M. Fetterman (ed.) Qualitative Approaches to Evaluation in Education: The Silent Scientific Revolution. New York: Praeger .
19.
Ministry of Finance (2001) Regeling prestatiegegevens en evaluatieonderzoek Rijksoverheid [Central Government Performance Data and Evaluation Research Arrangements]. The Hague: Ministry of Finance .
20.
Ministry of Finance (2002) Evaluatieoverzicht Rijksoverheid 2003 [Central Government Evaluation Overview 2003]. The Hague: Ministry of Finance .
21.
Patton, M. Q. (1996) ‘A World Larger than Formative and Summative’ , Evaluation Practice17(2): 131–144 .
Patton, M. Q. (2002) Qualitative Research and Evaluation Methods. Thousand Oaks, CA: Sage .
24.
Pawson, R. (2002) ‘Nothing as Practical as a Good Theory’ , plenary address presented at the European Evaluation Society conference, Seville, Spain: 12 October.
25.
Pawson, R. and N. Tilley (1997) Realist Evaluation. London: Sage .
26.
Popper, K. R. (1972) Objective Knowledge. Oxford: Oxford University Press .
27.
Rein, M. and D. A. Schön (1993) ‘Reframing Policy Discourse’, in F. Fisher and J. Forester (eds) The Argumentative Turn in Policy Analysis and Planning. London: UCL Press .
28.
Sabatier, P. A. and H. C. Jenkins-Smith (1993) Policy Change and Learning–An Advocacy Coalition Approach. Boulder, CO: Westview Press .
29.
Schön, D. A. (1982) The Reflective Practitioner. Cambridge, MA: Basic Books .
30.
Schwandt, T. A. (2001) Evaluation Practice Reconsidered. New York: Peter Lang .
31.
Schwandt, T. A. (2002) ‘Democratization, Globalization and Performativity: Implications for Learning in and from Evaluation’ , keynote address from the European Evaluation Society conference, Seville, Spain, 12 October.
32.
Shaw, I. F. (1999) Qualitative Evaluation. London: Sage .
33.
Simon, H. A. (1957) Models of Man. New York: Wiley .
34.
Swanborn, P. G. (1999) Evalueren [Evaluating]. Amsterdam: Boom .
35.
Taylor, S. E. and J. Crocker (1981) ‘Schematic Bases of Social Information Processing’, in E. T. Higgins, C. P. Herman and M. P. Zanna (eds) Social Cognition. Hillsdale, NJ: Lawrence Erlbaum .
36.
Teisman, G. R., F. B. van der Meer, E. H. Klijn, J. Edelenbos, H. L. Klaassen and M. A. Reudink (2002) Evalueren om te leren–naar een evaluatiearrangement voor de Vijfde Nota RO [Evaluation to Learn–Towards an Evaluation Arrangement for the Fifth Policy Document on Spatial Planning]. Rotterdam: Erasmus University .
37.
UK Treasury (2002) 2002 Spending Review–Public Service Agreements 2003–2006. London: HM Treasury .
38.
Van der Knaap, P. (1995) ‘Policy Evaluation and Learning–Feedback, Enlightenment or Argumentation’ , Evaluation1(2): 193–220 .
39.
Van der Knaap, P. (1997) Lerende overheid, intelligent beleid–De lessen van beleidsevaluatie en beleidsadvisering voor de structuurfondsen van de Europese Unie [Learning Government, Intelligent Policy–The Lessons of Evaluation and Policy Advice for the Structural Funds]. The Hague: Phaedrus .
40.
Van der Knaap, P. (2000) ‘Performance Management and Policy Evaluation in the Netherlands: Towards an Integrated Approach’ , Evaluation6(3): 335–350 .
41.
Verweij, A. O. (1995) Een kwestie van organiseren, De invoering van een etnische registratie in arbeidsorganisaties [A Question of Organizing, The Introduction of an Ethnic Registration in Labour Organizations]. Rotterdam: Institute for Socio-Economic Research .
42.
Weick, K. E. (1979) ‘Cognitive Processes in Organizations’ , Research in Organizational Behavior1: 41–74 .
43.
Weiss, C. H. (1990) ‘Evaluation Research for Decisions: Is Anybody There? Does Anybody Care?’, in M. C. Alkin (ed.) Debates on Evaluation, pp. 171–184. Newbury Park, CA: Sage . Reprinted from Evaluation Practice 9(1).
44.
Weiss, C. H. (2001) ‘What Kind of Evidence in Evidence-based Policy?’ , paper presented at the Third International Evidence-Based Policies and Indicator Systems conference, Durham, UK, 4–7 July. Available at http://www.cemcentre.org/eb2003/ (site visited 05/12/03).
45.
Wildavsky, A. (1987) Speaking Truth to Power–The Art and Craft of Policy Analysis. New Brunswick, NJ: Transaction Books .
