In recent years, evaluation research has taken an increasing interest in the proliferation of evaluation systems. One of the main assumptions being that such evaluation systems can enhance the use of evaluations and their findings. The current article analyses this assumption by empirically studying the workings of a specific evaluation system within the Danish Ministry of Employment. By applying the concept of constitutive effects, the article analyses how the evaluation system affects processes of policymaking within the central government. It is shown how the institutionalisation of an evidence-based evaluation system, at the ministerial level, structures the process of policymaking around a specific understanding of policy content, timeframes and expertise. Based on these findings, it is argued that evaluation systems can both enhance the instrumental use of evaluations and significantly alter policymaking processes.
AhonenP (2015) Aspects of the institutionalization of evaluation in Finland: Basic, agency, process and change. Evaluation21(3): 308–24.
2.
AlkinMCKingJA (2016) The historical development of evaluation use. American Journal of Evaluation37(4): 568–79.
3.
AlkinMCKingJA (2017) Definitions of evaluation use and misuse, evaluation influence, and factors affecting use. American Journal of Evaluation38(3): 434–50.
4.
AlkinMCKingJA (2018) The centrality of use: Theories of evaluation use and influence and thoughts on the first 50 years of use research. American Journal of Evaluation40(3): 431–58.
5.
AndersenNARandrupAG (2017) Evidensbaseret Politikudvikling: Viden som oplysning eller indskrænkning af beskæftigelsespolitikken?Tidsskrift for Arbejdsliv19(2): 41–56.
6.
BarbierJCLudwig-MayerhoferW (2004) The many worlds of activation. European Societies6(4): 423–36.
BjørnholtBLarsenF (2014) The politics of performance measurement: ‘Evaluation use as mediator for politics’. Evaluation20(4): 400–11.
10.
BourgeoisICousinsJB (2013) Understanding dimensions of organizational evaluation capacity. American Journal of Evaluation34(3): 299–319.
11.
BrodkinEZ (2011) Policy work: Street-level organizations under new managerialism. Journal of Public Administration Research and Theory21(2): 253–77.
12.
BrodkinEZMarstonG (2013) Work and the Welfare State: Street-Level Organizations and Workfare Politics. Washington, DC: Georgetown University Press.
13.
CorydonB (2015) Ministerens spørgetid – 59. møde onsdag den 25 februar 2015. Copenhagen: Folketingstidende.
14.
CousinsJBLeithwoodKA (1986) Current empirical research on evaluation utilization. Review of Educational Research56(3): 331–64.
15.
Dahler-LarsenP (2012a) Constitutive effects as a social accomplishment: A qualitative study of the political in testing. Education Inquiry3(2): 171–86.
16.
Dahler-LarsenP (2012b) The Evaluation Society. Stanford, CA: Stanford University Press.
17.
Dahler-LarsenP (2014) Constitutive effects of performance indicators: Getting beyond unintended consequences. Public Management Review16(7): 969–86.
18.
Dahler-LarsenP (2016) Evalueringsanvendelse. In: BredgaardT (ed.) Evaluering af offentlig politik og administration. Copenhagen: Hans Reitzels Forlag, 195–214.
19.
DuedahlPJacobsenMH (2010) Introduktion til dokumentanalyse – Metodeserie for Social- og Sundhedsvidenskaberne, Bind 2. Odense: Syddansk Universitetsforlag.
20.
DunlopCARadaelliCM (2013) Systematising policy learning: From monolith to dimensions. Political Studies61(3): 599–619.
21.
FischerF (1995) Evaluating Public Policy. Chicago, IL: Nelson-Hall Publishers.
22.
FischerFGottweisH (2012) The Argumentative Turn Revisited: Public Policy as Communicative Practice. Durham, NC: Duke University Press.
23.
FischerFTorgersonDDurnováA, et al. (2015) Chapter 1: Introduction to critical policy studies. In: FischerF, et al. (eds) Handbook of Critical Policy Studies. Cheltenham: Edward Elgar, 1–24.
24.
FlyvbjergB (1991) Rationalitet og magt – Bind 1: Det konkretes videnskab. Odense: Akademisk Forlag.
25.
FuruboJERistRCSandahlR (2002) International Atlas of Evaluation. New Brunswick, NJ: Transaction Publishers.
26.
HanbergerA (2011) The real functions of evaluation and response systems. Evaluation17(4): 327–49.
27.
HanbergerALindgrenL (2018) Evaluation systems in local eldercare governance. Journal of Social Work19(2): 233–52.
28.
HansenHFRieperO (2010) The politics of evidence-based policy-making: The case of Denmark. German Policy Studies6(2): 87–112.
29.
HøjlundS (2014) Evaluation use in evaluation systems – The case of the European Commission. Evaluation20(4): 428–46.
30.
JacobSSpeerSFuruboJ-E (2015) The institutionalization of evaluation matters: Updating the International Atlas of Evaluation 10 years later. Evaluation21(1): 6–31.
31.
JensenMS (2010) Evaluering bruges forkert. Djøfbladet2010(1): 28–9.
32.
JohnsonKGreenseidLOToalSA, et al. (2009) Research on evaluation use: A review of the empirical literature from 1986 to 2005. American Journal of Evaluation30(3): 377–410.
33.
JørgensenH (2002) Consensus, Cooperation and Conflict: The Policy Making Process in Denmark. Cheltenham: Edward Elgar.
34.
JørgensenH (2009) From a beautiful swan to an ugly duckling: The renewal of Danish activation policy since 2003. European Journal of Social Security11(4): 337–68.
35.
KirkhartKE (2000) Reconceptualizing evaluation use: An integrated theory of influence. In: CaracelliVJPreskillH (eds) The Expanding Scope of Evaluation Use (New Directions for Evaluation No. 88). San Francisco, CA: Jossey-Bass, 5–23.
36.
LarsenFCaswellDAndersenNA (2017) A new approach to helping the hard to place unemployed: The promise of developing new knowledge in an interactive and collaborative process. European Journal of Social Security19(4): 335–352.
37.
LasswellHD (1936) Politics: Who Gets What, When, How. New York: Whittlesey House.
38.
LeeuwFLFuruboJ (2008) Evaluation systems: What are they and why study them?Evaluation14(2): 157–69.
39.
LindgrenLHanbergerALundströmU (2016) Evaluation systems in a crowded policy space: Implications for local school governance. Education Inquiry7(3): 238–58.
40.
LødemelITrickeyH (2001) ‘An Offer You Can’t Refuse’: Workfare in International Perspective. Bristol: Policy Press.
41.
MarkMMHenryGT (2004) The mechanisms and outcomes of evaluation influence. Evaluation10(1): 35–57.
42.
MartinaitisZChristenkoAKraučiūnienėL (2018) Evaluation systems: How do they frame, generate and use evidence?Evaluation25(1): 46–61.
PattonMQ (1978) Utilization-Focused Evaluation. Beverly Hills, CA: SAGE.
45.
PowerM (1997) The Audit Society: Rituals of Verification. Oxford: Oxford University Press.
46.
PreskillHBoyleS (2008) A multidisciplinary model of evaluation capacity building. American Journal of Evaluation29(4): 443–59.
47.
RaimondoE (2018) The power and dysfunctions of evaluation systems in international organizations. Evaluation24(1): 26–41.
48.
SedlačkoMStaroňováK (2015) An overview of discourses on knowledge in policy: Thinking knowledge, policy and conflict together. Central European Journal of Public Policy9(2): 10–53.
49.
ShulhaLMCousinsJB (1997) Evaluation use: Theory, research, and practice since 1986. Evaluation Practice18(3): 195–208.
50.
SossJFordingRCSchramS (2011) Disciplining the Poor: Neoliberal Paternalism and the Persistent Power of Race. Chicago, IL: The University of Chicago Press.
51.
TorfingJ (2004) Det stille sporskifte i velfærdsstaten: En diskursteoretisk beslutningsprocesanalyse. Aarhus: Aarhus Universitetsforlag.
52.
TriantafillouP (2012) Decentralization as the exercise of power and autonomy: The case of Danish employment policy. Journal of Political Power5(1): 55–71.
53.
TriantafillouP (2017) Neoliberal Power and Public Management Reforms. Manchester: Manchester University Press.
54.
WeissCH (1972) Utilization of evaluation: Toward comparative study. In: WeissCH (ed.) Evaluating Action Programs: Readings in Social Action and Education. Boston, MA: Allyn & Bacon, 318–26.
55.
WeissCH (1979) The many meanings of research utilization. Public Administration Review39(5): 426–31.
56.
WoodM (2015) Puzzling and powering in policy paradigm shifts: Politicization, depoliticization and social learning. Critical Policy Studies9(1): 2–21.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
