This article presents some of the results from a study in progress, focusing on the influence of the institutional distance between evaluators and evaluees on the utilization of evaluations. The basis for the results presented here is an analysis of ten case studies from Switzerland. These cases involve evaluations that were carried out in different institutional contexts, with widely varying institutional distances between evaluators and evaluees. ‘Qualitative Comparative Analysis’ (QCA) has been used to interpret the cases, in order to allow a combination of case-and variable-centred comparisons. The analysis indicates that, under certain conditions, the institutional distance between evaluators and evaluees has no influence on the use of evaluations. In particular, formative objectives can be achieved quite independently of distance. When interpreting the results, however, one should not neglect the fact that they are solely based on a systematic evaluation of ten case studies with QCA. Generalization is not possible on this basis, nor is this the aim of the present article. On the contrary, the objective is to continue developing the debate about the influence of the institutional distance between evaluators and evaluees on the utilization of evaluations.
Alkin, M. C. (1985) A Guide for Evaluation Decision Makers. Beverly Hills, CA: SAGE .
2.
Balthasar, A. (2000) Energie 2000: Programmwirkungen und Folgerungen aus der Evaluation. Chur: Rüegger.
3.
Balthasar, A. (2003) Die Auswirkungen des institutionellen Kontexts einer Evaluation auf die Verwendung von Evaluationsergebnissen: Erster Zwischenbericht und Fortsetzungsgesuch zuhanden des Schweizerischen Nationalfonds. Lucerne: Interface Institut für Politikstudien.
4.
Clarke, A. (1999) Evaluation Research: An Introduction to Principles, Methods and Practice. London: SAGE .
5.
Cronqvist, L. (2003a) ‘Presentation of TOSMANA: Adding Multi-Value Variables and Visual Aids to QCA’ , paper presented at the COMPASS Launching Conference, 16-17 Sept., Louvain-La-Neuve (B) and Leuven.
6.
Cronqvist, L. (2003b) ‘Using Multi-Value Logic Synthesis in Social Science’ , paper prepared for the 2nd General Conference of the European Consortium for Political Research (ECPR), 18-21 Sept., Marburg (D).
7.
Danuser, J. and G. Regula (2002) Evaluation der Ökomassnahmen und Tierhaltungsprogramme, Bereich Artgerechte Tierhaltung: Schlussbericht Kategorie Milchkühe. Berne: Bundesamt für Landwirtschaft.
8.
De Meur, G. and B. Rihoux (2002) L’analyse quali-quantitative comparée: Approche, techniques et applications en sciences humaines. Louvain-la-Neuve (B): Bruylant-Academia.
9.
Eidgenössische Finanzkontrolle EFK (2002) Wirtschaftlichkeitsprüfung der Subventionen an Werkstätten für die Dauerbeschäftigung von Invaliden. Berne: EFK.
10.
Eidgenössisches Personalamt EPA (2002) Bericht an den Bundesrat über die zweite Frauenförde rungs periode in der allgemeinen Bundesverwaltung 1996-1999. Berne: EPA.
11.
Forss, K. , C. C. Rebienet al. (2002) ‘Process Use of Evaluations: Types of Use that Precede Lessons Learned and Feedback’ , Evaluation8(1): 29-45 .
12.
Gerheuser, F.W. (1999) Evaluation der Strategie und der Produkte des Ressorts Gewerbe von Energie 2000: Schlussbericht. Brugg, AG: Büro für Politikberatung und Sozialforschung.
13.
Greenberg, D. H. and M. B. Mandell (1995) ‘Research Utilization in Policymaking: A Tale of Two Series (of Social Experiments)’, in R. C. Rist (ed.) Policy Evaluation: Linking Theory to Practice, pp. 156-179. Aldershot: Edward Elgar .
14.
New Directions for Evaluation88: 85-98 .
15.
Henry, G. T. and M. M. Mark (2003) ‘Beyond Use: Understanding Evaluation’s Influence on Attitudes and Actions’ , American Journal of Evaluation24(3): 293-314 .
16.
Huberman, M. and M. Gather Thurler (1991) De la recherche à la pratique: Eléments de base. Berne: Peter Lang.
17.
Jaggi, K. and A. Näf (2000) Evaluation der Rückkehrberatung im Asylbereich: Schlussbericht zuhanden des Bundesamts für Flüchtlinge (BFF) und der Internationalen Organisation für Migration (IOM). Berne: BFF and IOM.
18.
Joint Committee on Standards for Educational Evaluation (2000) Handbuch der Evaluationsstandards: Die Standards des ‘Joint Committee on Standards for Educational Evaluation’. Opladen: Leske & Budrich.
19.
Kingdon, J. W. (1984) Agendas, Alternatives, and Public Policies. Boston, MA: Little, Brown & Co.
20.
Ledermann, S. (2004) Wozu denn wissenschaftlich? Untersuchung zur Verwendung von Evaluationen in der schweizerischen Entwicklungszusammenarbeit. Berne: Institut für Politikwissenschaft der Universität Bern.
21.
Lehmann, L. and A. Balthasar (2004) Quality Assessment of SDC’s External Evaluation Reports, Commissioned by the Evaluation and Controlling Division of the Swiss Agency for Development and Cooperation (SDC). Lucerne: Interface Institute for Policy Studies .
22.
Lester, J. P. and L. J. Wilds (1990) ‘The Utilization of Public Policy Analysis: A Conceptual Framework’ , Evaluation and Program Planning13: 313-319 .
23.
Mark, M. M. and G. T. Henry (2004) ‘The Mechanisms and Outcomes of Evaluation Influence’ , Evaluation10(1): 35-57 .
24.
Owen, J. M. (1993) Program Evaluation: Forms and Approaches. St Leonards, NSW: Allen & Unwin .
25.
Parlamentarische Verwaltungskontrollstelle PVK (2000) Das Kartellrecht: Standortbestimmung: Schlussbericht zuhanden der Geschäftsprüfungskommission des National-rates. Berne: PVK.
26.
Patton, M. Q. (1997) Utilization-Focused Evaluation: The New Century Text. Thousand Oaks, CA: SAGE .
27.
Patton, M. Q. (1998) ‘Discovering Process Use’ , Evaluation4(2): 225-233 .
28.
Ragin, C. C. (1987) The Comparative Method: Moving beyond Qualitative and Quantitative Strategies. Berkeley, CA: University of California Press .
29.
Rogers, P. J. and G. Hough (1995) ‘Improving the Effectiveness of Evaluations: Making the Link to Organizational Theory’ , Evaluation and Program Planning18: 321-332 .
30.
Sardi, M. and A. Ensmann (2002) L’evaluation de la Campagne nationale de prévention de la grippe 2001. Carouge, GE: Institut érasm.
31.
Schweizerischer Bundesrat (2002) Bericht über die Verkehrsverlagerung von 27. März 2002 (Verlagerungsbericht 2002): Bericht des Bundesrates an die parlamentarischen Kommissionen. Berne: Departement für Umwelt, Verkehr, Energie und Kommunikation.
32.
Scriven, M. (1997) ‘Truth and Objectivity in Evaluation’, in E. Chelimsky and W. R. Shadish (eds) Evaluation for the 21st Century: A Handbook, pp. 477-500. Thousand Oaks, CA: SAGE .
33.
Shulha, L. M. and J. B. Cousins (1997) ‘Evaluation Use: Theory, Research, and Practice since 1986’ , Evaluation Practice18(3): 195-208 .
34.
Sonnichsen, R. C. (1999) ‘Building Evaluation Capacity within Organisations’, in R. Boile and D. Lemaire (eds) Building Effective Evaluation Capacity: Lessons from Practice, pp. 53-73. New Brunswick, NJ: Transaction .
35.
Vanderborght, Y. and S. Yamasaki (2003) ‘The Problem of Contradictory Simplifying Assumptions in Qualitative Comparative Analysis (QCA)’ , paper prepared for the 2nd General Conference of the European Consortium for Political Research (ECPR), 18-21 Sept., Marburg (D).
36.
Varone, F. , B. Rihoux and A. Marx (2005) ‘A New Method for Policy Evaluation? Longstanding Challenges of Policy Evaluation and the Possibilities of Qualitative Comparative Analysis (QCA)’, in B. Rihoux and H. Grimm (eds) Innovative Comparative Methods for Policy Analysis and Evaluation, pp. 213-236. New York: Springer .
37.
Vedung, E. (1997) Public Policy and Program Evaluation. New Brunswick, NJ: Transaction .
38.
Vedung, E. (1999) Evaluation im öffentlichen Sektor. Vienna: Böhlau Verlag .
39.
Weiss, C. H. (1977) ‘Research for Policy’s Sake: The Enlightenment Function of Social Research’ , Policy Analysis3: 531-545 .
40.
Weiss, C. H. , E. Murphy-Graham and S. Brikeland (2005) ‘An Alternate Route to Policy Influence: How Evaluations Affect D.A.R.E.’ , American Journal of Evaluation26(1): 12-30 .
41.
Wholey, J. S. (1994) ‘Assessing the Feasibility and Likely Usefulness of Evaluation’, in J. S. Wholey , H. P. Hatry and K. E. Newcomer (eds) Handbook of Practical Program Evaluation, pp. 15-39. San Francisco, CA: Jossey-Bass .
42.
Widmer, T. and P. Neuenschwander (2004) ‘Embedding Evaluation in the Swiss Federal Administration: Purpose, Institutional Design and Utilization’ , Evaluation10(4): 388-409 .
43.
Widmer, T. , C. Landert and N. Bachmann (2000) ‘Evaluation Standards Recommended by the Swiss Evaluation Society (SEVAL)’, in C. Russen (ed.) The Program Evaluation Standards in International Settings, pp. 81-102. Kalamazoo, MI: Western Michigan University .
44.
Widmer, T., E. Rüegg and P. Neuenschwander (2001) Stand und Aussichten der Evaluation beim Bund: Schlussbericht zuhanden der Schweizerischen Bundeskanzlei vom 9. März 2001. Zürich: Institut für Politikwissenschaft der Universität Zürich.
45.
Williams, K. , B. de Laat and E. Stern (2002) The Use of Evaluation in the Commission Services: Final Report. Paris: Technopolis France .
