Past literature has identified several putative precursors of use, as well as alternative forms of use. However, important shortcomings still exist in previous work on use. In particular, inadequate attention has been given to the underlying processes that may mediate the effects of evaluation on attitude and action. In essence, a key part of the theory of change for evaluation itself is missing. To help fill this gap, we describe a framework designed to capture key mechanisms through which evaluation may have its effects. The framework includes change processes that have been validated in various social science literatures. It identifies three levels of analysis (individual, interpersonal and collective), each with four kinds of processes (general influence, attitudinal, motivational and behavioral). With a more comprehensive view of the mechanisms underlying evaluation’s influence, the field can move forward in relation to its understanding and facilitation of evaluation’s role in the service of social betterment.
Get full access to this article
View all access options for this article.
References
1.
Alkin, M. C. (2003) ‘Introduction (to Section on Evaluation Use)’, in T. Kelligan and D. L. Stufflebeam (eds) International Handbook of Educational Evaluation, pp. 189–196. Dordrecht: Kluwer Academic Press .
2.
Alkin, M. C., R. Daillak and P. White (1979) Using Evaluations: Does Evaluation Make a Difference? Beverly Hills, CA: Sage .
3.
Barnett, W. S. (1992) ‘Benefits of Compensatory Preschool Education’ , Journal of Human Resources27(2): 279–312 .
4.
Caplan, N. (1977) ‘A Minimal Set of Conditions Necessary for the Utilization of Social Science Knowledge in the Policy Formulation at the National Level’, in C. H. Weiss (ed.) Using Social Research in Public Policy Making. Lexington, MA: Lexington Books .
5.
Caracelli, V. J. (2000) ‘Evaluation Use at the Threshold of the Twenty-first Century’, in V. Caracelli and H. Preskill (eds) The Expanding Scope of Evaluation Use. New Directions for Evaluation 88. San Francisco, CA: Jossey-Bass .
6.
Cousins, J. B. (1996) ‘Consequences of Researcher Involvement in Participatory Evaluation’ , Studies in Educational Evaluation1(1): 3–27 .
7.
Cousins, J. B. (2003) ‘Utilization Effects of Participatory Evaluation’, in T. Kelligan and D. L. Stufflebeam (eds) International Handbook of Educational Evaluation, pp. 245–266. Dordrecht: Kluwer Academic Press .
8.
Cousins, J. B. and K. A. Leithwood (1986) ‘Current Empirical Research on Evaluation Utilization’ , Review of Educational Research56(3): 331–364 .
9.
Henry, G. T. (2000) ‘Why Not Use?’, in V. Caracelli and H. Preskill (eds) The Expanding Scope of Evaluation Use. New Directions for Evaluation 88, pp. 85–98. San Francisco, CA: Jossey-Bass .
10.
Henry, G. T. and M. M. Mark (2003) ‘Beyond Use: Understanding Evaluation’s Influence on Attitudes and Actions’ , American Journal of Evaluation24(3): 293–314 .
11.
Hofstetter, C. H. and M. C. Alkin (2003) ‘Evaluation Use Revisited’, in T. Kelligan and D. L. Stufflebeam (eds) International Handbook of Educational Evaluation. Dordrecht: Kluwer Academic Press .
12.
King, J. A. (2002) ‘Building the Evaluation Capacity of a School District’, in D. W. Compton, M. Baizerman and S. H. Stockdill (eds) The Art, Craft, and Science of Evaluaiton Capacity Building. New Directions for Evaluation 93, pp. 63–80. San Francisco, CA: Jossey-Bass .
13.
Kirkhart, K. (2000) ‘Reconceptualizing Evaluation Use: An Integrated Theory of Influence’, in V. Caracelli and H. Preskill (eds) The Expanding Scope of Evaluation Use. New Directions for Evaluation 88, pp. 5–24. San Francisco, CA: Jossey-Bass .
14.
Knorr, K. D. (1977) ‘Policymakers’ Use of Social Science Knowledge: Symbolic or Instrumental?’, in C. H. Weiss (ed.) Using Social Research in Public Policy Making. Lexington, MA: Lexington Books .
15.
Mark, M. M. (2001) ‘Evaluation’s Future: Furor, Futile, or Fertile?’ , American Journal of Evaluation22(3): 457–479 .
16.
Mark, M. M. (2003) ‘Toward a Comprehensive View of the Theory and Practice of Program and Policy Evaluation’, in S. I. Donaldson and M. Scriven (eds) Evaluating Social Programs and Problems: Visions for the New Millennium, pp. 183–204. Hillsdale, NJ: Erlbaum .
17.
Mark, M. M., G. T. Henry and G. Julnes (2000) Evaluation: An Integrated Framework for Understanding, Guiding, and Improving Policies and Programs. San Francisco, CA: Jossey-Bass .
18.
Melkers, J. E. and K. G. Willoughby (1998) ‘The State of the States: Performance-Based Budgeting Requirements in 47 Out of 50’ , Public Administration Review58(1): 66–73 .
19.
Melkers, J. E. and K. G. Willoughby (2001) ‘Budgeters Views of State Performance-budgeting Systems: Distinctions across Branches’ , Public Administration Review61(1): 54–64 .
20.
Nutley, S., H. Davies and I. Walter (2002) ‘Conceptual Synthesis 1: Learning from the Diffusion of Innovations’. Unpublished manuscript, University of St Andrews, Research Unit for Research Utilization.
21.
Patton, M. Q. (1997) Utilization-focused Evaluation: The New Century Text. Thousand Oaks, CA: Sage .
22.
Patton, M. Q., P. S. Grimes, K. M. Guthrie, N. J. Brennan, B. D. French and D. A. Blyth (1977) ‘In Search of Impact: An Analysis of the Utilization of Federal Health Evaluation Research’, in C. H. Weiss (ed.) Using Social Research in Public Policy Making. Lexington, MA: Lexington Books .
23.
Pawson, R. and N. Tilley (1997) Realistic Evaluation. London: Sage .
24.
Petty, R. E. and J. T. Cacioppo (1986) Communication and Persuasion: Central and Peripheral Routes to Attitude. New York: Springer-Verlag .
25.
Preskill, H. (1994) ‘Evaluation’s Role in Enhancing Organizational Learning’ , Evaluation and Program Planning17(3): 291–297 .
26.
Preskill, H. and V. J. Caracelli (1997) ‘Current and Developing Conceptions of Use: Evaluation Use Topical Interest Group Survey Results’ , Evaluation Practice18(3): 209–225 .
27.
Preskill, H. and R. Torres (1998) Evaluative Inquiry for Organizational Learning. Thousand Oaks, CA: Sage .
28.
Rog, D. J. (1985) A Methodological Analysis of Evaluability Assessment, Doctoral dissertation. Nashville, TN: Vanderbilt University .
29.
Shadish, W. R., T. D. Cook and L. C. Leviton (1991) Foundations of Program Evaluation: Theories of Practice. Newbury Park, CA: Sage .
30.
Shula, L. M. and J. B. Cousins (1997) ‘Evaluation Use: Theory, Research, and Practice since 1986’ , Evaluation Practice18(3): 195–208 .
31.
Walter, I., S. Nutley and H. Davies (2003a) ‘Developing a Taxonomy of Interventions Used to Increase the Impact of Research’. Unpublished manuscript, University of St Andrews, Research Unit for Research Utilization.
32.
Walter, I., S. Nutley and H. Davies (2003b) ‘Research Impact: A Cross Sector Literature Review’. Unpublished manuscript, University of St Andrews, Research Unit for Research Utilization.
33.
Weiss, C. H., ed. (1977) Using Social Research in Public Policy Making. Lexington, MA: Lexington Books .
34.
Weiss, C. H. (1979) ‘The Many Meanings of Research Utilization’ , Public Administration Review39(Sept/Oct): 426–431 .
35.
Weiss, C. H. (1988) ‘Evaluation for Decisions: Is Anybody There? Does Anybody Care?’ , Evaluation Practice9(1): 5–20 .
36.
Weiss, C. H. (1998) ‘Improving the Use of Evaluations: Whose Job is it Anyway?’ , Advances in Educational Productivity7: 263–276 .
37.
Weiss, C. H. and M. J. Bucuvalas (1977) ‘The Challenge of Social Research to Decision Making’, in C. H. Weiss (ed.) Using Social Research in Public Policy Making. Lexington, MA: Lexington Books .
38.
Williams, K., B. de Laat and E. Stern (2002) The Use of Evaluation in the Commission Services: Final Report. Paris: Technopolis France .
39.
Willoughby, K. G. and J. E. Melkers (2001) ‘Assessing the Impact of Performance Budgeting: A Survey of American States’ , Government Finance Review17(2): 25–30 .
