As part of a larger effort by members of the American Evaluation Association (AEA) Topical Interest Group on Evaluation Use (TIG- EU), we undertook an extensive review and synthesis of literature in evaluation use published since 1986. We observe several recent developments in theory, research and practice arising from this literature. These developments include: the rise of considerations of context as critical to understanding and explaining use; identification of process use as a significant consequence of evaluation activity; expansion of conceptions of use from the individual to the Lyn M. Shulha organization level; and diversification of the role of the evaluator to facilitator, planner and educator/trainer. In addition, understanding misutlilization has emerged as a significant focus for theory and to a limited extent, research. The article concludes with a summary of contemporary issues, particularly with regard to their implications for evaluation practice.
Get full access to this article
View all access options for this article.
References
1.
Alkin, M.C. ( 1990). Debates on evaluation. Newbury Park, CA: Sage.
2.
Alkin, M.C. (1991). Evaluation theory development II. In M. W. McLaughlin, & D. C. Phillips (Eds.), Evaluation and education: At quarter century (pp. 91-112). Chicago, IL: University of Chicago Press.
3.
Alkin, M.C., & Coyle, K. (1988). Thoughts on evaluation utilization, misutilization, and non-utilization. Studies in Educational Evaluation, 14, 331-340.
4.
Alkin, M.C., Daillak, R., & White, P. (1979). Using evaluations: Does evaluation make a difference ? Beverly Hills: Sage.
5.
Alkin, M.C., Jacobson, P., Burry, J., Ruskus, J., White, P., & Kent, L. ( 1985). A guide for evaluation decision makers. Beverley Hills: Sage Publications.
6.
Alkin, M.C., Koescoff, J., Fitz-Gibbon, C., & Seligman, R. (1974). Evaluation and decision making: The Title VII experience. Los Angeles: Center for the Study of Evaluation.
7.
Alvik, T. (1995). School-based evaluation: A close-up. Studies in Educational Evaluation, 21, 311-343.
8.
Ayers, T.D. (1987). Stakeholders as partners in evaluation: A stakeholder-collaborative approach. Evaluation and Program Planning, 10, 263-271.
9.
Beyer, J.M., & Trice, H.M. (1982). The utilization process: A conceptual framework and synthesis of empirical findings. Administrative Science Quarterly, 27, 591-622.
10.
Brown-McGowan, S., & Eichelberger, T.T. (1993). The utility of the knowledge use system in a higher education evaluation setting. Knowledge: Creation, Diffusion, Utilization, 14(4), 401-416.
11.
Burry, J., Alkin, M., & Ruskus, J. (1985). Organizing evaluations for use as a management tool. Studies in Educational Evaluation, 11, 131-157.
12.
Champney, T.F. (1993). Comments on the use of multiple roles to maximize the utilization of outcome research. Evaluation and Program Planning, 16, 335-336.
13.
Chen, H.T. (1994). A panel on theory-driven evaluation and evaluation theories. Evaluation Practice, 15(1), 73-74. (see also panel discussion, pp. 75-93).
14.
Cousins, J.B. (1995). Assessing program needs using participatory evaluation: A comparison of high and marginal success cases. In J. B. Cousins, & L. M. Earl (Eds.), Participatory evaluation in education: Studies in evaluation use and organizational learning (pp. 55-71). London: Falmer.
15.
Cousins, J.B. (1996). Consequences of researcher involvement in participatory evaluation. Studies in Educational Evaluation, 22(1), 3-27.
16.
Cousins, J.B., Donohue, J.J., & Bloom G.A. (1996). Collaborative evaluation in North America: Evaluators' self- reported opinions, practices, and consequences. Evaluation Practice, 17(3), 207-226.
17.
Cousins, J.B., & Earl, L.M. (1992). The case for participatory evaluation. Educational Evaluation and Policy Analysis, 14(4), 397-418.
18.
Cousins, J. B., & Earl, L. M. (Eds.). (1995). Participatory evaluation in education: Studies in evaluation use and organizational learning. London: Falmer Press.
19.
Cousins, J.B., & Leithwood, K.A. (1986). Current empirical research in evaluation utilization. Review of Educational Research , 56(3), 331-364.
20.
Cousins, J.B., & Leithwood, K.A. (1993). Enhancing knowledge utilization as a strategy for school improvement. Knowledge: Creation, Diffusion, Utilization, 14(3), 305-333.
21.
Duffy, B.P. (1994). Use and abuse of internal evaluation. In C. J. Stevens & M. Dial (Eds.), New directions for program evaluation: Preventing the misuse of evaluation. 64, (pp. 25-32). San Francisco: Jossey-Bass.
Fetterman, D. M., Kaftarian, A. J., & Wandersman, A. (Eds.). (1996). Empowerment evaluation: Knowledge and tools for self assessment and accountability. Thousand Oaks, CA: Sage Publications.
24.
Finne, H., Levine, M., & Nilssen, T. (1995). Trailing research: A model for useful program evaluation. Evaluation, 1(1), 11-31.
25.
Forss, K., Cracknell, B., & Samset, K. (1994). Can evaluation help an organization to learn? Evaluation Review, 18(5), 574-591.
26.
Goering, P.N., & Wasylenki, D.A. (1993). Promoting the utilization of outcome study results by assuming multiple roles within an organization . Evaluation and Program Planning, 16, 329-334.
27.
Greene, J.G. (1988). Stakeholder participation and utilization in program evaluation. Evaluation Review, 12(2), 91-116.
28.
Greene, J.C. (1990). Technical quality versus user responsiveness in evaluation practice. Evaluation and Program Planning, 13, 267-274.
House, E.R. (1995). Principled evaluation: A critique of the AEA guiding principles. In W. R. Shadish, D. L. Newman, M. A. Schreirer & C. Wye (Eds.), New directions for program evaluation: guiding principles for evaluators. 66, (pp. 27-34). San Francisco: Jossey-Bass.
31.
Huberman, M. (1995). The many modes of participatory evaluation. In J. B. Cousins, & L. M. Earl (Eds.), Participatory evaluation in education: Studies in evaluation use and organizational learning (pp. 103-111). London: Falmer.
32.
Huberman, M., & Cox, P. (1990). Evaluation utilization: Building links between action and reflection. Studies in Educational Evaluation, 16, 157-179.
33.
Jenlink, P.M. (1994). Using evaluation to understand the learning architecture of an organization. Evaluation and Program Planning, 17(3), 315-325.
34.
Johnson, R.B. (1993, April). Models of evaluation utilization: A meta-modeling synthesis of the literature. Paper presented at the annual meeting of the American Educational Research Association, Atlanta. (Educational Resources Information Center Document Reproduction Service No. ED 358 112).
35.
Keiny, S., & Dreyfus, A. (1993). School self-evaluation as a reflective dialogue between researchers and practitioners. Studies in Educational Evaluation, 19, 281-295.
36.
King, J.A. (1988). Research on evaluation and its implications for evaluation research and practice. Studies in Educational Evaluation , 14, 285-299.
37.
King, J.A. (1995). Involving practitioners in evaluation studies: How viable is collaborative evaluation in schools? In J. B. Cousins, & L. M. Earl (Eds.), Participatory evaluation in education: Studies in evaluation use and organizational learning (pp. 86-102). London: Falmer.
38.
King, J.A., & Thompson, B. (1983). Research on school use of program evaluation: A literature review and research agenda. Studies in Educational Evaluation, 9, 5-21.
39.
Levin, B. (1987). The uses of research: A case study in research and policy. The Canadian Journal of Program Evaluation, 2(1), 43-55.
40.
Leviton, L.C., & Hughes, E.F.X. (1981). Research on the utilization of evaluations: A review and synthesis. Evaluation Review , 5(4), 525-548.
Mathison, S. (1994). Rethinking the evaluator role: Partnerships between organizations and evaluators. Evaluation and Program Planning , 17(3), 299-304.
43.
Mowbray, C.T., (1992). The role of evaluation in restructuring of the public mental health system. Evaluation and Program Planning , 15, 403-415.
44.
Murphy, D.F. (1996). The evaluator's apprentice: Learning to do evaluation . Evaluation, 2(3), 321-338.
45.
Nevo, D. (1993). The evaluation minded school: An application of perceptions from program evaluation. Evaluation Practice, 14(1), 39-47.
46.
Newman, D.L., & Cai M. (1995, November). Teacher involvement in program evaluation and evaluation use: An empirical examination of the participatory evaluation approach. Paper presented at the joint meeting of the Canadian Evaluation Society and the American Evaluation Association , Vancouver.
47.
Owen, J.M., Lambert, F.C., & Stringer, W.N. (1994). Acquiring knowledge of implementation and change: Essential for program evaluators? Knowledge: Creation, Diffusion, Utilization, 15(3), 273-284.
48.
Owen, J.M., & Lambert, F.C. (1995). Roles for evaluation in learning organizations . Evaluation, 1(2), 237-250.
49.
Palumbo, D.J., (1994). The political roots of misuse of evaluation. In C. J. Stevens & M. Dial (Eds.), New directions for program evaluation: Preventing the misuse of evaluation. 64, (pp. 15-24). San Francisco: Jossey-Bass.
50.
Patton, M.Q. (1988a). The evaluator's responsibility for utilization . Evaluation Practice, 9(2), 5-24.
51.
Patton, M. (1988b). Six honest serving men for evaluation. Studies in Educational Evaluation, 14, 301-330.
Patton, M.Q. (1997). Utilization-focused evaluation: The new century text. (3rd Ed.). Newbury Park, CA: Sage.
54.
Patton, M.Q., Grimes, P.S., Guthrie, K.M., Brennan, N.J., French, B.D., & Blyth, D.A. (1977). In search of impact: An analysis of the utilization of federal health evaluation research. In C. H. Weiss (Ed.), Using social research in public policy making. (pp. 141-163). New York: D. C. Heath and Company.
55.
Preskill, H. (1991). The cultural lens: Brining utilization into focus . In C. Larson, & H. Preskill (Eds.), New directions in program evaluation: Organizations in transition: Opportunities and challenges for evaluation. 49 (pp. 5-15). San Francisco : Jossey-Bass Inc.
56.
Preskill, H. (1994). Evaluation's role in enhancing organizational learning. Evaluation and Program Planning, 17(3), 291-297.
57.
Preskill, H. & Torres, R.T. (1996, November). From evaluation to evaluative inquiry for organizational learning. Paper presented at the annual meeting of the American Evaluation Association, Atlanta.
58.
Scriven, M. (1996). Types of evaluation and types of evaluator. Evaluation Practice, 17(3), 151-161.
59.
Shadish, W.R., & Epstein, R.A. (1987). Patterns of program evaluation practice among members of the Evaluation Research Society and Evaluation Network. Evaluation Review, 11(5), 555-590.
60.
Shulha, L.M. (1994, November). Learning to participate in participatory evaluation: A case study in the transformation of design. Paper presented at the annual meeting of the American Evaluation Association , Boston, MA.
61.
Shulha, L.M. & Wilson, R.J. (1995). Inviting collaboration: Insights into researcher-school community partnerships. In J. B. Cousins & L. M. Earl (Eds.), Participatory evaluation in education: Studies in evaluation use and organizational learning (pp. 115-139). London: Falmer Press.
62.
Smith, N.L., & Chircop, S. (1989). The Weiss-Patton debate: Illumination of the fundamental concerns. Evaluation Practice, 5-13.
63.
Stahler, G.J. (1995). Improving the quality of federal human services national demonstration programs. Evaluation and Program Planning , 18(2), 129-141.
64.
Stevens, C.J., Dial, M. (1994). What constitutes misuse? In Stevens & Dial (Eds.), New directions for program evaluation: Guiding principles for evaluators. 64, (pp. 3-14). San Francisco : Jossey-Bass.
65.
Thompson, B. (1994, April). The revised program evaluation standards and their correlation with the evaluation use literature. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. (ERIC Document Reproduction Service No. ED 370 999)
66.
Thompson, B. & King, J.A. (1981, April). Evaluation utilization: A literature review and research agenda. Paper presented at the annual meeting of the American Educational Research Association LosAngeles CA: National Institute of Education. (ERIC Document Reproduction Service No. ED 199 271)
67.
Timar, T. (1994). Federal education policy and practice: Building organizational capacity through Chapter 1. Educational Evaluation and Policy Analysis, 16(1), 51-66.
68.
Tomlinson, C., Bland, L., & Moon, T. (1993). Evaluation utilization: A review of the literature with implications for gifted education. Journal for the Education of the Gifted, 16(2), 171-189.
69.
Torres, R.T., Preskill, H.S., & Piontek, M.E. (1996). Evaluation strategies for communicating and reporting: Enhancing learning in organizations. Thousand Oaks, CA : Sage.
70.
Vroom, P.I., Colombo, M., & Nahan, N. (1994). Confronting ideology and self-interest: Avoiding misuse of evaluation. In C.J. Stevens & M. Dial (Eds.), New directions for program evaluation: Preventing the misuse of evaluation, 64, (pp. 49-60). San Francisco: Jossey-Bass.
71.
Weiss, C.H. (1973). Where politics and evaluation research meet. Evaluation, 1(3), 37-45.
72.
Weiss, C.H. (1977). (Ed.). Using social research in public policy makingNew York: D. C. Heath and Company.
73.
Weiss, C.H. (1979). The many meanings of research utilization. Public Administration Review, 39, 426-431.
74.
Weiss, C.H. (1988a). Evaluation for decisions: Is anybody there? Does anybody care? Evaluation Practice, 9(1), 5-19.
75.
Weiss, C.H. (1988b). If program decisions hinged only on information: A response to Patton. Evaluation Practice, 9(3), 15-28.
