An area of concernfor evaluation research is the extent to which evaluations are utilized in
policy and program development and change. The current review critically discusses
definition of utilization and methodsfor detecting utilization. Five "clusters" of variables
have been found to affect utilization. These are: (1) the relevance of evaluation to the needs
of potential users; (2) extent of communication between potential users and producers of
evaluations; (3) translation of evaluations into their implicationsforpolicy and programs;
(4) credibility or trust placed in evaluations; and (5) commitment or advocacy by
individual users.
Get full access to this article
View all access options for this article.
References
1.
Agarwala-Rogers, R. (1977) "Why is evaluation research not utilized?" in M. Guttentag (ed.) Evaluation Studies Review Annual, Vol. 2. Beverly Hills: Sage.
2.
Alkin, M.C., R. Daillak, and P. White (1979) Using Evaluations: Does Evaluation Make a Difference?Beverly Hills: Sage.
3.
Averch, H. (1975) "Notes on improving research utility." Inquiry, 12, 3 (September): 231-234.
4.
Banta, H.D. and P. Bauman (1976) "Health services research and health policy." J. of Community Health2, 2 (Winter): 121-132.
5.
Bauman, P. (1976) "The formulation and evolution of the health maintenance organization policy, 1970-1973." Social Sci. and Medicine10, 3/4 (March/April): 129-142.
6.
Boruch, R.F. and P.M. Wortman (1979) "Implications of educational evaluation for evaluation policy," in D. Berliner (ed.) Review of Research in Education (Vol. 7). Washington, DC: American Educational Research Association .
7.
Brandl, J.E. (1978) "Evaluation and politics." Evaluation (Special Issue): 6-7.
8.
Brickell, H.M. (1978) "The influence of external political factors on the role and methodology of evaluation," in T. D. Cook et al. (eds.) Evaluation Studies Review Annual (Vol. 3). Beverly Hills: Sage.
9.
——— (1974) Data for Decisions: An Analysis of Evaluation Data Needed by Decision-Makers in Educational Programs. New York: Policy Studies in Education.
10.
Brown, R.D., L.A. Braskamp, and D.L. Newman (1978) "Evaluator credibility as a function of report style." Evaluation Q.2, 2 (May): 331-341.
Caplan, N. (1979) "The two-communities theory and knowledge utilization ." Amer. Behavioral Scientist22, 3 (January/ February): 459-469.
13.
——— A. Morrison, and R. Stambaugh (1975) The Use of Social Science Knowledge in Policy Decisions at the National Level. Ann Arbor, MI: Institute for Social Research.
14.
Carter, R.K. (1971) "Client reactions to negative findings and the latent conservative function of evaluation studies." Amer. Sociologist6, 2 (May): 118-124.
15.
Chelimsky, E. (1977) A Symposium on the Use of Evaluation by Federal Agencies, (Vol. II). Washington, DC: National Institute of Law Enforcement and Criminal Justice and Metrek, a division of the MITRE Corporation.
16.
Cook, T.D. (1978) "Introduction," in T. D. Cook et al. (eds.) Evaluation Studies Review Annual . Beverly Hills: Sage.
17.
——— and W.E. Pollard (1977) "Guidelines: how to recognize and avoid some common problems of mis-utilization of evaluation research findings." Evaluation4: 161-164.
18.
Cook, T.D., J. Levinson-Rose, and W.B. Pollard (1980) "The misutilization of evaluation research: some pitfalls of definition." Knowledge1, 4 (June): 477-498.
19.
Cox, G.B. (1977) "Managerial style: implications for the utilization of program evaluation information." Evaluation Q.1, 4 (December): 499-508.
20.
Datta, L.E. (1977) "Does it work when it has been tried? and half full or half empty?" in M. Guttentag (ed.) Evaluation Studies Review Annual (Vol. 2). Beverly Hills: Sage.
21.
David, J.L. (1978) Local Uses of Title I Evaluations. Report prepared for the Office of the Assistant Secretary for Planning and Evaluation, Department of Health, Education and Welfare. Menlo Park, CA: SRI International.
22.
Davis, H.R. and S.E. Salasin (1978) "Strengthening the contribution of social R&D to policy making," in L. E. Lynn (ed.) Knowledge and Policy: The Uncertain Connection. Washington DC: National Research Council.
23.
——— (1975) "The utilization of evaluation," in E. K. Struening and M. Guttentag (eds.) Handbook of Evaluation Research (Vol. 1). Beverly Hills: Sage.
24.
Davis, H.R., C. Windle, and S.S. Scharfstein (1977) "Developing guidelines for program evaluation in community mental health centers." Evaluation4 (Special Issue): 25-29.
25.
Dobson, A., J.G. Greer, R.H. Carlson, F.A. Davis, L.E. Kucken, B.J. Steinhardt, T.P. Ferry, and G.S. Adler (1978) "PSRO's: their current status and their impact to date." Inquiry15, 2 (June): 113-128.
26.
Downs, A. (1967) Inside Bureaucracy. Boston: Little, Brown.
27.
Eaton, J.W. (1969) "Symbolic and substantive evaluation research," in H. C. Schulberg et al. (eds.) Program Evaluation in the Health Fields. New York : Behavioral Publications.
28.
Fairweather, G.W., D.H. Sanders, and L.G. Tornatsky (1974) Creating Change in Mental Health Organizations . New York: Pergamon Press.
29.
Falcone, D. and B.J. Jaeger, (1976) "The policy effectiveness of health services research: a reconsideration." J. of Community Health2, 1 (Fall): 36-51.
30.
Florio, D.H., M.M. Behrman, and D.L. Goltz (1979) "What do policy makers think of educational research and evaluation? Or do they?" Educ. Evaluation & Policy Analysis1: 61-88.
31.
Fox, F.W. and B.M. Staw (1979) "The trapped administrator: effects of job security and policy resistance upon commitment to a course of action." Evanston, IL: Northwestern University. (unpublished)
32.
Glaser, E.M. (1976) Putting Knowledge to Use: A Distillation of the Literature Regarding Knowledge Transfer and Change. Beverly Hills: Human Interaction Research Institute.
33.
——— and S.H. Taylor (1973) "Factors influencing the success of applied research ." Amer. Psychologist28, 2 (February): 140-146.
34.
Guba, E.G. (1975) "Problems in utilizing the results of evaluation ." J. of Research and Development in Education8, 3 (Spring): 42-54.
35.
Hill, P. (1980) "Evaluating education programs for federal policy makers: lessons from the NIE Compensatory Education Study," in J. Pincus (ed.) Educational Evaluation in the Public Policy Setting. Santa Monica, CA: Rand Corporation.
36.
Huron Institute (1979) Proposal to National Institute of Education. Cambridge, MA: Author.
37.
Kiesler, C.A. (1971) The Psychology of Commitment. New York: Academic.
38.
Knorr, K.D. (1977) "Policymakers' use of social science knowledge: symbolic or mstrumental?" in C. H. Weiss (ed.) Using Social Science Research in Public Policy Making. Lexmgton, MA: Lexmgton Books.
39.
Larsen, J.K. and P.D. Werner (1981) "Measuring utilization of mental health program consultation," in J. A. Ciarlo (ed.) Utilizing Evaluation. Beverly Hills. Sage.
40.
Leviton, L.C. and R.F. Boruch (1980) "Illustrative case studies," in R. F. Boruch and D. S Cordray (eds.) An Appraisal of Educational Program Evaluations. Federal, State, and Local Levels. Washington, DC: U.S. Department of Education.
41.
Lynn, L.E. Jr. (1978) "The question of relevance," in L. E. Lynn (ed.) Knowledge and Policy: The Uncertain Connection. Washington DC: National Research Council.
42.
Lyon, C.D., L. Doscher, P. Mcgranahan, and R. Williams (1978) Evaluation and School Districts. Los Angeles: Center for the Study of Evaluation.
43.
Mechanic, D. (1978) "Prospects and problems in health services research ." Milbank Memorial Fund Q.56, 2 (Spring): 127-139.
44.
Menges, C.C. (1978) Knowledge and Action: The Rise of Social Science Evaluations in Decisions on Equal Educational Opportunity, 1970-1973. Washington, DC: National Institute of Education .
45.
Millsap, M.A. (1978) "The uses of evaluation in federal regulation writing: a case study." Presented at Evaluation Research Society Convention, Washington, D.C.
46.
Mintzberg, H. (1973) The Nature of Managerial Work. New York: Harper & Row.
47.
Mitchell, D.E. (1980) "Social science impact on legislative decision making: process and substance." Presented at the annual meeting of the American Educational Research Association.
48.
Mushkin, S.J. (1973) "Evaluations: use with caution." Evaluation1, 2: 31-35.
49.
Nielsen, V.G. (1975) "Why evaluation does not improve program effectiveness ." Policy Studies J.3, 4 (Summer): 385-389.
——— P.S. Grimes, K.M. Guthrie, N.J. Brennan, B.D. French and D.A. Blyth (1977) "In search of impact: an analysis of the utilization of federal health evaluation research," in C. H. Weiss (ed.) Using Social Research in Public Policy Making. Lexington, MA: Lexington Books.
52.
Pelz, D.C. (1979) Knowledge Utilization in Criminal Justice. A Research Proposal Submitted to the National Institute for Law Enforcement and Criminal Justice, LEAA. Ann Arbor, MI: Institute for Social Research.
53.
——— (1978) "Some expanded perspectives on use of social science in public policy," in J. M. Yinger and S. J Cutler (eds.) Major Social Issues: A Multidisciplinary View. New York: Macmillan.
54.
Pfeffer, J. and G.R. Salancik (1978) The External Control of Organizations: A Resource Dependent Perspective. New York: Harper & Row.
55.
Rein, M. and S.H. White (1975) "Can policy research help policy?" The Public Interest49 (Fall): 119-136.
56.
Resnick, S.M., C. O'Reilly, and A. Majchrzak "Utilization: different users different uses." Presented at Evaluation Research Society Convention, Minneapolis .
57.
Rich, R.F. (1979) "Problem-solving and evaluation research: unemployment insurance policy," in R. F. Rich (ed.) Translating Evaluation into Policy. Beverly Hills: Sage.
58.
——— (1977) "Uses of social science information by federal bureaucrats: knowledge for action versus knowledge for understanding," in C. H. Weiss (ed.) Using Social Research in Public Policy Making. Lexington, MA: Lexington Books.
59.
Schmidt, R.E., P. Horst, J.W. Scanlon, and J.S. Wholey (1977) Serving the Federal Evaluation Market. Washington, DC: Urban Institute.
60.
Staw, B.M. (1976) "Knee-deep in the big muddy: a study of escalating commitment to a chosen course of action." Organizational Behavior and Human Performance16, 1: 27-44.
61.
——— and F.V. Fox (1977) "Escalation: the determinants of commitment to a chosen course of action." Human Relations30, 5 (May): 431-450.
62.
Staw, B.M. and J. Ross (1978) "Commitment to a policy decision: a multitheoretical perspective." Admin. Sci Q.23, 1 (March): 40-64.
63.
Sundquist, J.L. (1978) "Research brokerage: the weak link," in L. E. Lynn (ed.) Knowledge and Policy: The Uncertain Connection. Washington DC: National Research Council.
64.
Tallmadge, G.K. (1977) Ideabook: The Joint Dissemination Review Panel . Washmgton DC: U.S. Office of Education and National Institute of Education.
65.
Tiffany, D.W., M. Phyllis and J.R. Cowan (1971) "A source of problems between social science knowledge and practice." J. of Human Relations19, 2: 239-250.
66.
Weinberg, H. (1979) "The role of cognitive biases in the interpretation of program evaluations." Dissertation in progress, Yale University.
67.
Weiss, C.H. (1978) "Improving the linkage between social research and public policy," in L. E. Lynn (ed.) Knowledge and Policy: The Uncertain Connection. Washington DC: National Research Council.
68.
——— (1977a) "Introduction," in C. H. Weiss (ed.) Using Social Research in Public Policy Making. Lexington, MA: Lexington Books.
69.
——— (1977b) "Research for policy's sake: the enlightenment function of social research." Policy Analysis3, 4 (Fall): 531-545.
70.
——— (1973) "Where politics and evaluation meet." Evaluation1, 3: 37-45.
71.
——— and M.J. Bucuvalas (1977) "The challenge of social research to decision-making," in C. H. Weiss (ed.) Using Social Science Research in Public Policy Making. Lexington, MA: Lexington Books.
72.
Weiss, J.A. (1979) "Access to influence: some effects of policy sector on the use of social science." Amer. Behavioral Scientist22, 3 (January/ February): 437-458.
73.
White, K.L. and J.H. Murnaghan (1973) "Health care policy formation: analysis, information and research." Int. J. of Health Services3, 1 (Winter): 81-91.
74.
Wholey, J.S., J.W. Scanlon, H.G. Duffy, J.S. Fukumoto, and L.M. Vogt (1969) Federal Evaluation Policy. Washington, DC: Urban Institute.
75.
Williams, S.R. and J.A. Wysong (1975) "The uses of research in national health policy: an assessment and agenda." Medical Care13, 3 (March): 256-267.
76.
Windle, C. and P. Bates (1974) "Evaluating program evaluation: a suggested approach," in P. O. Davidson et al. (eds.) Evaluation of Behavioral Programs. Champaign, IL : Research Press.
77.
Young, C.J. and J. Comptois (1979) "Increasing congressional utilization of evaluation," in F. Zweig (ed.) Evaluation in Legislation. Beverly Hills: Sage.
78.
Zaltman, G., R. Duncan, and J. Holbeck (1973) Innovations and Organizations. New York: John Wiley.
