The Design and Selection of Evaluation Studies

Abstract

User involvement in the design and selection of evaluation studies is a widely advocated strategy for increasing their use. In this paper a particular structure (the "user-oriented framework") for guiding such involvement is described . In addition, some tentative propositions are offered concerning the viability of this framework (and, more generally, user involvement in study design and selection). Finally, other strategies are identified that might be employed in conjunction with the user-oriented framework in order to increase its viability in certain contexts

Get full access to this article

View all access options for this article.

References

Ackoff, R.L. , with S.K. Gupta and J.S. Minas (1962) Scientific Method: Optimizing Applied Research Decisions New York: John Wiley.

Agarwala-Rogers, R. (1977) "Why is evaluation research not utilized?" pp. 327-333 in M. Guttentag (ed.) Evaluation Studies Review Annual, Vol. 2. Beverly Hills, CA: Sage.

Anderson, S.B. AND S. Ball (1978) The Profession and Practice of Program Evaluation San Francisco: Jossey-Bass.

Bennis, W.G. (1965) "Commentary on 'The researcher and the manager: a dialectic of implementation." Management Sci. 12: B13-B16.

Brown, R.D. , L.D. Braskamp , AND D.L. Newman (1978) "Evaluator credibility as a function of report style: do jargon and data make a difference?" Evaluation Q. 2: 331-341.

Campbell, D.T. AND J.C. Stanley (1966) Experimental and Quasi-Experimental Designs for Research Chicago: Rand McNally.

Caplan, N. (1979) "The two-communities theory and knowledge utilization." Amer. Behavioral Scientist 22: 459-470.

Connor, R.F. (1980)"Review of Utilization-Focused Evaluation by Michael Quinn Patton." Knowledge 1: 631-634.

Cook, T.D. AND D.T. Campbell (1979) Quasi-Experimentation: Design and Analysis Issues for Field Settings Chicago: Rand McNally.

10.

Dickey, B. , and E. Hampton (1981) "Effective problem-solving for evaluation utilization." Knowledge 2. 361-374.

11.

Dunn, W.N. (1981) Public Policy Analysis: An Introduction Englewood Cliffs, NJ Prentice-Hall.

12.

Egan, J. (1980) "The school that teaches how to run New York " New York 13 (November 3): 14-16

13.

Elmore, R.F. (1975) "Design of the Follow Through experiment," pp 23-45 in A. M Rivlin and P. M Timpane (eds.) Planned Variation in Education. Washington, DC. The Brookmgs Institution

14.

Greenberger, M. , M.A. Crenson , AND B.L. Crissey (1976) Models in the Policy Process New York: Russell Sage Foundation

15.

Guba, E.G. AND Y.S. Lincoln (1981) Effective Evaluation San Francisco . Jossey-Bass.

16.

Hatry, H. , L. Blair , D. Fisk , AND W. Kimmel (1976) Program Analysis for State and Local Governments Washington, DC: The Urban Institute .

17.

Heskett, J.S. (1976) Marketing New York: MacMillan.

18.

Johnson, K.W. (1980) "Stimulating evaluation use by integrating academia and practice " Knowledge 2: 237-262.

19.

Kilmann, R.H. (1981) "Organization design for knowledge utilization." Knowledge 3: 211-231.

20.

---and R.P. Herden (1976) "Towards a systemic methodology for evaluating the impact of interventions on organizational effectiveness." Academy of Management Rev. 1: 87-98.

21.

Kotler, P. (1976) Marketing Management Analysis, Planning and Control Englewood Cliffs, NJ: Prentice-Hall

22.

Lau, A.W. , A.R. Newman , AND L.A. Broedling (1980) "The nature of managerial work in the public sector." Public Administration Rev 40 513-520

23.

Leviton, L.C. AND E.F.X. Hughes (1981) "Research on the utilization of evaluation." Evaluation Rev. 5 525-548

24.

Lin, N. and G. Zaltman (1973) "Dimensions of innovations," pp. 93-115 in G Zaltman (ed.) Processes and Phenomena of Social Change

New York

John Wiley

25.

Lindblom, C.E. and D.K. Cohen (1979) Usable Knowledge Social Science and Social Problem Solving New Haven, CT

Yale University Press

26.

Lynn, L.E., Jr. (1978) "Introduction," pp. 1-11 in L E Lynn, Jr. (ed ) Knowledge and Policy: The Uncertain Connection.

Washington, D.C

National Academy of Sciences

27.

Mandell, M.B. (1984) "Strategies for improving the usefulness of analytic techniques for public sector decision making," pp. 289-310 in L. G. Nigro (ed.) Decision Making in the Public Sector. New York: Marcel Dekker.

28.

Mandell, M.B. (1979) The Development and Field Testing of a Framework for Improving the Effectiveness of Policy Analysis Ph.D. dissertation, Northwestern University.

29.

Mintzberg, H. (1971) "Managerial work: analysis from observation." Management Sci. 18: B97-B110.

30.

Mitroff, I.I. and D.D. Mitroff (1979) "Interpersonal communication for knowledge utilization: putting Freud and Jung back together — again." Knowledge 1: 203-217.

31.

Needleman, M.L. and C.E. Needleman (1974) Guenllas in the Bureaucracy The Community Planning Experiment in the United States New York. John Wiley

32.

Patton, M.Q. (1978) Utilization-Focused Evaluation Beverly Hills, CA: Sage

33.

Polivka, L. and E. Steg (1978)"Program evaluation and policy development bridging the gap " Evaluation Q. 2. 696-707.

34.

Quade, E.S. (1975) Analysis for Public Decisions New York: Elsevier.

35.

Rich, R.F. (1975) "Selective utilization ot social science related information by federal policymakers." Inquiry 13 239-245

36.

Rosenthal, S.R. (1980) "Providing analytic capacity for local urban policy the Boston Tax Policy Project" Knowledge 2 27-58

37.

Rossi, P.H. and H.R. Freeman (1982) Evaluation. A Systematic Approach Beverly Hills, CA

Sage

38.

Rothman, J. (1980) Using Research in Organizations A Guide to Successful Application Beverly Hills, CA

Sage

39.

Schmidt, R.E. , J.W. Scanlon , and J.B. Bell (1979) Evaluability Assessment. Making Public Program Work Better Human Services Monograph Series, 14 Rockville, MD

Project SHARE

40.

Sherman, M.A. , G.J. Rath , J.L. Schofer , and C.W.N. Thompson (1979) "Threats to the validity of emergency medical services evaluation a case study of mobile intensive care units " Medical Care 17 127-138

41.

Sundquist, J.L. (1978) "Research brokerage the weak link," pp 126-144 in L. E Lynn, Jr. (ed ) Knowledge and Policy The Uncertain Connection

Washington, DC

National Academy of Sciences

42.

Szanton, P.L. (1981) Not Well Advised New York: Russell Sage Foundation.

43.

U.S. General Accounting Office (1978) Assessing Social Program Impact Evaluations: A Checklist Approach, Exposure Draft Washington, DC: U.S. General Accounting Office.

44.

Van De Vall, M. and C.A. Bolas (1981) "External vs. internal social policy researchers." Knowledge 2: 461-482.

45.

Weiss, C.H. (1978) "Improving the linkage between social research and public policy," pp. 23-81 in L. E. Lynn, Jr. (ed.) Knowledge and Policy: The Uncertain Connection. Washington, DC : National Academy of Sciences.

46.

Weiss, C.H. (1980) "Knowledge creep and decision accretion" Knowledge 1 381-404

47.

Weiss, C.H. and M.J. Bucuvalas (1980) Social Science Research and Decision-Making New York: Columbia Univ Press

48.

Wholey, J.S. (1979) Evaluation: Promise and Performance Washington, DC The Urban Institute.

49.

Williams, S.R. and J.A. Wysong (1977) "Health services research and health policy formulation. an empirical analysis and a structural solution " J of Health Politics Policy and Law 2: 362-387.