Collaborative Evaluation in North America: Evaluators' Self-reported Opinions,Practices and Consequences1

Abstract

Despite growing theoretical interest in collaborative and participatory forms of evaluation, little is known about evaluators' perceptions about such evaluation practices or their views about its viability. This article reports results from a survey of Canadian and American evaluators concerning such perceptions. Five-hundred and sixty-four evaluators responded to questions about their views and opinions of collaborative evaluation and a subsample of 348 (61.7%) also described a specific collaborative evaluation project in which they had recently participated. The survey results suggested that evaluators support a utilization-focused, stakeholder-servicearientation to the role and believe that the evaluator has a primary responsibility of maximizing intended use for intended users. Reported practices suggest that most evaluators engaged in collaborative activities that resemble the conventional stakeholder-based evaluation approach. These data are neither able to, nor are they intended tosupport one side or the other in the professional debate about whether evaluators ought to embrace collaborative evaluation as a legitimate direction within the profession. Rather, they add to the empirical knowledge base concerning this type of evaluation.

Get full access to this article

View all access options for this article.

References

Alkin, M.C. (1991). Evaluation theory development: II In M. W. McLaughlin, & D. C. Phillips (Eds.), Evaluation and education: At quarter century, (pp. 91-112). Chicago, IL: The University of Chicago Press.

Altschuld, J.W. , Yoon, J.S. , & Cullen, C. (1993). The utilization of needs assessment results Evaluation and Program Planning, 16, 279-285.

Brandon, P.R. (1994, November). Involving program beneficiaries in reviews of evaluators' recommendations for curriculum

Paper presented at the American Evaluation Association Annual Meeting, Boston, MA

Brunner, I. , & Guzman, A. (1989). Participatory evaluation: A tool to assess projects and empower people In R. F. Connor, & M. H. Hendricks (Eds.), New directions for program evaluation: International innovations in evaluation methodology. (No. 42, pp. 9-17). San Francisco: Jossey-Bass.

Bryk, A. (Ed.) (1983). Stakeholder-based evaluation. New Directions in Program Evaluation, 17, San Francisco: Jossey-Bass.

Cousins, J.B. , Donohue, J.J. , & Bloom, G.A. (1995, November). Collaborative evaluation: Survey of practice in North America. Monograph presented at the joint meeting of the Canadian Evaluation Society and the American Evaluation Association, Vancouver, BC. (Submitted to Education Resources Information Centre).

Cousins, J.B. , & Earl, L.M. (1992). The case for participatory evaluation Educational Evaluation and Policy Analysis, 14(4), 397-418.

Cousins, J. B., & Earl, L. M. (Eds.). (1995). Participatory evaluation in education: Studies of evaluation use and organizational learning. London: Falmer.

Cousins, J.B. , & Leithwood, K.A. (1993). Enhancing knowledge utilization as a strategy for school improvement Knowledge: Creation, Diffusion, Utilization, 14(3), 305-333.

10.

Fetterman, D.M. (1994). Empowerment evaluation Evaluation Practice, 15(1), 1-15.

11.

Fetterman, D.M. (1995). In response Evaluation Practice , 16(2), 179-199.

12.

Garaway, G.B. (1995). Participatory evaluation Studies in Educational Evaluation, 21, 85-102.

13.

Green, L.W. , George, M.A. , Daniel, M. , Frankish, C.J. , Herbert, C.J. , Bowie, W.R. , & O'Neill, M. (1995). Study of participatory research in health promotion Vancouver, BC: Royal Society of Canada.

14.

Greene, J.G. (1988). Stakeholder participation and utilization in program evaluation Evaluation Review, 12(2), 91-116.

15.

Greene, J. (1994, November). Specific practices that facilitate action

Paper presented at the at the annual meeting of American Evaluation Association, Boston

16.

Huberman, M. , & Cox, P. (1990). Evaluation utilization: Building links between action and reflection Studies in Educational Evaluation, 16, 157-179.

17.

Jenlink, P.M. (1994). Using evaluation to understand the learning architecture of an organization Evaluation and Program Planning , 17(3), 315-325.

18.

Johnson, R.B. (1995). Estimating an evaluation utilization model using conjoint measurement and analysis Evaluation Review , 19(3), 313-338.

19.

King, J.A. (1995). Involving practitioners in evaluation studies: How viable is collaborative evaluation in schools? In J. B. Cousins, & L. M. Earl (Eds.), Participatory evaluation in education: Studies in evaluation use and organizational learning (pp. 86-102). London : Falmer.

20.

Larsen, J.K. (1985). Effect of time on information utilization Knowledge: Creation, Diffusion, Utilization, 7, 143-158.

21.

Levin, B. (1993). Collaborative research in and with organizations Qualitative Studies in Education, 6(4), 331-340.

22.

MacDonald, B. (1976). Evaluation and the control of education In D. A. Tawney (Ed.), Curriculum evaluation today: Trends and implications. London: MacMillan (Schools Council Research Studies).

23.

Mark, M. , & Shotland, R.L. (1985). Stakeholder-based evaluation and value judgments Evaluation Review , 9, 605-626.

24.

Mathison, S. (1994). Rethinking the evaluator role: Partnerships between organizations and evaluators Evaluation and Program Planning , 17(3), 299-304.

25.

McTaggart, R. (1991). When democratic evaluation doesn't seem democratic Evaluation Practice, 12(1), 9-21.

26.

Owen, J.M. , & Lambert, F.C. (1995). Roles for evaluators in learning organizations Evaluation, 1(2), 237-250.

27.

Patton, M.Q. (1986). Utilization-focused evaluation (2nd ed.). Newbury Park, CA: Sage.

28.

Patton, M.Q. (1994). Developmental evaluation Evaluation Practice, 15(3), 311-319.

29.

Preskill, H. (1994). Evaluation's role in enhancing organizational learning Evaluation and Program Planning , 17(3), 291-297.

30.

Shadish, W.R. , & Epstein, R.A. (1987). Patterns of program evaluation practice among members of the Evaluation Research Society and Evaluation Network Evaluation Review, 11(5),555- 590.

31.

Shapiro, J.P. (1988). Participatory evaluation: Towards a transformation of assessment for women's' studies programs and projects Educational Evaluation and Policy Analysis, 10(3), 191- 199.

32.

Shulha, L. , & Cousins, J.B. (1995, November). Utilization and social justice: Interconnections of meta-evaluation frameworks

Paper presented at the joint meeting of the Canadian Evaluation Society and the American Evaluation Association, Vancouver, BC

33.

Stake, R.E. (1975). To evaluate an arts program In R. E. Stake (Ed.), Evaluating arts education: A responsive approach. Columbus OH: Charles E. Merrill.

34.

Stufflebeam, D. (1994). Empowerment evaluation, objectivist evaluation, and evaluation standards: Where the future of evaluation should not go and where it needs to go Evaluation Practice , 15(3 ), 321-338.

35.

Torres, R.T. , Preskill, H. , & Piontek, M. (1996). Evaluation strategies for communicating and reporting : Enhancing learning in organizations. Newbury Park : Sage.