In this paper, the authors describe an approach to evaluating large-scale initiatives that span multiple project sites and respond to multiple evaluation needs at the project and program levels. Known as 'cluster evaluation' by those working with and for the W. K. Kellogg Foundation, a private philanthropic institution in the US, this approach encounters problems and offers solutions that are similar to other 'devolved' evaluation strategies for dispersed programs in the UK and several other European countries. This paper positions cluster evaluation in the literature and practice of evaluation by (I) describing cluster evaluation's key features, (2) relating it to core concepts in the evaluation field, (3) distinguishing it from other forms of multi-site evaluation, and (4) outlining a number of conceptual challenges that need to be resolved for the approach to achieve broader applicability and success.
Get full access to this article
View all access options for this article.
References
1.
Barley, Z.A. and M. Jenness (1993) 'Cluster Evaluation: A Method to Strengthen Evaluation in Smaller Programs with Similar Purposes', Evaluation Practice14(2): 141-7.
2.
Campbell, J.L. (1994) 'Emerging Issues in Cluster Evaluation: Issues of Cluster Evaluation Use'. Paper presented at the Annual Meeting of the American Evaluation Association, Boston; MA.
3.
Cottingham, P.H. (1991) 'Unexpected Lessons: Evaluation of Job-training Programs for Single Mothers', in R. S. Turpin and J. M. Sinacore (eds) Multisite Evaluations. New Directions for Program Evaluation series, No. 50, pp. 59-70. San Francisco, CA: Jossey-Bass.
4.
Glass, G.V., M.L. Byers and B.R. Worthen (1972) 'Recommendations for the Evaluation of Experimental Schools Projects of the U.S. Office of Education'. Final report (Grant No. OEG0-9-152075-3720. Boulder, CO: University of Colorado Laboratory of Educational Research.
5.
Hedrick, S.C., J.H. Sullivan, J.L. Ehreth, M.L. Rothman, R.T. Conms and W.W. Erdly (1991) 'Multiple Sites in Evaluation Research: A Survey of Organizational and Methodological Issues', in R. S. Turpin and J. M. Sinacore (eds) Multisite Evaluations. New Directions for Program Evaluation series, No. 50, pp. 19-31. San Francisco, CA: Jossey-Bass.
6.
Henry, R. (1992) 'A Delicate Balance: the Role of the Cluster Evaluator in Relation to the Foundation Project Directors'. Paper presented at the Annual Meeting of the American Evaluation Association , Seattle, WA.
7.
Jenness, M. and Z.A. Barley (1992) 'A Constructivist Approach to Cluster Evaluation' . Paper presented at the Annual Meeting of the American Evaluation Association, Seattle, WA.
8.
Jenness, M. and Z.A. Barley (1993) 'Light at the End of the Tunnel: Processes and Results of Cluster Evaluation'. Paper presented at the Annual Meeting of the American Evaluation Association, Dallas, TX.
9.
Johnson, R.M. (1993) 'A Model for Foundation Programs: The Kellogg Approach to Evaluating Clusters of Grants', in Council on Foundations, Evaluation for Foundations. San Francisco, CA: Jossey-Bass.
10.
Kellogg Foundation (1991) Information on Cluster Evaluation. Battle Creek, MI: Author.
11.
Kellogg Foundation (1993) Cluster Evaluation Information. Battle Creek, MI: Author.
12.
Kellogg Foundation (1994) Improving Cluster Evaluation: Some Areas for Consideration. Battle Creek, MI: Author.
13.
Kellogg Foundation (1996) 'Learning from Cluster Evaluation: a Synthesis of Observations and a Discussion of Implications' . Compiled contributions of cluster evaluators participating in the 1996 Annual Conference on cluster evaluation. Battle Creek, MI: Author.
14.
Mansberger, N.B. (1993) 'Making Sense out of a Mixed Bag of Data: the Use of Working Hypotheses'. Paper presented at the Annual Meeting of the American Evaluation Association, Dallas, TX.
15.
O'Sullivan, R.G. and J.M. O'Sullivan (1994) 'Evaluation Voices: Promoting Cluster Evaluation from within Programs'. Paper presented at the Annual Meeting of the Canadian Evaluation Society, Quebec, Canada.
16.
Parsons, B.A. (1994) 'Relationship and Communication Issues posed by Cluster Evaluation'. Paper presented at the Annual Meeting of the American Evaluation Association, Boston, MA.
17.
Pearl, J.B. and A. Rubino (1993) 'The Power of Cluster Evaluation Networking Conferences' . Paper presented at the Annual Meeting of the American Evaluation Association, Dallas, TX.
18.
Schmitz, C.C. (1995) 'Implications of Parallel Evaluation Systems for Formative and Summative Evaluation'. Paper presented at the International Evaluation Conference, sponsored by the Canadian Evaluation Society and the American Evaluation Association, 3 November, Vancouver, Canada.
19.
Seefeldt, M. (1992) 'Conflicts in Conceptual Framework, Design, Audience Expectation, and Evaluator Style'. Paper presented at the Annual Meeting of the American Evaluation Association, Seattle, WA.
20.
Smacore, J.M. and R.S. Turpin (1991) 'Multiple Sites in Evaluation Research: a Survey of Organizational and Methodological Issues', in R. S. Turpin and J. M. Sinacore (eds) Multisite Evaluations. New Directions for Program Evaluation series, No. 50. San Francisco, CA: Jossey-Bass.
21.
Stern, E. (1995) 'The Devolved Evaluation of Dispersed Programmes' . Paper presented at the International Evaluation Conference, sponsored by the Canadian Evaluation Society and the American Evaluation Association, November, Vancouver, Canada.
22.
Weiss, C.H. (1975) 'Evaluation Research in the Political Context' , in E. L. Struening and M. Guttentag (eds) Handbook of Evaluation Research (Vol. 1), pp. 13-26. Beverly Hills, CA: Sage.
23.
Wiseley, D.S. (1982) An Introduction to Foundation Evaluation. Indianapolis, IN: The Lilly Endowment, Inc.
24.
Worthen, B.R. (1995) 'Coercion, Conversion, or Convergence: Alternative Approaches to Developing Collaborative and Cooperative Relationships in Cluster Evaluation'. Paper presented at the International Evaluation Conference, sponsored by the Canadian Evaluation Society and the American Evaluation Association, 3 November, Vancouver , Canada.
25.
Worthen, B.R. and G.I. Latham (1982) 'An Evaluation of the Operation and Effects of Title IV Expenditures in Utah: 1980-82'. Logan, UT: Western Institute for Research and Evaluation. (Final summary report of three-year evaluation of Title IV programs, 2 vols, 527 pp.)
26.
Worthen, B.R. and A. Matsumoto (1994) 'Conceptual Challenges Confronting Cluster Evaluation' . Paper presented at the Annual Meeting of the American Evaluation Association, Boston, MA.
27.
Worthen, B.R., J.R. Sanders and J.L. Fitzpatrick (1997) Educational Evaluation: Alternative Approaches and Practical Guidelines. New York: Longman.
28.
Wunderlich, M. and C.C. Schmitz (1994) 'The Development of Standardized Indicators across Multiple and Diverse Consortia: The Kellogg Community-Based Public Health Cluster Evaluation'. Paper presented at the Annual Meeting of the American Evaluation Association, Boston, MA.
