Responding to the call for more research on the practice of evaluation and its link to theory, this article uses the interviews conducted with exemplary evaluators or exemplars in the American Journal of Evaluation as cases to describe and analyze the practice of exemplary evaluators. Dimensions of analysis included context, purpose, advance organizers, breadth and depth of stakeholder involvement, method proclivity, and use of results. Results showed that on these cases, evaluators differed on purpose, advance organizers, and choices on breadth or depth of stakeholder involvement. Determining merit and worth was a frequent and primary purpose for many. All used mixed methods, but varied in the confidence they placed in different methods. Instrumental use was identified in each evaluation. Some evaluators’ practice was congruent with theories; others were more atheoretical. Differences were often nuanced, influenced by both context and evaluator proclivity.
Get full access to this article
View all access options for this article.
References
1.
Alkin, M.C. (1991). Evaluation theory development II. In M. McLaughlin & D. Phillips (Eds.), Evaluation and education at quarter century. Chicago: University of Chicago Press.
2.
Alkin, M. C. (2003). Evaluation theory and practice: Insights and new directions. In C. A. Christie (Ed.), New directions for evaluation, No. 97. The practice-theory relationship in evaluation (pp. 81-89). San Francisco: Jossey-Bass.
3.
Alkin, M. C., & Ellett, F. S., Jr. (1985). Evaluation models and their development. In H. J. Walberg & G. D. Haertel (Eds.), International encyclopedia of education. New York: Pergamon.
4.
Alkin, M. C., & House, E. (1992). Evaluation of programs. In M. C. Alkin (Ed.), Encyclopedia of educational research. New York: Macmillan.
5.
Bickman, L., & Fitzpatrick, J. L. (2002). Evaluation of the Fort Bragg and Stark County systems of care for children and adolescents. A conversation with Leonard Bickman. American Journal of Evaluation, 23, 69-80.
6.
Christie, C. A. (2003). What guides evaluation? A study of evaluation practice maps onto evaluation theory. In C.A. Christie (Ed.), New directions for evaluation, No. 97. The practice-theory relationship in evaluation (pp. 7-35). San Francisco: Jossey-Bass.
7.
Cousins, J. B., & Earl, L. M. (1999). When the boat gets missed: Response to M.F. Smith. American Journal of Evaluation, 20, 309-317.
8.
Datta, L. (2003). Important questions, intriguing method, incomplete answers. In C. A. Christie (Ed.), New directions for evaluation, No. 97. The practice-theory relationship in evaluation (pp. 37-46). San Francisco: Jossey-Bass.
9.
Donaldson, S. I., Gooler, L. E., & Fitzpatrick, J. L. (2002). Theory-driven evaluation of the Work and Health Initiative: A focus on Winning New Jobs. Dialogue with Stewart Donaldson. American Journal of Evaluation, 23, 341-365.
10.
Fetterman, D. (2003). Fetterman-House: A process use distinction and a theory. In C. A. Christie (Ed.), New directions for evaluation, No. 97. The practice-theory relationship in evaluation (pp. 47-52). San Francisco: Jossey-Bass.
11.
Fetterman, D., & Fitzpatrick, J. L. (2000). Summary of the STEP evaluation. Dialogue with David Fetterman. American Journal of Evaluation, 21, 239-259.
12.
Fitzpatrick, J. L. (1997). Exemplary evaluations. Evaluation Practice, 18, 239-240.
13.
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2003). Program evaluation: Alternative approaches and practical guidelines (3rd ed.). Boston: Pearson.
14.
Greene, J. C., & Fitzpatrick, J. L. (2001). Evaluation of the Natural Resources Leadership Program, 1995 through 1998. Conversation with Jennifer C. Greene. American Journal of Evaluation, 22, 81-96.
15.
Henry, G. T., & Fitzpatrick, J. L. (2000). The Georgia Council for School Performance and its performance monitoring system. Dialogue with Gary Henry. American Journal of Evaluation, 21, 105-117.
16.
Henry, G. T. (2000). Why not use? In V. J. Caracelli & H. Preskill (Eds.), New directions for evaluation, No. 88. The expanding scope of evaluation use (pp. 85-98). San Francisco: Jossey-Bass.
17.
House, E. R. (2003). Stakeholder bias. In C. A. Christie (Ed.), New directions for evaluation, No. 97. The practice-theory relationship in evaluation (pp. 53-56). San Francisco: Jossey-Bass.
18.
King, J. A. (2003). The challenge of studying evaluation theory. In C. A. Christie (Ed.), New directions for evaluation, No. 97. The practice-theory relationship in evaluation (pp. 57-67). San Francisco: Jossey-Bass.
19.
King, J. A., & Stevahn, L. (2002). Three frameworks for considering evaluator role. In K. E. Ryan & T. A. Schwandt (Eds.), Exploring evaluator role and identity. Greenwich, CT: Information Age Publishing.
20.
Love, A. J. (1991). Internal evaluation: Building organizations from within. Newbury Park, CA: Sage.
21.
Mark, M. M. (2002). Toward a comprehensive view of the theory and practice of program and policy evaluation. In S. I. Donaldson & M. Scriven (Eds.), Evaluating social programs and problems: Visions for the new millennium. Mahwah, NJ: Erlbaum.
22.
Mark, M. M., Henry, G. T., & Julnes, G. (2000). Evaluation: An integrated framework for understanding, guiding, and improving policies and programs. San Francisco: Jossey-Bass.
23.
Mathison, S. (1999). Rights, responsibilities, and duties: A comparison of ethics for internal and external evaluators. In J. L. Fitzpatrick & M. Morris (Eds.), New directions for evaluation, No. 82. Current and emerging challenges in evaluation (pp. 25-34). San Francisco: Jossey Bass.
24.
Mueller, M. R., & Fitzpatrick, J. L. (1998). The evaluation of Minnesota’s Early Childhood Family Education Program. Dialogue with Marsha Mueller. American Journal of Evaluation, 19, 80-99.
25.
Riccio, J., & Fitzpatrick, J. L. (1997). MDRC’s evaluation of GAIN: A summary. Dialogue with James Riccio. Evaluation Practice, 18, 239-252.
26.
Rog, D. J., & Fitzpatrick, J. L. (1999). The evaluation of the Homeless Families Program. Dialogue with Debra J. Rog. American Journal of Evaluation, 20, 558-575.
27.
Scriven, M. (1996). Types of evaluation and types of evaluator. Evaluation Practice, 17, 151-162.
28.
Shadish, W. R., Jr., Cook, T. D., & Leviton, L. D. (1991). Foundations of program evaluation. Thousand Oaks, CA: Sage.
29.
Sonnichsen, R. C. (2000). High impact internal evaluation. Thousand Oaks, CA: Sage.
30.
Stake, R. E. (1975). Evaluating the arts in education: A responsive approach. Columbus, OH: Merrill.
31.
Stufflebeam, D. L. (2001). New directions for evaluation, No. 89. Evaluation models. San Francisco: Jossey-Bass.
32.
Worthen, B. R. (1990). Program evaluation. In H. J. Walberg & G. D. Haertel (Eds.), International encyclopedia of educational evaluation. New York: Pergamon.
