Attention to evaluation quality is commonplace, even if sometimes implicit. Drawing on her 2010 Presidential Address to the American Evaluation Association, Leslie Cooksy suggests that evaluation quality depends, at least in part, on the intersection of three factors: (a) evaluator competency, (b) aspects of the evaluation environment or context, and (c) the level and nature of supportive resources that are available in the evaluation community. In a brief reaction to her views, Mel Mark discusses selected implications of Cooksy’s approach and comments on the notion of evaluation quality itself.
American Evaluation Association Evaluation Policy Task Force. (2008, March). Comments on what constitutes strong evidence of a program’s effectiveness?Retrieved fromhttp://www.eval.org/aea08.omb.guidance.responseF.pdf
ChambersR.KarlanD.RavallionM.RogersP. (2009). Designing impact evaluations: Different perspectives. Working Paper 4. New Delhi, India: International Initiative for Impact Evaluation. Retrieved fromhttp://www.monitoreoyevaluacion.info/biblioteca/MVI_115.pdf
7.
ChelimskyE. (1987). The politics of evaluation. Society, 25, 24–32. doi:10.1007/BF02695393
8.
Chelimsky, E. (2009). Integrating evaluation units into the political environment of government: The role of evaluation policy. In W. M. K. Trochim, M. M. Mark, & L. J. Cooksy (Eds.), Evaluation policy and evaluation practice. New directions for evaluation (Vol. 123, pp. 51–66). doi:10.1002/ev.305
9.
CooksyL. J.CaracelliV. J. (2009). Metaevaluation in practice: Selection and application of criteria. Journal of MultiDisciplinary Evaluation, 6, 1–15.
10.
CronbachL. J. (1980). Toward reform of program evaluation. San Francisco, CA: Jossey-Bass.
Gawande, A. (2009). The checklist manifesto: How to get things right. New York: Metropolitan Books.
13.
Guba, E. G. & Lincoln, Y. S. (2001). Guidelines and checklist for constructivist (a.k.a. Fourth Generation) evaluation. Retrieved from http://www.wmich.edu/evalctr/archive_checklists/constructivisteval.pdf
14.
HouseE. R. (2008). Blowback: Consequences of evaluation for evaluation. American Journal of Evaluation, 29, 416–426. doi:10.1177/1098214008322640
15.
House, E. R. & Howe, K. R. (2000). Deliberative democratic evaluation checklist. Retrieved from http://www.wmich.edu/evalctr/archive_checklists/dd_checklist.PDF
16.
LeeuwF.VaessenJ. (2009). Impact evaluations and development: NONIE guidance on impact evaluation. Washington, DC: Network of Networks on Impact Evaluation.
17.
LincolnY. S.GubaE. G. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage.
18.
MarkM. M. (2006). The consequences of evaluation: Theory, research, and practice. St. Louis, MO: Presidential address presented at the annual meeting of the American Evaluation Association.
19.
PalumboD. (Ed.). (1987). The politics of program evaluation. Thousand Oaks, CA: Sage.
SchwandtT. A. (2008). Educating for intelligent belief in evaluation. American Journal of Evaluation, 29, 139–150. doi:10.1177/1098214008316889
22.
SchwandtT. A.HalpernE. S. (1988). Linking auditing and metaevaluation: Enhancing quality in applied research. Applied Social Research Methods Series, Volume 11. Thousand Oaks, CA: Sage.
23.
ScrivenM. (1996). Types of evaluation and types of evaluator. American Journal of Evaluation, 17, 151–161. doi:10.1177/109821409601700207
24.
StevahnL.KingJ. A.GhereG.MinnemaJ. (2005). Establishing essential competencies for program evaluators. American Journal of Evaluation, 26, 43–59. doi:10.1177/1098214004273180
25.
Stufflebeam, D. L. (1999). Program evaluation metaevaluation checklist. Retrieved from http://www.wmich.edu/evalctr/checklists/program_metaeval.pdf
26.
Stufflebeam, D. L., Goodyear, L., Marquart, J., & Johnson, E. (2005). Guiding principles checklist for evaluating evaluations. Retrieved from http://www.wmich.edu/evalctr/archive_checklists/guidingprinciples2005.pdf
27.
TrochimW. M. K. (2009). Evaluation policy and evaluation practice. New Directions for Evaluation, 123, 13–32. doi:10.1002/ev.303
28.
U.S. Government Accountability Office. (2005). Performance budgeting: PART focuses attention on program performance, but more can be done to engage Congress (GAO-06–28). Washington, DC: Author.
29.
U.S. Government Accountability Office. (2007). Government auditing standards (GAO-07–162G). Washington, DC: Author.
30.
YarbroughD. B.ShulhaL. M.HopsonR. K.CaruthersF. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Thousand Oaks, CA: Sage.
