What is Essential in Developmental Evaluation? On Integrity,Fidelity,Adultery,Abstinence,Impotence,Long-Term Commitment,Integrity,and Sensitivity in Implementing Evaluation Models
Restricted accessResearch articleFirst published online June, 2016
What is Essential in Developmental Evaluation? On Integrity,Fidelity,Adultery,Abstinence,Impotence,Long-Term Commitment,Integrity,and Sensitivity in Implementing Evaluation Models
Fidelity concerns the extent to which a specific evaluation sufficiently incorporates the core characteristics of the overall approach to justify labeling that evaluation by its designated name. Fidelity has traditionally meant implementing a model in exactly the same way each time following the prescribed steps and procedures. The essential principles of developmental evaluation (DE), in contrast, provide high-inference sensitizing guidance that must be interpreted and applied contextually. In lieu of operationalizing DE fidelity criteria, I suggest addressing the degree of manifest sensitivity to essential principles. Principles as sensitizing concepts replace operational rules. This means that sensitivity to essential DE principles should be explicitly and contextually manifest in both processes and outcomes, in both design and use of findings. Eight essential principles of DE are identified and explained. Finally, 10 threats to evaluation model fidelity and/or degree of manifest sensitivity are identified with ways to mitigate those threats.
BlumerH. (1954). What is wrong with social theory?American Sociological Review, 19, 3–10.
2.
ChilisaB. (2012). Indigenous research methodologies. Thousand Oaks, CA: Sage.
3.
CorynC. L. S.NoakesL. A.WestineC. D.SchröterD. C. (2011). A systematic review of theory-driven evaluation practice from 1990 to 2009. American Journal of Evaluation, 32, 199–226.
4.
CousinsJ. B. (2005). Will the real empowerment evaluation please stand up? In FettermanD. M.WandersmanA. (Eds.), Empowerment evaluation principles in practice (pp. 183–208). New York, NY: Guilford Press.
5.
CousinsJ. B.ChouinardJ. A. (2012). Participatory evaluation up close: An integration of research-based knowledge. Charlotte, NC: Information Age Publishing.
6.
CousinsJ. B.WhitmoreE.ShulhaL. (2014). Arguments for a common set of principles for collaborative inquiry in evaluation. American Journal of Evaluation, 34, 7–22.
7.
CruickshankD. R.KennedyJ. J. (1986). Teacher clarity. Teaching and Teacher Education, 2, 43–67.
8.
DaigneaultP. M.JacobS. (2009). Toward accurate measurement of participation: Rethinking the conceptualization and operationalization of participatory evaluation. American Journal of Evaluation, 30, 330–348.
9.
DicksonR.SaundersM. (2014). Developmental evaluation: Lessons for evaluative practice from the SEARCH program. Evaluation, 20, 176–194.
10.
DonaldsonS. I.PattonM. Q.FettermanD.ScrivenM. (2010). The 2009 Claremont Debates: The promise and pitfalls of utilization-focused and empowerment evaluation. Journal of Multidisciplinary Evaluation, 6, 15–57.
11.
FettermanD. M. (2005). A window into the heart and soul of empowerment evaluation. In FettermanD.WandersmanA. (Eds.), Empowerment evaluation principles in practice (pp. 1–26). New York, NY: Guilford Press.
12.
FettermanD. M.ShakehJ.KaftarianS. J.WandersmanA. H. (2014). Empowerment evaluation: Knowledge and tools for self-assessment, evaluation capacity building, and accountability. Thousand Oaks, CA: Sage.
13.
FettermanD. M.WandersmanA.KaftarianS. (2015). Empowerment evaluation is a systematic way of thinking: A response to Michael Patton. Evaluation and Program Planning, 52, 10–14.
14.
FSG. (2014). Next generation evaluation: Embracing complexity, connectivity, and change. Stanford Social Innovation Review. Retrieved fromwww.ssir.org/nextgenevaluation
15.
GoldsteinR. N. (2015). What philosophers really know. The New York Review of Books, LXII, 48–50.
16.
GundersonL. H.HollingC. S. (Eds.). (2002). Panarchy: Understanding transformations inhuman and natural system. Washington, DC: Island Press.
17.
JacobS.DesautelsG. (2014). Assessing the quality of Aboriginal program evaluations. Canadian Journal of Program Evaluation, 29, 62–86.
18.
KolataG. (2006a, 718). Race to the swift? Not necessarily. New York Times, Health and Fitness Section, p. 1.
19.
KolataG. (2006b, 1214). Study questions colonoscopy effectiveness. New York Times, p. A36.
20.
LamC. Y.ShulhaL. M. (2014). Insights on using developmental evaluation for innovating: A case study on the cocreation of an innovative program. American Journal of Evaluation, 36, 358–374.
21.
MacCoyD. J. (2014). Appreciative inquiry and evaluation – Getting to what works. Canadian Journal of Program Evaluation, 29, 104–127.
22.
MillerR. L.CampbellR. (2006). Taking stock of empowerment evaluation: An empirical review. American Journal of Evaluation, 27, 296–319.
23.
MurrayH. G. (2007). Low inference teaching behaviors and college effectiveness. In PerryR. P.SmartJ. C. (Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 145–183). Dordrecht, The Netherlands: Springer.
PreskillH.CatsambasT. T. (2006). Reframing evaluation through appreciative inquiry. Thousand Oaks, CA: Sage.
34.
RosenshineB.FurstN. (1971). Research on teacher performance criteria. In SmithB. (ed.), Research in teacher education (pp. 37–72). Englewood Cliffs, NJ: Prentice Hall.
35.
SchwandtT. A. (2001). Dictionary of qualitative inquiry (2nd Rev. ed.). Thousand Oaks, CA: Sage.
36.
ScrivenM. (1967). The methodology of evaluation. In TylerR. W.GagneR. M.ScrivenM. (Eds.), Perspectives of curriculum evaluation (AERA Monograph series on Curriculum Evaluation, 1, pp. 39–83). Chicago, IL: Rand McNally.
37.
WilliamsB. (2005). Systems and systems thinking. In MathisonS. (Ed.), Encyclopedia of evaluation (pp. 405–412). Thousand Oaks, CA: Sage.
38.
WilliamsB. (2008). Systemic inquiry. In GivenL. M. (Ed.), The SAGE encyclopedia of qualitative research methods (Vol. 2, pp. 854–859). Thousand Oaks, CA: Sage.
WilliamsB.HummelbrunnerR. (2011). Systems concepts in action: A practitioner’s toolkit. Stanford, CA: Stanford University Press.
41.
WilliamsB.ImanI. (2007). Systems concepts in evaluation: An expert anthology (American Evaluation Association Monograph No. 6). Point Reyes, CA: EdgePress.
42.
WilliamsB.van ‘t HofS. (2014). Wicked solutions: A systems approach to complex problems. Wellington, New Zealand: Bob Williams. Retrieved fromwww.bobwilliams.co.nz/wicked.pdf
43.
WilliamsM. (2004). “Operationism/Operationalism.” In Lewis-BeckM. S.BrymanA.Futing LiaoT. (Eds.), The Sage Encyclopedia of Social Science Research Methods (pp. 768–778). Thousand Oaks, CA: Sage.
