As theory-based evaluation (TBE) engages in situations where multiple stakeholders help develop complex program theory about dynamic phenomena in politically contested settings, it becomes difficult to develop and use program theory without ambiguity. The purpose of this article is to explore ambiguity as a fruitful perspective that helps TBE face current challenges. Literatures in organization theory and political theory are consulted in order to cultivate the concept of ambiguity. Janus variables (which work in two ways) and other ambiguous aspects of program theories are classified and exemplified. Stances towards ambiguity are considered, as are concrete steps that TBE evaluators can take to identify and deal with ambiguity in TBE.
AlkinM. C. (2004). Evaluation roots. Thousand Oaks, CA: Sage.
2.
BaierV.MarchJ. G.SætrenH. (1986). Implementation and ambiguity. Scandinavian Journal of Management Studies, 2, 179–212.
3.
Balle HansenM.VedungE. (2010). Theory-based stakeholder evaluation. American Journal of Evaluation, 31, 295–313.
4.
BehnR. D. (2003). Why measure performance? Different purposes require different measures. Public Administration Review, 63, 586–606.
5.
BergerP.KellnerH. (1981). Sociology reinterpreted: An essay on method and vocation. New York, NY: Doubleday.
6.
BestJ. (2008). Ambiguity, uncertainty, and risk: Rethinking indeterminacy. International Political Sociology, 2, 355–374.
7.
BickmanL. (1987). Using program theory in evaluation. New directions for program evaluation (Series No. 33). San Francisco, CA: Jossey-Bass.
8.
BirckmayerJ. D.WeissC. H. (2000). Theory-based evaluation in practice: What do we learn?Evaluation Review, 24, 407–431.
9.
ChenH. (1990). Theory-driven evaluations. Newbury Park, CA: Sage.
10.
ChenH. (2005). Program theory. In MathisonS. (Ed.), Encyclopedia of evaluation (pp. 340–342). Thousand Oaks, CA: Sage.
11.
CorynC. L. S.NoakesL. A.WestineC. D.SchröterD. C. (2011). A systematic review of theory-driven evaluation practice from 1990 to 2009. American Journal of Evaluation, 32, 199–226.
12.
CousinsB. J. (2003). Utilization effects of participatory evaluation. In KellaghanT.StufflebeamD. L.WingateL. A. (Eds.), International handbook of educational evaluation. Part One: Perspectives (pp. 245–268). Dordrecht, the Netherlands: Kluwer Academic.
13.
Dahler-LarsenP. (2001). From programme theory to constructivism: On tragic, magic, and competing programmes. Evaluation, 7, 331–349.
14.
DonaldsonS. I. (2007). Program theory-driven evaluation science: Strategies and applications. New York, NY: Lawrence Erlbaum Associates.
15.
DonaldsonS. I.LipseyM. W. (2006). Roles for theory in contemporary evaluation practice: Developing practical knowledge. In ShawI.GreeneJ. C.MarkM. M. (Eds.), The handbook of evaluation: Policies, programs, and practices (pp. 56–75). London, England: Sage.
16.
EmanuelE. J.EmanuelL. L. (1992). Four models of the physician-patient relationship. Journal of the American Medical Association, 267, 2221–2226.
17.
FitzpatrickJ. L.SandersJ. R.WorthenB. R. (2004). Program evaluation: Alternative approaches and practical guidelines (Vol. 3). Boston, MA: Pearson Education.
18.
ForssK.RebienC. C.CarlssonJ. (2002). Process use of evaluations: Types of use that precede lessons learned and feedback. Evaluation, 8, 29–45.
19.
FreedenM. (2005). What should the “political” in political theory explore?The Journal of Political Philosophy, 13, 113–134.
20.
FriedmanV. J. (2001). Designed blindness: An action science perspective on programme theory evaluation. American Journal of Evaluation, 22, 161–181.
21.
FunnellS. C.RogersP. J. (2011). Purposeful program theory: Effective use of theories of change and logic models. San Francisco, CA: Jossey-Bass.
22.
GallieW. B. (1956). Essentially contested concepts. Proceedings of the Aristotelian Society, 56, 167–198.
23.
GarganiJ. (2001). The challenge of evaluating theory-based evaluation. Paper presented at AEA conference in St. Louis, MO.
24.
HasenfeldY. (1982). Human service organizations. Boston, MA: Allyn & Bacon.
25.
JulnesG. E. (2012). Promoting valuation in the public interest: Informing policies for judging value in evaluation. Special Issue. New Directions for Evaluation, 133, 109–129.
26.
KahnemanD. (2011). Thinking, fast and slow. New York, NY: Farrar, Straus and Giroux.
27.
LedermannS. (2012). Exploring the necessary conditions for evaluation use in program change. American Journal of Evaluation, 33, 159–178.
28.
LeeuwF. L. (2003). Reconstructing program theories: Methods available and problems to be solved. American Journal of Evaluation, 24, 5–20.
29.
LevinthalD.RerupC. (2006). Crossing an apparent chasm: Bringing mindful and less-mindful perspectives on organizational learning. Organization Science, 17, 502–513.
30.
MarchJ. G.OlsenJ. P. (1976). Ambiguity and choice in organizations. Tromsø, Norway: Universitetsforlaget.
31.
MarkM. M.HenryG.JulnesG. (2000). Evaluation. An integrated framework for understand, guiding and improving public and nonprofit policies and programs. San Francisco, CA: Jossey-Bass.
32.
MayneJ. (2011). Contribution analysis: Addressing cause and effect. In SchwartzR.ForssK.MarraM. (Eds.), Evaluating the complex. New Brunswick, NJ: Transaction.
33.
OvretveitJ. (2003). Evaluation health interventions. Maidenhead, England: Open University Press.
RogersP. J. (2007). Theory-based evaluation: Reflections ten years on. New Directions for Evaluation, 114, 63–67.
38.
RogersP. J. (2008). Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation—The International Journal for Theory, Research and Practice, 14, 29–48.
39.
RogersP. J.PetrosinoA.HuebnerT. A.HacsiT. A. (2000). Program theory evaluation: Practice, promise, and problems. In RogersP. J.HacsiT. A.PetrosinoA.HuebnerT. A. (Eds.), Program theory in evaluation: Challenges and opportunities. New directions for evaluation (pp. 5–13). San Francisco, CA: Jossey-Bass.
40.
RossiP. H.LipseyM. W.FreemanH. E. (2004). Evaluation: A systematic approach. Thousand Oaks, CA: Sage.
41.
RothsteinH.HuberM.GaskellG. (2006). A theory of risk colonization: The spiralling regulatory logics of societal and institutional risk. Economy and Society, 35, 91–112.
42.
ShadishW. R.CookT. D.LevitonL. C. (1991). Foundations of program evaluation: Theories of practice. Newbury Park, CA: Sage.
43.
SimonH. A. (1996). The sciences of the artificial (Vol. 3). Cambridge, MA: MIT Press.
44.
StameN. (2004). Theory-based evaluation and types of complexity. Evaluation, 10, 58–76.
45.
StameN. (2013). A European evaluation theory tree. In AlkinM. C. (Ed.), Evaluation roots—A wider perspective of theorists’ views and influences (p. 355–370).Thousand Oaks, CA: Sage.
46.
StoneD. (2012). Policy paradox (3rd ed.). New York, NY: W. W. Norton.
47.
StufflebeamD. L. (2001). Evaluation models. New Directions for Evaluation, 89, 7–98.
48.
TaylorC. (2016). The language animal: The full shape of the human linguistic capacity. Cambridge, MA: The Belknap Press of Harvard University Press.
49.
TaylorD.BallochS. (2005). The politics of evaluation. Participation and policy implementation. Bristol, England: Policy Press.
50.
Tronegård-MadsenM. (2008). Virkningsevalueringens veje—Erfaringer fra Danmark [Master’s thesis]. Institut for Statskundskab, Syddansk Universitet, Denmark.
51.
VakkariJ.MeklinP. (2006). Ambiguity in performance measurement: A theoretical approach to organisational uses of performance measurement. Financial Accountability & Management, 22, 235–250.
52.
VedungE. (1997). Public policy and program evaluation. New Brunswick, NJ: Transaction.
53.
WarrenM. E. (1999). What is political?Journal of Theoretical Politics, 11, 207–231.
54.
WeickK. E.SutcliffeK. M.ObstfeldD. (2005). Organizing and the process of sense-making. Organization Science, 16, 409–421.
55.
WeissC. H. (1977). Research for policy’s sake: The enlightenment function of social research. Policy Analysis, 3, 531–545.
56.
WeissC. H. (1997a). Theory-based evaluation: Past, present, and future. New Directions for Evaluation, 76, 41–55.
57.
WeissC. H. (1997b). How can theory-based evaluation make greater headway?Evaluation Review, 21, 501–524.
58.
WilliamsM.MayT. (1996). Introduction to the philosophy of social research. New York, NY: Routledge.
