Since the beginning of the 21st century, evaluability assessments have experienced a resurgence of interest. However, little is known about how evaluability assessments have been used to improve future evaluations. In this article, we identify characteristics, challenges, and opportunities of evaluability assessments based on a scoping review of case studies published since 2008 (n = 59). We find that evaluability assessments are increasingly used for program development and evaluation planning. Several challenges are identified: politics of evaluability; ambiguity between evaluability and evaluation, and limited considerations of gender equity and human rights. To ensure relevance, evaluability approaches must evolve in alignment with the fast-changing environment. Recommended efforts to revitalize evaluability assessment practice include the following: engaging stakeholders; clarifying what evaluability assessments entail; assessing program understandings, plausibility, and practicality; and considering cross-cutting themes. This review provides an evidence base of practical applications of evaluability assessments to support future evaluability studies and, by extension, future evaluations.
AkintobiT. H.YanceyE. M.DanielsP.MayberryR. M.JacobsD.BerryJ. (2012). Using evaluability assessment and evaluation capacity-building to strengthen community-based prevention initiatives. Journal of Health Care for the Poor and Underserved, 23(2A), 33–48. https://doi.org/10.1353/hpu.2012.0077
2.
BarnesS. P.WethingtonH.CheungK. (2010). Early assessment initiative using the systematic screening and assessment method: Three case studies. New Directions for Evaluation, 125, 67–93.
3.
BelfordM.RobertsonT.JepsonR. (2017). Using evaluability assessment to assess local community development health programmes: A Scottish case-study. BMC Medical Research Methodology, 17(1), 1–12. https://doi.org/10.1186/s12874-017-0334-4
4.
BrisolaraS.SeigartD. (2012). Feminist evaluation research. In Hesse-BiberS. N. (eds), Handbook of feminist research: Theory and praxis (pp. 290–312). Sage Publications.
5.
Canadian Institutes of Health Research. (2019). How to integrate sex and gender into research. Retrieved February 15, 2019, fromhttp://www.cihr-irsc.gc.ca/e/50836.html
6.
ColquhounH. L.LevacD.O’BrienK. K.StrausS.TriccoA. C.PerrierL.KastnerM.MoherD. (2014). Scoping reviews: Time for clarity in definition, methods, and reporting. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2014.03.013
7.
CrenshawK. (1991). Mapping the margins: Intersectionality, identity politics, and violence against women of color. Stanford Law Review, 43(6), 1241. https://doi.org/10.2307/1229039
8.
D’Ostie-RacineL.DagenaisC.RiddeV. (2013). An evaluability assessment of a West Africa based non-governmental organization’s (NGO) progressive evaluation strategy. Evaluation and Program Planning, 36(1), 71–79. https://doi.org/10.1016/j.evalprogplan.2012.07.002
9.
DagenaisC.PlouffeL.GagneC.ToulouseG.BreaultA.-A.DupontD. (2017). Improving the health and safety of 911 emergency call centre agents: an evaluability assessment of a knowledge transfer strategy. International Journal of Occupational Safety and Ergonomics. 23(1), 50–59.
10.
DaviesR. (2013). Planning evaluability assessments: A synthesis of literature with recommendations (Working Paper No. 40). Department for International Development.
11.
DaviesR.PayneL. (2015). Evaluability assessments: Reflections on a review of the literature. Evaluation, 21(2), 216–231. https://doi.org/10.1177/1356389015577465
12.
DeGroffA.CheungK.Dawkins-LynN.HallM. A.MelilloS.Glover-KudonR. (2015). Identifying promising practices for evaluation: The National Breast and Cervical Cancer Early Detection Program. Cancer Causes and Control, 26(5), 767–774. https://doi.org/10.1007/s10552-015-0538-z
13.
DouthwaiteB.HoffeckerE. (2017). Towards a complexity-aware theory of change for participatory research programs working within agricultural innovation systems. Agricultural Systems, 155, 88–102. https://doi.org/10.1016/j.agsy.2017.04.002
14.
ElliottI. A.ZajacG. (2015). The implementation of circles of support and accountability in the United States. Aggression and Violent Behavior, 25, 113–123. Elsevier B.V. https://doi.org/10.1016/j.avb.2015.07.014
15.
FitzpatrickV. (2015). Evaluability assessment of adolescent pregnancy prevention and sexual health program, be proud! Be responsible! In New York State. State University of New York.
16.
Fletcher. (2015). Addressing gender in impact evaluation: A methods lab publication. Overseas Development Institute.
17.
GalportM.GalportN. (2015). Methodological trends in research on evaluation. New Directions for Evaluation, 148, 17–29. https://doi.org/10.1002/ev.20154
18.
GilchristJ. (2014). Evaluability assessment: Developing a process to determine if a program is ready and able to be evaluated. Simon Fraser University.
19.
HensonA. (2018). Strengthening evaluation research: A case study of an evaluability assessment conducted in a carceral setting. International Journal of Offender Therapy and Comparative Criminology, 62(10). 3185–3200. https://doi.org/10.1177/0306624X17723641
20.
HesterL. L.WilceM. A.GillS. A.DislerS. L.CollinsP.CrawfordG. (2013). Roles of the state asthma program in implementing multicomponent, school-based asthma interventions. Journal of School Health, 83(12), 833–841. https://doi.org/10.1111/josh.12101
21.
HightowerC.CaldwellC. (2010). Shifting sands: Science researchers on Google Scholar, Web of Science, and PubMed, with implications for library collections budgets. Issues in Science and Technology Librarianship, 63. https://doi.org/10.5062/F4V40S4J
22.
HiltonL.JonasW. B. (2017). Claim assessment profile: A method for capturing healthcare evidence in the scientific evaluation and review of claims in health care (SEaRCH). The Journal of Alternative and Complementary Medicine, 23(2), 96–101. https://doi.org/10.1089/acm.2016.0292
23.
HolvoetN.Van EsbroeckD.InbergL.PopelierL.PeetersB.VerhofstadtE. (2018). ‘To evaluate or not: Evaluability study of 40 interventions of Belgian development cooperation. Evaluation and Program Planning, 67, 189–199. Elsevier. May 2017. https://doi.org/10.1016/j.evalprogplan.2017.12.005
24.
HoneycuttS.HermstadA.CarvalhoM. L.ArriolaK. R. J.BallardD.EscofferyC.KeglerM. C. (2017). Practice to evidence: Using evaluability assessment to generate practice-based evidence in rural South Georgia. Health Education and Behavior, 44(3), 454–462. https://doi.org/10.1177/1090198116673360
25.
JoanisseM.StinchcombeA.YaminS. (2010). Evaluability assessment of a national driver retraining program: Are we evaluating in the right lane?Canadian Journal of Program Evaluation, 25(1), 27–50.
26.
LamS.DoddW.WhynotJ.SkinnerK. (2019). How is gender being addressed in the international development evaluation literature? A meta-evaluation. Research Evaluation, 28, 158–168.
27.
LaperrièreH.PotvinL.ZúñigaR. (2012). A socio-political framework for evaluability assessment of participatory evaluations of partnerships: Making sense of the power differentials in programs that involve the state and civil society. Evaluation, 18(2), 246–259. https://doi.org/10.1177/1356389012442445
28.
LevacD.ColquhounH.O’BrienK. K. (2010). Scoping studies: advancing the methodology. Implementation Science: IS, 5, 69. https://doi.org/10.1186/1748-5908-5-69
29.
LevitonL. C.GutmanM. A. (2010). Overview and rationale for the systematic screening and assessment method. New Directions for Evaluation, 125, 7–31. https://doi.org/10.1002/ev.318
30.
LevitonL. C.KhanL. K.RogD.DawkinsN.CottonD. (2010). Evaluability assessment to improve public health policies, programs, and practices. Annual Review of Public Health, 31(1), 213–233. https://doi.org/10.1146/annurev.publhealth.012809.103625
31.
LosbyJ. L.VaughanM.DavisR.Tucker-BrownA. (2015). Arriving at results efficiently: Using the enhanced evaluability assessment approach. Preventing Chronic Disease, 12, 150413.
LuS. K.ElliottS. J.PerlmanC. M. (2017). Evaluability assessment of a small NGO in water-based development. Evaluation, 23(2), 226–241. https://doi.org/10.1177/1356389017697620
34.
LundeenE. A.SiegelK. R.CalhounH.KimS. A.GarciaS. P.HoetingN. M.HarrisD. M.KhanL. K.SmithB.BlanckH. M.BarnettK.HaddixA. C. (2017). Clinical-community partnerships to identify patients with food insecurity and address food needs. Preventing Chronic Disease, 14, 170343. https://doi.org/10.5888/pcd14.170343
35.
McAdamsR.WasonD.AnandN.CraigN.InglisJ. (2017). Evaluability assessment of the expansion of early learning and childcare. National Health Services Scotland.
36.
McKinneyL. (2010). Evaluability assessment: Laying the foundation for effective evaluation of a community college retention program. Community College Journal of Research and Practice, 34(4), 299–317. https://doi.org/10.1080/10668920903527019
37.
NilsonL. G.DolnyL. L.NatalS.de LacerdaJ. T.CalvoM. C. M. (2017). Telehealth centers: A proposal of a theoretical model for evaluation. Telemedicine and e-Health, 23(11), 905–912. https://doi.org/10.1089/tmj.2017.0007
38.
Organisation for Economic Co-operation and Development. (2010). Glossary of key terms in evaluation and results based management. OECD.
39.
PangwaK. (2016). Evaluability assessment of the Gauteng science park incubation programme. University of the Witwatersrand.
40.
PattonM. Q. (1987). How to use qualitative methods in evaluation. Sage.
41.
PattonM. Q. (2003). Utilization-focused evaluation. In KellaghanT.StufflebeamD. L. (Eds.), International handbook of educational evaluation (pp. 223–242). Springer. https://doi.org/10.1007/978-94-010-0309-4_15
42.
PeersmanG.GuijtI.PasanenT. (2015). Evaluability assessment for impact evaluation. Overseas Development Institute.
43.
PetersonD. J.DowneyL. H.LeathermanJ. (2018). Informing youth health and well-being programs: A national 4-H healthy living evaluation study. Journal of Youth Development, 13(3), 212–236. https://doi.org/10.5195/JYD.2018.605
44.
PrasadV. (2017). Designing and implementing an evaluability assessment of a career-technical education program as the first step towards validating program effectiveness: A case study of Toledo Technology Academy. University of Toledo.
45.
RogD. J. (2010). The SSA method: Not just old wine in a new bottle. New Directions for Evaluation, 125, 111–118. https://doi.org/10.1002/ev.323
46.
Salvatierra da SilvaD.JacobsonS. K.MonroeM. C.IsraelG. D. (2016). Using evaluability assessment to improve program evaluation for the Blue-throated Macaw Environmental Education Project in Bolivia. Applied Environmental Education and Communication, 15(4), 312–324. https://doi.org/10.1080/1533015X.2016.1237904
47.
SanouA.KouyateB.BibeauG.NguyenV.-K. (2011). Evaluability assessment of an immunization improvement strategy in rural Burkina Faso: Intervention theory versus reality, information need and evaluations. Evaluation and Program Planning, 34(3), 303–315. https://doi.org/10.1016/j.evalprogplan.2010.10.005
48.
SchröterD. C.CorynC. L. S.CullenA.RobertsonK. N.AlyamiM. (2012). Using concept mapping for planning and evaluation of a statewide energy efficiency initiative. Energy Efficiency, 5(3), 365–375. https://doi.org/10.1007/s12053-011-9141-7
49.
SmithM. F. (1989). Evaluability assessment: A practical approach. Kluwer Academic.
50.
SmithM. F. (2005). Evaluability assessment. In MathisonS. (Ed.), Encyclopedia of evaluation (pp. 136-139). Sage.
51.
TaylorJ.ForsellG.PerweilerE.SienkiewiczM. (2018). Longitudinal evaluation practices of health workforce development programs: An incremental approach to evaluability assessment. Evaluation and Program Planning, 69. 68–74. https://doi.org/10.1016/j.evalprogplan.2018.04.007
52.
ThurstonW. E.PotvinL. (2003). Evaluability assessment: A tool for incorporating evaluation in social change programmers. Evaluation, 9(4), 453–469. https://doi.org/10.1177/1356389003094006
United Nations Evaluation Group. (2014). Integrating human rights and gender equality in evaluations. United Nations Publications.
59.
United Nations Evaluation Group. (2016a). Evaluation in the SDG era: lessons, challenges and opportunities for UNEG. UNEG.
60.
United Nations Evaluation Group. (2016b). Norms and standards for evaluation. UNEG.
61.
WalserT. M. (2015). Evaluability assessment in higher education: Supporting continuous improvement, accountability, and a culture of assessment. Journal of Assessment and Institutional Effectiveness, 5(1) 58–77. https://doi.org/10.5325/jasseinsteffe.5.1.0058
62.
WalserT. M.TrevisanM. S. (2016). Evaluability assessment thesis and dissertation studies in graduate professional degree programs: Review and recommendations. American Journal of Evaluation, 37(1), 118–138. https://doi.org/10.1177/1098214015583693
63.
WattsB. R. (2016). ‘Adaptation and use of a five-task model for evaluability assessment. Journal of Multidisciplinary Evaluation, 12(27), 67–78.
64.
WholeyJ. S. (1979). Evaluation: promise and performance. Urban Institute.
65.
WholeyJ. S. (1987). Evaluability assessment: Developing program theory. New Directions for Program Evaluation, 33, 77–92.
66.
WholeyJ. S. (1994). Assessing the feasibility and likely usefulness of evaluation. In WholeyJ. S.HatryH. P.NewcomerK. E. (Eds.), Handbook of practical program evaluation (pp. 15–39). Jossey-Bass.
67.
WholeyJ. S. (2004). Assessing the feasibility and likely usefulness of evaluation. In Handbook of Practical Program Evaluation (2nd ed., pp. 33–62). Jossey-Bass.
68.
WholeyJ. S. (2010). Exploratory evaluation. In NewcomerK. E.HatryH. P.WholeyJ. S. (Eds.) Handbook of practical program evaluation (3rd ed., pp. 81–99). Jossey-Bass.
69.
WholeyJ. S. (2015). Exploratory evaluation. In NewcomerK. E.HatryH. P.WholeyJ. S. (Eds.) Handbook of practical program evaluation (4th ed., pp. 88–107). Jossey-Bass.
70.
WillisonJ. B.WaltersJ. H.CramerL.FontaineJ.HorvathA.OwensC.RossmanS.WalshK.BarrickK.KennedyE.MarkovitsL.SheppardM. (2013). FY 2011 Second Chance Act Adult Offender Reentry Demonstration Projects Evaluability Assessment, Executive Summary. Urban Institute.
71.
ZintM. T.CovittB. A.DowdP. F. (2011). Insights from an evaluability assessment of the U.S. Forest Service More Kids in the woods initiative. Journal of Environmental Education, 42(4), 255–271. https://doi.org/10.1080/00958964.2010.538091
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
