Advocacy is an intentional act of influencing government and an important precondition for successful policy change in society. Drawing from an existing framework on policy influence, we propose an approach to quantifying the impact of policy influence efforts, specifically within the context of European Public Health (EPH) advocacy. The analysis hinges on the article “Moving from tokenism” which provides a starting point to conceptualize strategies to quantify impact. An exploratory case study approach allowed to integrate literature on advocacy evaluation in parallel with the internal documentation of a EPH advocacy organization We provide recommendations to advocacy organizations that aim to create an infrastructure towards quantifying the impact of their efforts. The framework is mostly tailored to the needs of EPH advocacy, but it can also have resonance beyond the scope of a specific sector.
ArabiA.RaF. (2014). Nurses’ policy influence: A concept analysis. Iranian Journal of Nursing and Midwifery Research, 19(3), 9. https://doi.org/10.1111/inr.12275. Epub 2016 Jun 21.
2.
ArenaM.AzzoneG.BengoI. (2015). Performance measurement for social enterprises. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 26(2), 649–672. https://doi.org/10.1007/s11266-013-9436-8
3.
ArensmanB. (2018). Negotiating effectiveness: The politics of results in advocacy for development and its evaluation [Wageningen University]. https://doi.org/10.18174/448390
4.
ArensmanB.van WaegeninghC.van WesselM. (2018). Twinning “practices of change” with “theory of change”: Room for emergence in advocacy evaluation. American Journal of Evaluation, 39(2), 221–236. https://doi.org/10.1177/1098214017727364
5.
BagnoliL.MegaliC. (2011). Measuring performance in social enterprises. Nonprofit and Voluntary Sector Quarterly, 40(1), 149–165. https://doi.org/10.1177/0899764009351111
6.
BarrettJ. B.van WesselM. G. J.HilhorstD. J. M.ArensmanB.KlaverD. C.RichertW.van BodegomA. j.van WaegeninghC.RaschE. D.WagemakersA. (2016). Advocacy for Development: Effectiveness, Monitoring and evaluation. Wageningen UR: Wageningen University, https://edepot.wur.nl/375995
7.
BiggsJ. S.FarrellL.LawrenceG.JohnsonJ. K. (2014). A practical example of contribution analysis to a public health intervention. Evaluation, 20(2), 214–229. https://doi.org/10.1177/1356389014527527
8.
BudhwaniS.McDavidJ. C. (2017). Contribution analysis: Theoretical and practical challenges and prospects for evaluators. Canadian Journal of Program Evaluation, 32(1), 1. https://doi.org/10.3138/cjpe.31121
9.
BuffardiA. L.HearnS.TilleyH. (2017). Refining advocacy assessment: Reflections from practice. ODI, Working Paper, 500, 13.
10.
CatalloC.SpaldingK.Haghiri-VijehR. (2014). Nursing professional organizations: What are they doing to engage nurses in health policy?SAGE Open, 4(4), 215824401456053. https://doi.org/10.1177/2158244014560534
11.
Chapman (2001). Advocacy in public health: Roles and challenges. International Journal of Epidemiology, 30(6), 1226–1232. https://doi.org/10.1093/ije/30.6.1226
Chapman, & Wameyo (2001). ActionAid's new strategy, Fighting Poverty Together, (1999-2003) gives a high priority to participatory policy making whi…58.
14.
ChiuP.CummingsG. G.ThorneS.Schick-MakaroffK. (2021). Policy advocacy and nursing organizations: A scoping review. Policy, Politics, & Nursing Practice, 22(4), 276–296. https://doi.org/10.1177/15271544211050611
15.
CoatesB.DavidR. (2002). Learning for change: The art of assessing the impact of advocacy work. Development in Practice, 12(3–4), 530–541. https://doi.org/10.1080/0961450220149870
CoffmanJ. (2007). Using the advocacy and policy change composite logic model to articulate an advocacy strategy or theory of change. HFRP: Harvard Family Research Project.
18.
CoffmanJ. (2009). A user’s guide to advocacy evaluation planning. Harvard Family Research Project.
19.
CohenG.SchroederJ.NewsonR.KingL.RychetnikL.MilatA. J.BaumanA. E.RedmanS.ChapmanS. (2015). Does health intervention research have real world policy and practice impacts: Testing a new impact assessment tool. Health Research Policy and Systems, 13(1), 3. https://doi.org/10.1186/1478-4505-13-3
20.
CullertonK.DonnetT.LeeA.GallegosD. (2018). Effective advocacy strategies for influencing government nutrition policy: A conceptual model. International Journal of Behavioral Nutrition and Physical Activity, 15(1), 83. https://doi.org/10.1186/s12966-018-0716-y
21.
DavidJ. L.ThomasS. L.RandleM.DaubeM. (2020). A public health advocacy approach for preventing and reducing gambling related harm. Australian and New Zealand Journal of Public Health, 44(1), 14–19. https://doi.org/10.1111/1753-6405.12949
De BruyckerI. (2019). Lobbying: An art and a science—five golden rules for an evidence-based lobbying strategy. Journal of Public Affairs, 19(4), e1949. https://doi.org/10.1002/pa.1949
De RaeveP.DavidsonP. (2020). Moving from tokenism. Health Europa, Quarterly, 13, 18–19.
26.
Devlin-FoltzD.FagenM. C.ReedE.MedinaR.NeigerB. L. (2012). Advocacy evaluation: Challenges and emerging trends. Health Promotion Practice, 13(5), 581–586. https://doi.org/10.1177/1524839912446482
27.
DurA. (2008). Measuring interest group influence in the EU: A note on methodology. European Union Politics, 9(4), 559–576. https://doi.org/10.1177/1465116508095151
28.
DybdalL.NielsenS. B.LemireS. (2011). Contribution analysis applied: Reflections on scope and methodology. The Canadian Journal of Program Evaluation, 25(2), 29–57. https://academic.oup.com/rev/article/31/1/118/6482054
29.
EbrahimA. S.RanganV. K. (2010). The limits of nonprofit impact: A contingency framework for measuring social performance. SSRN Electronic Journal. Harvard business school general management unit working paper no. 10-099, Harvard business school marketing unit working paper no. 10-099. https://doi.org/10.2139/ssrn.1611810
30.
FagenM. C.ReedE.KayeJ. W.JackL. (2009). Advocacy evaluation: What it is and where to find out more about it. Health Promotion Practice, 10(4), 482–484. https://doi.org/10.1177/1524839909339584
31.
GardnerR. G.HarrisT. B.LiN.KirkmanB. L.MathieuJ. E. (2017). Understanding “it Depends” in organizational research: A theory-based taxonomy, review, and future research agenda concerning interactive and quadratic relationships. Organizational Research Methods, 20(4), 610–638. https://doi.org/10.1177/1094428117708856
32.
GenS.WrightA. C. (2013). Policy advocacy organizations: A framework linking theory and practice. Journal of Policy Practice, 12(3), 163–193. https://doi.org/10.1080/15588742.2013.795477
33.
Harris-RoxasB.HarrisE. (2011). Differing forms, differing purposes: A typology of health impact assessment. Environmental Impact Assessment Review, 31(4), 396–403. https://doi.org/10.1016/j.eiar.2010.03.003
34.
HojnackiM.MarchettiK. M.BaumgartnerF. R.BerryJ. M.KimballD. C.LeechB. L. (2015). Assessing business advantage in Washington lobbying. Interest Groups & Advocacy, 4(3), 205–224. https://doi.org/10.1057/iga.2015.3
35.
IOB (2014). Evaluation of Policy Influencing, Lobbying and Advocacy. Term of Reference. Policy and Operations Evaluation Department (IOB), Ministry of Foreign Affairs - The Netherlands.
36.
IOB Evaluation (2015). Opening doors and unlocking potential. Key lessons from an evaluation of support for Policy Influencing, Lobbying and Advocacy (PILA).
37.
JonesH. (2011). A guide to monitoring and evaluating policy influence. London: Background Note, Overseas Development Institute. 12.
38.
KemmJ. (2001). Health impact assessment: A tool for healthy public policy. Health Promotion International, 16(1), 79–85. https://doi.org/10.1093/heapro/16.1.79
39.
KotvojsF.ShrimptonB. (2007). Contribution analysis: A new approach to evaluation in international development. Evaluation Journal of Australasia, 7(1), 27–35. https://doi.org/10.1177/1035719X0700700105
40.
LaboutkováŠVymětalP. (2017). Measures of Transparent Lobbying: How to Approach It and Evaluate It: A Preliminary Stage. 10.
41.
LampkinL. M.WinklerM. K.KerlinJ.HatryH. P.NatenshonD.SaulJ.MelkersJ.SeshadriA. (2006). Building a common outcome framework to measure nonprofit performance. Washington, DC: The Urban Institute. 17.
42.
LaneA. J. (2018). Impact evaluation of nurse advocacy center for underserved on population health. Online Journal of Rural Nursing and Health Care, 18(2), 88–112. https://doi.org/10.14574/ojrnhc.v18i2.518
43.
LangT.BrinsdenH. (2015). An introduction to public health advocacy: Reflections on theory and practice. Food Research Collaboration. https://doi.org/10.13140/RG.2.1.4874.7287
44.
LoweryD. (2013). Lobbying influence: Meaning, measurement and missing. Interest Groups & Advocacy, 2(1), 1–26. https://doi.org/10.1057/iga.2012.20
45.
MarchJ. G. (1955). An Introduction to the theory and measurement of influence. The America Political Science Review, 49(2), 431–451. https://www.jstor.org/stable/1951813.
46.
MayneJ. (2008). Contribution analysis: An approach to exploring cause and effect. 4.
NaeveK. (2017). Evaluating advocacy: An exploration of evidence and tools to understand what works and why. 3ie.
50.
NetaG.GlasgowR. E.CarpenterC. R.GrimshawJ. M.RabinB. A.FernandezM. E.BrownsonR. C. (2015). A framework for enhancing the value of research for dissemination and implementation. American Journal of Public Health, 105(1), 49–57. https://doi.org/10.2105/AJPH.2014.302206
51.
OxfamG. (2014). Monitoring, evaluation, accountability and learning.
52.
Rahn-TiemeyerK. (2015). Understanding dimensions of advocacy success: How evaluation can maximize the impact of nonprofit advocacy initiatives. SPNHA Review, 11(1, Article 5), 22. http://scholarworks.gvsu.edu/spnhareview/vol11/iss1/5
SpenceleyS. M.ReutterL.AllenM. N. (2006). The road less traveled: Nursing advocacy at the policy level. Policy, Politics, & Nursing Practice, 7(3), 180–194. https://doi.org/10.1177/1527154406293683
55.
StachowiakS.GienappA.ReismanJ. (2007). A guide to measuring advocacy and policy. The Annie E. Casey Foundation.
56.
StartD.HovlandI. (2004). Tools for policy impact: A handbook for researchers. Overseas Development Institute.
TelesS.SchmittM. (2011). The elusive craft of evaluating advocacy. Stanford Social Innovation Review, 9(3), 40–43.
59.
van WijkM.LallemanP. C. B.CummingsG. G.EngelJ. (2022). Public opinion leadership in nursing practice: A rogerian concept analysis. Policy, Politics, & Nursing Practice, 23(1), 67–79. https://doi.org/10.1177/15271544211071099
60.
VeermanJ. L.BarendregtJ. J.MackenbachJ. P. (2006). The European common agricultural policy on fruits and vegetables: Exploring potential health gain from reform. European Journal of Public Health, 16(1), 31–35. https://doi.org/10.1093/eurpub/cki166
61.
VeermanJ. L.BekkerM. P. M.MackenbachJ. P. (2006). Health impact assessment and advocacy: A challenging combination. Sozial- Und Präventivmedizin SPM, 51(3), 151–152. https://doi.org/10.1007/s00038-006-0034-z
