Evaluation blogs,podcasts and webinars in 2024: A mid-year review

Abstract

This is a further review of evaluation blogs, podcasts and webinars in the first half of 2024. It follows on from the previous ‘roundup’ of 2023 published in Evaluation in this year’s April issue. As before, hundreds of sources have been considered, and although this review is inevitably personal and selective, I believe it captures many of the important trends and debates informing current evaluation practice. The review is organised around key trends that emerged throughout the first half of the year, some of which continued and evolved from last year. It begins by considering ongoing debates about artificial intelligence and indigenous evaluation, before identifying further topics which have garnered particular attention so far in 2024.

Keywords

artificial intelligence bricolage decolonisation results scaling systems change theory-based evaluation

Get full access to this article

View all access options for this article.

References

Amazon-Brown

(2024) Hands on with GenAI: predictions and observations from The MERL Tech Initiative and Oxform Policy Management’s ICT4D Training Day. Available at: https://merltech.org/hands-on-with-genai-at-ict4d-in-accra/ (accessed 24 August 2024).

Apgar

Bradburn

Rohrbach

, et al (2024a) Rethinking rigour to embrace complexity in peacebuilding evaluation. Evaluation 30(3): 408–33.

Apgar

Snijder

Paul

(2024b) Breaking free from the theory of change straight jacket. CLARRISA. Available at: https://clarissa.global/getting-out-of-the-theory-of-change-straight-jacket-the-freedom-and-challenges-of-a-reflexive-approach/ (accessed 24 August 2024).

Aston

(2023) Changing theories. Medium. Available at: https://thomasmtaston.medium.com/changing-theories-e857aa8fba05 (accessed 24 August 2024).

Aston

(2024a) Evaluation blogs, podcasts, and webinars in 2023: A roundup review. Evaluation 30: 306–315.

Aston

(2024b) The fall of icarus. Medium. Available at: https://thomasmtaston.medium.com/the-fall-of-icarus-e941c4466108 (accessed 29 July 2024).

Aston

(2024c) Pivot or entropy: The end of the gold standard. Medium. Available at: https://thomasmtaston.medium.com/pivot-or-entropy-the-end-of-the-gold-standard-1e216ce7580b (accessed 29 July 2024).

Aston

(2024d) Higher, broader and deeper level results. Medium. Available at: https://medium.com/@thomasmtaston/higher-broader-and-deeper-level-results-b1cc150b8f1c (accessed 24 August 2024).

Aston

(2024e) Making sense of all the sensemaking. Medium. Available at: https://thomasmtaston.medium.com/making-sense-of-all-the-sense-making-c6d6eff2994f (accessed 25 August 2024).

10.

Aston

Apgar

(2022) The art and craft of bricolage in evaluation. CDI Practice Paper 25. Brighton: Institute of Development Studies.

11.

Aston

Goulden

(2019) Measuring the impact of advocacy: It’s not easy, but it’s not impossible. CARE Insights. Available at: https://insights.careinternational.org.uk/development-blog/measuring-the-impact-of-advocacy-it-s-not-easy-but-it-s-not-impossible (accessed 7 August 2024).

12.

Barnard

(2024a) Theory-based evaluations within a counterfactual paradigm. Substack. Available at: https://mattbarnard.substack.com/p/theory-based-evaluations-within-a?r=ct9ps&utm_campaign=post&utm_medium=web&triedRedirect=true (accessed 29 July 2024).

13.

Barnard

(2024b) Putting the theory of change in theory of change. Substack. Available at: https://mattbarnard.substack.com/p/putting-the-theory-of-change-in-theories?r=ct9ps&utm_campaign=post&utm_medium=web&triedRedirect=true (accessed 29 July 2024).

14.

Barnett

(2024) CDI at 10: An enduring curiosity in evaluating real-world impact. Bond. Available at: https://www.bond.org.uk/news/2024/02/cdi-at-10-an-enduring-curiosity-in-evaluating-real-world-impact/ (accessed 29 July 2024).

15.

Brown

(2016) Let’s bring back theory to theory of change. 3ie Impact. Available at: https://www.3ieimpact.org/blogs/lets-bring-back-theory-theory-change (accessed 24 August 2024).

16.

Centre for Development Impact (CDI) (2024) Practicing inclusive rigour: Moving from trade-offs to bright spots? Available at: https://www.ids.ac.uk/events/practicing-inclusive-rigour-moving-from-trade-offs-to-bright-spots/ (accessed 7 August 2024).

17.

Centre for the Evaluation of Complexity Across the Nexus (CECAN) (2024) Navigating systems change evaluation: How to know you are making progress in shifting systems. Available at: https://www.cecan.ac.uk/events/cecan-webinar-navigating-system-change-evaluation-reflections-on-how-to-know-you-are-making-progress-in-shifting-systems/ (accessed 29 July 2024).

18.

Chilisa

(2015) A Synthesis Paper on Made in Africa Evaluation. Accra, Ghana: Commissioned by the African Evaluation Association (AFrEA).

19.

Chilisa

(2019) Indigenous Research Methodologies. Thousand Oaks, CA: Sage.

20.

Chomitz

(2024) How RCTs inform scaling decisions. Global Innovation Fund. Available at: https://www.globalinnovation.fund/news/can-rcts-inform-scaling-decisions (accessed 29 July 2024).

21.

Delahais

(2024) A brief history of impact evaluation approaches. Miro. Available at: https://miro.com/app/board/uXjVOMGrWBY=/?share_link_id=949644593386 (accessed 29 July 2024).

22.

Evans

Raimondo

Hutton

(2021) The results agenda needs a steer – What could be its new course? World Bank. Available at: https://ieg.worldbankgroup.org/blog/results-agenda-needs-steer-what-could-be-its-new-course (accessed 29 July 2024).

23.

Fugard

(2023) ‘Counterfactual’ is not a synonym for ‘control group’. Available at: https://andifugard.info/counterfactual-is-not-a-synonym-for-control-group/ (accessed 20 December 2023).

24.

Gargani

(2024) Five big ideas about scaling impact. Available at: https://www.youtube.com/watch?v=_k49xuTZyu0&list=WL&index=22 (accessed 29 July 2024).

25.

Goldie

Davies

(2024) CECAN Webinar Navigating systems change evaluation: reflections on how you know you are making progress in shifting systems. Available at: https://www.cecan.ac.uk/events/cecan-webinar-navigating-system-change-evaluation-reflections-on-how-to-know-you-are-making-progress-in-shifting-systems/ (accessed 29 July 2024).

26.

Green

(2012) Theories of change = logframes on steroids? A discussion with DFID. From Poverty to Power. Available at: https://frompoverty.oxfam.org.uk/theories-of-change-logframes-on-steroids-a-discussion-with-dfid/ (accessed 28 August 2024).

27.

Griffith Centre for Systems Innovation (2024) Now we are all measuring impact – But is anything changing? It tells the long history of impact measurement. Available at: https://medium.com/good-shift/now-we-are-all-measuring-impact-but-is-anything-changing-1dc998b95c83 (accessed 1 August 2024).

28.

Guerzovich

(2023) Storytelling and evaluation can pathways be the theory missing in explanatory narratives. Medium. Available at: https://medium.com/@florcig/storytelling-and-evaluation-can-pathways-be-the-theory-missing-in-explanatory-narratives-0625a63adfc6 (accessed 24 August 2024).

29.

Haldrup

(2024) How to do M&E when you’re working with complex problems. Available at: https://medium.com/@undp.innovation/how-to-do-m-e-when-youre-working-with-complex-problems-b91622ebc9b2 (accessed 1 August 2024).

30.

King

(2024a) Mixing methods. Substack. Available at: https://juliankingnz.substack.com/p/mixing-methods-and-metaphors?r=8fet1&utm_medium=ios&triedRedirect=true (accessed 28 July 2024).

31.

King

(2024b) Mixed reasoning and cubist evaluation. Substack. Available at: https://juliankingnz.substack.com/p/mixed-reasoning-and-cubist-evaluation?r=8fet1&utm_campaign=post&utm_medium=web&triedRedirect=true (accessed 28 July 2024).

32.

Koleros

AMH

Adrien

M-H

Tyrrell

(2024) Theories of Change in Reality: Strengths, Limitations and Future Directions. Abingdon: Routledge.

33.

Lowe

(2023) Explode on impact. Medium. Available at: https://toby-89881.medium.com/explode-onimpact-cba283b908cb (accessed 20 December 2023).

34.

McClean

Gargani

(2019) Scaling Impact: Innovation for the Public Good. Abingdon: Routledge.

35.

Office of Evaluation and Oversight (2024) Boosting development effectiveness with adaptive management. Available at: https://www.youtube.com/watch?v=dOudjwC9UuE (accessed 28 July 2024).

36.

Pawson

(2006) Evidence Based Policy – A Realist Perspective. London; Thousand Oaks, CA; New Delhi, India: Sage.

37.

Rogers

(2024) What should be the role of evaluation in a changing global context? Available at: https://medium.com/@patriciarogers_5608/what-should-be-the-role-of-evaluation-in-a-changing-global-context-3f3c70bbb2d9 (accessed 29 July 2024).

38.

Shephard

Ellersiek

Meuer

, et al (2018) Influencing policy and civic space: A meta-review of Oxfam’s policy influence, citizen voice and good governance effectiveness reviews. Available at: https://policy-practice.oxfam.org/resources/influencing-policy-and-civic-space-a-meta-review-of-oxfams-policy-influence-cit-620462/

39.

Stern

Stame

Mayne

, et al (2012) Broadening the Range of Designs and Methods for Impact Evaluations: Report of a Study Commissioned by the Department for International Development. London: Department for International Development.

40.

Tasic

(2024) Dreams, generative AI and sensemaking. Available at: https://www.linkedin.com/pulse/dreams-generative-ai-sensemaking-ivan-tasic-caplf/?trackingId=DMqUI%2FjCec4OxLnhvZx5Bg%3D%3D (accessed 29 July 2024).

41.

Tolmer

(2024) Decolonizing M&E: What does African monitoring and evaluation look like? Medium. Available at: https://medium.com/@undp.innovation/decolonizing-m-e-what-does-african-monitoring-and-evaluation-look-like-926d97d6f8d1 (accessed 29 July 2024).

42.

Ton

(2024) Impact evaluation should be replaced with contribution analysis. Institute for Development Studies. Available at: https://www.ids.ac.uk/opinions/cdi10-replace-impact-evaluation-with-contribution-analysis/ (accessed 29 July 2024).

43.

United Nations Development Programme (UNDP) (2024) Decolonizing M&E: What does African monitoring and evaluation look like? Medium. Available at: https://medium.com/@undp.innovation/decolonizing-m-e-what-does-african-monitoring-and-evaluation-look-like-926d97d6f8d1 (accessed 7 August 2024).

44.

White

(2011) Unhappy marriage of impact evaluation and the results agenda. International Initiative for Impact Evaluation. Available at: https://www.3ieimpact.org/blogs/unhappy-marriage-impact-evaluation-and-results-agenda (accessed 29 July 2024).

45.

White

(2024) How not to learn what works: The scourge of results frameworks. Available at: https://www.gdn.int/how-not-learn-what-works-scourge-results-frameworks-0 (accessed 29 July 2024).

46.

World Bank (2024) World bank group announces new approach to measuring impact. Available at: https://www.worldbank.org/en/news/press-release/2024/04/09/world-bank-group-announces-new-approach-to-measuring-impact (accessed 29 July 2024).