The merit of narrative film methods to support participatory approaches and professional development has been increasingly demonstrated by research in several fields and education. However, the use of digital storytelling and other film methods in evaluation remains largely unchartered territory. This article provides a case study of a digital storytelling evaluation initiative in monitoring and evaluation (M&E) in an Australian community not-for-profit. The aim is to offer practical insights for evaluators and organizations considering digital storytelling and other film narrative methods for participant-centered evaluation. Embedding digital evaluation stories into M&E evolved through collaboration between the external evaluation team and organizational leadership, requiring capacity building in evaluation, digital and qualitative methods, and new systems and processes. Benefits include transformation into a participant-centered evaluation and learning culture. Several challenges are discussed, including the extent of organizational change required, the associated time, energy, and cost, and the positive bias of visual narratives.
AbbatoS. (2015). The case for evaluating process and worth: Evaluation of a programme for carers and people with dementia. In RussellJ.GreenhalghT.KushnerS. (Eds.), Case study evaluation: Past, present and future challenges (Vol. 15, pp. 107–132). Bingley, UK: Emerald.
3.
BaumF.MacDougallC.SmithD. (2006). Participatory action research. Journal of Epidemiology & Community Health, 60(10), 854–857. http://dx.doi.org/10.1136/jech.2004.028662
4.
BaumannS. E.LhakiP.BurkeJ. G. (2020). Collaborative filmmaking: A participatory, visual research method. Qualitative Health Research, 30(14), 2248–2264. https://doi.org/10.1177/1049732320941826
5.
BaumannS. E.MeranteM.FolbB. L.BurkeJ. G. (2019). Is film as a research tool the future of public health? A review of study designs, opportunities, and challenges. Qualitative Health Research, 30(2), 250–257. https://doi.org/10.1177/1049732319871251
6.
BernardH. R. (2002). Research methods in anthropology: Qualitative and quantitative approaches (3). Walnut Creek, CA: Alta Mira Press.
7.
BowerG. H. (1976). Experiments on story understanding and recall. Quarterly Journal of Experimental Psychology, 28(4), 511–534. https://doi.org/10.1080/1460747608400579
8.
BoyceC.NealeP. (2006). Conducting in-depth interviews: A guide for designing and conducting in-depth interviews for evaluation input. Watertown, NY: Pathfinder International Tool Series, Monitoring and Evaluation-2
9.
CampbellN. C.MurrayE.DerbyshireJ.EmeryJ.FarmerA.GriffithsF.GuthrieB.LesterH.WilsonP.KinmouthA. L. (2007). Designing and evaluating complex interventions to improve health care. British Medical Journal, 334(7591), 455–459. https://doi.org/10.1136/bmj.39108.379965.BE
10.
CatalaniC. E. C. V.VenezialeA.CampbellL.HerbstS.ButlerB.SpringgateB.MinklerM. (2012). Videovoice: Community assessment in post-Katrina New Orleans. Health Promotion Practice, 13(1), 18–28. https://doi/pdf/10.1177/1524839910369070.
11.
ChávezV.IsraelB.AllenA. J.IIIDeCarloM. F.LichtensteinR.SchulzA.BayerI. S.McGranaghanR. (2004). A bridge between communities: Video-making using principles of community-based participatory research. Health Promotion Practice, 5(4), 395–403. https://doi.org/10.1177/1524839903258067
12.
CousinsJ. B.ChouinardJ. A. (2012). Participatory evaluation up close: An integration of research-based knowledge. Charlotte, NC: Information Age Publishing.
13.
CresswellJ. W.Plano-ClarkV. L. (2011). Designing and conducting mixed method research (2). Los Angeles, CA: Sage.
14.
DavidsonE. J. (2005). Evaluation methodology basics: The nuts and bolts of sound evaluation. Thousand Oaks, CA: Sage.
de JagerA.FogartyA.TewsonA.LenetteC.BoydellK. M. (2017). Digital storytelling in research: A systematic review. The Qualitative Report, 22(10), 2548–2582. https://doi.org/10.46743/2160-3715/2017.2970
17.
De VecchiN.KennyA.Dickson-SwiftV.KiddS. (2017). Exploring the process of digital storytelling in mental health research: A process evaluation of consumer and clinician experiences. International Journal of Qualitative Methods, 16(1), 1–13. https://doi.org/10.1177/1609406917729291
EdwardsS. (2014). Finding a place for story: Looking beyond reflective practice. International Practice Development Journal, 4(2), 5. https://doi.org/10.19043/ipdj.42.005
20.
FettermanD. M.KaftarianS. J.WandersmanA. (2015). Empowerment evaluation: Knowledge and tools for self-assessment, evaluation capacity building, and accountability (2nd ed.). Sage. https://dx.doi.org/10.4135/9781483387079
21.
FoxJ. (2016). How to lead a quest: A handbook for pioneering executives. Milton, Queensland, Australia: Wiley.
22.
FunnellS. C.RogersP. J. (2011). Purposeful program theory: Effective use of theories of change and logic models. San Francisco, CA: Jossey-Bass.
23.
GibsonB. E. (2005). Co-producing video diaries: The presence of the “absent” researcher. International Journal of Qualitative Methods, 4(4), 34–43. https://doi.org/10.1177/160940690500400403
24.
GubriumA.HarperK. (2013). Participatory visual and digital methods. New York, NY: Routledge.
25.
GubriumA.HarperK.OtañezM. (2015). Participatory visual and digital research in action. New York, NY: Routledge.
26.
GubriumA. C.KrauseE. L.JerniganK. (2014). Strategic authenticity and voice: New ways of seeing and being seen as young mothers through digital storytelling. Sexuality Research and Social Policy, 11(4), 337–347. https://doi.org/10.1007/s13178-014-0161-x
GustomoA.FebriansyahH.GintingH.SantosoI. M. (2019). Understanding narrative effects: The impact of direct storytelling intervention on increasing employee engagement among the employees of state-owned enterprise in West Java, Indonesia. Journal of Workplace Learning, 31(2), 166–191. https://doi.org/10.1108/JWL-07-2018-0088
30.
HanB.FinkE. L. (2012). How do statistical and narrative evidence affect persuasion? The role of evidentiary features. Argumentation and Advocacy, 49(1), 39–58. https://doi.org/10.1080/00028533.2012.11821779
31.
HancoxD. (2012). The process of remembering with the forgotten Australians: Digital storytelling and marginalized groups. Human Technology, 8(1), 65–76. https://doi.org/10.17011/ht/urn.201205141653
32.
HoltD. T.ArmenakisA. A.FeildH. S.HarrisS. G. (2007). Readiness for organizational change: The systematic development of a scale. The Journal of Applied Behavioral Science, 43(2), 232–255. https://doi.org/10.1177/0021886306295295
33.
HorsagerD. (2009). The trust edge: How top leaders gain faster results, deeper relationships, and a stronger bottom line. New York, NY: Free Press.
34.
KotterJ. P. (2012). Leading change. Boston, MA: Harvard Business Review Press.
35.
KruegerR. A. (2010). Using stories in evaluation. In WholeyJ. S.HatryH. P.NewcomerK. E. (Eds.), Handbook of practical program evaluation (3rd ed., pp. 404–423). San Francisco, CA: Jossey-Bass.
36.
LabontéR. (2011). Reflections on stories and a story/dialogue method in health research. International Journal of Social Research Methodology, 14(2), 153–163. https://doi.org/10.1080/13645579.2010.492131
37.
LambertL. (2009). Where it all started: The center for digital storytelling in California. In HartleyJ.McWilliamK. (Eds.), Story circle digital storytelling around the world (pp. 79–90). Chichester, West Sussex, UK: Wiley-Blackwell.
38.
LawrenceR. L.BuckleyV. S.CuevaM.GiordaniT.RamdehollD.SimpsonS. (2006, October 4–6). Once upon a time: The power of story in research [Conference presentation]. Midwest Research-to-Practice Conference in Adult, Continuing and Community Education, St. Louis, MO, United States. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.95.5640&rep=rep1&type=pdf
39.
LenetteC.CoxL.BroughM. (2015). Digital storytelling as a social work tool: Learning from ethnographic research with women from refugee backgrounds. The British Journal of Social Work, 45(3), 988–1005. https://doi.org/10.1093/bjsw/bct184
40.
LunchC. (2007). The most significant change: Using participatory video for monitoring and evaluation. Participatory Learning and Action, 56, 28–32. https://pubs.iied.org/g02906.
41.
LundstrὄmM.EkborgM.IdelandM. (2012). To vaccinate or not to vaccinate: How teenagers justified their decision. Cultural Studies of Science Education, 7(1), 193–221. https://doi.org/10.1007/s11422-012-9384-4
42.
MainesM. J.PierceJ. L.LaslettB. (2008). Telling stories: The use of personal narrative in the social sciences and history. Ithaca, NY: Cornell University Press.
McCallB.ShallcrossL.WilsonM.FullerC.HaywardA. (2019). Storytelling as a research tool and intervention around public health perceptions and behaviour: A protocol for a systematic narrative review. BMJ Open, 9(12), 1–12. https://doi.org/10.1136/bmjopen-2019-030597
45.
MertensD. M. (2015). Research and evaluation in education and psychology (4). Thousand Oaks, CA: Sage.
MollS.Wyndham-WestM.MulvaleG.ParkS.BuettgenA.PhoenixM.FleisigR.BruceE. (2020). Are you really doing “codesign”? Critical reflections when working with vulnerable populations. BMJ Open, 10(11), 1–5. http://dx.doi.org/10.1136/bmjopen-2020-038339
48.
MoreauK. A.EadyK.SikoraL.HorsleyT. (2018). Digital storytelling in health professions education: A systematic review. BMC Medical Education, 18, 208. https://doi.org/10.1186/s12909-018-1320-1
49.
MulvaleG.MollS.MiatelloA.Murray-LeungL.RogersonK.SassiR. B. (2019). Co-designing services for youth with mental health issues: Novel elicitation approaches. International Journal of Qualitative Methods, 18, 1–13. https://doi.org/10.1177/1609406918816244
50.
PalinkasL. A.HorwitzS. M.GreenC. A.WisdomJ. P.DuanN.HoagwoodK. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administraion and Policy in Mental Health and Mental Health Services Research, 42(5), 533–544. https://doi.org/10.1007/s10488-013-0528-y
51.
PattonM. Q. (1990). Qualitative evaluation and research methods (2). Newbury Park, CA: Sage.
PattonM. Q. (2015). Misuse: The shadow side of use. In ChristieC. A.VoA. T. (Eds.), Evaluation use and decision-making in society: A tribute to Marvin C. Alkin (pp. 131–148). Charlotte, NC: Information Age Publishing.
57.
PattonM. Q. (2018). Principles-focused evaluation: The GUIDE. New York, NY: Guilford Press.
58.
PettyJ.TrevesR. (2017). Development of a digital storytelling resource to support children’s nursing students in neonatal care. Nursing Children and Young People, 29(2), 32–37. https://doi.org/10.7748/ncyp.2017.e857
59.
PlushT. (2013). Fostering social change through participatory video: A conceptual framework. Development Bulletin (Canberra), 75, 55–58.
60.
PreskillH.BoyleS. (2008). A multidisciplinary model of evaluation capacity building. American Journal of Evaluation, 29(4), 443–459. https://doi.org/10.1177/1098214008324182
61.
PreskillH.TorresR. T. (2001). The readiness for organizational learning and evaluation instrument (ROLE). In Russ-EftD.PreskillH. (Eds.), Evaluation in organizations: A systematic approach to enhancing learning, performance, and change (pp. 421–434). New York, NY: Basic Books.
62.
RiegerK. L.SchultzA. S. H. (2014). Exploring arts-based knowledge translation: Sharing research findings through performing the patterns, rehearsing the results, staging the synthesis. Worldviews on Evidence-Based Nursing, 11(2), 133–139. https://doi.org/10.1111/wvn.12031
63.
RiegerK. L.WestC. H.KennyA.ChooniedassR.DemczukL.MitchellK. M.ChateauJ.ScottS. D. (2018). Digital storytelling as a method in health research: A systematic review protocol. Systematic Reviews, 7(41), 1–7. https://doi.org/10.1186/s13643-018-0704-y
64.
RobinB. R. (2006). The educational uses of digital storytelling. In CrawfordC.CarlsenR.McFerrinK.PriceJ.WeberR.WillisD. (Eds.), Proceedings of SITE 2006—Society for Information Technology & Teacher Education International Conference (pp. 709–716). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE).
65.
RosensteinB. (2002). Video use in social science research and program evaluation. International Journal of Qualitative Methods, 1(3), 22–43. https://doi.org/10.1177/160940690200100302
SquireC.DavisM.EsinC.AndrewsM.HarrisonB.HydénL.-C.HydénM. (2014). What is narrative research?. London, UK: Bloomsbury.
68.
StakeR. E. (1995). The art of case study research. Thousand Oaks, CA: Sage.
69.
StakeR. E. (2005). Qualitative case studies. In DenzinN. K.LincolnY. S. (Eds.), The SAGE handbook of qualitative research (3rd ed., pp. 443–466). Thousand Oaks, CA: Sage.
70.
SunderlandN.RobinsonK.BurgessA. (2021). Overcoming future professionals’ fear of digital storytelling. Australian Social Work, 74(1), 13–28. https://doi.org/10.1080/0312407X.2020.1762098
71.
WangC.BurrisM. A. (1997). Photovoice: Concept, methodology, and use for participatory needs assessment. Health Education & Behavior, 24(3), 369–387. https://doi.org/10.1177/109019819702400309
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
