Abstract
A reflection on how we can better evaluate the impact of university extension programs by mapping the ripple effect.
Change flows along the lines of relationship.
I can’t take credit for this insight. I learned it from Kathy Allen, a systems-minded leadership educator who several years ago led a training for our staff at University of Minnesota Extension. If there’s one thing I know as a sociologist who has built a career doing program evaluation, this is it.
My work with Extension is to evaluate our rural community development programs. Extension’s work is educational, so the core emphasis of evaluation has historically been on individual outcomes. Are participants learning what we hoped they would learn? Are they behaving differently as a result?
But for me, and for funders and those above me in the university hierarchy, the more interesting stuff goes beyond the individual into the social realm—those “so what” questions about what happens as a result of teaching or creating connections among people. An economist colleague calls this the difference between “private value” and “public value.”
I’ve journeyed down several methodological pathways to answer these “so what” questions. First I collaborated with colleagues to create a community social capital model and survey. We hoped to use the survey as a pre/post measure of changes in social capital resulting from some of our more intensive community programs, but we found that data collection was too difficult and costly, and follow-up was nearly impossible. At least the model has been useful as an educational tool.
I partnered with a graduate student to create an econometric model to determine if one of our deeply engaged economic development programs was having an impact on per capita income at the county level. The results were inconclusive and our partners in communities learned nothing about themselves.
I began using social network analysis (SNA) to describe and measure the impact of programming on personal and organizational networks. This has been a great tool, but SNA is more about visualizing relationships than answering questions about what flows from them.
Then I learned about ripple effects mapping (REM). Extension colleagues in other states were using this qualitative, group participatory method to document changes in communities resulting from a community leadership program. I invited one of these colleagues to train me and others in Minnesota. Twelve years later, I’ve become a national leader in the use of this method and I co-authored, with several of those colleagues, a widely-used open access field guide to REM.
The REM process is not rocket science. It involves a face-to-face group session in which participants interview each other using Appreciative Inquiry questions, report out their interview findings, and the facilitator leads a group qualitative analysis of the data to generate themes and create a mind map documenting the chain of effects of the program in each thematic area. We always include a meal in the process. During the pandemic, we altered the method to make it possible over Zoom, but we did have to give up the meal.
I recently conducted a virtual REM evaluation about the Rural Equity Summit, an annual conference led by some of my Extension colleagues and representatives of two community-based organizations in south central Minnesota. The ripple mapping documented how the Summit has created networks of support for local leaders and residents to extend and deepen racial equity and justice efforts in rural communities. While the Summit is not the only source of equity, diversity and inclusion training in the region, it has clearly contributed to a synergy with other organizations and efforts working on equity issues.
The Summit has “rippled” into activities to promote equity in different sectors of rural communities, such as education, law enforcement, and the arts. It has inspired self-reflection and empathy, and has equipped local leaders and residents with tools and connections to promote individual, organizational, and community action. Commenting on a well-known trope, one participant of color commented that “it was refreshing to hear the ‘History of Minnesota Not So Nice’ [at the Summit]… I think just hearing those things was like a validation/confirmation that our experiences as people of color in Minnesota were legitimately not so nice!”
REM has reinforced for me that change flows along the lines of relationship, and it has actually helped me show it. REM has also changed me. It has taken me into the worlds of people working for anti-racism and anti-poverty education, healthy food access, community gardening, arts education, greenhouse gas reduction and sustainable development. I have something useful to offer them. They keep my hope alive.