The mixing of methods: A three-step process for improving rigour in impact evaluations

Abstract

This article describes a systematic process that is helpful in improving impact evaluation assignments, within restricted budgets and timelines. It involves three steps: a rethink of the key questions of the evaluation to develop more relevant, specific questions; a way of designing a mix of research methods to generate evidence that supports more valid conclusions; and a step that aims to make evaluation outputs more useful. The approach is illustrated through two examples: one on measuring income impacts in an irrigated horticulture programme in Nepal, Zambia and Ethiopia; and another on the assessment of changes in organizational capacities for collective marketing by smallholders in Bolivia. The article demonstrates that this simple, straightforward and structured three-step process helped to reduce the tendency to one-method designs. Enhanced critical reflection within the team allowed for greater sensitivity to validity threats and the creativity to find ways to handle them.