Testing mechanisms using large-N qualitative comparative analysis in realistic evaluations

Abstract

This article demonstrates how qualitative comparative analysis can be used to test mechanisms using large-N survey data (large-N qualitative comparative analysis) as part of realistic evaluation. Large-N qualitative comparative analysis may offer advantages when testing mechanisms in realistic evaluation: it can explore aspects of intervention complexity that are often harder to capture with regression-based methods, while simultaneously assessing the consistency and empirical relevance of mechanisms across a larger group of participants, thereby improving the credibility and generalizability of the program theory. Despite these benefits, qualitative comparative analysis is rarely used to test mechanisms with large-N data. The purpose of this article is to illustrate how this can be done by outlining a four-step process. A weakness of the proposed method is that it identifies the co-occurrence of conditions and outcomes without clarifying the generative processes that connect them. Therefore, the final part of the article discusses how qualitative comparative analysis can be combined with qualitative research.