What Good are Statistics that Don’t Generalize?

Abstract

Quantitative and qualitative inquiry are sometimes portrayed as distinct and incompatible paradigms for research in education. Approaches to combining qualitative and quantitative research typically “integrate” the two methods by letting them co-exist independently within a single research study. Here we describe intra-sample statistical analysis (ISSA) as a general technique for using quantitative tools to support qualitative inquiry so as to simultaneously provide warrants from qualitative and quantitative traditions. In certain circumstances ISSA makes it possible to relax the requirement that individual participants be treated as the unit of analysis in statistical models, and thus provides justification for coding qualitative observations and drawing statistically based conclusions about observations in a qualitative context. We developed ISSA and describe it here both because it can be used as a tool for qualitative research, and because it illuminates the relationship between method and interpretation in the research traditions that it bridges. In this article, we (a) summarize key features of qualitative and quantitative research relevant to ISSA; (b) describe ISSA as an analytical technique; (c) discuss the quantitative and qualitative justification for ISSA and the nature of the conclusions that can be drawn based on it; and (d) explore the more general implications of ISSA for qualitative and quantitative inquiry.