Rigor and Relevance in NDM: How to Study Decision Making Rigorously With Small N s and Without Controls and (Inferential) Statistics

Abstract

The purpose of this paper is to address concerns regarding the absence or lax of rigor in qualitative naturalistic decision making (NDM) studies. The paper argues that concerns regarding the rigor of NDM's qualitative studies are largely irrelevant, as they are typically conceptualized consistent with the quantitative/experimental paradigm. To redress this category mistake (discussing entity A in terms appropriate to a qualitatively different entity B; Ryle, 1949), the paper (a) conceptualizes rigor inclusively as “the disciplined application of reason to subjects related to knowledge and/or communication” (Allende, 2004); (b) differentiates this broad conceptualization into a substantive facet that is common to all paradigms (i.e., the judicious selection and application of methods in order to maximize the probability of producing trustworthy answers to the study's questions under the constraints imposed by its context) and a technical facet (the strict application of correct or prescribed methods) that admits paradigm-specific distinctions; and (c) based on this conceptualization, presents a set of appropriate guidelines for optimizing and evaluating the rigor of theoretical and empirical studies that is appropriate for inquiry-guided NDM qualitative studies.