Beyond the Facts: Limited Empirical Diversity and Causal Inference in Qualitative Comparative Analysis

Abstract

Qualitative Comparative Analysis (QCA) is a relatively young method of causal inference that continues to diffuse across the social sciences. However, recent methodological research has found the conservative (QCA-CS) and the intermediate solution type (QCA-IS) of QCA to fail fundamental tests of correctness. Even under conditions otherwise ideal for causal discovery, both solution types frequently committed causal fallacies by presenting inferences that were in direct disagreement with the underlying data-generating structure to be discovered by QCA. None of these problems affected the parsimonious solution type (QCA-PS). These findings conflict with conventional wisdom in the QCA literature, which has it that QCA-CS uses empirical information only and that QCA-IS is preferable to both QCA-CS and QCA-PS. The present article resolves these contradictions. It shows that QCA-CS and QCA-IS systematically supplement empirical data with matching artificial data. These artificial data, however, regularly induce causal fallacies of severe magnitude. Researchers who employ QCA-CS or QCA-IS in empirical analyses thus always risk moving further away from the truth rather than closer to it.

Qualitative Comparative Analysis (QCA) is a relatively young method of causal inference that continues to diffuse across management, political science, sociology, and neighboring areas of social research (Baumgartner and Thiem 2017a:955; Rihoux et al. 2013; Thiem 2017, 2018). At the same time, this diffusion process has been accompanied by a number of methodological debates about QCA. For example, a recent symposium in Comparative Political Studies around Paine (2016) and Thiem, Baumgartner, and Bol (2016) addressed the question of whether QCA’s epistemological foundations are really distinct from those of regression analysis; another symposium in Sociological Methodology around Lucas and Szatrowski (2014) focused in large parts on the question of whether QCA suffers from an inbuilt confirmation bias. ¹

Against this backdrop, Baumgartner and Thiem (2017b) have recently added an important front of inquiry by providing the first comprehensive evaluation of QCA’s three solution types: complex/conservative (QCA-CS), intermediate (QCA-IS), and parsimonious (QCA-PS). Across all of their sets of inverse-search trials, QCA-CS and QCA-IS proved unsound. Both of these solution types frequently committed causal fallacies of varying magnitude by presenting inferences that violated the very causal structure QCA was supposed to recover from a set of data, even under conditions otherwise ideal. For QCA-CS in particular, fallacies were severe: “between diversity values of 81.25 percent and 12.5 percent—the span covering the majority of configurational research designs—correctness does not even exceed 10 percent” (Baumgartner and Thiem 2017b:19-20).

Baumgartner and Thiem’s (2017b) results are all the more relevant because they conflict fundamentally with current convictions in a considerable part of the QCA literature, which have it that QCA-CS relies on empirical information only (Ragin 1987:110; Schneider and Wagemann 2012:162; Vis 2012:174; Wagemann and Schneider 2015:40) and that QCA-IS is preferable to QCA-PS and QCA-CS (De Meur, Rihoux, and Yamasaki 2009:155; Ragin 2008:171, 173; Ragin 2009:111; Ragin and Sonnett, 2005:196; Schneider and Wagemann 2012:175; Schneider and Wagemann 2013:211).

To date, a methodological explanation for Baumgartner and Thiem’s (2017b) results has not been put forward. The present articles fills this gap and resolves the seeming contradiction between Baumgartner and Thiem’s (2017b) findings and conventional views. More precisely, it demonstrates that, and how, QCA-CS and QCA-IS supplement empirical data with artificial data. However, these artificial data are often incompatible with the very causal structure QCA seeks to uncover from empirical facts, whereby causal fallacies of varying magnitude are induced. ²

Current Views on QCA-CS and QCA-IS

In his pioneering book on the comparative method, Ragin (1987) introduced QCA-CS some 30 years ago. Ever since, it has been argued that this solution type is the most conservative one because it draws only on empirical data. Ragin (2008:173) writes that “[t]he complex solution […] does not permit any counterfactual cases and thus no simplifying assumptions regarding combinations of conditions that do not exist in the data.” ³ In support, Schneider and Wagemann (2012:162) maintain that QCA-CS is “[c]onservative because […] the researcher […] is exclusively guided by the empirical information at hand.” ⁴

Despite the apparent advantage of relying on empirical information only, many methodologists soon began to criticize QCA-CS for the high complexity of the solutions it often produced, which supposedly detracted from the interpretability of its findings. In reaction to this perceived shortcoming of QCA-CS and the simultaneously growing skepticism toward QCA-PS, Ragin and Sonnett (2005) devised QCA-IS, which is now widely recommended to applied researchers as the most attractive one of QCA’s three solution types (cf. Baumgartner 2015:840). As Ragin (2008:171) argues, “the intermediate solution […] is optimal because it incorporates only easy counterfactuals, eschewing the difficult ones that have been incorporated into the most parsimonious solution. The intermediate solution thus strikes a balance between complexity and parsimony […].” In consequence, solutions produced by QCA-IS “are superior to both the ‘complex’ and ‘parsimonious’ solutions and should be a routine part of any application of any version of QCA” (Ragin 2009:111). Similarly, Schneider and Wagemann (2012:175) hold that “[t]he rationale for creating intermediate solution terms is that, on the one hand, the conservative solution often tends to be too complex to be interpreted in a theoretically meaningful or plausible manner and that, on the other hand, the most parsimonious solution term risks resting on assumptions about logical remainders that contradict theoretical expectations, common sense, or both” (see also Schneider and Wagemann 2013:211). ⁵

Explaining the Incorrectness of QCA

Given the aforecited claims in conventional QCA literature that QCA-CS avoids counterfactuals and that QCA-IS provides some sort of golden mean between QCA-CS and QCA-PS, many applied QCA studies published over the last decade in management, political science, and sociology have presented solutions based on QCA-CS or QCA-IS to their readers. Against this background, Baumgartner and Thiem (2017b) have demonstrated in the most elaborate method evaluation of QCA to date that both QCA-CS and QCA-IS, in fact, fail to meet fundamental criteria for empirical research methods of causal inference. Irrespective of the form and complexity of the causal structure they presupposed, and irrespective of the amount of data fed into QCA, QCA-CS and QCA-IS often presented inferences that were in direct disagreement with the structure that should have been uncovered. As it was outside the scope of their article, Baumgartner and Thiem (2017b) did not present a methodological explanation for this finding. The remainder of the present article fills this gap.

To understand why QCA-CS and QCA-IS regularly fail in their inferences, the relevant components of the theory of causation QCA posits must first be recalled. In QCA, causes of an effect, in relation to some specified background against which causation is to be inferred about, are functionally understood to be at least INUS conditions with respect to that effect (Cartwright 2007:34-35; Ragin and Strand 2008:431-32; Schneider and Wagemann 2012:79; Thiem and Baumgartner 2016a:803). ⁶ This means causes must be conditions that are insufficient but nonredundant parts of conditions that are themselves unnecessary but sufficient for the analyzed effect or conditions that are themselves minimally sufficient conditions of this effect or conditions that are the only minimally sufficient conditions of this effect (Mackie 1965; Mackie 1974:59-87). ⁷ A more formal paraphrase, one that can be used for methodological purposes, is the following:

Definition: Under the theory of INUS causation, every cause $X^{\{\cdot \}}$ of an effect $Z^{\{\cdot \}}$ relative to some causal field  $\mathcal{F}$ must be a configurational difference-maker to $Z^{\{\cdot \}}$ in $\mathcal{F}$ . Functionally, it is such a difference-maker if a pair of Boolean-algebraic conjunctions $\left(X^{\{\cdot \}}\Phi ,\neg X^{\{\cdot \}}\Phi \right)$ , where $\Phi$ denotes the conjunction of all conditions within $\mathcal{F}$ other than $X^{\{\cdot \}}$ , is instantiated by a set of empirical data $\delta$ , and it holds that $\delta$ establishes the truth of the proposition that $X^{\{\cdot \}}\Phi$ implies $Z^{\{\cdot \}}$ as well as the falsity of the proposition that $\neg X^{\{\cdot \}}\Phi$ also implies $Z^{\{\cdot \}}$ (cf. Baumgartner 2008:335-36; Graßhoff and May 2001:94-97; Thiem and Baumgartner 2016b:347-48).

As usual, “ $X^{\{\cdot \}}$ ” simply refers to an exogenous variable “X” taking on one of its values “ $\{\cdot \}$ ” and “ $Z^{\{\cdot \}}$ ” to an endogenous variable “Z” taking on one of its values $\{\cdot \}$ . Table 1 shows a truth table of the crucial elements contained in this definition and the two implicational requirements for establishing INUS conditionality.

Table 1.

Truth Table for Establishing Insufficient Nonredundant unnecessary sufficient (INUS) Conditionality.

Row	$X^{\{\cdot \}}$	$\Phi$	$Z^{\{\cdot \}}$	$X^{\{\cdot \}}\Phi \Rightarrow Z^{\{\cdot \}}$	$\neg \left(\neg X^{\{\cdot \}}\Phi \Rightarrow Z^{\{\cdot \}}\right)$
1	0	0	0	1	0
2	0	0	1	1	0
3	0	1	0	1	1
4	0	1	1	1	0
5	1	0	0	1	0
6	1	0	1	1	0
7	1	1	0	0	0
8	1	1	1	1	0

Pairs of Boolean-algebraic conjunctions $\left(X^{\{\cdot \}}\Phi ,\neg X^{\{\cdot \}}\Phi \right)$ are contained in rows $\left(3,7\right)$ , $\left(3,8\right)$ , $\left(4,7\right)$ , and $\left(4,8\right)$ . However, as it must additionally be true that $X^{\{\cdot \}}\Phi$ implies $Z^{\{\cdot \}}$ and that $\neg X^{\{\cdot \}}\Phi$ does not imply $Z^{\{\cdot \}}$ , the only pair of rows suitable for testing the causal relevance of $X^{\{\cdot \}}$ with respect to $Z^{\{\cdot \}}$ is $\left(3,8\right)$ . Pairs $\left(3,7\right)$ and $\left(4,7\right)$ are unsuitable because in row 7, the truth values for $X^{\{\cdot \}}$ , $\Phi$ , and $Z^{\{\cdot \}}$ are such that neither $X^{\{\cdot \}}\Phi \Rightarrow Z^{\{\cdot \}}$ nor $\neg \left(\neg X^{\{\cdot \}}\Phi \Rightarrow Z^{\{\cdot \}}\right)$ is true. Pair $\left(4,8\right)$ is also unsuitable because in neither row is $\neg \left(\neg X^{\{\cdot \}}\Phi \Rightarrow Z^{\{\cdot \}}\right)$ true.

For instance, if $Z^{\left\{1\right\}}$ was the outcome to be analyzed, and $X^{\left\{1\right\}}$ the condition whose difference-making power to be evaluated, then there would have to be at least two matching instantiations of conjunctions in $\delta$ , one of $X^{\left\{1\right\}}\Phi Z^{\left\{1\right\}}$ and one of $\left(\neg X^{\left\{1\right\}}\right)\Phi Z^{\left\{0\right\}}$ , that is, $X^{\left\{0\right\}}\Phi Z^{\left\{0\right\}}$ if X and Z are assumed to be bivalent with values $\left(0,1\right)$ . Without the latter, causal relevance could not be attributed to $X^{\left\{1\right\}}$ because the matching conjunction necessary for demonstrating a configurational difference-making situation in relation to Z when going from $X^{\left\{1\right\}}$ to $X^{\left\{0\right\}}$ was missing. Under such and similar circumstances, it can be expected that a correctly working method of empirical data analysis not go beyond the inference that, conditional on $\delta$ , evidence for the causal relevance of $X^{\left\{1\right\}}$ does not exist. At the very least, it can be demanded that the method not attribute causal relevance to $X^{\left\{1\right\}}$ .

Now consider the truth table of three exogenous factors A, B, and C and an endogenous factor F presented in Table 2, which has been adapted from Ragin (1987:106). ⁸ As this table shows limited empirical diversity—two of the eight possible configurations are not observed—the possibility for employing QCA-CS, QCA-IS, or QCA-PS comes into play. ⁹

Table 2.

Hypothetical Truth Table.

A	B	C	OUT $\left(F^{\left\{1\right\}}\right)$	Cases
0	0	0	0	a
0	0	1	1	b
0	1	0	0	c
0	1	1	1	d
1	0	0	0	e
1	0	1	1	f
1	1	0	?	—
1	1	1	?	—

Source. Adapted from Ragin (1987:106).

According to Ragin (1987:105), “[a] conservative statement of what the truth table shows […] is $F=aC+bC$ ,” which corresponds to $A^{\left\{0\right\}}{C}^{\left\{1\right\}}\vee B^{\left\{0\right\}}{C}^{\left\{1\right\}}\iff F^{\left\{1\right\}}$ in the syntax of propositional logic used herein. Given the declared goal of QCA to identify INUS conditions and their respective interplay, QCA-CS thus makes the following set of causal claims (1)–(5) about the unknown data-generating structure, which QCA is meant to uncover, ideally in as much detail as possible:

1. $C^{\left\{1\right\}}$ is causally relevant to $F^{\left\{1\right\}}\left(\text{is \hspace{0.17em} at \hspace{0.17em} least \hspace{0.17em} an \hspace{0.17em} INUS \hspace{0.17em} condition \hspace{0.17em} of \hspace{0.17em}}{F}^{\left\{1\right\}}\right)$ .

2. $A^{\left\{0\right\}}$ is causally relevant to $F^{\left\{1\right\}}\left(\text{is \hspace{0.17em} at \hspace{0.17em} least \hspace{0.17em} an \hspace{0.17em} INUS \hspace{0.17em} condition \hspace{0.17em} of \hspace{0.17em}}{F}^{\left\{1\right\}}\right)$ .

3. $B^{\left\{0\right\}}$ is causally relevant to $F^{\left\{1\right\}}\left(\text{is \hspace{0.17em} at \hspace{0.17em} least \hspace{0.17em} an \hspace{0.17em} INUS \hspace{0.17em} condition \hspace{0.17em} of \hspace{0.17em}}{F}^{\left\{1\right\}}\right)$ .

4. Conjunctively, $A^{\left\{0\right\}}$ and $C^{\left\{1\right\}}$ are causally relevant to $F^{\left\{1\right\}}$ on one path.

5. Conjunctively, $B^{\left\{0\right\}}$ and $C^{\left\{1\right\}}$ are causally relevant to $F^{\left\{1\right\}}$ on another path.

Clearly, these five claims are not all independent of each other: Claim (4) necessitates the truth of claims (1) and (2), claim (5) the truth of claims (1) and (3). Yet, how “conservative,” that is, how faithful to the empirical data, are these claims in actuality? Let us begin with claim (1) about $C^{\left\{1\right\}}$ because this condition appears in both disjuncts of the solution Ragin presents. Does $C^{\left\{1\right\}}$ fulfill the criteria of an INUS condition? There are three pairs of conjunctions instantiated by the data underlying Table 2, namely,

$\begin{array}{l}\left(A^{\left\{0\right\}}{B}^{\left\{0\right\}}{C}^{\left\{1\right\}}\right.\text{[case \hspace{0.17em} b]},A^{\left\{0\right\}}{B}^{\left\{0\right\}}{C}^{\left\{0\right\}}\left.\text{[case \hspace{0.17em} a]}\right)\text{,}\\ \left(A^{\left\{0\right\}}{B}^{\left\{1\right\}}{C}^{\left\{1\right\}}\right.\text{[case \hspace{0.17em} d]},A^{\left\{0\right\}}{B}^{\left\{1\right\}}{C}^{\left\{0\right\}}\left.\text{[case \hspace{0.17em} c]}\right)\text{, and}\\ \left(A^{\left\{1\right\}}{B}^{\left\{0\right\}}{C}^{\left\{1\right\}}\right.\text{[case \hspace{0.17em} f]},A^{\left\{1\right\}}{B}^{\left\{0\right\}}{C}^{\left\{0\right\}}\left.\text{[case \hspace{0.17em} e]}\right)\text{,}\end{array}$

each of which satisfies the requirements for attributing causal relevance to $C^{\left\{1\right\}}$ . The empirical evidence not only establishes the truth of the proposition that the first element in each pair, which contains $C^{\left\{1\right\}}$ , implies $F^{\left\{1\right\}}$ , but also the falsity of the proposition that the second element in each pair, which contains $C^{\left\{0\right\}}$ , implies $F^{\left\{1\right\}}$ . Therefore, $C^{\left\{1\right\}}$ can be declared an INUS condition and claim (1) is warranted.

What about $A^{\left\{0\right\}}$ and $B^{\left\{0\right\}}$ , to each of which QCA-CS attributes causal relevance on a separate path in combination with $C^{\left\{1\right\}}$ ? With regard to $A^{\left\{0\right\}}$ , can a matching conjunction for $A^{\left\{0\right\}}{B}^{\left\{0\right\}}{C}^{\left\{1\right\}}$ , instantiated by case b, or $A^{\left\{0\right\}}{B}^{\left\{1\right\}}{C}^{\left\{1\right\}}$ , instantiated by case d—the only two observed instantiations of a conjunction containing $A^{\left\{0\right\}}$ and implying $F^{\left\{1\right\}}$ —be found? The first candidate for comparison, $A^{\left\{1\right\}}{B}^{\left\{0\right\}}{C}^{\left\{1\right\}}$ (case f) does not qualify because it also implies $F^{\left\{1\right\}}$ , not $F^{\left\{0\right\}}$ ; and the second candidate, $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{1\right\}}$ , is not instantiated in Table 2. Respecting $B^{\left\{0\right\}}$ , can a matching conjunction for $A^{\left\{0\right\}}{B}^{\left\{0\right\}}{C}^{\left\{1\right\}}$ , instantiated by case b, or $A^{\left\{1\right\}}{B}^{\left\{0\right\}}{C}^{\left\{1\right\}}$ , instantiated by case f—the only two observed cases of a conjunction containing $B^{\left\{0\right\}}$ and implying $F^{\left\{1\right\}}$ —be found? The first candidate, $A^{\left\{0\right\}}{B}^{\left\{1\right\}}{C}^{\left\{1\right\}}$ (case d), again, implies $F^{\left\{1\right\}}$ , not $F^{\left\{0\right\}}$ , and is thus unsuitable; and the second candidate, $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{1\right\}}$ , is, again, not instantiated in Table 2. In other words, all of the necessary conjunctions required to empirically demonstrate the configurational difference-making powers of $A^{\left\{0\right\}}$ and $B^{\left\{0\right\}}$ , respectively, do not exist. Therefore, claims (2) and (3) are not warranted, and, by extension, nor are claims (4) and (5).

How can QCA-CS attribute causal relevance to $A^{\left\{0\right\}}$ and $B^{\left\{0\right\}}$ nonetheless, although the data underlying Table 2 do not contain a single case of $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{1\right\}}$ that also shows $F^{\left\{0\right\}}$ , whereby the falsity of the implication for matching conjunctions that lack $A^{\left\{0\right\}}$ , $B^{\left\{0\right\}}$ , respectively, could be established? From the viewpoint of empirical data analysis, the answer is extremely problematic: QCA-CS can only do so by supplementing empirical data with artificial data, namely the presence of $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{1\right\}}$ in conjunction with the absence of $F^{\left\{1\right\}}$ . Put differently, QCA-CS can only reach the inferences it presents by going beyond the facts. ¹⁰

However, the mere augmentation of the empirical data with matching artificial data does not, perforce, render QCA-CS incorrect. After all, QCA-CS may have outstanding predictive properties insofar as it lies about the current existence of a case, but not necessarily about the truth of the corresponding proposition in a future instantiation of that case. So as to understand what exactly causes QCA-CS to ultimately fail, all possibilities for a causal structure $\Delta$ that could have generated the data underlying Table 2 thus need to be considered first. ¹¹ As there are four possibilities for combinations of output function values on the missing configurations in Table 2, four such possibilities for $\Delta$ , call them ${\Delta }_1$ to ${\Delta }_4$ , exist:

$\begin{array}{ll}{\Delta }_1:C^{\left\{1\right\}}\hfill & \iff F^{\left\{1\right\}}\text{,}\hfill \\ {\Delta }_2:C^{\left\{1\right\}}\vee A^{\left\{1\right\}}{B}^{\left\{1\right\}}\hfill & \iff F^{\left\{1\right\}}\text{,}\hfill \\ {\Delta }_3:A^{\left\{0\right\}}{C}^{\left\{1\right\}}\vee B^{\left\{0\right\}}{C}^{\left\{1\right\}}\hfill & \iff F^{\left\{1\right\}}\text{, and}\hfill \\ {\Delta }_4:A^{\left\{0\right\}}{C}^{\left\{1\right\}}\vee B^{\left\{0\right\}}{C}^{\left\{1\right\}}\vee A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{0\right\}}\hfill & \iff F^{\left\{1\right\}}\text{.}\hfill \end{array}$

If ${\Delta }_1$ was the structure in operation behind Table 2, then it could never be true that $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{0\right\}}$ implied $F^{\left\{1\right\}}$ , whereas it could never be false that $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{1\right\}}$ implied $F^{\left\{1\right\}}$ [the combination of output function values $\left(0,1\right)$ on the missing configurations]. If ${\Delta }_2$ was that structure, then it could neither ever be false that $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{0\right\}}$ implied $F^{\left\{1\right\}}$ nor that $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{1\right\}}$ implied $F^{\left\{1\right\}}$ [combination $\left(1,1\right)$ ]. For ${\Delta }_3$ , it could never be true that $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{0\right\}}$ or $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{1\right\}}$ implied $F^{\left\{1\right\}}$ [combination $\left(0,0\right)$ ]. And if ${\Delta }_4$ was behind the data in Table 2, it could never be false that $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{0\right\}}$ did not imply $F^{\left\{1\right\}}$ , whereas it could never be true that $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{1\right\}}$ implied $F^{\left\{1\right\}}$ [combination $\left(1,0\right)$ ].

QCA-CS goes beyond the facts, but it would not commit a causal fallacy if either ${\Delta }_3$ or ${\Delta }_4$ was the true data-generating structure: None of the causal claims (1) to (5) would be fallacious under ${\Delta }_3$ or ${\Delta }_4$ . However, under ${\Delta }_1$ as well as ${\Delta }_2$ , QCA-CS adds artificial data that the causal structure behind the empirical data underlying Table 2 would never generate. ¹² Put differently, the evidence fabricated by QCA-CS may or may not put the researcher on the correct path to the discovery of that $\Delta$ which has given rise to the data being analyzed. In the case of Table 2, the supplementation of empirical data with artificial data by QCA-CS would thus lead to erroneous inferences for half of all data-generating structures that are theoretically possible, in circumstances otherwise ideal for causal discovery.

Although QCA-IS has also been shown to be incorrect by Baumgartner and Thiem (2017b), its performance has always been at least as good as that of QCA-CS, and usually better, but it has never been better than that of QCA-PS, and usually worse. This result can be explained by the distinction QCA-IS makes between so-called easy counterfactuals on the one hand and difficult counterfactuals on the other. It is only the set of difficult counterfactuals that influence whether QCA-IS will commit causal fallacies because only noninstantiated conjunctions of this type can be declared not to be implying the analyzed outcome.

As intermediate solutions are derived from pairings of models from a conservative solution and a parsimonious solution of an analysis that uses the same set of empirical data, QCA-PS must first be applied to Table 2. The parsimonious solution for the truth table in Table 2 consists of one model, namely $C^{\left\{1\right\}}\iff F^{\left\{1\right\}}$ (Ragin 1987:105-6). The set of causal claims made by QCA-PS reduces to claim (1), which violates none of the causal structures ${\Delta }_1$ to ${\Delta }_4$ .

Whether QCA-IS will commit a causal fallacy on $A^{\left\{0\right\}}$ and $B^{\left\{0\right\}}$ or not now depends on whether $A^{\left\{1\right\}}{B}^{\left\{1\right\}}{C}^{\left\{1\right\}}\Rightarrow F^{\left\{1\right\}}$ is declared to be a difficult counterfactual or not by the researcher. ¹³ In case of the former scenario, the solution derived by QCA-IS will coincide with that derived by QCA-CS, and QCA-IS will commit causal fallacies for ${\Delta }_1$ and ${\Delta }_2$ , but not for ${\Delta }_3$ and ${\Delta }_4$ . In case of the latter scenario, the solution derived by QCA-IS will coincide with that derived by QCA-PS, and QCA-IS will not commit any causal fallacy for any $\Delta$ . QCA-IS will, therefore, never fabricate more evidence for $\Delta$ than QCA-CS, and, in consequence, will never fabricate more evidence that, in fact, will conflict with $\Delta$ .

Conclusions

QCA is a relatively young method of causal inference that continues to diffuse across the social sciences. However, recent research has demonstrated that the complex/conservative solution type and the intermediate solution type of QCA fail fundamental tests of methodological correctness for configurational comparative methods. A methodological explanation for this finding has not yet been put forward so far.

The present article has filled this gap. It has been illustrated that, and how, conservative and intermediate solutions introduce matching artificial data which QCA supplements the empirical data with. Yet, these artificial data often lead to inferences that violate the very causal structure that had generated the empirical data in the first place and that QCA is meant to uncover. Researchers who employ QCA-CS or QCA-IS in empirical analyses thus always risk moving (much) further away from the truth rather than closer to it. For producing evidence-based findings not affected by such risks, QCA-PS provides the appropriate method.