Fewer Procedures,More Reflection: A Rejoinder to Duşa and Marx

We thank Adrian Duşa and Axel Marx for their comment on our evaluation of Marx and Duşa (2011; hereafter MD). We believe that such exchanges are indispensable for developing a full understanding of many procedures in qualitative comparative analysis (QCA) that have for too long remained entirely unquestioned. We welcome this exchange, but we will also demonstrate in our rejoinder why Duşa and Marx’s comment once more underlines the urgent need for more reflection on QCA, all the more so because QCA’s algorithmic core—Boolean optimization—has been swiftly imported in the 1980s from electrical engineering without much reflection at all. ¹

Contrary to our expectations, Duşa and Marx do not take issue with the core part of Thiem and Mkrtchyan (2023): the extensive simulations we have carried out and their results that have been summarized in Figure 2. Instead, their comment focuses on: (1) our failure to propose an enhancement of their benchmarks; (2) our use of constants in QCA; (3) our neglect of logical remainders; and (4) our reanalysis of the applied study by Muriaas et al. (2022). We successively address each of these points.

(1) “[T]heir paper does not present any replacement solution over MD, nor do they somehow improve on MD.”

If your doctor tells you that smoking is unhealthy, the best thing to do is to quit, not to replace cigarettes with cigars or to “improve” your cigarettes with flavored filters. To stop smoking is the most sensible fix. We did not seek to replace MD’s benchmarks, nor did we seek to enhance them. Our sole aim was to show that adherence to these benchmarks creates more harm than previously acknowledged, and that QCA does not lose its capability to distinguish real from random data. ²

(2) “First, all causal conditions except one are constant. […] No one would include 29 constant variables in a model.”

Here, Duşa and Marx take issue with the set-up behind Table 1 in Thiem and Mkrtchyan (2023). In note 9 of Thiem and Mkrtchyan (2023), we provided the full explanation of why we let only one exogenous factor in that table vary, and not two, 10, or 20. Duşa and Marx may have overlooked this note.

(3) “For 30 causal conditions where only two cases are observed, there are more than a billion remainders to be employed […]. That is anything but science.”

Here, Duşa and Marx again take issue with the set-up behind Table 1 in Thiem and Mkrtchyan (2023). This time, however, their criticism illustrates emblematically how disconnected the QCA community is from the very field from which it has once borrowed its algorithmic core. Since Ragin’s import of the Quine-McCluskey algorithm (QMC) from electrical engineering in Ragin et al. (1984), QCA researchers have worried about QMC’s use of don’t care (dc)-terms—called “logical remainders” in QCA—and have proposed various procedures, such as “intermediate solutions” or “enhanced standard analyses,” that manipulate some or all of these dc-terms so that QMC is blocked from accessing them (see Thiem 2022b).

More precisely, QCA researchers fear that the use of a dc-term by QMC is an “attempt to remove a contributing causal condition from a configuration displaying the outcome, on the assumption that this cause is redundant and the reduced configuration would still produce the outcome” (Ragin 2008:162). That is a serious misinterpretation. When QMC incorporates a dc-term during optimization, it does not mean this term is assumed to occur in conjunction with the outcome to remove a condition as redundant. The reason lies in the formal definition of the operation of Boolean conditionality, which—and this speaks for itself—no QCA textbook in the last 30 years has presented to its readers! We make up for this gap in Table 1 below, where “→,” as usual, represents the symbol for this operation. ³ The left-hand side of “→” is called “antecedent,” the right-hand side “consequent.”

Table 1.

Formal Definition of Boolean Conditionality.

Antecedent	Consequent	Antecedent → Consequent
True	True	True
True	False	False
False	True	True
False	False	True

Before QMC can use a dc-term, the Boolean conditional associated with this term must be declared true. But declaring the conditional to be true neither implies that the antecedent is declared true nor that the antecedent is associated with a true consequent. For an antecedent that is false, which a (non-existing or impossible) dc-term clearly is, a Boolean conditional is true no matter the consequent (rows 3 and 4 of Table 1). ⁴ In contrast, when QCA researchers demand that some dc-terms be declared “difficult counterfactuals” so that QMC is blocked from incorporating them, they declare the conditional to be false, thereby turning a non-existing or impossible antecedent into a true one in conjunction with a false consequent (row 2 of Table 1). Effectively, such manipulations are tantamount to the fabrication of data because dc-terms are turned into existing cases that show the absence of the analyzed outcome.

Irrespective of this misinterpretation of the Boolean conditional, an obvious question suggests itself: When being so afraid of QMC’s incorporation of dc-terms, why have QCA researchers not simply imported another algorithm that is equivalent yet does not use dc-terms? After all, electrical engineers have also developed QMC-equivalent algorithms that operate only on off-terms and on-terms, but not dc-terms (e.g., McCluskey 1962). Particularly for truth tables with many dc-terms, such algorithms are considerably faster. Had Ragin imported one of these algorithms instead of QMC in the 1980s, no QCA literature on logical remainders (which we estimate to make up the largest share of methodological QCA literature) would have ever seen the light of day.

(4) “It is not clear how they arrive at 22 rival models and how these models compare to the one(s) of Muriaas et al. (2022). […] Nor is it clear what they mean with ‘rival models that fit the data equally well.’ How do they assess that rival models fit the data equally well? Does this mean each model as the same consistency and/or coverage? We would be surprised.”

This last point of criticism does not relate to our methodological evaluation of MD’s benchmarks, either, but to the issue of model ambiguity in an applied QCA study. Model ambiguity in QCA occurs when multiple models fit a set of data equally well. This problem has first been noted in a QCA software review by Thiem and Duşa (2013) and has since been extensively analyzed in Baumgartner and Ambühl (2020), Baumgartner and Thiem (2017), and Thiem et al. (2020). In note 14 of Thiem and Mkrtchyan (2023), we pointed out that Muriaas et al. (2022) have disallowed the identification of model ambiguity by activating row dominance and deactivating the identification of all models in the software they employed. With the two omitted factors included, but while keeping the (incorrect) software settings of row dominance and model suppression activated, 22 models result, all of which have exactly the same consistency and coverage scores. These results can be easily verified with the replication material we have provided on https://osf.io/ea5df/. The link to this repository, which contains all replication material for Thiem and Mkrtchyan (2023), is also given in that article’s acknowledgment section.

Conclusions

We sincerely thank Adrian Duşa and Axel Marx for their comment on our evaluation of their benchmark tables of case-to-factor ratios in QCA. Such exchanges are indispensable for moving configurational research forward. However, and contrary to our expectations, their comment did not address the core part of our evaluation in Figure 2 of Thiem and Mkrtchyan (2023). Instead, it focused on the use of logical remainders and questions of model ambiguity, two additional issues about which a lot of confusion and disorientation seems to exist in QCA.

Given these larger problems, we do not want to conclude this rejoinder with a simple call to discontinue using MD’s benchmarks. Instead, we suggest a considerably more effective solution: Applied researchers who would like to employ configurational comparative methods in their research should have a look at coincidence analysis (CNA; Baumgartner and Ambühl 2020; Whitaker et al. 2020) and combinational regularity analysis (CORA; Thiem et al. 2022). Both CNA and CORA stand on a solid epistemological footing, have clearly defined search targets, and offer analytical capabilities far more advanced than those of QCA.