An incomplete recipe: One-dimensional latent variables do not capture the full flavor of democratic support

Abstract

Prominent recent works have measured democratic support using a single latent variable that purports to span a single dimension from steadfast opposition to whole-hearted support. This ignores ample evidence that support for democracy is complex and multidimensional. Here we provide a series of validation tests of the sort of cross-national time-series latent variable measures employed in recent research by reference to questions on support for liberal democracy and opposition to its erosion from multi-wave surveys conducted around the world. These tests show that, across countries and years, this latent variable is nearly orthogonal to measures of support for contestation and participation; civil liberties; institutional constraints on executive power; and prioritizing democracy over the economy, economic equality, or order. We conclude that support for democracy in any robust sense is simply not well captured by one-dimensional latent variable. Such measures are powerful but researchers must be mindful of their limitations.

Introduction

Recent threats to liberal democracy have manifested as the slow erosion of institutions and norms rather than the sudden and violent takeover of power, the acts of politicians and elected officials rather than admirals and generals. Considering that the ostensible backing for these antidemocratic office holders is some substantial share of the electorate, understanding the breadth and depth of citizen support for democracy is as important as ever. On one hand, the breadth of that support for democracy remains truly impressive: Anderson et al. (2021, 971–72), for example, estimates that some 90% of humanity agrees that democracy is the best form of government (see also, e.g., Kirsch and Welzel 2019; Wuttke et al. 2022). But on the other, a great deal of evidence indicates that these expressions of support for democracy do not run very deep, that such claims to favor democracy in the abstract do not consistently represent either commitments to liberal democracy or opposition to actions that undermine it (see, e.g., Bratton 2002; Schedler and Sarsfield 2007; Carlin and Singer 2011; Kiewiet De Jonge 2016; Bryan 2023). As a systematic review of this literature emphasizes, “it is important to know not just how strongly citizens support democracy, but also what kind of democracy it is that they support” (König et al. 2022).

These cautions notwithstanding, a number of prominent recent works have sought to better investigate democratic support by taking advantage of new latent-variable models of public opinion to estimate support for democracy across many countries and over long spans of time. In this approach, responses to many different survey questions tapping support for democracy in the abstract are combined to overcome the sparse and scattered collection of data available on any single question; the differences among the various questions used as indicators are accounted for with careful modeling (see, e.g., Claassen 2019; Solt 2020). The resulting measure is then uncritically employed to explore the trends, determinants, and consequences of democratic support (Claassen 2020a, 2020b; Claassen and Magalhães 2022; Jacob 2024; Tai et al., 2024).

Several narrow critiques of this line of research have already been raised (see Hu et al. 2024; Tannenberg 2022), but we offer here a more fundamental criticism foreshadowed by the many works reviewed in König et al. (2022): democratic support is multidimensional and therefore cannot be captured by any single measure. In other words, although the latent variables employed in this research adequately summarize the survey questions on which they are based—which ask the extent of support for democracy in the abstract or for some similarly abstract non-democratic regime—these questions do not account for the complexity of public attitudes to democracy. As Schedler and Sarsfield (2007, 637) noted nearly two decades ago, there exists a substantial number of people “who are sympathetic to democracy in the abstract, while hostile to core principles of liberal democracy in particular.” That piece listed four problems with questions about democracy in the abstract that could cause this pattern: the desire to give interviewers the socially acceptable answer, empty conceptions of democracy, competing conceptions of democracy, and other values that conflict with support for democracy (Schedler and Sarsfield 2007, 638–40; see also, e.g., Carlin and Singer 2011; Kiewiet De Jonge 2016; Bryan 2023). Regardless of the pattern’s sources, we contend that compiling many questions about democracy in the abstract into a single one-dimensional latent variable should not be expected to change the fact that such questions do not accurately assess this attitudinal complexity.

We therefore test the validity of the sort of one-dimensional measure used in the recent line of latent-variable-driven research by comparing it to responses to questions drawn from many surveys used in the broader literature to map the dimensions of support for liberal democracy and opposition to its erosion. The results of these validation tests are striking. Across countries and years, the latent variable of democratic support is essentially orthogonal to these measures of supporting contestation and participation; civil liberties; and institutional constraints on executive power; as well as of prioritizing democracy and political freedom over the economy, economic equality, and order. Support for democracy in any sort of robust sense is simply not well captured by a single latent variable.

We draw two conclusions. First, researchers who construct latent variables must be attentive to issues of multidimensionality and provide ample validation for their measures. The still novel approach to estimating latent variables across countries and over time is a powerful tool, but it does not alone solve all the difficulties of measuring public attitudes. Second, any research employing a single latent variable of democratic support—such as Claassen (2020a), which at the time of writing has already been cited nearly 300 times—should be viewed with profound skepticism.

The one-dimensional latent variable of democratic support

To provide the strongest test of the validity of the one-dimensional latent variable of democratic support, we create the best version of such a variable currently possible. Following the practice of previous research employing these latent variables, we start by identifying survey questions that asked respondents to choose between democracy and an undemocratic alternative or to evaluate either democracy or one of these undemocratic alternatives; the most frequently fielded of these survey questions asks respondents whether they strongly disagree, disagree, agree, or strongly agree with the statement, “Democracy may have its problems but it is still the best form of government,” the so-called Churchill item. Our collection of these questions of support for democracy in the abstract is similar to but expands on the data presented in Claassen (2020a, 2020b) and the larger set collected by Tai et al. (2024). In all, we identified 41 survey items on support for democracy in the abstract that were asked in no fewer than five country-years in countries surveyed at least three times; these items were drawn from 221 different survey datasets. In accordance with the advice offered by Hu et al. (2022) to avoid data-entry errors by automating data collection, we then used the DCPOtools R package (Solt et al., 2019) to compile the responses to these questions. Finally, we estimated a one-dimensional latent variable of democratic support from these responses using the population-level two-parameter ordinal-logistic item-response theory (IRT) model with country-specific item-bias terms presented in Solt (2020); that work shows that this model provides a better fit to survey questions on support for democracy in the abstract than the one-parameter logistic IRT model principally employed in the existing research on this topic, Claassen’s (2019) Model 5. The resulting set of one-dimensional latent variable estimates comprises 2,937 country-year observations in 136 countries. Full details on our collection of survey-question indicators of democratic support and on the resulting latent variable estimates can be found in Online Appendix A. The very similar results to the tests below that are obtained when using the estimates generated by Model five from the smaller dataset of indicator questions employed in Claassen (2020a, 2020b) and Tai et al. (2024) are presented in Online Appendix C.

Testing the validity of democratic support as a one-dimensional latent variable

To test the validity of the one-dimensional latent variable measure of democratic support, we compare it to a range of questions from multi-wave cross-national surveys that also address various aspects of support for democracy; that is, we provide a series of tests of convergent validation (see Adcock and Collier 2001, 540). Because these survey questions were not used to estimate the latent variable—again, those indicators ask respondents to evaluate “democracy” or a specific undemocratic alternative in the abstract—they constitute “external” validation tests (see Caughey, O’Grady, and Warshaw 2019, 684–85). We use questions that reference Dahl’s (1971, 3–4) two “theoretical dimensions of democratization,” public contestation and inclusive participation, and the institutionally guaranteed civil rights and liberties they comprise. These criteria of liberal democracy not only comprise “the central construct in the literature on citizen preference for democracy” (König et al. 2022, 2026) but are also directly implicated in the recent apprehensions regarding democratic backsliding (e.g., Gora and Wilde, 2022; Ananda and Dawson 2023; Meyerrose 2024). In light of recent concerns that citizens do not always prioritize the preferences for democracy they express in the abstract, we also draw on questions that assess democracy’s relative importance.

We start with survey questions that most directly address public contestation and inclusive participation. For example, the AmericasBarometer asks whether chief executives should “limit the voice and vote of opposition parties,” the Asian Barometer questions whether “people with little or no education should have as much say in politics as highly educated people,” and at the two dimensions’ point of greatest overlap, the Afrobarometer asks whether “we should choose our leaders in this country through regular, open and honest elections.” Scatter plots of the aggregate responses to these questions and other similar items against the one-dimensional latent variable of support for democracy in the abstract are presented in Figure 1. It is evident from a glance that all of these relationships are extremely weak, essentially null. Support for democracy in the abstract as measured by a one-dimensional latent variable has very little do with attitudes toward these core features of liberal democracy.

Figure 1.

Correlations between democratic support as a one-dimensional latent variable and polyarchy survey items.

Next, we consider the survey questions that examine the level of support for civil liberties. We compiled four items regarding free speech, three on freedom of the press, three more on freedom of assembly, and two that deal with freedom of association. Scatter plots of the aggregate responses to these questions against the one-dimensional latent variable of democratic support are presented in Figure 2. All of these correlations are weak to very weak, and several are in the opposite of the expected direction. For example, across five waves of the Asian Barometer, the one-dimensional latent variable of democratic support has a positive relationship (R = 0.14) with aggregate agreement with the statement, “The government should decide whether certain ideas should be allowed to be discussed in society.” Support for democracy in the abstract appears not to conflict with favoring government censorship in those countries and years. The strongest relationship with democratic support so measured is found with an item on free speech from the surveys from the AmericasBarometer, which asked respondents the extent of their approval of “people who only say bad things about” the country’s form of government “appearing on television to make speeches.” That correlation reaches only R = −0.39, and even then, it stands out as an exception rather than the rule. The one-dimensional latent variable of democratic support does not capture much of the variation across countries and over time of public demand that civil liberties are respected.

Figure 2.

Correlations between democratic support as a one-dimensional latent variable and civil liberties survey items.

Figure 3 turns to the institutions that guarantee citizens’ rights and liberties from government abuse, that is, courts and the rule of law, and those that make government policies depend on citizen preferences, that is, legislatures. The one-dimensional latent variable relates only at best weakly with aggregated rule of law items like the Asian Barometer question, “It is ok for the government to disregard the law in order to deal with the situation, when the country is facing a difficult situation.” Its correlations with items specifically on legislative checks, like the AmericasBarometer’s “when the Congress hinders the work of our government, our presidents should govern without the Congress,” are perhaps more consistent, but remain unimpressive. Support for democracy in the abstract, even in the form of a latent variable, has little relationship with support for the checks and balances of horizontal accountability. As mentioned above, recent research has documented that many who proclaim support for democracy in the abstract nevertheless have other preferences—such as those regarding policy or partisanship—that they may consider to be more important priorities (e.g., Graham and Svolik 2020; Simonovits et al. 2022; see also Krishnarajan, 2023). Survey questions on the public’s prioritization of democracy are relatively rare, but we found seven relevant items. Most straightforwardly asked respondents to choose between democracy or political freedom on the one hand and maintaining order, reducing economic inequality, or spurring economic development on the other, though the AmericasBarometer asked, “would a military coup be justified under the following circumstances: when social protest is high.” Figure 4 shows that the one-dimensional latent variable of democratic support bears almost no relationship at all to the aggregated responses to these questions.

Figure 3.

Correlations between democratic support as a one-dimensional latent variable and democratic institutions survey items.

Figure 4.

Correlations between democratic support as a one-dimensional latent variable and prioritization survey items.

Conclusions

We draw two straightforward conclusions, the first methodological and the second substantive. From a methodological standpoint, we note that advances in computer hardware and Bayesian software have made estimating latent variables a powerful tool for studying public opinion. But latent variables of public opinion, no matter how sophisticated, are not guaranteed to be good measures of the concepts that are important to us. That the latent variable of public opinion we examine here bears little relationship to survey items more directly tapping the concept this variable was meant to describe underscores that, as with any newly proposed measure, validation tests of such latent variables are crucially important (exemplars include Caughey et al., 2019, 686–91; Woo et al. 2023, 772–73). Measures that fall short in—or are not even subjected to—validation tests cannot serve as solid foundations for research.

To better capture the multidimensionality of democratic support, two potential approaches can be considered. The first is a confirmatory method, where separate one-dimensional models are fitted to distinct subsets of items, as illustrated by Caughey and Warshaw (2018) in its study of the multidimensional nature of public ideology. The second option is an exploratory approach, such as that employed by Pan and Xu (2018), to reveal the underlying dimensions. A more recent advancement in this field is presented in Berwick and Caughey (2025), which utilizes an exploratory group-level Bayesian IRT model to map the dynamic multidimensionality of public opinion within and across country contexts.

Substantively, our findings reinforce the extensive literature on democratic support. Support for democracy is a complex, multidimensional concept, and many who profess to support democracy in the abstract nevertheless also endorse a variety of illiberal and undemocratic actions. Therefore, one-dimensional measures—including those used in Claassen (2020a, 2020b), Claassen and Magalhães (2022), Tai et al. (2024), and Jacob (2024)—are inappropriate both as a general matter and in particular with regard to research on the erosion of liberal democracies: support for democracy in any robust sense is simply not well captured by a one-dimensional latent variable. Future research on the relationship between public opinion and democratic backsliding will need to take the multidimensionality of democratic support into account.

Moreover, at the individual level, there has been a sustained effort to identify the determinants of democratic support by comparing societies with different socioeconomic and political conditions. ¹ These studies contribute to understanding the psychological and sociological mechanisms behind the formation of democratic support, though many still treat democratic support as a one-dimensional scale—whether framed as support/opposition or support for procedural/outcome aspects of democracy. Given the country-level findings in this paper, it would be valuable for future research to explore whether similar issues arise with respondent-level indicators of democratic support. Such within-country discoveries could offer new insights into the formation and evolution of democratic support and help explain the inconsistencies in existing studies.

Supplemental Material

Supplemental Material - An incomplete recipe: One-dimensional latent variables do not capture the full flavor of democratic support

Supplemental Material for An incomplete recipe: One-dimensional latent variables do not capture the full flavor of democratic support by Yue Hu, Yuehong Cassandra Tai, Kyein Ko, Byung-Deuk Woo and Frederick Solt in Community Research & Politics.