Sage Journals: Discover world-class research

Abstract

Conjoint experiments are often used to mimic political choices that people face, such as voting for public officials or selecting news stories. Conjoint designs, however, do not always mirror the real-world decision-making contexts that individuals engage in because respondents are typically forced to select one of the available options. Theoretically, we illustrate how offering respondents an abstention option can produce average marginal component effects (AMCEs) of differing signs and magnitudes relative to a forced-choice outcome. This difference depends on (1) the proportion of respondents who would rather abstain than select profiles lacking their preferred attribute-levels and (2) those respondents’ preference orderings. Empirically, we replicate two conjoint experiments and demonstrate how omitting a realistic abstention option could lead to different AMCE estimates.

Keywords

Conjoint analysis abstention survey experiments

Introduction

Social scientists frequently use conjoint experiments to study a wide array of decisions that humans encounter. Conjoint experiments ask respondents to evaluate two or more alternatives that differ along a set of characteristics, often requiring participants to indicate which of the available alternatives they most prefer.¹ This “forced-choice” design has been used to measure individuals’ preferences over candidates in elections (Carlson, 2015), economic policies (Chilton et al., 2020), and news stories (Mukerjee and Yang, 2020). Yet, participants may ordinarily abstain from these types of decision-making processes entirely in daily life if they find none of the choices suitable.

We outline theoretically how the average marginal component effects (AMCEs) associated with attribute-levels in a forced-choice design can differ in sign or magnitude relative to the AMCEs for the same attribute-levels in a design allowing for abstention. Importantly, researchers cannot know beforehand how any of the AMCEs obtained in a forced-choice design compare to the estimates that would arise from a design with an abstention option. We then replicate two conjoint experiments that used forced-choice outcomes (Funck and McCabe, 2022; Mummolo, 2016) to investigate how including an abstention option could yield statistically and substantively different conclusions.

Forced-choice outcomes, abstention, and AMCEs

The standard conjoint experiment asks each respondent (i ∈ 1, …, N) to evaluate a fixed number of tasks (k ∈ 1, …, K). Each task presents respondents with a fixed number of profiles (j ∈ 1, …, J) that consist of randomly assigned levels for each attribute (l ∈ 1, …, L). After viewing the profiles in each task, respondents express their preferences toward those profiles, most commonly with a “forced-choice” outcome that requires respondents to indicate the profile they most prefer. These choices are then typically used to calculate AMCEs, which represent the probability a profile with a given attribute-level is selected relative to a randomly-generated profile with that attribute’s baseline level (Hainmueller et al., 2014: 19).²

A key assumption of the conjoint framework is that “respondents must choose one preferred profile j within each choice task k” (Hainmueller et al., 2014: 7). However, for many of the decision-making contexts to which conjoint experiments are applied, ample evidence exists that many people abstain when prompted to make a choice because they do not hold an opinion or do not approve of their choices. For instance, individuals’ decision to cast a ballot for one of the available candidates or parties in an election is far from universal, even in countries with compulsory voting (Blais, 2006).

Notably, abstention is often non-random and certain types of respondents, as defined by their preference orderings, might be more likely to abstain. For instance, partisans’ willingness to consume news is influenced by whether their party is advantaged or disadvantaged by salient events (Kim and Kim, 2021). Specific to survey taking, when given the option to respond “don’t know,” individuals in lower socioeconomic strata, women, and racial minorities may be more likely to abstain (Berinsky, 2008).³ As such, when forced-choice outcomes are employed in conjoint experiments, it is reasonable to anticipate that some respondents may be artificially induced to make a choice. We focus on the mechanical issues that arise when we force respondents’ to elicit preferences and how it may yield different estimates of the AMCEs than would manifest if respondents could abstain.⁴

Implications of a forced-choice outcome on AMCEs

We adapt a stylized conjoint experiment from Abramson et al. (2022) in which N = 5 voters are asked to evaluate pairs of candidates based on their positions on two policy issues, which serve as our attributes. For each issue, candidates can take one of two positions, which constitute our levels. First, say on abortion policy, voters can be pro-life (PL) or pro-choice (PC). Second, on tax policy, voters can support increasing (I) or decreasing (D) taxes on the upper-class. For simplicity, we assume all voters prioritize candidates’ abortion policy stances to candidates’ tax policy stances, and that voters’ abortion and tax policy preferences are ideologically coherent. In other words, all voters preferring pro-life candidates (PL > PC) are conservatives and therefore also prefer candidates pledging to cut taxes (D > I). To start, let three of the voters be conservatives and two of the voters be liberals. The full preference ordering for the four possible candidate types for the five voters is presented in Table 1a.

Table 1.

Individual and Aggregate Preferences Over Candidate Profiles With Differing Policy Positions.

(a) Voter preferences
Rank	V1	V2	V3	V4	V5
1	PL, D	PC, I	PL, D	PC, I	PL, D
2	PL, I	PC, D	PL, I	PC, D	PL, I
3	PC, D	PL, I	PC, D	PL, I	PC, D
4	PC, I	PL, D	PC, I	PL, D	PC, I

(b) Aggregate preferences
Comparison	V1	V2	V3	V4	V5	No abstentions	Uniform abstentions	Pro-life abstentions	Pro-choice abstentions
1 PL, D vs. PC, I	PL, D	PC, I	PL, D	PC, I	PL, D	3,2	3,2	3,2	3,2
2 PL, D vs. PC, D	PL, D	PC, D	PL, D	PC, D	PL, D	3,2	3,2	3,2	3,2
3 PL, D vs. PL, I	PL, D	PL, I	PL, D	PL, I	PL, D	3,2	3,0	3,2	3,0
4 PL, I vs. PC, I	PL, I	PC, I	PL, I	PC, I	PL, I	3,2	3,2	3,2	3,2
5 PL, I vs. PC, D	PL, I	PC, D	PL, I	PC, D	PL, I	3,2	3,2	3,2	3,2
6 PC, D vs. PC, I	PC, D	PC, I	PC, D	PC, I	PC, D	3,2	0,2	0,2	3,2

Notes: PL = Pro-life, PC = Pro-choice, D = Decrease upper-class taxes, I = Increase upper-class taxes. In the four rightmost columns of Table 1b, the first and second numbers indicate the number of voters preferring the first and second candidates in the comparison pair given respondents’ ability to abstain (as indicated by the column headings).

We use the voter preference rank-orderings in Table 1a to determine the electorate’s aggregate preferences for each unique combination of candidate profiles in Table 1b. The final four columns of the table provide the vote tallies for each candidate comparison under different forced-choice scenarios: (1) when voters must vote for the candidate they prefer (“No Abstentions”); (2) when all voters abstain if neither candidate possesses their most preferred level of their prioritized attribute, abortion policy (“Uniform Abstentions”); (3) when voters 1, 3, and 5 abstain if neither candidate is pro-life, but voters 2 and 4 always vote (“Pro-life Abstentions”); and (4) when voters 2 and 4 abstain if neither candidate is pro-choice, but voters 1, 3, and 5 always vote (“Pro-choice Abstentions”). In the first scenario, the first candidate profile wins each comparison by a 3 to 2 margin, but if abstentions are allowed, the winning candidate and margin of victory differ only when both candidates share the same abortion policy position (Table 1b, Rows 3 and 6).

Finally, as in Abramson et al. (2022), we use Proposition 3 in Hainmueller et al. (2014: 16) to calculate the AMCEs. To do so, we first obtain the difference in the number of votes a candidate with one level of an attribute would receive compared to a candidate with the other level of that same attribute, holding the second attribute constant, when pitted against each possible candidate. Following Hainmueller et al. (2014: 7), we code votes as 1 when a candidate is selected and 0 when it is not. We then sum these differences and normalize the sums by the product of the number of possible profiles (4), number of possible profiles with a fixed level of one of the two attributes (2) and the number of voters (5). So, the denominator is calculated as the number of unique profiles times the number of voters times the number of possible profiles with the unique levels of copartisanship and corruption (i.e., 4 × 5 × 2).

The AMCEs for each scenario are shown in Table 2. When using a forced-choice outcome that requires all voters to cast a ballot, the AMCE for pro-life is 0.10 (Table 2a, Column 1) and the AMCE for cutting taxes is 0.05 (Table 2b, Column 1). However, when voters are allowed to abstain if neither candidate in the matchup has their preferred abortion policy stance, the AMCE for pro-life increases to 0.15, while the AMCE for cutting taxes decreases to 0.025 (Table 2, Column 2).

Table 2.

AMCEs for Abortion and Tax Policies Varying Abstention Options.

	No abstention	Uniform abstention	Pro-life abstentions	Pro-choice abstentions
(a) Pro-life
1 $\bar{Y} (P L, D; P L, D)$ - $\bar{Y} (P C, D; P L, D)$	0.50	−0.50	0.50	−0.50
2 $\bar{Y} (P L, D; P L, I)$ - $\bar{Y} (P C, D; P L, I)$	1	1	1	1
3 $\bar{Y} (P L, D; P C, D)$ - $\bar{Y} (P C, D; P C, D)$	0.50	2	2	0.50
4 $\bar{Y} (P L, D; P C, I)$ - $\bar{Y} (P C, D; P C, I)$	0	3	3	0
5 $\bar{Y} (P L, I; P L, D)$ - $\bar{Y} (P C, I; P L, D)$	0	−2	0	−2
6 $\bar{Y} (P L, I; P L, I)$ - $\bar{Y} (P C, I; P L, I)$	0.50	−0.50	0.50	−0.50
7 $\bar{Y} (P L, I; P C, D)$ - $\bar{Y} (P C, I; P C, D)$	1	1	1	1
8 $\bar{Y} (P L, I : P C, I)$ - $\bar{Y} (P C, I; P C, I)$	0.50	2	2	0.50
AMCE	= 4/40 = 0.10	= 6/40 = 0.15	= 10/40 = 0.25	= 0/40 = 0.00
(b) Cutting taxes
1 $\bar{Y} (P L, D; P L, D)$ - $\bar{Y} (P L, I; P L, D)$	0.50	1.50	0.50	1.50
2 $\bar{Y} (P L, D; P L, I)$ - $\bar{Y} (P L, I; P L, I)$	0.50	1.50	0.50	1.50
3 $\bar{Y} (P L, D; P C, D)$ - $\bar{Y} (P L, I; P C, D)$	0	0	0	0
4 $\bar{Y} (P L, D; P C, I)$ - $\bar{Y} (P L, I; P C, I)$	0	0	0	0
5 $\bar{Y} (P C, D; P L, D)$ - $\bar{Y} (P C, I; P L, D)$	0	0	0	0
6 $\bar{Y} (P C, D; P L, I)$ - $\bar{Y} (P C, I; P L, I)$	0	0	0	0
7 $\bar{Y} (P C, D; P C, D)$ - $\bar{Y} (P C, I; P C, D)$	0.50	−1	−1	0.50
8 $\bar{Y} (P C, D; P C, I)$ - $\bar{Y} (P C, I; P C, I)$	0.50	−1	−1	0.50
AMCE	= 2/40 = 0.05	= 1/40 = 0.025	= −1/40 = −0.025	= 4/40 = 0.10

Notes: PL = Pro-life, PC = Pro-choice, D = Decrease upper-class taxes, I = Increase upper-class taxes.

To understand why the AMCEs change in opposite directions under uniform abstention, we focus on the matchups in which the vote tallies diverge under the forced-choice and uniform abstention scenarios. For instance, for the pro-life AMCE, consider the comparison of $\bar{Y} (P L, D; P C, I)$ - $\bar{Y} (P C, D; P C, I)$ (Table 2a, Row 4); voters’ aggregate preference for pro-life candidates is stronger when abstention is allowed because pro-life voters no longer cast votes for pro-choice candidates when no pro-life candidates are present (e.g., the PC, D; PC, I matchup). Put differently, voters no longer dilute their preferences over candidates’ abortion policy stances by selecting candidates who do not share their own position.

Alternatively, for the cutting taxes AMCE, because decreasing taxes is linked with being pro-choice, when there is uniform abstention along the abortion dimension there are fewer voters that prefer to cut taxes contributing to the AMCE (Table 2a, Rows 7 and 8). For instance, in the $\bar{Y} (P C, D; P C, I)$ - $\bar{Y} (P C, I; P C, I)$ comparison, voters who prefer a pro-life candidate will not vote if there is a pro-choice candidate, which reduces the aggregate preference toward cutting taxes.

In the final two columns of Table 2, we consider how the pro-life and cutting taxes AMCEs change when only pro-life or only pro-choice voters abstain. First, when pro-choice voters always vote but pro-life voters abstain if no candidates espouse pro-life positions, we see that the pro-life AMCE increases further to 0.25 (Table 2a, Column 3). Pro-choice voters diminish their aggregate preference for pro-choice candidates by casting votes for pro-life candidates when no pro-choice candidates are available. Yet, pro-life voters do not vote when confronted with only pro-choice candidates.

The cutting taxes AMCE, however, decreases to -0.025 indicating that the electorate’s aggregate preference on tax policy is in support of candidates pledging to raise taxes on the upper-class despite a majority of voters preferring candidates pledging to cut those taxes. Pro-life voters no longer express their preference for pro-choice candidates pledging to cut taxes (Table 2b, Column 3, Rows 7 and 8), but pro-choice voters continue to cast ballots for pro-life candidates pledging to raise taxes. In aggregate, these expressed preferences yield a negative, rather than positive, AMCE for cutting taxes.

Finally, when only pro-choice voters abstain, we observe that the AMCE for pro-life is 0.00 even though the majority of voters prefer pro-life candidates. This quantity arises because pro-choice voters no longer vote when only pro-life candidates are present (Table 2a, Column 4, Rows 1, 5, and 6), but pro-life voters still cast ballots for pro-choice candidates when no pro-life candidates are available. Meanwhile, the AMCE for cutting taxes increases to 0.10 as pro-choice voters no longer express their preference for increasing taxes by voting for pro-life candidates pledging to raise taxes (Table 2b, Column 4, Rows 1 and 2). As pro-life voters outnumber pro-choice voters, these abstentions increasingly boost the pro-life AMCE. These examples of uniform and asymmetric abstention showcase that the consequences of using forced-choice outcomes in conjoint experiments are not straightforward. Both the magnitudes and signs of AMCEs obtained using a forced-choice outcome may substantively differ from a design in which respondents are not forced to express preferences they do not hold.

To illustrate how the rate of abstention among voters with different preference orderings impacts the AMCEs, we extend our example to N-voters in the Supplemental Material. We find that the differences in the AMCEs are based on the distribution of persons in the sample with each preference ordering and the rate of abstention among those who abstain if none of the profiles presented contain the attribute-level they prioritize. In other words, these differences depend on which type of respondents would abstain, as described by their preference orderings, and how many respondents of each type would abstain.⁵

Unfortunately, researchers cannot know beforehand which types of respondents, as defined by their preference orderings, would rather abstain than select profiles lacking their preferred attribute-levels. Further, if respondents with certain preference orderings are more likely to abstain, forced-choice outcomes compel respondents to reveal preferences they would otherwise hold. Therefore, researchers cannot rely on any “rules of thumb” to speculate ex post on whether the directionality or magnitude of their AMCEs from forced-choice responses generalize to realistic contexts when abstention is possible.

Replication and extension of forced-choice conjoint

We explore empirically how forced-choice outcomes could produce different estimates of the AMCEs relative to a choice set allowing for abstention by replicating two published conjoint experiments. The first study examines how the complexity of the information environment affects the impact of scandals on vote choice (Funck and McCabe, 2022, henceforth “F-M”). The second study, described in the Supplemental Materials, explores how the topic relevance and source partisanship of news stories impact individuals’ media consumption (Mummolo, 2016).

In their study, F-M ask American respondents recruited through Lucid to view pairs of hypothetical candidates running for U.S. Congress. Partisanship is fixed at the task-level to mirror a general election, with one candidate randomly presented as a Democrat and the other as a Republican. For each task, respondents indicate the candidate for whom they would prefer to vote using a forced-choice outcome.

The two experimental attributes of interest for F-M’s primary “Information Hypothesis” are (1) whether candidates are accused of improper behavior and (2) the amount of information provided about the candidates. First, F-M randomly assign one of six levels of “Recent news” for each candidate; three levels are neutral or positive (“No recent news,” “Recently honored for public service,” or “Recently celebrated wedding anniversary”), while the others implicate the candidate in a scandal (“Recently accused of sexual harassment,” “Recently accused of cheating on spouse,” or “Recently accused of leaking confidential information”).

Second, the number of attributes provided for both candidates is randomly manipulated at the task-level. In the “Low Information” condition, respondents receive only the candidates’ party affiliation and recent news. In the “Medium” and “High Information” conditions, respondents receive randomly assigned levels of three or eight additional attributes of the candidates. In line with their “Information Hypothesis,” F-M find voters are less likely to select candidates associated with scandalous allegations, but the magnitude of this penalty shrinks as the information environment’s complexity increases.

We replicate F-M’s experiment using 2254 respondents from Amazon’s Mechanical Turk (MTurk). We match F-M’s protocol with a few modifications. First, half of our respondents were assigned to a forced-choice arm. These participants were required to indicate which candidate they preferred in each task. The other half of our respondents were placed in a abstention option arm where they could indicate their preferred candidate or abstain from making a selection. Second, after completing their six tasks, respondents were presented again with the first profile pair they evaluated but were provided the outcome measure from the opposite experimental arm (e.g., forced-choice arm respondents were given the option to abstain).⁶

Following F-M, we estimate our AMCEs using ordinary least squares regression with standard errors clustered by respondent. Our outcome is a binary indicator for whether a respondent selected an available profile, coded as “1”, and all other profiles in the task are coded as “0.” If a respondent abstained, all profiles in the task are coded as “0.” Importantly, respondents abstain often when allowed to do so. In our abstention arm, respondents selected none of the profiles in 25.8% (1764) of tasks in that arm and 31.0% (699) of respondents abstained at least once.⁷ This descriptive finding highlights that response options in choice-based conjoint experiments may induce meaningfully different behavior from respondents. We interact all non-baseline attribute-levels with a binary indicator for each respondent’s outcome measure arm so we can compare directly the AMCEs yielded when respondents are forced to choose a candidate versus when they are allowed to abstain. Finally, because F-M’s hypotheses implicate differences in the AMCEs recovered for attribute-levels of “Recent news” across facets of information environment complexity, we focus on those differences as opposed to the underlying AMCEs.⁸

Forced-choice design can result in different conclusions

In Figure 1, the left pane displays the difference between the AMCE for each non-baseline level of the “Recent news” attribute completed in the “Low Information” environment and the corresponding attribute-level in the “Moderate” and “High Information” environments among respondents in the forced-choice (black circles) and abstention option (grey triangles) arms. The right pane of Figure 1 presents each of the differences in AMCEs between the estimates yielded by the forced-choice and abstention arms (black squares). First, when comparing the “Low” and “Moderate Information” conditions, we see that all of the differences corresponding with the forced-choice and abstention option arms lead to substantively similar conclusions. In other words, we do not replicate F-M’s initial results; we find no difference in how much respondents prefer any “Recent News” attribute-level between the two lowest information categories.

Figure 1.

Effect of scandalous news and information complexity on vote choice. Notes: The left pane of this figure presents the differences in the AMCE for each non-baseline “Recent News” attribute-level for conjoint tasks situated in a “Low Information” conjoint task relative to when the same attribute-level is situated in “Moderate Information” or “High Information.” Black circles and grey triangles represent the differences for each attribute-level among respondents in the forced-choice and abstention arms, respectively. The right pane of the figure displays for each attribute-level in each information environment the difference in the differences in AMCEs obtained in the forced-choice and abstention arms. Lines in both panes represent Bonferroni-adjusted 95% confidence intervals (α = 0.05/20 = 0.0025) for each attribute-level among respondents in the forced-choice arm (Liu and Shiraito, 2023). See the Supplemental Materials for full model summary.

Some distinctions emerge when comparing the “Low” and “High Information” conditions in each of the two arms. For two of the three negative events (“Cheating on spouse” and “Sexual harassment”), our forced-choice arm replicates F-M’s finding—respondents penalize candidates less for transgressions in the “High Information” condition than in the “Low Information” condition (i.e., respondents are more likely to vote for corrupt candidates in the “High” compared to the “Low Information” condition)—but these differences do not manifest in our abstention option arm. However, we cannot reject the null hypothesis that the estimates associated with the forced-choice and abstention arms are not different from each other. The right pane of Figure 1 shows that the Bonferroni-corrected 95% confidence intervals for the differences in the differences of the AMCEs across the two arms all include zero.⁹

Similarly, while the two positive news stories (“Honored for public service” and “Celebrated wedding anniversary”) are not of theoretical relevance for F-M’s “Information Hypothesis,” the differences in AMCEs associated with these attribute-levels between the “Low” and “High Information” conditions are not statistically distinguishable in the forced-choice arm, but are distinguishable in the abstention arm (though, again, the differences in these estimates across arms are not themselves distinguishable).

In the Supplemental Materials, we consider one potential explanation for why the differences in AMCEs for negative “Recent News” attribute-levels across information conditions are smaller (albeit not statistically distinguishable) in our abstention arm relative to the forced-choice arm: respondents who are forced to choose are reluctant to select for scandalized candidates in low information environments but are more willing to do so when additional information can obscure or compensate for scandals. Yet, when respondents can abstain and prioritize candidates’ reputations, they decline to make a selection when presented with scandalized candidates irrespective of other information. In the Supplemental Materials, we probe the profile- and task-level characteristics associated with abstention and find that respondents are more likely to abstain when both candidates or the candidate who shares their party affiliation are implicated in scandals. Further, we find that as the information environment becomes more complex, abstention becomes less common only if the respondent’s copartisan is scandalized; otherwise, the probability of abstention increases as the information environment becomes more complex.

Conclusion

We have shown theoretically and empirically that forcing participants to reveal a preference when they would otherwise prefer to abstain may lead to AMCEs of different magnitudes or signs than would be recovered if an abstention option were offered. In some contexts, such as when political elites must hire civil servants (Liu, 2019) or select which lawmakers to lobby (Miller, 2022), abstention is unlikely and undesirable because a choice must be made as a key function of the respondents’ occupations. Additionally, when researchers are chiefly interested in measuring respondents’ underlying preferences as opposed to how they express those preferences in decision-making contexts and they have reason to believe respondents hold preferences on their concept of interest, using rating outcomes or omitting abstention options may be appropriate (e.g., Ganter, 2023). However, we advise that researchers interested in modeling respondents’ expressed preferences in decision-making contexts provide an abstention option if the data generating process naturally includes one. For example, some studies have included an abstention option to model the selection over candidates in elections (Agerberg, 2020; Hanretty et al., 2020) and locations to migrate (Ghosn et al., 2021). Including an abstention option is particularly important in light of the non-negligible abstention rates observed among the respondents in our abstention arms—across both replications, respondents abstained in approximately one-fourth of all tasks and roughly one-third of respondents abstained at least once.

If researchers wish to use forced-choice designs, we encourage them to be explicit about the contexts to which their findings might apply. The AMCEs recovered with a forced-choice outcome reflect the revealed preferences of respondents when a choice is required. Alternatively, if researchers are concerned with modeling the entire decision-making process, it may be better to modify the research design to make use of an appropriate estimating procedure that accounts for task attributes that impact respondents’ decision to select as well as preferences if a selection is made.

Supplemental Material

Supplemental Material - Preferential abstention in conjoint experiments

Supplemental Material for Preferential abstention in conjoint experiments by David R. Miller and Jeffrey Ziegler in Research & Politics

Footnotes

Acknowledgments

We very much appreciate the willingness of Amy Funck, Katherine McCabe, and Jonathan Mummolo to publicly share their data to make our replications possible. We thank Connor Hamby and Minh Vu Duc for valuable research assistance. We are grateful to the editorial staff of Research and Politics, our anonymous reviewers, Gustavo Diaz, Jonathan Homola, Yusaku Horiuchi, Shiyao Liu, Erin Rossiter, Yuki Shiraito, Anton Strezhnev, Michelle Torres, and Teppei Yamamoto for careful feedback, as well as the participants of the 2022 International Conference on Method Triangulation, 2021 LAPolMeth, and 2022 PolMeth Europe for constructive discussions.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The replication studies were supported in part by Vanderbilt University’s Center for the Study of Democratic Institutions. Research assistance was funded in part by the TRiSS Academic Research Fellowship.

ORCID iDs

David R. Miller

Jeffrey Ziegler

Supplemental Material

Supplemental material for this article is available online.

The replication files are available at: .

Notes

References

Abramson

Koçak

Magazinnik

(2022) What do we learn about voter preferences from conjoint experiments. American Journal of Political Science 66(4): 1008–1020.

Agerberg

(2020) The lesser evil? Corruption voting and the importance of clean alternatives. Comparative Political Studies 53(2): 253–287.

Bansak

Hainmueller

Hopkins

, et al. (2021) Conjoint survey experiments. Advances in Experimental Political Science. Cambridge, UK: Cambridge University Press.

Berinsky

(2008) Survey non-response. In: Donsbach

Traugott

(eds). The Sage Handbook of Public Opinion Research. London: Sage, 309–321.

Blais

(2006) What affects voter turnout? Annual Review of Political Science 9: 111–125.

Carlson

(2015) Ethnic voting and accountability in Africa: a choice experiment in Uganda. World Politics 67(2): 353–385.

Chilton

Milner

Tingley

(2020) Reciprocity and public opposition to foreign direct investment. British Journal of Political Science 50(1): 129–153.

Egami

Hartman

(2022) Elements of external validity: framework, design, and analysis. American Political Science Review 117: 1070–1088.

Elkjær

Wlezien

(2024) Estimating public opinion from surveys: the impact of including a “don’t know” response option in policy preference questions. Political Science Research and Methods 1–17.

10.

Funck

McCabe

(2022) Partisanship, information, and the conditional effects of scandal on voting decisions. Political Behavior 44: 1389–1409.

11.

Ganter

(2023) Identification of preferences in forced-choice conjoint experiments: reassessing the quantity of interest. Political Analysis 31(1): 98–112.

12.

Ghosn

Chu

Simon

, et al. (2021) The journey home: violence, anchoring, and refugee decisions to return. American Political Science Review 115(3): 982–998.

13.

Hainmueller

Hopkins

Yamamoto

(2014) Causal inference in conjoint analysis: understanding multidimensional choices via stated preference experiments. Political Analysis 22(1): 1–30.

14.

Hanretty

Lauderdale

Vivyan

(2020) A choice-based measure of issue importance in the electorate. American Journal of Political Science 64(3): 519–535.

15.

Kim

(2021) Temporal selective exposure: how partisans choose when to follow Politics. Political Behavior 43(4): 1663–1683.

16.

Krosnick

Holbrook

Berent

, et al. (2002) The impact of ‘No opinion’ response options on data quality: non-attitude reduction or an invitation to satisfice? Public Opinion Quarterly 66(3): 371–403.

17.

Leeper

Hobolt

Tilley

(2020) Measuring subgroup preferences in conjoint experiments. Political Analysis 28(2): 207–221.

18.

Liu

(2019) The logic of authoritarian political selection: evidence from a conjoint experiment in China. Political Science Research and Methods 7(4): 853–870.

19.

Liu

Shiraito

(2023) Multiple hypothesis testing in conjoint analysis. Political Analysis 31(3): 380–395.

20.

Miller

(2022) On whose door to knock? Organized interests’ strategic pursuit of access to members of congress. Legislative Studies Quarterly 47(1): 157–192.

21.

Mukerjee

Yang

(2020) Choosing to avoid? A conjoint experimental study to understand selective exposure and avoidance on social media. Political Communication 38(3): 222–240.

22.

Mummolo

(2016) News from the other side: how topic relevance limits the prevalence of partisan selective exposure. The Journal of Politics 78(3): 763–773.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.45 MB