Sage Journals: Discover world-class research

Abstract

Research shows that Americans have a generally poor understanding of welfare programs. Providing information about such programs has the potential to shape public preferences, but we argue that such effects may differ based on the content of the information and its correspondence with existing ideological beliefs. Using original survey experiments embedded in the Cooperative Congressional Election Study and through Amazon’s Mechanical Turk, we analyze how the relationship of ideology with welfare programs varies in response to different types of negative information about the program, and different descriptions of policy design. We find that information about inadequate benefits has a larger negative impact on welfare support for liberals than for conservatives but that both liberals and conservatives may be equally concerned about fraud and inefficiency. Other information about policy design has the expected conditional effect: state (as opposed to federal) funding and short time limits for benefits are more appealing to conservatives than liberals.

Keywords

public opinion policy information the submerged state traditional welfare programs

It is well-established that few Americans have a detailed understanding of public policies. Scholars have particularly noted citizens’ poor understanding of the American welfare state (Kuklinski & Quirk, 2000; Soss & Schram, 2007), including “hidden” or “submerged” welfare programs that provide aid in a complex or indirect manner (Howard, 1997; Mettler, 2011). In the absence of detailed information about such programs, many people form judgments about them based on partisan cues (Cohen, 2003; Mettler et al., 2023) and symbolic considerations, such as the perceived “deservingness” of those who benefit (Gilens, 1999; Mårtensson et al., 2023; Petersen, 2012; Schneider & Ingram, 1993; Soss et al., 2011) or assumptions about their race and ethnicity (Garand et al., 2017; Gilens, 1999; Lieberman, 2011). This naturally leads to the question of whether providing Americans with factual information about these programs can shift their beliefs (Callaghan & Jacobs, 2017; Guardino & Mettler, 2020; Kuklinski & Quirk, 2000). We argue that while information can shift beliefs about social welfare policy, the effect of information depends on the individual’s underlying beliefs. The same set of facts is likely to hold a different meaning for a liberal egalitarian than it would for a conservative individualist, for example.

In this study, we present the results of paired survey experiments from a high-quality nationally representative survey, as well as a conjoint experiment using a convenience sample, testing the effect of two types of negative information about social welfare programs: information about fraud and abuse (a particular concern of conservatives) and information about the inadequacy of the program to lift people out of poverty (which should be especially disconcerting to liberals). On balance, the results of these experiments support the expectation that liberals are more responsive than conservatives to information about the inadequacy of social welfare programs, but not that conservatives are more responsive to information about fraud and inefficiency. Indeed, liberals appear to be at least as concerned as conservatives about the latter issues. Conservatives did behave as expected in response to information about program design, however – describing a program as being funded solely by state government or having a short time limit for the receipt of benefits boosted conservatives’ support more than that of liberals.

Our findings have important implications for the research on public opinion toward welfare programs. First, previous studies in this field often neglect the effects of vulnerabilities of welfare programs on citizens’ welfare attitudes, despite the considerable problems of the American welfare state (Andrade et al., 2019; Headworth, 2021; Kim & Maroulis, 2018). In this study, we conceptualize these problems as falling into two categories: fraud and inefficiency on the one hand, and inadequacy on the other. Our experimental results demonstrate that information about these problems can indeed move public opinion, but in different ways for people with different sets of beliefs. These findings also buttress the argument that vulnerabilities of welfare programs, such as welfare fraud and abuse, are instrumental in undermining political support for key social welfare programs (Kohler-Hausmann, 2015). Our study also contributes to the growing literature on the effects of policy design (e.g., policy delivery mechanisms and work requirements) on public attitudes toward welfare policies (Ashok & Huber, 2020; Haselswerdt, 2022). This research helps to promote more comprehensive thinking and exploration of the effects of policy problems and design elements on the politics of the welfare state.

Public Opinion and Social Welfare: The Roles of Ideology and Information

In our theoretical framework, political ideology plays a fundamental role in the formation of initial opinions when citizens consider a welfare program. In other words, when citizens learn of the function and basic goal of a welfare policy, ideology helps them make their first judgment on the policy. This is supported by extensive research finding that ideologies or values are central to public opinion toward such programs (Blekesaune & Quadagno, 2003; Feldman & Steenbergen, 2001). In general, the political values or ideologies relevant to social welfare can be divided into liberalism and conservatism. Liberals or those with “left-wing” ideology hold egalitarian and humanitarian values and usually support welfare programs, while conservatives or those with “right-wing” ideology hold individualistic values and tend to oppose welfare policies (Andreß & Heien, 2001; Blekesaune & Quadagno, 2003; Feldman & Steenbergen, 2001; Feldman & Zaller, 1992; Gilens, 1999).

While ideology shapes initial judgments, providing additional information about policies may also affect people’s opinions toward these policies (Campbell, 2011), since most citizens lack a detailed understanding of how social welfare policies work. Soss and Schram (2007, p. 123) find that public perceptions of welfare have only the “loosest relation” to the programs’ actual features, and many citizens do not have “a firm cognitive grip on the material nature of welfare programs.” Public understanding is thought to be even weaker with respect to the “hidden welfare state” (Howard, 1997) or “submerged state” (Mettler, 2011), which includes policies like tax expenditures that provide government aid indirectly. The subtlety and complexity of these policies obscure their operational logics and social effects, making them especially difficult for general audiences to grasp (Guardino & Mettler, 2020).

This lack of understanding suggests that providing citizens with relevant policy information can help them form judgments about social welfare policies. New information can supplement their general political knowledge, facilitate issue comprehension, shape cognition, and influence opinion (Barabas & Jerit, 2009; Feldman et al., 2015; Jerit et al., 2006; Price & Zaller, 1993). Previous research shows that specific information about racial and ethnic identities of welfare recipients (Callaghan & Olson, 2017; Peffley et al., 1997) and policy design elements, such as policy delivery mechanisms (Ashok & Huber, 2020; Ellis & Faricy, 2020; Faricy & Ellis, 2014; Haselswerdt & Bartels, 2015), work requirements (Haselswerdt, 2022), inclusiveness or universalism (Jordan, 2013), and distributional outcomes or benefits (Cook et al., 2010; Faricy & Ellis, 2014; Guardino & Mettler, 2020) can affect citizens’ attitudes toward welfare policies. Other research shows that unfavorable or negative information has a greater impact on citizens’ impressions and political perception than does favorable or positive information, across a wide variety of situations (Ronis & Lipinski, 1985; Soroka, 2006), suggesting that information on a policy’s vulnerabilities is likely to have a bigger impact than information on its strengths.

In light of this, it is surprising that existing work on social welfare attitudes has given relatively little attention to policy flaws, especially since the American welfare state suffers from considerable problems. Briefly, these problems can be roughly classified into two categories: inefficiency and fraud on the one hand, and inadequacy on the other. According to the White House Office of Management and Budget (OMB¹), in 2021, improper welfare payments, including welfare fraud and abuse, were estimated to be 15.2% (more than $160 billion) of all federal welfare payments. This rate varies widely for specific programs, from 9.3% for Temporary Assistance for Needy Families (TANF) in 2008 to 15.6% for Medicaid and 31.6% for the Earned Income Tax Credit (EITC) in 2022 (Headworth, 2021; Kim & Maroulis, 2018; Office of Management and Budget, 2023). Notably, the latter estimate does not subtract overpayments recovered by the Internal Revenue Service (IRS), a recent and controversial measurement decision by OMB (Greenstein et al., 2019). Prior to this change, improper payment estimates for EITC hovered around 25%, still considerably higher than traditional social welfare programs like TANF.

In addition, due to stingy benefits, administrative burdens, social stigma, low benefit take-up, and perverse incentives to keep earnings and savings low (income and asset limits), key welfare programs like TANF have not been effective in helping impoverished families and reducing income inequality (including the racial wealth gap) in recent years (Andrade et al., 2019; Hamad et al., 2022; Herd & Moynihan, 2018; Michelmore & Lopoo, 2021; Wagmiller et al., 2020). Indeed, the very stringency and strictness that contributes to TANF’s lower-than-average improper payment rate limits the reach of the policy (e.g., Wang, 2021). Notably, the EITC performs better on some of these dimensions: the administrative burden and stigma of the program (operated through the tax code rather than a social welfare bureaucracy) are relatively low, and a gradual benefit phase-out mitigates disincentives to increase earnings. As a result, the credit succeeds in lifting millions of low-income families out of poverty (Marr et al., 2015). On the other hand, the EITC does not help poor families with no earned income, and waiting to receive benefits until tax time, particularly since a 2015 policy change slowed the processing of EITC claims, negatively impacts beneficiaries’ well-being (Kondratjeva et al., 2022).

Taken together, the problems of fraud and inadequacy result in a welfare state that distributes resources both too generously and not generously enough. Information about these issues has the potential to affect the way Americans think about these programs.

Crucially, however, we should not expect the effect of any piece of policy information to be the same for everyone. A person’s ideology shapes not only their initial judgments of welfare programs but their receptivity to additional information (Zaller, 1992). For example, information about the inadequacy of a program for helping people in need is more likely to depress support among liberals than conservatives, since it speaks more directly to the policy concerns of the former group. Conversely, information about fraud and abuse in a social welfare program should have the largest negative impact for conservatives, while liberals are more likely to dismiss such concerns as conservative talking points.

Political ideology may also affect the formation of divergent public opinion toward welfare programs when citizens receive information about the design of welfare programs. Provisions like time limits or work requirements likely appeal to conservatives, who usually prefer welfare programs intended to get people working again and oppose open-ended benefits. Conservatives are also wary of the federal government, and thus likely to be more favorable toward state-funded social welfare programs, whereas liberals are more comfortable with federal programs (Feldman & Zaller, 1992; Grafton & Permaloff, 2005; Weaver et al., 1995).

Research Design and Data

Our study uses two survey experiments² that test the effects of negative information or criticisms of social welfare programs on respondent support. Both studies randomize the presentation of information about flaws or vulnerabilities in social welfare policies and gauge the impact of this information on policy support among ideological subgroups. Since our focus in the present study is on negative information per se, these experiments do not test the impact of information about positive aspects of policy, or provide context that might mitigate negativity. They resemble fact-based political attacks on these programs from opposing political actors rather than balanced and complete assessments of the programs. As such, these experiments can be viewed as “most likely cases” for the impact of negative information on welfare attitudes.

The first experiment was embedded in a module on the 2020 Cooperative Congressional Election Study (CCES) (conducted September 29 – November 2, 2020), a nationally representative survey run by the survey firm YouGov. The full CCES includes 61,000 respondents, all of whom answer a “Common Content” battery of demographic and political questions that make up half of the survey. The other half is made up of questions written by university-based teams that are presented to randomly selected modules of 1000 respondents each. Both the Common Content and module questions are distributed between pre-election and post-election surveys fielded to the same respondents. The questions for the present study were fielded in the pre-election survey.

We randomly divided the sample of 1000 respondents into two conditions. All respondents were presented with descriptions of both the EITC and the TANF program in random order. In addition to a basic description of each policy, each treatment also included information about weaknesses or problems with the program. This negative information falls into two categories: administrative inefficiencies and fraud, which should be of greater concern to conservatives, and the inadequacy of the policies for solving issues of economic inequality, which should be of greater concern to liberals. Respondents who were presented with information on fraud-related problems with the EITC were also presented with information about inadequacy problems for TANF, and vice versa. Notably, there is no true “control” treatment in either experiment in which respondents are asked about programs in the absence of information about policy vulnerabilities, a choice that maximizes the number of respondents per group but does sacrifice some inferential leverage. We return to this issue in the conjoint experiment discussed below. Table 1

Table 1.

Experimental Design: Two Treatment Groups with Different Descriptions of Each Type of Problems of the Two Policies.

Treatments
	Condition 1	Condition 2
Description & question 1	Description of EITC + fraud	Description of EITC + inadequacy
Description & question 2	Description of TANF + inadequacy	Description of TANF + fraud

Respondents are randomly assigned to one of two treatment arms and see one problem type for each policy. Within each arm, the two items are presented in random order.

summarizes the treatment groups, and the full wording of each experimental treatment can be found in Appendix C.

After each description, subjects were asked to give their opinions of the policy using a five-point scale ranging from “oppose strongly” to “support strongly,” with “neither support nor oppose” as the midpoint. Responses to these questions form the dependent variables for our analyses, most of which use ordered logit models.

Other variables for this study are drawn from the Common Content questionnaire. These include ideological self-placement (five-point scale), political knowledge (an index constructed from eight factual knowledge questions about the partisanship of elected officials and federal and state government institutions), party identification (seven-point scale), age, sex, education, race, and annual household income. All of these covariates were measured prior to the experiment on the pre-election survey.

While the CCES provides a high-quality nationally representative sample, the limited space and sample size limit the leverage we have to test our hypotheses. To explore the effects of more types and combinations of information, we also conducted a conjoint survey experiment on Mturk with 2177 Amazon Mechanical Turk workers from September 6 to October 12, 2021. The conjoint experiment is a survey-experimental technique that allows researchers to obtain reliable measures of multidimensional preferences and estimate the causal effects of multiple attributes on hypothetical choices or evaluations. When making choices among multiple options, citizens normally will not only rely on their preferences on one or two dimensions, but also make trade-offs across dimensions (Bansak et al., 2021). This method allows us to analyze how attitudes toward welfare policies vary based on multiple dimensions such as policy problems, program cost, and program participant population, and if so, which dimensions are substantively and statistically significant.

We present respondents with several tables that show profiles of two semi-hypothetical welfare programs. These programs are based on the real-life EITC and TANF, but some attributes and randomized combinations of attributes depart from the characteristics of the real programs in important ways. The resulting program profiles are fictional but realistic, sharing features of these and other real social welfare programs. The prompt reads: “We are interested in what Americans think about different types of social welfare programs…. We will ask a few questions about each pair of programs.” We then show respondents a table that contained information about two welfare programs side by side, described as “Welfare Program A” and “Welfare Program B.” The tables contain two “policy problem” attributes, one for fraud problems and one for inadequacy problems. Each policy has two fraud problems and two inadequacy problems that could be displayed, along with a blank space which allows effects to be assessed relative to a baseline of no information – a true control condition that is not present in the CCES experiments. Other key attributes include the funding source for the program (state, federal, or both) and the time limit for receiving benefits, if any. These attributes allow us to test for heterogeneous effects of these design elements by respondent ideology. Table 2 shows an example table from our experiment.³

Table 2.

An Example Conjoint Table.

	Welfare Program A	Welfare Program B
Policy description	This policy is a benefit for low-income working people. Under this policy, many people don’t pay taxes on their income, and some can receive a payment from the government if their income is low enough. (EITC description 1)	This policy is designed to help poor families with children. Under this policy, people receive cash assistance from the government so that their children can be taken care of at home. (TANF description 2)
Policy problem 1	This policy is not enough to solve inequality in society. Since benefits are only available at tax time, the credit doesn’t necessarily help poor people when they need it most. (EITC inadequacy problem 1)	This policy is not enough to solve inequality in society because families in poverty who qualify for the program usually receive too small of a benefit to lift them out of poverty. (TANF inadequacy problem 2)
Policy problem 2	This policy has some administrative problems. For example, half of tax returns completed by paid preparers overclaim the credit. (EITC fraud problem 2)	- (No fraud problem displayed for this profile)
Funding provided by	Federal government	Federal and state government
Total cost per year	$70 billion	$27 billion
Time limit for receiving benefit	8 years	4 years
Number of people on program	25 million	5 million

Text in parentheses is not displayed to respondents.

The levels of each attribute are randomly varied, with randomization occurring independently across all respondents, tables, and attributes, aside from a constraint that the EITC policy problems are not displayed for TANF and vice versa. Each subject was presented with eight randomly generated comparison tables on different screens, for a total of sixteen hypothetical programs. We use clustered standard errors at the participant level to correct for within-respondent correlation across observations. To provide respondents with a smooth experience, we fix the order in which attributes are presented across all tables for each respondent, although we randomize the order across respondents.

After showing each comparison table, we ask participants two questions to measure their attitudes about the programs presented: a seven-point Likert scale of program support and a forced choice between the two profiles. We randomize the order of these two questions at the respondent level.

Our conjoint design has several advantages. First, unlike the CCES experiment, it includes a true control condition with no policy vulnerabilities. Second, we are able to test multiple policy problem treatments in each category for each policy type, reducing the chances that observed effects are idiosyncratic to the wording of a particular treatment. We are also able to incorporate tests of our hypotheses on funding source and benefit time limits. Lastly, randomly varying other program characteristics (like program cost and the size of the beneficiary population) improves the generalizability of these results to welfare programs beyond TANF and EITC.

The conjoint experiment seeks to determine, first, what types of attributes increase or decrease a welfare program’s support on average when the other welfare program attributes included in the research design vary independently, and second, whether the effects of these attributes depend on respondents’ political ideologies. These questions can be answered by estimating conditional average marginal component effects (AMCEs), which are the average effects of the attributes conditional on survey participants’ measured characteristics (political ideology in this case) (Bansak et al., 2022).

Since attentiveness is an issue with MTurk respondents, we include the battery of screener questions recommended by Berinsky et al. (2021) as pre-treatment attention checks to identify inattentive respondents. Per our pre-analysis plan, the results we present and discuss in the main text include all respondents, though we present results that exclude the respondents that failed to answer at least two attention check questions correctly in Tables G.2 and G.4 in the appendix. The findings are generally similar, though as expected the experimental effects are somewhat stronger for the more attentive subset of respondents.

Hypotheses

From the above theoretical arguments and framework, we put forward a series of testable hypotheses on the effects of information, ideology, and policy design on individuals’ attitudes towards social welfare programs. These hypotheses were preregistered in the Open Science Framework Registry prior to the receipt of the data.⁴

Hypothesis 1

Information about inadequate benefits will have a larger negative impact on liberals’ support than on conservatives’ support.

Hypothesis 2

Information about inefficiencies and fraud will have a larger negative impact on conservatives’ support than on liberals’ support.

These hypotheses are consistent with our argument that political ideologies matter not just to initial judgments about policy but to receptivity to new policy information. As the deep foundation of public opinion formation, ideology serves as a lens through which individuals perceive information about policy. For the CCES study, which lacks a control condition of no information, we cannot test H1 and H2 in isolation, so we test a modified version of both:

Combined Hypothesis 1&2

Information about inadequate benefits will have a larger negative impact on liberals’ support than conservatives’ support relative to information about inefficiencies and fraud.⁵

The “ideological lens” pattern should also apply to other policy characteristics, such as their funding sources and limitations on benefits. Since conservatives are usually uncomfortable with a large federal government and with open-ended public assistance, whereas liberals are more comfortable with both, we expect that:

Hypothesis 3

Describing a program as being funded by state (as opposed to federal) government will have a larger positive impact on conservatives’ support than on liberals’ support.

Hypothesis 4

Describing a program as having a time limit for receiving benefits (especially a shorter time limit) will have a larger positive impact on conservatives’ support than on liberals’ support.

The CCES experiment provides tests of Hypotheses 1 and 2, while the conjoint experiment tests all four hypotheses.

Results

CCES Experiment

Even though all respondents in this experiment were presented with negative information about both policies, their evaluations were slightly more supportive than neutral, on average, with an overall mean of 3.34 (standard deviation 1.14) out of 5 for the EITC and 3.15 (standard deviation 1.18) out of 5 for TANF. Table 3 presents differences across treatment groups for both experiments. We had no hypotheses about unconditional treatment group differences, and we find no evidence of such differences in either experiment. When we consider respondents as one group, it does not appear that information about the inadequacy of social policies for addressing inequality is more or less compelling than information about fraud and administrative problems.

Table 3.

Mean Support for Social Welfare Policies Across Treatment Conditions.

Policy	Inadequacy Treatment	Fraud Treatment	$p$
EITC	3.34 (.05)	3.34 (.05)	.97
TANF	3.12 (.05)	3.18 (.05)	.43

Standard errors in parentheses. $p$ -values are two-tailed.

Our expectation is that this surface similarity across the treatment groups masks ideological differences in response to different types of information. We explore this possibility with ordered logit models of policy support, with treatment indicators and their interactions with respondent ideology as the main explanatory variables. Figure 1 presents the results of these analyses graphically, per the advice of Brambor et al. (2006).⁶ For the EITC experiment, we see the pattern predicted by the combined version of H1 and H2: the fraud treatment had a larger negative impact among conservatives relative to the inadequacy treatment than among liberals. Conversely, the inadequacy treatment had a considerable dampening effect on liberals’ support relative to the fraud treatment. The interaction term is statistically significant both with and without control variables ( $p < . 05$ ). For the TANF experiment, however, we observe no interaction effect to speak of – while the interaction term is signed as expected, it is statistically insignificant ( $p = . 52$ without controls and $. 88$ with controls), and the graphical presentation shows only a slight narrowing across the range of the ordinal ideology variable.⁷

Figure 1.

Predicted probabilities of strong policy support across treatment groups and respondent ideology. Predictions and 95% confidence intervals generated from ordered logit models with control variables.

MTurk Conjoint Experiment

We now turn to our conjoint experiment, which allows us to test a greater variety of informational treatments, and to test our hypotheses about policy design. The results are summarized in Figure 2, which displays the AMCEs with 90% confidence intervals for all levels and attributes for liberal and conservative respondents,⁸ using forced program choice as the dependent variable. The results support Hypothesis 1: information about the inadequacy of both types of social welfare policies had a more pronounced negative effect on liberals’ support than that of conservatives, though it is notable that these treatments also negatively affected conservatives’ support. To formally test this hypothesis, we follow the procedure recommended by Leeper et al. (2020), running regressions of forced program choice and program support on the four inadequacy problem treatment indicators, a dummy variable identifying conservatives, and interactions of this dummy with each of the treatments (or attribute levels), with ideologically moderate respondents excluded. We then conducted omnibus F-tests on the interaction terms. The interaction terms were jointly significant in each regression ( $p < . 01$ for both), confirming that the effects of the inadequacy treatments were significantly greater for liberals.

Figure 2.

Conjoint experiment forced choice results by ideological group (with all respondents and 90% confidence intervals).

For Hypothesis 2, though, the findings suggest that the fraud treatments had pronounced negative effects on policy support for liberals as well as conservatives, with little evidence of differences in effects between the two groups. Recall that while the CCES experiment included no true control group, meaning that we could only compare support in the fraud condition to support in the inadequacy condition, here the baseline of “none” for each policy problem attribute allows for a non-zero-sum comparison. In this design, liberals show a surprising level of concern about fraud and abuse in both types of welfare programs. In fact, while the EITC “larger credit” treatment had no effect for conservatives, it had a significant negative effect for liberals. In the omnibus F-tests for this hypothesis, the interaction terms are jointly statistically significant in the wrong direction for both dependent variables ( $p < . 10$ for both). Thus, Hypothesis 2 is clearly not supported in this study.

The results support Hypotheses 3 and 4, however: both state funding and time limits (especially a short time limit of two years) are more appealing to conservatives than liberals. To test H3, we ran regressions of the dependent variables on an indicator of the state funding level (setting “federal” and “federal and state” as the excluded category), an indicator for conservative respondents (dropping moderates so liberals are the excluded category), and an interaction of the two. The interaction term is statistically significant in both regressions ( $p = . 047$ for program choice and $p = . 023$ for program support). To test H4, we repeat the omnibus interaction F-test procedure with all the levels of the time limit variable and the liberal/conservative indicator. The interactions are highly significant for program choice ( $p = . 0001$ ), though not for program support ( $p = . 16$ ). Despite this ambiguous result for H4, these findings are consistent with the divergent ideas those on the left and right hold about the proper way to design and fund social programs.

Adjustments for Multiple Hypothesis Tests

Since each study tests multiple hypotheses across two dependent variables, there is the potential that some statistically significant findings may emerge purely by chance. To account for this, we use the Benjamini-Hochberg procedure (Benjamini & Hochberg, 1995) for adjusting for multiple hypothesis tests with a false discovery rate of 10% for each study, as specified in our preregistration.⁹ Details on the hypothesis tests and adjustments can be found in Appendix H. Adjustments do not change our conclusions about any of the hypothesis tests for the CCES study (the combined version of H1 and H2 is supported in the EITC experiment, while other results remain insignificant, see Table H.3). For the MTurk study (Table H.4), the significant findings for H1 (on inadequacy) and H3 (on state funding) are robust to adjustment for both dependent variables, as is the result for H4 (on time limits) with the forced choice dependent variable. The initially significant $p$ -values suggesting that liberals are more concerned about fraud than conservatives (counter to H2) are considered statistically insignificant after adjustment.

Conclusion

This study provides experimental evidence that citizens’ ideologies shape their response to information about social welfare policies. Specifically, we found that information about the inadequacy of social welfare programs to solve problems of poverty and inequality was more likely to depress support for the program among liberals than conservatives. This hypothesis received partial support in the traditional CCES experiment and strong support in the MTurk conjoint experiment. We also expected to find that information about fraud and abuse would be more salient to conservatives than liberals, but found no support for this hypothesis. In fact, the findings of the conjoint study suggest that liberals are surprisingly similar to conservatives in their concern about fraud and abuse in social welfare programs, a pattern that scholars of social welfare should explore in future studies. We also found that conservatives are less comfortable with federally funded programs and with open-ended public assistance than liberals, as we expected, though support for the latter is mixed across the two dependent variables.

Our findings of heterogeneous ideological effects help to reconcile two findings from the literature that seem to be in tension: Americans know little about social welfare programs (Mettler, 2011; Soss & Schram, 2007), yet their beliefs about them are nonetheless difficult to change (Kuklinski & Quirk, 2000). Our results suggest that these beliefs may be malleable in response to new information, but in a limited sense: the new information must speak to concerns relevant to a person’s existing beliefs. Still, our many null or unexpected findings (including for additional hypotheses discussed only in Appendix H) indicate that these patterns are not necessarily predictable. Some results, particularly the aforementioned finding that fraud is a major concern for liberals, suggest that it may be a mistake to take for granted that a self-reported ideological position corresponds to a particular policy preference. This lends credence to the distinction between “operational” and “symbolic” ideology that scholars in the ideology literature have identified (Claassen et al., 2015; Ellis & Stimson, 2012; Grossmann & Hopkins, 2016, pp. 14, 19). It is also an important reminder that the relationship of ideology with specific policy opinions should be tested rather than assumed.

Our findings point to several directions for future research. In general, they suggest that scholars should incorporate the concept of policy vulnerability into their research designs, but do so in a way that takes into account how people of different ideological predispositions might view those vulnerabilities. More specifically, future work should explore the types of information or considerations that dampen support for social welfare programs among liberals, the core constituency for such programs, but one that is rarely the central focus of research on the politics of welfare. Our findings suggest that liberals are less supportive of programs that do a poor job of helping people, but also that they are surprisingly concerned about beneficiaries cheating the system, much like conservatives. These future studies should also seek to move beyond our admittedly narrow focus on negative information, allowing for the possibility that positive or contextual information may dampen or eliminate the negative effects we document here. For example, information about the administrative efficiency of a program like the EITC may serve to mitigate the negative effect of information about the program’s drawbacks. Additionally, in considering the politics of policy vulnerabilities, future work should consider the role of information sources. While we show divergent effects for different ideological subgroups when messages are presented out of context, citizens in the real world are likely to encounter such information in communications from political elites or organizations or in reporting from media outlets. In these cases, an individual’s level of trust in the source is likely to play an important role in determining if and how they incorporate the information into their beliefs. Lastly, while we explore the impacts of several different forms of information and policy design elements in the present study, there are obviously many other avenues for future research in this vein, including administrative burden (Keiser & Miller, 2020), targeting versus universalism (Gugushvili & Van Oorschot, 2020), in-kind versus cash benefits (Liscow & Pershing, 2022), and more.

Supplemental Material

Supplemental Material - Ideology, Information, and Social Welfare Preferences

Supplemental Material for Ideology, Information, and Social Welfare Preferences by Hang Qi, and Jake Haselswerdt in American Politics Research.

Footnotes

Acknowledgements

The authors would like to thank Katherine McCabe, seminar participants at the 2021 Midwest Political Science Association annual conference, the editor, and anonymous reviewers for constructive comments and advice.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Kinder Institute on Constitutional Democracy at University of Missouri, Columbia, and Harry S Truman School of Government and Public Affairs at University of Missouri, Columbia.

ORCID iDs

Hang Qi

Jake Haselswerdt

Data Availability Statement

Associated Code available here:

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Andrade

F. C. D.

Kramer

K. Z.

Greenlee

Williams

A. N.

Mendenhall

(2019). Impact of the Chicago earned income tax periodic payment intervention on food security. Preventive Medicine Reports, 16, 100993. https://doi.org/10.1016/j.pmedr.2019.100993

Andrebeta

H. J.

Heien

(2001). Four worlds of welfare state attitudes? A comparison of Germany, Norway, and the United States. European Sociological Review, 17(4), 337–356. https://doi.org/10.1093/esr/17.4.337

Ashok

V. L.

Huber

G. A.

(2020). Do means of program delivery and distributional consequences affect policy support? Experimental evidence about the sources of citizens’ policy opinions. Political Behavior, 42(4), 1097–1118. https://doi.org/10.1007/s11109-019-09534-z

Bansak

Bechtel

M. M.

Margalit

(2021). Why austerity? The mass politics of a contested policy. American Political Science Review, 115(2), 486–505. https://doi.org/10.1017/S0003055420001136

Bansak

Hainmueller

Hopkins

D. J.

Yamamoto

(2022). Using conjoint experiments to analyze election outcomes: The essential role of the average marginal component effect. Political Analysis, 31(4), 500–518. https://doi.org/10.1017/pan.2022.16

Barabas

Jerit

(2009). Estimating the causal effects of media coverage on policy-specific knowledge. American Journal of Political Science, 53(1), 73–89. https://doi.org/10.1111/j.1540-5907.2008.00358.x

Benjamini

Hochberg

(1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B, 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x

Berinsky

A. J.

Margolis

M. F.

Sances

M. W.

Warshaw

(2021). Using screeners to measure respondent attention on self-administered surveys: Which items and how many? Political Science Research and Methods, 9(2), 430–437. https://doi.org/10.1017/psrm.2019.53

Blekesaune

Quadagno

(2003). Public attitudes toward welfare state policies: A comparative analysis of 24 nations. European Sociological Review, 19(5), 415–427. https://doi.org/10.1093/esr/19.5.415

10.

Brambor

Clark

Golder

(2006). Understanding interaction models: Improving empirical analyses. Political Analysis, 14(1), 63–82. https://doi.org/10.1093/pan/mpi014

11.

Callaghan

Jacobs

L. R.

(2017). The future of health care reform: What is driving enrollment? Journal of Health Politics, Policy and Law, 42(2), 215–246. https://doi.org/10.1215/03616878-3766710

12.

Callaghan

Olson

(2017). Unearthing the hidden welfare state: Race, political attitudes, and unforeseen consequences. The Journal of Race, Ethnicity, and Politics, 2(1), 63–87. https://doi.org/10.1017/rep.2016.26

13.

Campbell

A. L.

(2011). Policy feedbacks and the impact of policy designs on public opinion. Journal of Health Politics, Policy and Law, 36(6), 961–973. https://doi.org/10.1215/03616878-1460542

14.

Claassen

Tucker

Smith

S. S.

(2015). Ideological labels in America. Political Behavior, 37(2), 253–278. https://doi.org/10.1007/s11109-014-9272-6

15.

Cohen

G. L.

(2003). Party over policy: The dominating impact of group influence on political beliefs. Journal of Personality and Social Psychology, 85(5), 808–822. https://doi.org/10.1037/0022-3514.85.5.808

16.

Cook

F. L.

Jacobs

L. R.

Kim

(2010). Trusting what you know: Information, knowledge, and confidence in social security. The Journal of Politics, 72(2), 397–412. https://doi.org/10.1017/S0022381610000034

17.

Ellis

Faricy

(2020). Race, “deservingness,” and social spending attitudes: The role of policy delivery mechanism. Political Behavior, 42(3), 819–843. https://doi.org/10.1007/s11109-018-09521-w

18.

Ellis

Stimson

J. A.

(2012). Ideology in America. Cambridge University Press.

19.

Faricy

Ellis

(2014). Public attitudes toward social spending in the United States: The differences between direct spending and tax expenditures. Political Behavior, 36(1), 53–76. https://doi.org/10.1007/s11109-013-9225-5

20.

Feldman

Huddy

Marcus

G. E.

(2015). Going to war in Iraq: When citizens and the press matter. The University of Chicago Press.

21.

Feldman

Steenbergen

M. R.

(2001). The humanitarian foundation of public support for social welfare. American Journal of Political Science, 45(3), 658–677. https://doi.org/10.2307/2669244

22.

Feldman

Zaller

(1992). The political culture of ambivalence: Ideological responses to the welfare state. American Journal of Political Science, 36(1), 268–307. https://doi.org/10.2307/2111433

23.

Garand

J. C.

Davis

B. C.

(2017). Immigration attitudes and support for the welfare state in the American mass public. American Journal of Political Science, 61(1), 146–162. https://doi.org/10.1111/ajps.12233

24.

Gilens

(1999). Why Americans hate welfare: Race, media, and the politics of antipoverty policy. University of Chicago Press.

25.

Grafton

Permaloff

(2005). Liberal and conservative dissensus in areas of domestic public policy other than business and economics. Policy Sciences, 38(1), 45–67. https://doi.org/10.1007/s11077-005-1811-x

26.

Greenstein

Wancheck

Marr

(2019). Reducing overpayments in the earned income tax credit. Center for Budget and Policy Priorities. https://www.cbpp.org/sites/default/files/atoms/files/4-30-13tax.pdf

27.

Grossmann

Hopkins

D. A.

(2016). Asymmetric politics: Ideological republicans and group interest democrats. Oxford University Press.

28.

Guardino

Mettler

(2020). Revealing the “hidden welfare state”: How policy information influences public attitudes about tax expenditures. Journal of Behavioral Public Administration, 3(1), 1–15. https://doi.org/10.30636/jbpa.31.108

29.

Gugushvili

Van Oorschot

(2020). Popular preferences for a fully means-tested welfare provision model: Social and cross-national divides in Europe. International Journal of Sociology & Social Policy, 40(11/12), 1455–1472. https://doi.org/10.1108/IJSSP-03-2020-0108

30.

Hamad

Gosliner

Brown

E. M.

Hoskote

Jackson

Esparza

E. M.

Fernald

L. C. H.

(2022). Understanding take-up of the earned income tax credit among Californians with low income. Health Affairs, 41(12), 1715–1724. https://doi.org/10.1377/hlthaff.2022.00713

31.

Haselswerdt

(2022). Who benefits? Race, immigration, and assumptions about policy. Political Behavior, 44(1), 271–318. https://doi.org/10.1007/s11109-020-09608-3

32.

Haselswerdt

Bartels

B. L.

(2015). Public opinion, policy tools, and the status quo: Evidence from a survey experiment. Political Research Quarterly, 68(3), 607–621. https://doi.org/10.1177/1065912915591217

33.

Headworth

(2021). Broke people, broken rules: Explaining welfare fraud investigators’ attributions. Punishment & Society, 23(1), 24–46. https://doi.org/10.1177/1462474520928131

34.

Herd

Moynihan

D. P.

(2018). Administrative burden: Policymaking by other means. Russell Sage Foundation.

35.

Howard

(1997). The hidden welfare state: Tax expenditures and social policy in the United States. Princeton University Press.

36.

Jerit

Barabas

Bolsen

(2006). Citizens, knowledge, and the information environment. American Journal of Political Science, 50(2), 266–282. https://doi.org/10.1111/j.1540-5907.2006.00183.x

37.

Jordan

(2013). Policy feedback and support for the welfare state. Journal of European Social Policy, 23(2), 134–148. https://doi.org/10.1177/0958928712471224

38.

Keiser

L. R.

Miller

S. M.

(2020). Does administrative burden influence public support for government programs? Evidence from a survey experiment. Public Administration Review, 80(1), 137–150. https://doi.org/10.1111/puar.13133

39.

Kim

Maroulis

(2018). Rethinking social welfare fraud from a complex adaptive systems perspective. Administration & Society, 50(1), 78–100. https://doi.org/10.1177/0095399715587520

40.

Kohler-Hausmann

(2015). Welfare crises, penal solutions, and the origins of the “welfare queen. Journal of Urban History, 41(5), 756–771. https://doi.org/10.1177/0096144215589942

41.

Kondratjeva

Roll

S. P.

Despard

Grinstein-Weiss

(2022). The impact of tax refund delays on the experience of hardship among lower-income households. Journal of Consumer Policy, 45(2), 239–280. https://doi.org/10.1007/s10603-021-09501-4

42.

Kuklinski

J. H.

Quirk

P. J.

(2000). Reconsidering the rational public: Cognition, heuristics, and mass opinion. In Lupia

McCubbins

M. D.

Popkin

S. L.

(Eds.), Elements of reason (1st ed., pp. 153–182). Cambridge University Press. https://doi.org/10.1017/CBO9780511805813.008

43.

Leeper

T. J.

Hobolt

S. B.

Tilley

(2020). Measuring subgroup preferences in conjoint experiments. Political Analysis, 28(2), 207–221. https://doi.org/10.1017/pan.2019.30

44.

Lieberman

R. C.

(2011). Shaping race policy: The United States in comparative perspective. Princeton University Press.

45.

Liscow

Pershing

(2022). Why is so much redistribution in-kind and not in cash? Evidence from a survey experiment. National Tax Journal, 75(2), 313–354. https://doi.org/10.1086/719402

46.

Marr

Huang

C.-C.

Sherman

DeBot

(2015). EITC and child tax credit promote work, reduce poverty, and support children’s development, research finds. Center on Budget and Policy Priorities. https://www.cbpp.org/sites/default/files/atoms/files/6-26-12tax.pdf

47.

Mårtensson

Österman

Palme

Ruhs

(2023). Shielding free movement? Reciprocity in welfare institutions and opposition to EU labour immigration. Journal of European Public Policy, 30(1), 41–63. https://doi.org/10.1080/13501763.2021.1981980

48.

Mettler

(2011). The submerged state: How invisible government policies undermine American democracy. University of Chicago Press.

49.

Mettler

Jacobs

L. R.

Zhu

(2023). Policy threat, partisanship, and the case of the affordable care act. American Political Science Review, 117(1), 296–310. https://doi.org/10.1017/S0003055422000612

50.

Michelmore

Lopoo

(2021). Exposure to the earned income tax credit in early childhood and family wealth. RSF: The Russell Sage Foundation Journal of the Social Sciences, 7(3), 196–215. https://doi.org/10.7758/rsf.2021.7.3.09

51.

Office of Management and Budget . (2023). Improper payments. Office of Management and Budget. Retrieved from. https://www.paymentaccuracy.gov/

52.

Peffley

Hurwitz

Sniderman

P. M.

(1997). Racial stereotypes and whites’ political views of Blacks in the context of welfare and crime. American Journal of Political Science, 41(1), 30–60. https://doi.org/10.2307/2111708

53.

Petersen

M. B.

(2012). Social welfare as small-scale help: Evolutionary psychology and the deservingness heuristic. American Journal of Political Science, 56(1), 1–16. https://doi.org/10.1111/j.1540-5907.2011.00545.x

54.

Price

Zaller

(1993). Who gets the news? Alternative measures of news reception and their implications for research. Public Opinion Quarterly, 57(2), 133–164. https://doi.org/10.1086/269363

55.

Haselswerdt

(2021a). Ideology, information, and social welfare preferences. Open Science Framework Preregistration. https://doi.org/10.17605/OSF.IO/BTQWN

56.

Haselswerdt

(2021b). Ideology, information, and social welfare preferences: A conjoint experiment. Open Science Framework Preregistration. https://doi.org/10.17605/OSF.IO/7US6M

57.

Ronis

D. L.

Lipinski

E. R.

(1985). Value and uncertainty as weighting factors in impression formation. Journal of Experimental Social Psychology, 21(1), 47–60. https://doi.org/10.1016/0022-1031(85)90005-8

58.

Schneider

Ingram

(1993). Social construction of target populations: Implications for politics and policy. American Political Science Review, 87(2), 334–347. https://doi.org/10.2307/2939044

59.

Soroka

S. N.

(2006). Good news and bad news: Asymmetric responses to economic information. The Journal of Politics, 68(2), 372–385. https://doi.org/10.1111/j.1468-2508.2006.00413.x

60.

Soss

Fording

R. C.

Schram

(Eds.), (2011). Disciplining the poor: Neoliberal paternalism and the persistent power of race. University of Chicago Press.

61.

Soss

Schram

S. F.

(2007). A public transformed? Welfare reform as policy feedback. American Political Science Review, 101(1), 111–127. https://doi.org/10.1017/S0003055407070049

62.

Wagmiller

R. L.

Lee

K. S.

J. H.

(2020). The role of welfare in family income inequality: 1968–2016. Children and Youth Services Review, 119, 105615. https://doi.org/10.1016/j.childyouth.2020.105615

63.

Wang

J. S.-H.

(2021). State TANF time limit and work sanction stringencies and long-term trajectories of welfare use, labor supply, and income. Journal of Family and Economic Issues, 42(4), 650–696. https://doi.org/10.1007/s10834-020-09714-8

64.

Weaver

R. K.

Shapiro

R. Y.

Jacobs

L. R.

(1995). Trends: Welfare. Public Opinion Quarterly, 59(4), 606–627. https://doi.org/10.1086/269496

65.

Zaller

(1992). The nature and origins of mass opinion. Cambridge University Press.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.62 MB

Ideology,Information,and Social Welfare Preferences

Abstract

Keywords

Public Opinion and Social Welfare: The Roles of Ideology and Information

Research Design and Data

Hypotheses

Results

CCES Experiment

MTurk Conjoint Experiment

Adjustments for Multiple Hypothesis Tests

Conclusion

Supplemental Material

Supplemental Material - Ideology, Information, and Social Welfare Preferences

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

ORCID iDs

Data Availability Statement

Supplemental Material

Notes

References

Supplementary Material