Beliefs and opinions about “illegal” and “undocumented” immigrants: Conceptual replication of a null result

Abstract

Activists and politicians often express a belief that the language used to describe immigrant populations, such as “illegal” vs. “undocumented,” can impact policy opinions. Existing experimental studies, however, have found relatively little support for that conjecture. In this note, we extend this line of research by exploring whether language impacts perceptions about unauthorized immigrants, such as their association with criminal behavior. Our study uses a conjoint design with experimentally manipulated instructions: respondents are randomly asked to identify profiles that belong to “illegal” or “undocumented” immigrants. We find that language used in the conjoint instructions does not impact perceptions about unauthorized population. At the same time, our observational results show that respondents’ preferences for terms “illegal” and “undocumented” are related to attitudes toward immigration. We discuss the implication of these results for the debate regarding the role of labels in political conflict over immigration.

During President Joe Biden’s 2024 State of the Union Address, the heckling of Representative Marjorie Taylor Greene prompted Biden to address the murder of 22-year-old nursing student Laken Riley in Georgia. In an off-the-cuff response, President Biden used the term “illegal” to refer to the individual suspected of the murder. The use of this term, commonly regarded as pejorative by many on the political left, swiftly attracted criticism from progressive activists and Democratic politicians.

This reaction is seemingly at odds with empirical data showing relatively little support for the claim that the language used to describe unauthorized immigrants has direct attitudinal and political consequences. ¹ Survey-experimental studies across the social sciences have repeatedly found that presenting respondents with the term “illegal” instead of the less negatively charged word “undocumented” does not have an expected effect on immigration policy preferences (Callister et al., 2022; Knoll et al., 2011; Merolla et al., 2013; Ommundsen et al., 2014). ²

However, there may be reasons to reconsider these results. First, there is evidence that the terms “illegal” and “undocumented” can impact support for specific enforcement policies, even if this effect is not uniform (Chenane et al., 2024). Second, recent developments in American politics, such as growing partisan polarization on immigration (Sides et al., 2018), may have changed these dynamics. Third, some studies have shown that linguistic labels may exert significant effects on political attitudes in other domains (Vicuña and Perez, 2023), at least within specific demographic groups (Singer and Couper, 2014). Finally, the claim that rhetorical differences may impact beliefs about population without legal status—for instance, the perceptions of unauthorized immigrants as being ethnically distinct from the Anglo majority or prone to criminal behavior—remains untested.

In this note, we extend previous research by exploring whether the use of terms like “illegal” and “undocumented” can change how Americans associate unauthorized status with attributes like ethnicity, education, and criminal behavior. We use a conjoint design, in which one half of the respondents is randomly asked to identify profiles that belong to “illegal” immigrants, whereas the other half complete the same task for “undocumented” immigrants. Our results show that the terms in question do not impact beliefs about unauthorized immigrants, at least at a short time scale. At the same time, respondents’ personal preferences for the “illegal” vs. “undocumented” language are related to attitudes toward immigration.

Research Design

1083 respondents for our study were recruited in March 2024 on CloudResearch (formerly TurkPrime). See Section A of Supplemental Material for sample demographics. Respondents started from completing a conjoint task designed to measure perceptions rather than preferences (Myers et al. 2024). In the task, participants were presented with seven pairs of immigrant profiles (14 profiles total) and asked which of the two was more likely to be in the country without authorization. See Section B of Supplemental Material for conjoint instructions. The status was described differently across the two conditions: randomly, half of the sample was asked to identify “illegal” immigrants, whereas the second half was asked about “undocumented” immigrants (see Section C of Supplemental Material for sample pairs of profiles in the two conditions). The attributes used in the descriptions were modeled from a conjoint task by Flores and Schachter (2018), but the numbers of randomized values per attribute were smaller since our primary interest was the “illegal” vs. “undocumented” comparison. See Section D of Supplemental Material for the full list of attributes and values.

After completing the conjoint task, participants were asked to express opinions on four policies related to immigration enforcement: Deferred Action for Childhood Arrivals (DACA), state laws similar to Arizona SB 1070, sanctuary initiatives, and building the border wall. See Section E of Supplemental Material for the questions and answer options. Responses to these questions were recoded so that higher values indicated opposition to lenient policies and support for restrictive ones and then combined into an additive index. At the end of the survey, respondents were debriefed about its goals and asked to express their own preference regarding the “illegal” vs. “undocumented” word choice (see Section F of Supplemental Material). ³

Results

We start from estimating average marginal component effects (AMCEs) in the conjoint task independently for the two experimental conditions with standard errors clustered on the level of respondents. ⁴ The results are presented in Figure 1. Respondents believe that immigrants who are Hispanic, have less education, speak worse English, stay in the country for shorter times, and have police records are more likely to be unauthorized. At the same time, these perceptions do not depend on whether immigrants are described as “illegal” or “undocumented.” The only difference between the two conditions shows for the welfare dependency attribute: both “illegal” and “undocumented” immigrants are associated with receiving government benefits—but the magnitude is stronger for the latter. However, a joint test that accounts for multiple comparisons does not allow rejecting the null hypothesis of similar perceptions in the two experimental conditions (F_{10, 1082} = 1.73, p = .070). We also find that using the terms “undocumented” and “illegal” in conjoint instructions does not impact respondents’ policy preferences (see Section G of Supplemental Material).

Figure 1.

The effects of conjoint attribute values on guessing that an immigrant is “illegal” or “undocumented”. Note. See Section H of Supplemental Material for the results in terms of marginal means.

The effects of terms used to describe unauthorized immigrants may vary by respondents’ partisanship. ⁵ Democrats and Republicans tend to follow media outlets that differ in their usage of words like “illegal” and “undocumented” (Merolla et al., 2013; Nelson and Davis-Wiley, 2018), so these terms can activate disparate perceptions among supporters of the two parties. To explore that potential heterogeneity, we re-estimate the conjoint results separately for Democrats and Republicans (for the purpose of this analysis leaners are treated as partisans). Results are presented in Figure 2. They show mostly null effects of linguistic labels on perceptions among supporters of the two parties, although the difference in perceived welfare dependency between “illegal” and “undocumented” immigrants is driven by Democrats. Joint tests confirm no significant differences in overall perceptions by experimental condition for both Democrats (F_{10, 501} = 1.18, p = .300) and Republicans (F_{10, 531} = 1.06, p = .389).

Figure 2.

The effects of conjoint attribute values on guessing that an immigrant is “illegal” or “undocumented” by respondents’ partisanship. Note. See Section I of Supplemental Material for the results in terms of marginal means.

Does this mean that the language used to talk about unauthorized immigrants is politically irrelevant? To answer this question, we compare respondents by their own preferences regarding the words “illegal” and “undocumented” (preferred, respectively, by 59.3% and 40.7% of the sample). ⁶ Preferences for the latter term are higher among younger respondents, those with college education, and Democrats—while there are no significant differences between white and nonwhite respondents (see Section K of Supplemental Material for full regression results). We also find that preferences for the terms “illegal” and “undocumented” are strongly and significantly associated with opinions on immigration enforcement, even controlling for partisanship and demographics (see Section L of Supplemental Material).

Conclusion

In this note, we have reconsidered existing experimental findings regarding the null effects of “illegal” vs. “undocumented” language by assessing the effects of this word choice on perceptions about immigrants without legal status. Using experiment-within-experiment design, we have shown that while Americans associate unauthorized immigrants with lower education and higher likelihood of having criminal record, these beliefs do not change depending on whether the population in question is described as “illegal” or “undocumented.” Overall, we extend the existing experimental findings regarding the null causal effects of the “illegal” vs. “undocumented” language by demonstrating that the same null effect is found when beliefs about unauthorized immigrants are concerned.

Importantly, these results do not mean that the language used to discuss immigration is completely irrelevant. For instance, the finding regarding differential associations of “illegal” and “undocumented” immigrants with receiving government benefits can have implications in the context of the ongoing debate about “immigrationization” of welfare (Garand et al., 2017; Levy, 2021). In addition, changing minds is not the only purpose of political rhetoric—it can be used to set the agenda, build coalitions, and signal ideological commitments (Gillion, 2016). Indeed, our observational results suggest exactly the latter: respondents’ preferences for the word “illegal” or “undocumented” are related to their attitudes toward immigration. Still, manipulating these words in the context of a survey experiment does not have a causal effect on perceptions about immigrants. In combination with earlier null findings, our result suggests that the focus on terminology in contemporary immigration discourse may be somewhat exaggerated.

Supplemental Material

Supplemental Material - Beliefs and opinions about “illegal” and “undocumented” immigrants: Conceptual replication of a null result

Supplemental Material for Beliefs and opinions about “illegal” and “undocumented” immigrants: Conceptual replication of a null result by Kirill Zhirkov and Robert H. Brehm in Research & Politics