Estimating anti-immigrant sentiment for the American states using multi-level modeling and post-stratification,2004

Abstract

An estimate is provided of an innovative state-level measure of anti-immigrant sentiment for use in future policy and behavioral studies. State governments became increasingly active in adopting immigrant policies in the 2000s. Previous research highlights the role of public opinion, especially attitudes toward immigrants, in explaining policy priorities and outcomes. Unfortunately, most extant studies utilize political ideology or immigrant populations as rough proxies for public opinion. In this article, we estimate a reliable and valid measure of anti-immigrant sentiment at the state-level using survey aggregation with multilevel regression and post-stratification (MRP) for the period 2004 to 2008. We compare our estimates of anti-immigrant sentiment to alternative measures of immigrant presence and political ideology in predicting multiple variations of state immigrant policies. Ultimately, we find theoretical and statistical advantages of using anti-immigrant sentiment over previous measures in predicting immigrant policies.

Keywords

Anti-immigrant sentiment immigration public opinion state policy

Introduction¹

Over the last decade, state governments have increasingly played an active role in shaping immigrant policy by enacting more than 1500 pieces of legislation (Monogan, 2013). States have legislated a blend of both favorable and unfavorable policies with real-world consequences for immigrants. For example, Arizona’s SB 1070 included a provision allowing law enforcement to conduct immigrant status checks for routine police stops. Research has highlighted the central role of public opinion in determining the flavor of immigrant policies (Hero and Preuhs, 2007; Hopkins, 2010; Turner and Sharry, 2012). The core argument is that high or low levels of anti-immigrant sentiment, resulting from socio-psychological influences or sociotropic economic threat about growing immigrant populations, influence state immigrant policy adoptions. Heightened conditions of mass anti-immigrant sentiment create a window for the formation of an opinion–policy linkage where political elites adopt anti-immigrant policies to gain electoral support.

A critical task for assessing the opinion–policy linkage between anti-immigrant sentiment and state-level policies is accurately measuring public opinion. Previous research utilizes demographic proxies, such as the proportion of foreign-born in a state, or general mass attitudes, such as political ideology, to operationalize anti-immigrant sentiment (Hero and Preuhs, 2007; Nicholson-Crotty and Nicholson-Crotty, 2011). The use of indirect measures is due to data limitations from the lack of “a reliable measure of public attitudes about this issue available for all fifty states…” (Nicholson-Crotty and Nicholson-Crotty, 2011: 617). For example, Schildkraut (2001) found that states with ballot initiatives and a larger proportion of its population being foreign born were more likely to adopt English-only laws due to initiatives being a vehicle for anti-immigrant sentiment.

In this paper we generate reliable and valid estimates of anti-immigrant sentiment in all 50 American states. We discuss the utilization of a simulation approach called multi-level regression and post-stratification (MRP) that uses statistical modeling of national-level surveys to estimate aggregate state-level opinions (Lax and Phillips, 2009a, 2009b; Park et al., 2006; Pacheco, 2011). Using MRP, we provide estimates of anti-immigrant sentiment for the years 2004, 2006 and 2008. We find a strong relationship between the estimates of anti-immigrant sentiment and state immigrant policies, suggesting that our measure carries analytical value.

Conceptualization and measuring anti-immigrant sentiment

To measure anti-immigrant sentiment we used a commonly employed survey question. The American National Election Study (ANES) and the General Social Survey (GSS) asks respondents “Do you think the number of immigrants from foreign countries who are permitted to come to the United States to live should be increased a little, increased a lot, decreased a little, decreased a lot, or left the same as it is now?”² This question holds several advantages.³ First, it is commonly used to measure anti-immigrant sentiment in the political behavior literature (see Brader et al., 2008; Muste, 2013; Voss et al., 2013). Second, the question is included in several national surveys, allowing responses to be combined within a single year to increase the sample size per state. Finally, third, the regular inclusion of this question allows for the continuation of this dataset over time. The key drawbacks of this survey question are that it refers specifically to legal immigration, and the question is asking respondents about a preference for a generic immigration policy outcome, not particular state-level immigrant policies.

We choose to measure negative attitudes toward immigrants for theoretical reasons based on evidence from previous research. First, Monogan (2013) found that states adopt a mixture of policies toward immigrant populations, but that negative policies tend to withhold material benefits with real-world consequences (e.g. restricting access to Medicaid coverage) while positive policies tend to be symbolic gesturing (e.g. launching a task force that examines employment barriers for immigrants). Second, the impact of negative news stories about immigrants motivates individuals to contact their elected representatives, requesting them to adopt restrictive policies (Brader et al., 2008). Positive news stories create a much smaller motivation to contact representatives and, in some cases, actually decrease motivation in comparison to a control group. Based on these individual-level findings, we anticipated a stronger relationship between anti-immigrant sentiment and immigrant policies.

Mechanics of measuring anti-immigrant sentiment

The largest hurdle in estimating state-level measures of anti-immigrant sentiment is the lack of an individual survey with population coverage and population homology. To overcome this, we combined multiple surveys together when possible and used MRP to estimate anti-immigrant sentiment.⁴ The MRP model is a somewhat simple method, with two stages, but previous research has demonstrated that it can still produce highly accurate state-level estimates of public opinion (Butz and Kehrberg, 2015; Lax and Phillips, 2009a). The advantage of MRP is that state-level public opinion can be estimated from as little as a single survey with a national sample in a two-step process (Lax and Phillips, 2009a; 2009b; Park et al., 2006; Pacheco, 2011). The first step is to estimate a multi-level model for individual responses using demographic and geographic variables as predictors. The advantage of this model is that MRP uses respondents to create predictions regardless of location to overcome smaller state-level sample sizes found in national surveys. The individual level variables include gender (male or female), race (White, Black or Hispanic), age (four categories: 18–29, 30–44, 45–64, and 65+), and education (four categories: less than a high school education, a high school education, some college, and college graduate). Interaction terms of race*gender and age*education are included in the model. At the macro-level, the model includes measures for region (Midwest, Northeast, South, and West), the state, the proportion of the state population identifying themselves as evangelical Protestants or Mormons, and the Presidential vote for Kerry in 2004.

In the second stage, MRP takes the estimates for each demographic–geographic type in the multi-level model and weighs the estimates according to the proportion of each type in state populations using the 2000 US Census (Lax and Phillips, 2009b). Post-stratification corrects for over-sampling or under-sampling of demographic categories (Voss et al., 1995).

Previous scholars derived MRP estimates of political attitudes using national surveys based on simple random or stratified sampling. For example, Lax and Phillips (2009a: 384) used surveys by Gallup, Pew, ABC News, CBS News, AP, Kaiser and Newsweek to estimate public opinion on gay rights. Two of the most widely used surveys in social science research – the ANES and the GSS – use cluster sampling based on geographic locations, a procedure that potentially influences the quality of state-level estimates. Scholars using cluster sampling to estimate state-level attitudes have expressed several concerns: representativeness, stability, reliability, validity and utility (Brace et al., 2002).⁵ Brace and his co-authors (2002: 175) discussed the issue of clustered sampling with the GSS: ‘The states are not populations of interest for this national survey, and we cannot assume that the sampling frame employed for the GSS will produce representative estimates of state population characteristics’. The use of cluster sampling directly calls into question the representativeness, and in turn the validity, of using the ANES or the GSS for state-level estimates. The problem of cluster sampling can be mediated by including an additional state-level measure, such as election results, in the MRP model, by combining surveys with different sampling frames, such as we chose to do in this research (Stollwerk, 2013). The step of post-stratification also corrects for cluster sampling (Lax and Phillips, 2009b). We combined the ANES and GSS in 2004 and 2008 and used only the GSS in 2006 to estimate anti-immigrant sentiment.⁶ The combined sample size in 2004 was 2896 respondents (ANES = 1014; GSS = 1882) and in 2008 the sample size was 3138 (ANES = 1917; GSS = 1221). The sample for the 2006 GSS includes 1838 respondents.⁷

The difficulty in evaluating MRP output lies in knowing when the procedure is generating accurate estimates. The accuracy of MRP estimates does vary across policy areas and the estimates can be less accurate with weak geographic covariates and a low ‘ratio of opinion variation across geographic units relative to opinion variation within units’ (Buttice and Highton, 2013: 465). Strong geographic variables can account for a significant portion of the variation in public opinion and, as such, inclusion of state-level variables is a necessary condition for accurate MRP estimates. MRP estimates can be difficult to test without population measures of political attitudes per state. The existence of such population-level data would eliminate the need to use MRP. Encouraging for our measurement approach, previous research has found that MRP estimates cultural and social attitudes, such as immigration, more accurately than other policy preferences with the inclusion of state-level variables for presidential vote and religion (Buttice and Highton, 2013; Lax and Phillips, 2009b). In the final summation, MRP is found routinely to outperform other methods even when the data is not ideal.

Evaluating an MRP measure of anti-immigrant sentiment

In Figure 1 we show our state-level estimates of anti-immigrant sentiment for the years 2004, 2006 and 2008. The numerical values presented in Figure 1 are also reported in the Appendix. There are a few aspects of note. First, while significant variation exists across states, nearly all cases demonstrate a sizable amount of anti-immigrant sentiment. A non-trivial quantity of citizens holds anti-immigrant views within every state. Second, anti-immigrant sentiment increases and decreases across states throughout the sample years to a greater degree than demographic changes, which tend to be much smaller and slower. This greater degree of change in attitudes is due to immigration attitudes also being influenced by local economic conditions (Hopkins, 2010) and local media messaging (Branton and Dunaway, 2009; Dunaway et al., 2010). Third, the average trend in attitudes over the 2004–2008 time period bends toward immigrant accommodation. Fourth, our estimates indicate greater variance in attitudes across states than across time.

Figure 1.

State-level estimates of anti-immigrant sentiment 2004–2008.

In the following sections, we test the reliability and validity of our estimates of anti-immigrant sentiment.

Reliability

A comparison of the anti-immigrant sentiments in Figure 1 suggests that our measure is reliable. Overall, the estimates do not change dramatically from year to year. The correlations within and across survey years in Table 1 are strong, indicating that the estimates are similar between the GSS and ANES surveys for each year and across the years. The correlations across survey years also indicates that the MRP estimates for 2006, which only uses the GSS, are reliable regardless of using a survey with one sampling frame and cluster sampling. Overall, these results support the notion that the measure is reliable.

Table 1.

Correlations within and across years for estimates of anti-immigrant sentiment.

Year	Within years	Across Years
2004	0.88	2006 – 0.78 2008 – 0.91
2006	–	2008 – 0.77
2008	0.71	–

Note: Results are Pearson R coefficients and statistically significant at p < 0.01.

Validity

We examined the validity of our measure in three ways. First, we compared our measure of anti-immigrant sentiment to previously used proxies, immigrant population demographics and political ideology, to assess convergent validity. Second, we examined the correlation between anti-immigrant sentiment and multiple measures of state-level immigrant policies to test predictive validity. Third, we compared the coefficient for anti-immigrant sentiment to the coefficients of political ideology and the rate of change in immigration population demographics in predicting immigrant policy tone.

Previous research used ideology and immigrant demographics as proxies for immigration attitudes. Our measure of anti-immigrant sentiment should be strongly related to these proxies, and the cross-sectional correlations support this prediction. First, anti-immigrant sentiment is positively and significantly correlated with the change in foreign-born population levels (2004 r = 0.31; 2006 r = 0.50; 2008 r = 0.32; p < 0.05). Second, the correlation between the public opinion estimates and the proportion of foreign-born is significantly and negatively correlated (2004 r = −0.69; 2006 r = −0.56; 2008 r = −0.64; p < 0.001). Third, anti-immigrant sentiment is negatively correlated with political liberalism (2004 r = −0.49; 2006 r = −0.37; 2008 r = −0.41; p < 0.01). In sum, states with a recently growing immigrant population are more likely to have higher levels of anti-immigrant sentiment, while more liberal states and states with larger foreign-born populations have lower levels of anti-immigrant sentiment.

We assessed the construct validity of our measure of anti-immigrant sentiment by examining the relationship to multiple measures of state-level immigrant policies. In Table 2 we report the correlations between the estimate of anti-immigrant sentiment and five different measures of immigrant policies:⁸ Immigrant Policy Tone (Monogan, 2013), Immigrants’ Climate Index (Pham and Van, 2013), Integration Policies (Boushey and Luedtke, 2011), Control Policies (Boushey and Luedtke, 2011), and SB 1070 Copycat Bills (Wallace, 2014). As expected, anti-immigrant sentiment is negatively correlated with immigration policy tone, immigrants’ climate index, and integration policies since positive values for these measures indicate pro-immigrant policies. Examining the table further, anti-immigrant sentiment is positively correlated with immigrant control policies and the introduction of SB 1070 copy cat legislation, but the correlation does not achieve statistical significance for the SB 1070 bills.

Table 2.

Correlation between state immigrant policies and MRP estimates of anti-immigrant sentiment.

Immigrant policies	Anti-Immigrant Sentiment
Immigrant policy tone	−0.61***
Immigrants’ climate index	−0.40***
Integration policies	−0.54***
Control policies	0.26*
SB 1070 copy cat	0.19

Note: *p < 0.10; **p < 0.05; ***p < 0.01. Pearson R correlations. Immigrant Policy Tone 2005 to 2011 (Monogan, 2013); Immigrants’ Climate Index 2005 to 2012 (Pham and Van, 2013); Integration Policies 1997 to 2008 (Boushey and Luedtke, 2011); Control Policies 1997 to 2008 (Boushey and Luedtke, 2011); SB 1070 Copycat 2010 to 2011 (Weaver, 2014). All correlations are for the 2004 measure of anti-immigrant sentiment except for SB 1070 Copy Cat data, which uses the 2008 estimates.

In Table 3 we report the coefficients for political ideology, change in foreign-born population, and anti-immigrant sentiment, from a series of regression models predicting immigrant policy tone with control variables.⁹ Political ideology and change in foreign-born population are in the expected direction and statistically significant in predicting immigrant policy tone. Substituting anti-immigrant sentiment generates similar results as the proxies for immigration attitudes. Anti-immigrant sentiment and political ideology coefficients are in opposite directions, with political ideology coefficient being positive and anti-immigrant sentiment negative. As mentioned earlier, higher values for political ideology indicate more liberal states and, as such, we expect a positive relationship because higher values correspond with more welcoming policies for immigrant policy tone measure. In conclusion, we find evidence for the validity of our estimates of anti-immigrant sentiment in predicting immigrant policies.

Table 3.

Comparing anti-immigrant sentiment to political ideology and change in foreign population in predicting immigrant policy tone.

	Model 3A	Model 3B	Model 3C	Model 3D	Model 3E	Model 4F
Anti-immigrant sentiment	−0.072*** (0.021)			−0.048** (0.021)	−0.062*** (0.022)	−0.046** (0.021)
Citizen ideology		0.027*** (0.007)		0.022*** (0.007)		0.018** (0.007)
Change foreign-born			−0.005*** (0.001)		−0.004*** (0.001)	−0.002* (0.001)
Unified Republican government	−0.014 (0.035)	0.026 (0.038)	−0.018 (0.036)	0.028 (0.036)	−0.004 (0.033)	0.027 (0.035)
Per capita gross state product	0.024* (0.015)	0.033*** (0.011)	0.037*** (0.016)	0.023** (0.011)	0.023 (0.016)	0.022* (0.012)
Term limits	−0.309* (0.181)	−0.204 (0.178)	−0.298* (0.177)	−0.230 (0.170)	−0.309* (0.167)	−0.245 (0.164)
Legislative professionalism	2.384*** (0.766)	2.138*** (0.741)	2.454*** (0.799)	1.943** (0.740)	2.108*** (0.733)	1.850** (0.702)
Constant	2.619 (1.563)	−3.168*** (0.594)	−1.514** (0.699)	0.192 (1.559)	2.433 (1.531)	0.517 (1.419)
R ²	0.54	0.59	0.53	0.63	0.60	0.65
N	50	50	50	50	50	50

Note: *p < 0.10; **p < 0.05; ***p < 0.01. All models use OLS Regression to generate coefficient estimates.

In the last three columns in Table 3 we include anti-immigrant sentiment in models with citizen ideology and change in foreign-born population. Overall, anti-immigrant sentiment remains a significant predictor of immigration policy tone. Foreign-born population is no longer significant with the inclusion of anti-immigrant sentiment, offering additional evidence for the validity of our public opinion measure. Citizen ideology and anti-immigrant sentiment are both significant predictors of overall immigrant policy tone when included in the same model. It is likely that the variables are tapping into different effects in predicting immigrant policy tone, because both have meaningful and statistically significant effects. This supports the idea that conservative values of individualism, laissez-faire economics, distaste for change, and a preference for small government do not necessarily translate into hostile attitudes towards immigrants and other minority groups (Stenner, 2005). In addition, this result supports previous findings that ideology, as a more general worldview, predicts a wide range of attitudes even with the inclusion of issue-specific attitudes in the statistical model (Erikson et al., 1993; Lax and Phillips, 2012).

Conclusion

We have developed reliable and valid measures of anti-immigrant sentiment for each American state at three different time points. Policy or subject-specific attitudes can be stronger predictors of policy outcomes (Lax and Phillips, 2012). Unlike the relatively static demographic measures, anti-immigrant sentiment is sensitive to changes in public opinion and elite influences on political attitudes. As Turner and Sharry (2012) pointed out in their case study, Oklahoma shifted from adopting welcoming policies to unwelcoming policies once elites brought the issue of immigration to the forefront, purposefully and strategically enflaming anti-immigrant sentiment. Our measure allows researchers to capture these more dynamic and nuanced changes in anti-immigrant sentiment, allowing for more robust predictions of immigrant policy and additional relevant state-level outcomes of interest. In addition, we encourage scholars to use MRP to develop more direct and diverse measures of attitudes towards immigrants.

Footnotes

Declaration of conflicting interest

The authors declare that there is no conflict of interest.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Supplementary material

The supplementary files are available at .

The replication files are available at

Carnegie Corporation of New York Grant

The open access article processing charge (APC) for this articlewas waived due to a grant awarded to Research & Politics from Carnegie Corporation of New York under its ‘Bridging the Gap’ initiative.

Notes

References

Berry

Ringquist

Fording

Hanson

(1998) Measuring citizen and government ideology in the American states, 1960–93. American Journal of Political Science 42(1): 327–348.

Boushey

Luedtke

(2011) Immigrants across the US federal laboratory: explaining state-level innovation in immigration policy. State Politics and Policy Quarterly 11 (4): 390–414.

Brace

Sims-Butler

Arceneaux

Johnson

(2002) Public opinion in the American states: new perspectives using national survey data. American Journal of Political Science 46 (1): 173–189.

Brader

Valentino

Suhay

(2008) What triggers public opposition to immigration? Anxiety, group cues, and immigration threat. American Journal of Political Science 52 (4): 959–978.

Branton

Dunaway

(2009) Spatial proximity to the US: Mexico border and newspaper coverage of immigration issues. Political Research Quarterly 62 (2): 289–302.

Buttice

Highton

(2013) How does multilevel regression and poststratification perform with conventional national surveys? Political Analysis 21(4): 449–467.

Butz

Kehrberg

(2015) Immigration and social distrust in the construction of welfare policy across the American states. Politics and Policy 43(2): 256–286.

Dunaway

Branton

Abrajano

(2010) Agenda setting, public opinion, and the issue of immigration reform. Social Science Quarterly 91(2): 359–378.

Erikson

Wright

McIver

(1993) Statehouse Democracy: Public Opinion and Policy in the American States. Cambridge: Cambridge University Press.

10.

Hero

Preuhs

(2007) Immigration and the evolving American welfare state: examining policies in the US States. American Journal of Political Science 51 (3): 498–517.

11.

Hopkins

(2010) Politicized places: explaining where and when immigrants provoke local opposition. American Political Science Review 104 (1): 40–60.

12.

Lax

Phillips

(2009a) Gay rights in the States: public opinion and policy responsiveness. American Political Science Review 103 (3): 367–386.

13.

Lax

Phillips

(2009b) How should we estimate public opinion in the States? American Journal of Political Science 53 (1): 107–121.

14.

Lax

Phillips

(2012) The democratic deficit in the States. American Journal of Political Science 56 (1): 148–166.

15.

Lewis

Wood

Jacobsmeier

(2014) Public opinion and judicial behavior in direct democracy systems: gay rights in the American states. State Politics and Policy Quarterly 14 (4): 367–388.

16.

Klarner

(2003) Measurement of the partisan balance ofstate government. State Politics and Policy Quarterly 3(3): 309–319.

17.

Monogan

III (2013) The politics of immigrant policy in the 50 US States, 2005–2011. Journal of Public Policy 33(1): 35–64.

18.

Muste

(2013) The Dynamics of Immigration Opinion in the United States, 1992–2012. Public Opinion Quarterly 77 (1): 398–416.

19.

Nicholson-Crotty

(2011) Industry strength and immigrant policy in the American states. Political Research Quarterly 64(3): 612–624.

20.

Pacheco

(2011) Using national surveys to measure state public opinion over time: a guideline for scholars and an application. State Politics and Policy Quarterly 11(4): 415–439.

21.

Park

Gelman

Bafumi

(2006) State-level opinions from national surveys: poststratification using multilevel logistic regression. In: Coehn

(ed) Public Opinion in State Politics. Stanford, CA: Stanford University Press.

22.

Pham

Van

(2012) Measuring state-created immigration climate. Migration and Citizenship Newsletter 1(2): 22–27.

23.

Schildkraut

(2001) Official-english and the states: influences on declaring english the official language in the united states. Political Research Quarterly 54(2): 445–457.

24.

Squire

(2012) The Evolution of American Legislatures: Colonies, Territories, and States, 1619–2009. Ann Arbor, MI: University of Michigan Press.

25.

Stenner

(2005) The Authoritarian Dynamic. Cambridge: Cambridge University Press.

26.

Stollwerk

(2013) The application of multilevel regression with post-stratification to cluster sampled polls: challenges and suggestions. In: 71st Annual Meeting of the Midwest Political Science Association, Chicago, IL, 11–14 April.

27.

Turner

Sharry

(2012) From progressive pioneer to nativist crackdown: the transformation of immigrant policy in Oklahoma. Politics and Policy 40(6): 983–1018.

28.

Voss

Gelman

King

(1995) Preelection survey methodology: details from eight polling organizations, 1988 and 1992. Public Opinion Quarterly 59(1): 98–132.

29.

Voss

Kehrberg

Butz

(2013) The structure of self-interest: applying comparative theory to American immigration attitudes. In: Freeman

Hansen

Leal

(eds) Immigration and Public Opinion in Liberal Democracies. New York: Routledge Publishing.

30.

Wallace

(2014) Papers please: an analysis of state level anti-immigrant legislation in the wake of Arizona’s SB 1070. Political Science Quarterly 129(2): 261–291.

Estimating anti-immigrant sentiment for the American states using multi-level modeling and post-stratification,2004–2008

Abstract

Keywords

Introduction 1

Conceptualization and measuring anti-immigrant sentiment

Mechanics of measuring anti-immigrant sentiment

Evaluating an MRP measure of anti-immigrant sentiment

Reliability

Validity

Conclusion

Footnotes

Declaration of conflicting interest

Funding

Supplementary material

Carnegie Corporation of New York Grant

Notes

References

Introduction¹