We study the implications of post-treatment bias in the context of the globalization backlash. We discuss whether horse-race regressions can inform about the relative role of economic versus cultural drivers. We make three methodological points: (1) if and insofar as cultural variables are post-treatment with respect to economic factors, the estimates of the effect of economic shocks on voting are biased in regressions that include cultural controls (and vice versa); (2) for the same reason, such horse-race regressions do not allow to accurately estimate the relative role of economic versus cultural factors; (3) one cannot infer mediation effects from changes in regression coefficients for a given factor of interest before and after including post-treatment controls. We accompany the methodological discussion with empirical evidence on the relevance of post-treatment bias in studies of the globalization backlash, both by replicating and expanding on earlier studies, and by presenting novel cross-country results on the culture-economy nexus.
AcharyaA.BlackwellM.SenM. (2016). Explaining causal findings without bias: Detecting and assessing direct effects. American Political Science Review, 110(3), 512–529. https://doi.org/10.1017/s0003055416000216
AchenC. H. (2005). Let’s put garbage-can regressions and garbage-can probits where they belong. Conflict Management and Peace Science, 22(4), 327–339. https://doi.org/10.1080/07388940500339167
4.
AgnolinP.ColantoneI.StanigP. (2024). Replication data for: In search of the causes of the globalization backlash. https://doi.org/10.7910/DVN/MKJYV3
5.
AnduizaE.RicoG. (2023). Sexism and the far-right vote: The individual dynamics of gender backlash. American Journal of Political Science, 68(2), 478–493. https://doi.org/10.1111/ajps.12759
6.
AnelliM.ColantoneI.StanigP. (2021). Individual vulnerability to industrial robot adoption increases support for the radical right. Proceedings of the National Academy of Sciences, 118(47), Article e2111611118. https://doi.org/10.1073/pnas.2111611118
7.
AngristJ. D.ImbensG. W.RubinD. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455. https://doi.org/10.2307/2291629
8.
AngristJ. D.PischkeJ.-S. (2009). Mostly harmless econometrics: An empiricist’s companion. Princeton University Press.
9.
AutorD.DornD.HansonG.MajlesiK. (2020). Importing political polarization? The electoral consequences of rising trade exposure. The American Economic Review, 110(10), 3139–3183. https://doi.org/10.1257/aer.20170011
10.
AutorD.DornD.HansonG. H. (2021). On the persistence of the China shock (tech. rep.). Brookings Papers on Economic Activity.
11.
Ballard-RosaC.JensenA.ScheveK. (2022). Economic decline, social identity, and authoritarian values in the United States. International Studies Quarterly, 66(1), sqab027. https://doi.org/10.1093/isq/sqab027
12.
Ballard-RosaC.MalikM. A.RickardS. J.ScheveK. (2021). The economic origins of authoritarian values: Evidence from local trade shocks in the United Kingdom. Comparative Political Studies, 54(13), 2321–2353. https://doi.org/10.1177/00104140211024296
13.
BaronR. M.KennyD. A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182. https://doi.org/10.1037//0022-3514.51.6.1173
14.
BaroneG.D’IgnazioA.de BlasioG.NaticchioniP. (2016). Mr. Rossi, Mr. Hu and politics: The role of immigration in shaping natives’ voting behavior. Journal of Public Economics, 136(100), 1–13. https://doi.org/10.1016/j.jpubeco.2016.03.002
15.
BartelsL. M. (2002). Beyond the running tally: Partisan bias in political perceptions. Political Behavior, 24(2), 117–150. https://doi.org/10.1023/a:1021226224601
16.
CarrerasM.Irepoglu CarrerasY.BowlerS. (2019). Long-term economic distress, cultural backlash, and support for Brexit. Comparative Political Studies, 52(9), 1396–1424. https://doi.org/10.1177/0010414019830714
17.
CavailleC.MarshallJ. (2019). Education and anti-immigration attitudes: Evidence from compulsory schooling reforms across Western Europe. American Political Science Review, 113(1), 254–263. https://doi.org/10.1017/s0003055418000588
18.
ChanT. W.HendersonM.SironiM.KawalerowiczJ. (2020). Understanding the social and cultural bases of Brexit. British Journal of Sociology, 71(5), 830–851. https://doi.org/10.1111/1468-4446.12790
19.
ClaytonK.FerwerdaJ.HoriuchiY. (2021). Exposure to immigration and admission preferences: Evidence from France. Political Behavior, 43(1), 175–200. https://doi.org/10.1007/s11109-019-09550-z
20.
ColantoneI.OttavianoG. I.StanigP. (2022). The backlash of globalization. In GopinathG.HelpmanE.RogoffK. S. (Eds.), Handbook of international economics (Vol. V, pp. 405–477). Elsevier Science.
ColantoneI.StanigP. (2018b). The trade origins of economic nationalism: Import competition and voting behavior in western Europe. American Journal of Political Science, 62(4), 936–953. https://doi.org/10.1111/ajps.12358
23.
DustmannC.VasiljevaK.Piil DammA. (2019). Refugee migration and electoral outcomes. The Review of Economic Studies, 86(5), 2035–2091. https://doi.org/10.1093/restud/rdy047
24.
ESS. (2002). Ess round 1: European social survey round 1 data (2002) (6.6th ed.) [Data Archive and distributor of ESS data for ESS ERIC. The European Social Survey.
25.
ESS. (2004). Ess round 2: European social survey round 2 data (2004) (3.6th ed.) [Data Archive and distributor of ESS data for ESS ERIC. The European Social Survey.
26.
ESS. (2006). Ess round 3: European social survey round 3 data (2006) (3.7th ed.) [Data Archive and distributor of ESS data for ESS ERIC. The European Social Survey.
27.
ESS. (2008). Ess round 4: European social survey round 4 data (2008) (4.5th ed.) [Data Archive and distributor of ESS data for ESS ERIC. The European Social Survey.
28.
EvansG.FieldhouseE.GreenJ.SchmittH.van der EijkC.MellonJ.ProsserC. (2016). British election study internet panel wave 8 (2016 eu referendum study, daily campaign survey). https://doi.org/10.5255/UKDA-SN-8202-2
29.
EVS. (2020). European Values Study Longitudinal Data File 1981-2008 (EVS 1981-2008). GESIS Data Archive, Cologne. ZA4804 Data file Version 3.1.0. https://doi.org/10.4232/1.13486
30.
FerraraF. M. (2022). Why does import competition favor republicans? Localized trade shocks and cultural backlash in the US. Review of International Political Economy, 30(2), 678–701. https://doi.org/10.1080/09692290.2021.1980898
FerrariD.FranzeseR.JacksonH.WaiR.WuP.KimB.KimW.PollackE.SandersH. (2021). How socioeconomic malaise & decline fuel xenophobic nationalist extremism. University of Michigan.
33.
ForastiereL.MatteiA.DingP. (2018). Principal ignorability in mediation analysis: Through and beyond sequential ignorability. Biometrika, 105(4), 979–986. https://doi.org/10.1093/biomet/asy053
FranzeseR. J. J. (2019). The comparative and international political economy of anti-globalization populism. Oxford Research Encyclopedia of Politics.
36.
FriedenJ. (2022). Attitudes, interests, and the politics of trade: A review article. Political Science Quarterly, 137(3), 569–588. https://doi.org/10.1002/polq.13360
37.
FrischR.WaughF. V. (1933). Partial time regressions as compared with individual trends. Econometrica: Journal of the Econometric Society, 1(4), 387–401. https://doi.org/10.2307/1907330
38.
GallegoA.KurerT. (2022). Automation, digitalization, and artificial intelligence in the workplace: Implications for political behavior. Annual Review of Political Science, 25(1), 463–484. https://doi.org/10.1146/annurev-polisci-051120-104535
39.
GelmanA.HillJ. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.
40.
GelmanA.ImbensG. (2013). Why ask why? Forward causal inference and reverse causal questions (working paper No. 19614). National Bureau of Economic Research. https://doi.org/10.3386/w19614
41.
GidronN.HallP. A. (2017). The politics of social status: Economic and cultural roots of the populist right. British Journal of Sociology, 68(Suppl 1), S57–S84. https://doi.org/10.1111/1468-4446.12319
42.
GrazianoW. G.HabashiM. M.EvangelouD.NgambekiI. (2012). Orientations and motivations: Are you a “people person,” a “thing person,” or both?Motivation and Emotion, 36(4), 465–477. https://doi.org/10.1007/s11031-011-9273-2
43.
HangartnerD.DinasE.MarbachM.MatakosK.XefterisD. (2019). Does exposure to the refugee crisis make natives more hostile?American Political Science Review, 113(2), 442–455. https://doi.org/10.1017/S0003055418000813
ImaiK.KeeleL.TingleyD.YamamotoT. (2011). Unpacking the black box of causality: Learning about causal mechanisms from experimental and observational studies. American Political Science Review, 105(4), 765–789. https://doi.org/10.1017/s0003055411000414
46.
KitscheltH.RehmP. (2014). Occupations as a site of political preference formation. Comparative Political Studies, 47(12), 1670–1706. https://doi.org/10.1177/0010414013516066
47.
MargalitY. (2019). Economic insecurity and the causes of populism, reconsidered. The Journal of Economic Perspectives, 33(4), 152–170. https://doi.org/10.1257/jep.33.4.152
48.
McKayD. A.TokarD. M. (2012). The HEXACO and five-factor models of personality in relation to RIASEC vocational interests. Journal of Vocational Behavior, 81(2), 138–149. https://doi.org/10.1016/j.jvb.2012.05.006
49.
MilnerH. V. (2021). Voting for populism in Europe: Globalization, technological change, and the extreme right. Comparative Political Studies, 54(13), 2286–2320. https://doi.org/10.1177/0010414021997175
50.
MontgomeryJ. M.NyhanB.TorresM. (2018). How conditioning on posttreatment variables can ruin your experiment and what to do about it. American Journal of Political Science, 62(3), 760–775. https://doi.org/10.1111/ajps.12357
51.
MutzD. C. (2018). Status threat, not economic hardship, explains the 2016 presidential vote. Proceedings of the National Academy of Sciences, 115(19), E4330–E4339. https://doi.org/10.1073/pnas.1718155115
52.
NorrisP.InglehartR. (2019). Cultural backlash: Trump, Brexit, and authoritarian populism. Cambridge University Press. https://doi.org/10.1017/9781108595841
53.
ReissP. C.WolakF. A. (2007). Structural econometric modeling: Rationales and examples from industrial organization. Handbook of Econometrics, 6, 4277–4415.
54.
RosenbaumP. R. (1984). The consquences of adjustment for a concomitant variable that has been affected by the treatment. Journal of the Royal Statistical Society: Series A, 147(5), 656–666. https://doi.org/10.2307/2981697
55.
SamiiC. (2016). Causal empiricism in quantitative research. The Journal of Politics, 78(3), 941–955. https://doi.org/10.1086/686690
56.
SmithE. R. (1982). Beliefs, attributions, and evaluations: Nonhierarchical models of mediation in social cognition. Journal of Personality and Social Psychology, 43(2), 248–259. https://doi.org/10.1037/0022-3514.43.2.248
57.
TabelliniM. (2019). Gifts of the immigrants, woes of the natives: Lessons from the age of mass migration. The Review of Economic Studies, 87(1), 454–486. https://doi.org/10.1093/restud/rdz027
58.
University of Essex, Institute for Social and Economic Research. (2023). Understanding society: Waves 1-13, 2009-2022 and harmonised bhps: Waves 1-18, 1991-2009. [data collection] (18th) [SN: 6614]. https://doi.org/10.5255/UKDA-SN-6614-19
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
