Abstract
Public attitudes towards immigration and immigrants, captured through social surveys, are widely reported in the media and used to inform political decision making. However, it is important to consider whether public attitudes are being accurately measured. This article uses critical discourse analysis and critical race and post-colonial theories to examine questions in leading social surveys. The article also draws upon interviews with survey managers and methodologists. In many high-quality surveys a ‘white’ identity is often framed as the norm alongside negative narratives of identity and difference. For example, in one survey question attitudes towards immigrants are asked about alongside attitudes towards alcohol and drug use. The objectivity of the framing and language of many survey questions needs to be reviewed. In the context of evidence of increased levels of racial discrimination, a new discourse is required to more objectively measure and understand public attitudes towards immigration and immigrants.
Introduction
Immigration is viewed as one of the key issues facing the UK and social survey data are central to measuring public attitudes towards immigration (Blinder and Richards, 2021; Ipsos MORI, 2017). Narratives regarding immigration are often dominated by essentialised ideas of identity and difference and negative public attitudes towards immigrants (Jones et al., 2017; Yuval-Davis et al., 2018). However, there has been only limited research that has critically analysed the ways in which public attitudes towards immigration and immigrants are measured in surveys.
This article’s theoretical framework draws upon Foucault’s (1972) and Fairclough’s (1992) theories of critical discourse analysis (CDA), critical race theory (Allen, 2017; Delgado and Stefancic, 2012) and post-colonial theory (Coghlan and Brydon-Miller, 2014). This informs the analysis of questions from a sample of leading social surveys in order to develop the understanding of how knowledge about public attitudes is collected. The analysis also reports evidence from interviews with leading survey managers and methodologists.
The term ‘immigrant’ is often conflated by the public, and in the media, to describe people who are in very different circumstances including people seeking asylum and ethnically and racially minoritised communities (Anderson and Blinder, 2019). The United Nations (UN, 2021) defines the term ‘migrant’ to be any person who changes their country of residence and stays for at least one year. There is no legal definition of being a migrant and the term may be used to refer to people who have migrated under very different circumstances, including those who come as refugees. 1
This article’s discussion is located within the UK’s current and past political response to immigration. The 2012 ‘Hostile Environment’ policy agenda was enacted through the 2014 and 2016 Immigration Acts, but such an approach is not new to the UK’s response to immigration, which developed directly out of the end of the Empire. It reflects a drive to control the movement of racialised and dispossessed diaspora by presenting a myth of a racially inclusive Britain, while preserving a claimed ‘white’ Britishness and by implication a racially ordered narrative of citizenship (Byrne, 2014; El-Enany, 2020). 2
In recent years there has been a rise in nationalist rhetoric that constructs ‘them’ (the Other) as a threat to the nation and ‘those people who can claim an “authentic” national identity’ (May et al., 2020: 1056). In the UK this was evident in the vote to leave the European Union (EU) in 2016, which, it has been argued, was linked to concerns about immigration, multiculturalism and ethnic diversity (Clery et al., 2017; Valluvan and Kalra, 2019). The campaign organisation, Stop Hate UK (2017), reported an increase in racially targeted hate crimes in the three-month period following the Referendum on EU membership.
The Hostile Environment policy as well as the 1960s Commonwealth Immigration Acts are linked to the Windrush scandal surrounding the rights of the children of Commonwealth citizens. These people, who despite living and working in the UK for many years, have been threatened with deportation and in some cases deported (Gentleman, 2019). This has brought into focus the discrimination faced by ethnically and racially minoritised communities and immigrants (Gedalof, 2022; Wardle and Obermuller, 2019). The Black Lives Matter movement also highlights ongoing discrimination and structural racism (Hodgkinson et al., 2021; Joseph-Salisbury et al., 2020; Maqbool, 2020). Evidence also suggests increases in the number of racial and religiously motived hate crimes including Anti-Semitism and Islamophobia (Akel, 2021; Community Security Trust, 2020; Tell Mama, 2020). Moreover, during the COVID-19 pandemic there were increases in racially targeted hate crimes towards Chinese and South-East Asian communities (Gover et al., 2020; Home Office, 2020).
Bhabha (1994) has drawn attention to the role of stereotypes and discourse in the construction and representations of the ‘Other’. Dominant anti-immigration discourses construct immigrants as potentially illegal and threatening and so legitimise policies that restrict movement (O’Neill et al., 2019). The European Commission Against Racism and Intolerance has highlighted the negative role that parts of the UK media and also politicians themselves are playing in contributing to racial intolerance (ECRI, 2016). Furthermore, research by the Muslim Council of Britain has highlighted the levels of Islamophobia in British newspapers (Hanif, 2018). In relation to these issues, it is useful to reflect on the work of Hall et al. (1978) who found that in the UK in the 1970s certain identities and groups were, and arguably still are, singled out and framed through a discourse of criminality, in part as a result of institutionalised racism. The media may play a key role in this framing and the state can exert control through this discourse. In this context, crime is framed as the problem of the ‘Other’ rather than understood as a result of underlying racism, inequality, poverty and powerlessness. McMahon and Roberts (2011) have also highlighted the problematic issues raised by oversimplifying the links between poverty, crime and race.
Social survey data on public attitudes are situated within this wider social and political context. However, they can also play a role in constructing this context. There are complex relationships between anti-immigration discourses, public attitudes and policy making. Anti-immigration discourses such as those in the media can influence public perceptions of immigration (Blinder and Jeannet, 2014; Eberl et al., 2018). Public attitudes can feed back into discourses and anti-immigration narratives are often dominated by negative public attitudes and essentialised ideas of identity and difference (Jones et al., 2017; Smart et al., 2005; Tyler, 2013; Yuval-Davis et al., 2018). Public attitudes can drive, and also be used to justify, particular policy responses towards an issue including immigration (Boa et al., 2010; Boehm et al., 2013; Bourdon, 2002; Kolarz et al., 2017; Marshall, 2017; ONS, 2019; Talbot and Talbot, 2014).
Given the complex links between public attitudes and policy making, critically examining how public attitudes towards immigration and immigrants are being measured in surveys is an important concern. It can be questioned how much a survey response is an accurate personal reflection or a reflection of wider social and political narratives.
This article examines how survey questions can be understood as being situated within the wider social context, and therefore analysed as a discourse. The key research questions are: how do social surveys measure public attitudes towards immigration and immigrants? How do social surveys configure the relationship between identity, immigration and race? To what extent are the questions framed in an essentialised narrative of identity and difference, and how has this changed over time? What are the implications for the measurement and understanding of public attitudes?
Background: Measuring Attitudes Towards Immigration and Immigrants
Attitudes are closely related to values and beliefs and it is generally accepted that they are acquired behavioural dispositions (Inglehart, 2010). Developing a survey involves defining and translating concepts and issues into a form that is measurable (De Vaus, 2013). In high-quality surveys, questions and response options are usually pretested to improve validity and reliability (Collins, 2003; Presser et al., 2004). A number of factors may affect the answers given by survey respondents including: interviewer effects (relating to the interaction between respondent and interviewer); the wording and ordering of the questions and response options; acquiescence bias; primacy or recency effects and response fatigue (Bryman, 2018; Lavrakas, 2008).
Following on from Tourangeau’s (1984) discussion about how respondents engage in four cognitive operations when assessing survey items (comprehension, recall, judgement and response), Holbrook et al. (2003: 82) state that ‘a respondent must interpret the meaning and intent of each question, retrieve all relevant information from memory, integrate that information into a summary judgment, and report that judgment accurately’. Many social surveys are conducted by an interviewer and so there is a dialogical nature to survey questions. The questions are meaningful because the words invoke a particular social imaginary, both for the interviewer and the respondent (De Vaus, 2013).
Of particular importance here is satisficing theory (Krosnick, 1991), where, due to the cognitive effort involved in answering a survey question, the respondent circumvents the cognitive process to provide an answer that would seem plausible to the interviewer. This can be problematic because discourse both reflects and creates meaning, therefore particular discursive formations can become dominant (Edley, 2001). In relation to questions about immigration specifically, respondents may draw upon easily available discursive formations that they have acquired from their social networks, media and political discourse, as a means of shortcutting the cognitive process.
Findings from experimental research may support this argument. For example, Sturgis and Smith (2010) asked participants about their views on a number of fictitious issues and found that respondents were willing to provide their views. The authors argued that when respondents have a limited understanding of a question there is a potential for them to draw upon dominant discourses. Furthermore, an experimental study by Miura and Kobayashi (2016), conducted with Japanese participants found that people who were likely to strongly satisfice when answering survey questions gave more stereotypical responses about immigrants. It could be that respondents are drawn to the stereotypes of such people that often dominate the media and political campaigns.
Research by Blinder (2015) has shown that in the UK when respondents answered questions about immigration they were most likely to be thinking of asylum seekers. Survey respondents have also been shown to overestimate the number of migrants who live in their local area and to overestimate the number of prisoners who were born in a foreign country (Ipsos MORI, 2017; ONS, 2014). These research findings may reflect the widespread media coverage of the issues of immigration and asylum seeking during this period including the so-called ‘refugee crisis’ (Agustín and Jørgensen, 2019). The findings also suggest that wider discourses have the potential to influence survey responses.
Theoretical and Analytical Framework
Language supplies different ways of constructing an object, event or person (Edley, 2001). However, some constructions and formulations are more available than others (Fairclough, 1992). Critical discourse analysis (CDA) aims to identify how discursive formations become dominant and normative. In this article, Foucault’s (1972) and Fairclough’s (1992) theories of discourse are drawn upon as an analytical framework. For Foucault (1972), power is the discursive ability to determine what is normal; it is a way of constituting knowledge (Hall, 1997). For Fairclough (1992), who draws upon Foucault, discourse is not only constitutive but also is constituted. Foucault’s framework, along with Fairclough’s work allow the language, statements and discursive formations that construct and convey particular meanings and knowledge in social survey questions to be analysed.
Research into the discourses used by the media to frame immigration is extensive (e.g. Buchanan et al., 2003; Cap, 2016; Eberl et al., 2018; Fotopoulos and Kaimaklioti, 2016; Triandafyllidou, 2017). Evidence suggests that although the media framing varies based on different groups, the coverage is consistently negative, conflict-centred and focused on quantification (Eberl et al., 2018; Finney and Simpson, 2009; Fotopoulos and Kaimaklioti, 2016). Immigrants are often represented as criminals and immigration is framed as an economic, cultural and/or criminal threat (Eberl et al., 2018).
One dominant discursive formation that often frames immigration includes what Cap (2016: 16) has theorised as proximisation: a ‘discursive strategy of presenting physically and temporally distant events (including distance adversarial ideologies) as increasing and negatively consequential to the speaker and her addressee’. This is underpinned by the ‘container’ metaphor, which defines the nation state as a container with boundaries and limited capacity (Hart, 2010). Phrases often used in conjunction with immigration include: ‘wave’, ‘doors’, ‘limited capacity’, ‘bursting point’, ‘overwhelm’ ‘absorbed’, ‘influx’, ‘throw open’ and ‘full up’.
The racialisation of immigration is also an influential factor in discursive formations. The construction of race is shaped historically and can be used to form the basis of exclusionary practices (Erel et al., 2016). In the UK, ‘long-standing black and Asian communities and post-colonial diaspora remain viewed as migrants’ (Turner, 2020: 60). This highlights the importance of examining the role of race in constructing the way the Other is represented. Furthermore, as has been seen with past and present generations of, for example, Irish, Jewish and earlier Eastern European immigrants, as well as those born in the UK, what for many is perceived as a shared ‘whiteness’ does not mean they will not experience racial discrimination (Community Security Trust, 2020; Fox et al., 2012).
Critical race theory is drawn upon for the analysis in this article. This places race and racism, as well as power, at the centre of the analysis by focusing on assessing the impact of the social construction of race (Allen, 2017; Delgado and Stefancic, 2012). In addition, Mayblin and Turner (2021) argue that Britain’s imperial past still influences contemporary life, including attitudes and policies towards immigration and immigrants. Therefore, the analysis is also informed by post-colonial theory. This considers the continuation of colonial and hegemonic perspectives in the construction of knowledge and their impact (Coghlan and Brydon-Miller, 2014).
Methodology
Critical Discourse Analysis of Social Survey Questions
Sample
A search for the relevant social surveys to be analysed was conducted based on the researchers’ knowledge of surveys, alongside searches conducted using the UK Data Service. The search queries included: ‘immigration’, ‘attitudes to immigration’, ‘ethnic minorities’, ‘migrant’, ‘refugee’ and ‘asylum seeker’. Following this, a sample of leading UK and international social surveys was selected, that is, those that were considered to be the most robust in terms of design, sample and data collection and also influential in political and media debates regarding immigration. Consideration was also given to leading commercial survey organisations, which are often used by policy makers and cited in the media (Kolarz et al., 2017; Marshall, 2017).
The sample of 11 leading surveys selected for analysis either solely focused on capturing attitudes to immigration and immigrants or had a section that aimed to capture such attitudes. All the questions from the survey or section were included in the analysis, although only a selection of these questions is reported in this article. The focus was on specific waves of each survey rather than tracking change within a single survey. The sample included surveys that were conducted face-to-face, self-completion, by telephone and online. The mode of the survey is not considered in detail in this article, but is part of ongoing follow-up research. The selected surveys are listed in Table 1.
Sample of social surveys.
The British Social Attitudes (BSA, 2021) survey is a leading survey of social and political attitudes. It is an annual survey that began in 1983; has a sample size of 3000 people and is funded by charity donations and government funding. The European Social Survey (ESS, 2021) is a leading survey of social and political attitudes across around 40 countries with a minimum sample size of 1500 people, which began in 2002. It is funded by the European Commission and national funding councils. The World Values Survey (WVS, 2021) is a global survey of social and political attitudes, which includes around 100 countries. It has an average sample size of 1200 people, began in 1981 and is funded through charitable donations. You Gov and National Opinion Poll (NOP) are examples of leading commercial polling organisations (NOP is now part of the research company GfK).
Survey Questions Discourse Analysis
Individual questions were selected from each of the surveys for in-depth analysis. This included examination of question wording and response options, question ordering and the context in which the survey was conducted. The issues of commissioning and funding were also examined in the interviews with the survey managers and methodologists.
The questions from the international surveys were analysed in their English language versions. Fairclough’s (1992) three-dimensional model for CDA was drawn upon. First, by analysing the discursive practice, which focuses on the way that a discourse is produced. This was done through reviewing the survey documents and also during the interviews with survey managers and methodologists about the ways that survey questions are developed. Second, by analysing the text, which involved examination of the wording and response options of the sample questions. Fairclough’s (1992) framework includes a focus on the linguistic characteristics of a text including: interactional control, ethos, metaphors, wording and grammar. In addition, and specific to this research, was consideration of the ordering of the questions and response options. Finally, by analysing social practice, which contextualises the discourse within its broader social practice. As outlined above, critical race theory (Allen, 2017; Delgado and Stefancic, 2012) and post-colonial theory (Coghlan and Brydon-Miller, 2014) were drawn upon to address this aspect of the analysis. In the findings not all the surveys are discussed in detail, however, the entire analysis informed the discussion.
Semi-Structured Interviews with Survey Managers and Methodologists
Semi-structured interviews were conducted with a sample of leading survey managers and methodologists in order to explore the issues in more detail (Ayres, 2008). The aim was to gain an understanding of how survey questions are developed and tested, and to explore the interviewees’ views about how the questions are framed, and how this may have changed over time.
The interviewees were selected on the basis that they were working as part of the design and delivery team of a leading social survey. They were identified through the researchers’ knowledge and online searches of the surveys’ websites. The sample included: two male academics, both of whom had worked on developing large-scale social surveys conducted in the UK, and three survey managers (two male and one female). All the interviewees had many years’ experience in developing and using survey data.
The interview schedule was developed from: (1) a literature review of the way that survey questions are developed and tested; and (2) the findings from the CDA of the sample survey questions. The interview questions explored issues such as: question wording, response options, pretesting, change over time and the wider social and cultural context. The interviews were conducted via telephone and lasted approximately 30 minutes. To encourage participation, no audio recordings were made, however, hand-written notes were taken and exact quotes written down. Recording interviews can have intrusive effects and while only using hand-written notes can reduce the information collected, the approach can encourage the discussion of potentially sensitive issues (Block and Erskine, 2012; Rutakumwa et al., 2020).
Analysis of the interviews was conducted using a hybrid method of deductive and inductive coding (Fereday and Muir-Cochrane, 2006). Quotes were coded manually in relation to the different issues raised by each interviewee. Themes already known to the researchers from the literature search informed the analysis. In addition, inductive coding was conducted to allow new themes to be identified. Indicative quotes highlighting the key themes from the interviews have been selected.
Ethical Issues
Good practices in social research and data handling were followed (BSA, 2017). Informed consent was obtained from all the interviewees. All the information from the hand-written notes was anonymised. The research was approved by the University Ethics Committee.
Findings
In the following sections the key findings are examined: (1) Immigration, Racialisation and Control; (2) Threat through Discursive Strategies.
Immigration, Racialisation and Control
Identity and Difference through Question Wording
The discourse used in many of the survey questions constructed immigration as an essentially racialised phenomenon and some of the questions reproduced a ‘white’ hegemonic racial hierarchy. Examining survey questions from different periods allowed for exploration of this over time. Table 2 highlights example questions from a 1978 NOP face-to-face survey, which collected data on public attitudes towards political parties, immigration and the National Health Service.
National Opinion Poll (NOP) questions 1978.
The first question phrasing sets up a spatial distance between those within the discourse space (Self) and immigrants on the periphery of the discourse space (the Other). The wording ‘let into’ carries connotations of control and power. In the second question the phrase ‘might be done about coloured immigrants’ positions immigrants as subject to control. The term ‘coloured’ is historically associated with segregation. Although such terms as ‘coloured’ are no longer commonly used in survey questions, arguably the discourse still remains, framed by narratives that can conflate race and immigration. This is evident in the questions from the 2018 and 2020 face-to-face and self-administered ESS, which covered a wide range of social and political issues (Table 3).
European Social Survey questions 2018 and 2020.
‘Should’ in the sense ‘ought to’, not in the sense of ‘must’.
The use of the phrase ‘to what extent do you think [country] should allow people. . .’, in the first question reinforces the narrative of power over constructed racial and ethnic identities of the Other. The response options such as ‘some’, ‘a few’ and ‘none’ reinforce this distancing. The use of the term ‘race’ in these questions constructs a narrative of difference, of ‘us’ and ‘them’. Furthermore, the use of the term ‘most’ implies the idea of a singular homogenous nation, where a majority ethnic group or race is the ‘correct’ population. This, of course, oversimplifies the history of how countries came into existence and the ethnic diversity within countries. The use of the term ‘ethnic group’ alongside race in the questions also creates a narrative of essentialised identity and difference. Ethnic identity is a complex and multidimensional concept, but it is often measured using a predefined list (Burton et al., 2010; Simpson et al., 2016). However, it is important to acknowledge that ethnicity is recognised by many groups as a way of trying to ensure recognition, including, for example, in official statistics (ONS, 2016).
Across many of the surveys that were examined, singular categories of ethnically and racially minoritised people were juxtaposed with identities, based on imagined narratives of ‘this country’, often framing ‘white’ as the norm. This includes questions where the respondents were asked to state to what extent they felt ‘comfortable’ or ‘uncomfortable’ with the number of ethnic minorities in the country. Some of the survey questions that were examined (not shown here) focused on country-based identities such as Pakistan, India, Africa and the West Indies, while others focused on religious groups such as Muslims, as well as non-western immigrants and immigrants from inside and outside the EU. These question formats arguably construct Self and racialised Other identities. Such framing can be oversimplistic. For example, the BSA 2003 survey focused the questions on immigration on attitudes towards Muslims, which overlooks evidence from the UK Census that 47% of Muslims in the UK were born in the UK (ONS, 2013). Questions about attitudes towards Muslims in the context of immigration creates a narrative of difference and the idea that the UK has a ‘natural’ inhabitant and that Muslims are the Other.
In international surveys, such as the ESS some questions were also framed by the assumption that nations have an essentialised ethnic or racial population group and that the immigration of other people could be viewed as problematic. Although the WVS wave analysed did not use this framing, immigration was still primarily presented as posing problems for the country, which highlights the dominance of certain framing and norms in survey questions.
Evidence from the interviews with the survey managers and methodologists suggests that the way questions are framed and the question wording could influence the responses from participants. For example, one survey manager commented: Imagine you were campaigning for longer prison sentencing – if you ask the public ‘Do you think prison sentences should be lengthened?’ the majority will say yes. But if you are campaigning for more community-based sentences and you ask ‘Do you think prison is effective in rehabilitation?’ most will say no it’s not. The way you ask a question can determine the answer. (Male, Survey Manager)
This clearly highlights the powerful role discourse including framing, wording and response options can have in the way respondents answer questions. This is important when we consider the way survey questions can racialise immigration.
Framing in Personal Terms and the Primacy of Difference
The narrative framing of the Other in survey questions also extended to personal relationships and family relationships. This is evidenced by the questions presented in Table 4 from the 2013 BSA face-to-face survey.
British Social Attitudes survey questions 2013.
The questions arguably present the respondent with the primacy of racial difference and narratives of family purity. This framing can be seen as creating a hierarchy of ethnic differences and acceptability. Racial hierarchies are rooted in colonialism (Fanon, 1991) and Turner (2020) has argued Eurocentric ideas continue to dominate and regulate normative ideas of ‘family’. In this question we also see how Eastern European migrants can be represented as racialised Others. This reflects how constructions of Other categories are flexible and situated. The word ‘mind’ in the questions has a negative connotation. In addition, the respondent is given more than one option for whether they ‘mind’, which may lead them to be more likely to choose one of these options and imply to the respondent that ‘minding’ is more likely than ‘not minding’.
Threat through Discursive Strategies
Container Metaphor and Quantification
One of the most common types of framings identified in the analysis of the surveys were those that asked respondents whether they believed immigration should be increased or decreased. The questions frequently focused on numbers using relatively subjective response options such as ‘a little’ and ‘a lot’. This prioritises the container metaphor and the idea of Britain having a ‘capacity’.
A number of questions also employed a discourse that creates a distance between immigrants coming from ‘non-western countries’ (Other) and ideas of ‘white’ and Britain (Self). Questions also referred to immigrants and asylum seekers without making distinctions between people who can be in very different circumstances. The questions framed the respondent to be thinking about migration of people from largely post-colonial states, as opposed to the migration of people from Europe.
In the interviews with leading survey managers and methodologists the issue of the framing of survey questions about immigration and immigrants was also highlighted. The focus on numbers was something one survey manager drew attention to as being an issue both now and, in the past, he commented: Six or seven years ago it [numbers] was asked about a lot in terms of asylum policy, these days it is asked in the context of Brexit . . . freedom of movement, but you see the language hasn’t really changed much, just the political context. (Male, Survey Manager)
Threat and Oversimplified Ideas of Culture
A related example of the use of the negative language of quantification and framing in relation to ideas of culture is also evident in the 2018 and 2020 ESS, as shown in Table 5.
European Social Survey questions 2018 and 2020.
In these questions, the response terms of ‘bad’, ‘good’, ‘undermined’ and ‘enriched’ frame the issue in a very simplified narrative. In all three questions the negative option is given first. Additionally, in the second question, the use of the phrase ‘cultural life’ could imply that there is a singular and homogenous narrative of the nation. This could reinforce the narrative of the immigrant as threatening, not only in an economic sense, but also culturally. The third question, with its focus on the impact immigration has on making a place a ‘worse or better place to live’, gives primacy to immigration as the most important issue in determining how ‘good’ a place to live is. The question wording replicates a dominant narrative of the proximisation of threat.
Question Format and Response Option Ordering
Other survey questions also used the proximisation discursive strategy and conveyed the threat of immigration through question formatting and response option ordering as shown in Table 6.
British Social Attitudes survey questions 2017.
The survey was carried out soon after the vote to leave the EU. The ordering of the response options prioritises the response ‘Reducing immigration’, which could introduce response bias through primacy effects. Moreover, all of the other options could be viewed as positive outcomes. Respondents were arguably being primed to see immigration as a ‘priority’, and also as a problem.
Negative Hypothetical Questions and Comparisons
A number of the survey questions also reinforced negative narratives of threat by juxtaposing immigrants with social and economic problems. For example, the following questions in Table 7 are taken from the 2009 BSA survey.
British Social Attitudes survey questions 2009.
The first question makes a negative link between immigration and crime, which is a common anti-immigration narrative. Perhaps even more striking is the question shown in Table 8, which is taken from the 2018 face-to-face and telephone administered WVS. This survey collects information on public attitudes towards a wide range of social and political issues.
World Values Survey questions 2018.
The question creates a narrative of ‘neighbours’ as a homogenous group that could be threatened if one of these other groups were to move in. The question also juxtaposes immigrants/foreign workers – and a number of other groups – with drug addicts and heavy drinkers.
Politicised Language and Context
When asked to comment on the way that questions, and the terms used, may frame attitudes towards immigration, one survey methodologist stated that he was not sure how it could be avoided any more than they already do: The terms you use, particularly in a political debate are identified with a particular position. In this country people who talk about ‘illegals’ come from the right wing and the left talk about undocumented. Finding a language that is neutral is difficult when the things we are studying are politically charged, but we try to avoid it as much as possible. (Male, Academic Survey Methodologist)
Here, the methodologist recognises that the terms social surveys use may have particular political connotations even when they try to avoid them. One commercial survey methodologist claimed that they would specifically avoid politically charged words such as ‘asylum seeker’. He commented: I try to avoid the word asylum seeker, because the public don’t know what an asylum seeker is, and it has become a buzzword for negative immigration . . . I try to avoid using words that have a negative connotation when there is a neutral alternative available. (Male, Commercial Survey Methodologist)
The ESS in 2014 and 2016 included questions regarding the respondents’ attitudes towards refugees and specifically whether the government should be ‘generous in judging people’s applications for refugee status’ if they were fleeing persecution. The use of the term ‘generous’ could frame the respondent’s answer. Moreover, in 2016 respondents were also asked about the extent to which they felt most refugees ‘aren’t in real fear of persecution in their own countries’, which associates asylum seekers and refugees with dishonesty. It is notable that the media coverage of the issues was widespread during this time period.
The Challenge of Reflecting Social and Language Change
A challenge that all the survey managers and methodologists were concerned with was ensuring consistency. While terminology could change over time, if the aim was to compare changes in attitudes, the questions needed to remain the same. As one survey methodologist commented: One thing you have to keep an eye out for is time series. You can’t change the questions or response options. So things can be outdated because the question can’t be changed . . . The client is reluctant to change the time series. It’s about finding the balance between having to change the questions as time moves on and being able to compare results across time. (Female, Independent Survey Methodologist)
This survey methodologist highlights the way language can change and how words that may have once been considered neutral can be revealed to be negative and/or take on new meanings. Moreover, terms such as ‘immigrant’ can take on particular meanings when placed within their wider social context and within discourses surrounding immigration. The desire to measure change by using the same words over time is clearly a barrier to capturing how public attitudes may have changed.
Question Testing
Although the survey managers and methodologists discussed cognitive interview pretesting methods and their importance in designing attitudinal questions, detailed information on the process and the findings was not easily available to those outside of the survey team. This makes scrutiny of the survey development process a challenge. Moreover, one survey manager described how the use of cognitive interviews was dependent on the survey’s budget: Cognitive interviews should always be carried out for attitudinal questions, but the client does not always have the money or time to do this. [. . .] Access to this [cognitive interview transcripts] would be difficult because it would need permission from the client. (Female, Survey Manager)
The impact of limited budgets for developing surveys was highlighted by all of the interviewees, even though this may be impacting on the overall accuracy of the data collected and therefore the understanding of public attitudes.
In the development and pretesting of survey questions, it was evident that there was only limited involvement of experts on immigration, refugee studies or critical race theory. Only one of the survey methodologists stated that such experts were part of the question development process and that this was dependent on costs: ‘Sometimes expert workshops are used and it depends on the type of questions, this could be academics, people from the government, immigration lawyers. But it also depends on the budget’ (Female, Survey Methodologist). The greater involvement of experts from a range of backgrounds in question design and testing alongside input from immigrants themselves, would clearly be of value in developing survey questions.
Survey Commissioning and Funding
A further issue in relation to funding and costs was the types of organisations that commission surveys. One of the commercial survey methodologists commented that, in his opinion, out of the three main types of clients they work for (media/journalists, academics and think tanks), academics and think tanks had more of an agenda. He commented: They all have an agenda obviously, but the media and journalists seem to be less worried about the agenda and the results as they have such a quick turnaround. They just say ‘right can you run me a poll on this’. Whereas academics and think tanks have more of an agenda. (Male, Commercial Survey Methodologist)
The point about academics and think tanks having more of an agenda may link to the in-depth approach of many academic studies and the political focus of many think tanks. Think tanks often aim to influence policy and can be funded by political organisations. However, it could also be that the agenda of the media is not as overtly evident, but more implicit in the overall framing of a particular survey or survey question. The commissioning and funding of surveys are clearly part of the challenge of understanding how surveys can capture public attitudes accurately.
Discussion and Conclusions
The research has highlighted a range of challenging issues about how public attitudes towards immigration and immigrants are captured. The findings show how some of the survey questions configured the relationship between immigration and race by presenting immigration as an essentially racialised phenomenon. The terms immigrant and asylum seeker were at times conflated, which oversimplifies people’s circumstances. This dehumanises immigrants by devaluing their diverse identities and experiences and can impact on the public’s understanding of their circumstances. Many of the survey questions contained negative framings by seeking attitudes regarding hypothetical situations, such as having immigrants as neighbours alongside attitudes towards problems such as alcohol and drug addiction. Another question gave primacy to the framing of a link between immigration and higher crime rates. This kind of discourse reflects colonial constructions of the Other.
The racialisation of immigration and the use of the terms such as ‘allow’ and ‘let into’ frames immigration in terms of racial hierarchical control. This narrative framing reflects Musolff’s (2015) argument that immigration discourses often aim to divide the ‘nation’. Discursive strategies such as the container metaphor and proximisation are powerful political tools; they can be used to legitimise restrictive immigration policies, which, it is argued disproportionately impact on ethnically and racially minoritised people (Liberty, 2018). Highlighting the use of these discursive strategies within survey questions is an important concern given that social surveys can be a key component of the policy-making process and in the justification of policy decisions.
Evidence from the interviews with the survey managers and methodologists highlighted some of the challenges when developing survey questions and the issue of negative framing. Budget and time constraints may be factors affecting the extent to which survey question pretesting is undertaken. Moreover, the results of any pretesting were not always easily available and the interviewees agreed that it could be difficult to provide this information publicly. Another constraint was the issue of measuring change over time and the need for consistent language, even when the meaning and usage of some of the language may have changed. It is clear that this could be at the cost of the more important issue of the accurate measurement of public attitudes, given the potential for certain words and phrasing to invoke particular social imaginaries of immigration (Blinder, 2015).
In considering the ways forward, there should be closer scrutiny of how surveys are developed and conducted, and also how findings are reported, including in the media. There must be greater involvement of experts from different backgrounds in the development and testing of survey questions. Moreover, the direct involvement of immigrants and people from diverse ethnic backgrounds as part of the development and testing of new survey questions should be standard practice. CDA-based approaches should also be part of the development of survey questions in order to ensure that they are examined in context. The issues raised by the ways in which the survey questions are framed, and the potential for power and ‘white’ norm framing to be reproduced should be part of good practice training for survey organisations, policy makers and the media. The development of survey questions and the pretesting stages should be made more transparent as part of the quality assurance process of a survey. New question framing and formats should be explored that do not simply replicate essentialised and negative narratives of identity and difference. For example, this could include asking respondents about their attitudes towards specific immigration policies rather than asking respondents about immigration in general. The challenges faced by many immigrants could also be highlighted in survey questions.
Surveys can be influential when used to inform and justify political decision making, but also in terms of informing public attitudes as part of a circular process. There is a need to review surveys’ fitness for purpose and the extent to which they may be leading rather than reflecting public attitudes and attitude change. In the context of evidence of increased levels of racial discrimination, a new discourse is required to more objectively measure and understand public attitudes towards immigration and immigrants.
Limitations
First, only a limited number of leading surveys were analysed and the international surveys were analysed in their English language versions. Second, only a small number of survey managers and methodologists were interviewed. However, all the interviewees had many years’ experience in developing, testing and analysing leading social surveys. Despite these limitations, the research makes a valuable contribution to the evidence base and ongoing debates in this important area.
Footnotes
Acknowledgements
The authors would like to thank Dr Aoileann Ní Mhurchú and the anonymous reviewers for providing feedback on earlier versions of the manuscript.
Funding
The authors received no financial support for the research, authorship and/or publication of this article.