We provide a systematic assessment of various methods to position political parties on immigration, a policy domain that does not necessarily overlap with left–right and is characterized by varying salience and issue complexity. Manual and automated coding methods drawing on 283 party manifestos are compared – manual sentence-by-sentence coding using a conventional codebook, manual coding using checklists, automated coding using Wordscores, Wordfish and keywords. We also use expert surveys and the Comparative Manifesto Project (CMP), covering the main parties in Austria, Belgium, France, Ireland, the Netherlands, Spain, Switzerland and the United Kingdom, between 1993 and 2013. We find high levels of consistency between expert positioning, manual sentence-by-sentence coding and manual checklist coding and poor or inconsistent results with the CMP, Wordscores, Wordfish and the dictionary approach. An often-neglected method – manual coding using checklists – offers resource efficiency with no loss in validity or reliability.
BakkerRVries CdeEdwardsE. (2015) Measuring party positions in Europe The Chapel Hill expert survey trend file, 1999–2010. Party Politics21(1): 143–152.
BenoitKLaverM (2006) Party Policy in Modern Democracies. London: Routledge.
4.
BenoitKLaverM (2007) Estimating party policy positions: comparing expert surveys and hand-coded content analysis. Electoral Studies26(1): 90–107.
5.
BenoitKConwayDLauderdaleBE. (2016) Crowd-sourced text analysis: reproducible and agile production of political data. American Political Science Review110(2): 278–295.
6.
BlackD (1948) On the rationale of group decision-making. Journal of Political Economy56 (1): 23–34.
7.
BudgeI (2000) Expert judgements of party policy positions: uses and limitations in political research. European Journal of Political Research37(1): 103–113.
8.
BudgeIFarlieDJ (1983) Explaining and Predicting Elections: Issue Effects and Party Strategies in Twenty-Three Democracies. London: George Allen & Unwin London.
9.
BudgeIKlingemannHVolkensA. (2001) Mapping Policy Preferences: Estimates for Parties, Electors, and Governments, 1945-1998. Oxford: Oxford University Press.
10.
BudgeIRobertsonDHearlD (1987) Ideology, strategy, and party change: spatial analyses of post-war election programmes in 19 democracies. Cambridge: Cambridge University Press.
11.
CastlesFMairP (1984) Left–right political scales: some ‘expert’ judgments. European Journal of Political Research12: 73–88.
12.
DäublerTBenoitKMikhaylovS. (2012) Natural sentences as valid units for coded political texts. British Journal of Political Science42(4): 937–951.
13.
DinasEGemenisK (2010) Measuring parties’ ideological positions with manifesto data a critical evaluation of the competing methods. Party Politics16(4): 427–450.
14.
EnelowJMHinichMJ (1984) The Spatial Theory of Voting: An Introduction. Cambridge University Press.
15.
Green-PedersenC (2007) The growing importance of issue competition: the changing nature of party competition in Western Europe. Political Studies55(3): 607–628.
16.
GrimmerJStewartBM (2013) Text as data: the promise and pitfalls of automatic content analysis methods for political texts. Political Analysis21(3): 267–297.
17.
GudbrandsenF (2010) Partisan influence on immigration: the case of Norway. Scandinavian Political Studies33(3): 248–270.
18.
HarmelRJandaKTanA (1995) Substance vs. packaging: an empirical analysis of parties’ issue profiles. In: Annual Meeting of the American Political Science Association. Available at: http://www.janda.org/bio/parties/papers/Harmel%20et%20al,%201995.pdf (accessed 11 October 2012).
19.
HoogheLBakkerRBrigevichA. (2010) Reliability and validity of the 2002 and 2006 Chapel Hill expert surveys on party positioning. European Journal of Political Research49(5): 687–703.
20.
HuberJInglehartR (1995) Expert interpretations of party space and party locations in 42 societies. Party Politics1(1): 73–111.
21.
KatzRSMairP (1992) Introduction: the cross-national study of party organizations. In: KatzRichardMairPeter (eds) Party Organizations: A Data Handbook on Party Organizations in Western Democracies 1960-90. London: Sage, pp. 1–20.
22.
KatzRSMairP (1994) The evolution of party organizations in Europe: the three faces of party organization. American Review of Politics14: 593–617.
23.
KlüverH (2009) Measuring interest group influence using quantitative text analysis. European Union Politics10(4): 535–549.
24.
KriesiHGrandeEDolezalM. (2012) Political Conflict in Western Europe. Cambridge: Cambridge University Press.
25.
KriesiHGrandeELachatR. (2006) Globalization and the transformation of the national political space: six European countries compared. European Journal of Political Research45(6): 921–956.
26.
LadnerASchwarzDFivazJ (2009) Die Schweizer Parteien im politischen Raum - eine Analyse der politischen Positionen ihrer Kandidierenden bei den Nationalratswahlen 2007. Working paper de l’IDHEAP, IDHEAP Lausanne2009(1).
27.
LaverM (2014) Measuring policy positions in political space. Annual Review of Political Science17(1): 207–223.
28.
LaverMBenoitKGarryJ (2003) Extracting policy positions from political texts using words as data. American Political Science Review97(2): 311–331.
29.
LoJProkschSOSlapinJB (2016) Ideological clarity in multiparty competition: A new measure and test using election manifestos. British Journal of Political Science46(3): 591–610. doi:10.1017/S0007123414000192.
30.
LoweW (2008) Understanding Wordscores. Political Analysis16(4): 356.
31.
LoweWBenoitK (2013) Validating estimates of latent traits from textual data using human judgment as a benchmark. Political Analysis21(3): 298–313.
32.
LubbersMGijsbertsMScheepersP (2002) Extreme right-wing voting in Western Europe. European Journal of Political Research41(3): 345–378.
33.
MarksGHoogheLSteenbergenM. (2007) Crossvalidating data on party positioning on European integration. Electoral Studies26(1): 23–38.
34.
McDonaldMDBudgeI (2014) Getting it (approximately) right (and center and left!): reliability and uncertainty estimates for the comparative manifesto data. Electoral Studies35: 67–77.
35.
McDonaldMDMendesSKimM (2007) Cross-temporal and cross-national comparisons of party left-right positions. Electoral Studies26(1): 62–75.
36.
OdmalmP (2012) Party competition and positions on immigration: strategic advantages and spatial locations. Comparative European Politics10(1): 1–22.
37.
PetrocikJR (1996) Issue ownership in presidential elections, with a 1980 case study. American Journal of Political Science40(3): 825–850.
38.
RabinowitzGMacdonaldSE (1989) A directional theory of issue voting. The American Political Science Review83: 93–121.
39.
RayL (2007) Validity of measured party positions on European integration: assumptions, approaches, and a comparison of alternative measures. Electoral Studies26(1): 11–22.
40.
RevelleW (2017) Psych: Procedures for Psychological, Psychometric, and Personality Research. Evanston: Northwestern University. Available at: http://CRAN.R-project.org/package=psych(accessed 18 May 2017).
41.
RohrschneiderRWhitefieldS (2012) The Strain of Representation: How Parties Represent Diverse Voters in Western and Eastern Europe. Oxford: Oxford University Press.
42.
RooduijnMPauwelsT (2011) Measuring populism: a comparison of two methods of content analysis. West European Politics34(6): 1272–1283.
43.
RovnyJ (2012) Who emphasizes and who blurs? Party strategies in multidimensional competition. European Union Politics13(2): 269–292.
44.
RovnyJ (2013) Where do radical right parties stand? Position blurring in multidimensional competition. European Political Science Review5(1): 1–26.
45.
RuedinD (2013) Obtaining party positions on immigration in Switzerland: comparing different methods. Swiss Political Science Review19(1): 84–105.
46.
Somer-TopcuZ (2015) Everything to everyone: the electoral consequences of the broad-appeal strategy in Europe. American Journal of Political Science59(4): 841–854.
47.
SteenbergenMRMarksG (2007) Evaluating expert judgments. European Journal of Political Research46(3): 347–366.
48.
Van der BrugWD’AmatoGBerkhoutJ. (eds) (2015) The Politicisation of Migration. Abingdon: Routledge.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
