Sage Journals: Discover world-class research

Abstract

For many, search engines are crucial gateways to (political) information. While extant research is concerned with algorithmic bias, user choices had been largely neglected. Yet, search queries are the key way in which searchers explicate their information need. Building on framing theory and selective exposure, we argue that queries are ingrained with (political) predispositions: issue frames in mind of searchers manifest themselves in search terms and queries. Using Dutch survey data (N = 1994), and manual coding and latent class analysis, we explore how types of people formulate search queries about immigration and climate change (RQ1). A regression analysis shows how these searcher types relate to political attitudes and socio-demographic characteristics (RQ2). Notably, searchers formulate queries in ways that are related to their political positions, but this differs for different issues. These findings imply systematic differences in user choices which future research needs to consider when auditing search engines.

Keywords

Personalisation polarisation political information search engines search queries user types

Introduction

A majority of people access news and (political) information through digital platforms, such as social media, search engines or news aggregators (Möller et al., 2019). Digital platforms allegedly afford the creation of echo chambers by allowing individuals to personalise their own information repertoire, which users are assumed to do according to their preferences (Sunstein, 2001). Some fear that this tendency is reinforced by algorithmic filtering which pick up on these explicit signals and trap people into filter bubbles where all attitude-challenging information is filtered out (Pariser, 2011). Some argue that citizens’ reliance on digital platforms is detrimental to the ideals of a deliberative democracy, which depends on a citizenry that is exposed to a wide range of news and perspectives to aid political decision-making (Helberger et al., 2018). Information environments that echo one’s own beliefs could potentially lead to increased polarisation (Stroud, 2010).

This article examines political search engine use, which has received relatively limited scholarly attention compared to social media. This is an important gap in the literature considering that search engines play a deciding role in shaping our political attitudes and decisions. Search engines are important gateways to news (Möller et al., 2019) and people use them to inform themselves on political events (Arendt and Fawzi, 2019). Moreover, they have the (misleading) reputation of being a reliable and unbiased source of information (Haas and Unkel, 2017; Pan et al., 2007) while evidence suggests that search engines are not content neutral (Courtois et al., 2018) and their results can significantly impact voter preferences (Epstein and Robertson, 2015).

Prior work on political search engine use has attempted to find filter bubble effects in search results (e.g. Puschmann, 2019; Robertson et al., 2018). A much overlooked, yet important, factor in political search engine use are search queries. Search engines offer the most ‘relevant’ web content in response to search queries while also taking into account other implicit factors (e.g. location, search history; Hannak et al., 2013). Consequently, queries are the key way in which users express their information need. While much concern exists about the consequences of consuming political information and news online, we know little about how people search for political information in the first place. Therefore, we ask how types of people search for political information on search engines in terms of the search queries chosen (RQ1), and how these types relate to political and socio-demographic characteristics (RQ2).

Building on literature from framing theory and selectivity in political information, we argue that individuals do not formulate search queries in neutral or generic ways, but they are ingrained with the searcher’s prior experiences, interests and beliefs. While information search is likely also impacted by situational factors, we focus on the role of user characteristics. We assess our questions in the Netherlands using survey data collected among a representative sample of the population. Specifically, we contrast search queries about immigration, a persistent polarising issue, with queries about climate change, a newer and less dividing issue. Most existing research focused on a bipolar partisan context (e.g. Arendt and Fawzi, 2019; Trielli and Diakopoulos, 2020), which do not always translate well to the European context that is more issue-focused.

We combine qualitative and quantitative coding, and latent class analysis (LCA), to identify searcher types. We then regress these types on issue attitudes, issue importance, political orientation and political interest as well as socio-demographic control variables to understand whether and how people search for political information in systematically different ways.

Theoretical framework and related research

Depending on their political predispositions, users self-select into information environments that follow a particular frame (here understood as an individual’s understanding of a situation; Lecheler and de Vreese, 2018) when they search for information online. For example, Figure 1 depicts the Google Search results for two immigration-related search queries: ‘‘immigrants’’ and ‘‘refugees’’. The first is a generic term used across different perspectives while the latter reflects a so-called ‘humanitarian’ frame where immigrants are viewed as victims (Urbániková and Tkaczyk, 2020). Because the search results are different, the seemingly minor difference in search query choice exposes the searcher to very different information. Although search engine algorithms curate search results, user decisions impact the search results to a large extent.

Figure 1.

Google Search results pages for two search queries: ‘‘immigrants’’ and ‘‘refugees’’.

Research exploring political search query choice is scarce, but some first evidence suggests that search queries reflect political preferences. For example, Borra and Weber (2012) predict the political orientation of websites with the search queries that led there, indicating that the queries themselves exhibit a political preference. Recent work by Trielli and Diakopoulos (2020) show that there are topical variations in search queries used by Democrats and Republicans to find information on their own political candidates and those from the other side during the US midterm election. Slechten et al. (2021) studied how selectivity impacts various steps of online search about political topics and found that people tend to formulate attitude-consistent search queries.

Two areas of research can help us understand the relationship between search query choice and political characteristics: framing theory and selective exposure.

Frames in search queries

Although framing theory is predominantly used to understand effects of mass-mediated communication, it also provides a useful theoretical framework to study why search engine users self-select through search queries into different information environments. There is ample evidence from framing research that people internalise issue frames from the media (for an overview, see Lecheler and de Vreese, 2018). In short, issue frames in the media communicate which aspects of an issue are deemed relevant and how one should express themselves when thinking about or discussing this issue. This process affects how individuals understand an issue: a ‘frame in thought’ or ‘audience frame’ (Druckman, 2001).

Framing research is mainly focused on the affective, attitudinal and behavioural consequences of exposure to frames (Lecheler and de Vreese, 2018). Less work attempted to study frames in thought directly. Yet, Price et al. (1997) found that exposure to different news frames affected the substantive thoughts people listed afterwards. Another study found that two seemingly similar terms – ‘‘global warming’’ and ‘‘climate change’’ – evoke different topical associations among the general public as well as between partisans in the United States (Schuldt and Roh, 2014). Others who use similar approaches – exposure to a frame and measuring frames in thought with open-ended survey questions – come to similar conclusions for other issue areas: the way an issue is portrayed is able to focus peoples’ thoughts on particular aspects of the issue (Brewer and Gross, 2005; de Vreese, 2004).

This study contributes to framing literature by investigating a different consequence of issue frames: how issue frames in mind affect information searches. Specifically, how frames manifest themselves in one’s choice for search queries. Framing researchers distinguish between ‘emphasis’ and ‘equivalency frames’ (Druckman, 2001). Emphasis frames refer to the selection of one set of information over others, while equivalency frames are concerned with the differential portrayal of equivalent information. When it comes to the manifestation of frames in thought in search queries, both types of frames matter: depending on the issue frame in mind, searchers are more inclined to search for particular topics over others, and this is intertwined with the terms they use to describe their topic (e.g. ‘‘immigrant’’ vs. ‘‘refugee’’).

Although beyond the scope of this article, it is worth noting that frames in media content and search results are interconnected. Frames in mind reflect media frames to a certain degree (Brewer and Gross, 2005; de Vreese, 2004). These frames likely manifest themselves into search queries, and subsequently, search engines are expected to return information reflecting similar frames.

Selectivity in search queries

Selective exposure research helps us understand how people are guided by these frames in information searches. Much empirical evidence supports the claim that people tend to seek out attitude-congruent information because people generally prefer exposure to information supporting their own position (e.g. Knobloch-Westerwick et al., 2015; Stroud, 2008). Emerging from Festinger’s (1957) theory of cognitive dissonance, the driving force behind this tendency is often argued to be motivated reasoning (Taber and Lodge, 2006). The general premise of motivated reasoning is that people, next to forming accurate opinions, are also motivated to defend their existing beliefs. This manifests itself in a biased approach to new information: attitude-congruent information tends to be evaluated as stronger and more compelling than attitude-challenging information (prior attitude effect), attitude-challenging information is more likely to be approached with counterargument than attitude-congruent information (disconfirmation bias), and people are inclined to self-select themselves into confirming rather than challenging information when given the choice (confirmation bias) (Taber and Lodge, 2006).

Research on political selectivity is mainly focused on the selection of content. For instance, the selection of news outlets and websites reveals partisan tendencies (e.g. Garrett, 2009). Regarding search engines, evidence shows that people read and select search results that support their own issue position (Knobloch-Westerwick et al., 2015; Westerwick et al., 2013).

The claim that people also selectively avoid challenging information is, however, contested (e.g. Garrett, 2009).

In conclusion, despite reporting an interest in diversity (Bodó et al., 2019), research shows that in practice people tend to shape their information environment in ways that align with their views and interests. Search engines in particular offer more opportunity for self-selection because one has to actively search for information by entering queries, which can also be seen as the step before selecting information. Given this level of user agency, we expect search queries to reflect similar tendencies people exhibit when selecting information: a search query expresses which aspects of a political issue in the searcher’s frame of mind are considered relevant.

Antecedents of search queries

We expect search queries on a given political issue to group together in a specific way and form typical patterns of search query behaviour. These clusters (i.e. searcher types) represent the issue frame expressed in the queries. We consider several individual-level factors to shape search queries about political issues: issue attitudes, issue importance, general political orientation and political interest.

Based on the assumption that political beliefs, preferences and interests will drive how users search for political issues, we expect issue-specific attitudes to impact search query formulation. We also explore the role of political preference in a more general sense: political orientation on a left–right scale. Political orientation is a reflection of citizen’s position on various economic and cultural issues (Van der Brug and Van Spanje, 2009). Issue-specific attitudes and political orientation, in general, are considered relevant factors in political information-seeking (e.g. Knobloch-Westerwick et al., 2015; Westerwick et al., 2013) and research on issue framing (for an overview, see Lecheler and de Vreese, 2018).

Furthermore, we expect personal issue importance moderates the relationship between issue-specific attitudes and search query choice, sometimes conceptualised as issue publics (e.g. Brenes Peralta et al., 2017). Individuals are more likely to seek out attitude-consistent political information with more issue importance, given that motivated reasoning is heightened for those who attach much importance to the issue (Brenes Peralta et al., 2017; Knobloch-Westerwick et al., 2015; Westerwick et al., 2013). Translated to search queries, we question whether highly engaged individuals are driven stronger by their issue position in search than those less engaged individuals.

Finally, political interest is considered an important factor for engagement with news and political information, which may also be true for online search. Given that politically interested individuals access news more often via search engines (Möller et al., 2019), one explanation could be that politically interested users are better equipped to search for political issues because they have more knowledge or are more likely to have done so in the past.

Issue polarisation in the Netherlands

Research on political information and news use is predominantly focused on bipolar partisan contexts, usually the United States (e.g. online search for political; candidates Arendt and Fawzi, 2019; Trielli and Diakopoulos, 2020), whereas in multi-party systems citizens tend to align themselves on political issues rather than in two ideological blocks (Van der Brug and Van Spanje, 2009). This context therefore calls for an issue-specific approach.

We assess our questions in the context of the Dutch immigration and climate change issues. Immigration is a persistent issue along which the Dutch public is divided, which has been on the political agenda since the 1990s (Boomgaarden and Vliegenthart, 2007). Climate change is a more recent political issue. Dutch public opinion on climate change is not characterised by an opposition to climate change (i.e. climate change deniers), such as in the United States. Research on Dutch audience segments found that most are medium engaged and concerned with the climate, with relatively few people on either sides of the spectrum (Wonneberger et al., 2020). Consequently, citizens likely have formed stronger opinions and corresponding ‘frames of thought’ on immigration than climate change, which may impact information search to different degrees.

Data and methods

We collected survey data among a sample of the Dutch adult population (N = 1994) (Araujo et al., 2020).

Nationally representative quotas were set on age, region and gender interlocked with education. The achieved distribution closely match that of the national population. The data and its supplementary information,¹ and code and other materials,² are publicly available.

Search queries

Respondents were asked for three search queries about immigration and climate change (random order).³ Following pretest results, we formulated the question as open as possible to minimise its potential impact on the respondent’s language (e.g. avoiding the term ‘‘immigration’’), multiple queries were asked to go beyond generic queries and maximise variation, and the Google Search logo was placed above the text fields to resemble a search engine setting.⁴

Search query indicators

Search queries are short phrases, typically only a few keywords. Unfortunately, many (unsupervised) text analysis techniques do not perform well on short texts due to sparsity and ‘noise’ issues. We therefore group the queries into search query indicators.

The following steps were preformed for each issue separately. First, we preprocessed all unique search queries by lowercasing the text and removing punctuation (n_immigration = 2, 734; n_climate = 2,671). Then, all authors sorted the same subsample of 100 search queries inductively into categories. We quickly reached consensus on the final categorisation, which consists of categories related to search queries’ (sub)topic, terminology, format and effectiveness (see Supplemental Table A1 for descriptions; Appendix B for details). Multiple indicators can apply to the same search query. We also flagged invalid entries (i.e. unrelated to the topic or not a search query). Based on these bottom-up categories, the first author developed codebooks that allowed for top-down coding. An inter-coder reliability test of a 10% subsample with an additional coder revealed that sufficient agreement is reached on nearly all indicators (see Supplemental Table A2). The categorisation is then applied to the full data set. Because the qualitative inspection of the immigration-related queries revealed various terminologies to refer to immigrants, we also extracted commonly used terms using a dictionary-based approach.

With this approach, we aimed at uniting the ‘best of both worlds’: the openness of a bottom-up, qualitative analysis to generate our categories, and the generalisability and precision of a top-down, quantitative approach.

Note that we aggregate indicators to the respondent level (i.e. present in at least one of three queries) because we are interested in individual differences. The frequency of search query indicators are presented in Supplemental Figures A1 and A2.

Latent class analysis

The indicators serve as input for LCA which identifies underlying patterns in search queries that might not become visible by considering the indicators separately, and categorise users accordingly. The first step is class enumeration, in which we decide upon the appropriate number of classes. For each issue separately, we iteratively ran an LCA for an increasing number of classes. There is no agreement on a single fit statistic for LCA, and we therefore consult several fit indices which have been reported to perform well for class enumeration (Nylund et al., 2007) and Lo–Mendell–Rubin likelihood ratio tests (LMR-LRT) to compare nested models. Furthermore, we inspect the class-conditional item probabilities (i.e. the probability of each indicator to occur in each class) to interpret what characterises the queries used by the members of each class. The final number of classes is based on a combination of statistical fit and substantial interpretation. Finally, we assign each respondent to their predicted class, that is, searcher type.

Predictors

Next, we assess whether searcher characteristics predict searcher type membership (see Table 1 for descriptive statistics).

Table 1.

Descriptive statistics of predictors.

	Immigration		Climate change
	M /proportion	SD	M /proportion	SD
n	1,483		1,595
Immigration attitudes	4.17	1.51
Climate attitudes			4.85	1.51
Issue importance	3.31	0.97	3.88	0.97
Political orientation	5.11	2.30	5.09	2.29
Political interest	5.42	2.41	5.36	2.43
Education
Low	0.06		0.07
Medium	0.49		0.50
High	0.45		0.43
Age	46.58	14.87	46.40	14.87
Male (ref. = female)	0.51		0.51

SD: standard deviation.

Immigration attitudes were measured by asking whether people who come to live in the Netherlands from other countries (1) ‘‘make the Netherlands a worse or better place to live’’, (2) ‘‘is generally bad or good for the Dutch economy’’ and (3) ‘‘generally undermined or enriched the Netherlands’ cultural life’’ (strongly disagree to strongly agree, 7-point scale). We calculated the mean score on these items (α = .84), so that higher scores indicate more positive feelings towards immigration.

Climate attitudes were measured with two items: ‘‘Most statements about climate change are exaggerated’’ and ‘‘Stopping climate change is more important than economic growth’’ (strongly disagree to strongly agree, 7-point scale). We reversed the first statement’s scale and used the mean score (α = .71), so that higher scores indicate pro-climate attitudes.

Issue importance was measured with the personal importance attached to the immigration and climate issues (very unimportant to very important, 5-point scale).

Political orientation was measured by a self-placement question on a scale ranging from 0 (left) to 10 (right). 113 respondents who declined to answer are considered missing.

Political interest was measured on a scale ranging from 0 (not at all interested) to 10 (very interested). 10 respondents who did not answer this question are considered missing.

We used separate binary logistic regression models for each searcher type, comparing each type against all other types. A multinomial logistic regression is inappropriate considering the LCA results show that some classes are more alike than others, hence violating the Independence of Irrelevant Alternatives assumption.

An alternative for predicting class membership is the ‘one-step’ approach which estimates relationships between predictors and latent classes simultaneously with the LCA model. We separate these because we view classification and prediction as two substantially different analytical steps, and inclusion of predictors may distort the latent classification (Bakk et al., 2013). Our ‘three-step’ approach leads the predictors of regression analyses to potentially be downward-biased (Bakk et al., 2013). Therefore, the strength of the relationships should be seen as a lower bound, with the true strength probably being higher.

Sample selection

We selected respondents with three valid search queries to maintain three observations per respondent (n_immigration = 1,607; n_climate = 1,729).⁵ For the regression analyses, we further excluded respondents with missing values on the predictors (n_immigration = 1,483; n_climate = 1,595). The samples are similar regarding socio-demographic characteristics (see Table 1).

Results

For reasons of conciseness, we first describe the LCA class enumeration and regression analyses results, after which we discuss the type of queries and political and socio-demographic profiles of each searcher type. We do so for each issue separately.

Immigration

Identifying searcher types

Figure 2 visualises the development of the fit across models with an increasing number of classes (for detailed statistics, see Supplemental Table A3). The inflection points indicate the four- or five-class solution as appropriate solutions. But the LMR-LRT suggests that the fit increases significantly with more classes (see Supplemental Table A3). Because the four- and five-class solution both produce essentially the same three classes, but the latter additionally splits the fourth class in two distinct and theoretically meaningful types, we chose the latter. Based on the item probabilities (see Figure 3; see Supplemental Table A4 for detailed estimates), we label the five searcher types: Statistics and Facts, Refugees, Cultural Impact, Economic Impact and Asylum.

Figure 2.

Fit indices across immigration LCA solutions.

Figure 3.

Distribution of indicators across immigration searcher types (n = 1,607).

Predictors of searcher types

In the following binary logistic regression results, we explore who is likely to belong to which type in terms of their political and socio-demographic profile.

Immigration attitudes significantly and negatively predict the Asylum type, and significantly and positively predict the Refugees type, as compared to all other types (see Table 2, M1). In other words, the searcher types differ in their immigration attitudes, where the Refugees type is the most positive, followed by Statistics and Facts, Economic Impact and Cultural Impact, and Asylum as least positive.

Table 2.

M1, M3: Results of binary logistic regressions predicting immigration searcher type (n = 1,483).

	Asylum		Statistics & Facts		Cultural Impact		Refugees		Economic Impact
	M1	M3	M1	M3	M1	M3	M1	M3	M1	M3
Immigration attitudes	0.852*** (0.037)	0.883** (0.042)	1.073(0.055)	1.091(0.051)	0.980(0.045)	1.016(0.051)	1.129** (0.048)	1.052(0.049)	0.996(0.044)	0.996(0.048)
Issue importance	1.115(0.078)	1.106(0.078)	1.000(0.082)	0.999(0.081)	1.009(0.073)	1.003(0.072)	0.950(0.063)	0.958(0.064)	0.930(0.065)	0.930(0.065)
Political orientation		1.060(0.034)		1.026(0.036)		1.063(0.035)		0.892*** (0.026)		1.001(0.031)
Political interest	0.994(0.030)	0.989(0.029)	1.033(0.035)	1.033(0.035)	0.930* (0.028)	0.926* (0.028)	1.036(0.029)	1.040(0.029)	1.007(0.030)	1.007(0.030)
Education (ref. = low)
Medium	1.353(0.374)	1.363(0.377)	4.157* (2.489)	4.183* (2.505)	0.624(0.157)	0.630(0.158)	1.150(0.301)	1.120(0.294)	0.648(0.165)	0.648(0.165)
High	1.166(0.334)	1.175(0.336)	6.500** (3.895)	6.538** (3.917)	0.604(0.158)	0.609(0.159)	0.894(0.241)	0.868(0.235)	0.710(0.186)	0.710(0.186)
Age	1.009(0.005)	1.009(0.005)	0.989* (0.005)	0.990* (0.005)	1.002(0.005)	1.003(0.005)	1.007(0.004)	1.006(0.004)	0.990* (0.004)	0.990* (0.004)
Male (ref. = female)	1.190(0.164)	1.155(0.161)	0.947(0.147)	0.932(0.146)	1.639*** (0.238)	1.589** (0.233)	0.671** (0.087)	0.713** (0.093)	0.886(0.122)	0.885(0.123)
Constant	0.177*** (0.079)	0.116***	0.039*** (0.028)	0.032***	0.380* (0.167)	0.243**	0.161*** (0.068)	0.379*	0.761(0.320)	0.757
Log-likelihood	−742.16	−740.45	−623.94	−623.68	−696.7	−694.98	−816.63	−809.05	−746.98	−746.98
Pseudo R2	0.0323	0.0358	0.0442	0.0448	0.025	0.0287	0.024	0.0388	0.0111	0.0111

Estimates presented as OR. Standard errors in parentheses.

p < .1; ** p < .05; *** p < .01.

Furthermore, issue importance does not significantly predict searcher types, all else being equal, compared to all other types. Furthermore, the interaction between issue attitudes and issue importance also does not significantly predict belonging to a particular searcher type over others (see Supplemental Table A5). Given these insignificant results, it is unsurprising that adding this interaction to the model does not significantly improve their fits in log-likelihood ratio tests (with results ranging between χ²(1) = .003, p = .956 and χ²(1) = 2.680, p = .102).

In M3 (see Table 2), we add political orientation to the model which improves the model fit significantly for the Asylum (χ²(1) = 3.423, p = .0643), Cultural Impact (χ²(1) = 3.442, p = .064) and Refugees models (χ²(1) = 15.165, p = .000). Political orientation significantly explains membership to the Refugees type, controlled for immigration attitudes and all other variables in the model. Right-wing searchers are less likely to belong to the Refugees type. In fact, the effect of immigration attitudes is insignificant in this model, which suggests that general political orientation captures the effects one’s opinion on immigration has on their type of search queries on this topic. In contrast, the Asylum type remains significantly associated with negativity towards immigration, controlled for political orientation and all other variables in the model.

Finally, the Cultural Impact type is significantly less politically interested than the other types. Higher educational attainment and being younger are significantly associated with membership to the Statistics and Facts type. The Economic Impact type is also significantly younger than all other types. Female users are more likely to belong to Refugees whereas male users are more likely to belong to Cultural Impact.

Discussion of searcher types

Based on the item probabilities and regression results, we can characterise the searcher types in terms of their queries used and political profile.

Statistics and Facts (17.6%)

This type mainly formulates search queries containing neutral terms and about characteristics of the immigration process or the immigrants themselves, but, unlike other types, do not exhibit a search preference for topics related to the consequences or causes of immigration. It is also worth noting that this type enters search queries about data and statistics. While this finding could indicate that this group is concerned about the number of immigrants in the Netherlands, it is more likely that this group did not have a specific information need which led them to search for ‘facts’, given their positive stance towards immigrants. This group is also considerably higher educated than all other types, which could provide another explanation for this finding.

Refugees (21.1%)

This group explicitly does not use terms with negative connotations and all respondents in this class used ‘‘vluchteling’’ (refugee) in at least one query. This term is mainly used for the recent refugees that are part of the so-called European refugee crisis which started around 2015. They also search for the underlying reasons for migration (causes). In this sense, this type aligns with what prior work on immigration news framing has identified as the ‘victimization’ (Bos et al., 2016) or ‘humanitarian’ frame (Greussing and Boomgaarden, 2017) which portrays migrants as war victims and is generally focused on refugees’ motivations and backgrounds. This search preference is reflected in their political profile. This type consists of people who are most positive towards immigration.

Moreover, the regression results point out that this type is explained not solely by their positivity towards immigration, but also by their political leaning in general.

The other searcher types are characterised by queries about the perceived consequences of immigration and their use of negative terms.

Cultural Impact (18.8%)

The members of this type are formulating queries about the impact of immigration on national and cultural values (i.e. integration, culture, and religion and racism). They also use more negative terms (e.g. ‘‘buitenlander’’ (foreigner) and ‘‘allochtoon’’ (foreigner)).

Economic Impact (23%)

This type’s queries are about the impact of immigration on the economy, such as the labour and housing market. They use more negative terms, but also the generic term ‘‘immigrant’’. This is also the largest group in our classification.

The distinction between a cultural and economic type relates to findings in research on immigration attitudes and frames. Cultural and economic concerns are found to be drivers of anti-immigration attitudes (Schneider, 2007) and are present in frames in the Dutch public debate on immigration (Roggeband and Vliegenthart, 2007).

Asylum (19.5%)

Those belonging to this type search for information on accommodation centres, crime and integration as well as use ‘asielzoeker’ (asylum seeker) over other terms. These factors characterise them as concerned about the consequences of incoming asylum applications rather than the immigrants themselves and their background. In fact, this group feels most negative towards immigration, which cannot be explained by their general political orientation, which suggests that their feelings about immigration are separate from their general political stance. Whereas for other types, such as Economic and Cultural Impact, the effects of immigration attitudes are explained by general political orientation. This is unsurprising because these searchers do not only exhibit an anti-immigration gradient, but also connect immigration to other themes that are likely captured by one’s place on the left–right scale, such as national values or economic concerns.

Climate change

Identifying searcher types

The inflection points in Figure 4 (details in Supplemental Table A6) indicate a solution between four to six classes, but the LMR-LRT suggests an improving fit up to nine classes. While two types stay stable across solutions, the other two are split into multiple classes that cannot be interpreted in a meaningful way. Hence, the four-class solution is both statistically and theoretically an appropriate model. This solution generates four types that are distinct in the search queries used (see Figure 5 for item probabilities; detailed estimates in Supplemental Table A7): Solutions, Politics and Information, Consequences and Factors and Actors.

Figure 4.

Fit indices across climate LCA solutions.

Figure 5.

Distribution of indicators across climate searcher types (n = 1,729).

Predictors of searcher types

Next, we explore who belongs to which type based on their political and socio-demographic profiles. All else being equal, climate change attitudes does not significantly predict membership to a particular searcher type over the others (see Table 3, M1). In other words, a pro- or anti-climate stance does not explain the type of search queries one uses to search for more information on this topic. General political orientation also does not significantly explain membership to the any of the types in particular (see Supplemental Table A8) and does not result in significant improvement of the model fits (with results ranging between χ²(1) = .358, p = .550 and χ²(1) = 1.227, p = .268).

Table 3.

M1, M2: Results of binary logistic regressions predicting climate searcher type (n = 1, 595).

	Solutions		Politics & Information		Consequences		Actors & Factors
	M1	M2	M1	M2	M1	M2	M1	M2
Climate attitudes	1.060(0.078)	1.176(0.255)	1.005(0.051)	0.846(0.114)	1.021(0.050)	1.328* (0.184)	0.920(0.056)	0.786(0.129)
Issue importance	0.996(0.114)	1.111(0.272)	0.912(0.071)	0.766(0.113)	1.070(0.082)	1.396* (0.214)	1.038(0.099)	0.895(0.154)
Climate attitudes* Issue importance		0.974(0.051)		1.046(0.034)		0.945* (0.031)		1.041(0.040)
Political interest	0.993(0.034)	0.996(0.034)	0.990(0.024)	0.985(0.024)	0.986(0.023)	0.992(0.023)	1.043(0.031)	1.037(0.031)
Education (ref. = low)
Medium	1.100(0.393)	1.093(0.391)	0.814(0.180)	0.822(0.182)	1.301(0.286)	1.280(0.282)	0.853(0.224)	0.862(0.227)
High	1.221(0.447)	1.291(0.446)	0.834(0.191)	0.834(0.191)	1.144(0.261)	1.141(0.261)	0.946(0.258)	0.949(0.259)
Age	0.988* (0.005)	0.988* (0.005)	0.983*** (0.004)	0.983*** (0.004)	1.014*** (0.004)	1.015*** (0.004)	1.013** (0.005)	1.013** (0.005)
Male (ref. = female)	0.484*** (0.083)	0.486*** (0.083)	1.106(0.128)	1.099(0.127)	1.160(0.129)	1.170(0.130)	1.149(0.162)	1.142(0.162)
Constant	0.227** (0.117)	0.194(0.146)	1.743(0.594)	3.374* (1.978)	0.176*** (0.060)	0.064*** (0.039)	0.123*** (0.052)	0.217* (0.151)
Log-likelihood	−575.52	−575.39	−993.16	−992.21	−1043.4	−1041.3	−733.08	−732.55
Pseudo R2	0.0444	0.0447	0.0214	0.0231	0.0216	0.0251	0.0195	0.0205

Estimates presented as OR. Standard errors in parentheses.

p < .1; ** p < .05; ***p < .01.

Furthermore, issue importance does not significantly predict belonging to a particular type over others (see Table 3, M1). However, adding an interaction effect between climate change attitudes and issue importance to the model (see Table 3, M2) reveals that pro-climate attitudes in combination with low issue importance is significantly associated with belonging to the Consequences type. The chance to belong to this type decreases as those with pro-climate attitudes regard climate change as more important. Consequently, adding this interaction term improves the model fit significantly only for the Consequences model (χ²(1) = 4.185, p = .041).

Finally, political interest and education do not significantly explain membership to any type. Age, however, is a significant predictor of searcher type. Belonging to the solutions and Politics and Information types is characterised by being younger, whereas the chance to belong to the Consequences and Factors and Actors types increases with age. Finally, membership to the Solutions type is significantly associated with being female.

Discussion of searcher types

Based on the prior results, we can interpret the types in terms of their queries used and political profile.

Consequences (34.3%)

This type’s members mainly search for consequences of climate change, and rarely search for potential solutions, individual action, politics and policy or relevant actors. Notably, regarding the climate as a higher priority (combined with a pro-climate attitude) is related to search queries about the topics other than consequences of climate change. It seems probable the latter is more generic and will come to mind for those who do not regard it of high priority.

Politics and Information (33.5%)

This type enters queries about politics and policy as well as general climate change information (e.g. statistical or research related). Notably, they rarely search for solutions or individual behaviour, suggesting that they view politics as responsible rather than themselves. This type is younger rather than older.

Factors and Actors (20.2%)

This type formulate queries about relevant actors (i.e. agriculture, industry, transport) and factors (i.e. carbon and nitrogen emissions), and expand climate change by searching for other environmental problems. They also have an above average probability for search queries about (renewable) energy and individual behaviour, which often go hand-in-hand. The discussion about renewable energy sources is an established environmental discussion in the Netherlands, and considering solar panels in your home is relatively normal, even for those relatively unconcerned about climate change.

Solutions (12.1%)

This type mainly formulate search queries about potential solutions to climate change, including actions they can take themselves (e.g. ‘‘eating less meat’’). They are also likely to be female and young.

Discussion and conclusion

In this article, we explored how types of people formulate search queries about political issues (RQ1) and how these types relate to political and socio-demographic characteristics (RQ2). Our analyses demonstrate that people use a large variety of search queries to search about one political issue. Search engine users formulate queries about different subtopics and, depending on the issue, use different terms to describe it. Most notably, we show that people use search queries in systematically different ways. The variation in search queries corresponds to distinct searcher types which are related to political and socio-demographic characteristics to some extent, depending on the issue. For immigration-related searches, issue attitudes and political orientation mattered, as well as education, gender and age. For climate-related searches, issue importance is a more relevant factor in explaining search queries; issue attitudes and general political orientation did not matter. Age and gender also impact climate-related searches. The Dutch immigration debate is more polarised than the climate debate. It is therefore expected that issue attitudes are more relevant in the prior issue rather than the latter. Finding a relationship for two political issues that are clearly different, strengthens the notion that individuals search for political information in systematically different ways that are associated with their political position (i.e. issue stance or perceived importance).

Yet, we should note that this is a case study which affects the generalisability of our findings. The specific types and the extent to which and how these relate to political characteristics will be impacted by the issue and context. Future work should explore other issues and cross-national comparisons (e.g. across political systems).

Our study has important implications for the research on framing and selectivity. While framing research has studied how issue frames affect subsequent behaviour, we contribute to this work by studying how issue frames manifest themselves in information searches, which (to our knowledge) is rarely considered in this work. For immigration, both word choice and topics in search queries correspond with one’s political perspective, which speaks to the research on equivalency and emphasis frames (Druckman, 2001). While prior work found equivalency framing of climate change in the United States (Schuldt and Roh, 2014), our findings relate only to emphasis framing. This may be a consequence of how public opinion on climate change is structured differently in these contexts.

Research on selectivity finds that people tend to self-select themselves into attitude-confirming information when given the choice. We extended this argument from the selection of information to the formulation of search queries, an activity that particularly allows for selective tendencies. Our findings demonstrate that it is also important to consider this, especially when individuals increasingly rely on search engines for information.

Our findings also have important implications for research on search results, particularly work on filter bubbles. A systematic difference in user-input likely affects the search results, perhaps even more than algorithms. Consequently, users may self-select themselves into information environments that echo their own perspective. Although Trielli and Diakopoulos (2020) suggest that variation in search queries is not enough to yield substantially different search results, this earlier work is set in a bipartisan context and examines queries about political candidates during election times. This setting may contribute to a mainstreaming effect due to Google’s (n.d) prioritisation of recency and trusted sources for news-related queries. More research is needed to draw conclusions.

Other work auditing search engine algorithms have found little evidence for algorithm-induced filter bubbles by entering the same search queries with different settings (e.g. Haim et al., 2017; Puschmann, 2019; Robertson et al., 2018). We show that these studies may underestimate user-driven effects because this approach neglects that people with different information needs already explicate this in their queries.

This creates a complex picture of how users and algorithms co-create information environments. If filter bubbles exist, they are likely produced in the interaction between user choices and personalisation algorithms. Therefore, future work should consider both user choices and the system that responds to it when auditing algorithmically curated media (see also Slechten et al., 2021). Specifically, we need search engine audits that study the main and conditional effects of what is searched for and algorithmic personalisation factors (e.g. search history, browsing history, location) on search results, which is a methodologically challenging endeavour.

Furthermore, this study has some limitations regarding scope and method. Our analytical strategy is one reasonable way to analyse these data but one can also imagine other appropriate approaches as well as other predictors (e.g. literacy). Similar to Trielli and Diakopoulos (2020), we utilised survey data to gauge search query use rather than real-life search data. While search data comes with sparsity issues and privacy concerns, survey data allows for prompting users with searches for specific topics, connecting these easily to background characteristics, and isolating query formulation from the influence of search engine features (e.g. autocomplete, search suggestions). The search queries obtained in the survey seem to resemble search queries (details in Supplemental Appendix B) but using survey data comes at the expense of some ecological validity: it cannot account for the potential influence of search engine features nor can it reflect real iterative search behaviour. We also cannot account for situational factors, such as information exposure elsewhere (e.g. TV, social media) or important events that impact individuals’ information need, which is likely how online search is initiated. Yet, we find clusters of search queries that are connected, to some extent, to political views, which suggests that a component can be explained by user characteristics.

Future work should explore the following opportunities to increase ecological validity in research on (political) search queries: addressing the effects of both situational factors and characteristics, validating search queries from survey data with search histories, and exploring the use of individual search histories.

In conclusion, our findings demonstrate that it matters to consider the choices users make online because they make them in accordance with their (political) predispositions. Exploring what role users play on algorithmically curated platforms is an essential puzzle to understand as we move more of our information consumption online.

Supplemental Material

sj-pdf-1-nms-10.1177_14614448221104405 – Supplemental material for Searching differently? How political attitudes impact search queries about political issues

Supplemental material, sj-pdf-1-nms-10.1177_14614448221104405 for Searching differently? How political attitudes impact search queries about political issues by Marieke van Hoof, Corine S Meppelink, Judith Moeller and Damian Trilling in New Media & Society

Footnotes

Acknowledgements

We would like to thank the two anonymous reviewers for their constructive feedback, as well as Tasha de Vries for her assistance with manual coding.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the Amsterdam School of Communication Research, and by the RPA Communication and its Digital Communication Methods Lab. This work was also supported by the Dutch Research Council (NWO) grant nr. VI.Veni.191S.089 awarded to Judith Moeller.

ORCID iDs

Marieke van Hoof

Damian Trilling

Data availability statement

The data that support the findings of this study are openly available in the Open Science Framework at .

Supplemental material

Supplemental material for this article is available online.

Notes

Author biographies

Marieke van Hoof is a PhD Student in Political Communication and Journalism in the Department of Communication Science at the University of Amsterdam. Her research focuses on the interplay between individual characteristics, user choices and algorithmic selection in the digital media environment, particularly on search engines.

Corine S. Meppelink (PhD, 2016) is an Assistant Professor of Persuasive and Health Communication in the Department of Communication Science at the University of Amsterdam. Her research focuses on digital literacy skills, particularly in the health domain, and how inequalities in this respect impact the processing of and consequences of online (mis)information.

Judith Moeller (PhD, 2013) is an Associate Professor of Political Communication and Journalism in the Department of Communication at the University of Amsterdam. In her research, she focuses on the effects of political communication, in particular, social media and digital media. She was awarded a VENI talent track grant by the Dutch Science Foundation (NWO) to study the impact of algorithmic filter systems and artificial intelligence on specific segments of the population and niche audiences.

Damian Trilling (PhD, 2013) is an Associate Professor of Communication in the Digital Society in the Department of Communication Science at the University of Amsterdam. He is especially interested in how the changing media environment changes how people engage with news and current affairs.

References

Araujo

Boukes

Trilling

, et al. (2020) Communication in the Digital Society Survey in the Netherlands. https://doi.org/10.17605/OSF.IO/YU64R

Arendt

Fawzi

(2019) Googling for Trump: investigating online information seeking during the 2016 US presidential election. Information, Communication & Society 22(13): 1945–1955.

Bakk

Tekle

Vermunt

(2013) Estimating the association between latent class membership and external variables using bias-adjusted three-step approaches. Sociological Methodology 43(1): 272–311.

Bodó

Helberger

Eskens

, et al. (2019) Interested in diversity. Digital Journalism 7(2): 206–229.

Boomgaarden

Vliegenthart

(2007) Explaining the rise of anti-immigrant parties: the role of news media content. Electoral Studies 26(2): 404–417.

Borra

Weber

(2012) Political insights: exploring partisanship in Web search queries. First Monday 17: 4070.

Bos

Lecheler

Mewafi

, et al. (2016) It’s the frame that matters: immigrant integration and media framing effects in the Netherlands. International Journal of Intercultural Relations 55: 97–108.

Brenes Peralta

Wojcieszak

Lelkes

, et al. (2017) Selective exposure to balanced content and evidence type: the case of issue and non-issue publics about climate change and health care. Journalism & Mass Communication Quarterly 94(3): 833–861.

Brewer

Gross

(2005) Values, framing, and citizens’ thoughts about policy issues: effects on content and quantity. Political Psychology 26(6): 929–948.

10.

Courtois

Slechten

Coenen

(2018) Challenging Google Search filter bubbles in social and political information: disconforming evidence from a digital methods case study. Telematics and Informatics 35(7): 2006–2015.

11.

de Vreese

(2004) The effects of frames in political television news on issue interpretation and frame salience. Journalism & Mass Communication Quarterly 81(1): 36–52.

12.

Druckman

(2001) The implications of framing effects for citizen competence. Political Behavior 32: 225–256.

13.

Epstein

Robertson

(2015) The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. Proceedings of the National Academy of Sciences 112(33): E4512–E4521.

14.

Festinger

(1957) A Theory of Cognitive Dissonance. Standford, CA: Standford University Press.

15.

Garrett

(2009) Politically motivated reinforcement seeking: reframing the selective exposure debate. Journal of Communication 59(4): 676-699.

16.

Google (n.d.) Ranking results how Google search works. Available at: https://www.google.com/search/howsearchworks/how-search-works/ranking-results/ (accessed 15 March 2022).

17.

Greussing

Boomgaarden

(2017) Shifting the refugee narrative? An automated frame analysis of Europe’s 2015 refugee crisis. Journal of Ethnic and Migration Studies 43(11): 1749–1774.

18.

Haas

Unkel

(2017) Ranking versus reputation: perception and effects of search result credibility. Behaviour & Information Technology 36(12): 1285–1298.

19.

Haim

Arendt

Scherr

(2017) Abyss or shelter? On the relevance of Web search engines’ search results when people Google for suicide. Health Communication 32(2): 253–258.

20.

Hannak

Sapiezynski

Molavi Kakhki

, et al. (2013) Measuring personalization of web search. In: Proceedings of the 22nd international conference on World Wide Web, Rio de Janeiro, 13–17 May. New York: Association for Computing Machinery.

21.

Helberger

Karppinen

D’Acunto

(2018) Exposure diversity as a design principle for recommender systems. Information, Communication & Society 21(2): 191–207.

22.

Knobloch-Westerwick

Johnson

Westerwick

(2015) Confirmation bias in online searches: impacts of selective exposure before an election on political attitude strength and shifts. Journal of Computer-Mediated Communication 20(2): 171–187.

23.

Lecheler

de Vreese

(2018) News Framing Effects. 1st ed. Abingdon; New York: Routledge.

24.

Möller

van de Velde

Merten

, et al. (2019) Explaining online news engagement based on browsing behavior: creatures of habit? Social Science Computer Review 38: 616–632.

25.

Nylund

Asparouhov

Muthén

(2007) Deciding on the number of classes in latent class analysis and growth mixture modeling: a Monte Carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal 14(4): 535–569.

26.

Pan

Hembrooke

Joachims

, et al. (2007) In Google we trust: users’ decisions on rank, position, and relevance. Journal of Computer-Mediated Communication 12(3): 801–823.

27.

Pariser

(2011) The Filter Bubble: What the Internet Is Hiding from You. London: Penguin Press.

28.

Price

Tewksbury

Powers

(1997) Switching trains of thought: the impact of news frames on readers’ cognitive responses. Communication Research 24(5): 481–506.

29.

Puschmann

(2019) Beyond the bubble: assessing the diversity of political search results. Digital Journalism 7(6): 824–843.

30.

Robertson

Lazer

Wilson

(2018) Auditing the personalization and composition of politically-related search engine results pages. In: Proceedings of the 20 World Wide Web conference on World Wide Web – WWW’18, Lyon, 23–27 April. New York: ACM Press.

31.

Roggeband

Vliegenthart

(2007) Divergent framing: the public debate on migration in the Dutch parliament and media, 1995–2004. West European Politics 30(3): 524–548.

32.

Schneider

(2007) Anti-immigrant attitudes in Europe: outgroup size and perceived ethnic threat. European Sociological Review 24(1): 53–67.

33.

Schuldt

Roh

(2014) Media frames and cognitive accessibility: what do ‘global warming’ and ‘climate change’ evoke in partisan minds? Environmental Communication 8(4): 529–548.

34.

Slechten

Courtois

Coenen

, et al. (2021) Adapting the selective exposure perspective to algorithmically governed platforms: the case of Google Search. Communication Research. DOI: 10.1177/00936502211012154.

35.

Stroud

(2008) Media use and political predispositions: revisiting the concept of selective exposure. Political Behavior 30(3): 341–366.

36.

Stroud

(2010) Polarization and partisan selective exposure. Journal of Communication 60(3): 556–576.

37.

Sunstein

(2001) Republic.com. Princeton, NJ: Princeton University Press.

38.

Taber

Lodge

(2006) Motivated skepticism in the evaluation of political beliefs. American Journal of Political Science 50(3): 755–769.

39.

Trielli

Diakopoulos

(2020) Partisan search behavior and Google results in the 2018 U.S. midterm elections. Information, Communication & Society 25(1): 145–161.

40.

Urbániková

Tkaczyk

(2020) Strangers ante portas: the framing of refugees and migrants in the Czech quality press. European Journal of Communication 35(6): 580–596.

41.

Van der Brug

Van Spanje

(2009) Immigration, Europe and the ‘new’ cultural dimension. European Journal of Political Research 48(3): 309–334.

42.

Westerwick

Kleinman

Knobloch-Westerwick

(2013) Turn a blind eye if you care: impacts of attitude consistency, importance, and credibility on seeking of political information and implications for attitudes. Journal of Communication 63(3): 432–453.

43.

Wonneberger

Meijers

MHC

Schuck

ART

(2020) Shifting public engagement: how media coverage of climate change conferences affects climate change audience segments. Public Understanding of Science 29(2): 176–193.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.38 MB