Abstract
Introduction
The scholarship on policy feedbacks on mass publics reflects Schattschneider’s observation that “new policies create a new politics” (1935). The basic premise is that public policies are not only outputs of the political system. Rather, they are also inputs in the sense that citizens’ experience with these policies and interactions with public officials administering them are consequential for a large array of political attitudes and behaviours 1 (for recent reviews see Bussi et al., 2022; Busemeyer et al., 2021; Campbell, 2012; Larsen, 2019). Macro-level developments in the form of evolving public policies are thereby part of the context that is to be considered to account for the latter. For the past ten years, feedback studies on mass publics have gained renewed scholarly attention. From their original focus on the study of social policy in the United States, they have expanded to include other policy fields such as education, health and tax policies, the carceral state and state-police interactions (e.g. Michener, 2018; Rose, 2018; Lerman and Weaver, 2004; Mettler, 2011; Soss and Weaver, 2017; Walker, 2020). They have also discussed the political outcomes of several design features from a citizen’s perspective, like conditionality and supervision, and hidden as well as visible policies (e.g. Bruch et al., 2010; Gingrich and Watson, 2016; Watson, 2015; Soss, 2002). As a result, the knowledge on how citizens’ experiences with specific programmes contribute to shaping their sense of self-efficacy, their interest for politics or their turnout has incredibly widened.
While taking stock of these developments in the literature, prominent scholars in the field have been calling to further expand this body of scholarship. In a landmark article, Campbell asked for feedback studies to account for citizens’ multiple policy experiences and not only their experiences with single policies (2012). Moreover, Béland and Schlager (2019), echoing other scholars (Kumlin and Stadelmann-Steffen, 2014), suggested to take this body of research to non-US cases, including Europe. In Western European countries, citizens have a variety of policy experiences over their life course, including school experiences as a pupil or a parent, experiences of public transportation, the health care system, and the income tax policy; as well as, in some cases, tax incentives for home owners, unemployment benefits or social assistance; in addition to retirement policy and support to cultural institutions like movie theatres, youth houses, senior clubs, and many others. These scholars thus direct our attention to the real-life experiences of citizens who live in a policy-ridden environment.
However, methodologically, acknowledging the multiplicity of citizens’ policy experiences in Western Europe challenges the research designs and the causal inferences that are common in this body of work. First, mainstream feedback studies are designed to study how citizens’ experiences with a specific policy are consequential for specific political attitudes and/or behaviours. Research designs are thus predicated on the a priori selection of a single policy which is deemed relevant by the researcher. The selection of single policies drives, then, the collection and analysis of data, be it with qualitative or quantitative methods. Replicating such a design to study citizens’ multiple policy experiences would be aloof from reflecting citizens’ real-life policy experiences, at least because it would not recognize that some experiences may be more salient, and would not allow to study how these experiences interact. Second, and relatedly, most feedback studies build causal inferences between a policy experience and political outcomes at the individual level based on two main mechanisms, the resource and the interpretive effects 2 . These mechanisms are policy-specific as they hinge on specific policy design features. When the multiplicity of policy experiences is considered, individual causal mechanisms cannot simply be added up: the few studies that document citizens’ multiple policy experiences evidence that the combined political outcomes are not the sum of individual policies’ feedback effects (Rosenthal, 2021a; Bruch et al., 2010). Overall therefore, from a methodological perspective, the roots of the literature on policy feedbacks on mass publics are challenged by these calls to theoretically and empirically expand their scope.
In our research, taking Campbell’s (2012) as well as Béland and Schlager’s (2019) calls as a starting point, both challenges were present from the first day onwards. Our research started from the observations that European citizens live in a political context shaped by multiple public policies and the normative messages embedded in these policies about the state and other political authorities and what they ought to do. Following the shared standards of cumulative theory building (Lamont and White, 2009), we anchored our study in the existing scholarship on policy feedbacks on mass publics. Our main research question asks how public policies are related to citizens’ disaffection toward politics in Belgium, France and the United Kingdom 3 . We rely on a secondary qualitative analysis of a corpus that includes interview and focus groups data collected at several points in time since 1990 to 2019 by primary researchers as part of their respective research projects. Our approach is thus grounded in qualitative secondary analysis, whereby we have used already existing qualitative datasets. We have performed a collective and configurative secondary analysis. Collective qualitative secondary analysis involves generating a dialogue between team members to establish comparisons and linkages across studies in ways to achieve relational distance; whereas configurative qualitative secondary analysis refers to how existing data are brought into conversation with existing theory and evidence, typically in ways which exploit greater temporal distance (Hughes et al., 2022).
In this article, we show, from a methodological perspective, how both calls shake feedback studies to their core and, based on our own research, we discuss how we tackled both challenges. Specifically, in response to the first challenge, that of the adaptation of research designs to the study of multiple policy experiences, we discuss our approach that applies a comparative and longitudinal qualitative secondary data analysis with no a priori policy selection. Our approach raised a related challenge, that of the operationalization of multiple policy experiences and perceptions in policy-selection-free datasets, which we address as well. In response to the second challenge which pertains to drawing causal inferences from multiple policy experiences to political attitudes, we elaborate on a third feedback mechanism which is normative and collective (Svallfors, 2010). We also discuss how we dealt with another challenge stemming from this choice, that of the empirical study of collective norms linked to multiple policy experiences from individual-level data. Table 1 summarizes our approach and the methodological challenges we faced. Overall, we discuss how multiple policy experiences and perceptions, and policies’ normative messages observed in interview data could be conceptualized and operationalized as traces of a policy-ridden environment based on qualitative secondary analysis.
In the remainder of the article, we first present the methodological challenges raised by the study of citizens’ multiple policy experiences in Europe from the perspective of the literature on policy feedbacks on mass publics. We then turn to how we dealt with each of them in our own research. In the second section, we address the challenge to research design and operationalization while in the third section, we discuss how causal inferences can be drawn when studying citizens’ multiple policy experiences. In the concluding section, we discuss the contributions of our own approach grounded in qualitative secondary analysis to respond to the methodological challenges raised by calls to theoretically and empirically expand feedback studies on mass publics.
Methodological challenges arising from studying citizens’ multiple policy experiences in a policy feedback perspective and related methodological challenges based on our own approach
The methodological challenges faced by feedback studies on mass publics
The scholarship on feedback studies was initially built on the idea that policy experiences are likely to shape citizens’ political attitudes and behaviours. These experiences hinge on the relationships with street-level bureaucrats, as well as policy design features. Policies’ level of generosity, whether they are universal or targeted, the definition and social construction of target populations as (un)deserving, as well as their mode of delivery and whether they are obligations-oriented and supervisory or rights-based, are consequential. As an illustration, a now classic study by Soss compares the experiences of beneficiaries of two distinct policies, a social assistance benefit targeting families with children (AFDC) and disability schemes in the US (2002; 1999). His research lends support to the idea that universal and conditional programs trigger opposite outcomes. Soss’ explanation points to recipients’ relations with caseworkers whom beneficiaries consider to be government’s representatives. In the case of AFDC, a conditional and supervisory programme, he shows that recipients perceive their relations to state officials as being adversarial, inquisitive and partial. This generates a sense of powerlessness and triggers a depressed personal and political efficacy. In contrast, recipients of disability schemes, which are rights-based, consider that the process of implementation is fair and balanced, and that they are treated as full-fledged citizens. Their experience with this policy generates a sense of increased political efficacy.
Policy feedbacks on citizens’ political attitudes and behaviours are explained by two mechanisms that are policy-specific and depend on the policy’s design features 4 . The resource effect describes “how resources and incentives provided by policies shape patterns of behaviour” whereas the interpretive effect captures “how policies convey meanings and information to citizens” (Mettler and Welch, 2004: 500). The resource-based mechanism (Pierson, 1993) is typically illustrated by senior citizens in the US who benefit from Medicare and the public retirement scheme. They may be qualified as “über citizens” (Campbell, 2003) as their level of political efficacy is higher than would be expected from mainstream individual-based accounts of political participation. This explanation points at the resources both programs provide them with, in terms of money and time in particular, that are conducive to higher political participation. The seminal case of the interpretive mechanism, on the other hand, is provided by American veterans who benefited from the GI Bill and who became active citizens not only on the basis of the resources for political participation that education provided them with, but also due to a feeling of being considered as first-class citizens, greatly supported by their community and who wanted to give back to the community (Mettler, 2005).
A defining feature of mainstream feedback studies is that virtually all of them start from a specific policy or set of policies. The selection of the policy of interest comes first. In that respect, Campbell’s call to consider the “lived experiences” that ordinary citizens have in the course of their daily life was ground-breaking (2012). She argued that their political attitudes are likely to be shaped by the combination of multiple policy experiences. In fact, citizens tend to be exposed to a myriad of public policies which result in direct or indirect policy experiences. Hern makes a similar point when stressing that “the experiences (with one or a few policies) that studies with a narrow policy focus test do not match the scale of the theoretical underpinnings of policy feedback – that experiences with public policy condition citizens’ relations to their state, influencing their level of political participation” (2019: 60) and their political attitudes.
Recently, a few studies have produced evidence that showcase how integrating the multiplicity of policy experiences refines well-established findings in the literature (Mettler, 2018 Rosenthal, 2021a; Theiss and Kurowska, 2019; Rosenthal, 2019; Bruch et al., 2010). Rosenthal’s analysis of survey data on US citizens confirms, on the one hand, previous findings on the respectively participation-enhancing and participation–depressing effect of visible universal and means-tested social policies (2021a). On the other hand, he presents evidence that citizens’ simultaneous exposure to multiple social policies sending contradictory signals, i.e. means-tested and universal policies, lead to situations in which either the predicted effects of isolated policies simply cancel each other out, or where specific experiences’ feedbacks prevail over others. In that respect, observing policy experiences in isolation may lead to incorrect conclusions about their feedback effects.
Béland and Schlager’s call to expand the empirical reach of feedback studies beyond the US, and notably to Europe, further strengthens Campbell’s invitation to consider citizens’ multiple policy experiences (2019). Western European citizens do experience the state and other public authorities through manifold public policies. In addition, they live in policy-ridden environments where direct spending are comparatively higher and welfare states include more universal components in comparison to other welfare capitalist economies (Jacques and Noël, 2021; Bussi et al., 2022).
These calls do not simply yield theoretical and empirical consequences. They also bring along a frontal methodological challenge to standard research designs which are virtually all based on the a priori selection of a single policy or a set thereof. The issue is that of the adaptation of these research designs to the study of multiple policy experiences. First, should the a priori selection of policies be kept and, if this is the case, how should be determined what policies matter the most in the life circumstances of different subgroups of citizens (Mettler, 2018; Rosenthal, 2021a)? Or should this selection be left open? In that case, how to design a research in such a way that research participants can themselves indicate what their policy experiences are (Rosenthal, 2021b)? Second, at a deeper level, Campbell’s call to account for citizens’ multiple policy experiences invites to consider how the combination of citizens’ policy experiences feeds back into their political attitudes and behaviours, rather than just the addition of several experiences. This raises the daunting question of the varying salience of policy experiences within specific groups of citizens (Hern, 2019), and the variation thereof between groups. It also draws attention to the issue of how these multiple experiences interact and what research design could capture these policy combinations for different groups of citizens (for an attempt at doing that, see Dupuy and Van Ingelgom, 2016).
Furthermore, inviting scholars to study citizens’ multiple policy experiences comes with another methodological challenge which pertains to causal inference. As such, feedback studies face a general inferential problem related to endogeneity as citizens’ attitudes and behaviours are supposed to influence policy-making, which in turn is expected to feed back into these very political attitudes and behaviours. The scholarship has dealt with the issue by assuming that there is a reciprocal causality and, more specifically, it has elaborated on policy-specific causal mechanisms that hinge on each policy's specific design features. But, then, how could causal inferences be drawn from multiple policy experiences to political attitudes and behaviours? Should we stick to the analysis of the resource and the interpretive mechanisms – which practically could require the study of the design features of a virtually endless list of policies? This choice would come at the cost of studying the combination of policy experiences to focus instead on the addition of several experiences. Also, as argued by Hern, not only may citizens have different experiences from one policy to the other, but also “different types of policies or services are salient for different people, so similar experiences may generate different political effects, as a function of policy salience” (2019: 15). The tricky issue is then to account theoretically for the combination of policy experiences and variation across groups.
Research designs and the operationalization of multiple policy experiences
In this section, we start with the first challenge – how to adapt research designs to the study of multiple policy experiences – and discuss how we designed our research to study Belgian, French and British citizens’ multiple policy experiences in view of analysing their attitudes toward politics. We also reflect on the type of data we need to study citizens’ experiences with a policy-ridden environment based on our qualitative comparative and longitudinal design, and present our approach to qualitative secondary analysis. We then discuss a related methodological challenge that stemmed from our choice of data and data collection method, namely the operationalization of policy experiences and perceptions in qualitative secondary interview data. We focus particularly on coding, from building the codebook to coding decisions.
How to adapt research designs to the study of multiple policy experiences? (A)
A few studies have followed up on Campbell’s call to study citizens’ multiple policy experiences and, thus, supplemented the standard single-policy research design. One strategy is to study citizens’ experiences with several policies based on an a priori selection of public policies by the researchers. This strategy replicates the standard research design and applies it to several policies. For instance, seeking to elucidate the paradox of the simultaneous increase in the number of Americans that are hostile towards the federal government and those that benefit in one way or another from its social policies, Suzanne Mettler analyses survey data including information on not less than 21 different social policies (Mettler, 2018). Another related strategy (see Hern, 2019 for this distinction) is to study how citizens’ experiences with specific design features in combination with others result in specific political outcomes. As an illustration for this approach, Watson‘s (2015) study mobilizes British panel data and a difference-in-difference design to assess how different forms of conditionality in welfare policies impact individuals’ various indicators of political behaviour and attitudes. While fruitful, this first set of strategies is not fully attuned to capture the whole of citizens’ policy experiences as research designs focus on specific policies only. It possibly overlooks potentially significant policy experiences not included in the set of selected policy domains or design features. In addition, while these studies do analyse multiple policies, they mainly focus on singling out the effects of specific policies or design features and, thereby, pay less or no attention to the question of how the accumulation and combination of policy experiences influences citizens’ political attitudes.
In contrast, recent contributions have suggested a distinct approach that takes a bigger step away from the standard design and provides with an approach closer to citizens’ everyday policy experiences. In these works, no a priori policy selection drives the design. On the contrary, the research is designed to have participants describe their policy experiences as they think about them. Interestingly, this strategy is implemented both with quantitative and qualitative methods. In her survey of Zambians’ experiences with service provision, Hern asks questions about citizens’ overall experiences with their government and only later in the survey, based on a list, she asks about the specific services or projects they were able to access (2019). Rosenthal adopts a similar approach, free of policy selection, in his studies on government visibility and its variation across racial groups in the US (2021b; 2019). He conducts semi-structured interviews where interviewees are asked open questions on how they felt the government most impacted their life (Rosenthal, 2019: 516) or their associations with the government. In that respect, research participants do the policy selection themselves based on their lived experiences, and the analysis then goes from there.
In our own research, we were drawn to align with this policy-selection-free strategy. Our research started with the empirical observation of citizens’ persistent feelings of disaffection toward politics in Western European established democracies. We wondered how this widespread disaffection could be related to two major developments, the turn to neoliberal policies and their supranationalisation over time and in different countries. Our approach integrates the multiple relations that ordinary citizens have with policies both in terms of experiences and perceptions. Indeed, Soss and Schram (2007) and Simonsen (2021) reported that policy perceptions, based on media and politicians’ discourses, are as consequential as direct and personal policy experiences. Alongside multiple policy experiences, their research thereby supports including the analysis of policy perceptions when studying the political outcomes of policies. In that respect, citizens’ multiple relations with policies derive from their first-hand or indirect experiences with various policies, their perceptions of policies they hear about through acquaintances, the media and politicians, and their exposure to policies’ normative messages about the state, the bureaucracy or the market. Finally, alongside a policy-selection-free design, our research question also called for some form of comparative longitudinal analysis since our study focuses on how citizens’ relations to politics hinges on their policy experiences and perceptions as policies change over time. This key feature is what the next section focuses on.
Our qualitative comparative and longitudinal design based on secondary data (B)
We turned to qualitative secondary data based on a constructed corpus of qualitative datasets (for a synthetic presentation of the related debates, including epistemological discussions, see Vila-Henninger et al., 2022; Duchesne, 2017; Hughes and Tarrant, 2020). Secondary analysis is understood as “a research strategy which makes use of pre-existing quantitative or pre-existing qualitative research data to investigate new questions or verify previous studies” (Heaton, 2004: 16). Qualitative secondary analysis entails the use of already produced qualitative data – interviews (face-to-face or focus groups), field and observations’ notes – to develop new social scientific and/or methodological understandings (Irwin, 2013). To our best knowledge, no comprehensive dataset existed when we started our research that would allow for a longitudinal and cross-national comparison of different social groups’ policy perceptions and experiences and how the latter shape their relationships to politics in general. Moreover, obviously, no retrospective longitudinal and comparative qualitative data could be collected to cover citizens’ policy perceptions and experiences of the past thirty years, as no interview technics allows for such an investigation.
We thereby got interested in qualitative secondary analysis as we knew from previous experiences of collecting and analysing interview data that generated data is richer and usually extends far beyond the scope of the primary research – an observation that is common among qualitative scholars. In addition, two of us had previously conducted qualitative secondary analysis 5 , both of a continuous type, whereby “researchers ask[…] new kinds of questions of existing research, as well as using existing data to support the formulation of new questions for new research” and a collective form, where “members of existing research teams from similar, cognate or even entirely distinctive projects (…) are purposefully brought together in order to permit a dialogue of evidence and questioning (…)” (Hughes et al., 2022: 380). We thus experienced first-hand the potentials – and challenges – of qualitative secondary analysis to study new research questions based on existing data and, providing the defining features of data collection, to get access to insightful empirical material (Dupuy and Duchesne, 2017; Belot and Van Ingelgom, 2017).
A decisive step is building the corpus of secondary data that will be analysed. Qualitative repositories such as the Henry A. Murray Research Centre (Harvard), the Qualitative Data Repository (Syracuse), the Timescape Archive (Leeds, specialized repository of longitudinal data), the Australian Data Archive, Qualidata (now part of the UK Data Archive) or BeQuali (Sciences Po Paris) have flourished (Gaspare, 2017). In our research, we did not resort to already stored data. Instead, we relied on the expertise and networks of one of us to identify datasets that would feature what we were looking for, namely (i) semi-to non-directive interviews; with interviewees from varying age, gender and socio-demographics backgrounds; (ii) datasets sharing common research questions and literature of references – as no research had previously been conducted directly on our topic of interest; (iii) datasets that would include at least one of our country case of interest 6 ; and, last, (iv) datasets that would be generated at different points in time.
At the time of writing this article, our corpus includes 4 main datasets shared by primary researchers with us 7 (see Table 2) composed of face-to-face semi-structured interviews and focus groups 8 . Crucially, they share one commonality: they research everyday citizens’ relationships to Europe. First, Celine Belot’s dataset includes 31 semi-structured face-to-face interviews conducted in Durham (Tyne and Wear, UK) and Guildford (Surrey, UK) between May and June 1995 and in Grenoble (France) and Boulogne-sur-Mer (France) between February and April 1996 (Belot, 2000). Belot interviewed young people between 15 and 30 years old and the sample was diversified according to age (3 classes), gender and education.
Datasets included in our corpus
* Data from
Second, the CITAE dataset is composed of focus groups organized in Brussels, Oxford and Paris between January and June 2006 (Duchesne et al., 2013). It is based on 24 collective discussions involving four to six participants selected to be socially close but politically diverse. The focus groups were conducted with citizens from different socio-economic contexts (working class, white collars, managers and activists).
Third, Heidi Mercenier’s dataset aims to understand the ways young people perceive and understand politics in general, and the EU in particular. Her study is based on six focus groups involving 35 young people aged 16 to 26, who grew up in Brussels if not born in Belgium. They were conducted between November 2013 and May 2014. In each focus groups, participants shared similar social characteristics and were gender- and age- balanced while politically diverse. The participants did not know each other but lived in each case in the same neighbourhood. Considering the spatial segregation of Brussels, the choice of the neighbourhoods makes it possible to interview people with diverse socio-demographic features.
Fourth, the RESTEP dataset is designed to study when and how citizens politicize European issues and whether citizens’ opinions are structured by specific cleavages across social groups and national contexts (Beaudonnet et al., 2022). In the original dataset, 21 focus groups were organized with different socio-economic groups in four countries (France, Belgium, Portugal, and Italy) during a four-month time span in 2019. Only those conducted in France (Grenoble) and Belgium (Louvain-la-Neuve) were kept to be analysed as part of our secondary corpus. The focus groups gathered 69 participants from different age, level of education and with different political orientations.
Next to the four main datasets, a fifth one serves the purpose of a control as it focuses specifically on the welfare state in the UK (Taylor-Gooby and al., 2019). The ‘Welfare State Futures: Our Children’s Europe’ (WelfSOC) original dataset comprises focus group and democratic forum data from five European countries (Denmark, Germany, Norway, Slovenia, and the UK). There were six different WelfSOC focus groups: ‘Young,’ ‘Middle Class,’ ‘Retired,’ ‘Working Class,’ ‘Working Women,’ and ‘White British.’ For the present project, we used the data collected from the three democratic forums realized in the UK in 2015. Each forum consists of people with similar social status (self-employed, unemployed, ethnic minority). In total 34 individuals participated to the forums that spanned two non-consecutive days.
Overall therefore, our main corpus is composed of datasets that were generated at different points in time and in the same set of countries (France, Belgium and the UK), thereby offering variation on time and policy contexts. Each dataset also offers socio-economic and political variation as primary researchers selected participants to face-to-face interviews or focus groups based on both criteria. Moreover, the similarity of research questions – how do citizens perceive and understand European integration – and methods in the original studies – cross-national qualitative research – supported a sufficient degree of cross-study comparability (Hughes et al., 2021). As we are empirically interested in meaningful commonalities across time, place and / or social-economic background, the remaining heterogeneity in types of data collection and primary research questions may strengthen rather than weaken findings which emerge from otherwise independently designed and collected primary data sources.
One main corpus is particularly well suited to our research endeavours. Indeed, interviewees do discuss a lot of their policy experiences and perceptions, regardless of their individual characteristics, the country, point in time and design of data collection. This is an important observation as none of the primary research was designed to study citizens’ policy experiences and perceptions. Research participants were not prompted to think, reflect or react on questions tapping into their policy experiences and perceptions. Neither interview questions nor focus groups scripts asked about any policy in none of our datasets with the exception of one question on environmental policies in Belot’s research (2000) 9 . It is even the case that in some datasets, policy discussions were overlooked and side-lined by primary researchers in the course of the interview (Dupuy and Duchesne, 2017). Participants’ decision to bring up policy experiences and perceptions in the discussion was thereby not prompted by researchers. Also, as primary researchers did not structure data collection on policy-related questions, the datasets are free of an a priori policy selection. Both the mentions and talks about policy experiences and perceptions and the range of policy domains brought in by interviewees therefore result from their own way of addressing interview questions and collective discussions in focus groups – not from research designs.
In addition, both the interviews and the focus groups of each three main datasets were non-directive in their moderation, and participants were allowed to talk as much or as little as they please about any topic or issue (Van Ingelgom, 2020). Focus groups participants, in particular, could thereby explore and further discuss anything beyond the questions asked by the primary researchers. In each of our main datasets, these discussions were rather lengthy – interviews lasted for over an hour and focus-groups were typically two to three-hours long. Both the non-directive moderation and the length of individual and collective interviews warrant that the topics that interviewees discussed, here the multiple policies they brought into the discussion, reflect their lived policy experiences and perceptions as they think about them, and are not only prompted by public discussions in the media, politicians’ discourses at the time of fieldwork or top-of-the-head examples without further relevance for the participants. Put differently, both the salience and the relevance of expressed policy perceptions and experiences are warranted by these features of data collection.
Altogether, our secondary dataset includes 31 semi-structured interviews and 37 focus groups, gathering 302 research participants in total. Our approach is aligned with a ‘depth to breadth’ qualitative secondary analysis which ‘mov[es] from work with small data samples to working across multiple datasets in a process of cumulative or ‘synthetic’ analysis. Here, ‘synthetic analyses’ refers to the synthesis of different types of data by bringing them into conversation with one another, as well as with evidence, findings, theory and developments in the social world beyond the original study contexts’ (Hughes et al., 2022: 377). Scaling up qualitative analysis (Davidson et al., 2019), our secondary dataset is a compilation of existing datasets where the year and place of fieldwork, as well as the socio-political characteristics of the participants were coded using descriptive codes. The selection and description of this new dataset constitutes the early process of undertaking analysis across existing datasets.
As underlined by Irwin, Bornat and Winterton (2012), working on existing datasets and comparing them is instrumental to bringing into light each project’s specific context. By constructing and describing our secondary dataset, we could thereby explore the ‘contextual embeddedness’ of the original datasets, in terms of the broader contexts in which there were formulated (literature, methods), the purposes for which they were produced and primary researchers’ specific research questions and analytical interests (Hughes et al., 2021). Working across datasets made it clear how different the data that we were dealing with were rather than relying on the ‘naïve’ treatment of research data as ‘data’ (Silverman, 2017).
For secondary qualitative analysis, it is a challenge for researchers to familiarize themselves with data depending on their distance to the formative contexts of data production (Hughes et al., 2022). In our research, team members have varying degrees of proximity and distance with the collected datasets: one of us was a primary researcher or participated to the fieldwork in three of our main datasets and three team members participated to the fieldwork of the RESTEP project. Other team members were not involved in any capacity in the process of data collection of any of the main datasets. By way of familiarization, we extensively read the publications of primary researchers, i.e. the methodological, analytical and theoretical pieces alike. Likewise, we studied all accompanying documents pertaining to data collection that primary researchers shared with us (e.g. notes on interviews and how they went, observations on the selection of participants, summary tables on the characteristics of interviewees, interview guide and focus groups scenario and their rationales). We also met with them so that they could give us a behind-the-scene account of the process of primary data collection. Last, but not least, we had several rounds of reading the transcripts of interviews, before, during and after coding.
The type of cross-national qualitative secondary analysis that we report on questions the way we might conceive of, and work collectively with data and findings from thematically linked qualitative research across national contexts and in different period in time. Discussing our operationalization of policy experiences and perceptions sheds light on how concretely our mode of empirical engagement with secondary qualitative data can be characterized as a collective and configurative qualitative secondary analysis (Hughes et al., 2022).
Operationalizing policy experiences and perceptions (C)
Our data covers a wide array of policies and policy issues as our interviewees brought in their policy experiences or referred to policies to discuss their relations to the Europe Union and to politics more generally. Our corpus is ridden with policies. Because primary data collection was not structured by questions prompting interviewees to discuss policies, the identification of policy experiences and perceptions has been a key issue. The challenge was to meaningfully distinguish interviewees’ references to policy experiences and perceptions from broad, fuzzy or vague references to something related to policies.
First, with respect to the operationalization of policy experiences and perceptions, the scholarship relies on two main non-exclusive strategies. The direct strategy relies on the first-hand collection of citizens’ experiences with the policy under study by semi-structured interviews and, sometimes, observations of the interactions of citizens and street-level bureaucrats (e.g. Soss, 2002; Levitsky, 2014). Some research expands the notion of policy experiences to include not only first-hand interactions but also second-hand interactions, like that of a close relative or friend (Maltby, 2017; Walker et al., 2020). In both cases, the standard operationalization of a policy experience is its tangible presence in one’s life, or a close one’s, as well as its concrete and immediate impacts (Soss and Schram, 2007). The indirect strategy uses mostly quantitative data on policy recipients and their political attitudes (e.g. Watson, 2015; Gingrich, 2019). Policy experiences are not directly observed. Instead, they are inferred from the theory-driven analysis of that policy’s design (for an exception, see Hern, 2019). Given our main research question and our data, we went for the direct strategy and operationalized citizens’ policy experiences and perceptions based on their own references to them and how they talked about them.
Second, we had to operationalize policy experiences and perceptions in datasets not structured by policy-related prompts. It came with specific issues not unknown from the literature on secondary qualitative analysis. On the one hand, our operationalization had to travel across datasets and, thereby, not to be restricted to the policy-related discussions in one of the datasets. Instead, at stake for us was to allow the study of citizens’ multiple policy experiences and perceptions by encompassing any policy domain mentioned or discussed by interviewees in different national and social contexts, as well as points of time. Here, the ‘translation’ across different international and social contexts was a key element of our collective work. As developed by Hughes and colleagues, it refers to “more than the linguistic translation of a word in one language into its cognate partner in another, but also of bringing into analytic alignment cognate theories, concepts, literatures, methodologies, fieldwork sites, study populations and so forth” (2021: 2) 10 . To do so, we started from the most comprehensive categorization of policy domains available, that of the ongoing Manifesto Research Group / Comparative Manifestos Project (Bevan, 2019), to build our policy codes. As these codes are defined at the level of the policy domain, they are in essence comparative. Discussions of country-specific policy instruments are coded under the same code if they pertain to the same policy domain. For instance, we coded under “social policy” mentions of policy instruments for minimum revenue provision in different domestic context like revenu d’intégration (Belgium), revenu de solidarité active (France), or guaranteed minimum income (UK). The codes are therefore instrumental to study whether policy experiences and perceptions within specific domains are more extensively discussed than others, and whether that evolves across time for instance. We included a few additional codes in order to adapt the codebook to the European focus in our datasets, such as Euro, European Integration, and Circulation. As a result, our codebook features 23 policy domains ranging from social policy over economic to cultural policy.
On the other hand, we had to operationalize citizens’ mentions of their policy experiences and perceptions. Initially, our research team had been admittedly very naive about this. We started from definitions of policy codes that virtually included nothing but the policy domain. That resulted in diverging coding practices with some team members coding extensively any vaguely related mention to policies, while others had a more restrictive conception of policies. The main point of discussion pertained to whether mentions of policy issues had to be coded under a policy code. For instance, the Mercenier focus-groups start with a question about the main problems of nowadays (“What are the most important problems we face today?”). This question was inspired by Jonathan White’s research on political bonds in the European Union (2011). From an applied political theory perspective, he argues that it is only when citizens consider that they share problems with others from other European countries, and when these problems can be tackled politically through policy-making, that political allegiance may be redefined at the European level. Collective problems or policy issues are thereby theoretically not unrelated to policies as such. However, we decided against coding policy issues under a policy code. We theoretically grounded our decision in feedback studies where citizens’ political attitudes are specifically explained by their policy experiences and their policy perceptions.
We thus needed to elaborate on coding criteria to exclusively code for policy experiences and perceptions. They emerged during an abductive process of collective coding that we describe elsewhere (Vila-Henninger et al., 2022). Our coding strategy was iterative in essence. It was open to exposure to multiple policies, but narrow enough to exclude broad references to policy issues. Based on our collective exchanges and analytical reflections, we thereby elaborated on four distinct criteria. Only when one or several of these criteria were met and when a policy domain was mentioned, was the excerpt coded under a policy code – depending on the policy domain under discussion. We present and illustrate in turn each criterion.
Our first criterion captures policy experiences and, specifically, a direct experience with a given policy or that of a close one. The following quote from the Mercenier dataset is illustrative. In the context of a discussion on the most important societal problems, Jordan, a 22-year-old man from the Brussels neighbourhood of St. Josse-ten-Noode, uses his mother’s (unsuccessful) efforts to obtain social housing to illustrate his opinion that social assistance rather benefits the undeserving and not those really in need. Thus, this quote was coded under the social policy code.
Jordan: Like me, my mother, she has been homeless for two years now. I mean, not really homeless because she does not live in the street because my father hosts her. But if my father did not host her, she would be in the street what. And she has two dependent children, I mean, three with me. I don’t know how long she has been asking the CPAS [Centre public d’aide sociale] to just have the CPAS or a social apartment, I mean something to house her children or at least to support them.
The other three criteria pertain to policy perceptions. The second criterion was participants mentioning (a group of) policy recipients or policy target. For instance, in the context of a semi-structured interview (Belot dataset, 1995), 29-year-old British citizen Alex who is himself unemployed, argues for the necessity that (young) unemployed should be the target of public efforts to improve their personal situation in order to prevent them from making bad life decisions. The quote was coded under the employment policy code.
Alex: I mean young unemployed people have aspirations, and they ought to be helped to get something. Not just to sort of, I don’t know, like there are loads of young men in Durham for example, from the area, they end up [3 seconds break], you know, going into crime.
The third criterion is the explicit mention of a specific policy measure or policy instruments. This was, for example, the case in the following quote where participants in a French focus group recruited from working class participants raised the issue of the 5% VAT on catering (CITAE dataset). We coded this instance as “economic policy” because participants talked about a specific policy tool in a discussion about the locus of decisional power and the perceived incapacity of French domestic politicians to deliver on promises without approval by EU institutions.
Lionel: Which means that when we try in France, when Chirac says: [imitating Jacques Chirac’s voice a bit] “don’t worry about the VAT on catering, I’ll lower it to five [percent] I swear to you “. After Villepin: “I swear to you, no problem”.
Albert: No, that’s Europe but [towards Lionel, Geoffroy approves him sideways]
Yasmina: He didn’t lower it.
Habiba [to Lionel]: Yeah.
Lionel: It’s been 24 months.
Yasmina: It wasn’t down. Yes, it has not been lowered.
Geoffroy [in front of him]: It hasn’t gone down.
Lionel: The restaurateurs are waiting. Chirac promised. Chirac doesn’t have it, he can’t.
The fourth criterion was applied when participants talked about a policy rationale, that is, the cognitive logics that underpin a policy. Aurélien, a participant in a CITAE focus group composed by Belgian political activists, illustrates this criterion. He praises the goal of industrial specialisation of the “Marshall plan”, a regional policy in Wallonia. He argues that considering that this specialization is likely to result in specific economic sectors in Belgium becoming European leaders is a valid rationale. The quote was coded under the economic policy code.
Aurélien: At the market level, I think that at the company level, there is extraordinary potential at the European level because I personally am a European nationalist. I prefer my radio cassette or computer to be made in the Czech Republic than in the Republic of China. And so, I see the European market as an opportunity, a specialization. This is exactly what the Marshall Plan does, well the so-called Marshall Plan in Wallonia [Stéphane chuckles]. We specialize in certain themes where we exactly want to be the leader at European level. It’s now something, a benchmark. If we didn’t have this European market with all the excesses that it generates, but if we didn’t have this European market, where would we be?
Building these four criteria was a decisive step in our research. The theoretical and analytical clarifications that came with it are illustrative of how we have dealt with secondary qualitative analysis as both a collective and a configurative engagement with the data. It was collaborative as it emerged from the dialogue between team members about comparing and building linkages between datasets. It was also configurative as the existing data were brought into conversation with broader sources of theory and evidence from the policy feedback literature (Hughes et al., 2022). In that sense, team members’ relative distance from the formative contexts of data production was instrumental in clarifying how we could operationalize citizens’ experiences and perceptions of public policies in comparative qualitative data through secondary qualitative analysis.
Building inferences from multiple policy experiences to feedback effects
In this section, we turn to the second methodological challenge raised by Campbell’s and Béland and Schlager’s respective calls to study citizens’ lived experiences outside of the United States, and in Western Europe specifically: how to draw causal inferences from multiple policy experiences and perceptions to political attitudes? Based on the existing scholarship and our own research, we start with causal mechanisms and introduce a normative mechanism as an alternative to the policy-specific mechanisms mostly relied on in the literature. We discuss the related operationalization challenge stemming from this choice, namely the empirical identification of collective norms linked to multiple policy experiences. While our preferred mechanism departs from mainstream feedback studies, we show that evidencing normative feedbacks in policy-ridden environments relies on a similar comparative strategy.
How to draw causal inferences from multiple policy experiences? (D)
The few studies that pay attention to citizens’ multiple policy experiences mostly draw from existing causal mechanisms, namely the resource and interpretive effects. They thereby stick to single-policy mechanisms and combine them to account for citizens’ multiple policy experiences. We see three main lines of study. One is concerned with the absence of theoretically expected resource and interpretive effects and investigates how the invisibility of specific policies may account for missing positive policy feedbacks. Mettler’s analysis of the “submerged” US state is prominent in this respect (2011). Her analysis relies notably on experiments where groups of citizens are provided with various information on state policy. She reports that groups of citizens that are informed on government spending and the scope of government intervention in citizens’ daily life have diverging, that is, more positive, preferences regarding the state in comparison to groups that lack such knowledge.
Another line of study deals with how policy experiences interact with one another. Rosenthal’s recent study (2021a) is here insightful. His analysis reflects, first, on the relative strength of resource and interpretive effects and, second, on the relative strength of various policies’ interpretive effects. To do so, he constructs several policy combinations composed of policies presenting varying design features. Each of these combinations, based on existing knowledge about single-policy interpretive effects in the US, is assigned an overall interpretive effect based on the theoretically expected balance of each single policy interpretive effects.
Rosenthal follows an additional, third, strategy of inference-building when studying citizens’ multiple policy experiences (2021b). This study is designed without an a priori policy selection and interviews start with an open question about interviewees’ experiences of the state. Based on their policy experiences, expressed in their own terms and without policy prompts, Rosenthal identifies the most salient experiences and accounts for the varying visibility of different policies of the US state to different racial groups in the US. Then, he investigates the interpretive effects that salient policies have for each group. Interestingly, interpretive effects are evidenced based on the cues that interviewees find in their manifold and racially distinct experiences with the US state. In that respect, Rosenthal expands the application of this causal mechanism by applying it not to specific policy design features, but to a set of salient policies in interviewees’ everyday experiences. Inference-building through this causal mechanism relies on a design where questions were asked to describe the cues that interviewees derived from their most salient experiences.
Another option would be to follow Svallfors who elaborates on a third type of policy feedback beyond interpretive and resource effect, namely the normative effect (2010, 2006). It captures how “public policies provide citizens with a sense (…) [of] the desirable state of affairs” and what governments ought to look like (2010: 120). The normative effect thereby entails the construction of collective norms through citizens’ policy experiences and perceptions. While in his own empirical research, Svallfors studies redistributive norms, the normative mechanism is not theoretically restricted to a specific type of norms; instead, it is theoretically open and allows to grasp a large range of collective norms that are conveyed through policies and broader institutions, whether they are about redistribution, the market or politics. This approach thereby complements mainstream feedback studies by integrating the normative components of policy experiences and perceptions, alongside their cognitive dimensions and the resources policies allocate, into feedback studies.
The normative mechanism could be applied to a single policy to study the collective norms associated to this specific public policy. However, as suggested by Svallfors’ empirical study, it may also be applied to a larger set of policies to grasp broader collective norms conveyed to individuals through their experiences with these multiple policies or their perceptions thereof (Verhaegen et al., 2021). Norms about redistribution, the market, or politics, that typically span across policy domains, are cases of these broader collective norms.
Additionally, the normative feedback mechanism is inherently group-based, in contrast to the resource and interpretive effects which are individual-level mechanisms. Norms are collective in essence and that is why the co-construction of norms through citizens’ policy experiences and perceptions has to be assessed at the group-level – which distinguishes it from Rosenthal’s use of interpretive effects of multiple policy experiences which is documented at the individual level. The definition of the group under consideration depends on the question guiding the research. In his seminal study of changes in citizens’ attitudes since the German reunification, Svallfors reports how institutionalized norms about redistribution, embedded in multiple social policies, are not only reflected in individual attitudes but also shared by generations who were exposed to and experienced similar policies (2010). His focus is on policy generations of Eastern and Western Germans, defining the groups he is interested in based on their territorialized and time-specific social policy experiences and perceptions. His empirical analysis distinguishes between different generations of Eastern and Western Germans based on the welfare state they came of age in. In order to do so, he used a threefold cohort categorization: (1) those who were already fully established in adult life at reunification (that is, born before 1950); those part of a middle cohort (born between 1950 and 1975) (3) and those who were still children at reunification (that is, born after 1975). Controlling for demographic factors and material circumstances at the individual-level, he reports that the younger generations of Eastern and Western Germans, who have experienced the same welfare state, share similar norms about redistribution and the role of the state therein. Similarly, he shows that ceteris paribus, younger and older generations of Western Germans share similar norms. In contrast, he reports that older Eastern Germans express distinct norms regarding redistribution in comparison to Western Germans and younger Eastern Germans. Moreover, the difference between the former and the latter has decreased over time, with older Eastern Germans normative preferences becoming closer to that of Western and younger Eastern Germans as their experience of living through a Bismarckian welfare state unfolds. Overall, he documents how social policy changes, that is the multiple policies of the German welfare state, as they are experienced and perceived by individuals over their life course are reflected in generational changes in shared collective norms and the related attitudes about redistribution (see also Verhaegen et al., 2021 on regionalization’s normative effect).
While the normative mechanism illuminates how policy experiences and perceptions are instrumental in conveying collective norms, it does not preclude that these collective norms are conveyed through other means as well, first and foremost by political ideologies. Instead, it is much in line with the broader scholarship on policy feedbacks on mass publics by focusing on the unique contribution of policy experiences and perceptions in shaping political attitudes and behaviours alongside individual, party-political and other explanations.
The causal inferences that the normative feedback mechanism draws between multiple policy experiences and perceptions and political attitudes rest on the collective norms the former convey and construct. Documenting a normative feedback mechanism thereby requires to evidence not only that policy experiences and perceptions are associated to norms, but also that these norms are in fact shared and collective. From a methodological perspective, this third feedback mechanism thereby shifts the focus away from the study of the resources allocated by a policy – grounded in an analysis of the policy design, or its interpretive messages – that typically rests on the analysis of the authority structure, design features and policy implementation.
Operationalizing norms linked to policy experiences and perceptions in interview data (E)
In our research, we build from Svallfors’ theoretical innovation and study how normative components of policies may feed back into collective normative orientations, or norms. In this subsection, we discuss how we operationalize the study of norms linked to policy experiences and perceptions in interview data, and will discuss in the next section how we evidence collective norms linked to multiple policy experiences and perceptions. Analytically, we consider norms to be a specific type of discursive resource in any discursive interaction, including but not exclusively an interview or a focus group, whose defining feature is to be widely accepted, while also possibly contested (Weber, 1978; Billig, 1987). Whether they are adhered to or not by a given citizen, or research participant, collective norms are thought to be the baseline of any discussion. This shift from individual attitudes to collective normative orientations raises methodological challenges of its own. How to identify such collective norms and assess their existence in empirical data? How to evidence that these subjective manifestations are not just individual phenomena, grounded in interviewee’s peculiar life circumstances and own views, but are also related to the multiple policies and related norms they are exposed to? In our research, we are notably interested in neoliberal norms, revolving on the primacy of market competition, and how policy changes that include neoliberal components have shaped citizens collective beliefs about market competition, market responsibility, and individualism. We will draw from this part of our analysis to illustrate our methods of data analysis in this subsection and the next.
Our empirical analysis of collective norms rests on interview segments where interviewees discuss their policy experiences and/or perceptions by using neoliberal legitimations or deligitimations. Legitimation is a social process that draws upon widely held beliefs, norms, and values (Weber, 1978; Thompson, 1971), and this is why we approach norms as they are relied on as legitimation or delegitimation by interviewees during political discussions. Concretely, these interview segments are empirically identified by combining our policy codes – that we presented in the previous section, and our neoliberalism codes (on the definition and use of code equations in qualitative research, see Vila-Henninger et al., 2022). Building on the literature (Rougier, 1938; Amable, 2011; Foucault, 2008; Harvey, 2005), our codebook for neoliberal ideology 11 isolated four major elements of neoliberal ideology: fair market competition, market individualism, market liberty, and human capital. The fair market competition code captures evaluations of the fairness of any aspect of market competition. The code market individualism, as a central tenet of neoliberalism, concerns individualism and individual autonomy more generally in market interactions. The key feature of market individualism is the belief that responsibility for market behaviour and outcomes does not lie with institutions, but rather with individual actors. Market liberty refers to the freedom of markets and freedom to participate in markets. Finally, human capital captures the idea that individuals develop a skillset for market competition. The idea behind is one of “entrepreneur de soi”.
For instance, in the UK CITAE working class Focus Group 1 excerpt below (Vila-Henninger and Dupuy, 2021), we see an example of “Market Liberty” applied to the policy of free movement in the EU and, specifically, its policy rationale. Robert discusses labour migration that is allowed by the EU circulation policy and he considers that it is positive and fair since migrants from Eastern European countries can come to Western Europe and “do good”. We also see the intersection of “Fair Market Competition” and “Market Individualism” in the excerpt from Brenda. She argues that immigrants who come to the UK cause “their countries[to] start to go downhill [because] like nurses come over here to work because they get more money what about their country.” Thus, in her mind, competition in the labour market enabled by the EU free movement policy and its embedded principle of market liberty, is unfair because it creates a “brain drain”. She brings in her perception of employment policy and suggests that the brain drain is the result of better conditions of employment, in the form of more favourable policy instruments of wage regulation, namely ‘more money’. We see “Market Individualism” here as well because Brenda blames the brain drain on the individual migrants and their decision to migrate–thus shifting responsibility for policy outcomes onto individuals.
Robert: but yeah that is true but I do think there has been a lot of positives from all the countries becoming part of the EU. There have been many positives you know for poorer countries (Brenda looks at Mary and smiles) like the borders have opened so people from tradesmen from the poorer countries can bring their trades over here and do good. You know what I mean, work so there is positives as well as negatives. You know what I mean? Brenda (to Robert): But their countries? They come here to do work yeah but what happens when their countries start to go downhill? Their people who have got like you know like nurses come over here to work because they get more money. What about their country?
Secondly, our empirical analysis proceeds by assessing whether the normative (de)legitimations interviewees draw upon while discussing their policy experiences and perceptions are indeed collective. Here, data collected through focus groups supports our investigation of shared norms as the focus group method enables the analysis of co-constructed meaning based on participants’ recorded interactions (Vila-Henninger and Dupuy, 2021). In the extract above, Robert highlights that the EU free movement policy brings positive outcomes and Brenda disagrees by pointing that some of its outcomes are unfair as free movement results in a competition in the labour market which she deems unfair. Later in the interview, Robert and Brenda, along with other participants, keep on discussing this issue of the competition in the labour market enabled by the EU free movement policy. Mary echoes Robert’s previous point by suggesting that English workers could go to Poland providing that they have good skills, just like Polish workers come to England – which she seems to assess positively. Participants then discuss language requirements for labour market participation. Robert notices that Polish workers have a job despite not speaking English. He then picks up on Brenda earlier comment regarding wage regulation and says that, regardless of the difficulties of the Polish language for English speakers, English workers would not want to migrate to Poland because “the money’s so bad.” These respondents argue that there is unfair competition between Poles and the English in the labour market because Poles have access to the Polish and English labour market, while according to participants, it is not economically viable for the English to migrate to Poland. In the course of the interview thereby, not only do Robert and Brenda come to build a consensus on the unfairness of the competition created by the economic liberty of the EU, but this norm of fair market competition forms the baseline of the conversation between participants as illustrated by Mary’s, Robert’s and Brenda’s contribution to the discussion.
Robert: That’s true but most of the tradesmen over here, they’re from Poland but can’t speak English but they still get jobs though.
Brenda: I don’t think I could work in Poland. The language is quite hard.
Mary: It depends what you want to do.
Brenda: There’s not many English people. You won’t find many English people in Poland.
Robert: That’s because the money’s so bad
Mary: If you were skilled at something, you’d be able to do it over there because they come over here and
Brenda: It depends.
Robert: But they won’t do it because of the money. The money’s so bad that’s why they are coming over here.
By studying the underlying normative assumptions of the discussion and how some of them build consensus, we can access the normative legitimations they share regarding the EU free movement policy – in that case, they agree on the fact that this policy should allow for a fair market competition, while they disagree on the extent to which this is what actually happens. More generally, because most of our data are focus group data, we have access to the negotiation of meanings through intra- and inter-personal debates (Van Ingelgom, 2020), thereby grounding our analysis of shared meanings and legitimations – while also allowing individuals to express (de)legitimations that are not shared by other research participants.
However, while the nature of data is well suited to our investigation, a consensus among research participants in one focus group and, in this case, a single policy, is not enough to evidence that a collective norm linked to multiple policy experiences and perceptions exists. We discuss in the next subsection how we leverage our comparative design to do so.
Leveraging our comparative design to empirically evidence collective norms linked to multiple policy experiences (F)
In our research, we investigate the normative effects of citizens experiences and perceptions of policies which include neoliberal components – we use the plural form as our research is sensitive to the diversity of collective norms pertaining to the role of the market that may emerge from our empirical analysis. This endeavour and, concretely, evidencing collective norms linked to multiple policy experiences, raises several questions, that we addressed with specific comparative designs. We discuss them in turn.
First, there is the question of how collective the norms co-constructed through policy experiences and perceptions are. In other words, what is the group the normative mechanism applies to? In feedback studies on mass publics, comparative research designs resting on the comparison between groups of recipients and non-recipients, or between groups of recipients experiencing different policy design features (e.g. levels of conditionality, rights-based vs. conditional) are very common. As they introduce variation on the main explanans, citizens’ experiences with a policy, they allow drawing inferences from its main features to individual attitudes. For instance, Watson examines how the conditionality of welfare programmes impacts on a range of political attitudes through the interpretive effect (2015). The implementation of conditional programmes relies on face-to-face interactions between caseworkers and recipients, where the compliance of the latter to specific attitudes is assessed by the former. Her study is not only set to investigate conditionality in general, but also to disentangle two dimensions of conditional programmes: the structure of authority (which pertains to the imbalance of power between caseworkers and recipients, and the related discipline and sanctions) and the stigmatization of target populations (which is higher when the programme is means-tested rather than contribution-based). In that effect, her study relies on two comparative designs: one of citizens’ experiences with policy programmes that are and are not conditional; and one of citizens’ experiences with conditional programmes that do and do not stigmatize their target populations.
When studying the normative effect, the empirical analysis is oriented towards looking for commonalities, shared norms, across social groups, rather than identifying differences between policy recipients and non-recipients as scholars investigating policy feedbacks in the form of resource or interpretive effects do. Thereby, based on the empirical identification of neoliberal legitimations and delegitimations relied on by participants in each interview or focus group, as discussed in the previous section, we implement a comparative strategy whereby we compare interviews and focus groups in each of our primary studies. We establish the existence of a given collective norm when it is shared among participants across key economic, social, or political divides (Vila-Henninger, 2017; Svallfors, 2006; Sachweh, 2012). This approach is instrumental in evidencing that in one location at one point in time, research participants across the main economic, social and political divides in our datasets share specific normative beliefs in relation to their policy experiences and perceptions. The fact that our main datasets are composed of interviews and focus groups with participants from different socio-economic backgrounds and political leanings serves our purpose well. For instance, in the CITAE data, there are three occupational categories: managers, white collar, and working class in each country case. In that respect, a collective neoliberal norm is evidenced only when specific neoliberal legitimations and delegitimations are found in working class and white collar and managers in a given country. Of course, the norm can be challenged and there can be dissensus – as our previous example demonstrates.
Second, the empirical study of the normative policy feedback raises the question of its scope: is it limited to a circumscribed set of policy experiences and perceptions or is it more pervasive and covers a larger range of these? At stake here is how various policy experiences and perceptions may combine in citizens’ everyday life to convey and support the construction of shared collective norms beyond a specific policy area. Collective norms about the role of the market are a typical case of norms that are conveyed by policies across policy domains – which grounds the study of their cumulative effect through the policy experiences and perceptions that citizens have. Methodologically, this calls for another layer of comparative approach: one that compares specific neoliberal legitimations and delegitimations across policy areas to establish their pervasiveness.
Taken together, the social-group and policy-sector comparisons allow to build evidence of a given normative policy feedback linked to a specified set of policy experiences and perceptions in one place at one point in time. The third and, final, dimension of our comparative design is to include variation in country cases and across time. In our research, we selected three country cases, Belgium, France and the United Kingdom, that vary along three main dimensions: our explanandum, namely the scope, nature and features of citizens’ disaffection toward politics; and our main explanans. Regarding neoliberalism, the case selection allows for the variation of its timing and differentiates between the rise of neoliberalism in the late 1970s and 1980s in the UK, and its subsequent diffusion and consolidation in the 1990s and thereafter, in France and Belgium (Hay, 2007; Schmidt and Thatcher, 2013). The first wave is marked by a highly politicized nature and may be referred to as normative neoliberalism, while the second wave is characterized by institutionalization, normalization and depoliticization and may be depicted as normalized neoliberalism (Hay, 2005, 2007). Regarding supranationalisation, in the European context, it has mainly taken the form of the Europeanization of public policies. The level of Europeanization varies greatly between countries and between policy areas: some of them are highly Europeanized, while others are very little Europeanized (e.g. Brouard et al., 2012). Also, the Europeanization of public policies as been diversely discussed in public debates in our three cases (Graziano et al., 2011).
The comparison over time, between the four time points that our main corpus includes, is crucial for us to document the emergence and change of specific collective norms linked to policy experiences and perceptions as well as their possible persistence over time in a given country case or across country cases. We thereby treat the different points in time to have a diachronic perspective and, thus, to leverage to possibilities of secondary qualitative analysis. We thereby approach time in a chronological way and consider each dataset, or each country-specific dataset in the case of comparative primary datasets, to be reflective of the policy context of that time and place. In that respect, we follow a classic strategy in feedback studies on mass publics where changes unfolding over (periods of) time represent a commonly analysed dimension of variation in the macro-level context. The inferences from such temporal comparative designs also provide direct variation on the main explanans – which is needed to demonstrate similarities and differences across settings. It is not an exclusive strategy as it is often combined with a cross-policy comparative design.
Overall, these three dimensions of comparison are key in our empirical strategy of both (re)contextualisation – how data express aspects of contexts of their production – and (re)connection – how data might be used to speak beyond such contexts (Hughes et al., 2022). Moreover, pondering evidence of similarities and differences along and across them is crucial in our inference-building endeavour and make statements about the empirical validity of our proposed normative mechanism. We rely also here on a classic comparative logic and build on theoretically meaningful patterns of commonalities and differences along and across dimensions (Bennett and George, 2005). Specifically, our operationalization of the (neoliberal) normative feedback mechanism rests on two main steps: first, the identification of specific (neoliberal) legitimations or delegitimations of policy experiences or perceptions in interview data and, second, the identification of these specific legitimations or delegitimations across the main socio-economic divides in a country at a point in time, across time and cross-nationally.
Conclusion
In this concluding section, we discuss the contributions of our approach to respond to the methodological challenges raised by calls to theoretically and empirically expand feedback studies on mass publics (Campbell, 2012; Béland and Schlager, 2019). Our article presents how these calls, while changing the core research questions and the empirical scope of the analysis, urge to reconsider the methodological roots of the policy feedbacks literature. We show how acknowledging the multiplicity of citizens’ policy experiences and perceptions in Western Europe challenges the research designs and the causal inferences that are common in this scholarship – which is mostly US-centric and largely focuses on single policy experiences (Bussi et al., 2022).
Our questioning of the core methodological choices of mainstream feedback studies on mass publics, as well as how we tackled both challenges to research designs and causal inferences, derive from our leading question. In our research, we ask how the turn to neoliberal policies and supranationalisation have affected citizens’ relations to politics and, specifically, are related to their (dis)affection toward politics over time and in different countries. In line with mainstream feedback studies, we thereby consider that “new policies create a new politics” (Schattschneider, 1935) through citizens’ policy experiences and perceptions. However, some of our methodological choices depart from this strand of scholarship (alongside other scholars Hern, 2019; Bruch et al., 2010; Rosenthal, 2019, 2021b, 2021a; Mettler, 2018).
This article makes three methodological contributions to feedback studies on mass publics. First, we develop an approach to the study of citizens’ lived, multiple, policy experiences and perceptions and, thereby, supplement single-policy research designs. We conduct a qualitative secondary analysis of datasets that are free of policy selection – while presenting other defining features that suit our research purposes. We also elaborate on a coding strategy to operationalize participants’ policy experiences and perceptions in interview data, both individual and focus-groups, with no policy prompts. Our coding is abductive (Vila-Henninger et al., 2022). Second, to tackle the inferential issue raised by the study of citizens’ multiple policy experiences with policy-specific mechanisms, we elaborate on the empirical study of a normative mechanism (Svallfors, 2010) that captures how multiple policy experiences and perceptions may co-construct collective norms. We present our empirical strategy for studying norms in interview data and evidencing policy-related collective norms based on a longitudinal, cross-national and social comparative design.
At the time of writing this conclusion, our research has two important results. First, in line with Andrea Campbell’s 2012 landmark article, we observe that citizens do in fact encounter multiple policies in their everyday life (Dupuy, Van Ingelgom and Defacqz, 2022). Research participants do discuss a large range of policies in interview settings which were set to analyse their relations to the EU and which excluded policy prompts. In each dataset, the entire range of policy domains is covered in participants’ discussions, with a few exceptions only – making clear that public policies matter a lot in their political discussions. Second, we demonstrate how this new approach to citizens’ multiple policy experiences and perceptions is particularly relevant for Western European cases which are policy-ridden. In Western European countries, citizens have a variety of policy experiences over their life course. But our analysis also demonstrates that policy perceptions are as consequential, if not more, as policy experiences once citizens’ whole daily life and multiple policy encounters are considered. Our research participants discussions are full of policy perceptions. This comes in sharp contrast to established results from single-policy studies which zero in on citizens’ direct experiences with a policy. Overall, as we elaborate on elsewhere (Dupuy, Van Ingelgom and Defacqz, 2022), these multiple policy experiences and perceptions ground normative, that is shared, understandings of the political community and politics in general.
Our approach is grounded in qualitative secondary analysis as we have used already existing qualitative datasets. Our third contribution is to introduce qualitative secondary analysis in feedback studies. Drawing from Hughes, Hughes and Tarrant (2022), we present our mode of engagement with secondary qualitative data. We account for our collective and configurative approach, by which we harness our epistemic and temporal distance to the primary datasets in order to raise new research questions and generate new empirical, analytical and theoretical insights from secondary qualitative data. This approach is also key to the process of (re)contextualization of data that we analysed. The development of qualitative secondary analysis has generated large debates on the new research possibilities it may offer (Davidson et al., 2019; Duchesne, 2017; Hughes et al., 2021; Hughes et al., 2022; Vila-Henninger et al., 2022) and this article intends to foster them from the perspective of feedback studies.
Admittedly, in our own research, the study of citizens’ lived experiences and their feedback effects is comparatively costly, both in terms of data and human ‘costs’ in the coding and analysis phases specifically. However, the methodological approach that we elaborate on in this article is not limited to secondary qualitative analysis and could be applied to primary research as well. While this article has certainly not settled each and every methodological challenge raised by Campbell’s and Béland and Schlager’s respective call, it opens new avenues of research that are worth considering to improve our collective understanding of existing democracies and, in particular, citizens’ broad disaffection towards politics.
Footnotes
Acknowledgements
We would like to thank the primary researchers, Céline Belot, the CITAE team, Heidi Mercenier and Laurie Beaudonnet and the RESTEP team, for trusting us enough to give us access to their data. We also thank the Qualidem’s Team members Margherita Bussi, Mauro Caprioli, Cal Le Gall, Damien Pennetreau, Thibaut Rioufreyt and Luis Vila-Henninger for their work on the project; as well as the Micro-Macro workshops participants, Camille Hamidi, Marine Bourgeois, Francesca Quercia, Boris Attencourt, Pierre Wokuri and Vincent Lebrou. A warm thank you to the two anonymous reviewers for their insightful and rich suggestions and comments. Last, but not least, we would like to specifically thank Sophie Duchesne for her support and for inspiring our work.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the project ERC Starting Grant Qualidem – Eroding Democracies. A qualitative (re-) appraisal of how policies shape democratic linkages in Western Democracies. The Qualidem project is supported by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement 716208). This article reflects only the authors’ view and the European Research Council is not responsible for any use that may be made of the information it contains.