Abstract
This article traces the evolution of the French policy PISA debate from 2001 to 2014 by analysing the results of two original qualitative researches. Theoretically, this debate is the outcome of specific policy configurations, which predetermine its scope, content and effectiveness. These configurations are themselves described through their political, institutional, professional and cognitive dimensions. Thus, three configurations are highlighted, all of which favour different public discourses: the Programme for International Student Assessment (PISA) is seen as a topic only interesting to insiders (2001–2004), then as a politicised ranking (2005–2010) and, finally, as a tool purely for communication (2010–2014). The article shows that the major effect of PISA on the French education policy debate was to confirm and legitimate pre-existing opinions and policy stances, and that PISA was progressively scooped out from its statistical content.
The effects of the Organisation for Economic Co-operation and Development’s (OECD) Programme for International Student Assessment (PISA) on domestic policy processes have been intensively scrutinised by scholars.
Six analytical perspectives, often interconnected, can be distinguished. Several authors focus on the “PISA shock” on various education systems. For one group of scholars, this shock may be regarded as a taken-for-granted piece of context to better analyse the effect of the survey on the development of national standards or curriculum reforms (e.g. Tveit, 2014). Another group compare the scope and the effective “impact” of this shock on various national education policies (e.g. Breakspear, 2012).
With the ambition of systematicity, other scholars provide typologies of “policy responses” or “reactions” to PISA in the continuation of Rita Steiner-Khamsi’s pioneer article (Grek, 2009; Steiner-Khamsi, 2003). Others analyse the media coverage so as to question the functioning of the media through a critical discourse analysis (e.g. Stack, 2006) or the circulation of policy models such as the “Asian Tigers” (Waldow et al., 2014). Other articles stress the diversity of media coverage from one country to another and its uneven impact on political leaders (Dixon et al., 2013).
Numerous authors wonder if the spread of PISA corresponds to the development of an evidence-based policy (EBP) in education. For instance, on 9 November 2015, 583 contributions registered in the Education Resources Information Center (ERIC) database mentioned both PISA and EBP in their abstract. Most of them were written by OECD experts or by specialists of globalisation in education. Their aim is to question either the methodological foundations of PISA, its political meaning or the implications of its development on governance.
Academic researchers also insist on the transformations of the State and the governance of education favoured by international comparisons like PISA (e.g. Meyer and Benavot, 2013). The results of this research may be found, for instance, in the “Transtate” project (e.g. Martens et al., 2014). The theory of the “comparative turn” is often quoted. It initially designated the evolution of the OECD itself but, progressively, it was used to conceptualise the changes of governance as a whole, which are supposed to be more and more based towards international comparisons. Several national cases are analysed within this framework; for example, France, which would be allegedly inspired by Finnish pedagogical methods and policy instruments (Dobbins and Martens, 2012). Lastly, the literature provides detailed national case studies in which authors scrutinise PISA uses in various policy spaces. If we look simply at English-speaking international literature, a high number of countries were already covered by such kind of analysis: Australia, China, Denmark, Finland, France, South-Korea, Germany, Japan, Spain, Russia, Turkey, etc.
This article follows the methods which correspond to that sixth perspective. It analyses how the French policy PISA debate evolved between December 2001 and September 2014. France is an interesting case for three reasons. First, internationally, it is a country often regarded as resisting globalisation. It is visible in its claims for “cultural exception”; its regular requests for translations of international documents in French; its uneven participation in international meetings; the methodological criticisms levelled at the first international consortium at the origin of PISA; and its attempt to promote an alternative methodology for international comparisons. Thus, France may constitute an interesting least-likely case: if PISA, even in this case, is used to reorient domestic policies, it means that the power of this survey is particularly strong. Second, policy reactions and lessons from PISA are all the more expected to happen since France has had average decreasing results since PISA 2000 and similar countries like Germany have implemented major structural changes. Third, France provides an interesting form of knowledge production and expertise distribution in the policy process. For deep historical and institutional reasons, “state sciences” (Pons, 2013) produced by central administrative elites remain powerful. This not only explains why France had already developed its own evaluation tools, but also why the adoption of PISA was only progressive.
This article shows that PISA had a major effect during the period 2001–2014 to confirm and legitimate pre-existing opinions and policy stances. My hypothesis is that this trend, identified in other European countries at least between 2001 and 2008 (Pons, 2012), is the outcome of specific policy configurations that predetermine the scope, the content, the salience and the effectiveness of public speeches on PISA.
That is why, after presenting my theoretical framework and my methodology, I distinguish three main periods in this policy debate. For each of them, I synthesise what is said about PISA and show that it can be understood as the outcome of a specific policy configuration. In the final section, I explain the form and the dynamics of these policy configurations via four types of factors: political, institutional, professional and cognitive.
Public speeches on PISA as policy configuration outcomes
My theoretical framework is based on three main notions. The first is that of policy debate, which I define as the study of a policy problem, which leads to a public discussion between actors with different opinions. This definition implies that there is a constitutive tension in a policy debate between the necessity to improve knowledge on the issue and the necessity to convince people with other opinions, sometimes by using argumentative strategies which hamper the detailed examination of the issue: simplification, selective omissions, metaphorical shortcuts, mastery of the sophist kayros evoked by Bourdieu (1984), etc. Such a tension may be stronger when the debate is public since the number of actors with conflicting visions may increase and specific frames for public speeches which favour a more or less deep examination of the problem may be introduced. This public character does not refer here to a theory of the public space but to the imperative to study, empirically, the changing frontiers of new “mosaic public spaces” and the kind of speeches that they enable (François and Neveu, 1999).
Public sayability is the second notion; that is, the possibility to say something publicly. As Bourdieu showed, public sayability is socially constructed: when we study a language, at the same time we study the social and linguistic conditions of the acceptability (and sometimes effectiveness) of this language. Nevertheless, Bourdieu’s approach towards language as an act of power is strongly influenced by the paradigm of (symbolic) domination. This paradigm is itself highly dominant in political science and invites authors to focus on the “competencies” of people to speak publically, on the “social hierarchies” of public speeches, on “self-censorship” and on the new dominant “registers of language” in the public space (François and Neveu, 1999). Yet, this vision is not the only way to conceive power, especially in a policy process in which there may also exist more situated, punctual, unstable and rapidly reversible relations of power. That is why I approach public sayability as the outcome of specific policy configurations.
Policy configuration designates a particular set of factors within the policy process that shape specific interdependencies between policy actors (Elias, 1993). These policy configurations define the scope of public speeches, their content, salience and effectiveness. In this approach, power is not an attribute that we possess but the ability to act on each other in a reciprocal social action (Simmel, 1981). One major theoretical challenge is to capture the factors that shape these interdependencies. The initial factors are political and refer to the effects of politics on policy: political circumstances, struggles between political forces and the shaping of political leaders’ agendas are key elements to understanding the public stances of policy actors on a topic. Other factors are institutional. By defining the formal rules of the games, institutional designs strongly predetermine the roles and routines of policy actors, their margins and spaces of liberty, and their degree and form of coordination. Others, again, are professional: the way in which policy actors define their job, embody it in specific competencies and struggle for it plays an important role in their involvement in the policy process. The final factors are cognitive and designate all the representations, ideas and pieces of knowledge that policy actors may share.
Two research devices
This article is based on materials drawn from two qualitative research studies. The first was conducted with Nathalie Mons in 2008–2009 for the Know&Pol project (www.knowandpol.eu; see Mons and Pons, 2009). We conducted 30 interviews with people who participated in the implementation of PISA in France and in the public debate about it between 2001 and 2008. We consulted the institutional and professional literature on PISA (public reports, articles in professional reviews, handbooks) and we made use of various bibliographic databases (PERIOD, ERIC, INIST and HAL-SHS). Research on publications issuing from unions, political parties and from websites was also conducted. Lastly, we carried out a press review which cross-checked a dataset of 270 press dispatches published from May 1999 to January 2009 using a press agency which specialised in education (Agence éducation et formation (AEF)), a dataset of 140 press articles published in 20 papers (national and regional, general and specialised) between November 2001 and January 2009 and a press review carried out by the Ministry of National Education itself.
The second qualitative research study comprises ongoing research into the morphologies and dynamics of policy debate in education in France, which I started in 2012. It consists of tracing, from the end of the 1990s onwards, the public debates on three significant policy issues, including PISA, in various fragments of the mosaic public space on education and to analyse how specific policy configurations shape public sayability on these issues. I focus more precisely on four fragments: political, institutional, media and academic. Each fragment is studied on the basis of one key source of information which has been previously identified for its relevance and the continuity of its data. This source is then cross-checked with interviews with policy actors and with the key sources of other fragments (Table 1).
Sources and datasets by fragments of the mosaic public space.
CAIRN, ERIC, IBSS, L’Harmathèque, Persée, PubPsych and Revues.org.
As they were not constituted for the same purpose, these two research devices are not totally symmetric. That is why I only take into account here the data that are common: Parliamentary debates, press articles in national and general newspapers, AEF press dispatches and scientific publications. On the contrary, all the interviews are mobilised for the analysis, even if others are still planned.
An insider affair (2001–2004)
Between 2001 and 2014, the policy debate on PISA moved through three main periods. The first one stretched from 2001 to December 2004 and was characterised by very few public speeches on PISA. Hence, the policy debate hardly went beyond the circle of policy actors who were pre-informed of its existence.
A narrow debate focused on methodology
In Parliament, this survey was very little discussed by deputies. It was essentially evoked in written questions from deputies of the right-wing majority who asked for additional pieces of information. Answers were, in general, the opportunity for the government to give more details on some findings, to legitimise current actions and to reassure its majority in Parliament.
The Ministry of National Education, especially through the department devoted to statistical analyses and evaluation studies (Direction de l’évaluation, de la prospective et de la performance, the DEPP), organised two press conferences in the period to present the official results of France and published these results in official documents. They insisted on the average position of France, on the needs to have further studies to confirm some trends and on the numerous methodological limits of the survey (like, for instance, the particular conception of the sample in France).
In the media, only five journals reported the results of PISA 2000. The headlines denounced the low level of French pupils, for instance, through classical metaphors. This limited coverage was, nevertheless, relatively detailed and technical. The indicators that were presented went well beyond the simple ranking and evoked the level of pupils with learning problems and the scale of academic inequality as it related to social origin. Typical French pupil behaviour that might explain these mediocre results was also singled out: the fear of responding when one does not know the right answer beforehand, for example, or the academic character of pupils who expect to be assisted in formulating questions and rarely show initiative. No one outside of the survey was interviewed, with the exception of the Ministry of National Education leaders ordered to explain what was seen as disappointing results (Mons and Pons, 2009).
General inspectors (GI) made little use of PISA conclusions. As a whole body, they organised no internal review concerning this study, nor did they transmit to the whole group the particular experiences of certain members. The very occasional initiatives taken were on an individual basis and undertaken by members whose previous work experience had already made them aware of international problems.
PISA’s first wave was marked by the near-total absence of a response on the part of the unions. Only two unions addressed the subject during the period. Starting with the survey’s first cycle, the Fédération Syndicale Unitaire (FSU), a left-wing federation of unions, began to examine the manner in which the survey had been conceived and the uses to which it might be put, including its eventual role in formulating policy. In January 2002, the federation organised a colloquium which brought together a representative of the OECD and four DEPP experts. The discussion principally concerned the methodological limits of the survey. In February 2004, the Confédération Française Démocratique du Travail (CFDT), a “reformist” secondary school union, also seized upon the OECD’s study and the secondary analyses developed by researchers in order to support its educational policy proposals. The union claimed in its contribution to the “Debate on the Future of the School” that the conclusions of PISA and those of scholars converged, demonstrating the need for supporting a single school in which student orientation would be put off for as long as possible (Mons and Pons, 2009).
Lastly, the survey did not really interest scholars beyond those among them who had already been solicited by the DEPP. Three kinds of scientific outcomes can be noted here. Members of the DEPP published articles in academic journals such as Psychologie et Psychométrie, and the two other major lines of research were favoured by a call for proposals ordered by the DEPP itself. An initial series of studies aimed first to analyse the PISA survey in order to better understand the tool, its interest and its limits. The University of Paris VIII was, for instance, invited to identify the social, socio-cognitive and, more generally, cultural dimensions of items and student responses, and to question the models of socialisation that were presupposed by the international evaluation. The second line of research tried to link the characteristics of educational systems to student performance to influence the elaboration of education policies. Several researchers from the Dijon-based Institute of Education (e.g. Denis Meuret and Nathalie Mons) actively contributed to this kind of questioning (Mons and Pons, 2009).
Collective ignorance as a policy configuration outcome
This collective ignorance of PISA is the outcome of a specific set of interdependencies between national policy actors (Figure 1).
Policy configuration 1. Towards a collective ignorance of PISA.
This ignorance came first from the stance adopted by the Ministry of National Education towards PISA 2000. Indeed, the results were published at a very particular political moment (December 2001), coming just one year after the departure of Claude Allègre, a turbulent Minister whose views gave rise to numerous protests by teachers, and a few months before the 2002 presidential election. The new Minister, Jack Lang, had to reassure teachers to count on their votes for the next elections. He had nothing to gain from fostering teachers’ mistrust by drawing upon a survey that showed average results and that was regarded by some influent unions of teachers, and through them some Left deputies, as the temple of neoliberalism in education. Moreover, the DEPP, which failed to strongly influence internationally the conception of PISA, which disagreed with several methodological choices and which also needed to reassert the centrality of its expertise after the Allègre’s challenging ministry, appealed for precautions of analysis.
Thus, the cabinet and the DEPP immediately agreed to issue a statement concerning the PISA results: France’s average but honourable result had to be underscored, as did the methodological biases of the survey. The objective was twofold: neutralising the negative effect produced by the end of illusions concerning the ‘French exception’ and supplying a legitimate explanation (i.e. methodological biases) that also served to discourage actors from looking more thoroughly into the questions raised by the survey.
Given that the Ministry of National Education has decided against making PISA a central element of its policy, the media – whose journalists did not specialise in education and had little training in statistical reasoning in any case, and who were also still discovering this new survey and strongly inclined to stick to government current affairs – have since been hardly encouraged to do more than simply mention the general results of the survey. Indeed, together with the teachers’ unions – for whom the burning policy issues were elsewhere at that time – evaluators (the Ministry’s general inspectors and statisticians) and scholars have generally given little attention to the conclusions reached by PISA. This has only further discouraged media interest in the survey, which limited the extension of the policy debate to other policy actors.
A politicised ranking (2005–2010)
This configuration changed in January 2005 and the policy debate on PISA became more political. It was increasingly linked with the implementation of a specific political offer coming from the ruling right-wing party, the Union pour un mouvement populaire (UMP; renamed Les Républicains in 2015). It moved progressively to the question of the expected reforms and through them, that of the values and goals of the education system.
Politicisation and extension of the debate
From January 2005, the right-wing government progressively put PISA at the heart of its political communication. It was visible during the preparation of the new Education Act (Fillon law, July 2005). PISA was explicitly mentioned in the motives at the basis of the new act to illustrate the “breathlessness” of the French education system, its weak democratisation and its average performances. Several deputies from the Opposition reacted to this presentation but their contestation was not really effective. For instance, subtle counter-arguments from some socialist deputies, arguing that French pupils failed to mobilise knowledge in new situations, which did not correspond to classical school exercises, were often discredited by the Majority because they were linked in the discourses of their opponents to former experiences launched by the Left which, in their opinion, had failed. PISA was then heavily utilised by the Right during the annual examination of the State budget in Parliament to justify the idea that more means were not correlated to better results. This argument served to legitimise cuts in public spending and its consistency was rarely contested by the Opposition. The survey was also regularly quoted by right-wing political leaders when they had to prepare the 2007 presidential elections, and in numerous announcements made by the government. The communication of the Ministry of National Education around the PISA 2006 results adopted, for instance, an alarmist tone to better introduce the need to reform primary and secondary education. Another example is that the methodological criticisms that were levelled during the former period were pushed in the background (Mons and Pons, 2009).
This new position of the Ministry of National Education had a direct impact on the work on PISA by the DEPP, as some of its publications show. The latter insisted less strongly on the particular character of the survey and only referred to contextual elements concerning education in France in order to explain certain results, particularly in mathematics. The members of the national group formed by the DEPP to analyse the French results underscored the “complementary” look of the OECD’s approach even as they foregrounded the pedagogical dimension in their interpretation.
The politicisation was also visible in the greater union responsiveness to PISA results. For example, the Union nationale des syndicats autonomes (UNSA) interviewed several times in its review scholars or senior civil servants who used PISA in their works. Many of them had worked in the DEPP. Their statements were drawn upon to supply know-how for the union’s arguments in favour of a group of reforms intended to increase equality within the educational system. The survey was also used as a tool for denouncing government policy. In contrast to PISA 2000, only two unions adopted positions denouncing the methodology of PISA 2003. The conservative Syndicat national des lycées et collèges (SNALC) union, which favours the suppression of the single school, called the study into question (the skills tests are not taught in France, Multiple choice questions (MCQ) disadvantage French students who have little experience in this type of evaluation) together with the Finnish model (Mons and Pons, 2009).
The media coverage of PISA was also more political. Journalists tended to give more exposure to new interlocutors. Among them, the OECD French-speaking experts saw their audience increasing and they started formulating policy recommendations. Articles tended, especially with PISA 2006, to concentrate on the country’s drop in the standings and the increase in the number of students experiencing difficulties in school, to better introduce then the question of the policy solutions that must be contemplated. Journalists used more and more foreign examples, especially the German one, to plead for a “PISA shock” and to stress its surprising absence in France. Lastly, scholars in pedagogy or in international comparisons, who specialised in the domains assessed by PISA, were invited to provide policy solutions.
This renewed attention to the survey also came from general inspectors – those from the disciplines covered by the survey started to investigate items. They regularly sent reports to the chiefs of the inspectorate, frequently mentioned PISA in their public reports and sometimes explicitly drew a connection with the recurrent themes of French educational policy, such as the problem of holding students back. The discrepancy of French students’ results in PISA, depending on whether or not they were held back, as well as the successful example of foreign countries that did not hold students back, were thus used to demonstrate the ineffectiveness of this measure in various forums. These initiatives gradually spread through the entire inspectorate, which organised in 2008 an annual general meeting on international comparisons (Pons, 2010).
Lastly, an academic debate progressively emerged with a distinction between those who still evaluate the methodological foundations and possibilities of PISA and those who regard it as an indicator of performance to better build international comparisons of education systems. The Revue française de pédagogie devoted two issues to international comparisons in 2006 (no. 157) and 2008 (no. 164), which addressed both dimensions. Several magazines linking researchers with teachers (Alternatives économiques) or with experts and administrators (Revue internationale d’éducation) took over the researchers’ conclusions to fuel the academic debate on PISA results. This movement reached its peak in 2009 with the publication of a book by Baudelot and Establet (2009). After asserting that the methodology was beyond reproach, the authors exploited PISA results to show the numerous defects of the French system: an inefficient early selection, the harmful effects of repeating years, the gap increase between the elite and students in difficulty and the social predestination of students.
The ripple effects of a new political offer
This inflexion of the policy debate is the consequence of a change of the policy configuration at its origin (Figure 2).
Policy configuration 2. Towards a politicised ranking.
From 2005, the political context evolved. At the European level, the implementation of the Lisbon strategy led member countries to identify three principal objectives for all educational systems and assign them various benchmarks. One of these was explicitly based on the PISA survey. In France, the Loi organique relative aux lois de finances (LOLF), a new finance law that came into force on 1 January 2006, implied an inventory of the production of statistical indicators and a series of audits to improve Ministry of National Education’s organisation and results. These two factors converged in the preparation of the Fillon law. This law is in conformity with the political offer that the Right-wing party has progressively shaped since 2001. This offer focused on the “success factors” highlighted by PISA (quality of teachers, pedagogical freedom, autonomy and evaluation of schools).
This new orientation had major effects on policy actors whose activity is directly linked with government’s current affairs. Unions had to train quickly and build specific policy stances to oppose themselves to the government. Some of them reactivated the arguments made by the DEPP on the methodological limits of the survey, while the latter had to reconsider its position. General inspectors had to investigate deeply this new survey massively quoted by “their” minister to remain an influent advisory body and not to be marginalised in a restructuring field of expertise. PISA was all the more quoted by journalists between two PISA sessions, who would comment on current policy issues, carry out international comparisons and plead for a major “PISA shock”. This movement also affected social researchers a second time, who were expected to confirm or invalidate comments on the survey and policy statements done on its base.
A commonplace tool of political communication (2010–2014)
From 2010, this politicisation led to a “trivialisation” of PISA, that is, a process which ended up with a lesser focus on its statistical content and more on its progressive merger with ancient debates like the “crisis” of education or the importance of social inequalities.
Policy inferences rather than knowledge productions
From 2010, the contributions by policy actors who are likely to fuel the debate with pieces of knowledge based on the survey dropped. The DEPP and the general inspectors intervened less than before. The DEPP only published its traditional note on French results and external interventions of its members were limited, as were those from general inspectors. Despite the relative success of the book by Baudelot and Establet (2009), scholars did not really investigate international comparisons through PISA and the contextualisation of the results obtained by various countries. 1 The academic debate focused on the global syntheses of this survey (e.g. Felouzis and Charmillot, 2012) and on its effects on the reorientation of French education policies.
Conversely, political speeches increased. They came first from the government. PISA was used massively by the right-wing minister Luc Chatel (2009–2012) to legitimise regular cuts in public spending (on the grounds that PISA showed the lack of correlation between means and results) and other policy initiatives: the reassertion of basic skills, the struggle against illiteracy, the reform of priority education, new evaluation devices, etc. PISA was also regularly quoted by the left-wing minister Vincent Peillon (2012–2014) from the Parti socialiste (PS). But it was done so as to legitimise other policies such as the change of schedules and learning rhythms of the pupils, another reform of priority education or the reform of pupils’ evaluation by teachers. These speeches also increased in Parliament (Figures 3 and 4).
Number of speeches on PISA in the National Assembly according to the left–right cleavage (2000–2014).
Number of speeches on PISA in the Senate according to the left–right cleavage (2004–2014).
The Right majority pleaded for the reduction of public spending, the individualisation of education offered, school autonomy and the development of new evaluation tools. The Left used the example of successful countries to ask for more money for disadvantaged pupils, better training of teachers and the end of the four-day week in primary education. It argued that countries with early childhood education better succeed in PISA and put forward the low staff–pupil ratio in comparison with other OECD countries, the under-provision of primary education, and the insufficient training and payment of teachers. The Right majority was progressively contained in the defence of its former policy on the basis of a communication tool that it actively contributed to the debate. It was deprived of one of its main sources of expertise. This dispossession was all the stronger after May 2012 since the new Left majority defined its governmental program “La Refondation” as the French version of the PISA shock.
The media both relayed political speeches and produced them. If an important platform was given to government’s speeches, the media also initiated a massive benchmarking by providing many articles on foreign education systems. France was classically compared to its neighbouring countries (like Germany or Italy) and to expected top performers (like Finland or Asiatic countries). They rarely provided contextualised and detailed analyses of the success of these countries, but rather developed various stereotypes: the obsession of order in Switzerland, the docility and lack of imagination of pupils in China and the strength of the German people in being able to stand up again after the PISA shock, as did this nation after Nazism. Moreover, within the quality newspapers, the editorial writers appropriated PISA to support their own diagnoses on the French schooling system and their own policy orientations. Lastly, an increasing number of media intellectuals or personalities used PISA to legitimate their repeated policy stances, as did several representatives of think tanks such as the “Institut Montaigne”.
This situation paved the way for massive policy inferences made by institutional actors on the basis of PISA according to their own interests and policy stances. PISA would show the need to abandon homework and to move towards a co-education between teachers and parents (Fédération des Conseils de Parents d’Elèves (FCPE), one main association of parents), the universality of Bourdieu’s theory (Christian Baudelot (AEF dispatch n°112043)), the lack of relevance of the reform of pupils’ school rhythms (National Association of Directors of Education in the Cities of France (ANDEV)), the necessary development of the syllabic method in the teaching of reading (SOS Éducation), etc. The official reactions of political parties to PISA 2009 and PISA 2012 results were close to a caricature of their policy stance. For the Left, the low performances were due to the policy of the Right in power since 2002. For the Right, they were the consequence of the reforms implemented by the Left when the pupils tested in PISA 2009 and PISA 2012 were in primary education. The reactions of teachers’ unions mainly consisted of taking PISA results as an opportunity to publicly reassert the relevance of their policy stances: the FSU reasserted the needs for more money and a more inclusive school; the Syndicat national unitaire des instituteurs professeurs des écoles et Pegc (SNUIPP) pleaded for an urgent plan of action in primary education; the Syndicat Général de l’Éducation Nationale (SGEN) criticised the permanence of a Republican elitism, etc.
A scooped out survey
This policy debate which progressively scooped out the survey from its statistical content is the result of a deep change in policy configuration (Figure 5).
Policy configuration 3. Towards a commonplace tool of communication.
Paradoxically, the starting point is the new possibilities offered by the survey itself. Indeed, after some French-speaking OECD experts established from PISA 2009 that: 1) cross-period comparisons were solid (with PISA 2000 in literacy); 2) France results were falling below the average of the OECD; and 3) social background was a key explanatory variable in the French case – France witnessed several changes in the speeches and the stances of policy actors.
For left-wing political forces, especially the Socialist Party, this new diagnosis was an opportunity to criticise the right-wing government record on the basis of its own indicator and to remobilise traditional discourses on the necessary struggle against the social inequalities of academic achievement. On the contrary, for some conservative forces – intellectuals, media like Le Point or Le Figaro, right-wing deputies – it constituted a renewed and statistical basis for their alarmist stances on the “decline” of the French Republican school. Thus, progressively, the policy debate on and from PISA melted into ancient, pre-determined and foreseeable policy debates.
This movement was particularly salient in Parliament. According to a boomerang effect, PISA was regularly used by left-wing deputies to severely criticise the right-wing government record and to argue that during its mandate, results fell and inequalities increased.
Of course, the media were strongly encouraged to cover this new political battle but they also fuelled it through classical media channels: articles from editorial writers, interviews or tribunes from popular intellectuals, reports on policy solutions available abroad, etc.
This politicisation of the debate contributed to its continuous extension but it also favoured its integration in the very classical debate on the “crisis” of education. Given that no major new pieces of knowledge had been produced by scholars and evaluators on PISA lessons since 2010, this integration resulted in a kind of “trivialisation” of PISA. Speeches on PISA progressively merged with the ordinary mechanisms of public communication between policy actors: unions reproached the government with implementing a non-egalitarian policy; right-wing media like the Figaro or Le Point regularly used PISA as an alarm signal for the decline of the French Republican school; and the Socialist Party denounced the government’s responsibility in PISA 2009 and PISA 2012 results, whereas the latter denounced its hypocrisy and so on.
The four dimensions of policy configurations
The forms and the dynamics of these three policy configurations can be explained by four types of intertwined factors.
The centrality of the Ministry of National Education
The first type of factors is political. The Ministry of National Education and its impulsion are decisive. This is particularly visible in the ripple effects of its various stances on other policy actors like the unions, the media, the official evaluators of the education system and the deputies. This was the case in December 2001 when it was decided not to communicate too much on PISA; in December 2007 when the minister declared that there was no more problem with PISA methodology; and in December 2013 when it translated its “Refoundation” program as the French version of the PISA shock.
This centrality comes from a long historical movement of centralisation that started in 1806. Even if it has been counter-balanced since the 1960s with several measures of “deconcentration” and decentralisation, this centralisation remains powerful since it concerns the definition of the curriculum, the recruitment and management of school personnel, budgets and the control, monitoring and evaluation of the education system. This centrality is also the consequence of a mixed political regime defined by the Constitution of 1958, with elements from both presidential and parliamentary regimes, in which the central executive power still has the initiative for policy implementation and has several powers in the legislative procedures.
This centrality has several consequences on the policy debate. It explains why this debate depends so much on political circumstances and when policy actors anticipate the next presidential elections. The situation in December 2001 is emblematic – this was when Jack Lang’s cabinet and the DEPP agreed to barely communicate on PISA. The inversion of power relations between the majority and the opposition in Parliament in spring 2012 is another illustration: the Right became weaker in the PISA debate when it was forced to defend its results in the perspective of the next presidential election.
This centrality also explains why the policy debate on PISA is so closely linked with other policy issues. Since the ministry of National Education is the central interlocutor of political and social forces, especially the unions, for all policy issues, PISA speeches cannot be analysed without taking into account the telescoping effects of agenda setting. The debates around the implementation of the Right’s policy in primary education in 2008 is a good example: if some unions insisted so much on the methodology at the origin of PISA during this period, it was to better contest this reform – how could an international survey of 15-year-old pupils’ competencies be a relevant source to conceive primary education reform? Another example is the evaluation policy itself. In a former study, I showed that this policy depended mainly on the competition between evaluators who all try to make their own conception of evaluation recognise as predominant by the government (Pons, 2010). This aspect is essential to understand the stances of the general inspectors and the DEPP in different periods to maintain their dominant position.
Institutional prisms
The institutional rules which prevail in each fragment of the mosaic public space are also a key source of explanation. As Van Zanten (2009) argues, in education in France, the circulation and the visibility of academic works are limited because of the weight of state elites, but also because scholars have mainly internal channels of communication of their works. Scientific seminars, symposia and publications still play a key role in knowledge production, in the reproduction of a scientific field and in the collegial regulation of this activity. Scholars are rarely encouraged to disseminate their knowledge to non-academic publics, and when they do, it is more often on the basis of individual viewpoints to take part in a burning political issue. This explains both the regular interventions in the debate of some scholars with a particular vision of the policy changes expected – like Denis Meuret or Nathalie Mons – and the fact that a majority of them stayed in the background once some findings were relatively stabilised, like the methodological limits of the survey or some key conclusions identified by Baudelot and Establet (2009).
In Parliament, public sayability is strongly codified by institutional procedures with different objectives. Written questions regarding information on the survey were mainly addressed by deputies from the majority, whereas oral questions were more oriented to the public discussion of government policy. The examination of a bill gave birth to more strategic uses of PISA either to legitimise a diagnosis or to stress its inconsistency. Furthermore, the deliberative power of Parliament seems to decrease in favour of more strategic uses of arguments in order to gain voters outside the hemicycle (Ferrié et al., 2008); this was particularly the case during the offensive from the Left in spring 2012. Lastly, Parliamentary sessions are often dramatisations of policy stances which are built outside of Parliament and are often mobilised to stage parliamentary work. The recurrent argumentation from the Right that more means do not lead to better academic achievement is a good illustration of this.
Concerning the media, the global drop in press that specialises in education does not favour more detailed analyses on PISA. Within media, various taken-for-granted representations strongly predetermine how media production is oriented. These are representations of the education sector – seen by journalists as an “institutional” sector in which power relationships between the Ministry of National Education and the unions predominate – and representations of journalism as a whole, with the symbolic domination of political journalism on other specialties. The whole functioning of the media field, with its circular circulation of information, its high degree of competitiveness and its structural amnesia, generally leads to massive mimetic processes and to a simplification and politicisation of educational stakes, especially when a ranking is covered. The combination of these elements explains both why the French media generally stuck to government current affairs throughout the period and why the extension of the media coverage of PISA resulted in its politicisation.
Lastly, the regulation of the French education system by institutional actors is traditionally based on three main pillars which are associated with specific visions and uses of knowledge in the policy process (Van Zanten, 2008). Among them, one is particularly important in the case of PISA: the neocorporatist mode of decision-making based on the interactions between state officials and unions’ representatives. This neocorporatism implies for policy actors to focus on political negotiation skills and leads them to a limited consideration for the knowledge produced by PISA and to a strategic use of it, mainly to build a policy stance. This aspect was close to caricature after the publication of PISA 2009 and PISA 2012.
The filter of professional identities
These political and institutional factors are often combined with specific professional identities of policy actors which may reinforce their power. Politicians, for instance, are now nearly all professionals of politics in the French context, at least at a national level; that is to say, people with an early and durable vocation for public issues, a remunerated activity (politics as a full-time job) and with specific competencies – being presentable, speaking well, knowing how to negotiate, etc. (Offerlé, 1999). According to Philippe Zittoun (2014), the main onus of their work resides in fabricating policy statements; in other words, in defining, formulating, disseminating and imposing in the public debate a policy solution to a problem that they contribute to shape. This work requires specific skills and knowledge: controlling broadcast channels; coupling efficiently problems, solutions, publics, responsibilities and solutions; arguing by taking into account the public and by anticipating counter-arguments; and communicating with persuasion. Thus, PISA’s strict content is less important for politicians than its argumentative possibilities. That is why they regularly focus on a small number of arguments and policy narratives (like the need to cut public spending for the Right), not always scientifically based but politically efficient (it is assumed).
Journalists are another interesting example. According to their professional trajectory, the degree of their specialisation, their vision of their job and their representation of the education sector, they are more or less likely to give details about the survey and to convince their editorial hierarchy to provide alternative visions of PISA. If a portion of journalists who durably specialised in education covered PISA from its beginnings and struggled to “sell” new topics to their hierarchy, most of those working in generalist media would recognise the level of dependence on their sources, the difficulties in covering such a technical survey in a short period of time and the lack of national experts available to help them interpret results. Moreover, some journalists do not want to specialise too much in this “institutional” sector in order to be able to communicate to a wider public.
Knowledge traditions and argumentative powers
The last type of factor is cognitive: PISA itself, its constitutive properties, its axiological plasticity and its evolving content provided policy actors with different opportunities during the period. If it was initially mainly regarded as a questionable “neoliberal” and “Anglo-Saxon” survey, especially from left-wing policy actors, important changes occurred with PISA 2009.
Moreover, some arguments may evolve differently in the policy debate according to their “convincing force”. For Francis Chateauraynaud (2004), the latter depends on the possibility for an argument to pass three kinds of tests: a “tangibility test” (its empirical relevance in the perceptible world), a compatibility test (its capacity to make converge different interests and representations) and a historicity test (its capacity to redefine the links between the past, the present and the future). This explains, with other factors, the career of the arguments initially developed by the DEPP on the methodological biases of PISA. These arguments never really passed the historicity test: relatively closed on themselves, they did not allow their advocates to project themselves in a new future. And their capacity to pass the tangibility and the compatibility tests decreased throughout the period. On the contrary, the argument from the Right saying that PISA showed that more means do not involve better results has been successful for a long time because it passed the three tests, at least until PISA 2009.
Lastly, knowledge traditions often prevented more detailed analyses on PISA to be publicly evoked. First, excluding people from the DEPP, the great majority of policy actors are not trained in statistical issues. Second, PISA is based on psychometrics whose production is both residual and dispersed in France because of many reasons: the evolution of differential psychologies, focus on other research issues, little room given to psychometrics in the training sessions of statisticians, preference given to careers advisers in studying pupils’ options, the centralisation of statistical data production within the DEPP, etc. Lastly, the administrative elites and their “state sciences” are particularly influential in France (Pons, 2013). This influence is visible in their centrality in the policy process (role of the DEPP, for instance, in the implementation of PISA in France), in the volume of their knowledge productions and in their participation in the editorial board of various reviews. Yet, their effective participation in the policy debate still depends on the minister’s authorisation. Thus, there is a gap between what these senior civil servants effectively know and what they can effectively say publicly.
Conclusion
The major effect of PISA on the French policy debate between 2001 and 2014 was to confirm and legitimate pre-existing opinions and policy stances. This conclusion invites the researcher not to take for granted the idea that an increasing number of speeches on PISA in a country would necessarily mean that this survey has an effect or an influence on domestic education policy or that a comparative turn is taking place. These speeches may also be the sign of a trivialisation of PISA and of domestic actors with a strong capacity to redefine educational problems according to their interests and representations.
These elements plead for theoretical frameworks that allow researchers to study the specificities of policy contexts, the dynamics of public speeches and the various potential “effects” of international surveys on them. For this purpose, I studied “public sayability” as an outcome of “policy configurations” which predetermine public speeches on PISA. This approach has the merit to take into account the diversity of policy contexts, to do it on the basis of a specific theory of social action (as reciprocal) and power (as a relation) and to establish theoretical links between discourses and policy processes.
This approach is limited by its specificities and a more detailed analysis of each policy configuration is always possible. A key research question is to know where and how to stop the investigation since policy actors are always involved in multiple forms of interdependencies. There could be more elements on how a policy configuration moves to another one. Lastly, this approach focuses on a specific form of agenda-setting through mediatisation and public debate, whereas more “discrete” processes are probably at work.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.