Abstract
This article analyses the trajectory of national testing in Portugal between the early 1990s and 2015 in order to unveil continuities and changes in the regulatory processes of education in Portugal, particularly the emergence of results-based coordination and control methods. Drawing on legislative materials, it identifies and describes the different forms or modalities of national testing and other assessment tools that have been used at basic and secondary school levels. Drawing on the texts of Ministers for Education (from documents and interviews), government programmes, and other official texts concerning the adoption of national testing, it describes and analyses the various ways of interpreting the regulatory rationale based on results of the aforementioned period. The article highlights three main areas of tension around which the different approaches to interpreting these results-oriented regulatory processes are structured in Portugal – accountability models, regulatory models and legitimating principles for the education system – and how they are overcome in the Portuguese context.
Introduction
Across Europe, national testing is widely used by public authorities and recurrently justified on the basis of three main functions: to judge student acquisitions; to steer and evaluate schools and educational systems; and to support the learning process (see Réseau Eurydice, 2009). The increase of national testing cannot be analysed without considering the weight of evaluation in our societies – that is, as a government mechanism and a new philosophy (Martuccelli, 2010/2011) and not solely as a managerial or pedagogical tool. In this article, national testing is examined as part of an “evaluacratic torrent”, which has impregnated the field of education (Correia, 2010).
In Portugal, over the last 25 years, intensification and diversification of evaluation are important features of the changes observed in the governance of education (Barroso, 2017). Although regarded as an emergent political technology already in the mid-1990s, within the scope of “mitigated educational neoliberalism” (Afonso, 1994, 1998), evaluation went on to acquire a key role in the following decade, when it became more effectively applied to different objects (Abrantes, 2010): students (through diverse standardised external assessment tools in basic education and secondary education), 1 teachers (appraisal of teachers’ performance), schools (external evaluation of schools by the National Inspection Board), and the education system (mainly through the participation in the PISA survey). Finally, evaluation gained even greater centrality throughout the first half of the current decade, with the increasing number of school years being subject to national testing. As has been noted, national testing has become a key element of the general evaluative architecture, as shown by the fact that students’ results serve to structure the benchmarks for teachers, schools, and the education system (Afonso, 2014).
Thus, in this paper, we analyse the trajectory of national testing in Portugal as illustrative of changes in the coordination and control of education systems – that is, changes in the regulation of the educational system (Barroso, 2005, Maroy and Dupriez, 2000). Specifically, we observe the activation of the standardised national tests and their legitimisation by the Portuguese public authorities, between the beginning of the 1990s and 2015. Our purpose is to describe and examine the different ways of interpreting a specific regulatory rationale – the results-oriented regulation – that has emerged over these last decades in Portugal.
We draw on four main sources: national legislation on students’ assessment in basic and secondary education between 1992 and 2015; government programmes; interviews with four Ministers for Education, 2 published in periodical press, which were selected for their relevance to national testing or to clarify governmental options with regard to the education sector; and other documents published by these Ministers for Education, as afterthoughts when “reviewing” or “justifying” their interventions.
The paper comprises two main sections. In the first part, we present the main argument supporting our study: national testing is a powerful instrument of the results-oriented regulatory processes that takes on specific forms in each national/regional context, depending on the tensions, conflicts, and agreements linked to the political-administrative and educational frameworks historically established in each context. In the second part, we portray and illustrate the three faces of the results-oriented regulatory processes in Portugal over the last decades: the “discrete” face; the “transformational” face, and their two variants (the State under evaluation and the State as the evaluator); and the “sanctioning” face. The paper closes with a discussion and conclusion.
National testing and the changes in the regulation of school systems
In 1974, dictated by larger changes in the school system that occurred during the transition from a totalitarian to a democratic regime in Portugal (see Stöer, 1986), the systematic use of national testing was interrupted. Since then, and for more than a decade, national testing became referred to as a strange part of the continuous assessment process, and was even abolished in 1983 in non-higher education (Dispatch 23/ME/83). Finally, in 1993 national testing was reintroduced, yet only at the end of secondary education (Year 12). The reintroduction was framed by five imperatives – quality, rigour, equity, selection, and synoptic information/vision of the school system – which are the five main elements of a narrative that has become a constant in the subsequent norms regarding students’ assessment and respective tools (see Normative Dispatch 338/93; Dispatch 60SEED/94; Normative Dispatch 644A/94; Dispatch 5437/2000; Decree-Law 6/2001; Decree-Law 209/02; Decree-Law 74/2004; Decree-Law 139/2012; Decree-Law 17/2016). These imperatives are associated, in policy texts, with an ensemble of requirements for the functioning of the school system, such as the urgency of ascertaining the truth in terms of what is taught and learned, the urge to control the achievement of objectives set for each level of education, and the urgency to serialise students as well as school trajectories.
The somewhat timid return of standardised tests in 1993 (exclusively at Year 12) became persistent over the coming years, by expanding from secondary to basic education, as national examinations: Year 9 (2002), Year 6 (2011), and Year 4 (2012), under centre-left governments (XIII Government (1995–1999), XIV Government (1999–2002), XVII Government (2005–2009), and XVIII Government (2009–2011)) and centre-right governments (XV Government (2002–2004); XVI Government (2004–2005); XIX Government (2011–2015)). Recently, in 2016, national tests of Year 6 and Year 4 were redesigned to take on diagnostic and feedback purposes. 3
The emergence of national tests has occurred within a trajectory of regulatory changes that occurred since the mid-1990s. At least rhetorically, control by norms gave way to a control through the results (Barroso, 2003: 75). These changes have also been stated in other national contexts along with other manifestations associated with the emergence of a post-bureaucratic regulatory regime (Maroy, 2009): strengthening of school autonomy; balancing centralisation and decentralisation; promoting school choice; diversifying educational provision; and increasing control over teachers’ work. Thus, the trajectory of national testing in policy texts may be understood as part of the emergence of regulatory processes based on the measurement of students, schools, and school systems’ performances.
These changes are not exclusive to the education sector. Principles of outcomes-based management, together with the adoption of explicit standards and benchmarks, and the increase of accountability, have been broadly applied in the public sector administration, accompanying the transnational diffusion and the national active and heterogeneous reception of New Public Management (NPM) ideas and instruments (Gunter et al., 2016; Verger and Normand, 2015; Verger et al., 2018). In fact, national testing may be seen as crucial to the accomplishment of major NPM principles (Mons, 2009: 8): public bodies must be regulated by outputs and not just by inputs; public services can be measured by tools with scientific validity; and actors with greater autonomy are more accountable for their actions to their managers and to citizens.
In this article, we intend to study the introduction of national testing in Portugal as an analyser of reception of the aforementioned regulatory modalities, under the NPM auspices. We examine the forms of regulation activated and served by the use of national testing and other standardised evaluation tools in Portugal by drawing on an analytical matrix that comprises three dimensions: (a) the types of accountability; (b) the driving forces (of regulation); and (c) the mandate(s) attributed to schooling.
The first dimension, types of accountability, takes into account that policy options regarding national testing materialise different accountability models. Harris and Herrington (2006, cited in Dupriez and Mons, 2011) distinguish market-based and government-based accountability, according to the actors and the means of action that lead the intended improving practices: the parents, represented as savvy consumers, supported by classificatory information on schools (in the model’s “purest” form conceived as being able to choose freely); or the public authorities, through the intervention of the central, regional, or local administrative bodies. We also take into account Maroy and Voisin (2014) typology, considering as extreme contrasting cases: hard accountability, which is based on legal, regulatory, or budgetary tools, designed in terms of incentives and constraints, and involving diverse consequences (e.g. financial, careers, individual, or collective images); and soft accountability, which is based on the supposition of involvement and reflexivity of the actors, and giving preference to self-evaluation rather than external sanction, and that focuses on training, monitoring, and other strategies that are expected to be supportive of reflection and self-oriented changes.
The second dimension, driving forces (of regulation), takes into account that students’ evaluation can be oriented, coordinated, and controlled from diverse contexts. They may be predominantly based on internal tools – that is, tools generated in each school, like global tests carried out by schools – thus giving each school the possibility to establish its own evaluation frameworks and benchmarks, serving its own educational project, and expecting that local actors will analyse school results with the purpose of changing their practices. Thus, the support for the regulatory processes would be predominantly internal to the (local) system’s and actor’s reflection (Maroy and Voisin, 2013). Or they may be predominantly external, with the fulcrum of the regulatory process, by means of “control and/or support”, exerted from outside each organisational context: based in the production of “information” about the whole system (and each of its units – the schools), through credible and reliable indicators. Therefore, the regulatory processes are predominantly determined by the strength of instruments external to schools over utilitarian and/or reflexive social actors (Maroy and Voisin, 2014).
The first and second dimensions deal basically with ways of administrating social units. However, the centrality acquired or attributed to national testing is not all about how to govern and administrate. It is also about “to what end” and “why”. Thus, a third dimension is needed to analyse the policy texts: the mandate(s) attributed to schooling. Surely, the centrality of national testing and quantitative learning measures cannot be dissociated from the educational perspectives that, according to Lingard et al. (2013), have redirected educational goals by giving primacy to human capital training, which is viewed as an investment in the accumulation of productive skills to supply countries with competitive advantages in a globalised economy context. Following these thoughts, it may be said that the expansion of national testing coexists with the preponderance of an educational framework in which the aim of providing different groups of students with the specific skills and knowledge needed to become productive citizens in the work activities required by the current economy replaces the aim of providing everyone with the skills and knowledge that make them responsible citizens (Labaree, 1999). However, this is a matter that lacks empirical proof. Consequently, it should also be understood in consideration of another central issue: the historically rooted disputes and agreements regarding schooling, particularly in terms of determining the place and role of education in society. Thus, the trajectory of national testing in Portugal from the 1990s to the near-present needs to also be analysed from the perspective of unveiling the tensions, conflicts, and arrangements between different legitimising principles of school knowledge and forms (Correia, 1999; Lima and Afonso, 2002).
The three faces adopted by the results-oriented regulation in Portugal
In Portugal, over the past 25 years, it has been possible to differentiate three faces of results-oriented regulation, which correspond to three interpretative approaches on the part of the authorities: the discrete, the transformational (comprising two variants – the State under evaluation and the evaluator State), and the sanctioning. Each form, temporally located, reveals diverse resolutions according to the aforementioned dimensions (types of accountability, driving forces, and mandate(s) attributed to schooling). We typify these three faces in Table 1.
The faces of results-oriented regulation in Portugal: a typological attempt.
The discreet face
In opposition to the reformist impetus that had characterised the intervention of Portuguese governments in the 1970s and 1980s, the so-called “gradualist” positioning that presided the political strategy of the second half of the 1990s highlighted the role of evaluation: “[c]hanges in education must, in this sense, be gradual, centred on schools and educational communities, subject to evaluation and a constant and participatory process of adjustment to reality” (Portugal, 1995: 117). Thus, the presence of the standardised external evaluation throughout the 1990s followed the emergence of evaluation as an effective control device in schooling. This emergence was, however, rather moderated, reflecting the same mitigated manifestation of the NPM proposals in the education sector in Portugal at that time: it was included in the political rhetoric, as well as in the adoption of new policy instruments, such as the autonomy contract, but it was not implemented (see Barroso, 2003).
Types of accountability
Within the scope of school-focused State interventions, a rhetorical call for evaluative practices was visible along this period, associating a (hypothetical) reinforced autonomy with a greater demand for school responsibility: Autonomy implies responsibility and a continuous inspective and evaluative attitude. (Grilo [Minister for Education, 1995–1999], 1999: 136) Autonomy means less weight but greater control on the part of the administration; A controlled autonomy, subject to constant evaluation …. That is why autonomy and evaluation appear in parallel … in an action that we launched, the evaluation of the students, in national terms, for the 4th, 6th and 9th years of schooling. … It is necessary to make national tests to know what the knowledge reached by all the children that finish …. (Grilo, 1999: 101–102)
However, and significantly, the ambition to develop a “culture of evaluation” within the education system was not expressed in the most emblematic document of the period – the “Education Pact for the Future” – a text presented as being a proposal, to all education stakeholders, of strategic objectives and immediate commitments (Benavente, 2004). Moreover, none of the 10 major proposed commitments of the Minister for Education referred to evaluation, not least to the evaluation of results (see Grilo, 1999: 7–8).
Thus, accountability emerged in the policy texts but does not materialise either as a global strategy or through the implementation/activation of specific regulatory instruments.
Driving forces
Throughout the second half of the decade, the driving forces of regulation were basically internal: student assessment was predominantly based on internal tools, formative and summative (global tests carried out by schools), complemented by the use of standardised external evaluation through national testing, but only at the end of secondary education (national testing, introduced in 1993, was launched in 1995 for Year 12). 4
Furthermore, at a rhetorical level, the presence of the standardised external evaluation and its tools were not understood, in policy texts, as being determinant of the changes need in the school system. Instead, they were seen as subsidiaries of changes in which schools were the epicentres, based on the reflection conducted therein: “… schools that take responsibility … are the ones that work best” (Grilo, 1999: 159). This rationale encouraged the adoption of internal summative assessments in the form of global tests, seen as “a way [for schools] to evaluate their own functioning” (Grilo, 1999: 121), as well as the promotion of best practices as a system regulatory tool: “The ‘schools of reference’ … demonstrate best practices” (Grilo, 1999: 163).
Mandate(s) attributed to schooling
The key words of this period, given by a Minister who for years was linked to major international organisations with influence in the education sector (World Bank, OECD, UNESCO), gathered the main elements that punctuate the contemporary educational discourse – effectiveness, equity, and quality – as the following excerpts illustrate: The process of democratizing access to education must be developed by articulating the universalization of basic schooling with an increased diffusion of secondary and higher levels, while ensuring the rigor and quality of teaching and learning as renewing social trust in school and education. (Grilo, 1999: 12) If we do not take on this dual aspect [being in school is a right and a duty], we are condemning education or making it a reproductive reality of injustice, or a system that can level underneath – which is intolerable. There is a dilemma in contemporary democratic societies that we must respond to. Quantity and quality, democracy and exigency, autonomy and responsibility, oblige us to permanently link contradictory and different elements and compulsory education has to harmonize these factors. (Grilo, 1999: 113) I think that the evaluation system, whatever it may be, whichever is the model, in 5 or 6 or 10 years, must reward the merit because if it only serves to punish it will not reach the objectives. (Grilo, 2009, interview)
These texts were sensitive to the developmental dimension of education, in its various dimensions, including the economic one. Also, they recalled concerns about democratisation (access for all social groups to different schooling cycles) and emphasised the exigency and merit, thus bringing together diverse mandates for schooling, of social effectiveness, democracy and meritocracy.
The “transformational” face and its variants
The mitigated emergence of standardised external evaluation that occurred throughout the 1990s gave place to a much more intense presence in the subsequent decade, and under a new rational, which took evaluation and assessments as the path needed to follow in order to bring change and reform – transformation – to the Portuguese education system.
National testing was expanded to basic education at the end of Year 9. Legislated at the beginning of the first decade of the 21st century, under a centre-right government (2002–2004), it was implemented in a continuous way by the centre-left government that followed (2005–2011). But evaluation and assessments has increased, throughout the decade, in many other forms and targeting more actors in the school system.
Results-oriented regulation emerged with its transformative features along both periods. In fact, the last decade of the 20th century was marked by the manifest adoption of NPM’s preconisation and the recurrent use of evaluation as an important regulation instrument, by both political forces, associated, respectively, to the centre-right and centre-left: … the Government shall implement the following measures in the field of education and training: evaluation of the performance of schools, dissemination of results and creation of a system of distinction of merit and support for those who are most in need; … creation of conditions for the modernization and professionalization of the management of educational establishments, by making processes simpler; clarifying responsibilities and respecting the figure of the school principal. (Portugal, 2002: 109–111) The trend towards decentralization and the reinforcement of schools’ autonomy requires the development of mechanisms for external evaluation, monitoring and control to ensure compliance with the rules of transparency and rigour in the management of public resources, as well as the fulfilment of the mission assigned to schools. The external evaluation programmes of schools, textbooks, teachers and non-teaching staff have been launched and implemented. (Rodrigues [Minister for Education, 2005–2009], 2010: 23–24)
The case was not only one of introducing evaluation as a new technical apparatus. The effective introduction of evaluation was said to represent the very cultural change of the education system: [it is] essential to introduce at all levels a culture of evaluation of institutions, teachers, staff and pupils which has an impact on their performance and organizational, professional and human development. (Portugal, 2002: 108). [One of the “five ambitions of the legislature”] to root the culture and practice of evaluation and accountability in all dimensions of the education and training system. Evaluation of pupils’ performance and national curriculum, evaluation of educators and teachers, evaluation, according to performance criteria, efficiency and equity, of schools and the technical services that support them. (Portugal, 2005: 43)
Consequently, the results became the epicentre of educational and governmental reflection and action. Evaluation was referred to as the dynamic force of educational change, through which the coupling and the mediation among the diverse levels and actors of the regulatory processes were to be exerted. Thus, in this period, evaluation appears as an instrument and a strategy of government.
Driving forces
The transformational face is characterised by a predominance of external evaluation. This pattern inverted the trend observed in the previous decade when there has been the coexistence of external and internal evaluation instruments, and the preponderance of the latter. External evaluation increases in the form of national tests and other standardised tests and international surveys, always in the name of “assuring the quality of learning” (Justino [Minister for Education, 2002–2004], 2005: 44-45; Rodrigues, 2010: 22). 5
At the same time, the fulcrum of the regulatory process shifted to external control and support, based in the production of “information” on the system through credible and reliable indicators: The excellence and strictness that is intended to lend to the functioning of the system implies the transparency and responsibility of the various services and entities that condition the life of schools and the performance of the students. Therefore, it is a government imperative to consolidate and make an updated, credible, coherent and accessible information system available. (Portugal, 2005: 49)
Thus, the regulation logic became predominantly determined by the strength of instruments external to schools, though associated with local actors’ responsibility or reflexivity, therefore, moving between a conception of a utilitarian social actor and a reflexive social actor: One of the most evident conclusions of the four-year experience of standardised tests [with feedback purposes] was their progressive devaluation in the eyes of students and many teachers. As “they count for nothing”, it is not necessary to study, practice, and much less to develop any effort in order to accurately answer the test questions. … The fact remains that the best results of standardised tests were obtained in the first year, precisely when it was “feared” that the results would have consequences. (Justino, 2005: 16) You see, for 10 years we had, examinations in secondary education … but these results were never systematically returned to schools, nor was anything ever required of schools or teachers to respond to those results. … And to me, what seemed evident was that the results of the national tests had to be returned to the schools, to the teachers, it was necessary to create conditions for reflection in the schools to be promoted on the basis of their results, and for the teachers to look at that and think “Why do we have such poor results in Physics or Chemistry? Why is this group different from that one?”. (Rodrigues, 2009: interview)
Despite this common pattern, the transformational face takes differentiated expressions throughout this period, related to the type of accountability promoted and the mandate advocated for schooling. These differences allow us to distinguish two variants of the results-oriented regulation transformational face: the State under evaluation (2002–2005) and the State as the evaluator (2005–2011) (Table 2).
The variants of the transformational face of results-oriented regulation.
Types of accountability
In the “State-under-evaluation” variant, the national testing and other standardised external evaluation tools respond to the purposes of certification, selection, and diagnosis. But they are also enacted as devices that should generate important information for the families’ “choices” and reveal examples (positive and negative) – as the following quotations from the 2002–2004 Portuguese Minister for Education illustrate: … the regular publication of national tests classifications, from school to school, subject to subject, in addition to enshrining citizens’ rights, gives greater transparency to the process of choosing schools and mitigates the real social inequality in access to information. Conditioning this access only strengthens the capacity gap of each individual, each family or each group, to access and effectively use this information. (Justino, 2005: 30) … [D]ifferentiating good and bad schools, schools that progress and schools that regress, well managed and quality schools, schools without quality and without educational management. This is one of the central problems of education in Portugal. (Justino, 2005: 32) The simple confrontation, from school to school, between the classifications obtained in national tests with their internal frequency ratings may reveal situations that deserve regulatory action. (Justino, 2005: 90)
This market-orientated accountability aimed to promote a change in the choices of (informed) parents and to force the adaptive and competitive dynamics of “supply”. Also, it was to be supported by the amplifying effect created by the public dissemination of results and through the rankings made by the written press: We compete in the economy, we compete in our professional careers, and we compete in politics, in the day-to-day of our social life. Why can we not compete in teaching, in pursuit of best practices, in promoting better performance, in achieving better results? (Justino, 2005: 95, originally quoted in Público, April 2001) The contribution of rankings, with the benefits, limitations, and even the damage caused by some, has acted solely as a means of accelerating a culture of evaluation that has been confined to some schools. Everyone knows how the mere publication of results has been tinkering with the school environment. (Justino, 2005: 44)
In the case of the “evaluative state” variant, public authorities and the various levels of their administration were enhanced, with the coordination and control of the school system reliant on the devices they create. The emphasis given to standardised tests clearly demonstrates this orientation: The improvement of the national evaluation system by standardised tests, as the most appropriate system to evaluate the development of the national curriculum and the provision of schools, in basic education. … The change of students’ evaluation system, so that the use of rigorous criteria, in the transition between phases or years of schooling and in the completion of study cycles, has the useful effect of implementing programmes to support the recovery of students with learning difficulties and not their pure and simple exclusion or their relegation to a disqualified path. (Portugal, 2005: 46)
The emphasis placed on testing cannot be dissociated from a question that was raised, during this government, over teachers – whether they would be able to respond to the challenges of a “school for all”, or stuck in a “school for a few”, from the past: Since 1986, the mission assigned to the school has been different: to integrate all students, even those who are not motivated or who do not have the best conditions to learn. … And the doubts that arise are many: do teachers’ expectations about their role mostly align with the past or current paradigm? … What is the distance between actual performance and expected performance? (Rodrigues, 2010: 35)
This challenge had as its most important political expression the launch, in 2007, of a model of teacher evaluation, based on rewarding merit mechanisms, and promoting competitiveness and the (vertical) differentiation of teachers through their performance levels.
Mandate(s) attributed to schooling
Each variant of the transformational face relies on a specific interpretation of the school system’s legitimacy.
On the one hand, Justino’s (Minister for Education, 2002–2004) texts focus on the societal lack of confidence in schools’ capacity to categorise students by merit and to promote fair social mobility. In other words, the mistrust in schools by those social groups that are strategically active in the struggle for school credentials: One of the most frequent interpretations of the results is geared towards the search for “cheaters”. The competition … has led to practices of alleged “tampering” or “inflation” of classifications. Everyone recognizes that the often “desperate” willingness to enter a medical school – just to mention the most paradigmatic example – leads to the search for the best averages. (Justino, 2005: 46) The school is not an isolated space and, providing a fundamental service to society, it is expected that the quality of this same service will correspond to the trust that the families place in the school. (Justino, 2005: 81, statement previously introduced in the Project Law no. 422/VIII submitted to the discussion in the Parliament on 26 April 2001.
In contrast, in Rodrigues’ (Minister for Education, 2005–2009) texts, the concern is not that of justice in the occupation of social positions in terms of individual investment, but that of justice in treating all social groups equally, in the context of what she pointed out as a major political challenge in Portugal – the success of all students: The mission assigned to the school is … to integrate and teach all students, even if they are not motivated or do not have the best conditions to learn. Today, it is required that the school responds for the results of all students, by guaranteeing the success of all students, and requiring new pedagogical practices, new teaching methodologies, new attitudes and a new way of looking at students. (Rodrigues, 2010: 34)
The “sanctioning” face
Between 2011 and 2015, under the auspices of a centre-right government (2011–2015), and during the so-called “Troika years” – that is, under the Financial Assistance Programme (2011–2014), 6 the evaluation mechanisms of the previous decade were enacted differently. Indeed, the emergence of the sanctioning face of the results-oriented regulation relate to changes in the three dimensions of analysis: an emphasis towards a quasi-market orientation; a greatest relevance given to external rewards/sanctions as a driver of change; and a specific rationale for a (re)centralised national curriculum associated to meritocracy.
Types of accountability
The results-oriented regulation was made explicit in the XIX Government Programme itself. In a section entitled “Guiding the organization of the Department of Education towards results”, there was an emphasis towards a quasi-market orientation, thus affecting the role of the State in the provision of education.
This became visible in the importance devoted to the public delivery of information on students and school results as a way of giving more visibility to and a focus on performance in national testing. In fact, the results and their diffusion are seen as relevant and strategic information to the school–family–market triangle, as a way of encouraging “families to make more informed decisions in the exercise of their freedom of choice” within a social atmosphere of “transparency” and “trust” that enables them “to make good decisions” (Portugal, 2011: 114–115).
Manifest in a discourse that promotes autonomy and school choice, the quasi-market regulatory framework is also evident in the strategic political goals pursued throughout this period: the establishment and extension of autonomy contracts as “one of the key policies for ensuring diversity and the merit award in schools”; the creation “of a new culture of discipline and effort”; the development of “initiatives of freedom of choice for families in relation to the available supply”; and the implementation of “decentralized models of school management” (Portugal, 2011: 114).
Driving forces
The XIX Government (2011–2015) introduced national testing in Years 4 and 6, under the argument of a culture of rigour and meritocratic excellence. Throughout this period, external evaluation became more and more valued. Several indicators attest to the great relevance of external evaluation: the creation of a national system of indicators “aligned with best international practices”; the generalisation of evaluation by means of national tests for Years 4, 6, and 9, the latter two affecting students’ classification at the end of basic education cycles 2 and 3 (XIX Government Programme: 114–115); the introduction of a new tool, an indicator of the effectiveness of education, directly linked to the results of national testing in basic (Years 1–4) and secondary education (Years 10–12), and used for the allocation of hourly credits to schools; the replacement of the Educational Evaluation Office, the body of the Department of Education previously responsible for the production of external testing, by the Institute of Educational Evaluation, presented as distanced from a so-called interference of political power and, thus, as being able to develop and consolidate a culture of evaluation in the education system, in a stable and credible way (MEC, 2015: 65).
In this scenario, the external evaluation is portrayed as “a stimulus to success” (Crato [Minister for Education, 2011–2015], 2015: 8), associated with the idea of improvement: Evaluation is not just for Teachers, nor does it begin with Teachers. Evaluation begins with the results of the School, and the results of the School are the Students. … The Final Cycle Tests are fundamental to understand what is happening…. Students’ assessment needs to be broadened, and to be better. … We are doing Teacher Assessment and also School evaluation. All this is necessary. Evaluation must be a generalized principle. Evaluation has to be seen as a principle of something that helps everyone to improve. (Statements by Crato, launch of the academic year, September 2011, MEC, 2015: 63)
Moreover, it was written that quality and good results would only be possible “with determination and rigour …, work, discipline and demandingness”, in order to replace “… slackness with effort, pedagogic dirigisme with scientific rigour, indiscipline with discipline, centralization with autonomy” (Portugal, 2011: 13).
The point of support for regulation is then essentially external, dependent on external sanctions, without referring to the reflexivity of the social actors: these thus, it is expected that these will always act in accordance with the presence/absence of symbolic or material rewards.
The mandate for education
Merit became a central value along this period. According to the Minister of Education of the XIX Government, the improvement of schools that were successful in “managing their resources well and making progress in school” should be rewarded (Crato, 2015: 12). And this reward would become visible in the awarding of prizes to the best schools, which would be no more than “a basic foundation of meritocracy, of proven efficiency in the private sector” (Crato, 2013: 21).
Parallel to this, a struggle developed around the curriculum and its centralisation through the establishment of national goals, whose regulatory effect acquired solidity when it acted in combination with external evaluation, within the framework of a global evaluation policy, transversal to all levels and actors of the educational system (teachers, school organisations, students, and curricula): The broadening of the external evaluation in Portugal, combined with the establishment of very precise curricular objectives, embodied in the Curricular Goals, fosters a rigorous implementation and measurement of the knowledge and skills that are being taught, learned and developed in schools. The aim of these measures has been to increase the demand and quality of the education system and ensure the early detection of difficulties, with a view to taking immediate action in order to promote success. (MEC, 2015: 64)
The creation of a self-entitled “results-oriented culture of transparency” was thus imagined to be achieved through the creation of national curricular “targets”, associated with “principles of rigour in evaluation, of high-demanding and merit in results” (MEC, 2015: 35).
Conclusion
Between the beginning of the 1990s and 2015, there has been an intensification in the use of national testing by the Portuguese national authorities. This intensification manifested itself in two ways. First, by the increase in the number of tests and the school population subjected to them, as they expand from secondary education (Year 12, 1995) to basic education (Year 9, from 2002; Year 6, from 2011; and Year 4, between 2012 and 2015). Second, by expanding its consequences due to the intentional or non-intentional combination with other regulatory tools, like the limitation of granting places in the access to higher education, the publication of test results and preparation of media ranking of schools (from 2001), or the importance of students’ achievement in teacher evaluation (from 2007) and in the external evaluation of schools, led by the General Inspectorate of Education and Science (from 2006).
The analysis of the national testing trajectory also shows the presence of diverse scopes and intensities of the standardised tests’ stakes: from no or minor consequences for students, schools, and teachers (the case of standardised external tests with diagnostic and feedback purposes) to stronger consequences, in the progression of students (like the national tests at the end of secondary education, where the classification must fall within a certain percentage for academic progression and access to higher education). Moreover, the policy adoption or withdrawal of assessments along the basic education years (namely, in Years 4, 6, and 9) tends to increase or decrease the stakes for students, as they may affect their objective careers differently (e.g. progression from one year to the next) and their subjective careers (e.g. decisions related to the choice of school and academic/vocational routes).
Despite all of the aformentioned diversity in scope and intensity, external standardised tests – and, in particular, national testing – became accepted in policy texts as multi-purpose tools: besides certification and selection purposes, they have been called on to steer the implementation of standardised curricula, to monitor students’ learning, to support the adoption or the “creation”, by teachers and schools, of new pedagogical and organisational responses, and to make social units (schools) and actors (teachers and heads) more accountable. Furthermore, national tests and their results are turning into nodes of a network of scattered assessment tools – a network that links tools that focus on students’ results to tools that target schools, teachers, and also government programs. Thus, standardised external evaluation seems to have become accepted as the preferential way to keep the various actors that participate in the regulatory processes in continuous and disciplined communication, at diverse levels of the education system (national, regional, and local), focused on – and oriented by – results.
In this paper, focusing on the justifications held in policy texts concerning the adoption of national testing and other evaluation tools, we have analysed the patterns of intensification described herein as indicators of changes in the coordination and control of the education sector in Portugal, and as part of broader transnational changes in the regulatory processes of education. From what we have described and analysed in this paper, we derived the following three main conclusions:
three different enactments of results-based regulation exist in Portugal;
the different configurations observed depend on the ways in which the government of the educational system is understood and performed, as well as on the mandates assigned to the educational system; and
the reinforcement of results-based regulation by public authorities in Portugal materialised through two forms of explicit opposition to bureaucratic-professional regulation based on norms – one market-oriented and the other government-oriented.
Regarding the existence of different enactments of results-based regulation, we differentiate three faces of results-oriented regulation: the discrete, the transformational, and the sanctioning.
Throughout the second half of the 1990s, results-based regulation was emergent but discreet. Neither evaluation nor assessment of results were central elements in the Minister’s statements regarding policy objectives and commitments. And this lesser relevance is particularly significant because in other sectors, namely in health and science, evaluation (and specifically the evaluation of results with identifiable consequences) had already been enacted as a significant regulatory tool. In the education sector, however, the bureaucratic-professional regulation remained strong, with the dominance of command and control strategies affecting school systems’ inputs and processes.
During the first decade of the 21st century, the mobilisation of the evaluative tool occurred as a strategic tool to foster change, and results-oriented regulation has been consolidated. The transformational face of results-oriented regulation develops along with the strengthening of the Europeanisation processes and runs in parallel with the criticism of the bureaucratic-professional agreement. Along this period of time, both governments and ministers (Justino, centre-right, and Rodrigues, centre-left) strictly adhered to the Lisbon Strategy, committing themselves to the pre-defined “problems” and “ambitions”, while conditioned by European benchmarks and the monitoring of results. And both called for changes in the way the education system is regulated – each one of them used external evaluation as an element to confront the established regulation based on norms.
The sanctioning face emerged in the context of the Troika and under Crato’s ministry (2011–2015). A quasi-market orientation comes to the front, affecting the role of the State in the provision of education. Policy texts thus linked the dissemination of annual information on schools’ quality, based on national testing results, with the symbolic encouragement of families’ school choice, or even the transfer of students from State to private schools. Finally, in parallel with the reinforcement of external assessments, the centralisation of curriculum occurred, thus strengthening the control purpose of evaluation.
Over the years under analysis, the development of results-oriented regulation in Portugal has been marked by a common element: the perspective of standardised external evaluation tools as drivers of educational change, while simultaneously turning the internal guiding tools (e.g. autonomy contracts, school’s educational projects, and self-evaluation) subsidiary to external ones. At the same time, national testing has become associated with the purpose of steering and monitoring the education system (although there is still insufficient evidence of its effective use, as such, by public authorities).
However, when we observe the two other dimensions – type of accountability and mandate for education – some differences emerge. And these, at least in part, are related to the political-ideological area of the policymakers. A government-oriented accountability tends to prevail in policy texts from centre-left policymakers (although they do not refuse the use of a major market-orientated tool, such as the publication of rankings with the results of the examinations by school); while a market-oriented accountability prevails in the texts of the centre-right policymakers (without relinquishing strong control over standards and programs, using, among other tools, evidence drawn up by the State administrative bodies).
Furthermore, the diagnostic and feedback purposes of external evaluations tend to be more valued in the political texts from the centre-left policymakers, while the functions of certification and selection tend to receive more explicit importance in policy texts signed by centre-right policymakers. And this difference does not seem to be dissociable from another diverse disposition: while, in general, the purpose of “social efficiency” (providing different groups of students with the specific skills and knowledge needed to become productive citizens in the work activities required by the current economy) is present in all political texts, centre-left policy texts tend to emphasise the issue of social inclusion, where centre-right tend to stress the promotion of individual merit.
Throughout the first decade of the 21st century, the reinforcement of results-based regulation by public authorities in Portugal has materialised through an explicit opposition to bureaucratic-professional regulation based on norms. Such opposition, identified in policy texts, has materialised in two distinct forms – both related to the two variants of the transformational face of results-oriented regulation: the State under evaluation (2002–2004) and the State as evaluator (2005–2011).
In the first variant, the bureaucratic component was targeted. Policy texts were considered to be necessary to shift from the principle of bureaucratic regulation to the principle of decentralised regulation, with competitive but also cooperative decentralisation practices. Thus, external evaluation devices were used in order to raise market-oriented accountability. In the second variant, the professional component was targeted. The professional pillar was questioned and fought, from a different regulatory framework – the State being a force of change. The introduction of teacher assessment was the context in which such confrontation most manifestly occurred, involving high levels of conflict in the educational sector. In this latter variant, external evaluation devices were used to support government-oriented accountability.
These three main conclusions support a broader one: the emergence and consolidation of results-based regulation under diverse “faces” allows us to emphasise that the changes and nuances in the processes of coordination, ordination, and control of education systems may be well captured as “montages composites” (Boltanski and Thévenot, 1991; Maroy and Dupriez, 2000), comprising values, rules, and tools with diverse provenience, as a linear substitution of one modality by another, or as a shift from bureaucratic to post-bureaucratic regulation.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by national funds through the Fundação para a Ciência e Tecnologia, IP (FCT), within the scope of the Unidade de Investigação e Desenvolvimento em Educação e Formação, Universidade de Lisboa (UID/CED/ 04107/2016).
