Abstract
The focus in this article is on Brazilian education policy, specifically quality assurance and evaluation. The starting point is that quality, measured by means of large-scale assessments, is one of the key discursive justifications for educational change. The article addresses the questions of how quality evaluation became a significant feature of Brazilian education, and how international organisations are related to this. The notion of measurement emerged at the beginning of the 20th century among an active educational community, strongly inspired by US educational thinking and supported by instrumental educational cooperation with US agencies and universities. Large-scale assessment became more and more popular in the 1960s–1980s, mainly within this setting. Educational indicators have been under development since the 1990s, alongside the education-related programmes of international organisations. Drawing on earlier research and documents, the article analyses historical and political influences from both international and national actors on the emergence of quality assessment in Brazilian basic education. The authors also point out the need for further research from a more complex perspective on the interplay between the various actors responsible for policies focusing on educational quality.
Introduction
This article concerns Brazilian education policy, specifically quality assurance and evaluation (QAE). Related to the focus of this special issue, it is an interesting case in that Brazil has aligned its education policy with the Programme for International Student Assessment (PISA). The 2007 Education Development Plan (PDE, Plano de Desenvolvimento da Educação, see MEC 2016) sets out the goal of reaching the Organisation for Economic Co-operation and Development (OECD) average level in terms of learning achievement by 2021, and this is still valid in the National Education Plan (PNE, Plano Nacional de Educação, see Brasil, 2014). We address the questions of how large-scale assessment became a significant feature of Brazilian basic education, and the role of international organisations (IOs) in this development.
There has been a considerable amount of research on quality assurance and evaluation in education in the Brazilian context. Our aim in this article is to amalgamate this research and build a basis for the future analysis of Brazilian quality-in-education policy. Although many researchers have pointed out some of the challenges associated with quality of education in Brazil (e.g. Arias et al., 2004; Derqui, 2001; Ferraz et al., 2012; Krawczyk, 2014; Oliveira, 2007, 2011; Saviani, 2003; Soares et al., 2015), we focus on the growing importance of international actors in the Brazilian education sector, and on the institutionalisation of QAE in the educational system.
Latin American educational literature tends to interpret the relationships between domestic and international developments according to two lines of argument (see Beech, 2002). First, the regionalism argument refers to the shared socio-historical and economic paths of Latin American countries, the implication being that global political-economic changes have affected them and their respective educational development in a similar way. Second, the imposition argument posits that the Latin American path is the result of socio-economic, political and cultural domination. In other words, IOs and the US have channelled imperialism and neo-colonialism in particular through the government itself or related agencies. With regard to Brazilian education, scholars have stressed the consistency of US educational aid along with the spreading of capitalism on the one hand, engaging in imperialist relations to achieve this goal (e.g. Carnoy, 1974; Lowenthal, 1991; Pereira and Pronko, 2015), and the inadequacy of US conceptions despite their profound impact on Brazilian educational reforms on the other hand (e.g. Alves, 1968; Arapiraca, 1982; De Tommasi et al., 1996; Haddad, 2008).
Although these accounts provide significant insights, we adopt another perspective to enhance understanding of the role of international actors in the development of assessment in Brazilian basic education. In terms of transnational networks, the basic question concerns the extent to which the global discourse on quality shapes converging forms of educational governance and standardised testing. Informed by studies on educational transfer in the context of borrowing and lending (see, e.g., Steiner-Khamsi and Waldow, 2012), and also the idea of globally diffused scripts in the hands of international organisations (Meyer and Ramirez, 2003; Ramirez, 2012), we analyse Brazilian developments and dynamics. However, our understanding of educational transfer comes closer to the ideas put forward in the discussion on embedded policies (Ozga and Jones, 2006), which question the relevance of a dominant, global education discourse (see also Lawn and Ozga, 2012). Our perspective emphasises the fact that locally embedded practices and longer socio-historical, socio-economic, political and discursive conditions facilitate the import, whether factual or discursive, of models and ideas (see e.g. Beech, 2002; Carney, 2008; contributions in Steiner-Khamsi, 2004; Waldow, 2009). We therefore put forward a view that stresses Brazilian agency and interest in shaping political relations with international actors (e.g. Lafer, 2001), as well as educational cooperation and its outcomes (e.g. Centeno, 2010; Luschei, 2004). In other words, we aim to shed light on the interplay between Brazilian and international ideas of large-scale assessment, and on how this has contributed to the current situation.
As is evident from various large-scale assessments, the measuring of quality is pervasive in education. It is clear that QAE is a very significant framing factor for education today. However, it is clear that quality is not defined and pursued per se, but is rather used as a more general means of governing. It is ‘no longer extraordinary. It becomes simply what can be expected … Quality is conformity with standards’ (Ozga et al., 2011: 2). As long as quality as such is indefinable, its operationalisation assumes importance as a means of constructing its meaning. This, in fact, makes the measurement more important than the definition.
Indeed, measuring quality may create new spaces and networks. In the case of Europeanisation, a new space of operation is being constructed: ‘Europe … is being created, sorted, systematised, scrutinised and constantly improved through the new soft governance tools of comparison and benchmarking’ (Grek and Rinne, 2011: 29–30). Quality serves as a means of connecting national and international actors, facilitating communication between expert and policy networks. In this sense, quality measurement relates to how experts have become more important in creating data infrastructures (Lawn and Segerholm, 2011). Whereas from the perspective of some policy makers, data analysis and its easy access promote equality in terms of ‘anyone can do it’ (Barber, 2014: 82), there are also political and normative aspects to consider (see, e.g., Simola et al., 2013). Ultimately, QAE reshapes actor relations and their roles in governing education (Dahler-Larsen, 2011).
In our view, the recognisable general dynamic that could be labelled QAE describes the notion of measurement in education and how it facilitates the creation of categories for sorting the performance of individuals or organisations and for making decisions. We scrutinise large-scale assessments in particular, with their main focus on measuring learning outcomes. We do not, for instance, concentrate on commonplace educational practices such as inspection, the evaluation of programmes and policies, school personnel and institutions, accreditation, curriculum evaluation or self-evaluation (see Kellaghan et al., 2003). Whereas, as mentioned, the term QAE is used in a broad sense, we refer to assessment or large-scale assessment in explicating the narrower focus. In the context of this special issue, we are especially interested in probing the interface between the international and the national in shaping the idea of large-scale assessment in Brazil. 1
We conducted a scoping literature review and identified, selected and synthesised earlier research and the respective arguments relevant to the emergence of quality assessment in Brazilian basic education. The literature on QAE in Brazilian basic education surfaced in the 1990s and the review covers international and national academic journals in English and Portuguese (1990-2014). Our source was the Scientific Electronic Library Online (SciELO), an established Brazilian electronic library of selected scientific journals (see Hayashi et al., 2008). We used three main keywords (educação, qualidade and ensino) combined with 24 other keywords covering the main Brazilian and international terminology on the subject. The selection process, which was based on the abstracts, narrowed the results first to 169 articles (relevance to basic education), and second to 37 articles (relevance to QAE). To supplement this, we conducted some general searches for English literature in Ebscohost.
In addition to the review, in the case of earlier time periods we relied on the analysis of the Brazilian Review for Pedagogical Studies (RBEP, Revista Brasileira de Estudos Pedagógicos), the historical reference source in the field in terms of scholarly production. We also examined official key documents, taking into account non-official publications that had been reprinted or included older documentation (e.g. cooperation agreements), and consulted the influential Brazilian literature on the subjects of QAE and international cooperation.
We are not attempting to describe Brazilian education as a whole, given that there are currently 27 state systems and over 5,000 municipal systems. Instead, we are interested in looking at how Brazil has developed assessment capacity in connection with international networks. The structure of the article follows two historical periods. The first period, from the early 20th century until the 1990s, which we call ‘preparing the soil’, was when expert networks were built, and international organisations played an increasingly large role in Brazilian education. The second period, from the 1990s onwards, has seen ‘rapid growth’ in QAE measures, made possible due to the fertile soil and the blossoming of large-scale assessment internally, and the establishment of participation in IO’s programmes.
Preparing the soil: constructing education, building international cooperation, experimenting with assessments
Interest in large-scale assessment in basic education emerged only in the 1980s in Brazil, and the country quickly developed nationwide and comprehensive large-scale assessments during the following decades. This is an intriguing early development, given on the one hand that Brazil restored democracy in the latter half of the 1980s, only achieving relative economic and political stability some years later, and on the other hand that a new education law was established as late as in 1996, and a comprehensive national plan for education (PNE, Plano Nacional de Educação) in 2001 (which was replaced in 2014). Even though Brazilian scholars and politicians generally agree that the country is far from having a system of education that meets international standards, it has established complex and encompassing QAE policies, even holding a vice-chair position on the PISA governing board. What facilitated this development in a country with many basic educational needs and against such a historical background?
We explore this question in this section by shedding light on two significant developments that arguably constituted the basis of the later rapid growth in large-scale assessment, and that should therefore be addressed here in the guise of socio-historical contextualisation. First, the developments of the education policy and the scholarly educational community are strongly interlinked in Brazil. Brazilian educationists and scholars have always been socially and politically committed, some holding important administrative and political positions. Furthermore, the latter have been strongly inspired by US educational thinking, practices and policies, a significant number having been trained in US institutions. Second, IOs have featured in Brazilian education since the emergence of this political field, and have contributed to its development in one way or another. Even though cooperation could be interpreted as an imperialist strategy, Brazilian actors envisaged educational cooperation as a way of serving domestic political and economic interests, according to the political profile of the government in charge or the individual actors involved (see, e.g., Paiva and Paixão, 2002). Educational reforms were the result of hybrid educational development, and the agency existing between the parties (Centeno, 2010). These developments reflect a particular ‘socio-logic’ (Schriewer and Martinez, 2004), which denotes the existence of a well-established international horizon of ideas and actions in the construction of domestic educational policy and practice.
Constructing education and an active scholarly community
Brazilian interest in education measurement first emerged at the beginning of the 20th century and, although timid, established two important processes that remain in place today. The first of these refers to statistics in the field of education and reflects the fragmented educational stance (Table 1). The Constitution of the Federal Republic implemented a dual system of education in 1891, in which the different states were autonomous in terms of organising and implementing basic education, whereas the National Government (União) was in charge of higher and secondary education, as well as the Federal District’s educational system (Romanelli, 2007 [1978]). The victory of federalism not only accentuated the existing economic disparities among the states, it also highlighted the wide educational gaps. This fragmented situation provoked the Federal Government into organising a survey of schools and teaching staff, enrolments and grade repetitions based on Annual Brazilian Statistics starting in 1906. This educational survey survived until 1918, but only covered the Federal District. It was resurrected in 1936 and became national in scope (Horta, 2007).
The first steps in large-scale assessment (1910s–1950s).
INE: Instituto Nacional de Estatística; IBGE: Instituto Brasileiro de Geografia e Estatística; INP: Instituto Nacional de Pedagogia; INEP: Instituto Nacional de Estudos Pedagógicos Anísio Teixeira; CAPES: Coordenação de Aperfeiçoamento de Pessoal de Ensino Superior.
The other initiative resulted in the first Brazilian assessment of educational performance, which was introduced as a selection tool for university entrance in 1911 (Cunha, 1980). A combination of mere knowledge assessment and considerable autonomy among higher education institutions attracted criticism, but the vestibular is still used by several universities, and is one of the major educational legacies of the First Republic.
Although there was a lack of enthusiasm for research on educational assessment in the following decades (Gatti, 2002), a clear moment of political openness to educational issues in the 1930s emerged. In the context of the socio-economic and cultural changes of the early 20th century, the decline of oligarchies made way for the rise of new urban social classes, and a group of intellectuals sought to break up the elites in order to offer new perspectives to the Brazilian people (Passos Júnior, 2003; Xavier, 2002). This movement triggered the political and civil restructuration that led to the 1930 Revolution, through which Getúlio Vargas was installed as provisional President and a New Republic was proclaimed (1930–1937). An active educational community that had formed a renewal movement called the New Education Movement (Movimento da Escola Nova) saw this moment as an opportunity to put pressure on the government to make structural changes, drawing first on French (for the most part until the 1920s) and then US (from the 1930s onwards) ideas for its legitimation (see Dussel and Caruso, 1998). Vargas embraced some of the educationists’ ideas in seeing schooling as an instrument with which to construct the nation (Peixoto et al., 1995). An attempt was made to set up a national system of education (Derqui, 2001) with the establishment of a Ministry of Education and Health, education being recognised as a national question of public interest (Saviani, 2003).
Support for universal education and progressive teaching methods had an effect on both the first national education reform (1932) modernising Brazilian secondary schooling (Dallabrida, 2009: 190), and on the 1934 Constitution, which emphasised the Federal Government’s responsibility for national education (Romanelli, 2007 [1978]).
Following the educational zeitgeist, the Brazilian government showed an interest in measurement and assessment as part of educational planning in the 1930s (Derqui, 2001). The National Statistics Institute (Instituto Nacional de Estatística), which was also responsible for educational data, was established in 1934. Moreover, the combination of federal-level importance and progressive ideas led to the foundation of two major organisations that shaped later education policy: the Brazilian Institute of Geography and Statistics (IBGE, Instituto Brasileiro de Geografia e Estatística) in 1934 and the National Institute of Pedagogy (INP, Instituto Nacional de Pedagogia (INE)) in 1937, later renamed the National Institute of Pedagogical Studies (INEP, Instituto Nacional de Estudos Pedagógicos Anísio Teixeira). These two institutes are currently the main providers of educational data in Brazil. IBGE supplies relevant information to INEP, which is responsible for national large-scale educational assessments. INEP is named after one of Brazil’s most influential educationists, Anísio Teixeira, who was very knowledgeable about foreign developments.
These political actions of the 1930s represented an attempt to reconcile socio-political demands and educational stances. In fact, the previously united educational community rapidly drifted into conflicting factions known as the ‘progressive educationists’ and the ‘conservators’ (Nunes, 1992; Romanelli, 2007 [1978]). The educational debate reflected the political instability. When fragile political alliances broke up, Vargas himself overthrew the government and installed a dictatorial regime (Estado Novo, 1937–1945). A group of educationists had a key role in maintaining the educational momentum during the Vargas dictatorship, adapting their progressive philosophy to the political context and combining progressive ideas with military discipline and a hierarchical focus (Dussel and Caruso, 1998). Moreover, they held important administrative positions such as head of the INEP (Centeno, 2010; Gomes, 2000), and advised the Minister of Education (Swartzman et al., 2000 [1984]). The impact of certain educationists on education policy continued to be visible.
The educational debate resurfaced again during the democratic populist governments (1945–1964) that followed Vargas’ corporative regime. During that time the divide between progressives and conservatives deepened (Freitag, 1986). The pluralist Brazilian education community thrived until the establishment of the dictatorial military regime in 1964. The Brazilian Journal of Pedagogical Studies (Revista Brasileira de Estudos Pedagógicos (RBEP), 1944<), edited by the INEP and the Ministry of Education (MEC), was clearly in tune with the democratic government’s ideology and the stance of the progressive educationists. It favoured progressive over liberal, Catholic, conservative or authoritarian writers (Alvarenga, 2003). In addition, the journal covered international developments in education. A section entitled ‘Information from abroad’ (Informação do Estrangeiro) reported on perceived best foreign experiences, studies, practices and policies, and became an integral part of the journal. IO programmes and proposals, especially those from OAS (Organization of American States), UNESCO, UNICEF and, to a lesser extent, from USAID (US American Aid Institution) and the World Bank (WB), were also included. Dewey was the most commonly quoted foreign author in the RBEP until 1974 (Alvarenga, 2003), as well as in teachers’ manuals and textbooks, in which US organisations and foundations had a massive presence, followed by UNESCO (Silva, 2006). US policies and practices persisted as major good or inspirational models, the educationists referring to them in order to press for national reforms.
The programmes financed by the IOs were for the most part well received at the grass-roots level. They were carried out in cooperation with renowned domestic institutions, which were also, in fact, largely financed by these IOs, and supported by the educational and intellectual elite. The convergence of interests among Brazilian educationalists, politicians and foreign actors was evident (see, e.g., Centeno, 2010; Cunha, 1988; Paiva and Paixão, 2002).
Building international cooperation – tailoring hybrid development
In terms of educational cooperation, the democratic period (1945–1964) laid the groundwork for the presence of transnational actors in Brazil, as private and public domestic and foreign partners began to collaborate on educational policy and practice.
The Vargas ‘nationalism of ends’ foreign policy (Lafer, 2001) supported the identification of external resources for national development, a policy that populist governments assimilated later (1954–1964). Although other IOs (essentially UNESCO, see Haddad, 1996; Mendonça and Xavier, 2006), and countries such as France and Germany, played important roles in the development of Brazilian education, cooperation with the US was perhaps the most prominent. Both private and official agreements were, by and large, settled directly with Brazilian foundations and universities. Among the exceptions were three agreements made directly with the Federal Government (Aliança para o Progresso, 1964): the establishment of an audio-visual learning centre, which did not have major consequences; the setting up of the Brazilian–American Commission for Industrial Education (CBAI Comissão Brasileiro-Americana para o Ensino Industrial, 1945) and the Brazilian–American Assistance Programme for Elementary Education (PABAEE, Programa de Assistência Brasileiro-Americana ao Ensino Elementar, 1956). The two last-mentioned programmes had a strong impact on Brazilian education. At that time, quality in education was far from being connected with performance or assessment. Both the US representatives (e.g. Hart in Paiva and Paixão, 2002: 64–66) and the Brazilian educationists (e.g. Moreira, 1956: 48–59) associated the quality of schooling with educational provision on the one hand, and the meeting of growing demands for more and better schools, equipment and materials on the other: the educational structure and curriculum together with a mostly untrained teaching body were seen as major obstacles.
National, foreign policy and, consequently, educational multilateralism suffered a near reversal in the early 1960s: in response to massive inflation, the Jânio–Jango government proposed economic stabilisation, substituted the ‘nationalism of ends’ with an ‘independent foreign policy’ (Lafer, 2001: 97) and questioned the rationale of international cooperation (Centeno, 2010). The educationists seemed to take advantage of the international cooperation to achieve their domestic goals, but the government policy had strong repercussions on this bilateral cooperation. The educational community was largely favourable towards the Alliance for Progress programme (Lowenthal, 1991) that Kennedy launched in 1961, and INEP became responsible for providing the required education studies (INEP/CRPE, 1963). According to the educationists, cooperation would enhance educational development (e.g. Chagas Filho, 1962), infusing the ‘economic character’ of the Alliance framework with a ‘philosophic-humanist’ view through the operationalisation of the national plan (Fonseca, 2009). The Ministry of Education set up the Federal Council of Education (CFE, Conselho Federal de Educação) in 1962, which became responsible for the national plan for education 1962–1970, through which many of the goals of the Alliance for Progress were intended to be accomplished (Fazenda, 1985: 53).
However, the Jânio–Jango government sought detachment from the US as it became clear that educational cooperation entailed international political and ideological commitment, which the government was not willing to espouse (Centeno, 2010). It refused in principle to collaborate on new programmes (Alliance for Progress, 1965 [1961]) or to extend existing ones, and the Minister of Education publicly condemned the bilateral agreements. Yet, on the ground, institutions, foundations, universities and intellectuals supported US funding and ideas. First, in terms of new agreements, the USAID established more programmes between 1961 and 1963 than in the previous 10 years (Alliance for Progress, 1964), and the 11 agreements were all established directly with Brazilian universities and/or foundations. Second, the existing programmes, as in the case of PABAEE, were already under Brazilian direction.
Although educational bilateralism and multilateralism strongly increased during the democratic period, educational and socio-political issues, as well as governance procedures, followed domestic trajectories. The discourse on quality and assessment methodology was not yet problematised through more standardised QAE practices. The initial experiments in assessment began in the late 1960s and on the state level (Table 2).
The first steps in the development of large-scale assessment (1960s–1980s).
TDE: Testes de Desenvolvimento Educacional; EDURURAL: Northeast Basic Education Brazilian Project; MEC: Ministry of Education; WB: World Bank; INEP: Instituto Nacional de Estudos Pedagógicos Anísio Teixeira; SAEP: Sistema de Avaliacão do Ensino Público de 1. Grau.
At the beginning of the 1960s, Brazilian society was divided, as before, into sectors with conflicting interests (Ianni, 1975). A powerful think tank, the Institute of Research and Social Studies (IPES, Instituto de Pesquisas e Estudos Sociais) was established in 1961, which together with its political counterpart, the Brazilian Institute for Democratic Action (IDAB, Instituto Brasileiro de Ação Democrática), rallied the elite groups left outside the main intellectual, political and administrative arenas: liberals and conservatives; businessmen and centre-right intellectuals (Dreyfuss, 1981). The IPES/IBAD coalition promptly gained the support of the Brazilian War College (ESG, Escola Superior de Guerra), which was a rallying point for military opponents of civilian populist politicians, both being directly or indirectly sponsored by US companies and agencies (Bandeira, 1973; Dreyfuss, 1981; Skidmore, 1967, 1988). IPES/IBAD supported the military–civil coalition that overthrew the government in 1964, and little by little claimed total executive power (Germano, 2002). An extremist government with its severe measures remained in power until 1985.
The Military Government tried to gain the support of the educational community by placing educationists who were enthusiastic about the US propositions under INEP direction and in the Federal Council of Education. However, it failed to attract the full support of the educational field: student movements and professors’ interventions were supressed via repressive measures (Germano, 2002). On the other hand, the emergent intellectual elite comprising IPES/IBAD members became a shadow government with plans for action in fields such as educational reform, foreign investment and labour. From their strategic public positions (Plank, 1990), they supported the military administration and educational cooperation with the US and IOs (Centeno, 2010). The bilateral educational cooperation between the various Brazilian governments and the US during the military period (1964–1985) has been thoroughly studied (Alves, 1968; Arapiraca, 1982; Carnoy, 1974; Centeno, 2010; De Tommasi et al., 1996; Haddad, 2008; Lowenthal, 1991; Pereira and Pronko, 2015), and is therefore only briefly mentioned here. Through USAID, the cooperation took the form of programme loans, the first set of which corresponded to over 80% of the net Brazilian inflow of long-term capital between 1964 and 1967 (Skidmore, 1967). From an ‘independent foreign policy’, Brazil moved first to a foreign ‘interdependent policy’, trying to satisfy private investors and American supporters, and later to a ‘pragmatic policy’, progressively weakening its relations with the US while avoiding direct confrontation (Lafer, 2001). Educational cooperation, being closely intertwined with foreign policy, mirrored these developments.
American advisory teams tailored Brazilian education with the collaboration of the local elite and educational staff on the ground. Even though the two major reforms during the military period (the 1968 Higher Education Reform and the 1971 Reform of Basic Education) were designed under the MEC–USAID agreements, they could be seen as an ideological identification of the existing shadow government, with US educational and economic political views (Centeno, 2010; Cunha, 1988).
Whereas the educational reforms conducted during the dictatorship are well documented, little is known about the initial experiments on educational assessment. In general, it seems that evaluation did not attract the attention of the educational community between 1960 and 1986: only 7% of the scholarly work was in any way connected with it (Gatti, 1987), and the research on assessment in education and its outcomes was rather dispersed (see Gomes Neto and Rosenberg, 1995). The reasons behind this lack of scholarly interest may reflect the educational situation per se. The Education Law approved in 1961 established only four years of compulsory schooling, and it is known that the majority of Brazilian children in the 1960s stayed in school for less than three years (Brasil 1961; IBGE 2006; IPEA/PNUD 1996).
It is also known that the Centre of Studies on Tests and Psychometrical Research (CETPP, Centro de Estudos de Testes e Pesquisas Psicométricas / Fundação Getúlio Vargas) was established in 1966, and technicians were being trained to work on educational statistics and assessment at the Fundação Carlos Chagas. At the beginning of the 1970s, the CETPP devised tests to assess pupils’ achievements in four subjects (the last two years of secondary schooling, ensino médio), called Educational Development Tests (TDE, Testes de Desenvolvimento Educacional) and applied in just one state (Guanabara, nowadays Rio de Janeiro). Later, in 1980, the Department of Planning and Orientation at the Municipal Secretariat of Schooling in São Paulo (Departamento de Planejamento e Orientação da Secretaria Municipal de Ensino da Prefeitura de SP) administered performance tests to some cohorts on the Early Childhood and Basic Education (Educação Infantil and 1o. Grau) level, and this triggered lively discussion among educational staff (Gatti, 1987). However, the first Brazilian large-scale nationwide evaluation concerned higher education (1976). The Coordination for the Improvement of Higher Education Personnel (CAPES, Coordenação de Aperfeiçoamento de Pessoal de Ensino Superior), which was established in 1951, decided to evaluate all postgraduate study programmes in Brazil (Horta, 2007).
Experimenting with assessment – deepening international cooperation
More concrete discussions and studies on educational assessment in Brazil were conducted in the late 1980s, both through cooperation programmes between Brazil and various IOs and projects on the evaluation of schooling arising from Brazilian initiatives.
The earliest well-documented Brazilian pupil assessment was implemented within the Northeast Basic Education Brazilian Project (EDURURAL), which was initially put forward by the Brazilian Ministry of Education in 1977 with goals that were familiar from earlier programmes: enhancing access to primary education; reducing repetition and dropout rates; and improving pupil performance (Horta Neto, 2007). The programme was only implemented in 1980 through a loan agreement with the WB, which demanded an external evaluation of the outcomes of the project (Gatti et al., 1991). The evaluation period extended from 1981 to 1987, the aim being to determine the factors that influenced school achievement, and the cost-efficiency of educational inputs: it was a large-scale assessment covering three states (Gomes Neto and Rosenberg, 1995). Two Brazilian foundations (the Fundação Cearense de Pesquisa e Cultura backed up by the Fundação Carlos Chagas) supported the WB staff in its implementation. Although the results of this study were widely disseminated outside Brazil, they did not attract much public attention on the national level: MEC ignored the reports, and the discussion was restricted to education scholars (Gomes Neto and Rosemberg, 1995).
The MEC and the WB started to design the Second Northeast Basic Education Project (Projeto Nordeste) in 1984, although the agreement was not signed until 1994, after 10 years of discussions (Horta Neto, 2007). The project involved nine northeast states in addition to the Federal Government and the WB. One of the objectives was to provide a ‘systematic analysis of school results, measured in terms of pupils’ and teachers’ performance, and the school network’ (Brasil, 1994: 10).
The INEP designed a programme for the large-scale evaluation of primary and lower-secondary schooling in 1987. The aim was to provide information to the state Secretaries of Education about learning difficulties, and a Brazilian foundation implemented the programme (Gatti et al., 1991; Horta Neto, 2007). One year later the same foundation signed a contract with one state on the basis of an earlier INEP project, for the assessment of 30,000 pupils in 29 cities.
Several scholars, technicians, educational staff and students were actively involved in both projects, but the results apparently did not influence education policies (Gatti et al., 1991). Nevertheless, expertise and thinking developed through both programmes, arguably strongly affecting the later development of large-scale assessment in Brazil. The INEP acquired knowledge through its own project and also benefited from the Northeast Project (Coelho, 2008; Horta Neto, 2007). As a result, in 1987/1988 the MEC established the first systematic assessment scheme, the evaluation system of public primary and lower-secondary schooling (SAEP, Sistema de Avaliacão do Ensino Público de 1. Grau). The national launch was planned for 1989, but it was postponed for financial reasons (Horta Neto, 2007). SAEP prepared the ground for the first-generation ‘assessment of basic education in Brazil’ (Bonamino and Sousa, 2012), which took place in the 1990s.
The rapid growth of large-scale assessment
As discussed in the previous section, the ground for QAE had already been prepared before the post-dictatorial period and the dawn of a new era of stronger international involvement and economic growth in Brazil. International networks continued actively to feed the rapid growth in large-scale assessment schemes. Contradictory forces drove the developments, the decentralised government supporting the autonomy of municipal and state education providers while attempting to strengthen its central steering power through assessment. Meanwhile, the post-1980s changes in Brazil and more widely in Latin America were being challenged on two counts: the dire economic situation after the oil and debt crises and the related decentralisation tendency (O’Cadiz et al., 1999).
During the 1980s, the World Bank and the International Monetary Fund demanded structural changes that helped to reduce the size of the state sector and eased the regulation of salaries (O’Cadiz et al., 1999). These ‘structural changes’ aimed at ‘macroeconomic stabilisation’ also led Brazil into domestic economic stagnation: devaluation of the national currency, deregulation of the banking system, price dollarisation and decontrol, the privatisation of state companies, fiscal reform, higher inflation, unemployment, lower salaries, the marginalisation of a considerable proportion of the Brazilian population and gigantic cuts in the government’s social programmes (Chossudovsky, 1999).
In fact, decentralisation was a common post-dictatorial reaction in Argentina, Brazil and Chile – and was also a practice recommended by the World Bank in the 1980s (Meade and Gershberg, 2008). This was presented as an alternative to the former centralised management of public and social policy, favouring fragmentation and increasing the number of political actors (Oliveira, 2011: 325). Decentralisation was also evident in education policy. New institutional and functional state rules set out in the 1988 Federal Constitution reinforced the practice of compulsory and free-of-charge schooling, progressively extending it to secondary education and even to those who did not have access to school at the appropriate age (Brasil, 1988). A further effect was to stabilise the new organisation of national education, creating federal, state and municipal systems with different duties and obligations. Under the 1988 Constitution the municipalities became principal agents implementing public policy (Cavalcante, 2013). In terms of schooling they are responsible for preschool and primary education. The changes enforced school autonomy in three waves from 1987–1991: establishing school councils with responsibilities covering accountability, management and the school development plan; granting financial autonomy in the case of discretionary funds; and encouraging a shift away from politically elected principals (Meade and Gershberg, 2008: 309–310).
International organisations had a multifaceted role during this period, and in general were supportive of both decentralisation (for instance by increasing school autonomy) and more quality assessment. UNESCO has been involved in Brazil since the 1960s, and since the 1990s through the Education for All programme. According to Rambla (2012: 195), UNESCO was very prominent at the time, supporting ‘the Federal Government in order to compensate for the devastating effects of the structural adjustment. In the 2000s, it scaled down its activities to a role of technical consultant’. Later on, in 2001, UNESCO facilitated the Cochabamba Declaration committing Latin American governments to universalising basic education and improving its quality (Coben and Llorente, 2003: 102). During that period, the United Nation’s Economic Commission for Latin America and the Caribbean (CEPAL) and UNESCO emphasised a combination of equity and competitiveness as the key objective of the educational reforms (Drodge and Shiroma, 2004). UNESCO and CEPAL, and to some extent the World Bank, supported the reforms of the 1990s regarding school autonomy and teacher professionalisation in Brazil (Derqui, 2001: 567).
New QAE techniques were introduced to counterbalance the decentralisation. The 1990s saw a marked increase in liberal market reforms throughout Collor de Mello’s government (1990–1992), and continuing economic restructuring during the Franco (1992–1995) and Cardoso (1995–2003) governments (Derqui, 2001). The Brazilian government, in line with other countries, followed global managerial and efficiency trends in education, as well as in other spheres of public administration (see Secchi, 2009). Whereas the 1988 Constitution reshaped the governance of Brazilian education on the federal level via decentralisation, giving municipal-level governments the power to influence federal-level decisions on the local level (Borges, 2008; Fenwick, 2010: 156–157; Oliveira, 2011; Saviani, 2003), the Federal Government tightened its grip in the 1990s with evaluation (Derqui, 2001). Many researchers saw decentralisation as a problem that was linked to Brazil’s lagging behind all Latin American countries except Haiti in terms of performance indicators, to weakened economic competitiveness (Borges, 2008), and to increased disparities in educational quality in the regions, states municipalities and localities (Alves, 2007; Dourado, 2007; Durham, 2010; Franco et al., 2007; Gonçalves and França, 2008; Souza and Costa, 2009).
In general, the 1990s governments introduced stronger educational control in soft quality-control measures rather than direct control via legislation (Table 3). This could be understood as a feasible, and possibly the only, strategy through which to increase steering capacity in a decentralised system, as pointed out in a World Bank (2001) report. It involved a combination of increased autonomy and quality measurement. Indeed, the MEC gained more power through evaluation (Alves, 2007; Voss and Garcia, 2014). A federal system for the evaluation of basic education (SAEB, Sistema de Avaliação da Educação Básica), covering all schools, was introduced in 1988 during the democratic transition period, although Collor de Mello’s government was the first to use it in 1990. The Franco government reformed the SAEB and restructured the school census. The first reforms of the Cardoso government (1995–1999) on national curricular guidelines allowed more autonomy on the local level for choosing textbooks, but at the same time put more emphasis on evaluation (Burton, 2012: 96). In addition to implementing the national subject-based assessment of higher education ENC-Provão (Exame Nacional de Cursos, 1996–2003), Cardoso’s government introduced secondary-school assessment in the form of ENEM (Exame Nacional do Ensino Médio) in 1998, and an examination leading to a basic education degree for those who were not able to complete it before school-leaving age (Exame Nacional para Certificação de Competências de Jovens e Adultos (ENNCEJA)). Brazil also took part in the study organised by the Latin American Laboratory for Assessment of the Quality of Education (LLECE) and the UNESCO Regional Bureau for Education in Latin America and the Caribbean (OREALC) in 1997, and in the OECD’s PISA test in 2000 (see MEC, 2002; UNESCO, 2016).
Assessment in Brazilian basic education, 1990–2003.
SAEB: Sistema de Avaliação da Educação Básica; ENEM: Exame Nacional do Ensino Médio; ENNCEJA: Exame Nacional para Certificação de Competências de Jovens e Adultos; PERCE: Primeiro Estudo Regional Comparativo e Explicativo; LLECE: Latin American Laboratory for Assessment of the Quality of Education; OREALC: UNESCO Regional Bureau for Education in Latin America and the Caribbean; UNESCO: United Nations Educational, Scientific and Cultural Organization; INEP: Instituto Nacional de Estudos Pedagógicos Anísio Teixeira; PISA: Programme For International Student Assessment; OECD: Organisation for Economic Co-operation and Development.
It was not only the development of performance indicators, but also the funding distribution, that was being controlled more tightly on the national level. FUNDEF (Fundo de Manutenção e Desenvolvimento do Ensino Fundamental e de Valorização do Magistério), a 10-year programme equalising the regional distribution of funding, was introduced in 1997 to address the performance challenge (Borges, 2008). FUNDEF ‘guaranteed that 15 per cent of municipal and tax revenues would be allocated directly to primary schools (of which 60 per cent would be spent on teachers’ salaries)’ and ‘set a minimum level to be spent per primary school student’ (Burton, 2012: 96).
By the end of the Cardoso government, national education was ‘fragmented, segmented, flexible and sprawling’ (Oliveira, 2011: 327). Indeed, the government turned the decision-making in a more managerial direction (Fonseca, 2009: 168). On the one hand, as Fernandes (2007: 8) claims, having the kind of synthetic indicator of educational development that was introduced later on during the Luiz Inácio Lula da Silva (Lula) government was desirable in terms of detecting schools and education systems with low-performing pupils and monitoring the temporal evolution of pupil performance in them. On the other hand, it was clear that the quarter of a century of SAEB had established the position of managerialism in Brazilian educational governance (Sousa, 2014).
The general change in steering persisted in the Lula government (2003–2010), during which the QAE policies were substantially reformed through the restructuring of predecessor initiatives and the development of new assessment indicators. International large-scale assessments such as the OECD’s PISA and TALIS (Teaching and Learning International Survey) and the new LLECE evaluations have also become commonplace since the Lula government (Table 4). The government broadened the FUNDEF scheme via FUNDEB (Fundo de Manutenção e Desenvolvimento da Educação Básica) in 2006, supporting basic education, with a 20% contribution from state and municipal funding and a federal contribution of 5% (Burton, 2012: 96), and expanding the coverage from early-childhood education to secondary school.
Assessment in Brazilian basic education, 2003–2015.
In 2005, the SAEB included the large-scale assessments of ANEB (Avaliação Nacional da Educação Básica, National Evaluation of Basic Education) and Prova Brasil (officially ANRESC Avaliação Nacional do Rendimento Escolar, National Evaluation of School Achievement), the latter covering all public institutions on a universal basis: in 2013 it also included the ANA (Avaliação Nacional da Alfabetização, National Evaluation of Literacy).
Prova Brasileira do Final do Ciclo de Alfabetização (Brazilian Test at the End of the Literacy Cycle) is an initiative of the ‘Todos Pela Educação’ movement, in partnership with the INEP (Instituto Nacional de Estudos Pedagógicos Anísio Teixeira).
SAEB: Sistema de Avaliação da Educação Básica; IDEB: Index of Basic Education Development; ENEM: Exame Nacional do Ensino Médio; ABC: ; ANA: Avaliação Nacional da Alfabetização; PISA: Programme for International Student Assessment; GIP: Grupo Iberoamericano de PISA; SERCE: Second Regional Comparative and Explanatory Study; LLECE: Latin American Laboratory for Assessment of the Quality of Education; OREALC: UNESCO Regional Bureau for Education in Latin America and the Caribbean ; TERCE: Third Regional Comparative and Explanatory Study; UNESCO: United Nations Educational, Scientific and Cultural Organization.
Some of the most influential quality-assessment policies were developed during Lula second government. SAEB was reformed to incorporate two large-scale assessments covering all schools instead of only a sample (Table 4). Furthermore, the 2007 IDEB (Index of Basic Education Development) combined the SAEB scores and school census (Censo Escolar), for example. Moreover, the 2007 PDE is considered the strongest initiative of the Lula government in the field of education, reflecting the influence of the international organisations. In fact, the World-Bank-funded FUNDESCOLA (Fund for School Strengthening and Development, Fundo de Fortalecimento da Escola) programme was re-formed as the PDE-Escola (School Development Plan, Plano de Desenvolvimento da Escola), which was part of the PDE (Fonseca, 2009: 170). It has been claimed that it promotes the necessary educational reforms to allow Brazil into the global development project, aligning Brazilian education with the PAC (Programa de Aceleração do Crescimento, Growth Acceleration Programme) (Ghiraldelli, 2009). The Federal Government has assumed responsibility for educational politics, pursuing partnerships with municipal bodies and valuing governance among subnational levels and civil society (Oliveira, 2011; Saviani, 2003: 196–197). The PDE-Escola currently works in conjunction with the IDEB in that priority is given to schools with lower IDEB scores (MEC, 2016).
Furthermore, the explicit goal of the PDE in 2007 was for the IDEB score to reach the OECD average level by 2021 (INEP, 2016; PDE, 2007), thereby achieving full alignment with international large-scale assessment models. In addition to the other large-scale activities indicated in Table 4, the Rousseff (2011<) government lists 20 goals in the National Education Plan (Brasil, 2014) for 2014–2024, of which 11 are directly linked to the quality of basic education (MEC, 2014). The plan follows the systemic model of the 2007 PDE, the objectives including the eradication of illiteracy, the universalisation of basic schooling (from four to 17 years old) and the expansion of school access and service on all educational levels. Increases in public-education financing to 7% of Brazil’s gross national product in 2018, and 10% in 2024, are planned to ensure that the aims of the PNE will be achieved (Brasil, 2014). In the document, the goal referring to quality has the greatest number of strategies. Among these, the 2007 IDEB score goals are reinforced and the connection with PISA is now rearticulated with scores for mathematics, literacy and science goals for 2015, 2018 and 2021 (Brasil, 2014).
Despite the federal-level attempts at QAE steering, the main responsibility for education still lies with the foremost public providers of basic education: the states and the municipalities. Brazilian states have maintained their strength since the end of the monarchy, assuming responsibility for education provision, together with the municipal authorities, since the early republican era (Borges, 2008: 237–238). The levels of coordination between the different levels vary. Azevedo and Santos (2012) found in their study of the Recife metropolitan area, based on evidence from documents and interviews with municipal managers (gestores municipais), that there was little vertical and horizontal cooperation between federal and municipal governments. Indeed, the parallel existence of federal, state and municipal education is also reflected in the existence of multiple policies and quality-assurance systems: States manage primary and secondary education and some universities. They have to implement their own PDE. The richer ones, mostly governed by opposition parties, have established their own performance-based schemes of teacher salaries, and reject the Federal requirement to guarantee a minimum. (Rambla, 2012: 195)
In other words, the national large-scale assessments operate in parallel with different state and municipal systems.
Although large-scale assessment and QAE measures were boosted in the post-dictatorial period, there are various views on the reforms in the education field. Gusmão (2013) identified three sets of actors in Brazilian education policy with differing views on quality. Those in the first group (including the WB) see the IDEB as a learning-assessment instrument for enhancing the quality of education. Members of the second group (MEC and UNICEF) recognise its legitimacy as a quality indicator, but also its limits. The actors in the third group, primarily comprising people involved in the processes and life conditions of the students and the teachers’ work, point out the limits of the evaluation indicators in that they ignore the fundamental factors and aspects of the educational process.
Exemplifying the growing Brazilian cooperation with international partners, Table 5 shows the activity of the World Bank in Brazil, and more specifically in the field of education. First, there was a rising trend in WB activity after 2000. Second, activities related to education started to gain momentum first in the 1990s and then in the 2010s. As noted above, the MEC began strongly to encourage concrete discussions and studies about educational assessment in Brazil in the 1980s. The programmes of the IOs significantly affected this new governmental stance. As Jones (2004) aptly points out, funding is always bound up with ideas. Its acceptance always implies acceptance – however defined – of the perspective to which the subvention is attached. International (e.g. EDURURAL with the World Bank) and bilateral (mainly with the US through USAID) cooperation aimed at promoting equality of educational chances for all was centred on the rationalisation of the school system (Rodrigues, 2007).
Number of World Bank (WB) projects in Brazil: all and education-related projects.
Source: World Bank (2015).
The development of quality assurance and evaluation has been open to international influences, intertwined with cooperation involving IOs and the networks that were built during the 20th century. The institutionalisation of QAE, especially via INEP, was seen as a way of regaining some control of a situation that arose from post-dictatorial decentralisation. Regardless of the government’s place on the political map, QAE in education has been growing in strength as a policy. However, given the varying roles of the IOs in this process, and on the basis of the material we have here, it is impossible to establish a cause–effect relation. What is clear is that as Brazil has gained a more prominent place in the world, and supposedly is less dependent on funding from IOs, and there has been not only a strengthening of the political will to produce more data through QAE, but also a growing number of education-related projects with international partners.
Conclusion
Against the background of earlier ideas about education transfer in the context of comparative education, research on educational quality and the recent history of Brazil, our research gives a rather multifaceted picture of how large-scale assessment became a significant feature of Brazilian education policy, and of role of the IOs in this. We do not claim to have given an exhaustive account, as more empirical research is needed on the matter, but more importantly, our findings raise questions for further study on QAE in Brazilian education.
When we started our work we did not subscribe to the regionalism argument or the imposition argument, both of which permeate the literature on Latin American research (see Beech, 2002). Indeed, Brazil has worked with IOs in times of economic stringency and strength. Education-indicator programmes have blossomed since the 1990s, alongside global education-related IO programmes. Independently of their political position, all the post-dictatorship governments took steps towards implementing assessment with a view to improving quality in education, a strategy that mirrored global developments. In this development we have pointed out the importance of the long build-up of QAE capacity and webs of experts.
In addition, from the perspective of comparative education we analysed the agency of Brazilian actors and how they forged relationships with international actors. In enhancing understanding of the transnational dynamics of QAE in basic education in Brazil, we have attempted to develop a more complex picture of Brazilian interaction with international partners in education policy.
Given the results of our analysis we conclude that endogenous factors such as the existence of a vivid and influential educational community facilitated the Brazilian QAE build-up, and its development was closely intertwined with Brazilian socio-political developments. In addition, the historical pattern of ‘externalisation’ processes in the Brazilian educational field points to a particular ‘socio-logic’ (Schriewer and Martinez, 2004). The educational community and its actions and debates were permanently established through the adoption of, or simply with reference to, foreign ideas, policies and practices, which were filtered according to contextual Brazilian conditions, problems and issues. International examples, mostly US ideas, persisted both in the development of theories, practices and policies, and in the legitimation or de-legitimation of political stances. However, the same questions of quality persist: educational provision; schooling progression and achievements; and teaching and administrative capacities. Furthermore, similar questions concerning structural inequality and wealth distribution have been on the agenda since the establishment of the educational field, in terms of thinking and policy. Despite the divergence in reasons and motives among the groups, the channelling of international cooperation according to different interests and purposes and the increasing presence of IOs in Brazilian education show that global influence and imposition do not capture the complexity of the relationships between the actors.
Subscribers to the world-culture approach could explain how the global script (Meyer and Ramirez, 2003; Ramirez, 2012) of measuring quality became the bread and butter of Brazilian education. Brazilian experts working in different institutions acquired some of their expertise from abroad, and it could be argued that they adopted the practice of measuring quality by means of quantitative indicators. Nonetheless, the pedagogical tradition and divisions among intellectuals served to formulate questions for national purposes. In fact, we argue that Brazilian education experts adopted a form of communication rather than a script. The adopted thinking related to conveying the problems, but the local problematisation remained the same, close to the idea presented in the research of educational transfer emphasising the importance of the local context (Ozga and Jones, 2006). There was no clear import of a set model, but as pointed out above, QAE was embedded in local ideas of quality, especially in the context of decentralisation. It was not a script for a structure that emerged from the local conditions, but networks, new institutions and a shift in understanding of what is relevant knowledge. However, the idea of a global script gains some support from the fact that the Brazilian large-scale-assessment scheme is oriented towards PISA.
All in all, our review leaves open a few questions for further analysis. There is a need for more empirical research on the interplay between IOs and national actors. We have gone into some detail about past developments in the Brazilian community of educational experts, and given the increasing involvement and influence of social movements and non-governmental organisations there is now a more specific need for research into recent developments. We call for further analysis with more degrees of complexity, and we hope that our aggregative work will serve as a basis for continuing the investigation.
Footnotes
Acknowledgements
We would like to thank Roseli Kuhnen (Federal University of Santa Catarina) for her comments on an earlier draft of this article.
Declaration of Conflicting Interest
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by the Academy of Finland (grant number 273871).