Sage Journals: Discover world-class research

Abstract

This article aims to understand how new accountability instruments in the context of the French-speaking Belgian educational system are appropriated by schools. After having characterised the specific nature of those instruments in the context of a traditionally highly decentralised system involved in a significant process of centralisation, we identify their effects through the case study of three schools. Using a new institutionalist lens, the analyses show that these instruments refer, in the French-speaking Belgian context, to a specific demand from the political environment of schools: developing and framing a common educational landscape, rather than to a logic of teacher evaluation. The data also indicate a reaffirmation, against this specific political demand, of three traditional ways of functioning tied up to the requests made by local educational communities. Thus, the analyses show a conflict between inherited institutions highly embedded in local contexts and the political signal associated with the new accountability instruments aiming to institutionalise common norms at the system level.

Keywords

Educational policy accountability testing new institutionalism New Public Management French-speaking Belgian school system

Introduction

The main objective of this article is to analyse the local implementation of specific accountability instruments in the French-speaking Belgian school system. This system is characterised by a twofold principle enshrined in the Belgian constitution: freedom of educational provision (for private and public providers); and freedom of school choice (for parents). This historical twofold freedom (1831) has given rise to a decentralised school system and a quasi-market regulation (Danhier, 2018; Vandenberghe, 1998), independently from any New Public Management doctrine. By the mid-1990s, specific accountability instruments made their appearance in this system: mainly educational standards and testing, coupled with recommendations on classroom practices for teachers. The emergence of these instruments is generally interpreted in French-speaking Belgium as a process of centralisation, against an educational system historically characterised by local autonomy (Draelants et al., 2011; Dupriez and Maroy, 2003).

Developing a neo-institutional (NI) argumentation, we argue that these instruments aim at imposing a common cognitive framework in this educational system. Studies developed in other contexts (e.g. Spillane et al., 2011), have highlighted that under certain conditions, accountability instruments allow to recouple local cognitive frameworks to political demands and instruments. This article analyses whether and how this occurs, in a context historically characterised by decentralisation and quasi-market regulation, a matching between the common framework established by these structuring instruments and the local institutions.

In the first section of this paper, we situate the accountability instruments in both the historical and contemporaneous context of educational policy in French-speaking Belgium, and we define our main theoretical concepts anchored in the sociological new institutionalism. Next, we present three case studies showing a very partial recoupling between the educational norms involved into the accountability instruments and the local institutions. Finally, we discuss our results by returning to the specific nature of the coupling/decoupling dynamic in the reflexive accountability context.

The Belgian French-speaking case: Testing and ‘reflexive’ accountability as instruments of progressive centralisation

The education system in French-speaking Belgium emerged as an autonomous structure in 1989, in the context of a major reform of the Belgian state that decentralised its organisation. This reform gave rise to three distinct education systems corresponding to the three cultural communities in the country: one Dutch-speaking, one German-speaking and a third – our focus here – French-speaking. Each of these systems was to inherit the Belgian constitutional structure, which, in the area of education, allows considerable freedom for initiatives of varied origins, whether private or public. This freedom was initially intended to permit each of the society’s two founding pillars (secular and Catholic) to create institutions adapted to its community and its cognitive references. Mainly, two categories of schools may thus be observed: on the one hand, those organised by the public authorities (the state itself, the municipalities and the provinces) and, on the other hand, schools tied to the Catholic community.

Secularisation and social fragmentation contributed to a double renewal of the meaning of the freedom established at the beginning of the 19th century. First, one may observe schools with characteristics quite different from this original bifurcated cleavage, involving alternative teaching methods, denominational schools associated with other religious orientations and so on. Secondly, the choice of schools by families has increasingly become a strategic one, based on an attempt to match personal preferences and an educational programme, rather than a desire to take a position in the ‘historic’ divide opposing secular and Catholic pillars (e.g. Draelants et al., 2011).

This context explains that the French-speaking Belgian educational context can be considered as an educational quasi-market (curricular and pedagogical autonomy, school choice and vouchers) (Vandenberghe, 1998) even if this inherited quasi-market system has not resulted from any New Public Management policies.

In this historical quasi-market context, the political authority has imposed educational standards and testing since the early 1990s in secondary schools. However, as shown in Table 1, classical accountability instruments do not exist in this educational system: contractualisation on targets, sanction and rewards associated with results and rankings are absent.

Table 1.

Accountability instruments in French-speaking Belgium.

Accountability instruments (secondary schools).
Harmonisation instruments		Distrust instruments
Testing	Teaching	Contractualisation on targets	Sanction and	Rankings
	standards		rewards
CE1D (year 8)^a	Yes	No	No	No

Note: ^asciences, French, mathematics, and modern languages. The lower secondary cycle also includes a ‘differentiated first cycle’ (premier degré différencié), which is a stream intended for young people who did not pass the primary school tests necessary to obtain the Basic Studies Certificate (Certificat d’Études de Base – CEB).

In French-speaking Belgium, teachers receive test score analysis as well as ‘didactic guidelines’ and benchmarks comparing the scores obtained with those of similar schools. These instruments (listed in Table 2) are provided to the schools after external tests. Some tests lead to certificates, such as the ones organised as part of the CE1D¹, which this article is focused on. These instruments can be seen as an application of a ‘reflexive’ accountability model based on trust in teachers’ autonomy (Klieme et al., 2004 in Mons, 2009; Mons and Dupriez, 2011), as it does not include sanction/reward, contractualisation or rankings. In this model, instructional change relies more on the use of ‘reflexive instruments’, amongst which the tests scores, that give teachers reliable information about their ability to reach the common educational standards on which the tests are based (Klieme et al., 2004; Thélot, 2002). In addition to the results at the external assessments, we can find amongst those ‘reflexive instruments’ many documents (Table 2) that must, in a way, fuel and structure reflexion by giving to the teachers directly usable examples of new didactical attitudes and practices corresponding to the will of the educational authority (Dupriez, 2015).

Table 2.

Reflexive accountability instruments.

Testing	Low-stakes ‘reflexive’ accountability instruments
	− Digital versions of previous years’ tests are made available to teachers on the official French-speaking community website
	− Teacher and student tests’ outcomes are transmitted to school administrations
CE1D (year 8)
	− Anonymous comparison with similar schools (in terms of student body) is produced and transmitted to the head teacher. Failure to comply with the prohibition on distributing these scores is subject to heavy penalties

Note: Tests in mathematics, French, sciences and modern languages

Standards, testing and ‘reflexive instruments’ (listed in Table 2) constitute mainly in French-speaking Belgium instruments aiming at the harmonisation of teaching practices (Carette and Dupriez, 2009). A vast body of literature shows that this process reveals a rise of the public authority in this fragmented context, rather than the development of managerial logics or competition between schools (Draelants et al., 2011; Dupriez and Maroy, 2003). The absence of sanctions, rewards and contractualisation in favour of ‘reflexive instruments’ can be explained by the context discussed above, namely the ‘historical’ educational quasi-market that has given rise to strong differences and a high-level of between schools’ segregation (Danhier, 2018). The centre-left political authority dominating French-speaking Belgium without interruption since its creation has formally prohibited the publication of test scores in order to avoid sparking competition between schools (Draelants et al., 2011).

Theoretical framework

Our contribution is situated within the literature aiming to understand the implementation of accountability policy and standard-based reforms (Ball, 1993), and more specifically with a cognitive reading of the implementation (Spillane et al., 2002) inspired by the new institutionalism in sociology. From a cognitive perspective, a key dimension of the implementation process is whether, and in what ways, implementing agents and organisations come to understand their practices, potentially changing their beliefs and attitudes in the process. In this paper, we consider that each school organisation, where implementation takes place, is characterised by specific ‘local cognitive scripts’ (Scott, 1995) that constitute together the sets of cognitions shared by a group’s members and lead them to consider the reality where they act in the same way. These constructions make sense of local attitudes and beliefs by providing common cognitions to the actors on key issues: How to teach? What is a good student? What is the purpose of a school? How to organise and evaluate learning? From this perspective also, political instruments (e.g. accountability instruments) are not only considered as regulation devices operating through the regulative pillar of the institutions (DiMaggio and Powell, 1983), but also as embedded within worldviews and cognitive frameworks guiding the perception and the comprehension of the social world. The implementation is thus analysed as the relationships, potentially conflictual, between the locally embedded cognitions and the worldviews embedded into political instruments and discourses. Empirically, this means understanding to what extent accountability instruments imposed by their political environment influence (or not) the cognitive scripts fuelling the Belgian schools which are, given the historical decentralisation, quite different from each other. This perspective on policy implementation is different from the one informed by the neo-Foucauldian concept of governmentality which insists much more on the (new forms of) power relations between ‘the centre and the periphery’ (e.g. Simons, 2007). From this theoretical position, testing and accountability instruments are centrally-imposed technologies that can enable both auto and mutual surveillance (Dean, 2010). In the NI apprehension of the implementation, the concept of power is also present (see Lawrence, 2008), but it does not constitute the core component of the approach. More specifically, we thematised the issue of the relationship between locally embedded cognitions and politically driven worldviews with the concept of decoupling. This concept has composed the conceptual apparatus of the organisational NI since its origins, to insist on the disconnection between the formal organisational structure and the ‘real’ observable flows of activities taking place inside organisations. Schools, being embedded in a highly institutionalised field, have served to support the empirical demonstration of the decoupling. The argument is that the school formal organisational structure is adopted in a ceremonial way, mainly for legitimacy reasons. This structure represents political demands from the institutional environment (Meyer and Rowan, 1977) but does not affect local cognitive scripts that frame daily actions and actors’ representations (Scott, 1995).

In recent years the concept of (systematic) decoupling has been the focus of major theoretical debates within the organisational NI. Those debates, initially centred on the integration of strategic and institutional views on legitimacy (see Suchman, 1995), have progressively emphasised on the importance of the ‘local’ in the institutionalisation process (‘coalface institutionalism’, Barley, 2008). The idea is that the institutional change (and stability) does not only come from institutional forces situated at the field level (the cognitive, regulative and normative pillars), but is also a matter of interactions between actors within organisational contexts. We should add that, within the educational research sphere, this argument has received significant attention, in particular to understand new kinds of instructional reforms (e.g. standard-based reforms and accountability policies) that have been precisely for objectives to influence more directly or monitor the instruction within the classroom context. Neo institutional scholars have identified, in high-stake testing contexts, the emergence of a recoupling process, where the formal structure is no longer simply adopted in a ceremonial way, but ‘incarnated’ in practices and views of teachers (Diamond, 2012; Hallet and Meanwell, 2016; Rowan, 2006; Spillane et al., 2011).

Drawing on an extensive review of the literature on accountability (beyond the NI approach), Hallet (2010: 69) suggests four arguments theoretically in favour of such a recoupling hypothesis in a context of high-stakes accountability policies. The first is that the implicit reasoning underlying the accountability rationale – making teachers accountable of their responsibilities – destabilises the traditional confidence in their own skills, which constituted one of the key elements in the decoupling process. Second, while ‘classic’ decoupling was explained in large part by the existence of a grey area relative to the teachers’ performances (‘what we attend to and how we respond’, in Hallet’s words), test scores now provide more precise indications about the effects of classroom work. Third, standards, which are at the heart of accountability rationales, also prevent decoupling between policies and practices because their diffusion creates benchmarks and new constraints that come to influence teacher practices. And last of all, the existence of rewards and sanctions also explains the closer surveillance of practices with the new policy demands (Hallet, 2010: 69).

However, this NI literature, mostly developed in a context of high-stakes accountability, leaves unanswered the issue of appropriation in low-stakes ‘reflexive’ accountability contexts such as the context of the French community of Belgium studied in this article, where the instruments are different in their consideration of the professional autonomy of teachers and the importance given to the ‘reflexive’ dimension (Dupriez, 2015). How are these low-stakes ‘reflexive’ accountability instruments locally implemented? Does their use in the schools lead to a dynamic of recoupling between locally embedded cognitions and the worldviews promoted by instruments? Is this recoupling process similar to the one described by the literature in high-stakes accountability contexts?

Method: Three case studies

Our objective in this article is to analyse to what extent and how low-stakes ‘reflexive’ accountability instruments recouple schools’ local institutions with those embedded into the instruments. In order to achieve this purpose, we conducted three case studies in three secondary schools selected after an exploratory study conducted with a sample of 29 schools (removed for blind review). Following theoretical sampling criteria, we have chosen three schools having potentially demonstrated tensions between local institutions and worldviews promoted by the instruments. Each of the schools (see Table 3) differs from the central norm promoted by the instruments for a specific reason. The first one for being an ‘elite’ school (Saint Ignace, a highly demanded school providing general education), the second for having an alternative teaching approach (where assessment is above all formative) and the third one for being a vocational school (with students from low socio-economic backgrounds).

Table 3.

Site selection.

	Information collected and observations			20 interviews
	Socio-	Curriculum	Pedagogical	Round 1	Round 2	Round 3
	economic level	orientation	rationale	9-2013	12-2014	6-2015
Saint Ignace	Very high	General education	Classical and elitist	Principal and teachers	Teachers	Teachers
Rousseau II	Weak	Vocational school	resocialising	Principal and teachers
Claparède	Quite high	General education	Alternative

Between September 2013 and June 2015, 20 interviews with school stakeholders were carried out (see Table 3). Those interviews were supplemented by a copy of the school mission statement and non-formalised observations in the faculty room. For each school, we had also access to a synthetic sociocultural index (between 1 and 20) based on a formula with five distinctively weighted variables ( students’ areas of residence, per capita income, diploma level, unemployment rate, professional activities and residential comfort levels), as well as to databases describing the neighbourhoods in which each of these schools are located in terms of age, unemployment level, and proportion of population with foreign nationality.

During the first round of interviews in September 2013, we met with school principals and teachers (14 interviews in total). The interview guide derived from a study previously conducted in primary education (Cattonar et al., 2010: 62). We slightly adapted the items to understand the within-school institutions framing attitudes and beliefs and how they (mis)match with the worldviews promoted by the educational standards, the tests and the reflexive accountability instruments. We made a vertical analysis, by school, of all our data (qualitative and complementary data). This analysis describes for each case the cognitive scripts structuring perceptions and uses of tests and related instruments. During the second round of interviews (December 2014), we met with three teachers in order to complement our data. Finally, during the last round (June 2015), we presented our tentative interpretations to three teachers in each school to enrich those interpretations.

Results

The presentation of results starts by describing the three schools and the cognitive scripts that structure locally the attitudes and beliefs towards the tests. Afterwards, we explore the decoupling hypothesis and possible related factors.

Looking inside the schools’ cognitive scripts

The first school, Saint Ignace, is a denominational school located in a socio-economically diverse area with six competing schools. It has a socio-economic-status composition higher than surrounding schools, thus showing that the families choosing Saint Ignace are amongst the most privileged of the area. According to the school principal and the teachers, parents are attracted by the school’s historical reputation combining rigour and discipline:

I know many teachers or principals of rural elementary schools near that, according to the pupil’s academic level, say: ‘You can try Saint Ignace in high school. Based on the academic level’. (Principal, Saint Ignace, September 2013, 228–231)

Teaching at Saint Ignace is inspired by norms such as academic excellence and meritocracy, which explains why teachers insist on huge efforts required from the students as well as on the strong selectivity of tests organised by teachers compared to the level expected at the compulsory tests such as the CE1D:

We warn the parents when we think that a child won’t pass into the third grade. They often leave by themselves because they feel like they do not belong since success is so hard to achieve. (French Teacher, Saint Ignace, December 2014, 412-415)

(…) Over 250 students in the second grade, I had between 40 and 50 failures in French, between 40 and 50 in mathematics. Which means between 20% and 25% of the school. (Principal, Saint Ignace, September 2013, 35-38)

At Rousseau II, the project also relies much on the history of the school and the existence of its ‘noble’ identity but as a vocational school of excellence (particularly in some sectors such as catering). However, teachers and the principal insisted on recent changes in the students’ recruitment linked to the highly deprived area in which the school is located. Those changes imply more centrality in the mission of socialising young people identified as ‘marginalised’. In this regard, local institutions forming the basis for learning diverge significantly from the ones described above at Saint Ignace. The objective at Rousseau II is not to achieve academic excellence but rather to restore the self-confidence of these young people and to help them to (re)construct a personal project:

We try to restore students’ confidence. These are often students who have encountered many difficulties before. So here, we make small products [for instance pastries or a main dish] in the workshop that they can take back home to show parents that they are no longer in failing school (…). I do not put any students in failure. (Practical Teacher, Rousseau II, September 2013: 119–122)

My principal (…) considers every socio-economic factor and what’s important to her is that the children feel valued and active members of the school (History-Geography Teacher, Rousseau II, December 2014: 314–317)

At Claparède, history also matters to explain the origins of the school’s alternative educational model. The school differentiates itself from surrounding schools by its innovative instructional methods and assessment. Student assessment is only formative and qualitative in nature, and this characteristic seems very central in the school’s cognitive script:

There is no grade average in the assessment of these skills. Periodically, a report card shows how the student is getting by in his learnings. Comments indicate the strengths and weaknesses of the student. (Quote from the pedagogical project of Claparède)

The socio-economic index of this small school is very high and the surrounding area (an airy urban setting) is characterised by a low unemployment level and an average income well above that found in the rest of the city:

It [the external test] doesn’t measure very much …or nothing. We’ve reached such a variety of student profiles. Between the students who’re here at Claparède and those who are, I don’t know, at Seraing or Charleroi [Belgian municipalities that are clearly less well-to-do than the neighbourhood where Claparède is located], we’re not at all in the same kind of background. These aren’t students with the same cultural capital. (History–Geography Teacher, Claparède, September 2013: 121–125)

Importantly for our purpose, not only our analysis shows highly differentiated local institutions, but also that those institutions lead teachers to consider test scores at standardised tests as poor descriptors of the quality of their work. Test scores seem rather interpreted by teachers as the reflection of the three school’s student profiles rather than of the quality of their work: students are socio-economically privileged at Saint Ignace and at Claparède, and students are in deep difficulty at Rousseau II:

I’m in a privileged school, my students do well. The external evaluation is a detail, it’s really something where they’re going to score 95, 97, 98%. (French Teacher, Saint Ignace, December 2014: 75–77)

On top of this argument, the low significance of test scores at Claparède is also linked to the misfit between the standardised assessment method and the pedagogy promoted in the school, which insists on differentiated learning paths and alternative assessment methods:

As we start [according to the local pedagogy] from the student’s assumptions and what they bring us (…) there are things we study in the first grade and in the third grade [while the testing session takes place at the end of the second grade]. (French Teacher, Claparède, June 2015: 243–249)

Towards recoupling?

This perceived disconnection between teaching quality and test scores arguably has not encouraged teachers to adopt the supposed ‘reflexive’ attitude (Klieme et al., 2004; Mons and Dupriez, 2011), which might explain the very limited reported impact of tests on classrooms’ realities. At Saint Ignace and at Rousseau II, teachers explicitly reported using rarely or never the didactical guidelines and previous tests to ground their instructional practices:

We teach as we did before [before the CE1D’s introduction].

(History–Geography Teacher, Saint Ignace, September 2013, 567)

Teacher: To be honest, I didn’t read them when I received them. I haven’t read them at all. The first time I consulted them (…) was for an academic work, otherwise I wouldn’t have opened them.

Researcher: Do you think your colleagues did the same thing?

Teacher: Oh yes! Totally! For sure! It isn’t a work instrument at all for us. We don’t rely on it. (French Teacher, Saint Ignace, June 2015: 171–177)

Some teachers in the two schools however seemed to include some parts of the ‘reflexive’ instruments (didactical guidelines, previous tests, etc.) in their practices, but in a fashion that does not compromise their usual practices. This ‘customised’ implementation is again in tension with the low-stakes accountability model according to which the instructional change should be triggered by accountability instruments and not the reverse (Klieme et al., 2004; Thélot, 2002):

If a text interests me and the question raised is – in my opinion – valid for what I myself do for skills assessment, then I use it, obviously. So yes, these are things I’ve already used. But I don’t do it on a regular basis. (French Teacher, Rousseau II, September 2013: 103–107)

I read them from time to time to have an idea of the type of exercises (…). If relevant for me. (Mathematics Teacher, Saint Ignace, September 2013: 139–141)

It is in the alternative school (Claparède), that the decoupling appeared the strongest. Teachers reported no implementation of instruments, and justified this regarding the fundamental conflict with their views on instruction and student assessment:

On the pedagogical level set, for us, it’s a real problem. We only assess a part of the skills we value as fundamental to pass a class. For instance, we assess note-taking and the quality of the researches a lot. Therefore, they can’t be assessed with a CE1D. (French Teacher, Claparède, June 2015: 70–75)

At some point in the school year, students at Rousseau II and Saint Ignace are deliberately exposed to the tests from previous years. Nonetheless, these ‘familiarisation’ periods aimed precisely to keep students from being destabilised by the form of these tests (type of questions, duration, etc.). It does not consist in an intensive training leading to learnings, nor in an influence from the pedagogical standard structuring the external assessment, but rather in a specific practice unrelated to the activities during the rest of the school year which is protected from possible influences of the instruments:

I do a lot of group assignments, games, we try to have a lot of exercises. Now, during the period before CE1D, it is pure training, that’s for sure. (History–Geography Teacher, Rousseau II, December 2014: 294–296)

Contrary to the general trend of decoupling, a minority group of teachers in two schools (Rousseau II and Saint Ignace) showed themselves to be more open to the tests and their possible implications for classroom practices. At Saint Ignace, this minority group is mainly composed of younger teachers describing themselves with distance towards the institutionalised values of meritocracy and excellence:

I think many are opposed to this because of the academic requirement’s level. I try to think: ‘If the level changes a little bit, that’s a good idea’ (Mathematics Teacher, Saint Ignace, September 2013, 381–383)

With my colleagues, we made the entire second grade lesson again (…). We drew many things from the ‘didactical benchmarks’ from the previous years. We reused many texts. (French Teacher, Saint Ignace, December 2014: 113–115)

There’s a kind of elitist tradition at the school. It’s very clear, we have to be honest about that. An elitist tradition that the young teachers aren’t particularly into. (French Teacher, Saint Ignace, June 2015: 339–340)

At Rousseau II, a minority of teachers also reported a more positive attitude towards the tests and used them significantly to reflect on their practices and to a certain extent modify those with some test materials. Those teachers have in common not to teach the Vocational Education and Training courses, which possibly explain why they do not perceive themselves that much at distance with the standards imposed by the tests:

There are even teachers who do CE1D drills during their lunch hour for the students who want them. (History–Geography Teacher, Rousseau II, December 2014: 311–313)

We devote much more time to giving them tests from previous years. (French Teacher, Rousseau II, September 2013: 344–345)

We gather, and we mark the assessments. Three of us do the marking (…) and thus we have the list of every student with every grade. We see clearly, we say ‘I didn’t think that student would pass and yet he did’. And what I observe as a French teacher is that reading is always a problem. So, my way of assessing them during the school year must not be enough, it’s not good enough yet. (…). But there is a betterment since the beginning, when I started to give the CE1D (…) only 40% of my students passed. Now it’s 60 of them. (French Teacher, Rousseau II, September 2013: 198–212)

Disruption and identity claims

Despite the protection strategies leading to some persistent decoupling, we observed some disruptions in the schools’ cognitive scripts, linked to the certification of students based on the standardised tests and the compulsory nature of the tests. At the vocational school for instance, significantly more students are failing the tests than the internal examinations previously organised by the school, which questions the institutionalised definition of educational quality and students’ success:

The CE1D level is rather basic and yet most of my students, in mathematics for instance, fail their test which means that their level is lower than the minimum asked to the children (History–Geography Teacher, Rousseau II, December 2014: 4–6)

At Claparède, compulsory tests lead teachers to introduce quantitative assessment in their pedagogy, which is conflicting with the alternative institutional choices that they had internalised through specific training programmes:

You’re going to start with the tests again, ranking them from the best mark to the less good, making a list of prize winners at the end of the school year and awarding diplomas. That’s going to change the spirit of the school! (Principal, Claparède, September 2013: 1005–1007)

At Saint Ignace, where usually many students fail at the end of the first two years, students’ performance at the standardised test clearly reveals much better achievement than the one at the assessment usually organised by the school, which raises conflict with the school’s institutionalised norms of excellence and meritocracy:

What we’re afraid of (…) is that students will tell themselves: ‘There’s no need to work anymore. Everybody is accepted in third grade’. (Principal, Saint Ignace, September 2013: 142–144).

Out of the 250 students in the third year [after the second grade when the CE1D is scheduled] my coordinator’s going to meet some sixty parents. Sixty parents with a student who’s failed everything. It’s inevitable that we let everyone get by with the CE1D [held at the end of the previous school year]! (Principal, Saint Ignace, September 2013: 599–602)

Two reactions appear to come out of this disruption in the schools’ habits and norms. The first one is a feeling of distrust described by some teachers regarding their methods and criteria of evaluation, and beyond, regarding their pedagogy:

I spend five hours a week with my students, sometimes more because there’s also support outside of the classes. I’ve taught them certain things and so I know where they’re coming from, I know where I want to take them and an evaluation that doesn’t consider the students’ possible progress doesn’t make much sense. (French Teacher, Saint Ignace, June 2015: 518–522)

I might be the one teaching but as soon as policies take over the school policies and over the pedagogical projects of the schools… I teach for the child, so that the students can grow and if I am forced to do some things I believe not to be ad hoc for the students, I feel like it’s a lack of confidence in me. (French Teacher, Claparède, June 2015: 128–142)

This feeling of distrust is then doubled by some reclaiming of the locally embedded institutions that seem to offer sounder foundations than the views of pedagogy and assessment embedded into the instruments:

We have a pedagogy that seems to work well, and has been for years. We will not change it for that. (Principal, Claparède, September 2013: 462–463)

We should write a letter to the French-speaking community and say, ‘We don’t agree and we’re not going to go along with this kind of thing’. (French Teacher, Claparède, June 2015: 159–160)

I think it’s sad. I think we dutifully do our work and, if at the end of the school year we say: ‘This student doesn’t have what it takes to pass to the third grade’, he does not. Then… some authority comes in and tells us: ‘Yes he does’. While in fact he does not! (French Teacher, Saint Ignace, September 2013: 456–460)

Discussion

This article is a neo institutional analysis of the local reception of low-stakes ‘reflexive’ accountability instruments in the French-speaking Belgium school system, historically characterised by a quasi-market regulation (Draelants et al., 2011; Vandenberghe, 1998). Similar studies to ours dealt with high-stakes contexts and suggested a significant recoupling between institutional demands behind policy and local institutions framing daily practices and representations (e.g. Diamond, 2012; Spillane et al., 2011). Our study thus aimed to add to the literature by analysing the recoupling hypothesis in a context of soft and reflexive accountability where educational standards and tests have been implemented without any sanction/reward or ranking. To analyse the local implementation of reflexive instruments, we conducted three case studies selected theoretically for their a priori distance towards the worldviews embedded into the accountability instruments. Our guiding hypothesis was precisely that accountability instruments are supported by attempts to harmonise the cognitive landmarks in the educational systems, potentially creating tension with long-term institutionalised educational norms and practices within school contexts.

Our results show a pattern of institutionalisation and recoupling very much at distance to the one identified by the literature on high-stakes accountability that emphasises on the substantial recoupling between institutional demands legitimising accountability instruments, organisational infrastructures and teaching practices. First, our results show that teachers do not associate test scores much with the quality of their teaching practices, which is nonetheless a necessary condition for engaging teachers in a reflexive dynamic around the tests. Second, reflexive instruments are hardly used (Rousseau II and Saint Ignace) or not at all (Claparède), and when they are, it is in a fashion indicating a persistent mismatch between local worldviews and institutional demands spreading these instruments. Only some recoupling was observed in two schools, but in minority groups of teachers. The first, at Saint Ignace where a group of young teachers, feeling distant from the interpretations most prevalent among their fellow teachers, reported using these documents to conceive their lessons. The second, at Rousseau II (the school training young people for manual trades), for a very small number of permanent teachers in ‘classic’ disciplines such as French who also reported more intense use of the instruments to inspire their practices. Third, our results indicate that testing matters anyway despite the persistent decoupling, partly because against the will of teachers, tests have an impact on the success of students and their orientation. However, teachers perceive this imposition of standardised testing as a lack of confidence from authorities about their usual assessment practices, and find ways to reclaim their singularity, in particular towards parents.

How to understand such different patterns between high-stakes and low-stakes accountability contexts? In our view, a first important argument is linked to the deep meaning of those instruments in the institutional context of French-speaking Belgium. Historically this educational system has been organised on a liberal and communitarian basis that provided the foundations for the development of differentiated educational and pedagogic projects. In new institutionalist terms, this means that the institutional infrastructure of the system was not legitimised centrally by political authorities, but rather by the cognitive references imprinted by traditional communities composing the Belgian landscape. What we have observed during the last 30 years (since the regionalisation of education matters) however is some attempts to structure the guidelines for the system by political authorities in a continuous process of centralisation. Nevertheless, what appears clearly in our findings is that this attempt does not really concretise within schools. On the contrary, long-term institutional scripts still seem to prevail over new ones embedded into accountability instruments, and this is probably understandable regarding the nature of the institutionalisation of those instruments. We may argue indeed that the institutionalisation operates like in a vacuum, without grounding itself into the cognitive and normative pillars (Scott, 1995). Therefore, accountability instruments tend to appear to teachers to be like technical norms without the substance to legitimise the common norms they are trying to institutionalise, which might explain why local institutions tied to the historical regulation of the market maintain and conflict with the capacity of the educational system to institutionalise itself.

What comes out of it are new types of decoupling (Bromley and Powell, 2012). While traditionally the decoupling in an institutional field (i.e. education field) was vertical, it turns out to be horizontal. The horizontal decoupling means here a means–ends decoupling, where policy is implemented (in our case, the tests) but with an opaque relationship to ends and outcomes. This is particularly true at Claparède for instance, where the introduction of the instruments does not prevent some attitudes opposed to the standards behind the instruments. Based on our results, we suggest that this new kind of decoupling might be more prevalent in the low-stakes accountability context where constraints are put on the implementation of standardised testing, but not on the use of test scores for reflexive purposes (no sanction/reward or ranking). The point is that teachers implement the standardised testing that has some significant consequences for the students and the organisation of the work, but still rely on their proper local cognitive references to orient their work. In other words, the theory of change supposed by the reflexive accountability model does not seem to hold up, as teachers do not interpret the low constraints on the use of instruments as a mark of confidence from the political authorities. Therefore, they implement the mandatory part of the accountability instruments while avoiding a real penetration of the instruments into the cognitive scripts of their school.

From a methodological point of view, it is important to conclude by underlining the limits of our research design. To test our interpretations in the form of hypotheses, future studies would benefit from relying on a larger and more representative sample and from including schools less remote from the institutional requirement than the three selected here. In addition to these aspects, future studies about recoupling in the low-stakes accountability contexts would benefit from the inclusion of observation moments of the teaching practices in the classrooms.

Conclusion

This article focused on three low-stakes accountability instruments introduced in the French-speaking community of Belgium: educational standards, testing and so-called ‘reflexive instruments’ (i.e. didactical guidelines). Our objective was to understand the local implementation of these instruments. We looked in particular at the capacity of those instruments to recouple historically differentiated cognitive scripts of schools with the standards promoted by the political authorities. The study case of three atypical schools indicates a very low recoupling and to a certain extent, the reclaiming of long-term institutionalised scripts, which we interpreted regarding both the nature of institutionalisation of accountability instruments in the context of French-speaking Belgium, and the institutionalisation of new kinds of decoupling.

Footnotes

Acknowledgements

The authors would like to thank Sébastien Dellisse (Faculté de Psychologie et des Sciences de l’Éducation, Université Catholique de Louvain-la-Neuve, Belgium) for his participation in the data collection and Miriam Rosen for the translation.

Declaration of conflicting interest

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

ORCID iD

Samir Barbana

Author biographies

Samir Barbana holds a PhD in Education Sciences from Université Catholique de Louvain (GIRSEF). His reasearch interests are political tools in the French-speaking Belgium educational system. His work combines political reading of the instrument rationality with organizational approaches of their implementation for understanding if and how those tools participate to restructure the identity of schools and their functioning.

Xavier Dumay is professor of Educational sciences at the Catholic University of Louvain and senior researcher at the Interdisciplinary Research Group in Socialization, Education and Training (GIRSEF). His research interests lie in the educational policies, the globalisation and the fragmentation of education.

Vincent Dupriez is professor of Educational sciences at the Catholic University of Louvain and senior researcher at the Interdisciplinary Research Group in Socialization, Education and Training (GIRSEF). He develops research in the areas of education policy, educational inequalities, school administration and comparative education.

References

Ball

(1993) What is policy? Texts, trajectories and toolboxes. Discourse: Studies in The Cultural Politics of Education 13(2): 10–17.

Barley

(2008) Coalface institutionalism. In: Greenwood

Oliver

Suddaby

(eds) The SAGE Handbook of Organizational Institutionalism. London: SAGE, pp.491–518.

Bromley

Powell

(2012) From smoke and mirrors to walking the talk: Decoupling in the contemporary world. Academy of Management Annals 6(1): 483–530.

Carette

Dupriez

(2009) La lente émergence d’une politique scolaire en matière d’évaluation des élèves: Quinze ans de transformations en Belgique francophone [The slow emergence of a school policy in student assessment: Fifteen years of transformation in French-speaking Belgium]. Mesure et Evaluation en Education 32(3): 23–45 [in French].

Cattonar

Dumay

Mangez

, et al. (2010) Evaluations Externes dans L’Enseignement Primaire en Belgique Francophone. Réceptions et Usages d’Outils de Régulation Basés sur les Connaissances [External Evaluations in Primary Education in French-Speaking Belgium. Receptions and Uses of Knowledge-Based Regulation Tools]. Rapport de recherche. Louvain-la-Neuve: Presses universitaires [in French].

Danhier

(2018) How big is the handicap for disadvantaged pupils in segregated schooling systems? British Journal of Educational Studies 66(3): 341–364.

Dean

(2010) Governmentality: Power and Rule in Modern Society. 2nd edn, London: SAGE.

Diamond

(2012) Accountability policy, school Organization, and classroom practice: Partial recoupling and educational opportunity. Education and Urban Society 44(2): 151–182.

DiMaggio

Powell

(1983) The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review 48(2): 147–160.

10.

Draelants

Dupriez

Maroy

(2011) Le système scolaire en Communauté française [The school system in the French Community]. Dossiers du Crisp 59: 3–116 [in French]. Available at: https://www.researchgate.net/publication/44832967_Le_Systeme_scolaire_en_Communauté_francaise (accessed May 13, 2019).

11.

Dupriez

(2015) Peut-on Réformer l’Ecole? Approches Institutionnelle et Organisationnelle du Changement Pédagogique [Can We Reform the School? Institutional and Organizational Approaches to Educational Change]. Louvain-la-Neuve: De Boeck [in French].

12.

Dupriez

Maroy

(2003) Regulation in school systems: A theoretical analysis of the structural framework of the school system in French-speaking Belgium. Journal of Education Policy 18(4): 375–392.

13.

Hallet

(2010) The myth incarnate: Recoupling processes, turmoil, and inhabited institutions in an urban elementary school. American Sociological Review 75(1): 52–74.

14.

Hallett

Meanwell

(2016) Accountability as an inhabited institution: Contested meanings and the symbolic politics of reform. Symbolic Interaction 39(3): 374–396.

15.

Klieme

Avenarius

Blum

, et al. (2004) Le Développement de Standards Nationaux de Formation: Une Expertise. Rapport de Recherche [The Development of National Training Standards: An Expertise. Research Report]. Bonn: Ministère fédéral de l’éducation et de la recherche [in French]. Available at: https://edudoc.educa.ch/static/web/arbeiten/harmos/develop_standards_nat_form_f.pdf (accessed May 13, 2019).

16.

Lawrence

(2008) Power, institutions and organizations. In: Greenwood

Oliver

Sahlin

, et al (eds) The SAGE Handbook of Organizational Institutionalism. London: SAGE, pp.170–197.

17.

Meyer

Rowan

(1977) Institutional organizations: Formal structure as myth and ceremony. American Journal of Sociology 83(2): 340–363.

18.

Mons

(2009) Les Effets Théoriques et Réels de l’Evaluation Standardisée, Complément à l’Etude « Les Evaluations Standardisées des Elèves en Europe: Objectifs, Organisation et Utilisation des Résultats » [Theoretical and Real Effects of Standardised Assessment, Complement to the Study ‘Standardised Assessments of Pupils in Europe: Objectives, Organization and Use of Results’]. Brussels: Eurydice [in French]. Available at: http://www.aede-france.org/Evaluations-standardisee.html (accessed May 13, 2019).

19.

Mons

Dupriez

(2011) Les politiques d’accountability. Du changement institutionnel aux transformations locales [Accountability policies. From institutional change to local transformations]. Education Comparée 5(1): 7–16 [in French].

20.

Rowan

(2006) The new institutionalism and the study of educational organizations: Changing ideas for changing times. In: Meyer

H-D

Rowan

(eds) The New Institutionalism in Education. Albany: State University of New York Press, pp.15–32.

21.

Scott

(1995) Institutions and Organizations: Ideas, Interests and Identities. London: SAGE.

22.

Simons

(2007) Educational policy in a European context: Exploring processes of governmentalisation in Flanders. Tertium Comparationis: Journal for Comparative and Multicultural Education 13(2): 217–233.

23.

Spillane

Parise

Sherer

(2011) Organizational routines as coupling mechanisms: Policy, school administrators and the technical core. American Educational Research Journal 48(3): 586–619.

24.

Spillane

Reiser

Reimer

(2002) Policy implementation and cognition: Reframing and refocusing implementation research. Review of Educational Research 72(3): 387–431.

25.

Suchman

(1995) Managing legitimacy: Strategic and institutional approaches. Academy of Management Review 20(3): 571–610.

26.

Thélot

(2002) Evaluer l’école [Evaluate the school]. Études 397(10): 323–334 [in French].

27.

Vandenberghe

(1998) L’enseignement en communauté française de Belgique: un quasi-marché [Teaching in the French community of Belgium: a quasi-market]. Reflets et Perspectives de la Vie Économiques 37(1): 65–75 [in French].

Local implementation of accountability instruments in the French-speaking community of Belgium

Abstract

Keywords

Introduction

The Belgian French-speaking case: Testing and ‘reflexive’ accountability as instruments of progressive centralisation

Theoretical framework

Method: Three case studies

Results

Looking inside the schools’ cognitive scripts

Towards recoupling?

Disruption and identity claims

Discussion

Conclusion

Footnotes

Acknowledgements

Declaration of conflicting interest

Funding

Notes

ORCID iD

Author biographies

References