School actors’ enactment of a performative accountability scheme in Russia: Tensions,dilemmas and strategies

Abstract

In European and global educational debates, performative or test-based accountability has become central to modernizing and raising the performance of education systems. However, despite the global popularity of performative accountability modalities, existing research finds contradictory evidence on its effects, which tend to be highly context-sensitive. With the aim of gaining a deeper understanding of the mechanisms and contextual factors that explain the effects of performative accountability, this study investigates the enactment of a performative accountability scheme adopted in the Russian school system. The analysis is based on interview and observation data collected during an in-depth qualitative study of two neighbouring schools with contrasting logics of action. Our findings illuminate the specific ways in which accountability policy outcomes are mediated and shaped by schools’ context and agency. We show how schools with different logics of action react to external pressures, and how different professional groups within schools experience policy pressures in dissimilar ways. We conclude that performative accountability mechanisms reinforce instrumental, and impede expressive, logics of action in schools. In both cases they produce tensions, particularly for schools in disadvantaged areas.

Introduction

In recent decades, concerns about the quality and equity of education systems have put significant pressure on policy-makers around the world to reform education. In European and global educational debates, performative or test-based accountability has become central to modernizing and raising the performance of education systems. Promoted by supranational organizations and networks, including the European Union (EU) and the Organisation for Economic Cooperation and Development (OECD), performative accountability reforms are expected to improve efficiency, academic excellence and equity by holding schools and school actors accountable for the achievement of externally defined standards, often measured by means of standardized achievement tests (Ranson, 2003; Verger and Parcerisa, 2017a). The growing tendency to hold key educational stakeholders, in particular teachers, accountable for learning standards is based on a widely shared belief among policy-makers that teachers are the number one factor contributing to student achievement (e.g. Darling-Hammond, 2000; Hattie, 2003). Moreover, by attaching consequences of some kind to performance measures, motivation and feelings of responsibility are expected to increase, with more effective teaching as a result (Rosenkvist, 2010).

Despite the global popularity of performative accountability modalities (Lingard et al., 2017; Verger et al., 2019), the expectations of policy-makers are not always met (Falabella, 2014). Studies find inconclusive and contradictory evidence of the enactment and effects of performative accountability (Maroy, 2015; Sahlberg, 2016; Verger and Parcerisa, 2017a, 2017b). Moreover, an increasing number of studies shows that performative accountability modalities can generate responses such as curriculum narrowing (Berliner, 2011; Jones and Egley, 2004), teaching to the test, and teacher-centred pedagogy (Au, 2007), as well as opportunistic behaviour such as cream-skimming (Jennings, 2010), educational triage (Booher-Jennings, 2005), cheating (Jacob and Levitt, 2003) and other gaming strategies (Berliner, 2011), harming education quality and inclusion (Allan and Artiles, 2016; Au, 2007; Falabella, 2014; Lipman, 2004; Nicolas and Berliner, 2007; Thiel et al., 2017).

According to Lingard et al. (2017: 2), accountability has to be understood ‘from the perspective of its definition and practice in particular circumstances, rather than according to a generic essence that can be subject to a moral evaluation’. How particular performative accountability modalities play out is context-sensitive, depending in part on the underlying ideas and regulatory setting of the accountability contingency (Holloway et al., 2017), as well as on the ways in which schools ‘do policy’ (Ball et al., 2012) within their particular local and school contexts (Ball and Maroy, 2009; Braun et al., 2011). Teachers and other key stakeholders are not mere objects of performative accountability reforms, but rather play a key role in putting policy into practice, enabled and constrained by their specific contexts, thereby mediating and shaping policy outcomes (Ball et al., 2012).

Thus, in order to understand why performative accountability reforms have certain effects in certain contexts, it is necessary first to unpack the policy modality and, second, to open up the black box of how schools ‘do policy’, taking context ‘seriously’ (Braun et al., 2011). With the aim of developing a more realistic and holistic understanding of how schools put policy into practice, Ball et al. (2012) developed the theory of policy enactment. Criticizing studies that perceive policy implementation as a linear and mechanical process, the authors argue that policy enactment can be understood as a negotiated process, whereby school actors make strategic choices about what they can and will put in practice, and what they will not, a process mediated by the local context and school-specific factors (Ball et al., 2012; Braun et al., 2011). The complexity of institutional policy enactment often goes unrecognized within school improvement policies, which rely upon generic practices, failing to take account of how contextual factors may enable or constrain enactment processes (Lupton, 2004). In a similar vein, within much research on the enactment of accountability policies, the material, institutional and social contexts of schools remain unexplored (Holloway et al., 2017; Verger and Parcerisa, 2017a, 2017b), thereby failing to provide a more holistic explanation of why particular policy outcomes may occur under particular circumstances.

In this paper, we aim to contribute to the understanding of the varied and complex ways in which teachers and other school actors interpret, make sense of and perform within particular performative accountability schemes, and how this process is mediated by contextual factors. To do so, we build upon a case study: the enactment of a performative accountability scheme adopted in the Russian school system. In Russia, large-scale standardized assessments of student achievement were launched nationwide in 2009 and were followed by the introduction of school rankings and performance-based payment for teachers linked to students’ examination scores. Teachers and school administrators are faced with new policy demands, which include a stronger emphasis on raising students’ test and examination scores, and the use of student assessment data to improve teaching practices and school management. While accountability reform at a national level in Russia has been discussed in Russian and international publications (e.g. Bolotov, 2018; Minina, 2016; Piattoeva, 2015), an analysis of this reform enactment in schools is still lacking. Our data, which consist of observation notes and interviews, were collected in 2015, during an in-depth qualitative study of two schools located in the ‘working-class’ part of a major city in Russia. The material was analysed through a qualitative content analysis procedure, building upon the theory of policy enactment developed by Ball et al. (2012). The framework of school-specific factors identified by Braun et al. (2011) and the typology of schools’ logics of action (Ball and Maroy, 2009) are used as analytical tools to examine how the process of policy enactment interacts with the local context and school-specific factors.

The paper is structured as follows. First, a brief review of the theoretical perspectives and analytical tools is provided. Then, we explain the Russian case study in more depth. This contextual elaboration is followed by a description of the data and methods and a presentation of the results. The article ends with a discussion and conclusion in which we elucidate the ways in which the enactment of a performative accountability scheme can be mediated by school context and logics of action, and how at the same time it has the potential to change schools’ dominant order. We argue that performative accountability mechanisms reinforce instrumental, and impede expressive, logics of action in schools, producing tensions in both cases. At the same time, we hypothesize that different rationales can lead to certain similar behaviour patterns in response to accountability pressures.

Theoretical perspectives and analytical concepts

Ball et al. (2012) developed the theory of policy enactment in an attempt to gain a better understanding of how schools ‘do policy’. The theory emerged as a critique of studies that perceived policy implementation as a linear and mechanical process, assuming a straightforward and direct translation of a policy text into practice. According to Ball et al. (2012), policy enactment is a dynamic, non-linear, negotiated and relational process, whereby a range of different policy ‘enactors’ make sense of and translate abstract policy texts in complex and creative ways, enabled and constrained by local and school-specific contextual factors. With the aim of gaining a better understanding of how policy is decoded and re-coded in schools, Ball et al. (2012, 43) developed a heuristic distinction between policy interpretation and translation. Whereas interpretation concerns the process of constructing meaning and understanding of a policy text and making sense of policy demands, translation relates more closely to the language of practice and the strategic choices made by actors regarding whether and how to put a new policy idea into practice (Ball et al., 2012).

The ways in which school actors make sense of and put policy ideas into practice depends in part on the regulatory nature of the policy, as well as the local and school context (Ball and Maroy, 2009; Braun et al., 2011). Performative accountability reforms differ considerably with regard to who is expected to give the account, the forum to which the account is to be given, and the consequences attached to the account (Au, 2007; Bovens, 2007; Leithwood and Earl, 2000). These factors are likely to condition policy enactment and policy outcomes (Verger and Parcerisa, 2017a). In addition to the nature of the policy, the complex and multiple ways in which teachers, school administrators and other school actors interpret and translate abstract policy ideas shape policy outcomes. Braun et al. (2011) identify four types of school-specific factors that mediate policy enactment processes: (a) situated, (b) material, (c) professional, and (d) external. In addition to the framework of school-specific factors, we use the typology of schools’ logics of action, developed by Ball and Maroy (2009), as a tool to analyse how the process of policy enactment is mediated by, and interacts with, schools’ dominant orders. These orders, or logics of action, have been defined as ‘consistencies deriving, ex post facto, from the observation of practices and decisions in the school in relation to diverse aspects of their functions’ (Ball and Maroy, 2009: 99). The authors distinguish between instrumental and expressive logics of action, two ‘ideal-types’ that serve as heuristic tools.

Expressive schools are ‘markedly socially heterogeneous and this is celebrated’ (Ball and Maroy, 2009: 102). Tolerance, inclusion and social heterogeneity are core values, and promoting equity is a key objective. Expressive schools often have an ‘open intake’ and tend to accept students who have been rejected or excluded by other schools. A common curriculum is applied and ability grouping or other differentiation practices are minimized. Teacher–pupil relations are ‘interpersonal or articulated in terms of “caring” and pastorality’ (Ball and Maroy, 2009: 102). There is regular collaboration between staff, while principals tend to act as ‘leading professionals’. In contrast, instrumental schools value homogeneity and are more likely to have a selective intake of students. Performance and academic excellence are core values and central features of the school’s reputation, while a discourse on equity is largely absent. Differentiation practices such as ability grouping are much more often applied. Teacher–pupil relationships, as well as staff relationships, are closed, formal and ‘based on authority and positional control’ (Ball and Maroy, 2009: 102). Collaboration is limited, while teacher autonomy is emphasized. Principals tend to act as ‘managers’ and sole decision-makers (Ball and Maroy, 2009). While every school has both instrumental and expressive orders, one tends to dominate over the other, with the exception of ‘hybrid schools’, which are ‘either “in balance”, in a stable condition of blended logics, or “in movement”, in the process of shifting between logics’ (Ball and Maroy, 2009: 104). The dominant school’s order is likely to shape the policy enactment of performative accountability, while the latter is at the same time shaped by enactment processes.

Based on a review of the literature on the enactment of accountability reforms, Verger and Parcerisa (2017a) show that teachers in various contexts employ rather critical attitudes towards performative accountability (e.g. Jones and Egley, 2004; Mintrop, 2003; Müller and Hernández, 2010), in some cases more so than school administrators (e.g. Assaél et al., 2014). Negative perceptions with regard to the meaningfulness and impact of policy measures are likely to shape the ways in which teachers respond to new demands. In this light, Verger and Parcerisa (2017a) report on studies portraying a variety of responses, moving beyond resistance and consent, and including negotiation, evasion and creative transformation (Hardy, 2014; Kostogriz and Doecke, 2011; Palmer and Rangel, 2011; Robinson, 2012). To understand the type of responses that teachers employ, Verger and Parcerisa (2017a) highlight the mediating role of principals and their leadership style. While the key role played by educational stakeholders in policy enactment of accountability reforms has received significant attention, the mediating role of school-specific factors and the local context remains little explored (Verger and Parcerisa, 2017a). This despite the fact the few studies taking context into account have shown that contextual factors mediate policy enactment of accountability reforms and thereby shape policy outcomes (e.g. Diamond and Spillane, 2004; Douglas, 2005; Falabella, 2014; Keddie, 2014; Mintrop, 2003).

Russian accountability reform

Since the 1990s, Russia has been paying increasing attention to new means of quality assurance in education. In the Soviet period, education policies did not include any national assessments of student achievement, let alone performance-based schemes for school financing (Bakker, 1999, 2012; Webber, 2000). It was assumed that the centralized curriculum and teacher training were sufficient to ensure the quality of education (West and Crighton, 1999). School graduation tests were organized by schools, and the main measures of school performance were students’ school grades, the percentages of students successfully transitioning to the next school year and stage, and the numbers of graduates who received special achievement prizes (see e.g. Bakker, 1999). The evaluation of teacher quality based on their students’ progress in learning was prohibited (Kukulin et al., 2015).

Within a decade of the dismantling of the Soviet system, diversity and inequality in Russian schools peaked as a result of the abrupt transition to a market economy, severe underfunding of schools, and liberalization of the school curriculum and teaching methods (Polyzoi and Dneprov, 2011). In the 2000s, the Russian Ministry of Education and Science issued several strategic and legislative documents which called for efficiency, accountability and transparency in education. As a key instrument to ensure academic excellence and equality of opportunity across the diverse country, the all-Russian Unified State Exam (USE, or GIA-11) was introduced on an experimental basis in 2001 and launched nationwide in 2009. The examination combined the functions of school graduation test, national university entrance test, and source of information on educational achievement for evidence based policy-making.

In 2015, when our empirical material was collected, students’ scores in national examinations, the GIA, conducted after grades 9 and 11, were used as an important measure of teacher and school performance. The System of Evaluation of Education Quality in the case region of our study featured 80 numerical indicators for desired outcomes in education. The ‘quality of educational results’ of schools was measured through students’ grade point averages; average scores of students in GIA; number of failures in GIA; number of prizes won in subject Olympiads; and educational contests. These indicators served as the criteria in school rankings, performance-based salary schemes for teachers and principals, and for the promotion of teachers to higher professional categories. If a school showed a performance significantly below the regional average, it could be inspected, with potential consequences such as lay-offs of members of the school administration, although such severe measures were rare (Gurova et al., 2018). However, even in the case of poor student performance, schools would not be closed as long as they complied with state regulations and had a sufficient number of students relative to their facilities. The main part of school funding was distributed per capita – that is, based on the number of students – and a teacher’s basic salary was calculated by the number of his/her contact hours.

The reform set new demands and expectations for teachers and schools. By holding them accountable for student test scores through performance-based pay, grant funding, and procedures for awarding qualifications, the reform established the mechanisms of an external accountability system in which the stakes were high, with student scores as an ‘objective’ measure of teacher quality. Moreover, schools became accountable for student results not only to the state but also to the public. This was assured by such means as rankings and school choice, coupled with per capita funding. As well as for teachers and schools, the stakes are high in the standardized test for students; their educational prospects depend on test scores. The reform aimed to increase teachers’ responsibility for student performance, through the requirements to include student performance data in school strategic plans and in teachers’ reports. Schools workers also received training in how to analyse test results and employ such instruments as ‘diagnostic tests’ in order to improve students’ learning.

Data and methods

With the aim of examining the enactment of the Russian performative accountability scheme, this paper adopts a case study, an in-depth qualitative study of two schools located in Cheboksary – the main city of a mid-size, middle-income region in the European part of Russia. Both the schools observed are public and situated in the same relatively disadvantaged city district. ¹ The first school, school A (approx. 1100 students), was identified by local authorities as a ‘best practice’ school in the region, in terms of how it implemented the performative accountability reform. It was ranked high in the regional school rankings, participated in international studies and enjoyed the position of a ‘magnet’ school in the neighbourhood. At the same time, despite this advantageous position, the school still struggled to retain its best-performing students, many of whom transferred in the upper grades to elite schools ² of the same city or to schools in Moscow that offered more specialized education. The second school, school B (approx. 700 students), was selected after one month of observations in school A; the object was to deepen understanding of accountability policy effects by selecting a contrasting case. It appeared that school A’s strategy was focused on high achievement and demonstrating outstanding results. Hence, to capture the diversity of policy enactment, school B was selected because it had a different orientation, as shown in the school’s public strategic documents: to create a supportive atmosphere and retain students. In the regional school rankings, it occupied a place below the average.

Building on the heuristic device developed by Braun et al. (2011), we examined the school-specific contexts of both schools. More specifically, based on semi-structured interviews and informal conversations with 11 school administrators and 19 teachers in both schools, as well as school observations, we explored the situated contexts of schools A and B, including their location, history, reputation and student composition. In addition, we examined their professional contexts, focusing on each school’s professional culture, as well as values and commitments of its staff. With regard to the schools’ material contexts, we looked at their buildings, budget, staffing and technological facilities. Finally, we analysed the external context of both schools, examining the role of the local authority in providing support as well as putting pressure on schools, in addition to local legal requirements and responsibilities, and pressure exerted by local league tables. A deeper understanding of the schools’ contexts enabled us to explore how contextual conditions interacted with school organizational dynamics, educational practices, and policy enactment.

To examine how Russian school administrators and teachers interpreted and translated the performative accountability scheme, we used notes from observations of lessons, examinations, teacher meetings, teacher–parent conferences and administrators’ offices; as well as 25 semi-structured interviews and numerous informal conversations with administrators, teachers, students and parents in both schools. Observation periods were selected to include all the main procedures that determine schools’ performance, as identified from policy documents and schools’ work plans: state examinations after grades 9 and 11, subject Olympiads and contests, internal school examinations and assessments (e.g. end-of-quarter or end-of-year tests).

The collected material was analysed through a qualitative content analysis procedure, focusing on: (a) the school’s specific contextual conditions (what interviewees said about the school, students, staff and relationships, and what material and social characteristics were observed); (b) teachers’ and administrators’ interpretations of the accountability policy (student performance as a measure of teacher and school performance, performance-based payment schemes, using data from student assessments to improve teaching or school management); and (c) teachers’ and administrators’ practices with regard to translating policy demands into materials, routines, procedures and orientations. Recurring themes in the analysed data were grouped into categories and sub-categories. Of the different characteristics of school context, recurring themes related to three categories: characteristics of student body (including families of students); principal’s leadership style and organizational culture; and school’s resources and infrastructure.

By means of this small-scale in-depth study we intend to bring the complexity and heterogeneity of the process of policy enactment of accountability reforms to the fore, and to identify rationales underlying school actors’ behaviour in response to a performative accountability scheme. Moreover, we aim to examine how contextual factors interact with policy enactment. The presentation of the findings is structured as follows: first, the school-specific contexts and logics of action of both schools are described; then, we proceed to the analysis of how teachers and administrators in both schools interpret and translate performative accountability policy demands.

Analysis of school actors’ policy enactment

Different logics of action within the same external context

The geographical proximity of the two schools observed is the reason for many similarities in their conditions. The schools operate within the same legal framework, and report to the same local educational authorities. In other words, both schools have a common ‘external context’ (Braun et al., 2011). As public schools, they have buildings with a typical Russian public school architecture. Being located in a less advantaged part of the city, both schools enrol students from disadvantaged families. Their teachers are mostly of local origin, having received teacher training in the same regional pedagogical university. However, the data collection revealed that there are more differences in the schools’ immediate local contexts than could be assumed based on the schools’ public information. The schools differed in regard to their local areas (Lupton, 2004) as well as their school-specific contexts (Braun et al., 2011).

Below, we provide detailed pictures of the two schools, touching on the schools’ material contexts, their history and reputation, but concentrating on their professional contexts and characteristics of the student body. These factors were particularly emphasized by our respondents when explaining the specifics of their schools. In turn, these descriptions also make explicit the different logics of action adopted by the two schools, which are likely to relate to small differences in the schools’ local area, as well as to the strategic choices and value orientations of both school administrations.

School A: ‘Let each of us take responsibility!’

School A has a large building with spacious classrooms and corridors. The main entrance leads to a bright foyer with colourful stands describing school achievements and projects. The building looks recently renovated and well-kept. Staff meetings are hosted in a large assembly hall on a weekly basis, during which the administration informs teachers of plans, announcements and concerns, praises the best staff and reprimands those who did not fulfil their tasks.

Hard work and strict discipline are expected from all staff members. During school holidays, few teachers and no administrators are allowed to take leave. Working on Saturdays is the norm. The principal hesitates to award free Saturdays to teachers, notwithstanding their good performance: ‘If I give him [free Saturdays], everyone will start to ask [for that].’ Besides regular teaching activities, projects and events organized by teachers ³ are so numerous that they complain about not being able to follow ‘what is going on in the school’. During examination periods, the atmosphere is enormously stressful: an administrator takes sedatives before meeting the principal to report on students’ failures.

Discipline and control also prevail in the classes. Teachers demand concentration and diligence from all students. Those who consistently fail to perform are ‘cleaned out’ (the term used by city’s teachers for the informal practice of getting rid of low-performing students, primarily through persuading parents to transfer their child to a different school – officially, the practice is prohibited). There are plenty of supplemental paid-for study courses that students can take after regular classes. Some classes are devoted to exam preparation, and these ‘are not really voluntary’ (S-26); ⁴ all students are expected to participate and the fee is affordable. The most cherished quality in students is their ‘motivation to study’: teachers are ready to offer a lot of support, including extra (free) lessons, to those who are low-performing but are considered ‘motivated’. However, only students who passed GIA-9 with good results can continue to grades 10 and 11.

Attracting and retaining high-performing students is a primary concern of administrators and teachers. Through a paid-for school preparation programme, in which twice the number of students is enrolled as the school can take into grade 1, the school attracts and informally selects able and motivated pre-schoolers. This is possible because the school has a reputation as a ‘very good school’ and even parents from outside the catchment area seek to get their children enrolled in it. According to our respondents, before the current principal was appointed three decades ago, the school did not stand out among other schools in the area; it was the principal’s management that made it high-achieving, well known in the city, and a ‘magnet school’ in its neighbourhood. However, in the upper grades, despite all efforts to retain them (improving the school’s reputation, persuading parents and head-hunting good teachers), the best-performing students still leave for elite institutions that demonstrate even better results in national examinations and subject Olympiads. A teacher explains to parents during a class meeting:

In grade 9 class performance has dropped. Six people left – kids who set the standard for others. When they left, the knowledge level of the class fell. […]. Not everyone understands that they will need to take exams. There is a concentration deficit. We move to grade 10 only those who are motivated, who proved themselves in some way. ⁵ Not each and every one [is admitted]. And even from grade 10 we have sometimes excluded [students]. (S-22)

Many parents take an active part in school activities, such as organizing school events or decorating the school for New Year celebrations. At parent meetings, most parents are present. They often ask for more paid-for classes for their children, and in grades 9 and 11 they support paid-for ‘mock examinations’. When teachers talk to each other about low-performing students, they explain their ‘lack of motivation’ as primarily due to lack of parental support and control. At the same time, teachers do not use this argument during administrative meetings, and the usual reaction to a reported problem is: ‘Let each of us take responsibility!’ (rather than referring to someone else’s fault). An administrator described the school, comparing it to her previous employment:

I am very satisfied with how the work in [this] school is organized. If a student does not work, they do not wait for this [problem] to ‘solve itself’, but actively start [to intervene]. […] [In my previous school] they worked from inspection to inspection, here – all the time. [Here it is] more intensive work, you need to constantly develop. (S-7)

School B: ‘They know their students like nobody else does’

The entrance to school B looks very different from school A’s. The building is smaller, there is no large foyer, and a visitor is met by a caretaker who asks the purpose of the visit. In the morning, the principal stands by the entrance, so that students know their attendance is being monitored. At an all-parents meeting, a representative of the city police explains that the entrance should indeed be controlled so that students do not skip classes or smoke outside during breaks. Teachers say they feel much safer with restricted access to the school; there were instances in the past of drunk and violent parents bursting into classes. The building looks old, classes are rather small, and all administrators, apart from the principal, share offices.

When describing the school, staff members always start with characteristics of the student body. The neighbourhood is considered dysfunctional. Teachers often refer to the poor health of low-performing students, resulting from parental neglect and substance abuse. They stay with such students after classes to offer extra lessons in small groups or individually; there are no paid-for courses, but in grade 9 teachers give additional exam-preparation lessons. Previously, the school taught children from the local orphanage. Teachers and administrators mention this to characterize the school as having historically dealt with a challenging student body. Admission to the school is not competitive, and most students are from its catchment area, where it is considered ‘an ordinary school, not a bad one’. It is also recognized for its students’ achievements in sports and active physical education instructors. An interviewee from the city authorities described the school as ‘doing its job well’ and emphasized that it is not among the weakest or most problematic schools they deal with. At the same time, an administrator complained that the challenging conditions and good work of her school were not recognized in the city, and the school was sometimes portrayed as ‘ineffective’ in the local newspapers. Comparing her school with school A, an administrator comments:

We know that they [in school A] clean out (get rid of low performers). They don’t keep weak students. And we take everyone, all those who were excluded by other schools. Well, they need to study somewhere, after all. (S-12)

The atmosphere both at staff meetings and in classes is friendly and relaxed. Meetings that take place in a narrow staff room are sometimes followed by an informal celebration of someone’s birthday; the staff room is also visited by pedlars from whom many teachers buy goods. In class, teachers often encourage students to ‘help each other’, to ‘work together’; they let students consult textbooks when answering and give them hints during internal assessments. Even at examination time the overall mood is peaceful: as teachers explain, ‘if we scare them, they simply won’t come’. Teachers also complain about parents’ attendance: while each class has from 25 to 30 students, fewer than 10 parents normally show up at class meetings. Rude behaviour of some students, as well as students’ learning problems, are the main sources of teachers’ stress. A teacher describes the situation:

The collective of this school is good, a good atmosphere. As for students … This is such an area [of the city] … I wanted to quit [already] on the second day. […] In three years of working in this school I have seen so many teachers who come – and leave at once, they do not want to work with this student body. (S-33)

Most respondents in our study, however, have worked in this school for many years. Only seven teachers joined school B in the last 20 years. One of these ‘newcomers’ notes:

Many teachers have been here since this school started, for 50 years. This school is dear to them. Many studied here. They know their students like nobody else does. If a girl comes to class in the morning, and her parents caroused all night, her homework is, of course, not done, she has not slept well – but still she woke up and came to school. Teachers know these kids, and they work with them specifically, [take] an individual approach. But they will never have [high] results. (S-31)

Administrators in school B stress the importance of ‘taking care of teachers’: ‘Otherwise, who is going to work?’ (S-12). They explain that the administration never presses teachers to organize any extra activities or take part in city contests and projects. The school also does not apply for any grants: ‘We tried [it] once, we prepared for a contest, applied for a grant … A lot of work, but the result was zero, we won nothing’ (S-12). In school holidays very few people are present at school; and during the school year the principal sometimes takes a week off. On Saturdays, only those teachers attend who are giving lessons. Administrators often have to remind teachers to prepare a certain report, and many activities required by local education authorities are carried out only as a formality. The principal often underscores that ‘the school works as it should work under the law’. An interviewee who used to work in school B as a teacher and administrator, but had recently transferred to teach in school A, compared the two schools:

Kids [in school A] are much more motivated to study. [This school has] much more opportunities to prepare kids, to do extra work with them. In school B teachers only do it at their own expense, but [in school A] there are very many extra [paid-for] classes. […]. [The] administration in both schools tries to direct kids to study, but here (in school A) opportunities are much better. (S-29)

Case schools as demonstrating different logics of action

The schools observed face similar challenges because of the common external context as well as their location within a less advantaged part of the city (Lupton, 2004). However, differences in their local areas became apparent during the observation periods. The schools also varied in the way they dealt with these challenges.

Schools are not allowed to share information on their students’ socio-economic status, but data collected in the two schools revealed that their student composition differed. Under Russian law, parents can apply to any school of their choice, but families living in the school’s catchment area have priority in the admission process. For this reason, both schools had students from disadvantaged families, but in school B a greater percentage of students appeared to have a poor socio-economic background. This can be attributed to the schools’ differing admission practices and reputations as well as to the differences between the local areas. With regard to the former, school A practises selectivity in student intake, and during the years of schooling it ‘cleans out’ students that consistently demonstrate low motivation for study, makes efforts to retain high performers, and sets barriers so that only students who perform well can move to the upper grades. School B, on the other hand, has care and acceptance of differences as major working principles, and the student intake is non-selective. With regard to the schools’ local areas, the immediate neighbourhood of school B appeared to be relatively rural; the inhabitants have lower incomes and are less educated. Student composition and parental engagement differ between the schools and cause stress in both, but for different reasons. In school B, stress is mainly related to students’ behavioural problems and lack of parental attention to their education, while school A’s staff are pressured by some parents who are not only more active and willing to help improve performance, but also more critical and demanding.

Furthermore, the schools differ in their professional values and commitments, and leadership styles. In school A, the administration places emphasis on achievement and diligence. Relationships between staff members as well as between teachers and students can be characterized as hierarchical, authority- and discipline-oriented. In contrast, in school B, relationships between staff members are based on cooperation, and administrators see their role as one of ‘taking care’ of teachers. Personal and informal approaches, cooperation and support also characterize teacher–student relationships and classroom practices.

As such, it can be argued that both schools have adopted different logics of action, with school A demonstrating characteristics of the instrumental logic of action, and school B of the expressive order (Ball and Maroy, 2009).

At the same time, despite the different logics of action that the two schools have developed, similarities in their experiences could be observed. Staff members of both schools refer to the challenges of working with parents of low-performing students. In both schools, teachers collaborate, though for different reasons and with different aims. In school A, collaboration is a way to achieve higher results by combining resources and efforts. Teachers work together in a variety of school projects and extracurricular activities. School B teachers cooperate informally to offer support and show empathy to a stressed colleague. As we show below, school-specific contexts and logics of action interact with how school actors make sense of and translate accountability pressures and demands.

School actors’ enactment of performative accountability

Interpreting the policy as a shift in responsibility

Despite the differences between the two schools outlined above, teachers’ interpretations of the performative accountability policy were fairly similar. The underlying principle of the reform, as perceived by teachers in both schools, is a shift of responsibility for students’ learning to the teacher. Teachers strongly disagreed with this. In their view, student grades reflect the abilities and motivation of students, and the support of their parents, not merely the teachers’ qualifications and effort. Hence, the shift of responsibility for students’ scores to the teacher can be damaging to students’ motivation:

Responsibility for the quality of education should be shared by teacher, student, and parent. And now, everything is shifted to the teacher, kids don’t feel responsible for their studies. The state raises freeloaders who expect that others will solve their problems and provide everything to them. (S-19, school A teacher)

[There should be] three parties [investing in education]: parent, student, teacher. If there are at least two, there will be success, development. But often there is only the teacher. (S-31, school B teacher)

Being critical of the shift of responsibility for students’ results to teachers, teachers in both schools considered performance-based salary schemes and rankings to be inadequate and unfair. In addition to the influence of student and family characteristics, they pointed to the many aspects of teachers’ work that such schemes disregarded. For example, although teachers work collaboratively, stand in for each other and are assigned to teach different classes in different years, their performance scores are calculated individually. ⁶ Moreover, a teacher’s contribution in one subject can help improve students’ performance in other teachers’ subjects, and no account is taken of this. Nor is account taken of the contribution of private tutors, with whom many students take private lessons to prepare for exams. Finally, it is more difficult to prepare all students for compulsory examinations (as in the case of GIA in mathematics and Russian language) than to prepare a few for examinations in the subject of their choice.

Those who teach compulsory GIA subjects – the school gives them an incentive (adds additional points to the performance-based calculation of salary). But even with these [additional points], teachers of mathematics and Russian language have fewer points than anyone else. Plenty of contact hours, and no points. The primary school [teachers] have a lot [of performance points]. Physical education teachers – more than anyone! Because [in their subjects there are] many contests, and no compulsory examinations. (S-34, school B teacher)

Students from [a better performing school] take GIA in their school, but take private lessons from me, the teacher of [school B]. It is we who prepare them. (S-12)

In both schools, administrators share the teachers’ view that student results greatly depend on students’ abilities and effort, as well as family characteristics, and recognize that measuring teachers’ performance by student scores can be problematic. Administrators in both schools also assert that school rankings should take into account the school’s context, not only student results. In school B, administrators express even more dissatisfaction with the accountability policy than teachers. They see it as provoking conflicts and not doing justice to teachers’ qualifications and the amount of work they do. In contrast, administrators in school A, despite their critical attitude towards the school performance criteria used in rankings, seem to appreciate criteria-based assessment of teachers as a management tool.

Putting policy into practice: tensions, dilemmas and strategies

Despite their critical attitudes towards (some aspects of) the policy, school actors in both schools are ready to take on the responsibility of preparing students for examinations, for three main reasons. First, examinations determine students’ opportunities to continue education to the next level, and schools see assisting students in this task as an important professional and moral obligation. Second, school staff see themselves as public servants, and they recognize national examinations as the government standard according to which schools should teach. Third, especially in school A, teachers and administrators also mentioned the school’s prestige and their own professional reputation as factors that motivate them to improve and demonstrate good performance. As a result of this reasoning, both schools make efforts to prepare students for examinations, offering extra support to some of them, organizing ‘mock examinations’ to hone their exam-taking skills, and changing teaching practices based on analyses of test results. School B workers complain that their students’ families are not eager to pay for the ‘diagnostic tests’ provided by a local company, and administrators have to borrow test materials from other schools.

At the same time, the performance-based schemes attached to the policy create professional and moral conflicts for teachers, and damage cooperation between staff members. One school A teacher complained that she was forced to give students better grades, because otherwise she would be reprimanded by the municipal educational authority, ‘and the principal does not protect us from the municipality because her salary depends on [criteria applied by this authority]’ (S-19). Another teacher commented: ‘We work for the ranking, not for education’ (S-22, school A teacher). Depending on student results, teachers face a moral dilemma: to encourage an able student to transfer to another school that can offer better opportunities for learning, or to try to retain the student in order to benefit from his/her high scores: ‘My students transferred in grade 9 to the [elite school], they will take the GIA in [my subject] there. I myself advised them to do so. And I will now have low [performance] indicators’ (S-19, school A teacher). Another contradiction is between different measures of performance: for example, students who often participate in contests have less time to prepare for GIA and sometimes pass GIA with lower scores. Teachers at both schools complain that accountability measures impede teaching rather than improve it, taking time away from the ‘actual teaching’.

While teachers claim that their actions are first and foremost in the interest of the student, there are also some indications of teachers’ gaming strategies to improve performance scores and receive a better salary. For example, a school A teacher describes how she prefers to participate in paid-for online contests rather than open municipal ones: ‘In the online contests you can be sure that the child will do well, as you can sit next to him/her’ (S-23) – a strategy also applied in school B. Another strategy is to discourage students from choosing examinations in a subject in which they generally do not perform well:

I wouldn’t wish these [students] to choose [my subject to take the GIA exam in], since they are not likely to pass it. If there were better students, then this would of course be nice, extra points for the performance-based part [of the salary]. (S-33, school B teacher)

Seeking to distance themselves from such self-interested behaviour, and claiming that they were motivated mainly by professional interests and relationships with students, interviewees often reported cases of ‘some other teacher’ using the new policy to her/his own benefit, ignoring the interests and needs of students and the school.

Some school B teachers showed indifference to the new accountability requirements – something we did not see in school A. For example, when asked what can be done to improve the performance of the least motivated students in GIA, a teacher in school B answers: ‘Well, what can we do? There will be failures … Maybe they will reprimand us … But these are not Stalin’s times, they won’t fire us for the “fails”’ (S-32). Another school B teacher explains that it is not possible to follow the recommendations of the city authorities and train students to GIA in small ability-based groups, because the school receives no funding for that. School A teachers, on the other hand, say they feel pressured by the accountability policy. This is probably related to the school administration’s response to the policy: it has embraced competition as a mechanism to improve performance.

Perceiving criteria-based assessment of teachers as a useful management tool, school A’s administration actively employs rankings, ‘not to determine who the best teacher is, but just to motivate’ (S-1). And, according to another respondent, the ‘performance-based part of salary really works here’ (S-7). Moreover, school A administrators appreciate ‘diagnostic tests’ not only for revealing students’ learning gaps, but also for indicating the standing of school A in the city. Administrators look for opportunities to promote their school through participation in contests and rankings. At a meeting, one of them explained why teachers should prepare students for a paid-for ‘metacognitive skills Olympiad’: ‘Only [a small fee], and our results in forming metacognitive skills will be all over Russia, so we should participate’ (S-7). Managing the composition of the student body by attracting and retaining high performers and getting rid of low performers is an important strategy to improve school results: ‘After grade 8 – all our “stars” left, 13 students, some [other students] came, but they are weaker. We want [to include] something in the school’s development plan, to retain such students. We attract [new] staff’ (S-7). The school also practises ability grouping in test-preparation classes, following the recommendations of the city teacher-training sessions ‘on the improvement of national examination results’.

In contrast, in school B, administrators aim to minimize the damage done by the performative accountability policy. In line with their critical view towards many aspects of the policy, including performance-based pay, school B administrators make efforts to compensate for its perceived unfairness:

Twice a year, when we pay the performance-based part [of salary] – teachers are stressed, offended … Of course, half of this performance-based part I give right away for work intensity, to those who have more working hours. But at first our physical education teachers had the biggest performance-based part because they participate in some events all the time – and subject teachers were offended. And it may be that a teacher who does not participate in any contests, but can explain [study material to students] well, is a good teacher … Competition between teachers – who needs it? (S-2, school B administrator)

School B administrators monitor their school’s results in comparison with other schools in order to ‘not lag behind our neighbours, not be the worst [school in the neighbourhood]’ (S-12). Knowing teachers’ strategies of ‘inflating’ performance scores, administrators are sceptical towards such practices. To improve students’ results, they encourage teachers to devote extra time to close the learning gaps of low performers. They also ask teachers to retain high-performing students by ‘working with parents’. Finally, the school seeks to avoid the risk of failures by discouraging participation in optional GIA tests:

‘[Teachers get performance scores] from online contests, paid-for ones, there are so many of these today. [If a teacher has] one able student, he/she participates in all these contests – [then] a teacher gains points as a Stakhanovite (a highly productive worker). […]. Though the real work is still in the classroom. (S-12)

Interviewer: Why did only a few students choose to participate in optional GIA-9?

Administrator: Since we are held accountable for GIA scores, we did not greatly encourage kids to take an optional GIA. If they fail, then you have problems. (S-12)

Overall, school B teachers and administrators express disappointment over the new accountability policy. They reminisce about the support and supervision of local teaching methods by organizations that have since been restructured. To show compliance with the law, the school, it seems, implements the new policy only as a formality:

I wish [authorities] would give us, as before, clear instructions about how we should work. We would teach according to them. And now [the regional minister for education] only tells us: ‘Switch your heads on!’ Well, we switch them on. [We] retell old stuff in new words. Though we will teach the same way as we used to. Well, we have multimedia devices now – these are all the [real] changes. (S-12, school B administrator)

Discussion and conclusion

In this paper, we have examined how a performative accountability scheme adopted in the Russian school system is interpreted and translated by teachers and school administrators in two schools within a disadvantaged city district, building upon the theory of policy enactment developed by Ball et al. (2012). Specific attention has been paid to how policy enactment interacts with the local and school contexts and school- and actor-specific conceptions of accountability. While earlier research has established that education policy outcomes are mediated and shaped by schools’ context and agency, the specific mechanisms of this influence in the enactment of performative accountability have been little explored (Verger and Parcerisa, 2017a). This paper endeavoured to fill this gap. Our findings confirm that the process of putting a performative accountability scheme into practice, as is the case of other policies, is far from straightforward. Many aspects of the new policy conflict with the professional values of teachers and administrators, causing moral conflicts, particularly in circumstances where the interests of students, teachers and the school no longer align. Moreover, being located in a disadvantaged city district brought specific challenges which are less commonly faced by schools in more advantaged neighbourhoods (Lupton, 2004). While such challenges may restrict policy enactment, as has been shown in the case of school B, the effects of context are rarely acknowledged in school improvement policies, fearing that contextual factors ‘might be used as an excuse for poor practice’ (Lupton, 2004, 4). Our paper calls attention to the need to take into account the perceptions of key stakeholders, as well as their opportunities, if one aims to understand the enactment and outcomes of a performative accountability reform.

More specifically, the accountability policy introduced in Russia has multiple components, some of which hold schools and teachers formally accountable to the state, the public and parents, while others seek to increase teachers’ internal feelings of responsibility and change teaching methods informed by test results. Teachers in both schools accept only a few of the policy’s components and are critical of other policy demands and related procedures. Their view of accountability as inadequate and even harmful is probably related to the schools’ location within a disadvantaged city district: these schools have to work with a diverse student body, and cases of students’ lack of commitment to studies and poor parental support are common. These challenges were particularly evident in school B, located within a local area characterized by more severe deprivation. At the same time, there are aspects of the accountability reform that teachers internalize. They recognize that performance in examinations is crucial to the students’ future, and they also view examinations as the government standard for what should be taught in schools. For these reasons, they take on the moral obligation to help students prepare for examinations.

Administrators’ reactions to the policy differed in several aspects from teachers’ reactions, presumably because of their managerial roles and pressures that local authorities and parents exert on school administrations. Administrators at school B expressed even greater dissatisfaction with the policy than their teachers, since they regarded it as an additional source of stress for the teachers, while hiring and retaining staff in school B was reported as challenging. The accountability scheme also takes into account mainly the academic achievements of students, and other contributions by the school to students’ development and well-being go unnoticed, which makes a significant part of school B’s work invisible to the authorities and other external observers, including journalists and families exercising school choice. In contrast, administrators at school A expressed greater appreciation of the policy than school A teachers. In this school, the policy provides a useful management tool to steer teachers’ work in line with the school’s academic achievement-oriented strategy. The policy also makes the school’s results more visible, which helps administrators to defend the school’s efforts in the face of demanding parents, reaffirm its reputation and keep the student intake competitive. In sum, administrators’ and teachers’ professional concerns are to some extent diverging, and they are affected differently by the pressures of accountability; hence their interpretation of the policy differed.

Building upon the typology of schools’ logics of action (Ball and Maroy, 2009), we demonstrate how school administrators translate policy demands in line with their logics of action. In school A, where the instrumental logic of action dominates, the policy’s focus on achievement and competition is embraced. The administration applies performance-based pay and teacher rankings to stimulate teachers to improve student performance. Moreover, school A strives to improve its reputation by participating in contests, and ‘cleans out’ low-performing students in order to improve its ranking. In contrast, in school B the characteristics of an expressive logic of action predominate, conflicting with the principles of competition and individual accountability on which the new policy is based. The expressive order emphasizes equality and cooperative relationships, which teachers and administrators believe are damaged by performance-based instruments. By partly resisting the policy, which favours competition and self-interest, school B can attract and retain teachers despite its weaker resource base and more challenging student body. Administrators at school B seek to ‘protect’ teachers from the effects of the accountability policy. They criticize the incentive system introduced by the new policy, emphasize that the ‘real work is in the classroom’, rather than taking part in contests, and attempt to organize payment in a more egalitarian way than through performance criteria.

Our results corroborate existing evidence on the effects of high-stakes performative accountability in different national contexts. Similar findings in schools pursuing performance improvement through a selective intake of students from upper-/middle-class families and more ‘able’ students, while excluding ‘disruptive’ students coming from less privileged backgrounds, are found in studies comprising research from the UK, USA, Chile, Australia and New Zealand (Falabella, 2014; Jennings, 2010). Moreover, previous studies show how schools focus on reputation and spend more on marketing activities in order to raise their profile and attract more and better-performing students (Ball, 2006; Gewirtz, 2002), a strategy employed by school A. Finally, there is also evidence of diminishing collaboration between school staff members, while individualism and competition are enhanced (Darling-Hammond, 2007; Falabella, 2014; Gewirtz, 2002).

These results of previous studies resonate with our observation that performative accountability reinforces instrumental logics of action and undermines expressive school orders. While different logics of action dominate in school A and B, we observed the employment of educational strategies and pedagogical practices predominantly associated with the instrumental logic of action within both schools in response to the performative accountability policy. For example, we found evidence for gaming strategies in both schools. Teachers in the two schools attempt to gain more performance points by participating in paid-for online contests, and in other ways, to capitalize on the high performance of certain students, while other students’ needs and more meaningful teaching activities might get less attention. Both schools also discourage low-performing students from participating in optional examinations. While there is an apparent contradiction between such strategies and the principles of an expressive order, a closer look at the rationales behind these actions can provide an explanation. In our interpretation, in school B gaming is a way to prevent the policy having too great an effect and to retain a diverse student body, despite the external pressures to demonstrate high performance. In school A, gaming is probably the ‘darker side’ of the achievement-oriented behaviour. In addition, teachers in both schools complain about the damaging effects on cooperation between staff members that performative accountability produces. However, somewhat paradoxically, we observed active teacher collaboration in both schools, even though the theory argues that collaboration is a distinctive characteristic of expressive order. As in the case of gaming, we assume that there are different rationales that underlie similar behaviour in schools with different orders. While in the expressive order school, collaboration is a way to offer support and compassion to colleagues, in the instrumental order school it is a means to achieve higher results by combining resources. It seems that in both schools, performative accountability creates conditions that discourage ‘care-based’ cooperation, whilst promoting ‘ambition-based’ cooperation, and thus a dialectic tension emerges between enhanced competition and collaboration.

With regard to pedagogical practices, we find that the two schools employ similar pedagogical tools to assist students in preparing for examinations. Both schools accept the idea of ability grouping as an effective pedagogical instrument to enhance test scores, even though ability grouping is thought to be associated with social exclusion (Falabella, 2014) and employed mainly in the instrumental order (Ball and Maroy, 2009). We also observed frequent use of ‘diagnostic tests’ and ‘mock examinations’ in both schools, which is a common effect of the introduction of high-stakes tests, but a contested practice since it displaces more creative teaching methods (see e.g. Firestone et al., 2004). Specific training that teachers receive as a part of accountability policy implementation in the case locality is the probable reason why both schools accept these tools rather uncritically. There is a need to raise awareness among educators of the effects produced by different test-preparation techniques, so that schools make a more informed choice of teaching methods in line with their pedagogical aims.

Finally, our analysis illuminates how subtle differences in the schools’ location and history may mean schools are inclined to adopt different logics of action. The catchment area of school B appeared to be more severely deprived, and the school historically worked with students from the local orphanage. The emotional problems of the school’s student population, many of whom lacked parental attention and care, meant that teachers felt the need to offer support and partly take on a parental role. Low income of most students’ families also restricted school B’s ability to use additional paid-for preparatory tests, and students’ ability to hire private tutors. In contrast, whereas teachers in school B complained about the lack of parental interest and engagement, teachers in school A experienced parental pressure to obtain high academic results. The fact that more middle-income parents with higher education live in the catchment area of school A may explain the value attributed by these families to education. Hence, school A’s administration had both a greater urge and more resources to develop the instrumental order and implement the performative accountability scheme. Highlighting the potential of the schools’ contexts to both enable and constrain their logics of action, we note that it would be inaccurate to derive simple causalities between contextual conditions and schools’ strategies. Value orientations and strategic choices of school staff are likely to have contributed to their interpretations of the most important aspects of the schools’ contexts. An expressive order school is likely to pay greater attention to the challenges and needs of its student body, while an instrumental order school might emphasize the necessity and possibility of high achievement in any conditions.

To sum up, our study contributes to research on policy enactment by analysing the interaction between a performative accountability scheme and schools’ logics of action and local context. Our research offers insights into how schools with different logics of action can react to external pressures, and how different professional groups within schools experience policy pressures in dissimilar ways which condition their diverse reactions. We thereby provide a deeper understanding of why particular outcomes of accountability policy may occur under particular circumstances. We conclude that performative accountability mechanisms reinforce instrumental, and impede expressive, logics of action in schools, while in both cases they produce tensions, in particular for schools in disadvantaged areas. Demonstrating how policy pressures promote behaviour predominantly associated with the instrumental logic of action within both schools, we contribute to the understanding of factors that can trigger the process of shifting between logics. We also open up the notions of ‘instrumental’ and ‘expressive’ orders by looking closer at the administrative and pedagogical rationales and sources of occupational stress in two schools that show characteristics of different orders, but at the same time have some unexpected similarities. This investigation allows us to hypothesize that different rationales can lead to similar behaviour patterns in response to performative accountability pressures.