Data warehousing organization: Infrastructural experimentation with educational governance

Abstract

Organization is increasingly entwined with databased governance infrastructures. Developing the idea of ‘infrastructure as partial connection’ with inspiration from Marilyn Strathern and Science and Technology Studies, this article proposes that database infrastructures are intrinsic to processes of organizing intra- and inter-organizational relations. Seeing infrastructure as partial connection brings our attention to the ontological experimentation with knowing organizations through work of establishing and cutting relations. We illustrate this claim through a multi-sited ethnographic study of ‘The Data Warehouse’. ‘The Data Warehouse’ is an important infrastructural component in the current reorganization of Danish educational governance which makes schools’ performance public and comparable. We suggest that ‘The Data Warehouse’ materializes different, but overlapping, infrastructural experiments with governing education at different organizational sites enacting a governmental hierarchy. Each site can be seen as belonging to the same governance infrastructure but also as constituting ‘centres’ in its own right. ‘The Data Warehouse’ participates in the always-unfinished business of organizational world making and is made to (partially) relate to different organizational concerns and practices. This argument has implications for how we analyze the organizational effects of pervasive databased governance infrastructures and invites exploring their multiple organizing effects.

Introduction

If you visit the Danish Ministry of Education’s webpage ¹ ‘The Data Warehouse’ (TDW), dashboards reporting the performance of Danish schools ² will populate your screen. The dashboards compare how schools perform based on statistics of, for example, surveys of pupils’ well-being, graduation grades and absence, benchmarking those measurements against national averages. It is mandatory to include selected dashboards in the ‘quality report’ that schools and municipalities must publish every other year. This report has become a central accountability tool, which shapes relationships between the ministry of education, municipalities and schools in Denmark, and the report reflect tendencies in international educational governance towards ‘data-driven’ management and accountability (Lawn et al., 2011; Ozga, 2009). In Denmark, TDW is now an important component in the infrastructuring of governance, ‘databasing’ inter-organizational relations between the ministry, the municipalities, and the schools. For organization studies, TDW is interesting. As a device that remains unsettled, it enables us to inquire the influence of databased governance infrastructures on organization and on inter-organizational relations. In this article, we engage these matters by exploring three sites in educational governance: the ministerial agency responsible for the development and maintenance of TDW, a municipality and a school. We explore the concerns and practices that emerge at each site in relation to the TDW and how TDW enacts a governmental hierarchy.

Studies of databased educational governance systems often claim that data infrastructures profoundly influence both intra- and inter-organizational relations. In her study of databased school inspection in England, Ozga (2009) states that data ‘give[s] the appearance of deregulation, [while] the centre maintains control through its management and use of data’, reducing municipalities ‘to distribution points in the flow of data around the system’ (pp. 140, 160). Similarly, Selwyn (2015) suggests that database governance infrastructures lead to a ‘recursive state where data analysis begins to produce educational settings, as much as educational settings producing data’ (Selwyn, 2015: 72; see also Williamson, 2016). These studies propose that databased governance infrastructures are extremely powerful as they amount to self-reinforcing feedback loops, which expansively colonize education. In this view, data structure organizational practices, centralize power and come to define inter-organizational relations in a governance hierarchy.

These observations are important but they may also hinder us from addressing organizational effects other than those fitting this systems logic. Following Daniel Neyland (2015), we suggest not ‘to commit to a singular (…) metaphor’ (p. 130), such as control or recursion, in studies of how database or other information technologies and organization intersect. We propose, instead, to address databased governance infrastructures such as TDW ethnographically, because it allows for attending to situated organizational concerns and practices, which might challenge ideas of such an overall logic and how it operates. With the idea of ‘infrastructure as partial connection’ Strathern (2004) we invoke to theorize how relations between what we normally consider different ‘levels’ of governance entail both connection and disconnection. We connect this concept to discussions in Science and Technology Studies (STS) of ‘infrastructures’ as open-ended and ontologically entwined with organizing (e.g. Edwards, 2017; Jensen and Morita, 2015). This allows us to explore how TDW both enacts a hierarchy with a pervasive and persuasive presence in the organizational ecology of educational governance and how this has multiple organizing effects. Each site, as our analysis suggests, is then both part of the ‘same’ infrastructure, but they also enact this infrastructure differently through experiments with making TDW relate to situated concerns (Jensen and Morita, 2015). This challenges the idea that an all-encompassing governance context determines organizational concerns and inter-organizational relations while acknowledging that the idea of such context is certainly in operation.

In the following section, we present the theoretical and empirical resources for this article and we discuss the implications of the concept of ‘infrastructure as partial connection’ for our method of multi-sited ethnography (Marcus, 1995). An introduction to TDW and its recent history follows, setting up the case for analysis. In the first analytical section, we examine the National Agency for IT and Learning (NAIL), the agency under the Danish Ministry of Education, which is responsible for developing and maintaining TDW. We argue that TDW figures here as experiments with making data available to many different users while also configuring those users (Woolgar, 1990). At the second site, which is a municipality, we examine experiments with combining figures from TDW with narrative and qualitative ‘sampling’ of the schools under scrutiny. The third site is a school in another municipality where TDW dominates its ‘valuation’ (cf. Muniesa, 2011). This brings about experiments with accounting for performance data by recontextualizing these in verbal accounts. In a situation where databased infrastructures are becoming ubiquitous, the conclusion addresses implications of exploring data infrastructures as partial connection for organization studies. Rather than assuming different ‘levels’ or ‘scales’ of organization (cf. Taylor and Spicer, 2007), partial connection allows us to keep open the question of scale and connection across ‘levels’. What is part or whole, centre or periphery, inside or outside the governance infrastructure varies depending on how different situated practices experiment with TDW, as well as how we situate the analysis and juxtapose our observations. This collapses the distinction between infrastructure and organization, and we might think of organizing with infrastructure as processes of making and cutting relations (cf. Strathern, 1996).

Infrastructure as partial connections

Technology is integral to the making of organization, and information technologies ‘afford changing organizational forms’ (Petrakaki et al., 2016; see also Zammuto et al., 2007). As Brian Bloomfield (1995) notes in a discussion of technology and organization, however, analysis is not a question of ‘how the “two” separate objects are to be interrelated but requires that they cannot be separated out’ (p. 495). The implication is that the TDW should not be understood simply as a technical foundation upon which organizational activities take place (Jensen and Winthereik, 2013). Rather, our assumption is that TDW is integral to socio-technical processes of organizing.

We further this processual and non-deterministic understanding of information technology and organization by developing the idea of infrastructure as partial connection. This shifts the focus of inquiry from a question of technology and domination (e.g. Bloomfield, 1995) to one of how organizing with information technology involves making and cutting relations, practices which materialize in different intra- and inter-organizational relations.

Edwards (2017) offers the view that infrastructures are ‘complex ecologies whose component systems must continually evolve to match the changing characteristics of the related systems around them’ (p. 340). This idea invites us to explore infrastructures as decentralized and distributed material-semiotic networks with no demarcated boundaries. The emphasis on ‘related systems’ in the concept of infrastructure entails exploring TDW as a part of situated practices. Even if a dashboard measuring a school’s performance looks the same on the screen, whether it is located in the Ministry of Education, the municipality or the school, ‘it’ is taken up differently and has different organizing effects. The argument here is relational: TDW becomes something different in different organizational practices, depending on how it is made to relate. This means, then, that the infrastructure of which TDW is a part is (re)configured locally, while no locale is isolated from its surroundings.

Mol’s (2002) suggestion that objects are not singular but multiple and practical is helpful in eliciting the distributed and decentralized becomings of infrastructure. Mol (2002) suggests that realities ‘are brought into being, sustained, or allowed to wither away in common, day-to-day, sociomaterial practices’ (p. 6). She discusses this as the multiple ontology of objects, enacted in different, overlapping versions rather than singular existences. The ‘ontological politics’ (Mol, 2002: 165) of information infrastructures – the question of what is enacted – is first and foremost practical, and it speaks of how different organizational realities materialize with TDW. Although TDW expresses an attempt to represent organizations, the use of its dashboards is a practical and situated matter and we cannot assume that they always acquire the same meaning or response. This view of ‘representation’ keeps the world flat: representation is not a question of reference between world and text (or dashboard, in this case). Rather, any representation can be considered an addition to a network, producing new associations and capacities while cutting ties with others (Latour, 1988). Jensen and Morita (2015), also inspired by Mol, suggest that infrastructures should be understood as ‘ontological experiments’, emphasizing that they are active components in the ongoing making of reality. The concept of infrastructure, in this view, addresses ‘how worlds are concretely made, conjoined or transformed by the co-evolving relations of multiple agents; people, technologies, materials, spirits, ideas – or what have you’ (Jensen and Morita, 2015: 82). Databased infrastructures materialize in situated practices, adding new elements, concerns and capacities to organizing, while backgrounding others. One important consequence of this idea for organization is that infrastructures are socio-technical and practical accomplishments, which participate in organizing, including processes of inclusion and exclusion.

Larkin (2013) argues that in being both a ‘thing’ and ‘the relation between things’, infrastructures have a ‘peculiar ontology’ (p. 329). While we have accounted above for how we analyse TDW as integral to situated practices of organizing, we still need to conceptualize the relation between those ‘versions’. Strathern’s (2004) famous notion of ‘partial connection’ can be used to specify how we understand the relations between (and within) our three sites. Strathern (2004) uses the term to theorize ‘social relationships as connections at once part and not part of her or himself’ (p. 40). In the context of our analysis, Strathern’s ambiguous concept implies that we need to explore ethnographically how TDW and organizations are made to relate. Insofar as TDW enacts a chain of accountability, it also simultaneously differentiates the entities in that chain.

Partial connection is also a way to refocus the current emphasis on data as a facilitator of flows between organizations (Green et al., 2005). While TDW affords the circulation of data between different sites, this is not only a matter of connecting but also of separating. This resonates with other ethnographic studies of infrastructures that have demonstrated the functional indeterminacy of infrastructures as well as how they divide (Harvey et al., 2017; Reeves, 2017). Analysing infrastructure as partial connection entails paying attention to how it rearranges complex orderings, producing new concerns and opacities. If infrastructures do produce new organizational realities, then these are not stable or closed worlds but consist of ongoing frictions and experiments, for instance, when actors challenge, ignore or repurpose the intentions of those who designed the infrastructures (Akrich, 1992). We therefore do not see TDW as a determinant in the making of educational organizations (as the metaphor of recursion, for example, invites us to do). Rather, we use the concept of infrastructure as partial connection to explore TDW as multiple, both (a) in its enactments in situated organizational practices through experiments with possible associations and (b) as partial connection within and between organizations.

What does this mean for our ethnography? It commits us to assume that TDW does not simply materialize a steering hierarchy from ministry to school, with the former emerging as ‘center of calculation’ (Miller and Rose, 1990) and the latter as the periphery to be represented by the former. Much more is going on. As Barbara Czarniawska observed in another organizational ethnography: ‘although calculation centers still do exist, the activity of calculation has been dispersed in economic organizations. Multiplication of centers of calculation led to a situation in which it is pointless to speak of “centers” or “specific sites”’ (Czarniawska, 2004: 778). The situation is similar in our case. We do not downplay the important role of the Ministry of Education in the proliferation of databased governance in Danish education, but we keep open the question of what TDW means for organizing at the three different sites we investigated. Each site may be considered a ‘centre’ in its own right with the other sites of governance figuring as parts of their wider ecologies.

Partial connection and multi-sited ethnography

Anthropologist George Marcus (1995) famously coined the term ‘multi-sited ethnography’ as an alternative to classic single-sited ethnographic participant observation. He advocated ‘tracing things in and through contexts’ (p. 107). Czarniawska (2004) too has observed that the bounded field-site directly prohibits the analysis of organization in a situation in which information infrastructures are pervasive.

While our ethnography in this study encompasses three sites, and in that sense is multi-sited, the notion of partial connection has implications for Marcus’s idea. First of all, ‘tracing things in and through contexts’ (Marcus, 1995: 107) implies that context(s) pre-exist the object of study. As anthropologist Matt Candea (2007) notes, this is a ‘problematic reconfiguration of holism (on a grander scale)’ (p. 167). Rather than assuming that TDW sits within, or produces, one overarching context, we explore how the organizing in which it participates involves experiments with contextualization. How, for instance, can the NAIL make TDW a ‘relevant context’ for decision-making in municipalities and schools? And how does TDW lead municipalities and schools to articulate ‘other contexts’ when giving accounts of their performance? As ethnomethodologist Lawrence Wieder (1974) writes, contexts (or codes) are resources for people to account for their behaviour, rather than a setting. This means that contexts are mutable and contextualization involves precisely the question of partial connection – how to see TDW as a part of or not a part of the organizational realities it seeks to represent. For us, studying TDW then entails holding the question of context open; infrastructures as partial connections, we argue, involve negotiations of TDW through different enactments of context.

Methodologically, we would also like to clarify Marcus’s characterization of multi-sited ethnography as a technique that ‘follows’ an object across different sites (which reminds us of the crudely formulated but famous methodological shorthand associated with actor network theory: ‘follow the actor’). In studying TDW, we do not encounter a clear-cut object that moves. Rather, we encounter and carve out enactments of TDW in relation to different organizational sites and moments (cf. Mol, 2002). Consequently, ‘following’ is an empirically and analytically open question of locating processes of organizing associated with TDW. In relation to this, we do not claim that our analyses offer exhaustive descriptions of organizing processes related to TDW. Instead, addressing the TDW at these three sites allows us to attend to some specificities in the ongoing infrastructuring of educational governance.

TDW: a short background story

TDW exists as the result of accountability requirements introduced by the Danish Ministry of Education in the 2000s, and it is central to the ministry’s governance of the municipalities’ administration of Danish primary and lower secondary schools. While the schools are subject to national legislation, objectives and oversight, they are owned and operated by municipalities. Since 2007, schools have been required to produce a quality report every other year. This report gives an account of the school (e.g. the number of pupils, employees, expenditures) and evaluates its pedagogical activities using quality indicators such as graduation examination grades in Danish and math, pulled from TDW, as well as evaluations of previous interventions and action plans for improvements (Danish Evaluation Institute, 2008: 7). The quality report was one of several measures taken by the ministry to improve and standardize the municipalities’ inspection of schools in the wake of a critical governmental report on Denmark’s first participation in the Organization for Economic Co-operation and Development (OECD)’s educational flagship programme PISA (Programme for International Student Assessment). The report criticized ‘the [ministry’s] assumption’ that municipalities and schools were competent to govern their own operations (Ekholm et al., 2004: 29). The PISA report further noted that a lack of data was a central hindrance to improving the quality of education: ‘Schools usually do not receive any feedback from the ministry, such as information about how schools perform compared to earlier years or compared to schools with similar pupil populations’ (Ekholm et al., 2004: 31). ³ In 2008, an evaluation of the municipalities’ first quality report echoed the call for more data: ‘it is not possible to undertake immediate comparisons between municipalities when they calculate numbers differently’ (Danish Evaluation Institute, 2008).

In the same period, NAIL developed the ‘data bank’, an online spreadsheet service where municipalities could access rows and tables with officially authorized data. Like TDW, the data bank is a collection of data gathered from other IT systems. In 2014, the data bank was partially replaced by a new web portal, the ‘management information system’ LIS (Danish: Ledelsesinformationssystemet), which was introduced as part of a grand national school reform process. Like the data bank, LIS provided access to reports on the performance of schools and municipalities in relation to national quality and performance indicators. Compared to the ‘data bank’, the main change was that LIS automatically generated quality indicators and other key figures for local quality reports, relieving administrators of labour-intensive indicator querying in the data bank (Local Government Denmark, 2014).

During 2016, the name LIS was dropped and the more generic term ‘Datavarehuset’ (TDW), which was already used for upper secondary institutions, was adopted for the primary and lower secondary school as well. This move situated primary and lower secondary school data as material to be used for quality reporting in relation to other data: data on upper secondary education and also as data for ‘the general public’: In 2016 the Ministry launched The Openness Initiative with the purpose of making it easier for parents to compare schools. The report format was expanded, too, towards interactive dashboards and maps that invite users to zoom in and compare institutions with reference to different quality indicators. Rather than aiming for a specific user and use – as LIS, was aimed for the municipalities and their needs when writing quality reports –the TDW became a more generic technology. These developments – the changes in name and visualization – testify that TDW has never been a static entity but, to repeat Edwards (2017) observation of infrastructures, are the subjects of ongoing developments.

Data massage: or how to make data available

NAIL is responsible for developing and maintaining TDW (and its forerunners). Tasks related to TDW include routine work such as cleaning and aggregating data gathered from the schools’ and municipalities’ administrative IT systems, then feeding data cubes to TDW. A managing government officer described online access to data as a matter of enabling a particular kind of governance:

This is the State’s capacity: to make available data. This is a different kind of governance than to hit someone on the head (…) [This is] to massage a culture of evaluation into the system, into all relations of governance. (…) Eventually we will offer each school principal some very detailed data about his school. Then he can glance down and say: ‘All right now, how is the fourth grade doing? Is it the boys or the girls?’ It is not because we would then sit (…) and inspect things from here – we do not have the capacity to do that at all. (Government officer, Ministry of Education, spring 2015)

The officer contrasts ‘massage’ with notions of data-based surveillance and control, associating techniques of relaxation or therapy (defined in Merriam-Webster as ‘therapeutic treatment especially of bodily, mental, or behavioural disorder’) to data. Data as a means to ‘massage a culture of evaluation into all levels of governance’ is a rendering of governance by numbers that emphasizes its scopic aspect. The very possibility of ‘glancing’ at performance data afforded by TDW is imagined as instructing school principals’ (and thus also teachers’) perception of their own performance. Enacting the dominant trope of the managerial gaze that can ‘glance’ and view the organization from above (Oliason and Sørensen, 2013), the TDW is tasked with shaping municipalities’ and schools’ conduct of themselves (Rose, 1996). Data is envisioned as a recursive input into reflexively understanding one’s own performance; tasked with directing local managers’ attention and making them ‘act’ on data. The idea that data can bridge managerial decisions and pedagogical activity in schools resonates with Bloomfield and Vurdubakis’s (1997) study of how IT can be imagined as producing new connections inside organizations. This hopeful aspiration, however, is also a partial connection, reflecting the belief that municipalities and schools have not yet internalized such a ‘culture of evaluation’. Like other infrastructures, TDW has an affective dimension, ‘intimately caught up with the sense of shaping modern society and realizing the future’ (Larkin, 2013: 322).

At NAIL’s Centre for Data and Analytics, the statisticians worked to make such scopic activities possible by means of TDW. Other concerns also materialized these processes. Reflecting on the difference between present and earlier versions of TDW, a statistician explained how previous versions of the website had included more notes, which provided more context to data:

The data bank aimed at specialized users who had command of the technology. The notes [documents published on the ministry’s website] were for the general public. This is reversed today. With the dashboards, everybody has quick and easy access to data. Our notes now form the background to the data rather than being the news. But my feeling is that we have less influence on how people interpret data, like, what does this difference in performance between two years mean, is it big or small? (Interview, May 2017)

As dashboards become the main form of report, and is made accessible to everybody, concerns about data interpretation emerge. Compared to earlier reporting, the dashboard reverses the relationship between numbers and text. The earlier reports were text-based, describing the statistics behind the numbers and offering an interpretation. The dashboards are visualizations of data, benchmarking and comparing schools and municipalities for quick overview. These highly suggestive visualizations collapse the distance between data and interpretation. As a result, interpretation is no longer reserved for those people with ‘technical expertise’, as the statistician explained later in the interview. The fact that the notes on method and interpretation have been moved from foreground to background, reflects an assumption that few people are interested in how data are produced and how figures are calculated. Much like ‘data massage’, we see here a concern with a construction of easiness often associated with IT (Markussen, 1995); yet it also produces a sense of partial and incomplete connections – the statisticians now have less influence on how users interpret data.

Before going online with the dashboards, the Centre for Data and Analysis held several meetings at which they presented mock-ups of the dashboards to senior personnel in NAIL. As part of the ministerial flagship project – The Openness Initiative –the project was of high priority. Discussions in those meetings included at what ‘scale’ data would make sense to users. While municipalities own primary and lower secondary schools, upper secondary education schools (age 16–19) are self-governing institutions. Some civil servants felt that the latter might be better assessed at the regional level than that of municipalities. Questions of where to place metadata, and especially notes on the production of data, also emerged. Should meta-commentary show as a link, a tab, a footnote, or centrally on the screen? The latter suggestion arose from the concern that the media might simply take screenshots of the dashboards, and make graphs and comparisons public, while ignoring all notes on their limitations and statically valid interpretability.

The ontological experiment here entails testing what it means to ‘make data available’. The questions of what kind of technology TDW should be, what kind of user it should presuppose, or how much information and guidance should accompany the data were not settled. Over time, the idea of ‘access’ takes on new meanings: from access as a question of offering data on a web portal, to how user-friendly the database is, including how much technical knowledge TDW presupposes on part of (imagined) users. The idea of open access involves pre-empting the use and misuse of data (for instance, by media), and configuring potential users and their informational needs (cf. Bloomfield et al., 1997; Woolgar, 1990). Working with TDW thus also prompted a kind of technological reflexivity: the statisticians saw TDW as a mutable entity that might easily become something other than they had intended.

Statisticians did not attribute this mutability solely to new political priorities such as The Openness Initiative. ‘Technical’ aspects of the data infrastructure also impacted this conception. As the two statisticians explained in an interview:

Statistician 1: Well, it’s not a real data warehouse [in the sense where] we define very strict master data [that organizes data across all the different it systems from where data originates] such as ‘what is a pupil’ (…) that we have a data model that encompasses everything

(…)

Statistician 2: It’s a softened version where (…) you build cubes from other sets of data (…) Normally, a data warehouse models data and ensures that things can speak to one another (…). We do this manually for the time being. (Interview, statistician September 2015)

The statisticians made sense of TDW by comparing it to an ideal-type data warehouse, automatically integrating data from different sources into one system and allowing for advanced and ‘intelligent’ analysis of data. The present warehouse, by comparison, was characterized by a lack of consistent metadata (data about data – for instance, defining a pupil). This forced the statisticians to do what they referred to as ‘manual work’. The (lack of) data infrastructure mobilized statisticians to organize their own work so as to attend to data quality and produce connections between data sets.

Working with TDW in NAIL thus implied working with an object that is continuously displaced, both by emerging political priorities, by new reporting and visualization formats and issues of data integration and quality. Insofar as TDW enacts the Ministry of Education as a centre of calculation that can represent other organizations, it simultaneously appears both fragile and internally diverse. Questions of how to visualize data and how to enable and limit the interpretation of data came up again and again, and these questions illustrate how laborious the work of producing commensurability and comparability is. Yet the idea that TDW can massage evaluation into the whole ‘system’ also reflects an ongoing experiment with governance by numbers: an experiment emphasizing reflexivity, learning and self-organization as central aspirations (cf. Morgan, 1986: 81–87).

Data gaps: experiments in sampling

As noted above, the Danish municipalities are obliged to use data from TDW in their quality reports. The quality report is, however, only the tip of the iceberg of a cascade of interactions around documents, including negotiations of development contracts and action plans. Pors (2011) has characterized these activities as inducing school principals to internalize political objectives: ‘[the aim is to bring] schools to understand the importance of the goals formulated by politicians so that they take it upon themselves to make their own goals reflect these’ (p. 155). This is descriptive of NAIL’s idea of data as a means for massaging a culture of evaluation into relations of governance – TDW seen as a stage for generating similar objectives across politicians and schools.

A development manager in the municipality that we studied, however, told us that since the introduction of quality reports in 2007 they had found it necessary to develop and test different evaluation practices to complement the quantitative data in their quality reports. As she put it: ‘I wouldn’t like to have a dialogue about the quality report [with representatives from a school] with only figures. It isn’t close enough to the teaching or children’s learning’. The TDW partially connects the municipal administration to its schools by adding figures to that relation; yet at the same time, that relation quickly seems incomplete. While TDW may produce knowledge of schools’ performance at a glance (to reiterate the government officer’s term from before), along with that glance comes a sense of lack: the municipality may know that a school’s performance is average (as the development manager expressed it), but not why this is the case or how the school could change this.

To accommodate this sense of lack, the municipality recently developed a new concept: ‘Learning Environment Evaluation’, to complement the quantitative data from TDW with qualitative observations. This expands the TDW infrastructure with new procedures for observation and results in a series of written inscriptions. The municipality assembles an observation team consisting of a municipality representative, a school principal and a teacher from another school. Equipped with a form that contains six focus points based on ‘knowledge about what increases learning’ – for instance, ‘classroom management’, a rather popular concept in Danish schools – the observation team spends a day at the school under inspection. The host school decides in advance which activities (teaching, meetings) the team is to observe at the school, and after the observations the team shares their reflections with the school’s management team. These reflections are subsequently condensed into two written narratives: a positive narrative about ‘good practices’ and a narrative about ‘things that made them wonder’. The municipality then assembles a document, which they refer to as a ‘data package’, containing both the quantitative data from TDW and the two narratives (structured by the six focus points). These are sent to schools in advance of the quality report meeting. In this situation, quantitative data, often accused of epitomizing a ‘shift away from the tangible and observable reality’ (Kallinikos, 2009: 185), surprisingly seems to enable the production of new qualitative forms of knowing organization. Rather than expressing an opposition between quantitative and the qualitative, between data and narrative (cf. Manovich, 1999), they come to operate alongside one another.

We might understand the qualitative narratives as an experiment in sampling and as way to produce new partial connections. As the officer said, ‘the narratives do not offer a “total picture”. We don’t visit all classes and when we visit, we only see glimpses, not the entire day. For example, we can’t make claims such as “they don’t evaluate” but only note that we didn’t observe any evaluation’ (interview April 2017). As observation days are costly in terms of workforce, these had been reduced from 2 days to 1 day of observation every year cutting them down to 1 day every other year was being considered. In spite of the limited representative capabilities of these narratives, as they are based on somewhat random glimpses, the development manager would not do without them:

The narratives give me a different entry point to a dialogue with the school principal. If the narratives note a lack of teaching differentiation [teaching that differentiates according to individual pupils as opposed to a one level fits all teaching] along with bored and uneasy pupils, we might ask whether a focus on variation and differentiation in teaching can improve grades and test results. We can actually include their teaching in our conversation. It’s not only based on figures or the school principal’s assumptions. It’s very concrete elements and we’re closer to the teaching practice. (Interview April 2017)

The narratives are seen both as partial and as getting ‘closer’ to practice. TDW, in this case, engenders a distance and following a need to know practices qualitatively: it produces a knowledge gap that begets bridging and infrastructurally expands into other practices of knowing. While the narratives are not deemed representative, they are infused with greater explanatory power than the quantitative data in TDW. TDW may thus produce a will to relate to schools in new ways. Rather than reducing organization, as some critiques suggest, TDW may have the effect of multiplying the organization into different forms of representation. In this sense, TDW engenders experiments with knowing, here through ‘sampling’. What does it mean to know practice? What to observe? How much? How can qualitative narratives help interpret the quantitative data? As the official reflected: ‘Worst-case scenario is that we just stare at data and then nothing happens’. The narratives, she elaborated, bring with them different commitments to interventions because ‘practice’ becomes visible in a way that it is not in TDW. This speaks to Singleton and Michael’s (1997) point, that incompleteness and indeterminacy constitute a productive problem. Here, they engender the forging of new relations between municipality and school. Indeed, it seems that the limitations in dashboards in TDW strengthen relations between the municipality and schools.

In that sense, we see an experiment with inter-organizational relations. Reflecting on the term ‘wonders’ rather than ‘flaws’ or ‘points of critique’ as the basis of one of the two narratives, the development manager noted: ‘We don’t want to come as fault-finders (…) One effect of this [appreciative approach] is that school principals become less defensive [in quality report meetings]; it fosters curiosity about what others have observed’ (interview April 2017). Indeed, the idea that school principals should not only be inspected but engage in reflections on observation and take part in observation teams at other schools illustrates that TDW does not solely enact a hierarchy of governance with school principals at the bottom. Rather, school principals, usually the object of evaluation, also take on the role of co-evaluator. In that sense, the infrastructural experiment with knowing has ontological effects: it produces school principals with new capacities for evaluation and new relations between the municipality and schools.

Reflecting on schools in socioeconomically marginalized areas, notorious for scoring low in TDW, the development manager elaborates:

The learning environment observation team also notes their competences. We acknowledge that they do what they have to but it’s difficult for them to raise the [performance] figures. The qualitative data softens up the conversation and we can have a more constructive conversation about this. (Interview April 2017)

This resonates with Larkin’s (2013) observation that infrastructures mobilize affect and ‘senses of desire, pride, and frustration’ (p. 10). While we have already discussed the narratives as a matter of knowing differently, they are also delegated with the task of setting in motion different affects than those produced by TDW. Positive affects, she hopes, will result in improved figures for the schools in TDW. Returning to the metaphor of massage, these activities aim to motivate school principals to practise evaluation. Here, however, what is in focus is not the activity of ‘glancing’ (performances) but rather ‘storying’ (making narratives). While this is a matter of how to represent practice, as accounted for above, representation here is not so much an issue of epistemology as one of generating productive and trusting accountability relations. Knowing that as a developing manager she will always be separate from the activities at the school, the narratives help her to partially connect to those practices and make school principals open up.

Accounting for data: negotiating contexts

Our final interlocutor was the school principal of a school in a different municipality. This school principal was generally concerned with ‘data’, as he told us. The school is located in a socioeconomically marginalized area, and several pupils at the school are deemed ‘vulnerable’ and ‘at risk’ due to parental neglect and high crime levels in the neighbourhood. Therefore, ensuring that teachers would produce and act on information about children at risk was important to him, and the school had established several procedures for teachers to register observations that ‘concerned’ them regarding the pupils’ well-being (Ratner, 2019). The school principal highlighted the importance of teachers’ sensations of children’s well-being as a part of the relevant ‘data’, emphasizing that with such matters, the quantitative data did not offer the whole picture. As we will emphasize later in this section, one challenge here was making such ‘locally’ produced data relate to the dashboards in TDW.

When asked about the role of TDW at the school, the school principal showed us different IT administrative systems on his computer, and said:

We only access TDW very rarely, and mainly in context of the quality report [every second year]. Basically, we already have all the data in these other systems. So if I need data on pupils’ absence, I enter Tabulex [IT for registering absence, among other things]. Data here is much more detailed. (August 2015)

The different administrative systems he showed us are the data sources of TDW. While NAIL, in the words of a managing government officer, focuses on ‘giving back data’ to the municipalities and schools, the school principal did not see a need for that, as he already possessed that data. From his perspective, he was the one submitting data to the Ministry of Education. On this site, the infrastructure around TDW emerges differently than in NAIL. For the statisticians and civil servants in NAIL, data from other IT administrative systems figure as input to TDW. For the school principal, the administrative IT systems are the main point of access to data. This is not only, as one might expect, due to greater familiarity with these IT systems; it also relates to the form of the data. TDW visualizes data in an aggregated form (e.g. it displays the average absence over a year per school). The school principal, in contrast, often needs detailed data about individual pupils or classes, within shorter time frames. TDW enables comparison with other schools – to see the absence rate compared with the average municipal and national figures. This kind of comparison valuates the school in relation to the performance of others. However, when engaging with ‘absence’ as an everyday problem, the need is rarely to evaluate performance but to identify patterns or instances at a more granular scale.

The school principal further explained that TDW is ‘imposed’ on them during quality assessments conducted by the municipality. In this particular municipality, the figures from TDW direct the quality conversation. Located in a socioeconomically disadvantaged district, they struggle with reaching politically fixed targets in, for example, national tests. In quality meetings, the school principal has to reflect orally on numbers, and illustrate the local challenges by drawing on the information collected locally on children’s well-being. TDW here affects experiments with articulation because the conversation about quality between municipality and school is also a negotiation of what counts as a legitimate ‘explanation’ of why a dashboards shows a performance far below national and municipal averages. In an interview, the school principal characterized his latest quality report meeting in the following way:

He [the managing official] kept focusing on how we could improve our results. But 30% of our pupils have cases at the social services department, we have parents who lack parenting abilities, it is difficult to be a child here and also difficult to be a school (…) we introduce all these factors [at the meeting] but it ends up sounding like a defence. But it’s part of data, it’s our context, it’s what surrounds the quantitative data. However, this fixation on indicators comes to equal quality in teaching. And we never do a good enough job. We feel very exposed. (School principal, January 2017)

Questions of how to contextualize data are central to how data are interpreted. What is defined as a cause? Who is held responsible? Who has the authority to determine whether a school principal’s account of the context has the same weight as the quantitative data? Can the school principal generate a relation to external account (in the sense of not being represented by the TDW) to displace its meaning? In this particular case, TDW materialized a steering hierarchy in which the municipalities took a strong role as inspecting agent, with the right to refuse the school principal’s explanation and demand better results before the next round of quality reporting.

Returning to the development manager from the municipality in the previous example, her experience with the quality conversation (although in her case that included qualitative data as well) may cast light on the uncertainty of the governing body in these meetings:

Sometimes they [school principals] have explanations, other times they make up excuses. They signal ‘we can handle this – do not touch us, don’t ask questions’ and enter a defensive mode. Others become curious when they see the data, they want to go explore. (…) They can be very open and honest about these things and then we can wonder about the data and explore the context together. (School principal, January 2017)

According to this manager, the question is not only about the quality of explanations, but also about how they are presented and what impression the school principal offers. Do school principals seem genuine? Are they open for exploration? Either way, the ‘lack’ of contextual knowledge around the data produces the need for more articulation. These articulations seem then to become part of an affective economy, in which some accounts are considered more authentic than others.

TDW thus occasions experiments with what a valid articulation of the context for data might be. We might consider these experiments as organization in the making (cf. Jiménez, 2007: xix). What counts as a good explanation of data is not given. TDW itself offers one version of ‘context’: the so-called socioeconomic reference, used on national test results and graduation examination grades. The socioeconomic reference was introduced in 2011 and is based on a statistical calculation of how pupils’ social and economic background (gender, heritage, parents’ education, employment status and income) influences examination and test results (Ministry of Education, 2016). The aim is to ‘see whether the school’s pupils have results that are better or worse or at level with pupils nationwide with similar background conditions’ (Ministry of Education, 2016). The indicator aims to make sure that schools are not held responsible for ’conditions external to the school’, such as parents’ unemployment. We might understand the socioeconomic reference as an attempt at framing (Callon, 1999). Obviously, this framing does not contain the effect of all possible relations. Rather, framing is about excising relations that contribute to the calculations from the ‘multitude of relations that are ignored’ (Callon, 1999: 186). As the school principal at this site noted, the socioeconomic indicator does not include pupils with cases pending at the social services, a factor which, in the school’s experience, is often associated with poor performance.

In spite of good positive performance scores when taking the socioeconomic reference into account, the school principal explained that he still needed to account for their overall low performances. While the socioeconomic reference on one hand may provide a ‘socioeconomic’ context, it may not enter the evaluation of other figures or dashboards in TDW. We see here that the logic of articulating potential relevant ‘contexts’ and justifications seems potentially infinite, and that what happens here is also an experiment in boundary drawing. Which issues are considered separate from the school’s performance? The school principal’s attempt to bring in local data gathered by teachers (such as the number of pupils who were of ‘concern’ or had cases pending with social services) at the quality meeting is an attempt at reframing. This articulates similar concerns as the socioeconomic reference but in a different form. The salient question here is to what extent a school will be held accountable for poor results in TDW. This depends on how the context is assembled, the extent to which local data are allowed to influence the evaluation and the interpretive flexibility of the actors involved. In contrast to the previous example, which generated a paper format that included both quantitative and qualitative data, here data outside TWD are not formally considered a part of the conversation and need to be recognized as relevant before acquiring a role in the infrastructure. In that sense, infrastructures do not only connect, but also constrain, as Edwards (2017) notes: ‘The routines of thought and action they facilitate can obscure other potentially valuable approaches, sometimes with serious consequences’ (p. 341).

Concluding discussion

Critical scholars concerned with the increased ‘datafication’ (Kitchin, 2014) of organization often focus on the totalizing effects of ITC and governing infrastructures. Jannis Kallinikos (2009) suggests, for instance, that computation entails the ‘relentless analytic reduction of the composite character and complexion of the world’ (p. 183). Other examples include Dahler-Larsen’s (2007) claim of performativity (that ‘what gets registered and measured is what counts’), organizational logics such as ‘audit culture’ (Strathern, 2000) or data as ‘misrepresentation’ (Kallinikos, 2009; Lingard et al., 2012). These critiques reflect the general concern that the enumeration, quantification and computation of organization are ‘ill-suited (…) for the messy, human, cultural phenomena to which they are being applied’ (Seaver, 2013: 2).

If we think of data infrastructures as partial connections that involve ontological experimentation, we can add empirical nuance to and differentiate these concerns. Databased governance infrastructures are not simply tools of oppressors or distortions of human organization. The concepts of partial connection and ontological experimentation compel our attention to the frictions, translations and gaps that emerge alongside their powerful effects. Perhaps they are even intrinsic to the production of hierarchical power relations. Applying these ideas to TDW allowed us to tease out its multiple enactments rather than echoing the idea of an overall logic of governance. We explored TDW as organizing and infrastructuring in the making, and approached each site as containing its own redistributive practices that are only partially related to the other sites. The idea of partial connection has elicited how infrastructures produce gaps along with relations. TDW and the data it provides seem never to be capable of standing alone but always to require additional relational work, both within and across sites. In that sense, the TDW generates and sustains relations that are ‘incomplete’ and indeterminate. Exploring data infrastructures as partial connection has further showcased the ontological politics of data. What counts as data, even in a national databased accountability system, is not settled, and drawing the boundary between data/noise is a political act.

Dashboards, data warehouses and other digital infrastructures of governance are not just becoming prevalent in Danish educational settings but are central in the government and management of organizations worldwide. Ethnography is important in eliciting the subtleties and varied effects of the increased datafication of organization; and it offers a healthy inoculation against grand or totalizing analytical claims. Further ethnographic study of infrastructures as partial connections are necessary if we are to understand how databased governance organizes in spite of, or perhaps even because of, infrastructural flexibility and variation. Organization does not simply conform to data, but data do have strong effects. In a context of increased concerns with the power of algorithms, numbers and the datafication of organization, our contention is that infrastructure as partial connection expands our horizons of imagining their (dis)organizing effects, including how questions of scale, what emerges as centre and periphery, are momentarily up for grabs. This implies thinking not only infrastructures, but also organizing itself as a series of ontological experiments, a continued accomplishment of organization including negotiating, extending and rethinking its relational capacities.