Sage Journals: Discover world-class research

Abstract

There is an abundance of enthusiasm and optimism about how governments at all levels can make use of big data, algorithms and artificial intelligence. There is also growing concern about the risks that come with these new systems. This article makes the case for greater government transparency and accountability about uses of big data through a Government of Canada qualitative research case study. Adapting a method from critical cartographers, I employ counter-mapping to map government big data practices and internal discussions of risk and challenge. I do so by drawing on interviews and freedom of information requests. The analysis reveals that there are more concerns and risks than often publicly discussed and that there are significant areas of silence that need greater attention. The article underlines the need for our democratic systems to respond to our new datafied contexts by ensuring that our institutions make changes to better protect citizen rights, uphold democratic principles and ensure means for citizen intervention.

Keywords

Big data government democracy transparency accountability

Introduction

Governments worldwide are trying to make greater use of big data by linking up their own datasets, combining them with other data points and analysing this data to generate significant insights. Across fields of study and sectors varying terms are used when discussing the activities that comprise this shift towards greater data governance. These terms include algorithmic governance, artificial intelligence, machine learning, predictive analytics, probabilistic policymaking and Big and Open Linked Data (Janssen and Van den Hoven, 2015; Mergel et al., 2016).

Governments possess massive datasets about citizens, these are often historical, personal and continually updated, making them incredibly valuable from a data analytics perspective and risky from a privacy, security and rights perspective. There is great promise and a lot of excitement about how big data systems might be used to make discoveries, improve services, better understand local and global contexts, recover revenue by detecting fraud and tax avoidance, allow governments to respond in real time to pressing issues and events, identify and support those in need of help earlier and better, and improve the quality of life for citizens generally (Maciejewski, 2016). There is also a lot of money involved as governments purchase cloud and analytics services and support. Much corporate and academic literature in computer science, business and health tends to focus on big data opportunities (Clarke, 2016). Others point to a growing ‘backlash’ and the increasing attention devoted to detailing the negative consequences of big data-driven policy, decision-making and services (Dalton and Thatcher, 2014).

A ‘Five Eyes’ survey

As governments make greater use of big data systems concerns are being raised both within and outside of government. Australia, Canada, New Zealand, the United Kingdom and the United States are perhaps known best, in terms of big data applications, for their surveillance practices. The leaks by Edward Snowden revealed their involvement in an intelligence alliance known as the ‘Five Eyes’ (Wood and Wright, 2015). However, these nations have all also identified a desire to be leaders in public sector uses of big data more generally. They are used here to provide an indication of how countries are attempting to ‘datafy’ their public services and the opportunities and risks being identified as they advance these efforts. For the sake of space the survey is limited to the five eyes countries.

In the U.S. the Obama government launched a ‘Big Data Research and Development Initiative’ in 2012 and invested $200 million in the programme (Kim et al., 2014). When it comes to public services, big data is being used to mediate access to social services and benefits; make decisions about trial strategies, sentencing, policing, rehabilitation and child welfare decisions; influence access to education; and moderate healthcare. Concerns have been raised about the ways that big data systems can socially sort and exacerbate inequality in harmful ways. Other concerns relate to a lack of transparency, accountability and oversight that surrounds their use (Barocas and Selbst, 2016; Citron and Pasquale, 2014; Eubanks, 2018; Gangadharan et al., 2014; Muñoz et al., 2016; Podesta et al., 2014; Ramirez et al., 2014; O’Neill, 2016).

In the UK the government is investing heavily in expanding big data applications across the public sector. In 2013 the Government identified big data as one of the ‘eight great technologies’ and invested £189 million in big data technologies (STC, 2016: 7). In the UK the Government’s Digital Service, the Departments of Work and Pensions and Education, the Ministry of Justice and the Home Office are investigating uses of data analytics (Bright et al., 2014; Dunleavy, 2016). The government has established a Data Science Campus to help more departments and agencies make use of big data. Predictive policing is being used in the UK (Dencik et al., 2015). Local councils are starting to use predictive analytics in child welfare (Symons, 2016). The NHS is using big data to make health services more effective and efficient. There is a push from within government to make data sharing between departments and with outside organizations easier (STC, 2016). The Science and Technology Committee and the Alan Turing Institute’s Data Ethics Group have recognized some of the ethical implications of changing systems, but most of their discussions focus on issues of privacy, security and potential misuses of data (Drew, 2016; STC, 2016). Issues of fairness and discrimination have been raised by the Royal Society report on machine learning and the Cabinet Office’s Data Science Ethical Framework (Hancock, 2016; RS, 2017).

The Australian government introduced a Big Data Strategy in 2013 with the aim of becoming a world leader in public sector uses of big data analytics (DFD, 2013). The government set up a whole of government Data Analytics Centre of Excellence to develop and build uses of data analytics across government departments. The aim of these big data efforts is to improve data management, personalize services, advance problem solving and decision-making capabilities, and increase productivity and efficiency (Gamage, 2016). Government efforts have not been without controversy and negative impacts. Examples of contested practices include the government’s automated debt recovery scheme commonly referred to as ‘robo-debt’ as well as plans to use big data to identify and intervene in the lives of those it deems at risk of long-term unemployment or ‘welfare dependency’ (Carney, 2018). The latter was first developed and implemented in New Zealand. New Zealand has introduced an ‘investment approach’ to welfare management that uses data analytics to try and identify those it deems most likely to become dependent on benefits and intervene. Predictive analytics have also been introduced in areas of social care, taxation, health, justice and the administration of social support and benefits. These efforts have also come with controversy, an early example being the government’s plans to use predictive analytics in child welfare. After the predictive model was released to the public for review, it was criticized for its inaccuracies, and embedded biases that would disproportionately punish the poor (Gillingham and Graham, 2017; Keddell, 2015).

As with other nations, the Government of Canada is taking steps to advance its digital position and is under pressure to do so quickly (Clarke et al., 2017). In many ways this Government of Canada case study is representative of what is happening in other countries: (a) Government officials in Canada are experiencing internal and external pressures to make greater uses of big data; (b) civil servants are identifying a range of benefits, risks and challenges in making use of algorithmic practices and decision-making and (c) most of these new practices and the internal debates that accompany them remain hidden from public view.

Data governance and its implications

Previous research has drawn attention to how big data is changing the way governments research, prioritize and manage public services (Bertot et al., 2014; Bhushan, 2014; Margetts and Sutcliffe, 2013; Redden, 2015). These new systems are said to shift operations and understanding (Amoore and Piotukh, 2015; Bartlett and Tkacz 2017; Dunleavy, 2016; Mittelstadt and Floridi, 2016). Others note the importance of attending to the ‘mobilizing effects’ of big data systems and the way these systems ‘actualize and legitimize’ worlds and ways of being (Ruppert, 2016).

As detailed by Kitchin (2014a), big data signals the creation of a ‘fundamentally different epistemology’. Of concern, in terms of governance, is how uses of big data may profoundly change state–citizen relations and the way governments understand and respond to citizens. As governments make greater use of big data there is a range of real and potential implications we must be alert to:

Citizens become knowable, traceable and trackable across lifespans, social and professional networks, government interactions and geography in new ways as citizens are transformed into ‘data subjects’ (Ruppert, 2016)

The collection and maintenance of data about citizens is encouraged

Governments are compelled to link up their datasets, make use of external data points and share their data internally and externally

More services and decision-making processes can be automated and inscrutable

New state citizen power dynamics are created as citizens become infinitely ‘knowable’, but have little ability to interrogate how their data is being collected and used

Emphasis shifts from causation to correlation and from prevention to pre-emption, prediction and probability

More public–private partnerships and corporate involvement become encouraged, particularly with technology companies, in the management and running of public services.

In combination, these are profound changes in the ways democratic states learn about, engage with and respond to citizens and the information about them. Without adequate transparency, accountability, oversight and means for citizen intervention these systems could be used in ways that infringe upon citizen rights. Key here is the recognition that with big data adverse effects can be unintentional. At the moment citizens do not have the information or resources needed to meaningfully engage with these changes while they are happening.

Transparency and accountability

In an age of datafied governance, democracies require new systems to ensure transparency and accountability. A challenge is that transparency and accountability are relative concepts (Janssen and Van den Hoven, 2015) and will require broader public debate about what systems of transparency, accountability and intervention should look like (Fink, 2017). One of the major challenges with transparency, as argued by Ananny and Crawford (2016), is that embedded in the transparency ideal is the assumption that greater transparency will lead to greater civic involvement and response. The problem, as they note, is that too often in practice greater transparency simply means shifting responsibility for oversight and accountability to individuals already limited in time and resources. As detailed by Ananny and Crawford (2016: 5, citing Heald (2006) and Fox (2007)) transparency can ‘reveal and obscure’, it can come with ‘soft accountability’ or ‘hard accountability’, it can leave inside people and processes inscrutable, it can be event based instead of continual and it can be in retrospect instead of real time. A further challenge, as argued by McKelvey, is that in the age of big data, we must also think about transparency in relation to how we can communicate data-related changes ‘into knowledge that is conducive with democratic debate’ as well as the democratic responsibility to do so (McKelvey, 2014: 599).

Transparency must be coupled with accountability. O’Neill (2016) argues that accountability must extend across the life of projects. Also stressed is the importance of ensuring due process (Brauneis and Goodman, 2017; Campolo et al., 2017; Pasquale, 2015). Calls for solutions that go beyond techno-centrism are coming from inside and outside governments and those who argue that any system of accountability will need to address the multiple forms of power imbalance that exist in big data systems and their connections to our wider society: ‘Technical approaches that look for a one-time ‘fix’ for fairness risk oversimplifying the complexity of social systems’ (Campolo et al., 2017). In addition, accountability must include the option to stop data practices and/or identify no go areas where the risk is too great. Data governance is another area where it becomes clear that ‘tech justice’ and ‘social justice’ concerns must be combined (Dencik et al., 2016).

In practice the ability for citizens to engage with politicians and public servants about big data practices is limited because at the most basic level, no national government provides a map of where and how big data systems are being used and any internal debates are not made public. Given that we are in the midst of a data-driven social and political transformation, a shift so significant some are calling it a paradigm shift and argue that it is ushering in a new capitalist order (Couldry, 2018; Kitchin, 2014b), surely if ever there was a time for governments to share their internal debates and invite public comment and critique it is now.

Encouraging public debate and inviting dissent is also important because we know that large government IT projects can and do go wrong. Failure comes at great financial and often human cost. Some recent examples include the UK’s e-borders system development, which ended with £224m being owed to Raytheon. There was the £10bn NHS national programme for IT that was shelved (Syal, 2013). More recently, failures linked to the Universal credit programme have received widespread attention because of the negative impact on people’s lives (Omar et al., 2017). Other examples of failure include attempts to automate welfare services in the United States. In Indiana, Florida and Texas such attempts led to system collapse, high costs and devastated the lives of many (Eubanks, 2015). In Canada, the government has been criticized for its handling of efforts to consolidate and update computer and data systems (Bagnall, 2016). In surveying reviews of failures a common refrain is that when ‘modernizing’ IT systems and services failure often occurs when governments do not account for the complexity of the efforts they are undertaking.

Theoretical framework

The research approach for this study and the analysis conducted is informed by Critical Data Studies. Critical Data Studies is emerging as an interdisciplinary field of research investigating data production, organization, analysis and employment (Kitchin and Lauriault, 2014). As argued by Dalton and Thatcher, a critical study of ‘big data’ requires recognition that ‘big data’, both the staggering data points being collected and analysed as well as the methods used to conduct these analyses, is not neutral. Taking a critical approach to these practices means recognizing that ‘big data’ ‘always shapes and is shaped by a contested cultural landscape in both creation and interpretation’ (Dalton and Thatcher, 2014). Here the influence of critical theory is important as big data practices are viewed as not inevitable, but as context specific. Investigating data governance requires attention to changing government systems, processes and services, as well as developing a deeper understanding of big data as socio-technical systems comprising people, infrastructures and processes of sense-making (boyd and Crawford, 2012; Dalton and Thatcher, 2014; Kitchin, 2014b; Ruppert, 2012). Data and their use, as argued by Kitchin (2014b): ‘constitute in Foucault’s terms a form of power/knowledge; a means through which people, phenomena and territory can be surveyed and regulated’ (Kitchin, 2014b: 16). For this reason, attention to how data uses are socially shaped and shaping is crucial.

This study employs counter-mapping, a tool that Dalton and Thatcher (2014) pointed to in their initial call for critical approaches to big data. Mapping is often used in qualitative research as a means to organize, classify and make sense of information (Aurini et al., 2016). In referring to counter-mapping, Dalton and Thatcher are referring to a research practice used by critical geographers Harris and Hazen (2005). As argued by Harris and Hazen, counter-mapping can be used to overcome power hierarchies and as a means to pose alternative imaginaries that challenge or complement standard representations. While their work focuses on uses of counter-mapping to overcome biases in cartography, I view counter-mapping as a provocative research tool with more widespread applications. In this study counter-mapping is employed to map government big data practices and internal discussions of the risks and challenges that come with big data. The map of big data applications and discourses is ‘counter’ in the way that it differs from dominant outward facing government discourses about big data applications. These discourses often focus on principles and ethics and rarely dwell too long on concerns and challenges. In contrast, meaningful democratic engagement requires access to a range of positions and an informed citizenry. The mapping provided in this study aims to be ‘counter’ in its systematic approach to mapping discourses and practices and by adding specificity to what is known about internal big data debates.

Methodology

There are three research questions being addressed in this article. Research question one asks: Where and how is big data being integrated into government departments and agencies in Canada? Research question two asks: What are the challenges and risks being identified by civil servants? Addressing these research questions, in combination, provides a map of government practices and discourses rendering both more visible than they are at present. Research question three asks: What are the areas of silence in terms of government discourses, and how might this matter? This question employs previous research, largely from Critical Data Studies, to imagine how internal discourses could be otherwise by considering what is missing.

The methods used to address these research questions include freedom of information requests, semi-structured interviews and document analysis. Interviews were conducted with 23 people including civil servants, data analysts, consultants and those in the non-profit sector between 2013 and 2015. Details of the interviews are provided in Table 1 (online Supplemental material). A combined approach of cold calling based on job title, snowball sampling and approaching senior level civil servants proved somewhat successful in gaining access. Interviews ranged from 20 minutes to an hour. Most of the interviews were conducted by phone or Skype. Two interviewees opted to provide written responses to questions. Of the 23 people interviewed, 16 were public servants. Public servant interviewees included people in a range of positions from data scientists, project managers, senior bureaucrats to communications staff. The interview protocols were designed to draw out how data analytics are being used and what interviewees think about the benefits and challenges of using big data applications. Transcripts were produced for all interviews. Only the transcripts for interviews done with public servants were used in the analysis of internal debates about challenges and risks outlined below.

Table 1.

Document analysis questions.

1. How are departments or agencies using big data?

2. What are the promises, promising areas, being linked to big data?

3. What risks are being identified or linked to big data uses?

4. What challenges in using big data are mentioned?

5. Are discussions of the limits of big data approaches present?

It was difficult to find out about the specifics of big data applications from interviews. It also proved difficult to find publications outlining government big data applications, a problem that exists across nations. For these reasons freedom of information requests were used to collect internal documents and communications about big data practices. My use of freedom of information requests is in line with an increasing number of social science researchers who are using this method in their attempts to find out about government digital and data practices (Fink, 2017; Larson and Walby, 2011; Monaghan and Walby, 2012).

As detailed in Table 2 (see online Supplemental material), 17 Freedom of information requests were sent to 16 Government of Canada departments and agencies (two different requests were sent to the Privy Council Office). Given the high number of federal departments, agencies and crown corporations, a decision was made to narrow the request by focusing on those departments and agencies interviewees and news accounts had identified as making use of big data and to also focus on applications in human services versus the sciences. The requests were all sent in July 2016. At the date of writing responses had been received from 13 of the 17 requests, four government bodies requested extensions. The text for most of the requests involved asking for: ‘Any reports, briefing notes, papers or summaries about any big data or data analytics projects or pilot projects operating within (department name) between Jan. 2014 to June 2016.’ An additional request to the Privy Council Office specifically requested any reports or documents related to the Deputy Minister’s Committee on Policy Innovation study of big data in government. Further, the Government of Canada enables citizens to search previously completed freedom of information requests and ask for copies. Through this method a copy of a response from the Privacy Commissioner’s Office was also obtained. In addition, a copy of a Canada Revenue Agency report provided to the CBC was published and publicly available online. Titles of the documents provided as part of the responses to the freedom of information requests are provided in Tables 3 and 4 (see online Supplemental material).

In combination the transcripts of interviews with public servants, departmental written responses and documents received through FOIs are used to gather facts about how different departments and agencies are investigating and applying big data applications. Document Analysis is a useful method to supplement interview data, as it provides an indication of what is being presented as important and reveals areas of silence (Bowen, 2009). Document and transcript analysis involved an initial superficial examination to gain familiarity with the content. A list of five key questions were identified and used as coding categories to sort and organize content. Any content identified that addressed the following questions was selected and grouped together (see Table 1).

A second coder was employed to go through the documents and categorize content to ensure no relevant content was overlooked.

This article focuses on the risks, challenges and limits that come with big data. The risks, challenges and limits identified in the documents and transcripts were placed into categories as detailed in Table 7 (see online Supplemental material). The table categorizes content by challenge or risk mentioned, but also indicates the source of the reference as well as discursive examples. The coding categories for challenges and risks were derived from previous research which identified the following challenges and risks: privacy and security, skills needed, infrastructure and access, lack of legislative and policy framework, data quality and inaccuracy and ethics. Categories were also derived from work in Critical Data Studies, these included: discrimination, citizen and consumer power, citizen data literacy and knowledge. An initial piloting of these categories for coding led to the identification of four further categories: culture, missed opportunity, replacement of workers and ensuring human influence. Although the latter two categories involved only one mention each. The selections were then analysed and compared to identify patterns, central themes, differences and areas of silence.

Before moving to the findings it is important to recognize that the documents that make up my sample were produced for various purposes and at various times. In total, transcripts from 16 interviews with public servants were analysed as well as 1468 pages of documents produced by 12 Departments and Agencies. These documents as well as the interviews provide an indication of the kinds of discourses present about government big data uses. The material collected and analysed here must be recognized as a snapshot in time. The data remains relevant as ‘situated products’ which provide an opportunity to ‘listen in’ on internal discussions about big data applications. They are valuable as they provide an opportunity to learn from civil servants and others involved in the integration of big data practices. However, the fact that they are dated speaks to the limitations of Freedom of Information requests as a method of investigation (Brauneis and Goodman, 2017; Fink, 2017).

Government of Canada and big data

Interviewees note that Canada is still at an early stage of big data integration. Two public servants argued that generally the departments and agencies that are ‘data rich’ and have a history of working with data have been the fastest to incorporate big data approaches (PS L and M, 2015). The examples in Table 6 (see online Supplemental material) show big data applications across government and build upon an earlier published survey (Redden, 2015). Not detailed here are Government of Canada efforts in relation to open data or how Canadian intelligence agencies, the Communications Security Establishment and the Canadian Security Intelligence Service, are using big data. It is widely recognized that uses of big data among Canadian intelligence agencies are much more advanced than the rest of government (Deibert, 2014). This is connected to the long history of surveillance practices in Canada and the legislative powers that enable greater access and uses of data (Whitaker et al., 2012).

The overview in Table 6 (online Supplemental material) provides a running record of where and how big data is being used but provides only a superficial and initial mapping. Better understanding of these applications in practice requires more investigation of big data uses as part of socio-technical systems dependent on contexts (Kennedy, 2016).

Benefits and promise

General and department-specific areas of promise and benefits are associated with big data applications and identified across interviews and documents. These include the ability to accelerate research, customize and improve programme and service delivery and development; better evaluate the success of programmes; strengthen enforcement and compliance efforts; prevent crime; save money; promote health; better manage agricultural and natural resources; develop new treatments for diseases; better gauge reactions and adjust communications to stakeholders and the public; improve relations with stakeholders and the public; generate efficiencies in internal operations and enhance internal management; improve safety and security outcomes; personalize services; create customized campaigns; improve performance and productivity; and create wealth for shareholders and stakeholders; improve data (CH, 2016: 189; Justice, 2016; PCO, 2016; PS, 2016; SC, 2016: 150–152; SSC, 2016). For example, civil servants at Global Affairs note that through uses of big data there is the potential to better identify private sector or consular needs, strengthen external communications, better understand and predict ‘external realities such as multilateral and bilateral negotiation contexts and crises, market trends, barriers and opportunities and development needs/impacts’ (GA, 2017: 1). Some of those who work with data analytics note its limitations and stress that it should only be used as one line of evidence among many (PS L, M, N, 2015; see Table 1 in Supplemental material).

This study focuses on the challenges and risks that come with big data systems. Throughout the documents risks and challenges associated with big data applications are raised. Most often concerns are dismissed and not discussed in detail. The exceptions to this are the Department of Justice which is in the process of developing plans to make greater uses of big data and algorithmic systems, Statistics Canada and the Department of Global Affairs who have been extensively investigating and using big data applications. The freedom of information requests received from each of these organizations provide lengthier discussions about the limits and risks of big data.

Risks and challenges

Given that more than 1400 pages of documents were analysed, the number of times concerns about big data risks and challenges are raised is limited. A detailed breakdown of concerns and challenges identified and a representative listing of how they appeared in discourse can be found in Table 7 (online Supplemental material). Although limited in terms of space, the concerns mentioned are wide ranging, emerge from across departments and indicate ongoing internal debate. It is likely that a similar set of FOI requests submitted now would reveal even more concern than detailed between 2014 and 2016, as there has been more public debate on the topic.

As indicated in Figure 1, concerns about the need for improved infrastructure and greater access to datasets in order to enable big data applications are mentioned most at 67 times. Indicating that internally this is represented as the most pressing concern. This issue is raised by nine departments and across interviews. Concerns about this are mentioned less by data-rich departments like the Canada Revenue Agency and agencies like Statistics Canada. The latter focuses more attention to issues of data quality and accuracy. The Department of Justice raises most concerns about infrastructure and access. This is heavily dependent on context. In 2015 the Department hired a boutique consultancy firm to investigate emerging trends and identify options for the Department. Ultimately, the report supported greater big data use, arguing that ‘law firms that fail to plan for the implementation of new technologies will find themselves at a significant competitive and cost-effective disadvantage’ (Justice p. 150). The Department is considering using data-driven applications to (a) improve operational efficiency by better measurement of performance and compliance, (b) inform policy by helping to predict environmental trends and (c) to manage risks throughout trials by using artificial intelligence and data analytics to aid in discovery and predict case outcomes (Justice p. 257). The challenge identified across Justice documents, as with those of other Departments, is that there is a ‘complex matrix of legislation and practices that surround the use of personal information’ (Justice p. 11). It is noted that Justice will require investment and restructuring to make it easier to share and access files. While the documents clearly indicate a Department that feels compelled to make use of big data or fall behind, there is reference to a wide range of privacy and security concerns that would come with such a move. Much discussed is the challenge of anonymization in practice and the ongoing threats of hacking and data breaches. Justice, like Canadian Heritage, the Privy Council, Public Safety and the Treasury Board Secretariat are concerned about public opinion and recognize that citizens have a ‘growing sense of unease’ about the ability to control their information (Canadian Heritage, p. 1). Across Departments concern is raised about the need to avoid ‘creepiness’ in the customization of services as well as the potential to increase negative public opinion if things go wrong and undermine public confidence in government (Justice p. 10; Privy Council p. 15). Given this it is somewhat surprising that there is so little attention devoted to detailing how the public might be more meaningfully informed and engaged about big data, algorithmic governance and artificial intelligence applications.

Figure 1.

Risks and challenges mentioned. The numbers indicate the number of mentions across documents and interviews.

There are some indications we might expect greater government transparency about its data practices in the future. The government recently invited public critique of its draft white paper on ‘Responsible AI in the Government of Canada’ and its draft impact assessment tool (Karlin, 2018; TBS, 2018). The government has also posted a prototype of its Algorithmic Impact Assessment Tool. Karlin and Corriveau (2018) have suggested assessments produced by the tool may be public. It remains to be seen if and how this kind of transparency will work in practice and if it will be extended to ongoing big data applications.

There is seemingly broad agreement among Deputy Ministers that for government to make more and better use of big data it will need to review privacy frameworks and barriers to data sharing. How data sharing is done, with what data and with who should be the subjects of widespread public debate. Here the trials and errors of Europe’s General Data Protection Regulation efforts could be useful to fuel transnational civic engagement on these subjects. In summary, the above listed challenges and risks are the most identified across Government of Canada documents and are also some of the most often cited concerns in other nations, such as the UK (Kim et al., 2014; Malomo and Sena, 2017; STC, 2016).

Accuracy

Civil servants are raising concerns about how big data practices can provide results that are inaccurate. Accuracy and data quality are most discussed by Statistics Canada and Global Affairs in reference to their own trials with big data. These issues are also referenced six times across Canadian Heritage documents. Global Affairs notes that analytical programmers and those in charge of data collection make choices about sources, and also that the data collected may be ‘products of an inferential procedure of some kind’. The latter can introduce bias and white noise. Global Affairs draws attention to the obvious but seldom mentioned point that big data leaves out ‘non-digital and non-digitizable forms of information’. Further, big data is powerful when it comes to making correlations but not at inferring causality (GA, 2017: 4). Both Global Affairs and Statistics Canada highlight that with big data one must account for the fact that the data points being used were ‘not developed for the purposes to which they are being put’ (GA, p. 8). As argued by Statistics Canada: ‘problems surrounding selectivity and representativeness are more common when dealing with big data than more traditional sources of data and require special attention’ (SC, p. 111). These and other methodological issues that come with using big data lead Statistics Canada to conclude in one report that veracity is the most important V in assessing big data’s potential (SC, p. 111). All of the above are obvious points to data scientists, less so for others. These points counter some of the myths about the accuracy, objectivity and neutrality of big data approaches.

Organizational culture

There were some surprising findings to emerge from the analysis. The number of references to the need to for changes in ‘mind-set and culture’ is perhaps not surprising given the dominance of this narrative in corporate literature. What is surprising is how often this narrative is repeated within the public service, how similar it is to the language in technology circles and how uncontested this narrative is. At 20 mentions across documents and interviews this is the fifth most mentioned concern, ahead of concerns raised about changing power dynamics, discrimination, data literacy and ethics.

I suggest that this ‘cultural barriers’ narrative requires interrogation. The narrative is that there is a culture of resistance within government to big data and that a change in culture is needed for civil servants to embrace big data approaches. For example, an interdepartmental steering committee report argues that ‘For big data to be operative, it requires organizations to change their culture to be more open to analytics, data sharing and evidence-based decision making.’ Another version of the same report argues that ‘Organizational culture is powerful and ubiquitous. Even when official policies and legislation allow data sharing, risk aversion can impede this practice’ (SC, 2016: 159). An IBM blog promoting this narrative is cited in a Canadian Heritage document (CH, 2016: 204; Helms, 2015).

There is little doubt that uses of big data challenge traditional hierarchies of knowledge and power, and that new information sources and practices can be disruptive. However, dismissing the concerns and resistance raised by civil servants about greater data sharing and the challenges they see in using big data applications make it harder to learn from the critiques they are raising. Only Global Affairs followed suggestions that civil servants may be reluctant to share data with the note that there are ‘very real concerns to take into account such as privacy (including legal restrictions) in regard to the use of some databases’ (GA p. 18). The dismissal and simplification of internal reluctance to share data and make greater use of big data applications as a ‘culture clash’ is a problem. This narrative may be used to pre-emptively silence those within the public service who might raise legitimate challenges to changing practices, or pre-emptively influence those in management positions to misidentify such challenges as ‘cultural resistance’.

Public–private partnerships

The infrastructure, data science skills and access issues detailed above as well as a desire to reduce costs, increase efficiency and improve services are motivating government bodies to develop public–private partnerships in this area. Across the documents there are references to partnerships with a range of bodies. A 2015 Industry Canada presentation states that IBM and SAS offer the most advanced off-the-shelf software to analyse numbers and text and that software by both is being used by a number of departments. The Department of Justice hired a consultant to advise them about big data changes. Shared Services Canada notes meeting with industry to discuss architecture considerations. In another instance SSC refers to itself as a ‘cloud broker’, likely a reference to the work it has been doing to find companies to provide public cloud services (GoC, 2016). Public Safety has worked with external research organizations such as SecDev and Demos (PS, 2016: 9). Global Affairs references using third-party programmes such as those developed by SecDev or Recorded Future.

Technology companies are changing their strategy to be more involved in decision-making. These companies now see themselves as providing more than technical tools and instruction, but also helping to identify and solve social problems. For example, companies like IBM, Cisco, Microsoft and SAS have been reinventing themselves and their business strategy. All see data as key to their business futures. In its 2016 annual report, IBM writes that it has re-strategized and is now ‘much more than a hardware, software and services company; IBM is a cognitive solutions and cloud platform company, with a focus on industry capabilities and expertise’. IBM and others are shifting their focus to providing ‘cognitive solutions’ and intelligent clouds in an ‘era of Cognitive Government’. For example, companies like SAS are promoting their ability to assist in areas like security, criminal justice, healthcare, local government, child welfare, etc. As companies move into the area of problem solving and work more closely with governments in identifying and addressing social problems there are a range of important issues requiring civic engagement. As noted by Garrido et al. (2018) these include the extent to which government and business spheres become even more intertwined, growing private sector control over government data and services, how governance may be co-opted and shaped by private interests, how governments may become ‘locked-in’ the more they become reliant upon contracted technology providers at the expense of developing their own in-house expertise and the extent to which these contracts render some aspects of public–private operations inscrutable to the public (Kitchin, 2014b). The lack of concern raised about public–private partnerships across documents is glaring, only one interviewee expressed concerns about this area (PSC, 2015; Table 1 online Supplemental material).

Fairness, inequality and discrimination

Concerns about the discriminatory potentials of big data approaches are not raised often, but they are raised. These concerns are raised most by the Office of the Privacy Commissioner. Concerns are mentioned briefly in a Steering Committee report (TBS, 2016: 124). Concerns are also raised in a Canadian Heritage overview document (CH, 2016: 79). A Department of Justice report notes:

There are also ethical and moral questions about how Big Data might be used by government, or disclosed to others for possible misuse. There is a difference between government predicting and disclosing broad statistics about crime and cancer rates on a macro scale and using the data to focus on individuals. The more granular the information becomes, the more organizations might be tempted to use the information in negative ways. (Justice, 2016: 39)

There is brief reference within the Justice document to the ‘potential to profile, target or discriminate vulnerable people or groups’ through matching open data sources with private sources of data (Justice, 2016: 195).

Previous work on uses of big data in the public sector shows that things can and do go wrong affecting people’s employment, ability to travel and access to benefits (Eubanks, 2015; Hu, 2015). Research into predictive policing and sentencing shows bias as well as an over-monitoring and criminalization of the poor and ethnic minorities (Angwin et al., 2016; Starr, 2016). Previous work in New Zealand and Australia has raised concerns about how big data applications in public services can exacerbate inequality, punish the poor, be dehumanizing and prone to error (Chapple, 2013; Gillingham and Graham, 2017; Keddell, 2015). Given the widespread examples of data harms (Redden and Brand, 2017), going forward government administrations could gain much by inviting scrutiny and building greater room for interventions into their algorithmic systems from the beginning, particularly by encouraging interventions from those with direct experiences and knowledge of discriminatory practices.

Changing information systems and their implications

Across the interviews and documents there is relatively little discussion of the epistemological implications of changing information systems and how computer systems, and now big data systems, change the kinds of information known about people and therefore influence the kinds of decisions that are made about what should be done about social problems. The concern came up most in Statistics Canada documents. Statistics Canada argues that big data should be used as a complementary source of information and points to the need for further research into the larger implications of data quality and accuracy issues.

Added to questions about the knowledge value of big data-driven approaches are concerns about potentials to change ‘systems of knowledge’ which leads to the production of ‘particular forms of information and computational knowledge’ (Berry, 2011; boyd and Crawford, 2012). Previous work investigating the computerization of social services demonstrates the profound effect changes in technology can have on working practices, relationships, the kinds of information valued and what is ‘knowable’. (Gilllingham, 2011; Munroe, 2010; White et al., 2009). Studies of the data dashboards being introduced across public and private sectors details how these dashboards can both inform and mislead. Dashboards present a myth of objective and neutral data when in reality they present condensed data; tell stories through design and visualization; and obfuscate the weaknesses, assumptions and biases embedded in the data (Bartlett and Tkacz, 2017: 15; Kitchin et al., 2015). Further, big data systems, and their mainstay which includes predictive modelling and risk scores, feed into an ongoing neoliberal paradigm which privileges target cultures, risk assessments, marketization and financialization. What can get lost in this perfect storm of numerical rationalization is recognition of the messiness of real lives and the complex historical, social and economic factors that lead to social problems.

Conclusion

This study provides a partial map of where and how big data is being used across Government of Canada departments and agencies as well as of the kinds of discourses about risks and challenges circulating internally. In the absence of an official map, the research process has involved counter-mapping in order to render big data practices and internal debates more visible. The research provides insight into the kinds of big data and algorithmic applications being applied across government and also specific examples of the concerns being raised internally by those trying to make use of algorithmically driven systems. The risks and challenges most mentioned in government discourses are the need for better infrastructure, the difficulty accessing data, concerns about privacy and security, concerns about poor data quality, inaccuracy and recognition of the need for greater data analytics skills within departments and agencies. These concerns have been identified in other national contexts in previous research (Kim et al., 2014; Singh et al., 2014). The analysis identified other concerns seldom discussed in previous research. These relate to the problematic debates around culture and concerns about changing power dynamics. Some concerns voiced internally echo those being raised outside of government such as potentials for discrimination, missed opportunities, ethics, citizen data literacy, the need for legislative and policy frameworks, the need to ensure human influence in changing practices and how data systems may replace workers. The analysis identifies two significant areas of silence: (a) how datafied public–private partnerships may limit democratic systems and change the way problems are identified or responded to and (b) how changing information systems may change the way citizens and societies are understood and governed.

Overall, through detailing the ongoing and varied internal discourses about new and emerging data practices across government the article underlines the value of counter-mapping as a method, the need for an official map of big data practices and data sharing as a matter of government responsibility, and also the need for our democratic systems to adjust to protect citizen rights. The shift to datafied societies comes with considerable risk, risks that are recognized by those inside and outside of government (Justice, 2016; PS, 2016; Veale et al., 2018). This analysis demonstrates that risks are greater and more varied than often discussed. Enabling greater citizen involvement in changing practices will require more than basic levels of transparency, it will require that our democratic institutions change to respond to our current context by developing a means to make black-boxed processes and the implications of changing systems open to wider debate and intervention. The responsibility for oversight cannot fall solely on individuals given the limited time and resources of most people. There are a range of ideas being put forward to enhance accountability and oversight. For example, some suggest a national algorithm safety board (Schneiderman 2016), increased power for privacy commissioners or systematizing community oversight. An initial step towards greater transparency would be the production of maps detailing where and how big data applications are being used and where data sharing is taking place.

This study provides a broad overview, but depth is sacrificed for scope. Necessary are investigations of how big data is being integrated into government practices by looking at applications within their contexts. Such an approach would enable greater opportunity to learn about how data risks and challenges are being addressed in practice, where and how fairness is being incorporated into the stages of big data systems and the abilities for internal and external actors to intervene (Couldry and Powell, 2014; Gürses and van Hoboken, 2017).

Supplemental Material

Supplemental material for Democratic governance in an age of datafication: Lessons from mapping government discourses and practices

Supplemental Material for Democratic governance in an age of datafication: Lessons from mapping government discourses and practices by Joanna Redden in Big Data & Society

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research for this article was funded by the Social Sciences and Humanities Research Council of Canada (SSHRC).

References

Amoore

Piotukh

(2015) Life beyond big data governing with little analytics. Economy and Society 44: 341–366.

Ananny M and Crawford K (2018) Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society 20(3): 973–989.

Angwin J, Larson J, Mattu S, et al. (2016) Machine bias. ProPublica, 23 May. Available at: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing (accessed 2 September 2016).

Aurini

Heath

Howells

(2016) The How to of Qualitative Research, London: Sage.

Bagnall J (2016) Build to fail: Politics of sabotaged shared services before the Department got off the ground. Ottawa Citizen, 11 November. Available at: http://ottawacitizen.com/news/national/built-to-crash-the-ugly-sputtering-beginning-of-shared-services-and-how-politics-conspired-against-it (accessed 3 September 2017).

Barocas

Selbst

(2016) Big data’s disparate impact. California Law Review 104: 671–732.

Bartlett J and Tkacz N (2017) Governance by dashboard. Demos, April. Available at: https://www.demos.co.uk/wp-content/uploads/2017/04/Demos-Governance-by-Dashboard.pdf (accessed 1 October 2017).

Berry

(2011) The computational turn: Thinking about the digital humanities. Culture Machine 12: 1–22.

Bertot

Gorham

Jaeger

et al. (2014) Big data: Open government and e-government: Issues, policies and recommendations. Information Polity 19: 5–16.

10.

Bhushan

(2014) Fast data, slow policy: Making the most of disruptive innovation. SAIS Review of International Affairs 34(1): 93–107.

11.

Bowen

(2009) Document analysis as a qualitative research method. Qualitative Research Journal 9(2): 27–40.

12.

boyd

Crawford

(2012) Critical questions for big data. Information, Communication & Society 15(5): 662–679.

13.

Brauneis R and Goodman E (2017) Algorithmic transparency for the smart city. Yale Journal of Law & Technology. GWU Law School Public Law Research Paper. Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3012499 (accessed 18 October 2017).

14.

Bright J, Margetts H, Hale S, et al. (2014) The use of social media for research and analysis: A feasibility study. Department for Work & Pensions. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/387591/use-of-social-media-for-research-and-analysis.pdf (accessed 7 November 2016).

15.

Campolo A, Sanfilippo M, Whittaker M, et al (2017) AI Now 2017 report. AI Now. Available at: https://assets.contentful.com/8wprhhvnpfc0/1A9c3ZTCZa2KEYM64Wsc2a/8636557c5fb14f2b74b2be64c3ce0c78/_AI_Now_Institute_2017_Report_.pdf (accessed 18 October 2017).

16.

Carney T (2018) Robo-debt illegality: A failure of rule of law protections. Australian Public Law, 30 April. Available at: https://auspublaw.org/2018/04/robo-debt-illegality/ (accessed 4 September 2018).

17.

Chapple

(2013) Forward liability and welfare reform in New Zealand. Policy Quarterly 9(2): 56–62.

18.

Citron

Pasquale

(2014) The scored society: Due process for automated predictions. Washington Law Review 89: 1–33.

19.

Clarke

Lindquist

Roy

(2017) Understanding governance in the digital era: An agenda for public administration research in Canada. Canadian Public Administration 60(4): 457–475.

20.

Clarke

(2016) Big data, big risks. Information Systems Journal 26(1): 77–90.

21.

Couldry N (2018) Colonized by Data: The Capitalization of Life Without Limits. JOMEC Seminar Series. Cardiff: Cardiff University.

22.

Couldry

Powell

(2014) Big data from the bottom up. Big Data & Society 1(2): 1–5.

23.

Dalton C and Thatcher J (2014) What does a critical data studies look like, and why Do we care? Seven points for a critical approach to ‘big data’. Space and Society Open Site. Available at: http://societyandspace.org/2015/05/12/what-does-a-critical-data-studies-look-like-and-why-do-we-care-craig-dalton-and-jim-thatcher/ (accessed 5 September 2015).

24.

Deibert

(2014) Black Code: Inside the Battle for Cyberspace, Toronto: McClelland and Stewart.

25.

Dencik

Hintz

Cable

(2016) Towards data justice? The ambiguity of anti-surveillance resistance in political activism. Big Data & Society 3(2): 1–12.

26.

Dencik L, Hintz A, Carey Z, et al. (2015) Managing ‘threats’: Uses of social media for policing domestic extremism and disorder in the UK. Cardiff School of Journalism, Media and Cultural Studies, October. Available at: http://www.dcssproject.net/files/2015/12/Managing-Threats- Project-Report.pdf (accessed 7 November 2015).

27.

Department of Finance and Deregulation (2013) The Australian Public Service Big Data Strategy, Canberra: Australian Government.

28.

Drew

(2016) Data science ethics in government. Philosophical Transactions of the Royal Society 374: 2083.

29.

Dunleavy

(2016) Big data and policy learning. In: Stoker

Evans

(eds) Evidence-Based Policy Making in the Social Sciences: Methods That Matter, Bristol: Policy Press, pp. 143–168.

30.

Eubanks V (2015) Want to cut welfare? There’s an App for that. Slate, 27 May. Available at: https://www.thenation.com/article/want-cut-welfare-theres-app/ (accessed 3 September 2016).

31.

Eubanks

(2018) Automating Inequality, New York: Macmillan.

32.

Fink K (2017) Opening the government’s black boxes: Freedom of information and algorithmic accountability. Information, Communication & Society. Epub ahead of print 20 May 2017. DOI: 10.1080/1369118X.2017.1330418.

33.

Gamage

(2016) New development: Leveraging ‘big data’ analytics in the public sector. Public Money & Management 36(5): 385–390.

34.

Gangadharan SP, Eubanks V and Barocas S (2014) Data and discrimination: Collected essays. Open Technology Institute and New America. Available at: https://www.ftc.gov/system/files/documents/public_comments/2014/10/ 00078-92938.pdf (accessed 9 September 2015).

35.

Garrido S, Allard MC, Béland J, et al. (2018) IoT in the smart city: Ethical issues and social acceptability, Montreal: CIRAIG, February. Available at: http://ville.montreal.qc.ca/pls/portal/docs/page/prt_vdm_fr/media/documents/ido_vi_revue_litt_final_en.pdf (accessed 2 September 2018).

36.

Gillingham

(2011) Computer-based information systems and human service organisations: Emerging problems and future possibilities. Australian Social Work 64(3): 299–312.

37.

Gilllingham

Graham

(2017) Big data in social welfare: The development of a critical perspective on social work’s latest electronic turn. Australian Social Work 70(2): 135–147.

38.

Gürses S and van Hoboken J (2017) Privacy after the agile turn. Open Society Foundation. Available at: https://osf.io/ufdvb/ (accessed 10 July 2018).

39.

Hancock M (2016) Data science ethical framework. Cabinet Office. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/524298/Data_science_ethics_framework_v1.0_for_publication__1_.pdf (accessed 8 February 2017).

40.

Harris

Hazen

(2005) Power of maps: (Counter) mapping for conservation. ACME: An International Journal for Critical Geographies 4(1): 99–130.

41.

Helms J (2015) Five myths and five ways to create an analytics culture. IBM Center for The Business of Government, 16 March. Available at: http://www.businessofgovernment.org/blog/business-government/five-myths-and-five-ways-create-analytics-culture (accessed 9 August 2017).

42.

(2015) Big data blacklisting. Florida Law Review 67: 1735–1809.

43.

Janssen

van den Hoven

(2015) Big and open linked data (BOLD) in government: A challenge to transparency and privacy. Government Information Quarterly 32: 363–368.

44.

Karlin M (2018) A Canadian algorithmic impact assessment. Medium, 18 March. Available at: https://medium.com/@supergovernance/a-canadian-algorithmic-impact-assessment-128a2b2e7f85 (accessed 5 September 2018).

45.

Karlin M and Corriveau M (2018) The Government of Canada’s algorithmic impact assessment tool: Take two. Medium, 8 August. Available at: https://medium.com/@supergovernance/the-government-of-canadas-algorithmic-impact-assessment-take-two-8a22a87acf6f (accessed 5 September 2018).

46.

Keddell

(2015) The ethics of predictive risk modelling in the Aotearoa/New Zealand child welfare context: Child abuse prevention or neo-liberal tool? Critical Social Policy 35(1): 69–88.

47.

Kennedy

(2016) Post, Mine, Repeat: Social Media Data Mining Becomes Ordinary, Basingstoke: Palgrave Macmillan.

48.

Kim

Trimi

Chung

(2014) Big-data applications in the government sector. Communications of the ACM 57(3): 78–85.

49.

Kitchin

(2014a) Big Data, new epistemologies and paradigm shifts. Big Data & Society.1–12.

50.

Kitchin

(2014b) The Data Revolution, London: Sage.

51.

Kitchin R and Lauriault TP (2014) Towards critical data studies: Charting and unpacking data assemblages and their work. The Programmable City Working Paper 2. Programmable City, Social Science Research Network. Available at: http://eprints.maynoothuniversity.ie/5683/ (accessed 5 September 2018).

52.

Kitchin R, Lauriault TP and McArdle G (2015) Urban indicators and dashboards: epistemology, contradictions and power/knowledge. Regional Studies, Regional Science 2(1): 43–45.

53.

Larson M and Walby K (eds) (2011) Brokering Access: Power, Politics and Freedom of Information Process in Canada. Toronto: University of British Columbia Press.

54.

Maciejewski

(2016) To do more, better, faster and more cheaply: Using big data in public administration. International Review of Administrative Sciences 83(1): 120–135.

55.

McKelvey

(2014) Algorithmic media need democratic methods: Why publics matter. Canadian Journal of Communication 39: 597–613.

56.

Malomo

Sena

(2017) Data intelligence for local government? Assessing the benefits and barriers to use of big data in the public sector. Policy and Internet 9(1): 7–27.

57.

Margetts

Sutcliffe

(2013) Special issue: Potentials and challenges of big data. Policy & Internet 5(2): 139–146.

58.

Mergel

Rethemeyer

Isett

(2016) Big data in public affairs. Public Administration Review 76(6): 928–937.

59.

Mittelstadt

Floridi

(2016) The ethics of big data: Current and foreseeable issues in biomedical contexts. Law, Governance and Technology Series 29: 445–480.

60.

Monaghan

Walby

(2012) Making up ‘terror identities’: Security, intelligence, Canada’s integrated threat assessment centre and social movement suppression. Policing and Society: An International Journal of Research and Policy 22(2): 133–151.

61.

Muñoz C, Smith M and Patil DJ (2016) Big data: A report on algorithmic systems, opportunity, and Civil Rights. Executive Office of the President, May. Available at: https://www.whitehouse.gov/sites/default/files/microsites/ostp/2016_0504_data_discrimination.pdf (accessed 3 September 2017).

62.

Munroe E (2010) The Munro review of child protection. Part one: A systems analysis. Department for Education, 1 October. Available at: https://www.gov.uk/government/publications/munro-review-of-child-protection-part-1-a-systems-analysis (accessed 10 December 2016).

63.

Omar

Weerakkody

Sivarajah

(2017) Digitally enabled service transformation in UK public sector: A case analysis of universal credit. International Journal of Information Management 37(4): 350–356.

64.

O’Neill

(2016) Weapons of Math Destruction, New York: Crown.

65.

Pasquale

(2015) The Black Box Society: The Secret Algorithms That Control Money and Information, Boston, MA: Harvard University Press.

66.

Podesta J, Pritzker P, Moniz EJ, et al. (2014) Big Data: Seizing Opportunities, Preserving Values. Executive Office of the President. Washington, DC: The White House. Available at: http://www.whitehouse.gov/sites/default/files/docs/big_data_privacy_report_may_1_2014.pdf (accessed 10 June 2014).

67.

Ramirez E, Brill J, Ohlhausen MK, et al. (2014) Data brokers: A call for transparency and accountability. Federal Trade Commission, May. Available at: https://www.ftc.gov/system/files/documents/reports/data-brokers-call-transparency-accountability-report-federal-trade-commission-may-2014/140527databrokerreport.pdf (accessed 2 January 2016).

68.

Redden

(2015) Big data as system of knowledge: Investigating Canadian governance. In: Langlois

et al. (eds) Compromised Data: From Social Media to Big Data, New York: Bloomsbury, pp. 17–39.

69.

Redden J and Brand J (2017) Exploring social justice in an age of datafication. Data Justice Lab, December 7. Available at: https://datajusticelab.org/data-harm-record/ (accessed 2 September 2018).

70.

Ruppert E (2012) The governmental topologies of database devices. Theory, Culture and Society 29: 116–136.

71.

Ruppert

(2016) Big data economies and ecologies. In: Ryan

McKie

(eds) An End to the Crisis of Empirical Sociology? Trends and Challenges in Social Science Research, London: Routledge, pp. 13–29.

72.

Singh

Srivastava

Johri

(2014) Big data and the opportunities and challenges for government agencies. International Journal of Computer Science and Information Technologies 5(4): 5821–5824.

73.

Science and Technology Committee (2016) The big data dilemma. 10 February 2016, HC 468, 2015–16. Available at: http://www.publications.parliament.uk/pa/ cm201516/cmselect/ (accessed 10 November 2016).

74.

Schneiderman B (2016) Opinion: the dangers of faulty, biased or malicious algorithms requires independent oversight. PNAS 113(48): 13538–13540.

75.

Starr

(2016) The odds of justice: Actuarial risk prediction and the criminal justice system. Chance 29(1): 49–51.

76.

Syal R (2013) Abandoned NHS IT system has cost £10 bn so far. Guardian, 18 September. Available at: https://www.theguardian.com/society/2013/sep/18/nhs-records-system-10bn (accessed 9 July 2017).

77.

Symons T (2016) Datavores of local government. Nesta discussion paper, July. Available at: https://www.nesta.org.uk/sites/default/files/local_datavores_discussion_paper-july-2016.pdf (accessed 2 November 2016).

78.

Veale M, Kleek MV and Binns R (2018) Fairness and accountability design needs for algorithmic support in high-stakes public sector decision-making. Available at: https://arxiv.org/abs/1802.01029 (accessed 2 September 2018).

79.

Whitaker

Kealey

Parnaby

(2012) Secret Service: Political Policing in Canada From the Fenians to Fortress America, Toronto: University of Toronto Press.

80.

White

Broadhurst

Wastell

et al. (2009) Whither practice-near research in the modernization programme? Policy blunders in children’s services. Journal of Social Work Practice 23(4): 401–411.

81.

Wood

Wright

(2015) Before and after Snowden. Surveillance & Society 13(2): 132–138.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.55 MB