Abstract
Decision support systems, which incorporate artificial intelligence and big data, are receiving significant attention in the public sector. Decision support systems are sociocultural artefacts that are subject to a mix of technical and political choices, and critical investigation of these choices and the rationales they reflect are paramount since they are inscribed into and may cause harm, violate fundamental rights and reproduce negative social patterns. Applying and merging the concepts of sense-making and translation, this article investigates the rationales, translations and critical reflections that shape the development of a decision support system to support social workers assessing referrals concerning child neglect. It presents findings from a qualitative case study conducted in 2019–2020 at the Citizen Centre Children and Young People, Copenhagen Municipality, Denmark. The analysis shows how key actors through processes of translation construct, negotiate and readjust problem definitions, roles, interests, responsibilities and ideas of ambiguity and accountability. Although technological solutionism is present in these processes, it is not the only rationale invested. Rather, technological and data-driven rationales are adjusted to and merged with rationales of efficiency, return on investment and child welfare. Through continuous renegotiation of roles, responsibilities and problems according to these rationales, the key actors attempt to orchestrate ways of managing the complexity facing child welfare services by projecting images of future potentials of the decision support system that are yet to be realised.
Keywords
Introduction
Decision support systems (DSS in singular/DSSs in plural) are not new to social work with children. In the 1980s, Schoech and Schkade (1980) argued for a computer-based processing application designed to help child welfare professionals make complex decisions. Simultaneously, early expert systems that used artificial intelligence (AI) to identify professionals’ knowledge base and rationalise decision-making were developed in child welfare services (Kirk and Reid, 2002). Developments in computing power, machine learning technologies and the accumulation of digital data have spurred new and increased interest in the use of DSSs in the public sector (European Commission, 2019). In this article, we define DSSs as systems that incorporate algorithms and/or neural networks (AI) that have been trained and tested on large data sets to calculate and produce an output concerning the likelihood of a particular outcome in order to support professional decision-making. This may also be referred to as ‘predictive analytics’ (Gillingham, 2019). DSSs divert from automated decision systems insofar as they ‘only’ provide information supporting human decision-making. DSSs have been developed and implemented in child protection services in several countries including the USA, the UK, New Zealand, Australia, the Netherlands, Norway and Denmark (Jørgensen et al., 2021). In the USA, for example, child protection authorities in more than a dozen states use or are developing predictive analytics, among which the Allegheny Family Screening Tool is perhaps the best-known (Glaberson, 2019).
DSSs are sociocultural artefacts that are subject to a mix of technical and political choices. They are products of human decision-making across a range of actors in specific social, cultural and historical contexts, and they reflect their creators’ tacit and explicit rationales, assumptions, norms, meanings and values (Filgueiras, 2021; Lenz, 2021; Lupton, 2018; Seaver, 2017). They may involve social ordering and forms of categorisations of social and human phenomena pertaining to specific values and biases. This can ultimately reproduce and reinforce socially divisive categories, causing ethical problems and harm (Beer, 2017; Boyd and Crawford, 2012; Bozdag, 2013; Chandler and Fuchs, 2019; Dutton and Kraemer, 1980; Eubanks, 2018; Noble, 2018; O’Neil, 2017; Tolan et al., 2019). Thus, the rationales guiding the development of DSSs have significant ethical and social consequences (Keddell, 2015). To increase validity and transparency, and to enable critical examination of DSSs’ potential in social work, Gillingham and Graham (2017) suggest a reflexive data science approach in which developers record decisions in the construction of data transformation and knowledge. Problems and targets are not given, and data do not speak for itself. Rather, actors provide data with meaning in particular contexts through processes of interpretation and construction of narratives (Dourish and Cruz, 2018). In relation to this, Jaton (2021) emphasises the importance of critically investigating the heterogeneous social practices aimed at confirming the accuracy of computerised methods of calculation focusing on the definition of problems, the identification of the appropriate data and the qualification of the targets. Similarly, Ananny (2016) argues that we need to explore the rationales involved in technological development by critically exploring the contextual assumptions and construction of meaning among actors involved in the development of DSSs.
The aim of this article is to present knowledge about the rationales, translations and the potentials for critical reflections embedded in the development of a particular DSS. The article presents empirical findings from a qualitative case study conducted in 2019–2020 in Citizen Centre Children and Young People (CCCYP), Copenhagen Municipality, Denmark, where we explored the development of a DSS expected to support social workers in making more adequate and quicker assessments about the severity reported in referrals concerning child neglect. Empirical data include informal conversations, mail correspondences and interviews with actors involved in developing the DSS (a top manager, a digital consultant, data managers and data scientists), a test report, a report evaluating experiences of using the DSS in one local child welfare centre, digitalisation strategies documenting political rationales and aspirations and technical documents describing the DSS. The analysis is inspired by the merging of two concepts: organisational sense-making and translation. Organisational sense-making appears in day-to-day routines or in the face of indeterminacy in relation to how specific problems are handled (Weick, 1990) and involves rationalisation of what actors are, know and are capable of, which creates professional identity (cf. Abbott, 1988) and organisational order. Translation is the process through which heterogenous actors are enrolled, disciplined and aligned in networks through problematisation, interessement, enrolment and mobilisation (Callon, 1984). The article asks the following research questions:
How do actors involved in the development of the DSS for assessing referrals concerning child neglect make sense of the technology? What kind of translations and potentials for critical reflection appears in these processes of sense-making? What lessons can be learned from 1) and 2) in relation to the development of DSSs to support child welfare decisions?
As an organisational and socio-technical experiment, the development of the DSS in CCCYP creates a space for key actors to construct, negotiate and readjust problem definitions, roles, interests, responsibilities and ideas of ambiguity and accountability. They attempt to orchestrate ways of managing the complexity facing child welfare services by projecting images of future potentials of the DSS that are yet to be realised. A major finding is that although technological solutionism is present in these processes it is not the only rationale invested. Rather, technological and data-driven rationales are adjusted to and merged with rationales of efficiency, return on investment and child welfare.
Child protection trends in Denmark as a backdrop for developing a DSS
In many countries, legal and managerial reforms including the implementation of IT systems for making risk assessments in child protection cases have been the preferred responses to cases of severe child neglect in deprived families and the public services’ apparent incapacity to protect children (e.g., Featherstone et al., 2014; Munro, 2009; Poikolainen, 2020; Simpson and Nowacki, 2018; Sørensen, 2018; White et al., 2009). The development of DSSs in Danish child protection has a similar backdrop. In Denmark, a strong focus on child protection and risk assessment for preventing child neglect has supplemented a family-oriented focus on offering universal and social services for vulnerable families (cf. Gilbert et al., 2011). The amendment of the Consolidation Act on Social Services in 2011 emphasised public employees’ obligation to react on suspicion of child neglect. At the same time, the Integrated Children’s System (ICS) and the adjacent IT system, Digitalisation of Children and Young People (Socialstyrelsen, 2011), were implemented. The Danish Appeals Board carried out a critical investigation of Danish cases of severe child neglect and public authorities’ failure to respond adequately (Ankestyrelsen, 2012). This led to the 2013 ‘Child abuse package’, which to this day requires social workers to respond within 24 hours to information on a suspicion that a child is exposed to violence and/or sexual abuse. Following this, the number of referrals increased by 42% from 2015 to 2019 in Denmark and by 57% in Copenhagen Municipality. Thus, the administrative and professional burden of assessing and following up on incoming referrals has increased significantly. It is in this context that attempts to develop DSSs emerge. Municipalities, managers and professionals face a challenge in terms of mobilising resources and time for handling referrals concerning child neglect. Thus, the development of DSSs is inscribed in efforts to improve efficiency and effectiveness in the public sector (Filgueiras, 2021; Pencheva et al., 2020). In a Danish context, such efforts are also related to a socio-demographic and economic dilemma of how an increase in universal welfare needs and expectations of quality should be met for the same economic and professional resources (cf. Nissen, 2017; 2019; Nissen et al., 2018).
Rationales of DSSs
At a very general level, Ames (2018) argues that new technologies, which are enabled by massive data sets, increased computing power and new techniques in machine learning, capture and ignite a powerful contemporary social imagination of an algorithmic or technological sublime. New, complex and potentially transformative technologies stir feelings of astonishment, awe, terror and psychic distance, as Mosco (2005) argues. In a similar vein and with regards to big data, Boyd and Crawford (2012: 663) argue that there is a widespread mythological ‘belief that large data sets offer a higher form of intelligence and knowledge that can generate insights that were previously impossible, with the aura of truth, objectivity, and accuracy’. Zooming in on computer scientists, Seaver (2017) argues that they tend to enact algorithms as conceptual objects, indifferent to implementation details. This may relate to one of the most powerful narratives about data, which is that they are self-evident and require no interpretation or narration (Dourish and Cruz, 2018). The belief in indifferent algorithms and the belief in self-evident and objectively true data may entail what Morozov (2013) terms technological solutionism, which is the belief that we can solve social problems through one-size-fits-all technological solutions. Thus, research indicates that social imagination, feelings and myths can underpin a belief in the self-evident potential of technological development, if not technological solutionism. These are somewhat broad notions, myths and imaginaries, and there is a need to explore empirically the local and contextual sense-making processes involved in the development of DSSs as they may paint a much more complex picture of the rationales invested in DSS.
In this article, we focus on the sense-making processes involved in the development of a DSS that is to be implemented in the CCCYP in Copenhagen Municipality’s Social Services Administration. The purpose of the DSS is to assist social workers in deciding whether a referral should be responded to urgently or not. The system has been trained and tested on a data set consisting of approximately 15,000 historical referrals and social workers’ actual assessments of these referrals as acute or not acute. The system does not rely on administrative quantitative data, such as age, prior incidents, family income, etc. Instead, it applies natural language processes to assess open-ended texts. The system can recognise the 15,000 most frequent words in the training set. Through iterative trial and error processes, a neural network has identified patterns in the training set and has subsequently been employed to assess and categorise referrals in a test set. Once implemented, the DSS will perform the same operation on incoming referrals. The expectation is that the DSS can support social workers in discovering and prioritising acute cases out of an increasing number of referrals. The DSS has been developed and tested in-house in collaboration between the CCCYP, the Social Services Administration’s IT-Strategic Office and Copenhagen Municipality’s IT department.
Theoretical framework
The analysis is inspired by the concept of professional and organisational sense-making merged with the concept of translation. New technologies affect relations between organisational goals and the knowledge, decisions and actions of professionals and may consequently spur professional and organisational sense-making processes. Sense-making processes do not depend on top-down decisions. On the contrary, they often appear in day-to-day routines or in the face of indeterminacy in relation to how specific problems should be handled (Weick, 1990). Sense-making involves the ongoing retrospective development of plausible images that rationalise what people are doing. Viewed as a significant process of organizing, sensemaking unfolds a sequence in which people concerned with identity in the social context of other actors engage in ongoing circumstances from which they extract cues and make plausible sense retrospectively, while enacting more or less order into those ongoing circumstances (Weick et al., 2005: 409)
As this quote indicates, sense-making involves rationalisation of what actors are, know and are capable of. This creates a sense of professional identity (cf. Abbott, 1988) and organisational order. Importantly, organisations themselves create this order. They filter, translate and manage phenomena into problems and solutions, roles, decisions and actions through communication. These meaning-making activities contribute to a certain mode of understanding, explaining, reflecting and acting upon a reality, which includes the organisation itself (|Luhmann, 1996; 2012). This shaping of reality not only creates boundaries that include but also excludes, certain worldviews, and produces both potentials for and limits to critical reflection (Nissen, 2010).
In relation to this, Andersen and Pors (2016) argue that public welfare organisations are in a transition that enacts a certain form of management by potentialisation. Recognising that risks and harms cannot be contained and controlled completely, organisations increasingly manage complexity by projecting images of future potentials that are yet to be realised. This management by potentialisation involves expectations of innovation, which are not necessarily grounded in evidence (Andersen and Pors, 2016), and economic rationalities and imaginaries concerning potential outcomes and benefits (Beckert, 2016). Thus, welfare agencies may seek to manage complexity and indeterminacies in child protection through technological innovation. This will not necessarily ameliorate the lives of children, who are subject to neglect but will allow the organisation to inscribe itself into national and local rationales concerning the potential of digitalisation, thereby creating a sense of solving a problem. Seen from this point of view, critical reflection should involve recourse to not only the technology but also to the complexities and uncertainties of the managerial context and the purpose of the organisation (Andersen, 2019; Nissen, 2010).
One can say that sense-making processes involve shared agency through negotiation, collaboration and a network of heterogeneous actors including nonhuman entities (Cochoy, 2014; Law, 2009; Latour, 2005). In order to analyse sense-making as practical processes involving heterogenous actors, we suggest applying the concept of translation. More specifically, we suggest that the four stages in translation, which will be described shortly, provide a nuanced vocabulary with which to analyse important elements in professional and organisational sense-making processes creating meaning of new technology and shaping new sociocultural realities.
Michel Serres first defined translation as a kind of mediation that simultaneously transmit and transform a signal (Brown, 2002). Callon (1984) introduced and adapted the concept to Actor-Network Theory’s vocabulary, ontology and epistemology. Hence, translation, in Callon’s terms, designates the processes through which heterogenous actors are enrolled, disciplined and aligned in networks around a specific problem. Translation is a four-stage process consisting of problematisation, interessement, enrolment and mobilisation. During the problematisation stage, problems are identified, appropriate solutions are presented and key actors, their interests as well as the alliances between them are described. For example, the DSS in our case is presented as a solution to problems of inefficiency and ambiguity inherent in decision-making and child welfare. In the interessement stage, roles, competencies, identities, goals and inclinations are consolidated in practice. Interessement, according to Callon, designates the group of actions by which an actor attempts to impose and stabilise the identity of the other actors and their interrelations as defined in the problematisation stage. For example, besides providing welfare services, public agencies may also become data brokers collecting, handling and sharing citizens’ data. Similarly, professional social workers may not only be thought of as end users but may also gradually become expected to have competencies to understand, account for and qualify the DSS’s output. Compared to problematisation, then, interessement designates practice in addition to interpretation. In the enrolment stage, negotiation takes place, aiming at the acceptance of assigned roles and identities. Enrolment describes the group of multilateral negotiations, trials of strength and tricks that accompany and help interessement to succeed. For example, during the development phase, digital scientists try out various algorithms, decision trees and thresholds to find different ways in which the DSS may fit its role and deliver acceptable outputs. In the mobilisation stage, the actors who have not been directly involved in the problematisation, interessement and enrolment stages are mobilised aiming at aligning the network (Callon, 1984; Cochoy, 2014). For example, in the case explored in this article, implementation of the DSS in the organisation’s various local centres requires that additional actors such as social workers and local managers agree with the problem identified, the solution offered as well as their ascribed roles, competencies and identities.
The four stages in translation – problematisation, interessement, enrolment and mobilisation – connect actors around a rationale and construct new realities. Translation, however, also transforms signals, as Serres puts it. Besides connecting actors, translation also implies betrayal and treason and each stage in the translation process involves risks of destabilisation and resistance (Law, 2006). Thus, neither success, stability nor a uniform sense of meaning is guaranteed. Rather, translation processes are fragile and uncertain, and as networks expand, the likelihood of conflict, disagreement and new uncertainties increases. As illustrated in the analysis, human as well as non-human actors may be a source of resistance and challenge existing rationales with regards to how problems, solutions, identities and associations are interpreted and enacted.
The case study and methods
We explored the rationales invested in the development of the DSS through an in-depth case study, which aims to produce context-dependent, rich and reflexive knowledge about practices of sense-making (Flyvbjerg, 2003). We conducted the case study in 2019–2020 and designed it in collaboration with one of CCCYP’s two digital consultants. The digital consultant is a trained social worker, who co-ordinates digitalisation initiatives and acts as a domain expert in the development of the DSS. Data for this study consist of Copenhagen Municipality’s digitalisation strategies documenting political rationales for DSSs, a test report on the DSS’s accuracy, a report evaluating the DSS in one local child welfare centre using the DSS, technical documents describing and visualising the DSS, as well as informal conversations, mail-correspondences and interviews with actors involved in developing the DSS including the digital consultant, data scientists, the top manager and data managers. The digital consultant arranged and participated in all interviews. The interviews were conducted by one of the researchers and comprises an initial dialogue with the top manager of CCCYP, one group interview with three digital managers from the Social Services Administration’s IT Strategic Office, and two group interviews, each with two data scientists from Copenhagen Municipality’s IT department. The interviews were transcribed verbatim by the interviewer, and actors were given pseudonyms.
We adopted an abductive research approach going from an inductive reading and coding of the transcripts with an open focus on identifying rationales and processes of sense-making in the development of DSS to a more specific focus on and coding of the processes of problematisation, interessement, enrolment and mobilisation. These concepts worked as sensitising concepts that do not substantially determine the scope of perceivable findings (Blumer, 1954), but, in this case, allowed us to engage closely with the empirical forms of meaning generated in and among the actors in CCCPY. Both researchers conducted the reading and coding parallel to analysing with a dual attention to theory and empirical meanings appearing in the material which could not be deduced from theory (Timmermans and Tavory, 2012). Following this approach, we analysed the translation processes involved in professionals’ sense-making of the DSS and discovered how these were related to processes of critical reflection concerning the potentials, limitations and uncertainties associated with the DSS.
Analysis
In Denmark, the development of DSSs using AI and big data has become one of many governmental responses to challenges of a growing aging population pressuring the public economy, a problem of recruiting welfare professionals and higher user expectations of quality in their encounter with public services (Danish Government et al., 2016; Local Government Denmark, 2019). Research has shown that these challenges tend to set forward struggles for finding resources and time to make good quality social work often combined with demands for productivity and efficiency (Nissen, 2017; 2019; Nissen et al., 2018). In the following, we will analyse how actors in CCCYP involved in the development of the DSS for assessing referrals make sense of these challenges through processes of translation.
Problematisation: Efficiency, return on investment and increased legal and professional accountability?
Problematisation designates the identification of problems, the presentation of appropriate solutions as well as the description of key actors, their interests and their alliances.
Copenhagen municipality annually invests DKK one billion (134 million Euro) in digitalisation, which covers approximately DKK 600 million (80 million Euro) in operational expenses and approximately 700 working years (Københavns Kommune, 2019). As a result, there are significant economic interests at play. During an interview, the digital manager, Emil, indicated that in the initial process of forming the digital strategy of Copenhagen Municipality, the expectations and rationales for developing new digital technologies in the area of child protection were negotiated. In the face of a managerial and economic rationale focused on digital technologies as a means to save money by increasing efficiency, the digital team made an effort in translating this agenda into a problem of the accountability of child protection and later, the potential value of the DSS in terms of responding more adequately and faster for the benefit of vulnerable children:
In the quote above, the digital manager and consultant emphasise the value of their work as a means to improve the legal accountability and professional efficiency of child protection. We can interpret this with a reference to the problematisation stage in the translation process. In the process of negotiating the problem that the DSS should respond and provide solutions to, a narrow conception of the DSS as a means to save money and deliver a return on investment was not an imaginary that made sense to the digital managers and the consultant (cf. Beckert, 2016). In addition, they introduced a problem of accountability and efficiency among the social workers in child protection: the DSS could potentially prevent social workers from overlooking referrals ‘by coincidence in a letter box’ and shorten the response time, not only for the benefit of administration but for ‘paying extra attention to cases critical for vulnerable children’. This problematisation is also fertilised by the imagined possibility of making better discretions powered by computing technologies and tools, as Emil later told:
However, as a way of counterbalancing this technology-driven rationale, Emil explains that politicians and the top manager do not expect them to be ‘first movers’ but, rather, ‘ambitious second movers’ being very careful when developing a DSS supporting the accountability of social workers’ risk assessment and decision-making. In relation to this, it is interesting how Philip motivates the value of the DSS. He suggests that ‘it has to be of some value’ for vulnerable children, even if this value is impossible to calculate. This suggests that the DDS is made sense of and considered worth investing in, not only due to the prospect of an economic return on investment but because of the possibility of improving accountability by diminishing the risk of overlooking children at risk.
This problematisation was confirmed by the top manager of CCCYP, who explained that the DSS is a way of improving accountability by enhancing the capacity to respond and act adequately and in due time on suspicions of child neglect. Moreover, her hope was to reallocate sparse resources and time to what she considered to be the core of social work: to be in contact with, and work closely with, vulnerable children and families. Thus, the assumption in CCCYP was that the DSS could potentially improve the quality of social work in the face of sparse economic and professional resources. This rationale aligns with a long-lived imaginary about how digitalisation can potentially ease the burden of administrative tasks, thereby making it possible to release and allocating resources to the ‘warm hands’ of frontline workers (Jæger, 2003). As we shall see in the following, this problematisation implies that specific roles and identities are ascribed to the DSS, social workers and even notifiers.
Interessement: Distributing decision making and the problem of ambiguity
Interessement, understood as the group of actions by which some actors attempt to impose and stabilise the identity of the other actors, implies the ascription of roles, competencies and identities constraining actors’ possible actions.
In an interview, the data scientists, Liv and Michael, explain how the DSS handles two different types of data. Normally, referrals come in electronic forms filled out by either professionals or citizens. The forms allow a person to describe a concern qualitatively with open-ended text and categorise it by ticking off boxes. If the person, who fills out the referral form, ticks ‘Yes’, instead of ‘No’, as a response to the question, ‘Is there a suspicion of violence?’, the DSS instantly categorises the referral as acute without processing any text in the form. This is interesting, since the purpose of the DSS is to ‘sustain and eventually improve quality in terms of responding adequately’ (cf. previous quote of Philip). By relying automatically on the ticked-off box, the DSS extends the notifier’s experience of reality, a suspicion, by immediately translating it into a suggestion to act urgently. The notifiers are ascribed a role as an important decision-maker as a consequence of the way the system is constructed and operates. Retrospectively, the digital consultant makes sense of this role ascription by arguing that CCCYP is to act instantly on all suspicions of violence.
If a notifier has not answered ‘Yes’ to the question ‘Is there a suspicion of violence’? the DSS applies natural language processes to assess the open-ended text. Through iterative trial and error processes, a neural network has been trained to distinguishing between referrals and mark them with a red, a yellow or a green label. The data scientist, Michael, explains how they in this process seek to reduce complexity and provide unambiguous outputs with as little information as possible:
However, resolving ambiguity turned out to be very difficult and ultimately resulted in a decision to go for a better-safe-than-sorry rationale. More specifically, the yellow category caused problems of ambiguity, and this led the data scientists to renounce the ambition of being able to distinguish clearly between yellow and green referrals. The data scientists explained the decision to modify their ambition of the output with a reference to the rationale and current ‘scenario’ in the Danish child protection system: a better-safe-than-sorry rationale concerned with the risk of overlooking a child at risk. Consequently, and as the following excerpt from an interview shows, the data scientist first decided to focus on the DSS’s handling of red cases, thereby leaving risk assessment of ambiguous referrals to the discretion of social workers:
As the quote shows, the digital consultant, Philip, does not consider it an error if social workers act urgently on referrals that the DSS categorises as acute, but which upon investigation turn out not to be acute, even if this takes time from handling acute cases. The result of the modification is that the output of the DSS does not relieve the social workers from spending time on referrals that are not necessarily acute, and that the capacity of the DSS as a technology for increasing efficiency in correct prioritisation is reduced. In relation to this, it is interesting how the value of the DSS as supporting assessments of referrals is sustained while the role of the social workers in relation to assessing ambiguous referrals is expected to be stable: ‘read’ them and make a discretion concerning how to act and how fast. It shows that the DSS does not necessarily support the correctness and efficiency of social workers risk assessments and decisions beyond the confirmation of red cases: ‘if you are red, then you are red’. However, based on later tests showing false negatives in 9 out of 444 cases it was finally decided to ‘calibrate the thresholds’ for each category and ‘merge the red and yellow categories’. The aim was to ensure that the DSS ‘always chooses the red category if there is any probability of a case being red’, as stated in the test report. As such, the ambiguous yellow category was discarded altogether. What was initially viewed as a category of ambiguous referrals left to the discretion of social workers was in the end translated into seemingly statistically unambiguous acute cases potentially overrating risks and increasing the pressure to handle more referrals faster. As we shall see in the next section, this translation of the better-safe-than-sorry rationale enrols the social worker employing a specific knowledge base and providing responsibility and accountability.
Enrolment: Knowledge bases, responsibility and accountability
Enrolment, understood as negotiations aiming at ensuring acceptance of roles and identities, involves a return to and reflections about different knowledge bases involved in risk assessment, how they differ or resemble each other and the risks of and limits to DSSs.
The DSS’s knowledge base is the data that the neural network is trained on; the DSS learns from social workers’ previous assessments as they appear in the historical data set. The data scientists consider data as providing unique access to understanding and getting full information about the reality of risk assessments ‘equal to talking to all social workers […] listening to how they would have categorised individual cases’, as the data scientist Michael explains in an interview. From this perspective, there is not much of a difference between human risk assessment and the basis upon which the DSS estimates referrals. In relation to this, it is recognised that the quality of data is crucial and that, in the case of the DSS, quality depends on how social workers have previously assessed referrals. Therefore, whether the data set represents the ‘correct way of working’ is of great concern, as expressed by the digital manager, Emil, in an interview:
In the quote above, the social workers are indirectly enrolled as responsible for the quality of the data set used for the DSS’s learning process. In relation to this, the digital consultant argues that classifications based on algorithms are no less accurate, and perhaps even more accurate, than the social workers’ individual assessments of referrals based on a ‘theory book’, because algorithms are trained on a real and big data:
In the quote above, we can see how the digital consultant enrols himself as a professional with a ‘point of view’ that is data-driven instead of theory driven. Phillip considers this both the same (‘what is the difference?’) and different: Instead of a theory-driven risk assessment made by a social worker, the DSS offers an output based on knowledge from a big set of real data. However, this also implies that a distinct or transparent background of theoretical understandings and explanations does not accompany the output of the DSS. The digital managers Søren and Emil critically reflect on this lack of transparency and the risks associated with it in terms of making correct or incorrect categorisations. In relation to this, they address the ‘algorithm’ and ‘human beings’ (including themselves) as actors, and question who can be held accountable:
The reflections of Søren and Emil can be viewed as a negotiation of roles and responsibilities and limits to DSSs. The digital managers recognise the risk that DSSs may potentially make people subject to inaccurate or unnecessary forms of profiling and control. They also reason that lack of accountability may potentially erode citizens’ trust in institutions, their willingness to provide data and thus decrease the possibilities for technological innovation. Moreover, they connect these matters with a greater issue concerning the distribution of decisions between human beings and machines (cf. Callon, 1984). Because of the lack of transparency and the inability to explain how the DSS works and how the output is generated, they reason that there needs to be a responsible actor who can be held accountable. This, they suggest, is the ‘human being making the final decision’, in this case, the social worker. In relation to this, they emphasise that the system is supporting rather than making decisions characterising this distinction as ‘the key premise’. In this way, the DSS is completely dependent on the social workers’ knowledge base and how they perceive the reality of and their responsibility for the quality of risk assessments in child protection. If this is the case, then what is the role of the data scientists and the DSS?
In the excerpt, Phillip suggests that the DSS can potentially become a tool for testing the accountability and quality of social workers’ risk assessments and make corrections to their practices. This deviates slightly from the top manager’s and their own problematisation, interessement and enrolment of the social workers. Ideas about translating administrative tasks into ‘warm hands’ or supporting decisions for the benefit of vulnerable children are toned down and translated into a ‘a test of confidence’. Phillip’s notion can be viewed as a projection of the potentiality of new roles and responsibilities of human and nonhuman actors in the processes of making risk assessment and decision; at some point, the DSS might prove more correct in making risk assessments than the social workers, and then the responsibility for making final decisions may be distributed differently. This projection is an example of how the use of algorithms opens for simulations, reflections and projections of ideas about the organisational reality and what it may become in the future (cf. Nassehi, 2019; Andersen and Pors, 2016). However, so far, the capacity of DSS to support social workers risk assessment and decisions is yet to be tested.
Mobilisation: Testing the value of DSS and revisiting the problem, the roles and responsibilities
Although projections regarding the future role of DSS are made, the DSS is currently made sense of as a technology that does not, and should not, make decisions but only support professional decision-making. As we have shown above, this enrols social workers as indispensable human decision-makers responsible for the accountability of a decision supported by a DSS translating potentially ambiguous referrals into seemingly statistically unambiguous red cases. For specific problematisations, interests, roles, aims and goals to be connected successfully, key actors hitherto only represented indirectly in data and by data managers, consultants and scientists need to be mobilised and enrolled, disciplined and aligned in this network. Following an incremental implementation and test strategy, the DSS was first taken in use and tested by social workers in one local child welfare centre over the course of three months in 2021 prior to a potential implementation of the DSS in additional centres. The successive evaluation report concluded that new and inexperienced social workers are more inclined to consult the DSS’s categorisations when assessing incoming referrals and especially in periods with many referrals, but that the overall value of the DSS is deemed insignificant. The reason, which is clearly stated in the report, is that the social workers testing the DSS do not experience the problem that the DSS was supposed to solve: The test unit does not perceive the work with categorising referrals as either acute or not acute as problematic. Categorisation happens within a maximum of one hour, and neither assistants, screening guards nor managers find it difficult to assess whether a referral is acute or not. There is therefore no demand for the solution provided by the algorithm.
The social workers testing the DSS do not recognise the problem so meticulously constructed by digital managers, consultants and scientist connecting problems of efficiency with problems of accountability and quality of child protection. According to the evaluation report, the centre was chosen as a suitable and ‘safe’ test site, because it already conducts a systematic assessment process and has developed a structure ensuring speedy executions. In relation to the analysis of the translation and sense-making processes above, we can understand this as the consequence of translations: minimising the risk that the DSS would disrupt normal procedures and potentially overruling social workers understood as accountable actors responsible for making the final decisions. Playing it safe and within the confines of the translation process already well underway in this case, however, meant that the DSS was never really put to the test and that its affordances were questioned. Had the DSS been tested in other centres working under different circumstances, for example, a center experiencing problems with assessing referrals correctly, the DSS could have proven of greater value easing, qualifying and improving the estimation of incoming referrals, the evaluation report concludes. In the case of such an outcome, the responsibility for making final decisions could eventually become renegotiated, changed and distributed differently, as indicated by Philip (cf. above). However, this is a potential yet to be realised.
Conclusion and discussion: Lessons to learn for critical reflection on the reality of child protection
DSSs are sociocultural artefacts that reflect a mix of technical and political choices and therefore their creators’ tacit and explicit rationales, assumptions, norms, meanings and values. Since the rationales and choices guiding the development of DSSs can have significant ethical and social consequences for people affected by DSSs, it is important to investigate these in the specific contexts in which they are developed.
In this article, we have presented findings from a qualitative case study on the development of a DSS to support social workers in assessing referrals concerning child neglect. We posed three research questions: 1) How do actors involved in the development of the DSS for assessing referrals concerning child neglect make sense of the technology? 2) What kind of translations and potentials for critical reflection appear in these processes of sense-making? 3) What lessons can be learned from 1) and 2) in relation to the development of DSSs to support child welfare decisions? We have merged the concept of professional and organisational sense-making with the concept of translation to analyse sense-making processes facing indeterminacy, characterised by potentialisation, and involving a network of heterogeneous actors including nonhuman entities. The concept of translation (problematisation, interessement, enrolment and mobilisation) creates awareness of how sense-making is both socially and technically conditioned and constructs new sociocultural realities. As a concept, it attunes the analysis to points of conflict and negotiations and therefore, the uncertainties implicit in sense-making processes.
The analysis shows that the DSS was initially considered a potential solution to managing the complexity facing child welfare services (Andersen and Pors, 2016). The DSS was at different organisational levels and by different managerial and administrative actors presented as worth investing in and a potential solution to problems of controlling public spending, of handling an increasing number of referrals efficiently and accountable, and of using social workers (‘warm hands’) adequately. This organisational problematisation had some affinity with technological solutionism understood as a self-evidential belief in how technology can solve very complex problems relatively indifferent to the context and complexity of implementation processes. The analysis shows that the digital managers, -scientists and -consultant attempt to translate this belief into a problem they suggest the DSS can handle: a technology that can improve the efficiency and accountability of social workers' risk assessments by offering an unambiguous distinction between green and red cases. However, in the interessement and enrolment stages, where roles, competencies and responsibilities of the DSS and social workers were imposed and negotiated, the complexity and ambiguity inherent in child protection and risk assessments complicated this ambition. It was recognised that data depend on and represents social workers’ practice – good as well as bad – and that algorithmic risk assessments do not only depend on mathematics but also on choices regarding acceptable thresholds. Therefore, expectations with regards to the capacity of the DSS to offer a more accurate and distinct risk assessment were gradually modified, and the responsibility for risk assessments was (re)distributed to the social workers understood as inevitable actors with competencies to ‘making the final decision’ potentially subject to a ‘test of confidence’. However, a test of the DSS in a local child welfare center showed that efficiency and accountability is not necessarily a problem in social work practice. Therefore, the DSS did not make a difference in the social workers’ risk assessment, and the capacity of the DSS to support social workers’ decisions is a potential yet to be realised.
In the case investigated in this paper, the digital consultant, the digital managers and the data scientists are meticulously and continuously constructing and negotiating the problems, the interests, the roles and the affordances of the DSS. This involves reflections on various problems, forms of knowledge, uncertainty, ambiguity, accuracy, data quality, responsibility and accountability – combined with processes of making sense and adoption of the better-safe-than-sorry-rationale in Danish child protection. The result is a modified DSS, which is yet to prove valuable in terms of providing efficiency and accountability in relation to severe cases of child maltreatment. Thus, reflexive data science must, as Gillingham and Graham (2017) notes, not only increase transparency and validity in relation to data and the construction of the DSS but must also critically examine the extent to which the use of big data and algorithms provides new knowledge adequate for the complexity and challenges of child protection. In relation to this, this article offers at least two important lessons to learn.
Firstly, our article shows that social workers remain indispensable decision makers and therefore responsible for providing legitimacy and accountability to the decision-making process and outcome. Moreover, the result of the test of the DSS in a local child welfare centre serves as an example of what can happen when indispensable actors are not involved in the problematisation stage but are mobilised solely as end-users. As an alternative, one could imagine a reversed translation process aiming at mobilising social workers’ knowledge about practical challenges and needs for technological support. This does not exclude professional challenges in relation to increasing efficiency and accountability, but eventually also includes other challenges related to investigating and ameliorating the lives of vulnerable children. Social workers’ knowledge of the contextual and practical complexities of child protection, risk assessment and the needed support and help for vulnerable children and families is valuable to spur critical reflection on the reality of and imaginaries associated with DSSs. Child welfare agencies that invest in DSS must therefore consider how and to whom the responsibility for problematisation, interessement, enrolment and mobilisation is distributed in the development of DSSs.
Secondly, and closely related to the first, organisations and managers must reflect critically on, how they inscribe their specific organisation into national digitalisation strategies and translate the potential of new technological solutions. In Denmark, DSSs are understood as a solution to complex problems and cross-pressures stemming from scarce resources – economic, human, time, etc. – and higher user expectations in the encounter with the public services (Danish Government et al., 2016; Local Government Denmark, 2019). However, our article shows that technological solutionism, and in particular in the sense of a one-size-fits-all-solution, does not make sense to a highly specialised (public) welfare organisation dependent on professionalism in delivering services. Indeed, research shows that in such a context, organisational changes purely guided by managerial, administrative and technical incentives risk fostering unintended consequences and ‘defensive’ practices where child protection workers are more preoccupied with complying with IT systems than with seeing dangers related to child neglect and solving social problems (Whittaker, 2018, Broadhurst et al., 2010; Munro, 2011, White et al., 2010).
Future research
As for our research, important questions remain unanswered. For example, whether referrals marked as acute by the DSS reflect an actual problem of child neglect or not is an open question. Research on different ways of constructing DSSs to support the context and challenges of child welfare and child protection is still important (cf. Gillingham, 2019). Another unanswered question relates to variations and the possibility of alternative rationales constructed in the development of DSSs for child protection. Comparative research into different contexts and processes of translation and sense-making of DSSs could be valuable in order to understand and explain different managerial and professional rationales and strategies, e.g., with regards to problematisation, interessement, enrolment and mobilisation of actors. In the future, such research could also explore the long-term consequences of the use of DSS. Do problematisation, interessement, enrolment and mobilisation change due to critical reflection on and clarification of the potentialities and limitations to DSS? What are the effects of the implementation of DSSs, will they still be considered worth investing in, and will resources and responsibilities be distributed differently? For example, will the implementation of a DSS release resources for social workers to work closely with children and families, as the top manager in CCCYP suggested?
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.
