Abstract
Discussions about ethics of Big Data often focus on the ethics of data processing: collecting, storing, handling, analysing and sharing data. Data-based systems, however, do not come from nowhere. They are designed and brought into being within social spaces – or social milieu. This paper connects philosophical considerations of individual and collective capacity to enact practical reason to the influence of social spaces. Building a deeper engagement with the social imaginaries of technology development through analysis of two years of fieldwork with start-ups working on Internet of Things, this paper suggests that different action positions can emerge, with consequences for how data is understood and valued. The Disengaged, Pragmatist and Idealist ethical action positions identified in the paper reveal the ways individuals and groups negotiate possibilities for ethical action, through justifications, explanations and structuring of system features.
Introduction
Discussions about ethics of Big Data often focus on the ethics of data processing – ‘generation, recording, curation, processing, dissemination, sharing and use’ using algorithms (including machine learning and artificial intelligence) as well as corresponding practices such as programming, hacking and coding (Floridi and Taddeo, 2016: 1). Data-based systems, however, do not come from nowhere. This article shifts the focus of ethical discussion to the contexts of data production, particularly the ethical qualities of the social milieu in which data-intensive technologies are produced, and the practical reasoning that people undertake as they negotiate these social milieu, identifying that uncertainty and contingency in these spaces may be one reason that efforts to bring ethics into technology may not yet have borne fruit.
We focus on Internet of Things (IoT), a framework for data-producing connected technologies that grew from experiments with internet-connected machines in the 1980s. Since then, references to IoT and ‘connected worlds’ have permeated corporate discourse (Cisco, 2011; Thibodeaux, 2017) evoking a new kind of ‘digital sublime’ (Mosco, 2005). This contributes to a social milieu where, as Mansell (2012) argues, dominant social imaginaries stress the value of information (and data) as intellectual property, creating business models based on extensive data extraction and processing. For us, a social imaginary is ‘an ethos that enables people to make sense of developments in a society’ that can create ‘a shared sense of the legitimacy of the organization and conduct of a society’ (Mansell, 2012: 32–33). Within these imaginaries can develop particular moral reflections that guide social action. Not specifically ideological, these work implicitly because they are taken for granted. In this article, we focus on this moral aspect, using concepts from the virtue ethics tradition to provide greater specificity on how this implicit moral reflection occurs, and its implications.
The ethical social milieu of Internet of Things development
Increasingly, ethical issues concerning protection of personal information, freedom of action, ownership of data and self-determination have attracted attention (Mansell, 2012) along with issues related to bias, discrimination and justice. Connected and autonomous system design can also make it difficult to identify responsibilities and liabilities (Jasanoff, 2017; Taylor, 2017; Van der Ploeg, 2003). To better identify how IoT developers make ethical decisions in context, we focus on small companies and start-ups, which are constrained by expectations about rapid financial success through competitive innovation in creating IoT technologies. Unlike large corporations, start-ups have limited administrative, human and legal resources at their disposal and hence limited capacity for comprehensive ethical assessments. They also face uncertainty in their businesses, with an expectation that the majority of them will fail within two years after launch. We wondered whether these uncertainties would contribute to a social space with a different orientation to ethics than that of the big companies, which have made public commitments to principles of ethics by convening ethics advisory boards and publishing aspirational ethical guidelines but then dissolving these boards or deferring responsibilities to content creators (Financial Times, 2018; Murgia and Shrikanth, 2019; Waters, 2019). While influential, it is unclear whether these guidelines and advisory organisations shift IoT business development towards ethical and responsible innovation.
In this paper, we aim to address two gaps in the literature. First, we move away from analysing top-down efforts to define ethics made by powerful actors and instead examine the social spaces occupied by small companies. Second, we take a practice-based approach to ethics employing the virtue ethics tradition to assess the practices that people employ in engaging with ethics in technology development. This is necessary in order to expand the range of ethical perspectives used to analyse data-intensive processes, to ensure that ethical questions are considered fully and from a range of perspectives.
Our analysis is based on two years of multi-site ethnographic fieldwork with developers in/of IoT start-ups based in London, Barcelona, Geneva, Copenhagen and Belgrade as part of the VIRT-EU Project: Values and Ethics in Innovation for Responsible Technology in Europe. We attended industry meetups, hardware and software showcases, workshops and conferences, followed an accelerator programme and conducted in person interviews and co-design workshops with IoT developers, designers and entrepreneurs, undertaking more than 100 unique fieldwork visits between 2016 and 2018. As part of a related study, we were also able to examine ten years of data from the records of the IoT Meetups in Europe, and also conduct ethnographic research into London IoT Meetup group. Here, we present ethnographic material in the form of interview excerpts, ethnographic field notes and case study vignettes to illustrate the practice of ethics within IoT development and design. These are the result of several rounds of thematic coding and ethnographic analysis, with the first oriented towards the ways that ethics are expressed, and the second focused on positions within the social milieu, involving the development of ethnographic vignettes. These two rounds of coding and interpretive work followed the first round of fieldwork in 2016–2017. A third round of coding occurred with the addition of new fieldwork material in 2018, and focused on identifying ethical action positions. In these three rounds of coding, we asked: how do developers in start-ups and small companies practice ethical decision-making within the social spaces they engage in? What are the technological, business and social contexts that influence these practices?
To answer these questions, we begin with a conceptual framework grounded in the virtue ethics tradition of Alasdair MacIntyre and Elizabeth Anscombe, focused on ethical practice within what MacIntyre calls social milieu, or spaces where people build shared understandings of appropriate actions (Anscombe, 1958, 2005; MacIntyre, 2016, 1999). As explained earlier, we investigate these milieux as ways to foreground and investigate the modes of moral reflection within social imaginaries.
We present three different positions that individuals may take within the social milieu of technology development. These positions are the Disengaged, the Pragmatist and the Idealist. Each position – or action position – holds the capacity for alternative social imaginaries to flourish, as well as constraining action. By presenting these three perspectives, we show how data ethics are not only performed through data processing, but also through negotiations within social spaces.
Expanding ethical approaches to data-based technologies
In an article published in this journal, Andrej Zwitter notes that Big Data poses important challenges to moral agency, namely, free will and individualism. In addition, it can alter power dynamics by redefining relationships in relation to connectedness and combining these with risks related to privacy and profiling (Zwitter, 2014: 2–3). Zwitter’s account, along with others examining technology and information, concerns the consequences of actions (i.e. consequentialist ethics) (Berdichevsky and Neuenschwander, 1999). Ethical frameworks, however, operate in many ways as collateral realities: they would cease to exist, without those practicing them and turning what is being done into what necessarily has to be done (Law, 2009: 14). Therefore, this article takes a practice-based approach to ethics employing ideas from virtue ethicists, and focuses on the process of constructing practical reasoning in relation to the ethics of IoT. Charles Taylor’s (1995) explication of this approach identifies how this process of reasoning unfolds in understanding of scientific knowledge, specifically in a context of rapid societal, technological and economic change. Hence, our framework foregrounds the importance of examining practical reasoning and ethical actions, because ethical evaluations based on the implications of data processing do not fully account for how ethical reasoning and assumptions come to be established in the first place.
From this constructivist position, we take up virtue ethics as a perspective that can be developed in parallel to and in connection with the related ‘values in design’ frameworks (Coleman, 2012; Coles and Norman, 2005; JafariNaimi et al., 2015; Powell, 2016; Shilton, 2018), allowing us to processes that sustain, reinforce or challenge decisions made when building new technologies. We are primarily interested in inquiring into the values and virtues articulated and practiced by people who develop new technologies, rather than approaches that focus on consequences of the technologies once they are built (Vallor, 2010: 158). 1
One approach to examining technology development may be in relation to claims and efforts at goodness – the realm of virtue ethics. This ethical tradition can be traced back to the philosophical writings of Plato, Aristotle, Socrates as well as Stoics and Cynics. It is concerned with questions such as ‘What is a good life’, ‘What does it mean to be a good person’. In the mid-20th century, Elizabeth Anscombe advocated a return to virtue ethics to address the blind spots of moral philosophy which was largely consequentialist and focused on the final utility of individual actions as the site of judgement (Anscombe, 1958, 2005). Anscombe advocated consideration of how an ethical person would behave when faced with a particular ethical dilemma. Such a positioning holds a commitment to concepts such as excellence and virtue over implications, utility or greatest good for the greatest number (Anscombe, 1958, 2005).
Recent work in virtue ethics has built on Anscombe to develop more specific discussion on the relationship between an ethical agent’s character and their engagement with principles, rules and responsibilities within their own social context. MacIntyre (2016), in particular, has developed the notion that a virtuous agent acts in relation to his or her individual virtues but also in various collective contexts or ‘social milieu’. These become sites for the development of practical reasoning: individuals and groups from different places and backgrounds engage with the political and ethical realities of the social milieu they inhabit in various ways. Central to these processes are questions around the idea of human flourishing. This opens up a space for reflecting on the emotional structure of thoughts and actions in order to understand what it means for individuals and groups to make sense of things and to reason as they do (Williams, 1973). MacIntyre’s unique attention to the social context makes his philosophical perspective helpful to articulate with the idea of the social imaginary. 2
We use the invitation to think of virtue ethics as being enacted within social milieu to analyse how IoT developers engage in practical reasoning. This lets us examine which kinds of ethical actions become possible. This situated understanding of ethics opens a new space for understanding data-based technologies and explains how social milieu enables and constrains certain forms of ethical action.
The creation and use of new technologies produces new social imaginaries, as described above (Castoriadis, 1997; Mansell, 2012; Taylor, 2004). Here, we follow MacIntyre’s argument that people act not only in relation to their own virtues, but also in relation to various collective contexts (MacIntyre, 2007), creating different action positions. In other words, the relationship between individual responsibility and ethical action and broader political and social contexts can shape and constrain people’s capacity for ‘practical reasoning’ (MacIntyre, 2007, Taylor, 1995). For us, practical reasoning includes practices of explanation and argumentation as well as decisions about the structure of technical systems. This philosophical perspective resonates with other constructivist traditions, providing means to identify moral and ethical aspects of technology-making practices. Specifically, we situate these practices in relation to social imaginaries, which provide different ways to negotiate the potentials presented within technical development. Mansell (2012) describes how the social imaginaries of technology development encompass dominant business and innovation-focused imaginaries as well as their alternatives, while Powell (2008, 2018) shows how social imaginaries can hold contradictions as people attempt to negotiate different perspectives on technology. Here, we demonstrate how different imaginaries emerge and co-exist, and how they come to be sustained within social milieu. We connect the social space of moral negotiation (social milieu) with the space of technology design (social imaginary) by observing practices of explanation, argument and technology-making.
All of these practices have been associated with how values come to be embedded in technology, whether through infrastructural choices (Shilton, 2018), standard-setting (Braman, 2010; DeNardis, 2010) or software and hardware development practices in open-source culture (Coleman, 2012; Powell, 2016), and it is possible to assess relationships between particular practices and values. Previous work has analysed how argumentation and system design practices in open source software development not only expressed values but layered these with moral judgements related to ideal modes of human flourishing (Powell, 2018).
Here, we extend this discussion of values and design and directly engage with the idea of social milieu as a space for ethical engagement, deepening the concept of the social imaginary of technology. We use MacIntyre’s evocation of the role of social worlds in shaping how people develop ideas of the ‘good life’ and ‘responsibility’ to show how ethical issues related to the development of connected devices are framed in particular terms, often occluding or displacing issues that do not connect with the perspective of that particular social milieu, sometimes even influencing how people are able to situate their own subjectivity ethically (MacIntyre, 2007). Our connection between social imaginaries and virtue ethics lets us do two things. First, we are able to examine the ethical concerns and values of developers and how these come to be reflected in the technologies they come to build. Second, we can pay attention to the social contexts as they produce assumptions about the capacities and uses of these technologies. We accomplish this by developing action positions that illustrate how people imagine their capacity to act within their social milieu. This helps us expand the range of ethical perspectives used to reflect on and advance ethical considerations of emerging technologies, beyond consequentialist approaches, and show why making the technology industry ‘ethical’ cannot be a goal in itself.
Disengaged, Pragmatist and Idealist: Imagining capacities to act
In the previous section, we drew attention to the importance of social contexts, where organisational structures and cultural logics help develop and enact values (and/or virtues). Here we focus on how people engage with these social contexts, organisational structures and cultural logics in the field of IoT, showing how the social milieu helps to produce different action positions, and how the contingencies of these positions can be limiting as well as enabling.
Our findings indicate that the social milieu of technology development, being strongly focused on innovation, market share and corporate reputation, creates challenges for direct and explicit engagement with ethics, which forms a major constraint to systemic change in this area. Many people we spoke to considered ethics as important but could not find means to engage directly with ethical issues. As such, ethical conduct was rarely discussed in the online or offline encounters we observed.
Three positions we observed in our fieldwork illustrate points of engagement with ethical or moral concerns. These fold around particular kinds of action and we refer to them as action positions. They are, the Disengaged, the Pragmatist and the Idealist. Within the Disengaged position, many IoT developers described their space of action as demanding action on issues relating to finance and business sustainability, remaining ambivalent about the ‘use’ of ethics in technology development beyond compliance with legal regulation. A Pragmatist position advocated employing ethical principles pragmatically in response to external structural forces including increasing demand from consumers for ethical products, the need for data privacy and security, or changes in corporate governance as a means of mitigating financial liability. An Idealist position advocated action on values and principles by embedding them into products and business ventures they built or social networks. These positions are of course not the only ones that are available to people in these spaces, but include the most significant directions of engagement observed within our fieldwork. We outline the orientation towards action associated with each of these positions in relation to explanations, arguments and technology design choices.
The Disengaged: We can’t do ethics as it may slow down innovation
Our analysis suggests that ethics-by-design may be much-celebrated yet little acted-upon. Many people we spoke to were disengaged from the problems or potentials of ethical thinking by the perceived necessity to deliver particular products or reach investment milestones in their business development. This was especially true if they had raised investments through private equity firms (e.g. venture capitalists) or private investors (e.g. Angel investors). A feeling of being pressed for time, along with this financial concern, removed ethics from some people’s agendas: in some cases, we observed apathy towards ethics (participants explained that ethical deliberation slowed down the creative process). In other cases, we heard about the importance of ethics followed by a qualified ‘…but’. For instance, a developer who is one of the three co-founders of a wearable tech company that integrates IoT into sports-wear told us that she would be interested in thinking about ethics, ‘but investors do not care about ethics, they care about actual products’. 3 Through participation in an accelerator program, her company was trying to ‘aggressively raise money’ to be able to stay afloat until the next investment round. Similarly, in a round-table of technology investors, professionals in international development and IoT system designers, an IoT designer pointed out the importance of ethics and how companies should design products with ethics in mind from the outset ‘but’ also gave a warning. 4 He said that Google hired an ethicist to help them with incorporating ethical design to the new technologies they are working on, but shortly afterwards fired him, because ‘he asked too many difficult questions’ and this, according to him, had slowed down innovation processes.
In our study, the broader technology development social milieu in which these developers socialised rarely allowed room for engaging in ethical discussions beyond those specified by laws and regulations. Although there was little discussion of this principle before its introduction as an enforceable principle, once the European General Data Protection Regulation (GDPR) came into force in May 2018, participants taking this position began to explain privacy concerns not as broad ethical issues but as concerns about compliance with principles identified in the regulation. 5 To illustrate, within the nearly 90 meetups held by IoT London Meetup Group in the last 10 years, our analysis indicated that ethics has featured as a topic only once, while meetups focused on GDPR compliance were much more common.
In short, we identified distraction from and disengagement towards broader discussions about ethics, with a tendency to orient discussions towards what is required by law. The assumption that innovation works in a ‘beta version’, pushed out before being fully tested and in anticipation of improved future versions, was often used as a justification for a disengagement with potential ethical risks of technologies. Moreover, Facebook’s motto ‘move fast and break things’ was extended to ethical action. It was presumed that things might go wrong, but technology would be there to ‘fix it’. 6
The inherent uncertainty of start-up development, coupled with the push from investors to deliver and make a viable market intervention, was suggested as the main reason for not prioritising ethical concerns. Disengagement, therefore, did not always suggest unwillingness to engage in ethical discussions (though in some cases it did); rather, this action position was influenced by the social milieu prioritising the survival of business and the generation of profit. The ethical underpinnings or implications of technologies were often therefore discussed in relation to commercial and financial concerns and often became secondary to them.
The Pragmatist: Ethics as a business interest
In the second action position, the Pragmatist, a practical perspective is adopted for ethical decision-making. This practical position is framed in relation to increasing demand from consumers for ethical products or the need for strong governance of data privacy and security. This action position places ethical concerns squarely in relation to business interests, but not subsumed by them. We heard that ethics provided new market opportunities or allowed businesses to limit financial liability, and observed how these ‘pragmatic’ concerns influenced technological design.
For example, in a product showcase in London, it became apparent that several companies had decided to completely de-anonymise their data to comply with the GDPR. 7 Removing their responsibility for personal data spared them from having to ‘do ethics’ as one of the developers from an industrial IoT start-up told us. 8 We heard various ways that personal private data could be aggregated and anonymised in order to comply with the Regulation, though much less about why private data might need to be anonymised to begin with.
In our study, it was a widely shared view that data is valuable, and that even if it did not appear valuable at present, it might become valuable in the future. For instance, we heard many references to the Roomba Robot Vacuum Cleaner; how the company stored data on individual homes but did not use the data until a third party offered to purchase it (Astor, 2017). Roomba thus became both a hardware company and data processing company.
In response to examples like this, developers expressed that they just needed ethics to be more ‘doable’, given their financial, time and human resource limitations, and given their perception of the uncertainty of future business decisions and directions.
An effort at constructing a kitemark recognising ethical capacities illustrates how this position enacts its version of pragmatism. The Open IoT Trust Mark project started with the intention to create a visible or traceable trade or kite-mark for connected devices that would indicate the device bearing the mark would adhere to certain ethical standards, including governance and management of personal data, and sustainability of materials used for manufacture.
9
Indeed, values such as honesty, justice, courage, care, wisdom, flexibility and magnanimity (Vallor, 2016) underpinned the goals of the group that started the Mark. A participant in one of the IoT Mark meetings positioned ethical values as necessary for business:
While IoT innovation is important, fundamental human values must be preserved and respected. They must not be seen as a burden. This requires a cultural change of mind-set. We need a social contract.
10
Here, it is important to consider the possible space for action in relation to a particular social imaginary, and the extent to which this may or may not be able to challenge the social structures that could constrain ethical agency or ethical action. We observed during the five IoT Mark workshops held over two years that the loudest voices came from the software developers who outweighed other members by a large margin in terms of profession, gender and race (i.e. young-to-middle-aged white male engineers). These voices advocated for pragmatic interpretations of principles that could be easily adapted by businesses, and they clearly noted that unless business concerns were central to ethical discussions, they were bound to be ignored. They argued for features like security and pushed back on principles that could have undermined existing business practices. When defining the assessment criteria for the principles, they also voted for extremely specific, itemised list of actions that could be ‘ticked off’, rather than questions that provoked deeper engagement with ethical issues. In short, they removed divergent ethical principles from the Mark to avoid impacting business capacity to innovate or profit. We observed that the principles chosen for the Mark and their assessment criteria were similar to the guidelines already in existence in many other documents which highlighted the need for security, privacy, transparency, accountability and interoperability for IoT devices. 12 Indeed, the entire privacy section and their assessment criteria of the Mark were replaced by the eight principles of the GDPR. 13
We also heard support for the ‘value’ of ethics and ethics-by-design, given the increasing consumer interest in ethical and sustainable products. Here, although ‘doing good’ and building ethical products (or products with a social impact) were portrayed as the main goals, this was explained in relation to the ‘market opportunity’ for connected IoT products. The emergence of ‘tech for good’ accelerator programmes, social ventures and investors oriented towards social causes addressed through technology were given as illustrations that ‘doing good’ and creating profit were not mutually exclusive – perhaps even generating significant returns to investors (Chowdhury, 2017). However, participants foregrounded the potential for social ventures to create an ‘untapped’ market as a reason to consider ethical principles. 14
To illustrate, consider the story of Company X, a start-up providing solar-powered batteries to poor rural populations in order to sustain connectivity of mobile devices. The company website, displaying glossy images of Global South populations, promises to offer ‘Power for All’ and close the ‘digital divide’. The material explicitly references the UN Millennium Development Goals, #8 F in particular: ‘co-operation with the private sector, make available the benefits of new technologies especially information and communications’ (UN, 2015). Such references suggest the business was working towards the aims of social justice and humanitarianism, by setting up mobile connectivity as a basic human right (Willems, 2017). As the CEO of the company, John,
15
explained to us, the company came to focus on providing electricity to power mobile devices as a specific business aim. While he acknowledged that access to electricity was an important issue, he felt that mobile connectivity for the poor was an area that had untapped potential. He explained:
We needed a high rural population density. We found that in India a single national entity where the number of people living in rural areas amounted to more than the next ten national entities combined.
16
I think my views on development are controversial, maybe unpopular. I believe that you can either be a charity or a business, not both. Anything that tries to do both does neither well. There is no third way. I know ‘the capitalist’ is considered a bad word, but to be honest, I’m a capitalist.
If you can measure something, you can design for it; if you can design for it, you can create new value, if you can create new value, you can monetise it. Our aim is to create peace businesses (Taylor and Broeders, 2015).
The virtues demonstrated within the Pragmatist social milieu hinge on ‘mutually beneficial exchange’. Where MacIntyre strongly criticised the idea of a market economy exhibiting virtuous behaviours (MacIntyre, 2016), other ethicists argue that the values of cooperation and exchange (Miller, 2017) as well as expression of freedom through the exercise of choice (Zupan, 2011) are part of market dynamics; but also creating ‘value’ through creating a business and job opportunities. 19 Virtue ethicist Shannon Vallor (2016) explores how virtue ethics relates to technology, but her assessment does not clearly identify the role of business and markets. In our case, the virtues of the social milieu transcended those of individuals, and determined the ways that they explained their decisions. In such a context, ethics mattered once it was embedded into and carried by business interests.
The Idealist: Ethics as market disruptor
Unlike the Disengaged and the Pragmatist, some IoT developers situate themselves as making an intervention in what they see as the overly business-led perspective of the technology industry. These developers told us that they would like to build products with different ideals in mind, or at least start a critique, suggesting that they felt capable of expressing more virtuous positions. A series of IoT manifestos advanced some of these perspectives (Fritsch et al., 2018) and some developers we interviewed also positioned themselves and their individual trajectories along these lines. The Idealist action position includes developers who would like to challenge the main discourse in start-up world, demonstrating strong attachment to ethical values and are motivated by the lack of debate in the tech world about the kinds of futures developers are building, one product at a time. The ethical perspective here frames ethics as a relational, human-centric activity and ethical acts as a narrative construction, as an ‘individual-in-relationships’ rather than an isolated single person (Vallor, 2016: 57). We observed a strong identification with ‘we’ rather than ‘I’. Respondents stressed learning from the mistakes of one another, growing together collectively towards the attainment of shared ideals as values that bring them together. Here, we began to see the possibility of separating (and integrating) individual and collective subjectivities in relation to ethical concerns, and also an active engagement with the responsibility for producing ethical technologies (and futures).
Marcus, a developer in his forties, had worked in the tech and design industry for over twenty years. He proposed ‘deconstruction’ as a new ethical paradigm for thinking about IoT design and development, referring to the redesign of Information and Communications Technology networks in a decentralised manner so that citizens could gain more control over their data and become aware of the implications of their connectivity. 20 When asked about why he developed this ‘alternative’ thinking to technology decision-making, Marcus alluded to his own experience growing up next to a French border, on the German side of Strasbourg. While the border separating two countries certainly existed, it was also, to Marcus, ‘never really there’. As a result of his personal freedom to easily move between two countries, the idea of borders appeared as an artificial division, a way of segregating people economically, politically and culturally through arbitrary means. As Marcus notes, ‘This whole way of ordering the world seemed to create more issues than not’. Indeed, various developers we spoke to explained that they thought technology created artificial borders, as algorithms created black-boxed systems which non-developers could not understand. ‘Giving people their data back’ was identified as a potential ethical solution in our study, either in the form of data-boxes or in the form of full management rights where individuals are given the power to keep or ‘trade’ their own data. As one developer working in an individual data trading start-up explained, ‘This idea may never work. But we need to try it to prove that other ways of thinking are possible’. 21
This contextual understanding of ethics often came with a critical approach to the tendency to equate data with truth (Rieder and Simon, 2016). According to Marcus, for instance, data should not be interpreted as a ‘one size fits all’ solution. This speaks to the increasing tendency of government and industry to mine ‘truth’ from vast amounts of data collected from (IoT) sensors. Vallor (2016) makes a similar point when she observes that any kind of truth always has a context: ‘We cannot respect truth by stripping it of all reference to the concrete worldly situations that make it true’ (179). She has also evoked the danger of techno-social ‘lock-in’ emphasising that we should be mindful of the ‘gradual hardening of certain technical design choices that over time become increasingly difficult to undo, modify, or improve upon, as other systems, tools and practices are built to work with that initial choice’ (Vallor, 2016: 217). Indeed, several developers we spoke to stressed the often unacknowledged importance of ‘path dependency’ in technology. Sinister Kitty, a co-founder of an IoT middleware company, for instance, explained that he and his partner sought to ‘deconstruct’ the way IoT technologies speak to one another through disrupting the very communication technologies they are part of. 22 They used blockchain in the design of their middleware to, as they explained, foreground the necessity of security.
In addition to rebuilding systems, some developers also explained that technology industry business models need to be ‘disrupted’ in order to address ongoing ethical issues. Developers who considered themselves as creating a different non-market-based/oriented social milieu often highlighted that their activities revolve around ‘pushing back’ against ‘the system’. This illustrates the idea that ‘the likelihood that knowledge will be transmitted depends on the social organisation of knowledge, storage technology and who controls access to it’ (Mokyr, 2002: 8). Here, some developers stressed the importance of paying attention to both technology development and technology transfer. For instance, a developer working on tracking devices for wildlife conservation questioned what would happen if someone took his technology and adapted it to humans. 23 He said that he could imagine implications of tracking all activities of human life, including but not limited to invasion to personal privacy, as well as expanded surveillance and control, even if he was not certain that these risks would transpire. Another developer, who created a wearable technology for individuals with speech difficulties, described that she did not want her technology to be used in the entertainment industry – despite the potential financial benefit – since this would go against ‘the whole reason’ she set up her company: helping vulnerable individuals and their families. 24 She explained it was not money but her own ideals that mattered to her. Similarly, another developer who owns an IoT design studio in Amsterdam told us how he resisted putting facial recognition technologies into his products, although clients found them ‘cool’ or ‘cutting-edge’. This play of context echoes what Vallor refers to as ‘practical wisdom’: a kind of moral excellence exhibited by individuals and groups whose lives ‘are guided by appropriate feeling and intelligence, rather than mindless habit or rote compulsion to follow fixed moral scripts provided by religious, political or cultural institutions’ (Vallor, 2016: 25).
The spaces for the exercise of practical reason that we identified demonstrated that feeling and intelligence needed to be negotiated against the expectations and constraints of the environment. This included the ability of developers to ‘go against the flow’, 25 to perceive what ethical dilemmas might emerge from the technologies they are building, or to negotiate their ethical behaviour (Vallor, 2016) within a highly competitive business environment. For instance, ‘disrupting the market’ by putting ethics first could bring financial risks because a start-up might refuse investments from funds with questionable sources of income – as well as benefits for individuals who could separate themselves from companies with values they did not share. 26 We also heard that people negotiated this play of values by participating in ‘disruptive’ movements through meetups, GitHub forums and other online discussions while holding a steady job in a company with a more ‘business-as-usual’ perspective on ethics. 27
Action positions and the possibilities of the social milieu
Building on the virtue ethics tradition, which focuses on MacIntyre’s and Anscombe’s interpretations of the practical reasoning which people use to determine a course of action that they consider to be good, our analysis shows that a range of actions are enabled and constrained by the social milieu in which people find themselves (Anscombe, 2005; MacIntyre, 1999). These constraints motivate explanations of the difficulty of acting ethically due to expectations of how their products appear in a market (the Disengaged), justifications for interpretations of ethical principles that fit with consumer interests (the Pragmatist), as well as seeking to transform the potential design of IoT systems by rethinking the metaphors and models that underpin them (the Idealist). As Vallor suggests in her investigation of virtue ethics in relation to technology, design choices for technology are one of the ways that social milieu structure potential orientations for practical reason (Vallor, 2010). Therefore, the Idealist position that we identified was not a position idealising theoretical reason or speculating on potential courses of action but rather, as with the other positions, engaged in explanations, reflections, arguments and reorientations of their own products and services.
In light of our analysis, the extent to which individual subjectivity can influence the exercise of virtues may depend on the organisational environment a developer is embedded in. Acting ethically comprises a form of knowledge that surpasses a cognitive understanding of rules and principles to encompass emotional and social intelligence ‘awareness of the motivations, feelings, beliefs and desires of others; a sensitivity to the morally salient features of particular situations’ (Vallor, 2016: 26). Such capacities to act influence decisions that might otherwise appear to be inevitable outcomes of structural tendencies, such as the decision by a start-up with an original ethos oriented towards ‘creating better things’ to change direction to address the perceived demands of funding rounds or growth. Action positions are not determinate, but situational. This means that constraints are not external things to be overcome, but intrinsic in the milieu that technology developers are part of. This goes some way to explaining how ongoing conversations about using ‘technology for social good’ or ‘business with purpose’ gain traction while technology products continue to violate privacy, intensify bias and entrench social power. It is not that developers do not have virtuous intentions, but that the milieu structures the space for action in ways that compromise moral reflection. The implications here for policy, practice and transformation of the milieu include the fact that merely introducing ethics as a concern may not allow people to act on virtues that they hold for themselves and for their ventures, or may constrain the ways that they interpret these virtues. There may be enabling strategies to consider, such as creating different ways of sustaining businesses and opening up spaces of negotiation for developers to claim different outcomes for their products other than market viability.
Conclusion
Through our examination of the IoT social milieu, we illustrated the ethical spaces outlined by three different action positions. These action positions illustrate how ethical thinking and doing unfold through justification, explanation and decisions about how to design technological systems. The three action positions illustrate how practical reasoning is practiced and emerges through individual and collective negotiations within social spaces. The action positions we observed – the Disengaged, the Pragmatist and the Idealist – demonstrate that understandings of how IoT development cultures should (or could) operate shaped how developers assumed they could act.
In particular, by situating the action positions in relation to social imaginaries of technology, we identify the ethical negotiations that underpin the construction of what appears to be possible for new technological systems. Our extensive fieldwork within the ecosystem of small companies illustrates that ethical concerns and spaces for action in IoT development are set up in relation to structural factors including assumptions about business development, policy and regulation (including the GDPR). These structural factors also include the relative power of small versus large companies, making our empirical insights important in understanding the ethical spaces of action experienced by people working in start-ups, whose position is associated with greater potential for innovation and with less influence on the structural aspects of business models for data-processing.
Our specific focus on the development of IoT systems is also significant because it identifies an important direction for broader considerations of data ethics within an emerging technology that is data-processing heavy. Following a different ethical trajectory and broadening ethical concerns to encompass practices and processes of development, in addition to consequential concerns relating to data storage, ownership or surveillance, we demonstrate how contextual and relational features of social spaces influence the capacity for people to practically reason and to act. Structural features of the data-based and connected environments including economies of data extraction lead to consequences including surveillance, which in turn have well-identified ethical implications including bias, discrimination and manipulation. These are not in question.
Our analysis shows that business interest enacts certain kinds of moral reflection and, in some cases, it is used as a legitimisation for disengagement or for a more practical turn. In such cases, consumer interest is put forth as a potential to transform the existing social milieu – that if consumers pushed for more ethical engagement from the developers, producers and entrepreneurs of emerging technologies, change might happen. Similarly, regulation is increasingly suggested as a means for transformation, that if all businesses are required to comply, then not engaging with potential ethical implications of technology design and production would cease to offer a competitive edge. We think this position postpones engagement with ethics and defers responsibility to future consumers, organisations and regulations.
Nevertheless, while there is uncertainty here, there is also space for transformation. The existence of social milieu does not prescribe that this milieu will constrain virtuous or transformative action. In fact, the Idealist action position suggests that there is considerable space to work inside social milieu to transform the expectations of business.
This paper contributes to the field by showing that the creation and maintenance of the social spaces within which ideas about how technologies can and should be designed are also part of the ethical terrain. These aspects need more focused attention, and we identify that philosophical concepts can be deepened as well as made relevant through a sustained and careful connection to the social imaginaries that motivate technology development, and the contexts where they occur.
Footnotes
Acknowledgements
This article is based on research that comes out of the VIRT-EU project: Values and Ethics in Innovation for Responsible Technology in Europe (Principal Investigator for London School of Economics is Dr Alison Powell). We thank the VIRT-EU consortium members and Research Assistant Sebastián Lehuedé for their research contributions. We thank Professor Robin Mansell and Professor Sonia Livingstone for their helpful suggestions to the earlier versions of this article. We also thank the anonymous reviewers and editors of Big Data and Society for constructive commentary.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The VIRT-EU project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme (Grant Agreement No: 727040).