Do Algorithms Shape Character? Considering Algorithmic Ethical Subjectivation

Abstract

Moral critiques of computational algorithms seem divided between two paradigms. One seeks to demonstrate how an opaque and unruly algorithmic power violates moral values and harms users’ autonomy; the other underlines the systematicity of such power, deflating concerns about opacity and unruliness. While the second paradigm makes it possible to think of end users of algorithmic systems as moral agents, the consequences of this possibility remain unexplored. This article proposes one way of tackling this problem. Employing Michel Foucault’s version of virtue ethics, I examine how perceptions of Facebook’s normative regulation of visibility have transformed non-expert end users’ ethical selves (i.e., their character) in the current political crisis in Brazil. The article builds on this analysis to advance algorithmic ethical subjectivation as a concept to make sense of these processes of ethical becoming. I define them as plural (encompassing various types of actions and values, and resulting in no determinate subject), contextual (demanding not only sociomaterial but also epistemological and ethical conditions), and potentially harmful (eventually structuring harms that are not externally inflicted by algorithms, but by users, upon themselves and others, in response to how they perceive the normativity of algorithmic decisions). By researching which model(s) of ethical subjectivation specific algorithmic social platforms instantiate, critical scholars might be able to better understand the normative consequences of these platforms’ power.

Introduction

In the growing field of “critical algorithm studies” (Seaver & Gillespie, 2016), moral problems are pervasively raised, but rarely theorized. This article is an attempt to systematize how critical scholars consider the normativity of computational algorithms and propose a complementary perspective.

It begins by suggesting that this field tends to center on what I term the moral harm paradigm. ¹ According to it, the invisible and biased operations of algorithmic power violate universal values in ways that are concretely harmful to the autonomy of individuals and communities. From this perspective, the critique of the morality of algorithms is a particularly complex, but also a relatively narrow problem: it ultimately concerns what algorithms do to people. I then identify an alternative critical paradigm, which takes a deflationary perspective toward power and its moral consequences. This strand assumes that various “actors” of “algorithmic assemblages” can, to variable extents, perceive what algorithms do—including non-expert end users. Such insight grants to these users the possibility of moral agency, I contend. This, in turn, allows for the examination of how ordinary people reflect on the normative decisions made by algorithmic power and, based on this, consciously transform their ethical selves. Such end user-oriented ethics of algorithms concerns not what algorithms do to people, but what people intentionally do to themselves (and others) in reaction to what they think algorithms do to them.

The second part of the article proposes one way of critically conceptualizing these processes. Building on Michel Foucault’s peculiar kind of virtue ethics, I examine the uncertain ways Brazilian non-expert end users have intentionally adapted their political voices to what they perceived as the normative requirements of Facebook’s algorithms. This analysis serves as both an example of how to empirically approach this sort of self-formation and a theory-building exercise. From it, I propose algorithmic ethical subjectivation as a conceptual framework to consider that specific form of ethics of algorithms. Before concluding, the article unpacks three general features of this framework: plurality, contextuality, and potential harmfulness.

Contrasting Critiques of the Morality of Algorithms

Harmful Algorithms

The notion of harm pervades the moral issues invoked by scholars in relation to algorithms. To be sure, only some of the works in this moral harm paradigm explicitly theorize harms as moral. Concerns about morality are usually implicit in critical discussions on algorithmic power. While the paradigm comports a plurality of definitions of such power, these definitions usually rely on a common fundamental injustice: one between “sorters,” “those who are able to” produce and analyze digital data, and “sortees,” “those who find their lives affected” by these analyses (Andrejevic, 2014, p. 1683). By materializing and reproducing this structuring condition of inequity, algorithmic power would violate core moral values and lead to tangible harms to sortees’ lived autonomy.

Research has demonstrated that a form of fragmented, participatory, and hypertrophied “dataveillance” (Albrechtslund, 2008; Clarke, 1988) incessantly violates sortees’ privacy to turn their behavior into readable data points (e.g., Citron & Gray, 2013). This “post-panoptic” monitoring hinges on proprietary algorithms. They probabilistically derive ever-fluid data selves, based on which corporate and governmental decision are made, but over which sortees have no control (Cheney-Lippold, 2011, 2016; Turow, 2011; Yeung, 2016). If perceived, dataveillance might harm individuals’ ability to express themselves freely; if not, it may underpin other harmful procedures (Calo, 2011). Second, undetectable algorithms are said to automate previous forms of digital “echo chambers,” giving rise to “filter bubbles” of personalized content (Pariser, 2011; see though Boxell, Gentzkow, & Shapiro, 2017; Dubois & Blank, 2018). By decreasing exposure to divergent views, facilitating the circulation of so-called “fake news” and allowing the microtargeting of voters, such violation of diversity would harm sortees’ capacity to tolerate others and make informed decisions (Sunstein, 2017). Third, “learning algorithms” can invisibly reproduce and deepen various forms of prejudiced social classification (Barocas, 2014; Lauriault, 2017), manifesting a new form of “rational discrimination” (Gandy, 2009; McQuillan, this issue). The consequences of these violations of fairness are compounded as they tend to inform an increasing number of non-algorithmic decisions, harming people’s “life-chances” ² (Angwin, Larson, Mattu, & Kirchner, 2016).

Arguments on moral harms can be seen as premised on three implicit assumptions. The first is definitional. This paradigm assumes algorithms to be digital opaque artifacts, controlled by sorters and constituted by three elements: (1) biased computational code which analyze (2) biased digital data to find patterns and invisibly make, or assist in the making of, (3) normative decisions on behalf of sortees: what they ought to see, with whom they ought to interact, what they ought to know, and so on. As theories that identify, in the unknowability of power, the essence of its efficiency (Lukes, 1974), the opacity of algorithms is assumed as a central constitutive component of the moral harms produced through any of their elements. As lack of “technical literacy,” “corporate or state secrecy” or a mismatch between human and artificial heuristics (Burrell, 2016, p. 1), opacity poses an unprecedented threat to the capacity of users, academics, and policy-makers to be aware of, study and regulate digital platforms, scholars agree (cf. Pasquale, 2015).

This discussion matters because works in this paradigm tend to assume that the opacity of code, data, and algorithmic decisions entails an asymmetrical distribution of moral agency between sorters and sortees. If human moral agency is the capacity of an individual to govern her behavior according to moral standards (Himma, 2009, p. 21), sortees could only be agents if they could intentionally entertain moral considerations about their relationship with algorithms. However, ignorant of or poorly informed about these artifacts, these users emerge from this paradigm mostly as victims. That is, as individuals who eagerly engage with their unknowable perpetrators without being able to consider the moral implications of such engagement’s algorithmic component. ³ On the other hand, due to their considerable (not limitless) comprehension of and control over algorithms, sorters appear in this paradigm to be endowed with a vast moral agency. Therefore, their actions toward these artifacts would be amenable to moral critique and accountability.

The third assumption regards which conception of morality can make sense of the relations between those particularly defined “artifacts” and shaped moral subjects. While authors rarely pick philosophical positions, this paradigm appears to operate between consequentialism, as suggested by the focus on harms, and deontology, as indicated by the recourse to universal moral values as a measure of good (see Sandvig, Hamilton, Karahalios, & Langbort, 2016). One way of making sense of this hybrid conception is with what Bernard Williams (1985) called “morality system” (p. 174). Characterized by the centrality of general obligations, this third-person perspective of the moral, of which both deontological and consequentialist accounts are part, stresses the importance of a distinct view of blame—one that, conceived of as objectively founded, overlooks the contexts of moral actions. This paradigm frequently assumes these contexts to be either mostly inexistent (sortees’) or inaccessible (sorters’). This article argues that the notion of harm can be treated from a different conceptual perspective, founded on distinct assumptions about what algorithms are, the ways they affect moral agency, and how morality can be conceived of. I turn to these problems below.

The Deflationary Paradigm

An alternative paradigm within critical algorithm studies is less interested in denouncing moral harms than in complicating what algorithmic power is and how it can be tamed, taking thus a “deflationary” stance (Neyland, 2016; Ziewitz, 2016). Usually relying on ANT (Actor-Network Theory), it defines algorithms as assemblages of human and non-human actors engaged in mutually constitutive relations (Scott & Orlikowski, 2009). From this view, studying one actor in isolation (say, code) is in principle misleading because actors’ attributes are not possessed, but enacted—there is no “inside” to be objectively “revealed” (Ananny & Crawford, 2016). This co-constructivist take and expands what counts as these assemblages’ actors. They include not only those controlling data collection, code design, and algorithmic decisions but also users of platforms. These may be activists (Milan, 2018), search engine optimization specialists (Ziewitz, 2012), media professionals (Gillespie, 2017) and, of specific interest for this article, non-expert end users. ⁴ There is an emerging, empirically based understanding that this kind of user can perceive the inclinations of behind-the-screen content curation. Termed “folk theories” (Eslami, Vaccaro, Karahalios & Hamilton, 2017) or “imaginaries” (Bucher, 2016), such lay comprehension is depicted as informing how algorithmic power operates. Echoing earlier sociological accounts of quantification (e.g., Espeland & Stevens, 2008), this paradigm argues that these perceptions explain how ordinary individuals behave differently from what would be expected in the moral harm paradigm. They might internalize “theories” and “imaginaries,” self-regulating their actions to comply with what they think algorithms expect (Introna, 2016), devise strategies to game the system (boyd, 2017), and employ these perceptions to frame their resistance to platforms’ unilateral changes (DeVito, Gergle, & Birnholtz, 2017).

It is my contention that this other definition of “algorithm” enables us to challenge the extent to which non-expert end users are devoid of moral agency. For, if these individuals can somehow apprehend (theorize, imagine) platforms’ algorithms, they can also somehow entertain moral standards when consciously acting in response to algorithmic decisions. This possibility entails a proliferation of empirical entry points into the normativity of algorithms. Data, code, and some of the most relevant sorters might still be inaccessible—but ordinary people’s perceptions and practices are some of the prototypical objects of analysis of qualitative researchers. They might be observed via ethnographic work and explored through self-reporting methods (Couldry, Fotopoulou, & Dickens, 2016; Kitchin, 2016; Markham, 2017). The deflationary paradigm, and in particular Ananny’s (2016) insightful piece, do hint at these reconfigurations of moral agencies—usually to discuss algorithmic accountability. But they neither take account of these other ordinary entry points nor unpack the “somehow”s mentioned above. The rest of this article draws on a different conception of what has been called so far “morality” to propose one way of doing both.

An End User-Oriented Ethics of Algorithms

The investigation of this moral agency qua an element of algorithmic social platforms ⁵ might be termed an ethics of algorithms, but oriented toward non-expert end users of platforms. Bernard Williams posits that an “ethics”—as opposed to “morality”—enables a first-person, “ethnographic stance” (Williams, 1986, p. 204) toward the irreducible and multiple contexts shaping how individuals practically address the Socratic question (how should one live?) and, in so doing, shape their character. In relation to algorithms, this might mean critically investigating a basal process which has been so far neglected by the reviewed literature: how non-expert end users’ perceptions of platforms’ algorithms are related to the construction of their ethical selves. But, differently from Williams, I do not downplay concerns raised by a moral view. As defined here, the ethics of algorithms sits between the two paradigms. On one hand, it assumes that algorithms can only be understood relationally—and are perceivable by users. On the other hand, it attempts to rearticulate the problem of harms—but by nesting it within a conceptualization of ethics that is interested in the agency of those who, so far, have been mostly assumed to be passive targets of sorters’ actions. In this way, a “morality of algorithms” and an “ethics of algorithms” are complementary, not opposing.

As it becomes clearer below, Michel Foucault’s critical form of virtue ethics is particularly well-positioned to tackle the sort of experience I am interested here. Foucauldian ethics has become influential among philosophers of technology (Dorrestijn, 2012) and anthropologists (Faubion, 2011; Laidlaw, 2014), but has not been openly explored by Foucauldian-inspired scholars of algorithmic power (e.g., Bucher, 2012; Introna, 2016; Reigeluth, 2014; Rieder, 2017). The projection of his view onto the said self-formation process makes up what I call algorithmic ethical subjectivation. ⁶ Detailing it demands, first, a detour to Foucault’s thought.

Foucault as (Virtue) Ethicist

While not usually labeled as a virtue ethicist, Foucault came to share with virtue ethicists the belief that the ethical phenomena could be better grasped by studying how “one ought to form oneself as an ethical subject” (Foucault, 1984, p. 26; see Levy, 2004, for an extended treatment of these similarities). Two aspects of Foucault’s take on ethics interest me.

First, as with much of feminist moral theory (e.g., Tronto, 1993), his view is integrally built upon a critique of power. Arguably Foucault’s central preoccupation, the formation of the subject in relation to evolving structures of control was viewed differently by him in the last part of his life (Koopman, 2013). The earlier oppositional relation between freedom and power that marked “the dark Foucault” (Ortner, 2016, p. 51) was replaced by an association of co-dependency. Government (“the way in which the conduct of individuals or of groups might be directed”—Foucault, 2000, p. 341), and self-government (“the government of the self by the self”—Foucault, 2000, p. 364) became generatively tied. Therefore, his “ethical subject” is neither autonomous (as the Kantian universal legislator) nor dominated by ideological normative stances (as the Bourdieusian subject; see Pellandini-Simanyi, 2014). Rather, she is the provisional and unstable product of the relations between the freedom to act and the power-laden codes of conduct which she must consider when deciding on what to do and who to be. Ethics is thus the reflexive practice of freedom (Foucault, 1994, p. 284).

Second, Foucault proposes specific “parameters” to examine how distinct ethical subjects emerge in relation to the same codes of conduct and change over time. The first parameter of subjectivation is the “ethical substance; that is, the way the individual has to constitute this or that part of himself as the prime material of the moral conduct” (Foucault, 1984, p. 26). It might regard one’s sexuality (Foucault’s object), political subjectivity (as this article will argue), or any other dimension informed by moral considerations. The second one is the “mode of subjectivation,” or “the way in which the individual establishes his relation to the rule and recognizes himself as obliged to put it into practice” (Foucault, 1984, p. 27), also defined by him as “the way in which people are invited or incited to recognize their moral obligations” (Foucault, 1994, p. 264). It cannot be mistaken for any objective code of conduct—in reality, it suggests the process of understanding a rule and justifying its application. That is, while not prescriptive—the word “virtue” is mostly absent from Foucault’s writings—his take presumes individuals’ lives to be pervasively influenced by an ensemble of normative experiences and reasoning. Third, there is the “ethical work [. . .] that one performs on oneself, not only to bring one’s conduct into compliance with a given rules, but to attempt to transform oneself into the ethical subject of one’s behaviour” (Foucault, 1984, p. 27). These practices, directed by and toward the individual (but with consequences for others), have also been called technologies of the self. And this leads to the fourth parameter: “ethical telos”, the ethical position a subjectivation aspires to—which is crucial, since “a moral action [. . .] tends to the establishing of a moral conduct that commits an individual [. . .] to a certain mode of being” (Foucault, 1984, p. 28). These parameters also help us to understand what constitutes such subject after all—goals, practices, perceptions.

Two Stories of Ethical Becoming

Following Faubion’s (2011) empirical analysis of these parameters and the methodological tradition interested in analyzing the relations between identity and culture through narratives of self-development (e.g., Sandelowski, 1991), I propose that algorithmic ethical subjectivations might be studied through the stories of how individuals have transformed themselves on and through algorithmic social platforms, as perceived and recounted by themselves.

Based on this definition, I examine the stories of how two non-expert end users’ ethical selves were transformed by their political usage of Facebook. These two individuals shared their stories with me in São Paulo (Brazil) in early 2017. Their interviews are part of a larger research project on the emergence of novel forms of political citizenship among ordinary Brazilian individuals on and through Facebook during Brazil’s current political crisis. ⁷ I conducted semi-structured intensive interviews, asking participants about what they see, how they are seen, and what they do to become more or less visible when talking about or doing politics on Facebook. The stories compose an analytic exercise whereby I exemplify how algorithmic ethical subjectivation may be researched and revisit some elements of Foucault’s original theory to explore what they might mean in relation to an end user-oriented ethics of algorithms.

Unexpected Viral Episodes

The first individual is Cesar, ⁸ a politically progressive Black man raised in the impoverished outskirts of São Paulo. In 2016, he published on his own Facebook timeline a text criticizing the elitism of Brazilian leftists. Although he had a small network of Facebook friends, the post caught the attention of some “influencers—mostly journalists,” Cesar said, and ended up having more than 17,000 reactions, 6,000 shares, and 600 comments. He believes that, in total, around 1 million people were exposed to the text. Overnight, he had thousands of friendship requests. “I had never experienced the power of the viral,” he told me. Until that day, he recalled having a “banal presence” on Facebook, seldom writing on politics. At the time, he was working as a clerk, but wanted to become a writer. “After that, I thought: what now?,” he said, adding that he figured out that the sudden visibility was a chance to somehow change his life.

The second individual is Sheila, a White woman also raised in a poor neighborhood of São Paulo. Her upbringing was mostly apolitical, and until some years ago the main subject of her posts was the Carnival. In the beginning of 2015, she posted a short video of herself. In it, Sheila, who is politically conservative, argued that Brazilian right-wingers should unite around one large street protest and avoid fragmentation. The video went viral after being unexpectedly shared by a journalist, what brought to Sheila waves of new followers. From 2013 until that moment, she had been talking about politics on Facebook “out of curiosity.” But, as with Cesar, she saw the event as an opportunity. In her case, to try to disrupt what she calls the Brazilian “communist establishment,” which she said she hates.

They both told me that, if they were to seize the chances they envisioned, they would have to, at some level, transform themselves to avoid disappearing in the cacophony of Facebook. Let me explain why this transformation can be conceived of as ethical.

Ethical Substance: Political Voices

The first parameter is the “ethical substance,” that is, the component of the self that is altered through a subjectivation. As explained below, the stories of Sheila and Cesar suggest that their conscious attempts to continue to be heard consisted centrally in the transformation of their political self-expression on Facebook. (None of them offered explanations or justifications for this, what suggests that this choice was not weighted, or that they did not even perceive it as a choice.) Here, “self-expression” may be conceptually approached through the notion of “voice.” By this term I mean, with Couldry (2010), the capacity of “giving an account of oneself” and the obligation to value frameworks and decisions which “themselves value voice” (pp. 1-2). And we can term it “political” in as much as, through this “account,” one positions oneself in relation to a somehow public matter in a somehow public channel. Defined as such, political voice is inherently ethical. For a proper political subjectivity depends on one’s capacity of not only speaking, but having her political voice heard (Honneth, 1996; Dayan, 2013). It is certain that visibility does not automatically grant voice (more on this shortly). What matters, at this point, is that Cesar and Sheila understood that, to be properly heard, they had to be visible on Facebook. This leads to the second parameter of subjectivation.

Mode of Subjectivation: Speculative Personal Codes of Conduct

In their struggle for visibility, Cesar and Sheila told me that they had to consider the following question: what/who one ought to be/do to become more visible on Facebook? Their narratives indicate that this question entails three separate problems.

Is There a Rule Defining How Visibility Is Conferred on Facebook?

Despite having no technical training, they were differently aware that a non-random rule defines how visibilities are regulated on the platform. In our talk, Cesar demonstrated a sophisticate and critical understanding of what he named “the algorithm,” “a set of rules to [supposedly] make the user’s experience better,” but whose real objective is “keeping you connected to Facebook,” for commercial purposes. Sheila described a mechanism that works as a “filter”: “[Facebook records] what I see more often and then tries to filter [what I will see].” Cesar had read about algorithms in books and journalistic articles; Sheila had not. But they both told me that it was through the practical engagement with Facebook that they better understood how the mechanism works (see Bucher, 2016). Cesar told me that he ran “some tests,” changing aspects of his posts to assess whether these shifts increased their visibility. “It is just logic,” Sheila said when explaining how she learnt that the “filter” started to show her more political content after she began posting and interacting with political content more often.

Who or What Governs Such Rule?

Neither Cesar nor Sheila indicated the “algorithm” or the “filter” as governed exclusively by Facebook. They perceived themselves as also having some control over it. Cesar, for example, said, “We are the ones who generate its [the algorithm’s] behaviour. Sometimes you make choices—click on certain things. I have some control. I choose not to see some contents, and ask to see some people first.” Sheila was also aware of her ability to manually change how the “filter” works, and used it to block content on cruelty against animals—but not on politics. “About politics, I want to see everything. It does not matter if it pleases me or not.”

What Does This Rule Define as Obligations?

Since both Sheila and Cesar suggested that visibility is not managed unilaterally by an “algorithm” or a “filter,” but feeds on interactions from end users, their perceptions coalesced around one general obligation: to be seen more often, one must make people “like”, comment on, and “share” one’s Facebook activity. Since the struggle for visibility entails the struggle for engagement metrics, the manipulation of how the “filter” manages their own visibility involved active attempts to steer other users’ responding actions. This general obligation is then realized through more specific duties, they told me. For Cesar, the most important duty involves readiness. To be visible, one is obliged to have opinions about all topics, particularly political ones, given the political crisis in Brazil. A second duty regards optimizing the timing of activities—to get visibility, one must post in “peak times” and many times per week: “The times that you post, the subject that you speak about . . . It might not go viral, but changes the repercussion [of the post].” For Sheila, to increase one’s visibility, what she calls “populism” is essential. “Populism,” for her, involves “zoeira,” an idiomatic Brazilian expression which indicates the use of humor, falsehoods, and exaggerations to attack an opponent. “Without ‘zoeira,’ it does not go viral. [But] it’s a massive futility, just a gratuitous aggression.” In sum, while the basic tenet of Facebook’s algorithmic logic seemed to be intelligible to them both in quite similar ways (as reflected in what I called “general obligation”), the reasons of this tenet were opaquer, thus subject to a wider variety of interpretations (the “more specific duties”).

As such, their perceptions gained a clearer ethical meaning when they became speculative personal codes of conduct. By this I mean that these “codes” were constituted as an abstract and individualized (as opposed to written and shared) bundle of duties, derived from concrete observations and information but working according to a unifying logic which could only be speculated (as opposed to be known).

Ethical Work: Immoderation/Moderation of Political Voices

These codes guided their ethical work—but did not impose compliance. Cesar initially employed “marketing techniques” to engage followers. The once politically quiet man transformed his Facebook voice, he said. He began to post several times a week and express opinions about the details of the Brazilian political crisis. It worked: he managed to turn many of his posts “viral” and cemented what he came to see as a loyal public. But then the hectic involvement with his own new practices of expression and new readers prompted ethical doubts. Cesar told me,

You start to ask yourself: I am going to go viral with this post, but going viral for what? You learn you must [verbally] assault a politician to be seen, then you do that over and over again to remain visible. Is this ethical? I have my rules on how to write [and they] are not the rules of Facebook.

He sensed he no longer needed to aggressively pursue visibility, and refrained from it. “It is not really honest. I thought I should have a more responsible presence on the Internet.” This involved talking less and less controversially.

Sheila also transformed her political voice on Facebook. She posted almost exclusively about politics (rather than the Carnival), created two fan pages, and began to publish conservative content (including memes of her own creation) at least three times a day—which requires professional-like discipline, she said. In line with her own code of conduct, she constantly attacked Brazilian “lefties.” She gave me the example of how calling feminists “potential whores” was an efficient technique. “I do not really think all feminists are potential whores. But who cares? This is what goes viral.” At any rate, these aggressive practices triggered a snowballing effect: the more she attacked, the more she got attacked, which increased her willingness to keep attacking. “I became more and more polarized.” This is not to say that she enjoys the criticisms—or even “personally” wants more visibility. But, according to her, those are the “price” to be paid for her project of undermining “communism.”

Both Sheila and Cesar used techniques I would term immoderate. To increase their visibility, they talked more often and more aggressively. However, the differences between their trajectories point to the indeterminacy of subjectivations. Despite having similar notions about Facebook’s algorithmic management of visibility, their individual conduct evolved differently. For Cesar, these immoderate practices of self-formation prompted critical self-reflection and led him to adopt an opposing set of practices, aimed at self-control—in consonance with his belief about what counts as an “honest” voice. Sheila, on the other hand, is conscious that her immoderate practices have radicalized her, but is unwilling to halt them due to the political quest she finds herself immersed in.

Ethical Telos: Two Modes of Political Voices

Sheila and Cesar initially seemed to share the same ethical telos of political recognition—they aspired to be not only visible, but recognized as political subjects. However, by engaging in the practices incentivized by this logic, their similar initial telos got bifurcated. For Cesar, visibility and recognition drifted apart. The more he struggled for engagement metrics to become algorithmically visible, the more he got consciously uncomfortable with the ethical consequences of this struggle. He came to realize that what he aspired to was not to simply “go viral,” but be recognized as a responsible political voice—even if this meant limiting his reach. Sheila also reflected on her actions, as evidenced by her awareness of the questionable ethical nature of her visibility-enhancing strategies. But this reflection did not lead her to curb her voice. Instead, she got consciously radicalized. By struggling for visibility, she realized she aspired to be recognized as an antagonistic political voice—even if this meant being attacked by those she antagonizes with.

Algorithmic Ethical Subjectivation: Three Features

However limited, the analyses of Cesar’s and Sheila’s stories allow me to trace the contours of algorithmic ethical subjectivations. I understand them as the plural, contextual, and potentially harmful processes through which non-expert end users, guided by the normative meaning they assign to social platforms’ algorithmic decisions, intentionally engage in ordinary reflective practices to transform elements of their selves and become who they aspire to be. While congruent with or explicitly contemplated by Foucault’s view, plurality, contextuality, and potential harmfulness gain determinate forms when the subjectivation is algorithmic, as I propose next.

Plurality

From my brief analytical exercise, algorithmic ethical subjectivations emerge as triply plural. First, algorithmic power does not seem to automatically “produce” the same kinds of subjects. As exemplified by Cesar’s and Sheila’s trajectories, different ethical works and telos might unfold in relation to the same algorithmic logic—and even change during the process. Second, algorithmic subjectivations may involve an axiological plurality. As such, they can gravitate not only toward privacy, diversity, fairness, and autonomy but also to other values such as voice. The study of algorithmic ethical subjectivations should thus be sensitive to all forms of duties, goods, and virtues, which are at stake in specific processes. Third, there is a plurality of ethical actions. Drawing the exact boundaries of the ethical domain is a lingering difficulty (Cassaniti & Hickman, 2014). It might, but need not be limited to certain special moments commonly thought of as “ethical”—for example, the violation of rules and the assignment of responsibility. With Williams (1985), I argue that transformations of the self in response to algorithmic decisions will be ethical every time they are intentionally entangled with the question of how one should live. And with Das (2017), I see these actions as ethical even when their authors do not name them as “ethical,” as Sheila did not. Such pluralities might be explained by how an algorithmic ethical subjectivation can only take place in a multiply contextualized manner.

Contextuality

Invoking “context” to discuss digital datafication is hardly new (boyd and Crawford, 2012; Seaver, 2015). Yet most of the approaches concern the actions presided over by sorters, and how policymakers and technologists can prevent or ameliorate harms (e.g., Nissenbaum, 2010). The notion of context is differently defined here. For me, they are conditions, which, while not depending on end users’ algorithmic ethical subjectivations to exist, make these processes possible.

Predefining which contexts are the “correct” ones is a losing game (Burke, 2002). But some appear to be necessary to my discussion. The central necessary context comprises the decisions algorithms make on behalf of end users (e.g., Facebook’s news feed). From the non-expert end user perspective, they are a form of sociomaterial context instantiated not through complexes of computational code and digital data, but on a deceptively simple interface. It is the deliberate attempt by an individual to change herself in response to what she speculates as the logic behind these decisions that defines an ethical subjectivation as “algorithmic.” Even when she ignores the term “algorithm,” as Sheila did. This kind of context encompasses thus a direct consideration of the power laden-structures which “are proposed, suggested, imposed” to users (Foucault, 1994, p. 291).

Another necessary context is epistemological, which allows individuals to assign normative meanings to algorithmic decisions (and other contexts). They include, for instance, not only self-reflexivity and critical thinking but also literacies and shared narratives about algorithms. Consider, for instance, the personal characteristics, skills and information that allowed Sheila and Cesar to understand Facebook’s regulation of visibility in the way they did.

Finally, there are ethical contexts. By this, I mean the individuals’ set of moral dispositions and beliefs which must be in place before going through any transformation (e.g., Cesar’s notion of honesty). Other contexts are necessarily present, but not necessarily invoked in all algorithmic ethical subjectivations. By this I mean, for instance, “sociopolitical” contexts—which also point to power structures. It includes broad political circumstances (e.g., the Brazilian crisis), which framed the processes I analyzed above, but also, societal classifications, such as class, gender, and race. Cesar’s subjectivation, for instance, was partly motivated by his wish to transcend the place he was assigned in Brazil’s acutely classist and racist social order. There are also contexts which involve personal emotions and desires, which routinely and consciously inform individuals’ moral deliberation and actions. Take Sheila’s hatred against Brazil’s “communists,” which fueled her quest for visibility.

Therefore, while the normative meaning assigned to algorithmic decisions (the said “mode of subjectivation”) is the lodestar of the process I have been discussing so far, it cannot, in isolation, explain such a process. “Contextuality” means not only paying attention to which contexts play which role in which subjectivation but also to how these roles are shaped by the relations between contexts. If algorithmically mediated communication is neither “fundamentally liberating” nor “wholly exploitative” (Mansell, 2017, p. 46), a rigorous attention to these contexts might help us understand under which conditions subjectivations become harmful, as explained next.

Potential Harmfulness

According to the moral harm paradigm, harms are directly inflicted upon sortees by the power of external algorithms to invisibly analyze data and make (or assist in the making of) normative decisions. My proposal points to a different mechanism.

When individuals translate the normativity of algorithmic power into personalized codes of conduct, such power does not directly and necessarily produce harm. Rather, it might incentivize end users to perform harmful acts. The stories analyzed above indicate two modalities of these acts. The first one is self-inflicted. Consider Cesar’s trajectory, and how he initially undermined his own understanding of what is “ethical” to get his political voice heard. The second is inflicted on others by someone going through an algorithmic ethical subjectivation. This is exemplified by Sheila’s conscious attempts to increase her visibility by demeaning political opponents’ dignity. These kinds of harms add conceptual and political complexity to discussions of accountability. Conceptually, it becomes more difficult to discern to what extent non-expert end users are ethically responsible for harms practiced by themselves, but made possible by an algorithmic logic they are not responsible for (Gerlitz, 2016). Politically, sorters can further exploit end users’ moral agency as a discursive pretext to shift blame (Sandvig, 2015).

The framework this article has advanced does not offer straightforward responses to these difficulties. But it does provide largely unexplored analytical avenues to recognize and understand the role played by algorithmic contexts in these types of harms. Such critique would look into which “models” of subjectivation are “proposed for setting up and developing relationships with the self” (Foucault, 1984, p. 29) on different algorithmic social platforms. That is, how the sociomaterial contexts enabled by platforms, always vis-à-vis other contexts, incentivize and suppress certain subjectivations.

While I cannot properly answer here which ethical subjectivation model(s) Facebook’s algorithmic regulation of visibility “proposes,” my empirical analysis suggests a path. This regulation appears to invite users to believe that having their political voice heard (being recognized) comes at the cost of a basic disposition needed for a proper coexistence with others (the willingness to recognize opponent’s dignity). As if the proper constitution of the political self was detrimental to the constitution of the political other. This of course echoes a core concern of the moral harm paradigm. However, it primarily undermines not a universal value (diversity), but a mode of being ethical (a telos) that aspires for the possibility of a community. The resulting “invitation” to reject a communal telos might not be irresistible (e.g., Cesar), but is certainly alluring (e.g., Sheila). This is a stunning contradiction to Facebook’s stated mission of bringing “the world closer together” (Zuckerberg, 2017).

Conclusion

Much of the critical literature “positions ‘the algorithm’ as the thing to be explained” (Gillespie, 2016, p. 53). This is usually the case with approaches concerned with how opaque algorithms produce moral harms. An alternative view has argued that, in fact, social platforms’ algorithms can “become legible, meaningful, and contested” to some of their ordinary end users (Gillespie, 2016, p. 69). But the ethical ramifications of this possibility have been largely ignored. This article attempted to theorize one of them.

I have argued that the normative meaning end users assign to algorithmic decisions has constitutive ethical properties. It may orient how they intentionally change certain components of themselves through certain practices on themselves to achieve a certain mode of being ethical. That is, it may transform who they are as ethical subjects. Studying these algorithmic ethical subjectivations, as I have termed these processes, entails two movements. First, decentering algorithms and their controllers in favor of end users’ multiply contextualized realities and perceptions. Second, pausing consequentialist and deontological assumptions in favor of a critical virtue ethics—such as the Foucauldian thought I built on.

My approach suggests that, once the meaning produced by end users is taken into consideration, the idea that algorithms mold users’ identity despite their control should be nuanced. As my examination of the stories of ethical becoming suggests, the objective logic behind algorithmic decisions neither imposes any precise meaning about them nor defines how such meaning is acted upon by an end user. That is, we can alter how we conduct ourselves as a deliberate response to algorithms’ perceived attempt to conduct us. But this process cannot be explained away by the built-in normative leanings of these algorithms. It does not follow that algorithmic decisions do not engender harmful subjectivations, but that these harms are neither necessary nor necessarily inflicted upon users from the outside. Instead, it might be that by trying to comply with what we think algorithms want from us, users consciously act in ways that harm themselves’ and others’ autonomy.

It is evident that the usefulness of my proposition is limited by individuals’ awareness of platforms’ inner workings. Therefore, some of the most fundamental consequences algorithms are involved in remain best addressed by a third-person conception of morality. Consider violations of privacy and fairness, much of which are operationalized beyond the very possibility of end users’ perceptions. What algorithmic ethical subjectivation does offer is a distinct point of departure to broaden what can be known about the “inherent humanness” (Lemov, 2016) of datafication normativity.