Thinking Outside the Black-Box: The Case for “Algorithmic Sovereignty” in Social Media

Abstract

This article is an interdisciplinary critical analysis of personalization systems and the gatekeeping role of current mainstream social media. The first section presents a literature review of data-driven personalization and its challenges in social media. The second section sheds light on increasing concerns regarding algorithms’ ability to overtly persuade—and covertly manipulate—users for the sake of engagement, introducing the emergence of the exclusive ownership of behavioral modification through hyper-nudging techniques. The third section empirically analyzes users’ expectations and behaviors regarding such data-driven personalization to frame a conceptualization of users’ agency. The fourth section introduces the concept of “algorithmic sovereignty.” Current projects that aim to grant this algorithmic sovereignty highlight some potential applications. Together this novel theoretical framework and empirical applications suggest that, to preserve trust, social media should open their personalization algorithms to a social negotiation as the first step toward a more sustainable social media landscape. To decentralize the immense power of mainstream social media, guarantee a democratic oversight, and mitigate the unintended undesirable consequences of their algorithmic curation, public institutions and civil society could help in developing and researching public algorithms, fostering a collective awareness so as to eventually ensure a fair and accountable “algorithmic sovereignty.”

As algorithms shape what we read about and how important we consider different issues, they should be recognized as an extension of our will. We should be able to consciously build our own algorithms, change them whenever we want, and not have to delegate this decision to a commercial entity with opaque functioning and objectives.

—tracking.exposed manifesto ¹

Online personalization is our interface with the infosphere. It is essential to managing information overload so as to preserve individuals’ time and attention. It is, however, an imperfect tool, subject not only to technical challenges but, in particular, also to commercial and political manipulations. Today’s social media landscape is indeed characterized by an oligopolistic market in which corporate non-state actors treat users’ attention as a commodity in what has been described as “surveillance capitalism” (Zuboff, 2015). A small group of platforms—above all Google, YouTube, Twitter and Facebook—act as the ultimate gatekeepers for billions of users worldwide. They provide a personalized experience, but little individual or group user control over information filtering processes (Harambam et al., 2018). This control asymmetry translates into an industry enriched by collecting, attributing, and inferring data points. The behavioral analysis produced perpetuates an epistemic imbalance in which platforms know people more than people know themselves. As such, social media platforms can exploit people’s vulnerabilities. To sustain this market, short-term pleasure, soft news, homophily, narcissism, entertainment, and advertisement predominate social interactions. In general, the ability to engage users is increasingly of concern in various contexts including possible negative psychological effects (Verduyn et al., 2017), the development of addictive usage patterns (Deibert, 2019), the potential filter bubble effect (Pariser, 2011) and its role in reinforcing echo chambers (Sunstein, 2017) and, in turn, in spreading misinformation (Del Vicario et al., 2016).

In this article, an interdisciplinary critical analysis of the algorithmic black-box problem in social media is provided. As such, we draw from various disciplines, in particular from media and data protection law, sociology and psychology, design and political theory. We conclude highlighting the need for more public discussion about improving data-driven personalization practices in social media through a radical user empowerment and social scrutiny. The article is structured as follows; first, a literature review of data-driven personalization in social media examines its challenges and risks. In particular, algorithms’ ability to persuade and eventually manipulate users for the sake of engagement is critically analyzed. Next, an empirical and critical literature review analysis of the expectations and behaviors of users regarding personalization shows how, to fulfill civic rights and duties, the average user cannot be fully responsible to navigate alone all the complexities of online data-driven personalized media consumption. Finally, we introduce the concept of algorithmic sovereignty, preliminarily evaluating its strengths and weaknesses with respect to current potential applications.

Personalization and Its Challenges

Personalization is ubiquitous online and it shapes every social media engagement as it filters and prioritizes information to make it “fit” each user’s profile. As such, it legitimizes social media’s gatekeeping role. User interests become more important than content quality or social significance (DeVito, 2017). This rhetoric makes data-driven personalization very appealing in our hyper-individualized societies. Yet, social media platforms are not just communication companies that uniquely shape social networks and their dynamics processes, but also large advertising agencies driven by profit (Fuchs & Marisol, 2015). Personalization can in fact be understood as pursuing the logic of market segmentation until each individual user is reduced to a unique market (Yeung, 2018a).

This process occurs largely beyond the control of users as it is based on implicit personalization—behavioral data collected from subconscious activity ² —rather than on deliberate and expressed preferences, and this might become a default choice in future personalization (Thurman & Schifferes, 2012). ³ In general, engineers’ approach can take two directions: build more advanced AI techniques, or simplify the user interface to convert the recorded human activity in a defined set of possibilities, reducing complexity (Harris, 2019). Modern advancements in technology use both. This inevitably raises several concerns.

To begin with, profiling technologies that allow personalization create a kind of knowledge that is inherently probabilistic. They cannot produce or detect a sense of self. They can, however, influence the individual’s sense of self, especially in everyday social media information consumption. This is concerning for two main reasons.

On the one hand, since long has been warned that the development of identities might be undermined by profiling algorithms that are stuck in the past, that may steer individuals to conform to the status quo of their past actions chosen by past selves. Individuals may start to want what is recommended to them without even recognizing it, in a sort of self-fulfilling prophecy. This may perpetuate existing inequalities and other pathologies, narrowing the “aspirational self,” while threatening the foundational microeconomic principle of preference formation and modification. ⁴

On the other hand, profiling is a matter of pattern recognition, which is comparable to categorization, generalization and stereotyping. Personalization thus relies on the comparability or even sameness (by some simplified categories) of the user with others (e.g., collaborative filtering). As such, personalization paradoxically denies individual uniqueness through a “smart” homogenization that negotiates the diversity of humankind (Yeung, 2018a). It is in this context that is particularly discussed the notion of “group privacy” as the right that is held by a group to safeguard information that is not only private to individuals, but which reveals something about that same group (Taylor et al., 2016).

Related to the above concerns, personalization of media content can limit the diversity of information, from exposure to discovery. Yet, information filtering processes take place on three intertwined levels; not only technological (e.g., affordances) but also individual (e.g., selective exposure) and social (e.g., sharing practices). The major risk is the creation of “informational bubbles”: at the individual level, filter bubbles (Pariser, 2011) and, at the group level, echo chambers (Sunstein, 2017). The crucial difference is that the former may not depend on user’s autonomy and awareness (Bozdag & Timmermans, 2011)—therefore is mainly caused by technological affordances—while the latter pre-existed digital age and thus it is primarily individual and social. These phenomena, however, are poorly defined and, thus, are used more as metaphors.

Yet, personalization—in particular news personalization—could reduce opportunities to self-determine and could negatively affect information finding by reducing the exposure to alternative points of view in the “marketplace of ideas” (Pariser, 2011; Sunstein, 2017) and, more generally, to serendipitous encounters (Reviglio, 2019b). There could be several consequences; from the limitation of personal creativity to a reduction in the ability to build productive social capital. Moreover, personalization could weaken media pluralism, solidarity and make people more politically polarized, narcissistic and vulnerable to (self)propaganda. This, in turn, would make people more susceptible to polarizing messages, help to spread misinformation, and ultimately, erode interpersonal trust (Del Vicario et al., 2016).

Critics, however, argue that these are mere moral panics, and that personalization allows the media to inform people better and more effectively while fostering the cultivation of expert citizens with stronger group identities. In any case, personalized techniques enable potentially manipulative political microtargeting (Kohl et al., 2019). Even so, a prominent concern remains “epistemic inequality.” That is, the wealthier the social networks, the higher the education level, meaning more personalization benefits, and vice versa.

Currently, research on personalization is more often contradictory, ambiguous, and thus unreliable (Bodo et al., 2017; Tucker et al., 2018; Zuiderveen Borgesius et al., 2016). The risks of social media usage (Verduyn et al., 2017) and filter bubbles (Zuiderveen Borgesius et al., 2016), in fact, are very hard to prove and, eventually, to mitigate. Most research is ultimately inconclusive because it is generally survey-based, or is correlational or based on a small or unsatisfactory sample. In light of the fast-changing media landscape, many studies become rapidly outdated. This contributes to the current crisis with regard to the study of algorithms (Bodo et al., 2017). They are indeed “black-boxed” (Pasquale, 2015) which means their functioning is opaque and their interpretability ⁵ may not even be clear to engineers (Albanie et al., 2017). While insights on the main causes and risks of personalization has been currently understood (Tucker et al., 2018), we still lack evidence with regard to the extent of their consequences. Given the heterogeneity of users, in fact, causes and effects of personalization vary widely. Yet, concerns are growing because of the rise of persuasive technologies and the ability of Big Data to nudge individuals, make them addicted and ultimately deceive them (Yeung, 2018a; Zarsky, 2019).

Another related theoretical issue with personalization and its future development is that they can increasingly gratify users to the extent to which they come to accept manipulation as benevolent. This argument has been discussed in terms of “psychological hedonism” (Gal, 2017): if personalized systems become “pleasure machines,” able to predict our choices and simply grant them to users, are these users willing to give up their autonomous choice? And, if so, under which conditions? Indeed we need to think seriously about the way in which the choices we make affect our values, identities, and the meaning and content of our lives. In other words, whether we as individuals actually strive for hedonism—that is the view that all human action is ultimately motivated by the desire for pleasure and the avoidance of pain—and the extent to which this is beneficial for society at large. Importantly, democracy requires a certain amount of discomfort. This leads to fundamental theoretical questions: where positive individual effects go along with negative societal effects what should be done? Should the state override personal autonomy to safeguard personal autonomy itself? If yes, to what extent and in which cases?

Addicted by Design

Today’s Internet (Alter, 2017) —especially social media (Deibert, 2019)— is already “addictive by design.” This threatens individual autonomy (Gal, 2017; Zarsky, 2019). Human behavior can indeed be manipulated by priming and conditioning, using rewards and punishments. Even algorithms can autonomously explore manipulative strategies that can be detrimental to users (Albanie et al., 2017). For example, they can use A/B testing to experiment with various messages until they find the versions that best exploit individuals’ vulnerabilities (Calo, 2013). ⁶

Manipulation and deception become easier thanks to affective computing (or “emotional AI”), captology—the study of computers as persuasive technologies (Fogg et al., 2002)—and the emergence of psychographic techniques focusing on demographic characteristics and “affect detection techniques,” along with diverse types of data such as location-based tracking, real-time data, or keyboard usage. Consider that just dozens of Facebook Likes can reveal useful and highly accurate correlations (Youyu et al., 2015). Implicated data include what the European General Data Protection Regulation (GDPR) classifies as specially protected data: sexual orientation, ethnicity, religious and political views, personality traits, intelligence, use of addictive substances, parental separation, age, gender, and perhaps most importantly, emotions. ⁷

Despite the detection of emotions is clearly reductionistic and prone to bias, the resulting persuasive techniques can deeply affect individuals’ self-control, self-esteem, and even self-determination. These can indeed stimulate users in a powerfully subconscious and hormonal way. Facebook’s infamous, large-scale emotional contagion experiment exemplifies this point (Kramer et al., 2014), showing how mainstream social media can affect emotions and exploit vulnerabilities in human psychology.

Compulsion loops are already found in a wide range of social media, and especially online games (Deibert, 2019). Research suggests that such loops may work via “variable-rate reinforcement” ⁸ in which rewards are delivered unpredictably. This unpredictability affects the brain’s dopamine pathways in ways that magnify rewards. Also, design facets intentionally trigger dopamine rushes or other emotional highs, stimulate popularity contest or implicit social obligations (Kidron et al., 2018)—use a whole host of additional brain hacks, probably many of them not even publicly known. Among those known, moral outrage can be exploited to increase engagement. New York University (NYU) psychology researchers found that each word of moral outrage added to a tweet raises the retweet rate by 17% (Harris, 2019). At times, users may also be captured in a spiral of ever more extreme, conspiratorial content—also known as the “rabbit hole effect.” ⁹

Similarly, under the auspices of maintaining Friends’ relationships, Facebook’s “NewsFeed” is primarily moderated by homophily, or similarity in affinity (DeVito, 2017). Excessive homophily, however, helps to spread misinformation, frequently resulting in homogeneous, polarized clusters reiterating emotionally charged and externally divisive content (Deibert, 2019; Sunstein, 2017). In other words, increasing social and political polarization is actually part of the business model of currently dominant social media platforms. Eventually, most users consume personalized content designed for implicit individual needs such as hedonism, sensationalism, and subtle self-propaganda.

Such Big Data-driven behavioral modification is defined by Yeung (2017) as a technique of “hyper-nudging” which dynamically configures the user’s informational choice context in ways intentionally designed to influence her decisions. For example, “dark patterns” which define instances where designers use their knowledge of human behavior (e.g., psychology) and the desires of end-users to implement deceptive functionality that is not in the user’s best interest (Gray et al., 2018). As such, hyper-nudging—in parallel to personalization—concerns all of the design process, not only algorithmic decision-making. This introduces a new form of power—a new “invisible hand” of behavioral prediction and modification (Zuboff, 2015).

Over time, as smart environments will likely permeate societies, users—especially young people—may be automatically plugged in and guided through life along algorithm-driven pathways nudged in the best interests of whoever owns or pays to use people’s data, at times with dramatic consequences. For example, addictive techniques are already concerning in the case of negative effects on children’s well-being, including increased risk of suicide and depression, conflicts with parents and adverse effects on cerebral and social development (Kidron et al., 2018).

Given this constellation of facts, the resultant, paramount concern is that technology, especially social media, is downgrading our attention spans, a common base of facts, and capacity for complexity and nuanced critical thinking, hindering our ability to construct shared agendas to help to solve our problems and the epochal challenges we all face. For some, this supposed degraded and degrading capacity for collective action arguably represents “the climate change of culture” (Harris, 2019). Still, existing scientific evidences on these consequences remain insufficient, as it seems the current legal framework and political response.

Citizens Before Users

Despite apparently increasing transparency along with the resultant efforts to reform, the current algorithm-driven advertising system that lies at the core of the surveillance economy persists. And it persists in systematically promoting extreme, inaccurate and radical content—regardless of what malicious actors may do to seed it. This is the case of political microtargeting, also known as “dark ads.” ¹⁰ Such massive power asymmetry between platforms and potentially malicious actors using them, on one hand, and individual users on the other, cannot be ignored. Since the Cambridge Analytica scandals implicating manipulative and possibly illegal social media use in Brexit and Trump 2016 campaigning, challenges and potential solutions are being discussed (Kohl et al., 2019).

To access social media platforms, users usually have to give up the data they generate. The profit-generating business of selling the resultant meta-data to advertisers is what ostensibly supports these “free” platforms. But Internet users are not simply or even primarily hedonistic consumers to persuade or even manipulate to maximize data extraction and profit generation. Rather, users are also often producers, citizens, and unpaid workers who generate economic value for these platforms through their “digital labour” (Scholz, 2012). And, still, only a minority of users actually understands how Facebook’s personalization works (Smith, 2018). ¹¹ This is the status quo of the current data-driven society, in which sociality and the “public sphere” are turned into economic value. Users occupy a subordinate position without any reasonable possibility to opt out rather than “agree” or leave their social networks.

One compelling explanation for why this became possible so rapidly is that nothing similar to Information and Communication Technologies development ever happened in the past, so there were few institutional defensive barriers. Information intermediaries have indeed been overestimated and treated as “emissaries of the future,” producing “an overwhelming sense of inevitability” (Zuboff, 2015, p. 85). As such, societies quickly came to depend on these new information and communication tools as necessary resources, and at times even as preconditions for social participation.

Internet regulators have usually approached personalization through the lens of pre-Internet legislation. Personalization algorithms, however, are novel and complex entities, and they require novel solutions and public discussions around their regulation. There are, in fact, three intertwined “paradoxes” that highlight the complexities to fairly govern personalization. First, the “personalization paradox,” that is, a trade-off between privacy and personalization accuracy. Second, and strictly related, the “privacy paradox,” that is, the infamous users’ inconsistent will to protect their own privacy. And third, the “paradox of choice,” that is, the more choices users have, the more easily they rely on simple-to-use personalized tools. ¹² In other words, in order for personalization systems to provide a “better service,” users are surveilled and their data captured and exploited. Even if they disagree—as they more often do—they do not proactively react. Even in cases where the users are provided with more agency, they are unlikely to take advantage.

GDPR—the new European Data Regulation in effect since 2018—grants novel rights for data subjects and duties for data controller. ¹³ Along with the e-Privacy regulation draft, the GDPR actually strengthens “data consumer protection”; users can indeed decide whether to enter into a contract, be informed, access the data generated, receive information about the logic involved and opt not to be subject to automated decision-making based solely on automated processing. Yet, the right to an explanation (Art. 15) and the right to non-discrimination (Art. 22)—the most relevant in this context—are highly disputable. The major problem actually lies in the apparent belief of lawyers as well as designers that input control alone is sufficient to alleviate any systemic concerns of personalization algorithms (Kohl et al., 2019). Data protection law remains indeed crucial—because personalization requires updated target profiles—but currently no legal framework addresses explicitly the outputs of personalization algorithms.

The approach of user empowerment often relies on informed consent, but this might not be an optimal solution because opt-in approaches lead to uncertainty and context dependence. People in fact cannot be counted on to navigate the complex trade-offs involving privacy self-management (Acquisti et al., 2015). Most people in fact neither read nor understand online privacy policies. These actually have also two inherently contradictory goals; to be understandable to consumers—which requires simplicity and brevity—and say something meaningful about how data are processed—which is complicated and requires a lot of details. Moreover, dark patterns are often employed during terms of conditions and privacy updates (Moen et al., 2018). Although user-centered solutions may better foster the transparency of personalization algorithms, they have significant shortcomings: they shift the responsibility and accountability for the protection of rights and utility to the users, and this usually results in inefficiency such as insufficient privacy protection.

An explanation for the widespread default trust to this system, and the lack of responsiveness toward privacy management can be traced back to behavioral economics as well as psychology. Drawing from the famous work of Kahneman (2011), for example, Thaler and Sunstein (2009) describe two major cognitive systems from which individuals take decisions: an “automatic system” and a “reflective system.” While the latter is the slow, effortful, and controlled way to think, the former is efficient, rapid, largely unconscious, and prone to systematic errors. This bounded rationality affects how people assess the likelihood of future events and how their individual biases and vulnerabilities might be exploited. This is why education on persuasive techniques is probably the most effective deterrent to deception and manipulation (Friestad & Wright, 1994). Similarly, when people have knowledge regarding their own vulnerabilities, the efficacy of persuasive attempts diminish. ¹⁴ Developing such a critical and aware approach to persuasion—“persuasion knowledge” (Friestad & Wright, 1994)—is indeed fundamental for citizens of information societies. Yet, it might not be sufficient to resist undesirable consequences of personalization and, ultimately, to fully benefit from its potential.

Algorithmic Sovereignty in Social Media

Mainstream social media—especially Facebook—explicitly counteract any possibility for its participants to gain sovereignty: it denies all possibilities of participation in the decision-making process of its own algorithms, as well as strictly regulates the opportunities of interoperability for the data it gathers. Therefore, when we speak of algorithmic sovereignty, we intend the moral right of a person to be the exclusive controller of one’s own algorithmic life and, more generally, the right and capacity by citizens as well as democratic institutions to make self-determined choices on personalization algorithms and related design choices. Self-determination is essentially at issue here, and the battlefield is whether or not users will be able to exercise their actual will when it comes to personalization. Until now this premise—essentially that it matters whether people control personalization and its algorithms—has never been seriously commented, stated, or even discussed by social media platforms.

Conceptually, the following analysis is based on a constructionist approach that posits that metaphors organize users’ perceptions and contribute to creating new realities (Krippendorff, 1993). To effectively address new Information and communications technology (ICT)-related challenges, it is indeed needed to enrich our conceptual toolbox (Floridi, 2015). Taking the notion of “algorithmic sovereignty” as a metaphor—first explored by Roio (2018)—we are interested in what could be emphasized or neglected in its use and the extent to which its use may contribute to reiterate or deconstruct mythologies of digital infrastructures, with a specific focus on personalization algorithms in social media.

In the last years, similar related concepts have been introduced and discussed in academia and media (Couture & Toupine, 2017), notably data sovereignty (e.g., De Filippi & McCarthy, 2012), digital sovereignty, and technological sovereignty (DiEM25, 2019). Such increasing interest in the notion of sovereignty has the ultimate goal to assert some form of individual and collective control empowerment over digital technologies. In particular, the notion of technological sovereignty is used to refer to initiatives that create alternatives to commercial and/or military technologies, often with a great emphasis on free and open source software and hardware. As such, it is also framed as an opposition to a hegemonic power, namely the United States and its biggest private tech companies (also called GAFAM: Google, Amazon, Facebook, Apple, Microsoft). In its broad sense, technological sovereignty is often meant as a rupture from “state sovereignty,” while other times—similarly to the notion of “computing sovereignty” by Richard Stallman (Couture & Toupine, 2019)—as the duty of public services to keep total control of their computing tasks in the benefit of citizens.

We can ideally distinguish between two poles of algorithmic sovereignty—weak sovereignty and strong sovereignty—between which different approaches may be situated: “weak algorithmic sovereignty” would refer to the negotiation between states and social media platforms to enact certain technical conditions, while “strong algorithmic sovereignty” would refer to the actual common ownership of algorithms by the people. This latter interpretation widely overlaps with the concept of technological sovereignty. Algorithmic sovereignty can thus be considered as a sub-group of the broader notion of technological sovereignty but, at the same time, it can provide more specific conditions on what the individual and social control over algorithms should be.

In this context, it is easy to think of the free software movement and the open source business models as the paradigms that enable the scrutiny of algorithms by thousands of experts, as well the freedom to modify them and distribute modifications (Roio, 2018). ¹⁵ Yet, even when the access to algorithms complies to these major ethical concerns, would that be enough to verify a condition of sovereignty for all living participants? Is transparency, as opposed to secrecy, a sufficient condition to make algorithms functional to the creation of an intelligible society? And how to effectively translate these principles in the context of personalization algorithms in mainstream social media?

Algorithmic Sovereignty in Practice

Personalization algorithms, more than others, are technologies that inscribe values, thus visions, ideas, and beliefs to satisfy needs and desires. This is a fundamental condition to understand the role of sovereignty and to enable users to relate to it. Exerting more control as hyper-connected citizens on personalization in social media is the collective achievement we call algorithmic sovereignty.

In light of all the consequences of personalization, it is fundamental to recognize all personalization algorithms in which individuals and societies participate as Commons (Ostrom, 2015), or, more precisely, as Knowledge Commons (Vercellone et al., 2015). To do so, it is necessary to give particular attention to features and uses of algorithmic languages, well beyond the legal and economic considerations concerning the contexts in which they are interpreted and executed (Roio, 2018). Similarly, politics for the digital age need to actually consider users as political subjects (Fuchs, 2015). For example, unions could be extended to digital media users. The exploitation of digital labor is in fact connected to the commodification of the commons, including the digital ones.

Nowadays, the only way for participants to gain sovereignty in mainstream social media is to create an entirely new territory. ¹⁶ This is hardly suitable due to network effects which occur when the value of a platform to any user increases exponentially with the number of already present users. Normally, this leads to monopolies (Lovink & Rasch, 2013). The history of alternative media is indeed a history of enormous challenges, as the consequence of a political economy that limits the possibilities for civil society because hearing alternative voices is a matter of money and political resources that afford visibility (Fuchs & Marisol, 2015). What has changed in the last decade is the vast societal dependency on information intermediaries’ networks.

The only opportunities for social media users to gain algorithmic sovereignty lies in two possible lines of action (Roio, 2018): either to “fork” ¹⁷ the project, with great expense of resources and effort and with an uncertain outcome—this might even be facilitated with the right to portability afforded by the GDPR—or to rely on a strong regulatory framework that opens Facebook’s governance to its participants. Yet, both options are very uncertain and unlikely to succeed. On the one hand, the “critical mass effect” does not make data portability a useful mechanism for migrating to other networks. On the other hand, the focus of the GDPR on data consumer protection does not actually guarantee the protection of collective rights.

To overcome social media monopolies, major media reforms are required. Among many, some scholars advocated proposals for gaining sovereignty such as a “Public Service Internet” that is completely non-profit (Fuchs, 2017) or “Public Service Algorithms,” namely the possibility to provide individuals institutionally coded algorithms (Bennett, 2018). Burri (2015), for instance, proposes a “Public Service Navigator” as a mechanism for “influencing the conditions of access to content, particularly its visibility, discoverability, and usability” (Burri, 2015). A practical example of an exploratory algorithmic tool is represented by the Massachusetts Institute of Technology (MIT) Media lab project gobo.social, a social media news aggregator with sliders that users can control to filter information; from political perspectives (left, right, center) to the extent of seriousness, rudeness, gender, and other parameters one can filter information with. This project shows what a user-friendly personalization algorithm might look like.

Yet, the idea that existing legal frameworks such as consumer data protection and antitrust regulation can be applied to the new challenges and that, more generally, media should be left to self-regulation is still very widespread. In the European media policy, it is in fact assumed that in most cases, soft law promotes self-regulation. To sustain at least a fair competition, another solution could be taxing large media (and other) corporations and channeling the resulting income into alternative non-commercial social media (Fuchs & Marisol, 2015). All these measures have their own limits, of course, but they represent constructive steps toward a more sustainable social media landscape. So far, however, there has not been sufficient evidence or political interest to justify any more radical proposals in practice. Certainly, there are still enormous social and technological issues at stake.

Nonetheless, the pursuit for more individual agency and social control of platforms is receiving attention. For example, the European observatory on Algorithmic Sovereignty, a collective place of documentation gathering research, developments, events, and projects related to the topic. ¹⁸ Also, the project Fediverse—with roughly 5 million users ¹⁹ —is an example of a social media where users approach an algorithmic sovereignty. The name is a portmanteau of two words “federation” and “universe,” a common name for federated social networks running on free open software on a myriad of servers across the world. The “Federation” refers to a global social network composed of nodes that talk to each other. Each of them is an installation of software which supports one of the federated social web protocols. Fediverse networks are designed to be run by anybody, free to choose and register on any server and choose the person who will be in charge of its data—the administrator of your server. Selecting a server for its politics of data, however, is an expert choice one cannot expect from the large public.

Another significant approach is taken by the project tracking.exposed. By scraping and collecting the data individually recommended to the user on supported platforms (Facebook and YouTube above all), it collects and stores evidence which is usable in new ways. A result of such data reuse is the comparison of personalized experiences. By assuming everyone has an individualized perception of the public discourse, the project extrapolates the topics and let users compare, which might allow for improved critical judgment regarding the quality of information received. Also, these data are shared for research purposes, with non-governmental organizations (NGOs) and academic teams. By creating new profiles, with as many of the possible variables (age, friends, pages followed, likes) under control, it is possible to test how algorithms lead to divisive perceptions. As a side effect, it has collected a library of public information. Inspired by the political concept of the European Data Commons (DiEM25, 2019), the project keeps data in the collective interest, as far as it does not expose any individual without his or her consent. Yet, full anonymity cannot be offered at the current state of research. Therefore, the team has to make a privacy impact assessment for each form of data reuse. ²⁰

All together, these proposals and projects question—and to some extent could even disrupt—the current power asymmetry between users and the public toward mainstream social media platforms and their personalization algorithms. Yet, they could also renew trust between platforms and users. To ensure algorithmic sovereignty, however, a comprehensive approach to personalization algorithms is needed: one that complements the principles advocated by the free software movement to accommodate the needs, skills, and motivations of the average user, not only experts, activists, or enthusiasts of civil society.

Algorithmic Sovereignty in Theory

With “algorithmic sovereignty” in social media, we generally intend to regulate the use of information filtering and personalization design choices according to democratic principles, setting their scope for private purposes and harnessing their power for the public good. In other words, to open black-boxed personalization algorithms of (mainstream) social media to users and public institutions.

While there have been discussions on the ethics of recommender systems and personalization (Bozdag & Timmermans, 2011; Helberger, 2019; Milano et al., 2020), and on how to fulfill main algorithmic principles such as fairness, transparency, accountability, accuracy, and privacy (algo:aware, 2018), in this hypothetical radical scenario, algorithms could even be exchanged, remixed, tested, plugged, and even sold or rented. On top of this opening, it is expected that not all users would have the knowledge to build their own algorithms. Moreover, one could assume that some influential organizations (such as news media, political parties, ideological groups) would also spread their own algorithms to their followers. Audience fragmentation and engagement optimization may still dominate the media landscape. Therefore, we identify and discuss prominent intertwined preconditions, principles, and goals—certainly not exhaustive—to grant in social media what we intend as algorithmic sovereignty.

First of all, and more obviously, there is a general need for “algorithmic literacy.” We can define it as the basic knowledge on how filtering mechanisms and design choices function and what their impact on one’s own life is and may be. This can be cultivated with regular updates on personalization research, visualization tools, and dashboards. Some paradigmatic experiments have already been done: (1) in general, through browser extensions (Reviglio, 2019a), (2) with interactive sliders (e.g., gobo.social), or (3) showing users their filter bubbles and help to “burst” them (Nagulendra & Vassileva, 2016). Clearly, to gain more control, algorithmic literacy need to go hand in hand with digital and media literacy.

Another significant aim to achieve is “content neutrality,” meaning platforms should become agnostic toward content. These should in fact return machine-readable, unfiltered, chronologically ordered data. Then, a reader device would receive these data, and an algorithm would filter and prioritize. This could be a fundamental architectural requirement for a strong algorithmic sovereignty. By making sure that the filtering happens on the client side, it is ensured that the platform becomes effectively neutral, and nobody, except the individual, will end up knowing what was watched and for how long. Such neutrality would also help to prevent deception by design and, eventually, to cultivate a related aspirational goal: “algorithmic neutrality,” which is the idea that content should spread freely without biases.

Content and algorithmic neutrality might be also achieved with a “right to profile transparency,” through an infrastructure that supports what the philosopher Mireille Hildebrandt (2015) defines as “counter-profiling,” meaning “to conduct data mining operations on the behaviours of those that are in the business of profiling whether ‘those’ are humans, computing systems or hybrid configurations” (p. 223). This is a general strategy to figure out how one is actually targeted. In addition, platforms and algorithm developers could routinely disclose to their users and the public any experiment that the users were subjected to. ²¹ An independent board to review and approve experiments in advance could also be established. This would be particularly in line with two intertwined newly discussed human rights: the “right to not be measured, analysed or coached” ²² (van Est et al., 2017) and a “right to cognitive sovereignty” that ought to protect individuals’ right to mental self-determination (Yeung, 2018b).

More generally, algorithm auditing needs to be supported as well as data sharing. In particular, it is fundamental to guarantee a strong separation between filtering mechanism and data. Filtering mechanisms should be, as much as possible, stateless and idempotent, that is, capable to return the same result if executed twice. This is a formal way to describe a more easily auditable machine. On the other hand, personalization algorithms that use neural networks keep updating their internal state, making it impossible to perform certain analyses either ex post or ex ante. If the same result can be accomplished using a simpler computing system, it would also be simpler to explain, standardize, teach, and assess outcomes. If it is possible to keep a simpler computing system as standard, the technical debt caused by the lack of explainability of algorithms might be avoided. Moreover, metadata which describes the content produced is fundamental to control. Functionalities of research, analysis, and prioritization might perform better with metadata usable by end-users. Even if it is a laborious process, mainly for skilled individuals, metadata represent an opportunity to differently index and retrieve content so as to eventually increase content quality over quantity.

There are, then, user-centered preconditions to negotiate. Being essential to personalized information consumption, design choices, and affordances should be to some extent adjustable, especially for what concerns attention management. For example, disabling the auto-loading of posts (similar to the auto-play function for videos which is usually set by default) and decide to scroll n posts anytime one logs in. Another interesting example is the possibility to hide metrics in order not to be influenced by comparative quantifications (such as the number of likes and shares). ²³ The quantification and maximization of social interactions, in fact, contribute to create a “culture of performance” (Castro, 2016) that seems to be negatively correlated with well-being (Verduyn et al., 2017). In essence, all those basic design choices that ultimately affect information behavior and personalize one’s information diet ought to be socially negotiated, implemented, and ultimately, adjustable.

The development of “natural metrics” usable to evaluate the informative experience produced by the algorithm is also another critical issue. For example, the percentage ratio of photos/videos/text, the time spent on the platform, the number of posts encountered, or which sources have been consumed. This information should be collected on the device and given back to the user to primarily increase self-awareness.

Furthermore, there are information self-determination and media pluralism concerns to cope with. In particular, the capacity to reach a balanced information diet from both an individual and a political perspective (Eskens et al., 2017) and, at the same time, make sure that a range of viable “perspective widening” tools are provided (Delacroix & Veale, 2020).

On the one hand, a “right to receive information”—as guaranteed by Article 10 of the European Court of Human Rights—recontextualized in the digital context could help to establish what news consumers legitimately may expect from the news media with respect to the diversity or relevance of personalized recommendations (Eskens et al., 2017). In practice, this right could result in an effective explicit personalization—especially related to political news.

On the other hand, a “right to information explorability” to increase serendipitous encounters and to reduce potential filter bubbles and echo chambers (Reviglio, 2019b). This could be achieved increasing information findability and discoverability, for instance, with the possibility to choose the accuracy and diversity of information filtering. In practice, this might imply more interactive control of the algorithmic outputs and, as such, discoverability throughout sliders, topic categories, filters to navigate the information by source and keyword, or even algorithmic recommender personae as specific avatars to filter information (Harambam et al., 2018). As Hildebrandt (2019) advocates, societies must demand companies to explore and enable alternative ways of datafying and modeling the same person. Multiple and dynamic profiling could be paramount to this scope.

Final Considerations

Certainly, the above outlined general suggestions need to be discussed further as they may have potentially unintended and undesirable consequences, impossible to systematically assess or even speculate in a short article such as this. Yet, we briefly identify some major weaknesses and critical issues.

If institutions empower users who cannot exercise their algorithmic sovereignty, they may rely on default choices, and this may only leave aside the weaker users and enable other actors as well as mainstream social media. This scenario is not different from a free market where the players are not equal, and it might even legitimize current mainstream platform at the expense of potential emerging competitors. Now, if we imagine the “liberalization” of algorithms would happen tomorrow, it is not hard to believe that a technocratic group of skilled individuals would try to claim their algorithm to be the best. By leveraging existing conditions of influences, they might just repropose the same form of algorithm oppression but with a variety of small actors in the market.

Also, this liberal market approach might lead to the misleading message that an algorithm can be better than another one. The perfect algorithm, in absolute terms, does not exist. Every one of us has different priorities, interests, and time availability. The correct algorithm is the one that best fulfills the needs of the individual, and this might not be true anymore if one begins to change. The fitting algorithm thus cannot be permanent. Instead, the above-mentioned natural metrics, dashboards, and tools are fundamental, and should be imagined as the regulation imposed in the food industry (to declare the allergens, ingredients, kilocalories etc.). It can be assumed that negotiating these parameters and design choices should be an effort made and/or supported by the current democratic institutions and expert network bodies, such as the World Wide Web Consortium (w3c), Institute of Electrical and Electronics Engineers (IEEE), or Internet Engineering Task Force (IETF). In any case, it is desirable for a multistakeholder governance to meaningfully involve all the relevant actors.

Because algorithm sovereignty is a political challenge, the solution cannot be only technical. Algorithmic sovereignty indeed implicitly calls for more responsibility over citizens which ought to be able to decide on their own instead of delegating to someone else. We believe that institutions like schools, academia, and public service media ought to be proactively responsible for the most significant preconditions: digital, media, and algorithmic literacy.

Conclusion

Personalization algorithms not only provide fruitful opportunities but also dystopian realities. Social network effects have indeed resulted in a few large transnational companies controlling the vast majority of social media use. Given this control, it is very difficult to establish alternatives that question the very principles on which the current business model and its perverse consequences exist. Nowadays personalization algorithms in social media tend to represent individuals and society as a body without contradictions and complexities, entities that can be reduced to a calculation to assure profitability. The outlined risks are fundamentally threatening individual progress and societal cohesion. There is a clear and increasing need to reimagine social media. To regain trust between users and platforms, personalization algorithms need to be seen not as a form of legitimate hedonistic subjugation, but as an opportunity for new forms of individual liberation and social awareness. For these reasons, a basic—and admittedly underdeveloped—notion of algorithmic sovereignty has been introduced and discussed: we pose that the individual and collective autonomy to govern personalization algorithms is a necessary step to imagine new forms of algorithmic arbitrage in social media.

Achieving such algorithmic sovereignty will likely be hard; there is a fundamental need to devise strategies that interrupt and in some cases outlaw surveillance capitalism’s foundational mechanisms. Opening the black-box of personalization algorithms is only one piece of the puzzle. Through further discussion, we need to realize how to reach a fair, effective, and accountable algorithmic sovereignty in social media. In this article, we outlined general and preliminary preconditions and principles. Despite vast technical, social, and political issues, the major problematic ultimately hinges on balancing conflicting democratic values, and translate them into code, design, and social practices. We believe that such endeavor is within reach and that public institutions and civil society can eventually sustain its realization.