Abstract
This study provides one of the first examples of de-platforming’s direct emotional and financial impact on Instagram and TikTok content creators at the margins. Both platforms provide significant opportunities toward creative and flexible work, allowing creators to maintain networks, promote work, express themselves and earn a living. However, their governance can severely disrupt certain forms of content creation, particularly for users who post online sex work and nude content. Through a qualitative survey, we gathered the experiences of 123 de-platformed Instagram and TikTok users who posted nude or sex work-related content. Our study provides crucial testimonies showing that the precarity of creator labor and platforms’ reliance on automated content moderation have negatively impacted creators’ well-being. Participants faced adverse psychological impacts resulting from job and income uncertainty, associated feelings of powerlessness, a loss of digital identity, and enforced isolation from a previously established social network. We conclude by providing platform governance recommendations based on these experiences.
Introduction
This study provides one of the first examples of de-platforming’s direct emotional and financial impact on Instagram and TikTok content creators at the margins, publishing much-needed practical and user-informed testimonies of the effects of censorship explored by platform governance studies.
Instagram and TikTok are largely free-to-use, high-profile social networks providing professional content creators with significant opportunities to maintain networks, promote work, express themselves, and earn a livelihood (Duffy, 2020; Duffy & Meisner, 2022). However, despite promoting equal access to the benefits of the creator economy, the same platforms are also subject to governance procedures that stigmatize certain forms of work and that can severely disrupt certain forms of content creation, particularly those by users who post nudity or sex work (Are & Paasonen, 2021; Blunt & Wolf, 2020).
Social media creators are “jack-of-all-trades entrepreneurs within a highly competitive industry, simultaneously videographers, editors, photographers, on-screen talent, brand ambassadors, merchandise producers” (Glatt, 2022, p. 4). They make a living by creating content, building relationships with and communities among users, and promoting products and services they post on platforms such as YouTube, Instagram, TikTok, Patreon, Facebook, and Twitch, generating revenue for themselves and for brands (Cotter, 2021).
The creator economy has facilitated new kinds of cultural production, also creating flexible, aspirational options for labor based on entrepreneurship (Banet-Weiser, 2018; Duffy & Meisner, 2022; Glatt, 2022; Nolan-Brown, 2022). The advantages of social media as a workplace include the apparent welcoming of everyman into cultural production and entertainment, flexibility to choose when, where and how much to work and, perhaps most importantly, the opportunity of unprecedented fame and wealth for those previously unknown and sometimes disadvantaged, such as BIPOC, LGBTQ+, female, disabled, working class, and other users (Glatt, 2022).
However, these new forms of creator labor have also escalated workers’ precarity (Duffy, 2020; Duffy & Meisner, 2022; Easterbrook-Smith, 2022; Glatt, 2022). Those working through social networks are part of the “gig” or “platform economy,” an “insecure, often short-term or piecemeal, employment, frequently facilitated by a platform or app” (Easterbrook-Smith, 2022, p. 3). Arising from the ashes of the 2008 financial crash, the gig economy is typically facilitated by apps such as Uber or TaskRabbit, although content creation—including influencer partnerships, social media self-promotion of offline services, and online sex work—is also part of the gig economy. Gig work offers flexibility but lacks the protection and benefits linked with traditional employee status (Easterbrook-Smith, 2022). In effect, the gig economy has helped to shift “patterns of employment in the cultural industries away from stable structures” (Glatt, 2022, p. 3).
Budding social media creators face additional challenges, as their success is related to a set of variables outside their control, such as elusive visibility, always within reach but not guaranteed (Duffy, 2020; Duffy & Meisner, 2022; Glatt, 2022). Banet-Weiser calls the social media landscape an “economy of visibility,” characterized by an accumulation of metrics such as likes, following, and engagement (Banet-Weiser, 2018, p. 2), “popularity metrics” which are a crucial aspect of content creators’ success (Glatt, 2022, p. 12). Affordances such as views, channel subscriptions, and followers, therefore, become essential to a user’s legitimacy, so much so that “depending on how many views and subscriptions a channel achieves, it is accordingly deemed relevant and attention-worthy or not” (García-Rapp, 2017, p. 234).
However, maintaining those views and following is anything but straight-forward: While the promise of visibility is afforded to everyone, success strictly relies on compliance with platforms’ often broad, unclear, and unequally applied rules, their processes, agendas, and aesthetics (Are & Paasonen, 2021). Achieving views and securing a following becomes essential to thrive—and to be seen to be thriving—meaning that those whose content or mere existence are stigmatized by platform rules are not guaranteed the same visibility, work, privileges, or earnings as other content creators (Are & Paasonen, 2021; García-Rapp, 2017).
This quest for visibility leads to a precarious and unbalanced power dynamic between users and platforms. Content creators are at the mercy of tech giants’ opaque and capricious management (Are, 2021b; Bishop, 2019; Cotter, 2021). They sit within a power structure exerted through algorithms, “codified step-by-step processes” through which platforms “afford or restrict visibility” (Bishop, 2019, p. 2590). Unsurprisingly, this results in users wishing to understand more about how said spaces are governed (i.e., Are, 2021b; Cotter, 2021; Duffy, 2020).
Algorithmic governance is largely to blame for the precarity of creator labor: the inscrutable “algorithmic boss” can frustrate and wreak havoc on the lives and livelihoods of creative workers (Duffy, 2020, p. 103). Social media users using platforms with algorithmically curated feeds have lamented the lack of agency and powerlessness (Are, 2022; Eslami et al., 2016). Recreational social media users have been found to develop and share “folk theories” about how curation algorithms work to plan their behavior in the face of opaque algorithmic governance (Eslami et al., 2016). These “communally and socially informed, theories and strategies pertaining to recommender algorithms” known as “algorithmic gossip” (Bishop, 2019, p. 2602), or “folklore” (Savolainen, 2022) are some of the few tools creators have to attempt to beat the algorithm and secure visibility. The creator economy is therefore characterized by “exploitation, insecurity, and a culture of overwork” (Duffy & Meisner, 2022, p. 1). Far from improving working conditions, platforms have instead lowered work opportunities for their workers, and particularly for workers at the margins (Glatt, 2022), replicating offline inequalities by disproportionately targeting marginalized communities—a phenomenon that started with sex workers but now swiftly moving onto the creator economy as a whole (Are, 2021c; Blunt & Stardust, 2021; Blunt & Wolf, 2020; Paasonen et al., 2019).
De-Platforming Sex (and Beyond)
Nowhere is the precarity of creator labor felt more keenly than by those who post content involving nudity and sexuality, such as sex workers and sex-positive influencers (Are, 2021c; Blunt & Stardust, 2021; Blunt & Wolf, 2020). On the one hand, these stigmatized communities have been swift to reap the benefits of the creator economy, reclaiming something “taboo” like sex and nudity as a form of self-definition (Are, 2021c; Blunt & Stardust, 2021; Blunt & Wolf, 2020). Online sex work, for example, has offered significant freedoms to sex workers who previously relied on third-party managers, aiding those affected by discriminatory hiring practices in mainstream or studio-produced porn “to control their own image, or to produce and perform in pornography” (Easterbrook-Smith, 2022, p. 5). Nolan-Brown (2022) writes: “For the first time, people with a truly diverse array of body types, looks, races, ethnicities, sexualities, gender identities, and kinks had direct access to the tools of porn production and distribution.”
On the other hand, these new benefits have been short-lived in an industry where creative labor is subject to stringent content moderation, or the practice of deleting and/or censoring online content, a crucial aspect of platform governance without which social media would be unusable (Diaz & Hecht-Felella, 2021). Through content moderation, social media and internet platforms make publisher-like decisions over the visibility of what is posted on their spaces, enforcing rules established via “community guidelines” or “standards” based on which a blend of human moderators and algorithms are trained to make decisions (Kaye, 2019).
Different social networks share similar community guidelines, banning content that is often already prohibited by the criminal law of most countries (Goanta & Ortolani, 2021). Yet, platforms have been known to over-censor content they found legal but potentially objectionable, with a set of repercussions on users’ freedom of expression (Goanta & Ortolani, 2021; Kaye, 2019). This conservative approach to censorship has been linked to platforms’ wish to protect their commercial interests by being overzealous in following recent legislation (Are & Paasonen, 2021; Blunt & Stardust, 2021; Blunt & Wolf, 2020; Nolan-Brown, 2022; Tiideberg & van der Nagel, 2020).
Content moderation—and particularly automated moderation—has also disproportionately targeted marginalized users, over-focusing on nudity and sexuality instead of on violence (Are & Gerrard, 2023; Duffy & Meisner, 2022). A case in point is the 2018 US law known as FOSTA/SESTA—the Allow States and Victims to Fight Online Sex Trafficking Act (FOSTA) and the Stop Enabling Sex Traffickers Act (SESTA), respectively. FOSTA/SESTA
The law changed the face of platform governance through increasingly sex-averse community guidelines focused on nudity, sexual activity, and solicitation (Blunt & Stardust, 2021; Blunt & Wolf, 2020; Paasonen et al., 2019). In this sense, content creators posting nudity and sex work find themselves between a rock and a hard place: on the one hand, an existing following and cross-platform linking are necessary to make adult content creation platforms profitable (Beebe, 2022; Easterbrook-Smith, 2022); on the other, platforms’ distaste toward nudity, sex and sex work made creators at the margins more invisible, dubbing their presence as inherently dangerous, and adding a further layer of precarity to their working lives (Are & Paasonen, 2021). On Instagram and TikTok, such changes mean that accounts going against community guidelines are deleted, often without warning (Instagram, n.d.; TikTok, n.d.), preventing creators from accessing what is effectively their workplace.
On Instagram, community guidelines prohibit “the offering and selling of sexual services” (Instagram, n.d.), with a two-step test to identify whether users are soliciting even implicitly, which identifies the offering or soliciting for sexual acts combined with posts presenting suggestive elements as implicit solicitation (Are, 2021a; Instagram, n.d.). Commonly sexual emojis (e.g., the aubergine emoji), suggestive poses, sexual slang or mentions or depictions of sexual activity count as “suggestive elements,” and an additional catch is that the solicitation itself can be implicit—for example, “send me a DM,” or “hit me up via email” or “link in bio” (Are, 2021a; Instagram, n.d.).
Instagram’s understanding of sexually suggestive elements has been notoriously opaque, inconsistent, arbitrary, and puritan (Are, 2021a, 2021c; Kaye, 2019; Paasonen et al., 2019) to the point where “offering sexual services” or “sexual solicitation” are not even defined within platforms’ terms of use. This means that users are not privy to Instagram’s understanding of sex work and solicitation—and it appears that the platform’s threshold for these behaviors is so low and unclear that even users outside of sex work have been de-platformed after having shared news or petition links (Are, 2022), while sex workers sharing petitions, news articles or any other link-based information have found themselves de-platformed even when not linking to adult content (Stefanello, 2021).
TikTok’s community guidelines also ban content that is “overtly revealing of breasts, genitals, anus, or buttocks, or behaviors that mimic, imply, or display sex acts” (TikTok, n.d.). The platform’s rules prohibit “[a]ny solicitation of nude imagery or sexual contact, through blackmail or other means of coercion” and “[c]ontent that depicts, promotes, or glorifies sexual solicitation, including offering or asking for sexual partners, sexual chats or imagery, sexual services, premium sexual content, or sexcamming” (TikTok, n.d.).
Both platforms’ juxtaposition of sex with harm is striking: instead of regarding sex, nudity, and sexuality as crucial aspects of human life, community guidelines “other” creators of sexual content, identifying them as dangerous by default, showing a puritan approach to governing bodies, and even refraining from
Health and Well-Being Consequences of Content Moderation
Faced with precarious working conditions, opaque governance systems, the stigmatizations of creator labor focusing on bodies, nudity, sexuality, and sex work, social media creators have been found to experience a set of negative offline consequences, such as stress and burnout (Cotter, 2021; Glatt, 2022).
Precarious employment is known to be a significant social determinant of health and well-being in both traditional and gig economies, with three forms of precarity tied to adverse psychological consequences (Allan et al., 2021): precarity
Job insecurity and financial anxiety go hand in hand, and both negatively impact psychological well-being (Dijkstra-Kersten et al., 2015; Fiksenbaum et al., 2017). While previous research has focused on employment precarity in the gig economy and its connected financial stress, inadequate healthcare, and absence of other forms of organizational support (e.g., Wood et al., 2019), workers within this system face highly diverse financial threats. For example, the Covid-19 pandemic led to a huge loss of income for drivers but relatively good income for food delivery workers (Apouey et al., 2020). Content creators depend upon precarious forms of patronage—that is, following and views—instead and also on exchanging key information around governance, policies, and payment systems that can discriminate against users like sex workers. As Uttarapong et al., (2022) note, content creation is far from “easy money” particularly when bodies are involved, and creators must be vigilant in navigating these spaces.
The power differential between employers and employees is another critical issue influencing worker well-being (Benach et al., 2014). In the traditional workplace, this power differential exists between known individuals, but the power imbalance is extreme between digital content creators and their faceless moderators: platforms and moderators are faceless and remote, and the sanctions are drastic since de-platforming results not only in loss of income but also in loss of existing work and contacts.
Other “protective” factors associated with precarious work include good social support, high job control, and low job strain (e.g., Madsen et al., 2017). While we know relatively little about how these psychosocial buffers work in the creator economy, it is notable that the online social networks of content creators are also destroyed by de-platforming. Hill (2021), for instance, has identified the concept of “algorithmic insecurity” as a determinant of poor psychological health. Although her work focuses on non-creative gig work (e.g., on Uber, Door-Dash workers), it is notable that psychological distress for gig workers comes largely from poor control over scheduling and working hours and reliance on mysterious ratings that provide a potential for gaining more work. In other words, Hill found workers often suffer from impenetrable decision-making algorithms and are frustrated by the lack of transparency and control.
Finally, precarity at work is also associated with the loss of identity, a known psychological risk factor. Job loss has long been found to lead to problems associated with the loss of a “work identity” with outcomes including low self-esteem and a disrupted sense of self (Allan et al., 2021; Sverke et al., 2002). For content creators, this identity loss is more tangible and potentially more devastating, first because the work they do is in large part about the deliberate and creative construction of digital identity (Alacovska & Kärreman, 2022), and second because de-platforming literally strips them of that carefully crafted identity. This means that identity loss is a particularly significant emotional well-being factor for content creators, as their posts as their creative selves will typically be more elaborate and require more investment than those of everyday social media users whose construction, conflict, and occasionally destruction of digital identities can also be a cause of psychological distress (e.g., (Beech et al., 2016; Lingel et al., 2014; Pitcan et al., 2018).
In summary, then, content creators work within a gig economy that is already precarious and reliant upon visibility. Those working with sex or nudity are more vulnerable to de-platforming, which has the potential for serious consequences on finance and on well-being. Given the potentially devastating impact of de-platforming on such content creators, this study responded to the following research questions:
Methods
Data to inform this article was gathered during a 3-month period through an anonymous qualitative survey (see also Are’s in press paper on malicious flagging as online abuse against creators at the margins, which draws from the same dataset).
Qualitative surveys ask participants a series of open-ended questions on particular topics with the aim of producing rich accounts of their experiences (Braun et al., 2021). Qualitative surveys are useful from “a social justice and inclusion point of view” because they “offer an accessible method to research beyond the ‘usual suspects’” (Braun et al., 2021, p. 643;644). We drew participants from marginalized communities who had to share personal and often traumatic experiences of de-platforming, meaning qualitative surveys provided the added benefit of allowing them to present their own narrative, in their own safe space, without interference.
Based on Are’s (2022) experiences of de-platforming, this study’s inclusion criteria were intentionally narrow: participants had to be over 18 years of age and have experienced
The survey informing this study was circulated through are social media networks—Facebook, Twitter, Instagram, and TikTok—because she had a sizable following of more than 350,000, comprising artists, athletes, sex workers, activists, researchers, and journalists with whom she built relationships and who helped her share the call for participants. By circulating the survey among communities that already knew her, we ensured users, and particularly sex workers, could feel safe and comfortable when sharing their experiences of de-platforming. XBIZ, a leading news website for the adult industry, also promoted this study’s call for participants through an article, which explains why sex workers are overrepresented in this paper (Parkman, 2022).
The survey questions were modeled on previous experiences of de-platforming (Are, 2022; Stokel-Walker, 2021a) and asked participants about their experiences of account removals, their understanding of the reasons behind their deletion, the steps they took toward account recovery, and the impact de-platforming has had on them. Screening questions were informed by previous research findings highlighting that platform censorship disproportionately targets BIPOC, LGBTQIA+, plus-sized, and disabled users (Haimson et al., 2021) and therefore asked participants about their profiles’ aims, their age, gender identity, sexual orientation, ethnic background as well as whether they were plus size or had a disability.
Data gathered through qualitative surveys are often organized through thematic analyses “illustrated by vivid and compelling excerpts from participants’ responses” (Braun et al., 2021: 650). Thematic analysis (TA) allows researchers to identify, analyze and report themes or patterns within data (Braun & Clarke, 2006). Characterized by minimal data organization, TA describes data sets in rich detail and shines a light on the realities of participants’ lives (Braun & Clarke, 2006), providing insights into lived experiences that become incredibly relevant, particularly when researching stigmatized communities.
Themes are typically “creative and interpretive stories about the data, produced at the intersection of the researcher’s theoretical assumptions, their analytic resources and skill, and the data themselves” (Braun & Clarke, 2019, p. 594), where the importance of a theme does not necessarily depend on quantifiable measures or replicability, but rather on whether it captures relevant details about the research question (Braun & Clarke, 2019).
Three limitations affected the nature of the data gathered. The first is that conducting such research on platforms
Second, the XBIZ article meant that the data collected was skewed toward sex workers’ experiences, although a variety of users from different backgrounds also took part in the study. However, since sex workers are overrepresented among de-platforming targets (e.g., Blunt & Wolf, 2020; Stardust et al., 2020), the responses to this survey arguably represent an accurate picture of the current platform governance landscape.
Finally, basing this study’s design on known experiences of de-platforming may result in the exclusion of users who had different experiences of de-platforming from Are’s. However, given platforms’ aforementioned opacity with regards to their processes, we preferred to base our inclusion criteria on documented experiences of de-platforming that have been previously fact-checked by journalists and confirmed by platforms (Stefanello, 2021; Stokel-Walker, 2021a).
Results and User Demographics
A total of 123 participants took part in this study. Out of these, 98 reported having been affected by both censorship
Most users reported using Instagram and TikTok to promote and/or sell their work (63). The users surveyed also utilize platforms to maintain networks of like-minded users (27), to engage in activism (26), to share pictures without a connecting theme (25), to share and receive education (18), and to document a journey (17).
Most of the users who took part in this study were aged 25–34 (18), with the second biggest group being 35–44 (13) and the third being 18–24 (5). Respondents were largely White (34), with only a few coming from a mixed race (3), Black (2), Latine (2), or Indigenous (1) background. Most of the users surveyed were cisgender women (28), followed by cisgender men (5), non-binary (3), and gender fluid (2) users, with only one transgender woman taking part. Most respondents were bisexual (13), followed by pansexual (12), heterosexual (7), queer (6), and gay (2) users. Only 8 users identified as plus-size, and 16 identified as having a disability.
Thematic Analysis
This article now illustrates its main findings through three themes: de-platforming’s economic, reputational, and emotional impact on users. Our focus on user experience aims to ground the study of platform governance in the experiences of those it directly affects, providing some yet undocumented experiences of the precarity felt specifically by creators of nude and sexual content.
“It Has Destroyed My Income”: Censorship’s Economic and Professional Impact on Creators
A recurring theme in users’ experiences of de-platforming is the loss of income and work opportunities arising from the deletion of their profile and content and from platforms’ opaque moderation in general. Consistent with previous research on the creator economy (Bishop, 2019; Duffy, 2020; Duffy & Meisner, 2022; Glatt, 2022), participants reported that being tied to platforms’ whims resulted in work instability and precarity, forcing them to rely on the “algorithmic boss” (Duffy, 2020, p. 103) and with many reporting significant financial losses as some of the most crippling and challenging effects of de-platforming. Some went as far as claiming to have lost thousands in earnings in the aftermath of their accounts’ deletions: [M]y photography work went from several thousand a month to near zero. COVID didn’t help but since I couldn’t make a new account until recently, I lost out on tens of thousands of dollars. It’s been terrible. I’ve had to do even more sex work for less money. It affected my (sic) massively in terms of finances. On my old account, I was able to utilize my visibility there to receive hundreds of new OF (OnlyFans) subs each month, as well as brand deals/sponsorships etc. Through OF alone I was making upwards of 8k a month, and even now with my new platform, I am often too scared to advertise, so my income has substantially dropped. Now I’m lucky to be making more than 3k a month. I could be making a lot more money if I felt welcome to participate in social media.
The financial losses were, for them, inevitably tied to their personae, their accounts, and their profession. Most respondents citing loss of income were sex workers, who also reported they had been pushed into less safe, more exhausting working conditions to survive post-de-platforming.
Given gig economy workers’ already precarious working conditions, the reliance on third-party platforms such as Instagram and TikTok to market and promote themselves is a concerning variable for these creators’ ability to earn a living: I have not really been able to come back to Pole studios like I did before quarantine. Los Angeles is heavily based on the cloud and IG numbers make a really big deal for teaching Pole. Some actual gigs to only book people with over 5000 6000 followers. So I’ve definitely lost teaching work and Hollywood gig work gig. People who had commissions from me lost access to contacting me. My website didn’t have any traffic while my accounts were in limbo. I had to start completely over again. So I experienced serious a financial blow every time. I’m less seen now and less searchable, so quite simply I have less opportunities for work and marketing in any category. [It’s] making it nearly impossible to grow our community and thus our business.
Users who report lost opportunities mentioned an inability to grow a community of fans and customers as well as lost commissions, gigs, and even website traffic. This is particularly important for creators engaging in Instagram and TikTok promotion for content on subscription platforms such as OnlyFans and Patreon, which do not have their own native promotion tools and so rely on the popularity and reach of social media or mainstream media (Easterbrook-Smith, 2022). This could, in itself, be a triggering factor for de-platforming, as both TikTok and Instagram prioritize native content, with the former even stating that its policies mean that writing “hit me up in DMs” or “link in bio” combined with an account presenting suggestive elements may already alert automated governance to the potential solicitation (Are, 2021a), even when the links used are to other platforms.
The experiences reported by participants are consistent with Hill’s (2021) “algorithmic insecurity” and its related inconsistency, lack of transparency and control, which result in poor psychological health for workers. Platforms like Instagram and TikTok seem to have created an entirely inhabitable space for sex workers and users who post nudity, de facto profiling, and excluding a whole set of workers by making them unwelcome and treating them as synonymous with
“It’s Affecting My Relevance”: The Reputational Impact of De-Platforming in the Creator Economy of Visibility
Intrinsically tied to financial loss is the experience of lost opportunities, resulting in less work, less brand awareness, and, ultimately, reputational damage. Consistent with Banet-Weiser, (2018) “economy of visibility,” de-platformed users claim that not being on TikTok and Instagram, or “not being there” with the high number of followers they once had, equals not being visible to potential work partners, fans and peers: I’m losing money at the clubs and online because I had a loyal fan base before. The whole censorship, from the flagging to deletion of content, to shadowbanning and deletion of my account has been affecting the reach of my work/ my bookings/ my income/ my relevance in the industry and community. When my artwork is removed, the ripple effects are wide and damaging. As a professional artist, I rely on access to social media and the ability to share my artwork with my peers, galleries, and institutions. I have experienced many instances of institutions I am working with being unable to promote our collaboration and my artwork because they experience deletions of my artwork on social media. This amounts to essentially being erased from the art world today.
Instagram and TikTok are not merely spaces for transaction: instead, they act as portfolios of creative work. Once a user is de-platformed, they feel they no longer exist as a worker, as an artist, as a creator, and they experience great reputational damage, arising from the lack of visibility and the loss of a sizable platform. With the erasure of online work then comes the erasure of offline work and its related opportunities and ultimately the erasure of their reputation.
Users seem to believe that more followers equal more visibility and offered them more legitimacy (contrary to research by Pittman & Abell, 2021 and by De Veirman et al., 2017 that found engagement to be more beneficial to brands than a big following). Brands or workplaces that seek out content creators and influencers also use a large following as a proxy measure of success (García-Rapp, 2017; Glatt, 2022). This inevitably means that creators’ success is almost synonymous with visibility (Duffy & Meisner, 2022) and relies upon metrics such as views and following (Banet-Weiser, 2018). As such, losing profiles built through years of digital labor to then have to start again is a severe reputational blow that, according to participants, has cost them work and opportunities.
Lacking healthcare, organizational support, or, indeed, any stability in keeping with general gig economy trends (Wood et al., 2019), gig workers such as content creators therefore experience a heightened version of the power imbalances seen between more traditional employers and employees (Benach et al., 2014). By being de-platformed by those they call the “faceless masters” of platforms, creators lose not only their income but also existing work, contacts, and reputation. In this sense, the experiences reported by the users surveyed are consistent with previous research on job insecurity and financial anxiety, showing that the precarity of their particular content creation work has negatively impacted their psychological well-being (Dijkstra-Kersten et al., 2015; Fiksenbaum et al., 2017).
Hopelessness, Shame, Grief, Defiance: De-Platforming as a Devastating Emotional Rollercoaster
Participants reported a variety of negative emotional effects arising from content moderation, showcasing a conflicting relationship with Instagram and TikTok that highlights how significant these platforms’ role is not only in terms of creative work but also in relation to marketing, self-branding, network-building, and memory-making. The feelings identified reveal a complicated, unequal, and, sometimes, toxic relationship with Instagram and TikTok, leading to feelings of disengagement, vigilance, shame, grief, and defiance. Our findings are directly analogous to the literature on toxic leadership within the traditional workplace (Bhandarker & Rai, 2019), which can generate high levels of psychological distress in workers, leading to feelings of low self-worth and a desire to withdraw from that environment. In this case, participants reported an inability or lack of will to continue sharing content and engaging with Instagram and TikTok due to their hopelessness in interacting with their faceless governance. For example, I have given up on Instagram entirely. Why support a brand that views fat sex workers as so abhorrent that we shouldn’t exist on their platform? The poor reach and censorship has also demotivated me on creating new pieces and made me scared to post anything they wrongly deem inappropriate and lose my account which will impact on my income even more. I am forever finished doing art full time.
The loss of a traditional, offline “work identity” is connected with low self-esteem and a disrupted sense of self (Allan et al., 2021; Sverke et al., 2002). Digital content creators, especially those who work through a carefully crafted identity and through their body, face an even deeper loss: a personal rejection of who they are, a feeling of not being worthy of a platform they need for work, but which deems them unacceptable and dangerous. Indeed, users’ reaction to the governance they experience often has a strong personal dimension, as if platforms were targeting them due to their physical characteristics and appearance, gender identity, sexual orientation, job, or hobby. Many described their feelings with words such as “shame,” “disheartening,” and “upsetting,” claiming they did not feel good enough to be visible on platforms: [Censorship] made me feel judged, sad, ashamed of myself once more. [. . .] Someone could take their own life soon. It’s so disheartening knowing that being a fat sex worker is seen as so harmful to the world that we need to be deleted from it. The situation just leaves me feeling like I’m not good enough for a platform that I put so much heart and soul into. [. . .] When TickTock (sic) doesn’t feel like you are worth being shown it really does affect you mentally. I know it’s just a platform but I have cried myself to sleep many of nights trying to figure out what I can do differently. I’m not good enough I’ll (sic) because of what I do to protect my family.
Such experiences show the damaging effects of social media platforms’ processes. It’s not just about “being beautiful enough” as may happen with the loss of self-confidence in teenagers (Sherman et al., 2016) and young women (Scully et al., 2020). Instead, de-platforming is perceived as a crippling fundamental attack on self-worth and personal well-being. Users were confused as to why Instagram and TikTok would encourage celebrities and verified influencers to produce sensual, nude, or sex work-adjacent content (Are & Paasonen, 2021) while denying them the same rights. By targeting them through censorship and exposing them to online abuse for doing the same, they feel platforms are indirectly saying that they are
Many participants described their experience as a kind of grieving process, where the loss of work and a community left them bereft. They described the loss as “crushing,” triggering feelings of “depression” and leading to what they felt was a “devastating” failure to work and exist in digital spaces: It was truly a horrendous grieving process for me. I’d lost everything, all my work and dedication. I started a new account which felt like starting a business again from scratch which could be taken away again at any moment. I lost my whole community, friends and contacts. I felt very isolated. I’ve lost money, a community, content, and an entire following. I worked so hard on it for the past two years. It happened almost a month ago and I’ve felt anxious, depressed, and sick since. My depression has been triggered so intensely from all of this that I barely paint or create at all anymore. I work a few different jobs now to try to make up for the fact that I’m now a failed artist. The impact is indescribable and devastating.
Respondents describe unquantifiable damage caused by de-platforming arising from a personal, deep emotional connection with platforms, most likely arising from Instagram and TikTok’s community-building affordances. Grieving caused by de-platforming blends the loss of personal and professional connections, work and opportunities, the efforts, trust, and digital labor placed on companies that do not value it and, even, on the memories made and forever lost with the flick of a switch. For some, this resulted in a sense of resignation, a recognition that they’d simply have to find work elsewhere, and in the refusal to engage with platforms’ moderation: It has destroyed my income, it has destroyed my connections in my industry, it has destroyed my relationships. I wish I could promote more such as my videos on Patreon since it is art at the end of the day, but the risks are too high and having your engagement cut does impact viewers and potential supporters joining. I didn’t have enough time or energy so don’t bother using IG anymore.
Others were left with an uncomfortable dependence upon platforms that didn’t seem to want them: “I wish I didn’t have to use Instagram to promote, but that’s where everyone seems to be,” a creator wrote. Participants described the stress of continuing to work in such a hostile environment, and the need to be more vigilant when posting content on their Instagram and TikTok profiles: Financially, it means I am unable to run my business as effectively as I am always thinking of how I can strategically post and share products. As an artist, it does put your creative process into question in the sense you begin to consider alternatives to making your art so it stays up which is a negative outcome of Instagram’s censorship. I had all my posts put under review for upwards of 6-8 hours before tiktok let them through. I honestly felt like I was being constantly surveilled/watched, it was unsettling. It caused me to not post again for some time.
With the constant threat of de-platforming, users reported a chilling effect on their ability to market themselves, promote their work or even to creatively express themselves as they were always second-guessing the acceptability of their content and looking out for the threat of deletion. Users have had to add an extra layer to their digital labor, monitoring algorithmic gossip (Bishop, 2019) and folklore (Savolainen, 2022) to avoid trouble. In absence of communications with Instagram and TikTok, and in the face of their opaque and often unequal governance, users have had to take on the burden of their own protection, plagued by a constant feeling of being watched. Such users are effectively experiencing their time on the platform as dead men walking, waiting for de-platforming to hit them because of the content they share.
Yet, despite the disheartening experiences many users report, they also engaged in several acts of defiance against governance systems they do not understand or resonate with. Users continued posting content against the rules, posting links to platforms known to be flagged by Instagram and TikTok such as OnlyFans (Are, 2021a), promoting adult material without directly posting sexually explicit content, for example: I let it go as technically it is against the rules. Still, there are very little places to promote queer and trans adult media or to share publicly information that helps people learn about sex and not feel so ashamed about it. While they didn’t show nudity the captions were promoting adult material or discussing sexuality so it was against Terms of Service.
By defying platform governance, even in ways that may prove detrimental to the survival of their profiles, users are reclaiming spaces, making a statement regarding the worth, demand for, and rights of their art. This defiance can be seen as a form of resistance and protest, particularly in a scenario where the space for sex and nudity online is shrinking (Are & Paasonen, 2021), and it represents a general understanding that, while Instagram and TikTok rule over content as private entities, they are also presiding over a space that users have reclaimed as a civic and creative zone (Are, 2020).
Conclusion and Recommendations
Building on previous research on the impact of precarity on gig economy workers, we found that de-platforming arising from algorithmic insecurity (Hill, 2021) and the “algorithmic boss” (Duffy, 2020) has concerning negative impacts on the well-being of Instagram and TikTok creators at the margins. Sex-working content creators, who have directly experienced the harmful consequences of FOSTA/SESTA and anti-sex technology policies, bear the brunt of de-platforming, but the policies that affect them have quickly been trickling down to other creators who post nude content (Are, 2022; Bronstein, 2021). We found that these users faced a range of adverse outcomes: job and income uncertainty, with associated feelings of powerlessness, a loss of digital identity, and enforced isolation from a previously established social network.
Our study provides crucial testimonies grounded in user experience, showcasing how Instagram and TikTok offer more than just a creative space to content creators: they are a workplace and offer networking and memory-making opportunities, the loss of which is devastating. Creators depend upon these platforms for their lives and livelihoods, yet those who use nudity in their work are surveilled by those same companies, who provide no opportunity for recourse for these creators to be heard or ask for help once they are locked out of their workplaces.
Instagram and TikTok are quick to censor content and de-platform creators, effectively coding
Sex workers and users who post nude content are not afforded the same rights or, even, humanity as other users. This is particularly concerning for those whose only work opportunity is online sex work or content creation. This is a particularly salient point given that 16 out of 41 respondents to this study were using content creation as an accessible form of work and expression because they have a disability (Stokel-Walker, 2021b).
Far from welcoming the everyman and the previously marginalized and excluded, platform governance and the creator economy are actually replicating offline inequalities (Glatt, 2022; Nolan-Brown, 2022). We therefore conclude our paper with three recommendations arising from participants’ experiences. To tackle the adverse impacts of their governance on users’ income, reputation, and well-being, platforms should:
Footnotes
Acknowledgements
We would like to thank everyone who shared and took part in this survey, particularly those who shared personal, traumatic experiences of de-platforming toward advancing knowledge in this field.
Correction (February 2024):
Article updated to correct the grant number in the funding section on page 10 to reflect as EP/T022582/1.
Data Availability
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Research Councils UK > Engineering and Physical Sciences Research Council EP/T022582/1.