Criminogenic and harm-enabling features of social media platforms: The case of sharenting practices

Abstract

Sharenting – that is, the sharing of identifying and sensitive information of minors, who are often overexposed online by parents or guardians – has, at times, criminogenic potential, as the information shared can enable both heinous crimes and other types of harmful conduct. Whilst most research on sharenting has focused on the sharenters and their agency, there is a gap in addressing whether and to what extent social media platforms display criminogenic or other harm-enabling features that can render sharenting risky for affected minors. By relying on an adapted crime proofing of legislation approach, our contribution analyses the self-regulations (in the form of corporate documents and forms of self-organisation) of five major social media platforms and identifies several risks and vulnerabilities to harmful sharenting practices embedded in the platforms’ policies. In doing so, the study demonstrates how criminological imagination can effectively contribute to the multidisciplinary debates on digital ecosystems and their regulation, paving the way for a reduction of criminogenic and harmful opportunities online.

Introduction

As digital technologies continue to permeate most aspects of social life from work and social networking to criminal justice, they are creating ‘digital paradoxes’ by enabling positive social changes and improvements to the quality of life, whilst generating new threats to safety and wellbeing (Fussey and Roth, 2020). These new techno-socialities (Escobar, 1994) are particularly interesting from a criminological perspective, beyond the mainstream cybercrime literature, as individuals interacting in non-criminal virtual spaces can give rise to criminal, deviant or otherwise harmful activities and behaviours (Powell et al., 2018; Lavorgna, 2021). In this context, mainstream social media platforms play a key role, providing highly populated virtual spaces. These platforms are best understood as sociotechnical assemblages and complex institutions (Gillespie, 2017), and are often conceptualised as composite human actors (users and, depending on the platform, moderators) and algorithm-driven nonhuman entities (automated tools and filters) embedded in their users’ general communicative practices (in line with Prochazka, 2019).

In this contribution which originates from the ESRC-funded project ProTechThem - Building Awareness for Safer and Technology-Savvy Sharenting, we hypothesise that virtual places have regulatory characteristics and loopholes that can enable harmful practices amongst users. We demonstrate this using the empirical example of harmful sharenting practices in online social media spaces. Sharenting is the potentially harmful sharing of identifying and sensitive information of minors, who are often overexposed online by parents or guardians.

Whilst most research on sharenting has focused on the sharenters and their agency, the characteristics of the social media platforms they inhabit, which are at the basis of their potentially harmful sharing activities, have been so far overlooked. To address this gap, we draw on insights from the ESRC project which analysed the self-regulatory policies and practices of five mainstream social media platforms. The aim was to examine whether and to what extent the platforms display problematic features, and whether there are regulatory loopholes that can render sharenting risky for affected children by enabling criminogenic and other harm-enabling opportunities. To achieve our objective, we utilised an adapted crime proofing of legislation approach which is a conceptual framework for identifying criminogenic opportunities in regulations and their implementations (see Savona, 2017). This allowed us to examine whether criminogenic or otherwise harm-enabling features are embedded in the policies. Of course, what we are addressing in our work is a complex sociotechnical phenomenon, based on the interplay between policies, platform design and developments, and social norms and attitudes. We do not aim to provide a comprehensive understanding of the issue, but rather to shed some light on one of the factors accounting for important (criminal and non-criminal) harms and risks that social media platforms are currently failing to address – that is, their self-regulatory practices. As such, in this contribution we are focusing on the orientation of social media platforms towards the legal risks relating to sharenting, rather than, for instance, on other relevant factors such as their design. In the next section, we offer a brief overview of the nature and limitations of self-regulation (via content moderation) as a core governance mechanism of social media platforms.

Framing the ecosystem: Self-regulation and moderation in social-media platforms

A key dilemma of contemporary digital policy relates to the issue of how best regulate online media, particularly social media platforms, as they are fundamental digital intermediaries in contemporary societies (Milosavljević and Micova, 2016; Cusumano et al., 2021). At the moment, self-regulatory mechanisms (and especially corporate documents and forms of self-organisation, consider for instance the terms of service or content policies created by the social media companies themselves) play a core role. After all, self-regulatory initiatives are not novel developments, as they are often used by industries to improve their public reputation (also relatively to their competition), and to avoid more costly regulation (Abbott and Snidal, 2009; Milosevic, 2017; Gorwa, 2019).

Of course, self-regulation alone is not enough; appropriate regulatory procedures are also required (consider, for instance, Zancova and Dimitrov, 2020; Tyler et al., 2021). More generally, there have been calls for increased accountability of social media platforms (for instance, see Leerssen, 2015) and for better regulations and controls backed by a strong sanction regime against misbehaving platforms (Wise, 2019). Concerns over the data practices of social media platforms and privacy violations, for instance, have led to harsh criticisms, with claims that they should not be permitted to behave like ‘digital gangsters’ in the online world, considering themselves to be ahead of and beyond the law (DCMS, 2019: 42). Beside self-regulation, in the European region, several multi-actor informal arrangements governing online content on social media platforms are in place. They are likely to play an increasingly important role in the coming years as a vital part of the corporate regulatory toolbox (Gorwa, 2019), which can help avoid issues of techlash (Douek, 2019). In recent years, social media platforms seem to be entering a new phase of responsibility towards the public, but solutions to commonly cited problems such as privacy violations and poor content moderation have not been offered. Thus, current self-regulatory strategies seem to constitute nothing more than a smokescreen or digital washing. Nevertheless, they remain the primary form of regulation and are, as such, pivotal to understanding the criminogenic features of the platforms and how they are addressed. Self-regulation also mostly defines the relationship between social media platforms and their users – that is, the digital ecosystem of our interest.

As digital intermediaries, social media platforms are constantly muddling through balancing acts between users’ ability to post freely (promoting individual participation and fostering discourse democracy) and preventing harms that may arise from posted content, whilst operating in a context whereby the platform is commodifying users’ content and data to make a profit (Johnson, 2017). Self-regulatory practices can favour this balance, creating a trusted partnership between users and social media companies (Schneble et al., 2021). But they still fall short, operating in a ‘logic of opacity’ (Roberts, 2018). It has been claimed, for instance, that terms and conditions, and rule of conducts, remain obscure to users, as regulations are written in long and complex language (Schneble et al., 2021).

In the context of self-regulatory practices, content moderation can be defined as the ‘governance mechanisms that structure participation in a community to facilitate cooperation and prevent abuse’ (Grimmelmann, 2015: 6). Moderation can take different forms: it can be automatised or manual, and it can range from centralised corporate moderation to user-driven moderation models (Seering, 2020). Moderation can have both a punitive and an educational role (West, 2018). Either way, moderation systems have a role in shaping affective relationship between users and platforms, and for users to assert their agency, for instance by seeking redress (West, 2018). As such, they are a first point of intervention for preventing and countering several criminal, deviant, or otherwise harmful activities online, in the effort to create, nurture or maintain a better social media platform ecosystem. Following Seering's (2020) distinction among ‘the platforms and policies perspective’ and ‘the communities perspective’ in moderation research, acknowledging that previous research has already stressed how platforms generally allow users significant leeway to self-moderate (for a review, see Seering, 2020), and fully recognising the importance of intra-group moderation, this study departs by looking at platforms and policies as – we claim – these provide the overarching frame within which also users’ intra-group moderation occurs.

Over the years, automated ways to moderate social media have been developed by social media companies, mostly through artificial intelligence (machine learning) tools based on natural language processing and sentiment analysis, with the intent of removing ‘bad’ content more effectively and quickly (see, for instance, Gorwa et al., 2020; Lim et al., 2020). These algorithmic moderation systems are increasingly used for user-generated content moderation at scale by all the major social media platforms, fuelled by growing public expectations for increased platform responsibility (for an excellent recent overview of algorithmic moderation, see Gorwa et al., 2020). Unfortunately, these systems are still opaque, unaccountable, and scarcely effective in complex sociotechnical systems, with the risk of further complicating and exacerbating issues of (lack of) fairness and justice (Gorwa et al., 2020) as it is extremely difficult for automated tools to make contextual decisions on composite, multifactorial concepts (Li and Williams, 2018). Additionally, intervening with an automated mechanism against something in violation of a platform's standards does not mitigate concerns on how certain standards are created, as at the core of these concerns are issues of transparency, fairness and depoliticisation (‘where do we draw the line between what is acceptable and what is not?’) that are socio-political and ethical, rather than technical, in nature (as discussed in detail in Gorwa et al., 2020). Yet, as stressed by Gorwa and colleagues (2020), these systems are here to stay, as they are now often mandated by both legislation and informal platform regulation.

In commercial content moderation, the involvement of more humane, trained and diverse moderators in the process is often seen as a practical solution; when manual moderation occurs, however, this is nonetheless done in less-than-ideal conditions, often by (outsourced) freelancers with poor working conditions, and exposed to extreme amounts of toxic content (Milosevic, 2017; Gillespie, 2018). Furthermore, also in those cases, moderation processes are generally still obscure, even if data disclosures are emerging (Gillespie, 2018; Keller and Leerssen, 2019). Opacity, in a way, seems to be a distinguishing feature of platform design, as content moderation practices are entangled in a nebulous web of rules and procedures. In this context, the key criterion in commercial moderation seems to be the potentially revenue-generation value of a certain content, rather than its meaning and intent; as such, platforms become an instrument for the reification and consolidation of pre-existing power structures (Roberts, 2018; Santos Rutschman, 2021).

From this brief excursus, it is evident that analysing self-regulatory mechanisms is necessary for assessing whether and to what extent social media companies are addressing the criminogenic or otherwise harming potential of their platforms. As already noted, we use harmful sharenting practices to explore this issue empirically, with a focus on whether the self-regulatory strategies instituted by social media platforms encompass potentially problematic features that can trigger or exacerbate sharenting risks and harms.

The next section will provide an overview of sharenting practices, risks, and harms. The section will clarify why these matter from a criminological perspective and present necessary multidisciplinary information.

Sharenting and its ecosystem

Sharenting can be defined as ‘making public by parents a lot of detailed information about their children in the form of photos, videos and posts through social media, which violate children's privacy’ (Brosch, 2018: 78). Especially over the last decade, sharenting has received scholarly attention by several disciplines, including law (e.g., Steinberg, 2017; Hancock, 2021), media, communication and cultural studies (e.g., Chalken and Anderson, 2017; Choi and Lewallen, 2018; Archer, 2019; Ranzini et al., 2020; Barnes and Potter, 2021); computer science (e.g., Ammari et al., 2015); educational sciences (e.g., Cino and Damozzi, 2017; Di Bari, 2017; Brosch, 2018); and psychology (e.g., Lazard et al., 2019). So far, however, this common social practice has been almost ignored by criminological scholarship (Lavorgna et al., 2022), which is surprising as we consider that, beyond risks posed by negative psychological repercussions in ignoring children's desire to having (or not) an online identity (Steinberg, 2017) or due to the perpetuation of gender and racial stereotypes (Choi and Lewallen, 2018), there are concerns regarding the potential for financial exploitation (Archer, 2019; Barassi, 2019), grooming and child abuse, cyber hate and identity crimes (e.g., Bezáková et al., 2021; Wachs et al., 2021; Williams-Ceci et al., 2021). Indeed, recent research has shown that, despite the potential under-emerging and underreporting of cases where sharenting has led to the victimisation of minors, there are systemic vulnerabilities in current sharenting practices that can cause the perpetration of harms (Lavorgna et al., 2022).

Sharenting has been linked to contemporary ‘societal showcaseisation’ (Codeluppi, 2007), the desire for visibility and the creation for money-making opportunities by parents working as social media influences (as in the case of so-called ‘mumpreneurs’, see Archer 2019, but also by ‘Instadads’ performing ‘sharenting labour’ as studied by Campana et al., 2020). In-depth analyses of sharenters, however, reveal a more complex reality, with many parents facing ethical dilemmas and hesitations in sharing information about their children online, as social media platforms offer a range of affordances that were once unimaginable (Chalklen and Anderson, 2017; Cino and Demozzi, 2017; Brownlie, 2018; Archer, 2019). Overall, they try to enhance their self-representational agency and ability to build community and social capital, whilst remaining mindful of surveillance and potential risks related to children's exposure in digital spaces (Chalklen and Anderson, 2017; Cino and Fomenti, 2021). At the same time, they must contend with data technologies that reinforce the cultural value of archival time and encourage users to track and document their lives (Barassi, 2020). Additionally, virtual spaces matter for parenting as places to gather experiential information, and social and practical support (Johnson, 2015; Kumar and Schoenebeck, 2015; Ranzini et al., 2020). Of course, the level of disclosure can vary, both in breadth (i.e., the amount of disclosed information, which includes the frequency and duration of disclosed contents), and in depth (which reflects the level of intimacy) (Wheeles and Grotz, 1976).

A core problem is that much of the focus of studies exploring digital parenting has been on offering guidance to adults on how to best manage screens and media for their children (Uhls, 2015). But parents (and, more generally, guardians, including teachers), whilst being expected to educate and protect children online, are unprepared for their role as ‘digital custodians’ (Buchanan et al., 2019: 175). They should be the first line of defence, but at times are not (Brosch, 2018). In most countries, there are currently no policies securing children's right to online privacy, and the decision on whether and how to disclose information online is left in the parent's hands. This makes them the principal gatekeepers of the personal information of their children online (Steinberg, 2017; Brosch, 2018).

It can be argued that sharenters’ agency is somehow constrained by the ‘systemic coercion of digital participation’ at the basis of the surveillance capitalism (Barassi, 2019: 415) so embedded in our ‘onlife’ (Floridi, 2015; see also Barassi, 2020; Chayko, 2020). This issue can be considered as part of the ‘next generation privacy problems’ (Zittrain, 2008: 205), as individuals are enabled to deeply compromise privacy through the generative technologies currently in use (generative meaning their capacity and purpose is beyond what was first imagined when they were created, see Zittrain, 2008). We also need to take into consideration that children nowadays are ‘datafied’ in numerous ways, as they digitally participate to society without their consent or control (Lupton and Williamson, 2017; Barassi, 2019).

As already noted, most research on sharenting focus on the sharenters and their agency. The structural, regulatory characteristics of the social media platforms they inhabit, which are at the basis of their potentially harmful sharing activities, have been so far ignored. Nonetheless, these platforms’ characteristics are pivotal, as they are an integral part of the digital ecosystem of interest. In what follows, we present our study which explored whether and how five mainstream social media platforms respond to potentially harmful sharenting practices. In doing so, we unravel several criminogenic and harm-enabling features of the self-regulatory strategies framing the platforms.

Methodology

Criminologists have long argued that regulations and their implementation can sometimes be counterproductively and unintentionally criminogenic (Sutherland and Cressey, 1978). In more recent years, crime proofing of legislation was developed as an approach to assess existing or future opportunities for crime due to legislation and indicate potential interventions aimed at proofing it against crime, and proved to be a valid form of risk assessment and management (Albrecht and Kilching, 2002; Russell and Clarke, 2006; Transcrime, 2006; Morganti et al., 2020). Crime proofing of legislation stems from crime opportunity approaches, sharing the core idea that opportunity is a root cause of crime (Clarke, 2012) and aiming to reduce crimes by looking for crime patterns in specific settings (Felson and Clarke, 1998) – including the regulatory ones. Crime proofing of legislation was initially developed to assess the risk of unintended consequences produced by legislative measures. It enabled policy makers to evaluate legislation in the law-making process and to suggest changes (e.g., textual changes) to reduce the crime risk (see, for instance, the MARC model in Transcrime, 2006; Savona, 2017). The approach has been adapted over the years, for instance to assist the systematic analysis of norms regulating a certain market to assess whether they might have an unintended criminogenic role, or whether there are major loopholes in their implementation that could be criminally exploited (consider, for instance, Lavorgna et al., 2018).

Our approach furthers this latter trend by using an adapted form of crime proofing of legislation as a framework to analyse aspects of the relevant text and identify in the self-regulations of social media platforms criminogenic or otherwise harm-enabling opportunities for harmful sharenting. In our revised model, we adjusted the approach used by Lavorgna and colleagues (2018, which was in itself an adaptation of Transcrime, 2006). This was necessary because we focused not only on fully-fledged crimes but also on harmful behaviours, and on existing self-regulations rather than on formal legislation under development. The intended consequences of the self-regulatory practice, as discussed above, are to govern and manage users’ activities and relationships (among themselves and with the platform), fostering their participation whilst preventing and mitigating crimes and harms, with a focus on the legal risks they might produce. The unintended consequences, for the scope of our work, are the facilitation of harmful sharenting practices, or the failure in addressing them. The mismatch between the intended and the unintended consequences was assessed by looking at a series of indicators.

First, we distinguished a regulatory risk element (rather than a ‘legislative crime threat’, as in previous studies), based on the following indicators: accessibility of information (which includes accessibility in practice, and clarity of content); consistency (temporal; internal; across platforms); regulatory gaps (to what extent harmful sharenting is addressed); and implementation mechanisms (to address harmful sharenting). Second, to assess sharenting vulnerabilities (rather than market vulnerabilities, as in previous studies), the following indicators were considered: attractiveness (which, in our case, depends on desirability of sharing potentially sensitive information in a specific social media context), shareability and availability (respectively, how easy is to share potentially sensitive information, and the capacity to access shared material by a third party), and lack of guardianship (moderation practices at different levels).

It is worth noting that, by adapting a method derived from crime opportunity approaches, we do not want to deny, or minimise, the fact that social media use reflects, and depends on, a range of cultural, psychological, and even emotional habits, patterns and needs (e.g., Waters and Ackerman 2011; Hu et al., 2018, Stsiampkouskaya et al., 2021; Bayer et al. 2022); also, we acknowledge that most users may not read or consciously agree to platform policies. Indeed, a major limitation of this approach is that it does not fully capture how sociotechnical affordances enable and constrain behaviours (Gibson, 1977; Hutchby, 2001; Bloomfield et al., 2010). Nonetheless, as discussed above, self-regulations – by defining an important part of the relationship between social media platforms and their users, hence fabricating and delimiting our digital ecosystem of interest – remain pivotal to assessing whether and to what extent social media companies are addressing the criminogenic or otherwise harm-enabling potential of their platforms.

For our analysis, we focused on what we expected to be five social media platforms with a major role in sharenting practices. They were selected considering their demographic distribution among the population and following a discussion with our non-academic project partners ¹ . The platforms are: Facebook; TikTok; Instagram; You Tube; and Twitter.

Textual data were retrieved (from the UK) from each of the platform's relevant self-regulatory documents (mostly Terms and Conditions, and Community Standards, in their English version). Some additional documents targeting, for instance, specifically parents were also identified and considered for the analysis. The data we used are publicly available online. After an initial screening, all sections of the documents that could (broadly) be relevant to sharenting, digital harms and crimes, moderation practices, minors, and their parents/guardians were selected for analysis (updated on November 30, 2021). These amounted to a total of 330 pages (108,502 words), available at [to be added after peer review]. We then performed a content analysis on those documents with the support of NVivo12, based on the following main codes, which are at the basis of the indicators mentioned above and discussed in the following section. The codes were created based on insights from emerging data and the extant literature, and they are: platform; relevant platform's metadata; platform's age limitation; provisions specific to minors (including definition); content monetisation; specific provision on someone else posting material on a subject; specific (harm/crime) risk that might occur if someone else is posting material on a subject/on behalf of a subject; crime/harm addressed; provision specific to parents/guardians/educators; sharenting if specifically addressed; content moderation practices; other. For the sake of brevity, a summary of selected descriptive results for exemplary purposes is available in Appendix A.

Identifying and discussing criminogenic features

Regulatory risk

Accessibility of information

In terms of accessibility of information, the regulations analysed are problematic. Even if they can be easily accessible online via the platforms or standard search engines by any interested party and the language used is generally clear, the relevant information is dispersed across a number of documents, sections and pages, making it hard to reach in practice. The scale of information to be navigated is extensive, making it unlikely that the standard user will be aware of relevant provisions. Parents/guardians and minors are addressed directly only in a few cases.

Consistency

Temporally, the documents analysed are relatively stable (considering the speed of innovations in the sector). Versions of the documents were last updated in the period ranging from May 2018 (Twitter User Agreement) to October 2021 (YouTube Channel Monetisation Policies and TikTok Privacy Policy). Updates, however, are not easily traceable by users, who realistically are not aware of changes in policies.

The internal of consistency of regulations (within each platform) is generally good, even if there are some provisions that could create confusion, especially when it comes to the age limitations set to use a specific platform or some of its services. These provisions are of interest in the case of sharenting practices as their presence suggests that a platform might have content inappropriate for those below a certain age but who, paradoxically, might find themselves exposed in that very same platform. On this matter, an example of internal inconsistency is provided by YouTube, where the Terms of Service states that minors under 14 can access contents only with parental guidance and with specific restrictions, but a separate section targeting specifically parents sets the access age at 13.

Consistency across platforms, not surprisingly, is far more problematic. Staying on the example of age limitations we can note differences regarding, for instance, the minimum age for subscription (generally 13 years, but 16 years for services such as TikTok Live Stream Program or Twitter Periscope). There are also disparities in policies on access to specific content, content creation, or the age under which parental control is necessary. These differences across platforms can create confusion in a context where most users operate across platforms in their digital modus operandi (Tandoc et al., 2019).

Regulatory gaps

Harmful sharenting falls within a regulatory gap in the platforms observed, with only some of its manifestations (namely, those linked to potential sexual abuse and some serious forms of antagonistic online behaviours) being taken into proper consideration in the platforms’ ecosystem.

For instance, we found a number of provisions addressing the possibility of someone posting material on another subject (e.g., a child), and the provisions at times also identified specific (crime/harm) risks, through detailed lists (as in the case of Facebook and TikTok) or through more general provisions. All platforms showed attention to the issue of posting material on another subject, e.g., Instagram:

‘You can't post someone else's private or confidential information without permission or do anything that violates someone else's rights, including intellectual property rights (e.g., copyright infringement, trademark infringement, counterfeit or pirated goods). […] Post only your own photos and videos, and always follow the law. […] Share only photos and videos that you've taken or have the right’.

But only some platforms had specific provisions regarding minors. None addressed sharenting in general as problematic, with platforms rather focusing on a limited number of sharenting-enabled crimes and harms.

TikTok, for example, focuses on the limitations concerning the use of material which is confidential or property of someone else, offering a detailed list of contents which can’t be shared/posted (e.g., ‘any material, i.e. deliberately designed to provoke or antagonise people, especially trolling and bullying, or is intended to harass, harm, hurt, scare, distress, embarrass or upset people’). Instagram offers (less detailed) provisions focusing on the need to avoid abusive behaviours (e.g., sharing nude or sexual photos/videos is considered violation of Instagram's Community Guidelines). But minors are never specifically mentioned in this context. Facebook, on the other hand, provides several specific provisions about avoiding posting material that could led to bullying and harassment (e.g., by forbidding ‘content manipulated to highlight, circle or otherwise negatively draw attention to specific physical characteristics […]’), or the sexual exploitation of children. Most of these previsions are then specified in a detailed way (e.g., the topic of non-sexual child abuse is addressed by considering imagery posted by news agencies depicting children in sensitive contexts). However, specific alerts concerning the ‘mere’ posting of textual or visual information on minors by parents or guardians are never addressed as a specific matter of concern, with the exception of the potential for sexual exploitation of children through the sharing of images. Statements illustrating this include:

‘We do not allow content that sexually exploits or endangers children. […] We know that sometimes, people share nude images of their own children with good intentions; however, we generally remove these images because of the potential for abuse by others and to help avoid the possibility of other people re-using or misappropriating the images’.

There are some provisions specific to parents, guardians and educators, but also in these cases sharenting is not addressed in its entirety. For instance, TikTok offers detailed provisions to parents/guardians/educators through its Terms of Service and a Guardians’ Guide. However, this information seems to address how teenagers use TikTok, and not issues concerning sharenting. In other words, the orientation that TikTok documents offer is useful to guide guardians in supervising or monitoring minors actions on the platform. But it does not mention directly the sharing practices of these guardians which instead are ruled by the general terms of use for every adult users. Similarly, YouTube's guidance to guardians is limited to the control of the kind of contents that children can watch, the time they spend on the platform, and similar issues. Thus, the platform highlights the responsibility of parents and guardians in monitoring children's activities on the platform. Compared to the other platforms, Instagram appears to address sharenting more explicitly, but again the focus is mostly limited to the potential for sexual exploitation:

‘People like to share photos or videos of their children. For safety reasons, there are times when we may remove images that show nude or partially nude children. Even when this content is shared with good intentions, it could be used by others in unanticipated ways. You can learn more on our Tips for parents page’.

Implementation mechanisms

In order to intervene in existing illicit conducts and to prevent future ones, all social media platforms have a number of implementation and enforcement mechanisms which include, among other things: reporting tools; blocking and filtering; geofencing (i.e., banning a certain geographical location to access a social media platform); various forms of manual or automated moderation; educational material (Milosevic, 2017). With reference to the provisions explored to far, overall, in all five platforms, implementation mechanisms are primarily based on some form of collective responsibility at community level involving the monitoring of material posted, and reporting it if and when appropriate (e.g., on YouTube: ‘If you see a video that you feel is inappropriate, flag the video’). As such, the reporting is delegated to the community (similar to what was noted by Milosevic in regards to cyberbullying) – as if it is ‘primarily the responsibility of the community to regulate itself rather than have the company regulate the community’ (Milosevic, 2017: 114). Additionally, posts are then removed if their content is among those listed as forbidden. As such, harmful sharenting is not addressed through specific enforcement mechanisms, with the exception of its more extreme manifestations as discussed above. Similarly, compliance with other requirements issued by legislators (such as age requirements verification) is not realised proactively. An example is the apparent neglect of age limits which are only enforced when reported by other users as already stressed by Schneble and colleagues (2021).

It is worth noting that, despite their presence on the platforms considered, minors are relatively disregarded as agentic users in the documents analysed, and rather addressed as an audience to be protected (e.g., YouTube: ‘You’re required to tell us that your videos are made for kids if you make kids content. As a YouTube creator, you are required to set future and existing videos as made for kids or not’). Specific provisions addressing minors generally focus on age limitations (as discussed above). In the case of Facebook there is a provision specifying that minors between 13 and 18 can report content if they notice a misuse of their images or videos. As such, according to Facebook, the following content also may be removed: ‘A reported photo or video of people where the person depicted in the image is: A minor under the age of 13, and the content was reported by the minor or a parent or legal guardian. A minor between the ages of 13 and 18 years old, and the content was reported by the minor’. Instagram seems to provide more attention to the topic, addressing minors directly, for instance, encouraging them to report bullying and harassment through its Help Center, and to reflect on the opportunity to share nude or sexual photos or videos.

Sharenting vulnerabilities

Attractiveness

The sharing of potentially sensitive information of minors can originate from motivations ranging from parental pride and social isolation, to the search for social influence and profit (Lavorgna et al., 2022). Whilst some of these factors are inherent in the nature of social media platforms (Haillikainen, 2015), some can directly increase the rewards for harmful sharenting practice. Consider, in particular, the monetisation of certain content. TikTok and YouTube, for instance, provide specific self-regulatory provisions about this issue, with a few references to minors and age limitations, but again, with minors considered as potential audience and not as potential ‘content’. For example, in defining contents which are not suitable for monetisation (e.g., ‘content that incites hatred against, promotes discrimination, disparages, or humiliates an individual or group of people’), there is no specific provision for protecting the exposed identities of minors.

Shareability and availability

In all the platforms considered, it is extremely easy for anyone enrolled to the platform to share potentially sensitive information of minors; there are no awareness raising tools or other mechanisms intervening, unless the content is manifestly forbidden as discussed above. Similarly, it is easy for a third party to access (and re-share) the material posted, with limited risk of detection. It is possible to set the privacy settings of both individual accounts and groups/pages to limit these possibilities, but these are not the default options.

Lack of guardianship

Relevant content moderation practices can occur at both platform and community/individual levels. Platform moderation generally takes place in automatised ways, with human moderators being kept ‘in the loop’ and intervening when needed. The ‘logic of opacity’ (Roberts, 2018) previously discussed is found also in this context, as the details of the training (of software; of human beings) informing moderation practices are not publicly available, and the information available on the standards applied to moderation is obfuscated. This opacity by design affects standard moderation and reporting practices, as well atypical procedures envisaged by some platforms, such as expert moderation or social reporting (e.g. on Facebook). It is expected that the latter in particular will be used alongside the regular reporting to try to resolve issues not listed by corporate policy. The procedures are accessible through a series of prompts whose wording is based on the user's characteristics. Such procedures might have the potential to counter digital harms, but they have to be regularly and independently evaluated (as discussed also in Milosevic, 2017: 122ff).

For the more serious manifestations of sharenting addressed by the platforms, there are actions to protect minors at the platform level. Consider, for instance, the following snippet from Facebook Community Standards:

‘We have built a combination of automated and manual systems to block and remove accounts that are used to persistently or egregiously abuse our Community Standards. […] We comply with: Requests for removal of an underage account. Government requests for removal of child abuse imagery depicting, for example, beating by an adult or strangling or suffocating by an adult. Legal guardian requests for removal of attacks on unintentionally famous minors. […] We may remove content created for the purpose of identifying a private minor if there may be a risk to the minor's safety when requested by a user, government, law enforcement or external child safety experts’.

But, again, there are no instruments to intervene in less extreme yet potentially harmful cases. As such, should an individual signal the presence of a potentially inappropriate post, which for instance, discusses some sensitive information concerning a minor, there is no clear remedy.

Here, also the discretionary power of the platforms is evident, not only because of the use of modal verbs such as ‘may’, but also because of the platforms’ ability to unilaterally decide whether and how to intervene. On this aspect, consider also the following excerpt from TikTok Terms of Service:

‘[…] We have the right to remove, disallow, block or delete any posting you make on our Platform if, in our opinion, your post does not comply with the content standards set out at Section 7 (Your Use of Our Services) above. In addition, we have the right – but not the obligation – in our sole discretion to remove, disallow, block or delete any User Content (i) that we consider to violate these Terms, or (ii) in response to complaints from other users or third parties, with or without notice and without any liability to you. […]’.

Instagram and YouTube offer a more balanced approach between the several moderation levels, requiring that the users become active in reporting harmful material, and even reminding users of their agency in controlling digital abuse. Consider, for instance, this snippet form YouTube Community Guidelines (Reporting and Enforcement):

‘We rely on YouTube community members to report, or flag content that they find inappropriate. Reporting content is anonymous, so other users can't tell who made the report. […] When something is reported, it's not automatically taken down. Reported content is reviewed along the following guidelines: Content that violates our Community Guidelines is removed from YouTube. Content that may not be appropriate for younger audiences may be age-restricted. […] If you see a video that you feel is inappropriate or which may violate our Community Guidelines, flag the video. […] YouTube policy specialists review flagged videos 24 h a day, 7 days a week. […] If you feel that your child's privacy has been violated (e.g., use of image or personal information without consent), please visit our Privacy Guidelines, where you can learn more about our privacy policy and how to file a privacy complaint’.

Twitter, on the other hand, emphasises the role of community moderation, by acknowledging the limits of its ability to automatically intervene from a platform's level (‘We may not monitor or control the Content posted via the Services and, we cannot take responsibility for such Content’). This suggests that, even if Twitter can take actions against violations, the responsibility to respect the rules is assigned to the users.

Further discussion and conclusion

We have discussed the regulation and moderation instituted by major social media companies which, through their self-regulations, set the boundaries of what contents and behaviours are allowed on their platforms. We argue that unpacking the self-regulatory strategies is imperative to deepen our understanding of their dynamics, rhetoric, and effectiveness (in line with Wyatt, 2008). This is necessary since they define the digital ecosystem formed by the relationship between social media platforms and their users. As such, they frame users’ perceptions and practices in a climate of increasing privatization of the digital public sphere (Milosevic, 2017: 47).

In the context of ‘deep mediatization’ (Hepp, 2020) permeating our social spaces, more and more aspects of our lives – including how we frame and manage risks – are seamlessly adapted to digital media logic and infrastructures. But even if users maintain their agentic capacities (Lupton, 2020; Lupton and Sutherton, 2021), their ability to act is both enabled and constrained by (technological and regulatory) social media affordances (e.g., Gibson, 1977; Hutchby, 2001; Bloomfield et al., 2010). By shaping conditions of possibility, and hence overcoming the limitations of technological determinism (Fussey and Roth, 2020), social media affordances leave space for human (individual or collective) choice or intervention. In the process, they hold us responsible for the technologies we make and use (Wyatt, 2008). Individual and collective users’ responsibility for harmful sharenting practices cannot, and should not, be minimised. Nonetheless, as evidenced in our findings, these practices can be enabled and even facilitated by a defective self-regulatory framework: by using an adapted crime proofing of legislation approach, we highlighted severe regulatory gaps, and how some attempts to regulate user behaviour might instead give rise to increased risk. Users such as parents/guardians sharing minors’ information, and their audiences and secondary distributors, have been defined as ‘slack and irresponsible’ (Lim et al., 2020: 96) for creating criminogenic opportunities online. We do acknowledge that our analysis of policy documents and guidelines may not fully capture the myriad of ways in which the lack of adequate policy we observed provides opportunities for illegal or otherwise harmful sharenting practices or how users may experience the inadequate policies instituted by platforms. Our findings demonstrate that we should not ignore the platforms’ responsibilities – even if the deniability of their legal responsibility has been at the core of the rise of contemporary digital giants (Gillespie, 2010; Andersson Schwartz, 2017; Klonick, 2017; Gorwa, 2019).

Of course, regulating the power of platforms is not an easy task (Lynskey, 2017; Bucher, 2018), and we firmly agree with the importance of self-regulation and its implementation through moderation systems. Yet, social media design should more carefully consider the impact of content moderation in practice on potentially harmful behaviour, as the mere presence of platform regulations does not guarantee efficient and effective practices (Chancellor et al., 2016).

From a criminological standpoint, having identified specific criminogenic features, Situational Crime Prevention techniques and mechanisms (see Clarke, 1992, 2009; Freilich and Newman, 2014) can provide useful guidance on how to mitigate the problems identified. The techniques offer a framework for mapping and devising potential interventions. Such interventions could be useful for addressing harmful sharenting practices. The interventions could also help address other comparable digital activities that are potentially harmful in the sense that they can harm individuals in ways that are currently escaping existing self-regulatory approaches. Discussing practical solutions would exceed the scope of this study. But our contribution has shown how criminological imagination can effectively contribute to multidisciplinary debates not only on sharenting practices but, more broadly, on digital ecosystems and their regulation, paving the way for a reduction of criminogenic and harming opportunities online.