Sage Journals: Discover world-class research

Abstract

This article presents the results of methodological experimentation that utilises machine learning to investigate automated copyright enforcement on YouTube. Using a dataset of 76.7 million YouTube videos, we explore how digital and computational methods can be leveraged to better understand content moderation and copyright enforcement at a large scale.We used the BERT language model to train a machine learning classifier to identify videos in categories that reflect ongoing controversies in copyright takedowns. We use this to explore, in a granular way, how copyright is enforced on YouTube, using both statistical methods and qualitative analysis of our categorised dataset. We provide a large-scale systematic analysis of removals rates from Content ID’s automated detection system and the largely automated, text search based, Digital Millennium Copyright Act notice and takedown system. These are complex systems that are often difficult to analyse, and YouTube only makes available data at high levels of abstraction. Our analysis provides a comparison of different types of automation in content moderation, and we show how these different systems play out across different categories of content. We hope that this work provides a methodological base for continued experimentation with the use of digital and computational methods to enable large-scale analysis of the operation of automated systems.

Keywords

Machine learning copyright enforcement YouTube content moderation automated decision-making Content ID

This article is a part of special theme on The Turn to AI. To see a full list of all articles in this special theme, please click here: https://journals.sagepub.com/page/bds/collections/theturntoai

How can we understand how massive content moderation systems work? The major social media platforms use a combination of human and automated processes to efficiently evaluate content that their users post against the rules of the platform and applicable laws. These sociotechnical systems are notoriously difficult to understand – we can see their results in individual cases, but their inner workings and systemic impact are often obscured (Gillespie, 2018). Most major platforms provide regular transparency reports, but these mainly provide high-level aggregations that are insufficient to really probe the contours and social effects of moderation systems (Suzor et al., 2019). This is a challenge that is widely acknowledged and quickly becoming more pressing; the lack of good information about how our shared digital environments are governed – what information is available, removed, and made more or less visible – has led to serious concerns about the potential for bias and the flow of harmful content (Suzor, 2019). This is likely to become more important and more difficult as nations continue to ask platforms to do more to regulate social media content – and to regulate more quickly by applying machine learning to filter material automatically or prioritise content for review.

In this article, we investigate copyright enforcement on YouTube as an important case study of a sophisticated and complex set of processes that are heavily automated and remain highly controversial. YouTube is a major target for the Digital Millennium Copyright Act (DMCA) notice and takedown copyright enforcement system, and it has also built Content ID, one of the most extensive automated systems for detecting copyright infringement. There are major concerns that YouTube’s enforcement system frequently incorrectly removes videos, at substantial cost to freedom of expression (Tushnet, 2014: 1461). Nevertheless, Content ID now serves as a model for the further deployment of ‘upload filters’ (Reda, 2019), and under the European Union’s recently approved copyright directive, more platforms will have a strong incentive to deploy automated systems that can monitor potential copyright infringement by their users (European Union Parliament, 2018). There is also real potential for these tools to be applied to censor or moderate other types of content, including hate speech and abuse (European Commission, 2018).

In this study, we use digital methods to try to make the content moderation system on YouTube – a system that relies on both automated and discretionary decision-making and that is applied to varying types of video content – more legible for research. Starting with a random sample of the text metadata of 76.7 million YouTube videos that includes information about whether and why each video was removed or blocked, we developed a machine learning classifier to categorise these videos into four categories. The categories represent ongoing controversies over online copyright enforcement: full movies, gameplay, sports content, and tutorials on copy control circumvention (hacks, cracks, and exploits). The core methodological problem that we confronted was how to reliably identify from a very large dataset of videos, relatively small subsets of removed videos that fell within our content categories. By solving this problem, we were able to examine how different types of automated and discretionary enforcement operate differentially within different categories of video content.

Essentially, in this study, we explore how digital and computational methods can be usefully combined with statistical and rich qualitative methods to study the digital traces of large-scale content moderation systems. We hope that this work will help to inform continual experimentation in the development of a new set of methodological approaches for interrogating pressing public policy research questions in the context of large-scale automated decision-making in digital media environments.

Background

YouTube’s baseline legal obligations for enforcing copyright are set out by the notice-and-takedown system established under the United States DMCA legislation and propagated around the world.¹ Notice-and-takedown has become an extremely important industrial mechanism for enforcing copyright; copyright owners employ rights management companies who use automated search tools to send hundreds of complaints of notices every year (Urban et al., 2017). Google, like other major targets of notice-and-takedown, has had to develop streamlined automated processes to deal with the massive volume of complaints it receives, but there is little public detail about how these processes work.

In practice, YouTube goes beyond its obligations under the DMCA when enforcing copyright, and has developed a series of additional tools and privately negotiated systems and policies (Bridy, forthcoming). The most visible of these tools is Content ID, YouTube’s automated rights management system that allows rightsholders to block, monetise, mute or track videos that contain their works. Rightsholders provide YouTube with a reference file of their work and the Content ID algorithm scans videos that are uploaded to YouTube to see if a match can be found in the database of reference files. Google reports that on YouTube, 98% of copyright matters are decided by Content ID (Google, 2016).

Automated, privatised copyright enforcement is an example of a controversial institutional shift from public to private modes of regulation. Private regulatory modes are controversial because they tend to lack important democratic features and due process safeguards (Black, 2001: 143; Zimmerman, 2014: 273). Private, automated regulatory systems in particular can lead to institutional convergence (Perel and Elkin-Koren, 2016: 481); that is, a tendency for lawmaking, enforcement and adjudication to occur in centralised private modes rather than through separate legislative and judicial institutions. The increased use of automation in private regulation raises additional concerns about transparency, accountability, and protection for fundamental rights (Citron, 2007; Elkin-Koren, 2014).

There are multiple sources of opacity – institutional, legal and technological – that make it difficult to evaluate automated private regulatory systems (Diakopoulos, 2015; Zarsky, 2016). Trade secret laws often prevent public access to these systems (Perel and Elkin-Koren, 2017). Their workings typically encode rules and priorities privately negotiated between stakeholders; Content ID, for example, was developed through partnerships between YouTube and large entertainment companies (Tushnet, 2014; Yafit, 2013: 248). Automated decision-making processes frequently occur in a ‘black box’ that is difficult or impossible to interrogate (Pasquale, 2015). The data and rules that human and algorithmic moderators are trained on is secret, and the outputs of these systems are often obscured by platforms who seek to avoid public scrutiny (Gillespie, 2018). Increasing accountability requires real transparency, but improving transparency will require substantial improvements in methodologies and collaborations (Suzor et al., 2019).

A second major controversy that marks automated decision-making is the capacity for error and bias. When undertaken on a very large scale, even very low rates of error in automated decision-making can create very large problems (Urban et al., 2017: 35). Automated decision-making systems are typically blunt instruments, operating with narrow objectives that can introduce systematic bias, and incapable of accounting for the full context that might impact human decision-making (Binns et al., 2017). For example, if Content ID’s primary objective is to enforce and commodify the use of content on YouTube, without the ability to distinguish between non-infringing and infringing uses, it risks serving the interests of large rightsholders at the expense of end users or smaller creators (Bridy, 2016; Kohl, 2013: 220).

In copyright governance, as more advanced automated enforcement systems have been deployed to protect the proprietary interests of media and entertainment companies, controversies have arisen over the potential for these systems to prioritise efficiency over accuracy (Gray, 2020). Content ID on YouTube, in particular, has been criticised for removing from YouTube content not subject to copyright protection such as reviews, parodies and educational videos that fall under a fair dealing or fair use exception (McSherry, 2014). Similarly, Content ID has been criticised for capturing content that rightsholders would have, if left to their discretion, not removed, such as gameplay videos (Boroughf, 2015). So long as rightsholders continue to pressure platforms to implement more streamlined and efficient system for removing content from the internet, the potential for over-enforcement of copyright online will remain a contentious issue.

The study of copyright takedowns has been a vibrant area for scholarship. Existing work has utilised a broad range of methodologies, including interviewing stakeholders (Urban et al., 2017); experimental uploading and interacting with platforms (Nas, 2004; Perel and Elkin-Koren, 2017); the analysis of information made available at the discretion of platforms or during legal disputes (Bar-Ziv and Elkin-Koren, 2018; Seng, 2014, 2015; Tushnet, 2014; Urban et al., 2017; Urban and Quilter, 2006); tracking discrete samples of publicly available material (Erickson and Kretschmer, 2018; Jacques et al., 2018); and laboratory experimentation (Fiala and Husovec, 2018). To this body of work, our study contributes a machine learning methodology that helps to identify particular categories of content for further qualitative and statistical analysis at a very large scale. Our hope is that this new approach will improve understanding of how content moderation systems that combine both automated and discretionary decision-making operate in practice, across different types of video content.

Interrogating content moderation at scale

This article has two main aims. First, we seek to compare trends in copyright takedowns, Content ID blocks, and terms of service (ToS) removals across several key controversial issues. We set out to categorise videos into topics that aligned with concerns in the literature about potential misuse of copyright enforcement mechanisms. By focusing on particular controversies, we hoped to be able to investigate the types of actors responsible for moderation within each category, as well as to compare rates of different types of moderation across categories and against the baseline average. Second, in order to undertake this analysis, we had to develop a methodology that could help us isolate videos in controversial categories from the much larger bulk of YouTube content.

One of the major challenges of understanding content moderation practices is the vast scale of content that is posted and moderated on major social media platforms. Overall rates of removal under the DMCA and Content ID on YouTube are relatively low – only approximately 1% of all videos uploaded are removed due to an apparent copyright violation (Suzor, n.d.). In order to examine trends within copyright removals, we required a very large initial sample and a suitable mechanism to isolate sufficiently large samples of videos that are likely to be relevant to the inquiry at hand. Given the extremely large volume and proportion of irrelevant videos, this is a task that is prohibitively difficult to undertake manually. A large-scale dataset is required to analyse and compare trends in types of content removal across different categories of content and it requires some form of computational analysis to assemble.

Methodology

Data collection

For this study, we used existing infrastructure (Suzor, n.d.) to obtain a random sample of metadata about YouTube videos and their availability. The infrastructure uses YouTube’s search API (‘list’ endpoint)² to generate a random sample of YouTube videos as they are published. Each of these videos was tested approximately two weeks after it was first collected, and we logged whether it was still available, and if not, what reason YouTube provided for its removal. The two week time period was selected to provide a sufficiently long enough time for a copyright takedown or terms of service removal – given that these systems tend to be most controversial when content is new and therefore most visible and most commercially valuable. When YouTube removes a video, it provides a detailed error message that explains why the video is not available and, in the case of copyright takedowns, who was responsible for requesting that the video be blocked. When a video is blocked by Content ID, YouTube will still host a link to the video and will provide an error message explaining that the video was blocked due to a copyright claim. It is possible that our infrastructure under-counts Content ID matches where a video is blocked before it is published, but it appears for many videos that Content ID blocking occurs some minutes or hours after initial publication. YouTube does not provide detail about what proportion of Content ID blocks take effect before publication.

We categorised the distinct explanations for removal that YouTube provides into six groups: the video was available, the video was not available because of a Content ID block (either globally or in the jurisdiction in which our server is based), the video was not available because of a DMCA notice or equivalent (either globally or locally), the video was removed by YouTube for violating its Terms of Service or Community Guidelines, the user’s account was terminated by YouTube, or the user removed the video themselves. We discarded videos that were unavailable for technical or other (unknown) reasons. Our final dataset consists of title, description, availability status, and error messages for 76.7 million YouTube videos collected between October 2016 and February 2019 (See Figure 1).

Topic selection

We first sought to divide the sample into categories of similar videos to better understand the types of videos that were removed for different reasons in this large dataset. We used topic modelling (Steyvers and Griffiths, 2007) on the title and description text of approximately 4000 videos (with English metadata) that had been removed, to explore the main types of content that are moderated. We used Latent Dirichlet Allocation (Blei et al., 2003; Pedregosa et al., 2011) to generate a series of topic models at varying degrees of granularity, ranging from 5 to 80 clusters. By varying the number of clusters and examining the most relevant words for each cluster, as well as a sample of documents that were most likely to fall into each cluster, we were able to identify some relatively strong groupings of topics for further analysis.

We used the topic modelling as a starting point to inductively identify some coherent clusters of similar videos that appeared to be frequently blocked. We cross-referenced these topics against controversial issues that have been identified in the literature, in order to develop a set of categories for further analysis. Ultimately, we developed a classification scheme for five categories, informed by existing controversies over copyright enforcement and notice and takedown:

‘Full movies’: videos that appear to conform to a traditional classification of movie piracy: full length copies of feature films (Pariser, 2016; Patry, 2009). In this category, we excluded movie trailers, movie soundtracks and videos containing only movie scenes.

‘Gameplay’: live streaming or recorded video game play. This category included reviews of games and game walkthroughs, tips, and guides that are vulnerable to takedown because they contain a lot of existing copyright content, including artwork, music, and dialogue (Boroughf, 2015; Burgess and Green, 2018). We excluded advertisements and other promotional content.

‘Sports’: videos of sporting event broadcasts either as live streams, recorded streams, or snippets. This category focused on recordings of televised sports content that is protected as copyright subject matter under most copyright regimes (Garrett, 2016; Jones, 2017). Premium sports content is highly lucrative, and is accordingly a hotly contested site for copyright enforcement. We excluded non-professional games recorded by users. We further subdivided this category to train the classifier to distinguish live broadcast streams from non-live sports content.

‘Hacks’: videos that provide tutorials on circumvention of Digital Rights Management (DRM) software and hardware, including game exploits, key and serial generators, and software cracks that can be used to infringe intellectual property rights (Gillespie, 2007). Sometimes called ‘paracopyright’, there are long-standing concerns that anti-circumvention law can be misused to reduce competition (Burk, 2003). Notably, the DMCA does not provide a procedure for notice and takedown of instructional material that teaches people to circumvent DRM – so it would be a sign of potential misuse if this category of videos had relatively high copyright takedown rates.

Classification

Our initial topic modelling was useful for exploration, but once we had identified our relevant categories, we elected to use a supervised classification technique to develop more robust samples for analysis. Recent advances in natural language processing have greatly improved the state of the art in text classification, improving the utility of deep neural networks in classification tasks. New techniques make use of ‘transfer learning’ – general models that are trained on very large existing datasets, and then fine-tuned for specific applications on much smaller labelled datasets (Mou et al., 2016). These general models are trained on large corpora to understand how different features of language relate to each other – learning, for example, how words are used in different senses by learning the contexts in which they appear in different sentences (Adhikari et al., 2019). We made use of the newly released Bidirectional Encoder Representations from Transformers (BERT), which has improved state of the art performance on many common natural language processing tasks (Devlin et al., 2018). BERT provides sentence-level representations trained on massive corpora of Wikipedia articles and digitised books. These pre-trained models allowed us to use supervised approaches to train a classifier to identify complex patterns within our specific dataset with relatively small training sets.

We used BERT to train a machine learning classifier to identify videos in each of our categories across our larger dataset. We use a transformer attention-based deep learning model – described as a ‘simple network architecture’ (Vaswani et al., 2017) – to perform classification on YouTube video titles and descriptions. We used the largest case-insensitive English-language pre-trained BERT model, with 24 encoder layers, 1024 hidden units, 16 heads, and 340 million parameters, and fine-tuned the model on Google’s Tensor Processing Unit cloud-based computing architecture.

We started by labelling a training set of example videos (title and description text) identified as likely to fall within our chosen categories by our topic model. We then went through several rounds of semi-supervised learning, where we ran our trained model to classify small batches of unlabelled records and manually corrected its results. We iteratively evaluated the model’s performance quantitatively by measuring its accuracy (f1 score – an average combined measure of false positive and false negatives in each category, weighted by the total number of results in each category) and qualitatively by closely examining the sample of predictions. We ultimately manually labelled approximately 10,000 videos to develop an adequate training set, although the majority of these (5801) were examples of videos that were not relevant to our categories.

We were able to achieve satisfactory results using only between 420 and 1696 examples in each category. To evaluate our model, we manually labelled each video in a sample of approximately 100 videos from each category (with the two sports categories combined) across the entire predicted dataset. The final results show an f1 weighted accuracy score of 90.6%. These results are quite good for our purposes – there are relatively few false positives within each of category, and few instances of confusion between the four categories under scrutiny. We will discuss particular trends and limitations below in our qualitative analysis of each category but, notably, we found that a relatively low amount of manual labelling was required to produce an accurate machine learning classifier using the ‘transfer learning’ technique.

In the final stage, we deployed the trained model to categorise our entire random sample of 76.7 million YouTube videos. Our model found 12,943,693 unique videos fell into one of our four categories. We used multinomial logistic regression to examine the relationship between takedowns, our predicted categories, and additional variables in the metadata, including links to external sites. This statistical method enabled us to estimate the relative influence of different factors on the likelihood of videos in each category being removed through different mechanisms, and provides a basis from which to speculate about the factors that might influence decisions to block content across different categories. Most importantly, however, the classification technique allowed us to identify video metadata for manual qualitative investigation.

We undertook qualitative analysis of the metadata of a sample of 12,000 videos across our categories, including both removed and available videos, to explore the types of videos classified, and supplemented this with targeted samples on discrete questions. This analysis was first conducted by reading through the titles and descriptions of each of the videos in each category, paying attention to the choices made by uploaders to describe their videos and make them findable by others, as well as the types of content that the classifier assigned to each category. We then watched a manually selected subset of videos in each category until we were confident that we could understand the types of content that the classifier was identifying (or mis-identifying) in each category. Our analysis below focuses on how our findings relate to some of the key controversies around these categories of user-generated content and online copyright enforcement broadly.

Results and discussion

One of the most important findings of this study is that we can see, at a large scale, the rates at which Content ID is used to remove content from YouTube. Previous large-scale studies have provided important insights into rates of DMCA removals (e.g. Seng, 2014; Urban et al., 2017), but information about Content ID removals has remained imprecise, provided at a high level of abstraction by YouTube. In this article, we provide the first systematic analysis of Content ID removals rates, including comparisons with other removal types and across different categories of content. We note that where rightsholders have opted for monetisation through Content ID, they will be included in the ‘available’ category. YouTube does not make public the information necessary to determine which rightsholders have opted to monetise which videos.

Across our entire dataset, videos were most frequently removed from YouTube by users themselves, followed by removals due to an account termination and then Content ID blocks (See Table I). DMCA takedowns were the least common removal type – Content ID removals occurred at seven times the rate of DMCA removals, and videos were on average five times more likely to be removed for terms of service violations than due to a DMCA notice. Our findings confirm the general trend identified in a recent study of Content ID and DMCA removals: in a sample of 1839 parody videos it was found that Content ID was used to block videos five times more frequently than DMCA notices (Jacques et al., 2018).

We built a simple Logistic Regression model to estimate the links between specific removal outcomes and the categories assigned by our classifier (See Table 2). We used 150,000 randomly selected records from each category to build the model, which estimates log odds of each type of removal for each category. We separated out music claimants (music rightsholders such as record labels or publishers) from the rest of the Content ID claims because music claimants’ behaviours tend to differ from other types of rightsholders quite significantly – they frequently make use of Content ID’s monetisation option rather than removing videos. The model also includes an interaction term for whether the video description has a link to an external website. We developed several models and selected this one based on its interpretability and its performance under a pseudo-R squared metric. The full regression results are in Appendix 1.

Overall, our analysis shows that in YouTube’s heavily automated content moderation system there is substantial discretionary decision-making, as well as a potential lack of contextual sensitivity. We found very high rates of removals for videos associated with film piracy and all types of sports content. We found that game publishers are largely not enforcing their rights against gameplay streams and that when gameplay videos are removed it is usually due to a claim by a music rightsholder. We also found high rates of removals in the hacks category but mostly for Terms of Service violations, which indicates that YouTube rather than rightsholders are more commonly taking action to remove content and terminate accounts that provide DRM anti-circumvention information.

Full movies

Film piracy has been one of the key issues of the ‘copyright wars’ (Patry, 2009). From early in its history, YouTube has been a central battleground in these wars, amidst major concern by both screen and music industries that YouTube’s core business model was built on copyright infringement (Burgess and Green, 2018). Content ID has cleaned up a lot of the direct copyright infringement on YouTube over the years, but there are ongoing concerns that YouTube continues to host copies of infringing content and that it provides a vector for infringement by directing viewers to streaming sites and filelockers where they can access copies of feature films. In particular, rightsholders continue to complain that movies that are removed often reappear quickly after removal (Van der Sar, 2018) and that filtering technologies like Content ID are vulnerable to gaming or circumvention by sophisticated users (Pariser, 2016: 19).

In our analysis, we sought to identify how well both Content ID and the DMCA process were working to keep what might appear to be clearly infringing content of YouTube. Overall, only 36% of videos in this category were available two weeks after they were first posted. When we break this down by removal type, we see that videos in this category are nearly 30 times more likely to be removed through a DMCA notice than the baseline for unclassified videos, and 11.5 times more likely to be blocked through Content ID. This category also had the highest rate of removals for terms of service violations (24.7 times more likely than baseline) and account terminations (14.5 times more likely). The high rate of account terminations suggests that the accounts used to post full movies or links to full movies are likely to be ‘repeat infringers’, in the copyright terminology, or are either frequently or flagrantly in breach of YouTube’s rules. The top claimants in this category were a mix of film studios, who used Content ID directly, and third-party rights management companies, who are generally responsible for sending DMCA notices on behalf of producers.

From our qualitative analysis of the video metadata, it was apparent that the primary types of content removed in this category were videos that purported to be full copies of feature films hosted on YouTube or videos that promoted links to third-party websites apparently hosting streams of full copies of feature films. Many of the links in this category were to URL shorteners, filelockers, and a long tail of domains that often appeared to be offering illicit downloads and streams, advertising farms, or malware sites, amongst a lot more that were impossible to efficiently classify.

To add these links to our regression, we created a binary category for links. We excluded the most common domains in our dataset, those that were not generally associated with copyright infringement (we did so by excluding domains that received >1000 links in our entire classified dataset). We found that videos with links to sites that are not amongst the most popular sites were much less likely to be removed by Content ID – between 66% and 89% less likely than videos without a link in this category. There was no statistically significant difference for DMCA notices. From the video descriptions we examined, this appears to be evidence of uploaders seeking to use YouTube to gain the attention of users searching for illicit content without uploading any infringing material in the video itself – and therefore avoiding detection by Content ID. The high rates of removals of all videos in this category under the DMCA, Terms of Service, and account terminations by YouTube, however, regardless of whether they have a link or not, suggests that this may not be a major problem for rightsholders.

Gameplay

A persistent concern about both notice and takedown and Content ID relates to how it might regulate transformative user-generated content, and videos made from computer games have long been a prime site of controversy (Burgess and Green, 2018). As game streaming took off, YouTube has become an important (if secondary) platform for live and recorded gameplay footage (Taylor, 2018). For many years, commentators have raised concerns about potential over-enforcement of copyright, because copyright law gives many different copyright owners the right to object to gameplay videos and it can be hard to evaluate a fair use claim (or equivalent; Taylor, 2015).

The most common type of content identified in this category was streams of game play, often with commentary by the player, either as a live broadcast or recording. There is some suggestion that early fears about the lawfulness of gameplay videos may have settled down as copyright owners have come to accept and even embrace recorded footage and streaming videos (Matsui, 2016). In our data, we clearly see that game publishers are not enforcing their rights against game streamers at any real scale: gameplay videos are 83% less likely to be removed by a Content ID claim than an unclassified video, and 93% less likely to be removed by a DMCA notice. In general, it appears that game streaming is an advanced case of ‘tolerated use’, where technically infringing content has become normalised and acceptable (Tehranian, 2011; Wu, 2007).

The problems with tolerated use arise when the discretion to tolerate or remove content is exercised in a way that could stifle legitimate expression or reflects systemic biases. When videos in this category were removed on copyright grounds, it was usually at the request of large music companies – not game publishers. Worryingly, music claimants were 41% more likely to block gameplay videos than the uncategorised average. This may be a variant of the ‘tragedy of the anticommons’ (Heller, 1998): even if a large proportion of music is available to reuse on YouTube through Content ID’s licensing scheme, gameplay streams may last for hours and include many different background songs, the owners of any of which can elect to block the entire stream. Since it can be difficult for streamers to know in advance which songs are made available to license, and particularly since there is little threat of gameplay streams substituting in markets for recorded music, this is one area where Content ID appears to remain a justified cause of frustration for ordinary users.

Of the small proportion of videos that were removed for terms of service violations, or where the uploader’s account was terminated by YouTube, the majority appeared to be misclassifications or overlap with the ‘hacks’ category: footage of cheating and exploits in video games. Our coding schema meant that guides on game exploits should be classified as ‘hacks’, not ‘game play’, but this is not always easy, and there is a degree of unavoidable overlap where users upload footage of cheating behaviour in multiplayer games. Our manual validation, in Figure 2, confirms that this is a particular area of confusion for our classifier: approximately 4% of videos in each of these two categories were mistakenly predicted in the other.

Figure 2.

Manual validation of approximately 100 randomly selected videos in each class (combining sports classes).

Figure 1.

Overall removal rates.

Sports

Live sports continues to be one of the major areas of controversy over copyright infringement on the internet. Because live sporting events are immensely popular around the world, and access is often limited to premium cable channels, pay-per-view, and streaming offerings (Garrett, 2016; Hull, 2010), we might expect a great deal of unmet demand from consumers who are dissatisfied with or cannot afford premium offerings, which in turn may lead to increased copyright infringement (Birmingham and David, 2011; Dootson and Suzor, 2015). Over the past decade, the infringement of live telecasts of sports broadcasts has become an increasingly pressing concern for sports organisations and, seeking to protect an important revenue stream, they have argued for stronger laws (Garrett, 2016: 2) and new practices to prevent internet users from sharing streams of live broadcasts of their sporting events (Mellis, 2007). At the same time, however, the strict copyright enforcement practices pursued by sports organisations have caused the removal or monetisation of works such as reviews, gifs, memes and other potentially fair uses of sports broadcast content (Jones, 2017; Wang, 2015).

In this category, the types of videos identified by our classifier primarily included live streams of sporting events, as well as videos containing parts of full recordings of sporting matches, for a wide variety of sports, from professional leagues of football, basketball, tennis, hockey, motor sports, wrestling and more. Videos in these categories were at high risk of removal by almost all avenues. Content ID removal rates were high for both live sports and highlights (4 times and 3.9 times more likely than baseline respectively), as well as for DMCA takedowns (18 times and 10.6 times baseline respectively). Users who posted videos that appeared to be live streams were at risk of having their account terminated 12.5 times more often than the baseline, and these videos were 9.6 times more likely to be removed for violating YouTube’s terms of service. Clearly both copyright owners and YouTube are heavily active in policing sports content on YouTube.

From our qualitative analysis of video metadata in the non-live sports subcategory, there was little discernible difference between types of takedowns. The high rate of removals in this category is somewhat concerning, since many clips of sports content may be lawful under fair use or other copyright exceptions, but this cannot be determined from the metadata. This area appears to be an important candidate for follow-up studies that are able to undertake fair use analyses (see e.g. Erickson and Kretschmer, 2018; Jacques et al., 2018) to determine whether sufficient care is being taken by YouTube, its partners, and copyright owners and their agents when evaluating whether to block sports videos.

Hacks

The final category we investigated primarily consists of video tutorials about circumventing DRM. This includes guides about jailbreaking smartphones, generating serial numbers for software, and downloading cracked software with the copy-protection removed. Because the tools are closely related, the classifier also identified videos about game exploits (modified versions of games that allow players to cheat, bypassing both copy protection and anti-cheat software), and serial generators for prepaid gift cards on app stores and ecommerce platforms. The classifier sometimes struggled to distinguish game cheats that required circumvention from ordinary cheats and exploits, although it appeared likely from the descriptions that many of the supposed exploits we identified were bait designed to lure traffic towards malware, advertising farms, or paid services.

This category had very high removal rates overall – only 54% of videos were still available two weeks after they were posted. Tensions over DRM generally and circumvention tools in particular have been a constant feature of copyright debates over the last three decades, and rightsholders have worked hard to ensure that other companies secure digital distribution channels (Gillespie, 2007). For this category, we sought to know whether copyright owners have been using the DMCA notice and takedown process to remove information about the circumvention of DRM – which would be clearly beyond the scope of the takedown regime. There was no evidence to support this – copyright owners were not sending notices at a statistically significant higher rate in this category compared to unclassified videos, and the odds of removal for Content ID were 72–88% lower.

Interestingly, however, videos in this category were nearly 10 times more likely to be removed for a breach of YouTube’s Terms of Service, and 15 times more likely to be removed because YouTube had terminated the uploader’s account. YouTube’s Community Guidelines prohibit instructional videos that ‘[show] users how to bypass secure computer systems’, and it clarifies that this includes ‘Showing users how to circumvent payment processes to download software or applications for free’ (YouTube, n.d.). This appears to be a rule that YouTube is enforcing quite extensively – perhaps not surprising given Google’s interests in securing the Android ecosystem and its need to maintain working relationships with many software developers and copyright owners.

Limitations

The most obvious limitation of this study is that we are attempting to classify the content of videos based on the text and description fields. First, we only trained the classifier on English-language metadata, and the language model is optimised for English – so most of the results are also English-language. Second, these fields are entirely generated by the user, and do not always accurately reflect the video content. Unfortunately, our infrastructure does not collect full copies of videos, and we do not have the computational power to accurately classify or recognise themes from video content at a large scale. Nevertheless, there are important benefits in classifying video content on textual metadata, since these text fields are used to aid users who search on YouTube to discover relevant content. The more problematic limitation is that we are unable to include other metadata fields in our analysis – the YouTube Search API endpoint only returns a selected ‘snippet’ of information. It would have been useful to have additional metadata to build our logistic regression models (video duration, for example). It would have also improved the accuracy of our classifier to have full length data for the video description field. This is an unfortunate limitation of the API and the quota imposed by YouTube that was unavoidable in this study.

An important limitation of our analysis strategy is that we do not account for changes in enforcement rates over time. Given the small proportions of videos in some of the categories we examine compared to the total number of videos in our overall random sample, we have elected not to further divide our categorised sets into daily or monthly aggregations. Because content moderation systems of platforms are continuously tweaked, and their features, cultures, and the behaviour of their users change over time (Burgess, 2015), and individual rightsholders may change their takedown strategies, we suggest that future longitudinal work could be extremely useful.

Another key limitation is that we have chosen to optimise our training data for precision over recall. Precision is a measure of the proportion of correct classifications within each category (number of true positives/total of true and false positives), and recall is a measure of the proportion of correctly identified classifications across the entire dataset (number of true positives/total of true positives and false negatives). In classification tasks, there is a trade-off between these measures: increasing the overall number of correctly classified results in any category generally means including more incorrect results as well (Buckland and Gey, 1994). The specific goals of the analysis should guide training and evaluation of a classification model (Sokolova and Lapalme, 2009). We focused on improving the quality of predictions within each category (minimising false positives), at the expense of not including some potentially relevant videos (false negatives). We were therefore conservative in allocating examples to categories in our training sets, and in the semi-supervised stages, focused on reducing the numbers of misclassified examples, especially because it was difficult to find additional positive examples within the very large set of unclassified and (for our purposes) largely irrelevant random sample. So, for example, from our manual validation, 97 out of 100 videos classified as ‘full movies’ appeared from the metadata to purport to be full versions of cinematic films or to link to sites where people could watch full versions of those films. The precision of this category is very high, and we can be comfortable drawing conclusions about how these types of videos are regulated. This high rate, however, suggests that there may be a problem of over-fitting: that our classification may be so tightly constrained to identify the particular patterns of our training videos that we miss other ways that people might share full versions of films on YouTube. This is a risk we guarded against by qualitative exploration of search results on film titles on YouTube – we did not find any major omissions that do not fit the patterns we identified in developing our training sets – but we are not able to exclude the possibility that our model is too narrow in identifying too few relevant videos. Ultimately, this choice means that we cannot make generalisations about the overall prevalence of videos matching our categories on YouTube, but we can be more confident that the videos in each category are correctly classified.

Conclusion

This experimental methodology has left us optimistic about the potential to use machine learning classifiers to better understand systems of algorithmic governance. The methodology improves our understanding of how an evolving system of both automated and discretionary content moderation operates in practice. Using a simple neural network infrastructure, a pre-trained BERT language model, and substantial cloud processing power, we were able to achieve satisfactory performance on a multiclass classifier over short texts (≤300 characters) with a relatively small number of training examples (between 420 and 1696 per class). The implications for computational social studies are exciting, and we have made our code available to help others extend BERT classification in other contexts.³ This methodological experiment proved useful for helping to identify and interrogate patterns in a large dataset of content moderation outcomes. Content moderation is a notoriously opaque area, where the training materials and performance of human moderation teams are kept confidential, as are the details of the classification systems that prioritise content for review and, in some circumstances, remove content directly. As nations around the world continue to pressure platforms to take a more active role in moderating harmful content, it will become increasingly important to develop mechanisms to hold these moderation systems to account. YouTube’s complex and highly automated copyright moderation systems are an excellent case study to develop and hone new methods.

A key benefit of our methodology is that it allows for the identification of trends in content moderation that would not be evident in small sample sizes or through experimental uploading. Our study has shown the potential to undertake large scale quantitative analysis on these systems at a level of detail that has so far not been possible. Most importantly, we hope that this methodological approach proves useful in the future for researchers who may undertake longitudinal analyses or undertake further detailed qualitative study of particular controversies.

As for YouTube’s copyright enforcement system itself, we have only been able to scratch the surface with this analysis, and we must leave some further detailed investigation for future work. It is clear, however, that both the Content ID and DMCA takedown system are used with a greater degree of discretion than was previously apparent: at an aggregate level, rightsholders make different decisions in relation to different types of videos. It also seems that in aggregate, the Content ID and DMCA systems are working relatively well to remove apparently infringing content from YouTube. Our study does, however, raise some concerns about potential misidentification and over blocking, particularly in the sports highlights category, as well as the large amount of discretion that music rightsholders are able to exercise to choose to block all types of content – including material such as gameplay that is unlikely to compete in the market for recorded music.

Identifying the factors that affect the decisions of rightsholders to remove content is an important area for further study, since these decisions have ramifications for freedom of expression and access to information. We suggest that future studies might fruitfully develop finer-grained classification categories and seek to collect more extensive metadata in order to facilitate more extensive qualitative analyses of the types of content that are most likely to be blocked. The high discretion and potential lack of contextual sensitivity evident in these systems is something that policymakers too should clearly evaluate and address before encouraging platforms to rely to a much greater extent on automated content moderation tools, either for copyright or for issues like hate speech and abuse.

Footnotes

Acknowledgements

We thank Rosalie Gillett for outstanding research assistance.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Suzor is the recipient of an Australian Research Council DECRA Fellowship (project number DE160101542). This research is also supported by an ARC Discovery Projects grant (DP170100122). This research was supported with Cloud TPUs from Google’s TensorFlow Research Cloud (TFRC).

ORCID iD

Nicolas P Suzor

Notes

References

Adhikari

Ram

Tang

, et al. (2019) DocBERT: BERT for Document Classification. Available at: https://arxiv.org/abs/1904.08398v1 (accessed 20 April 2019).

Bar-Ziv

and Elkin-Koren

(2018) Behind the scenes of online copyright enforcement: Empirical evidence on notice & takedown. Connecticut Law Review 50: 339–385.

Binns

Veale

Van Kleek

, et al. (2017) Like trainer, like bot? Inheritance of bias in algorithmic content moderation. In: International conference on social informatics, 2017, pp. 405–415. Berlin: Springer.

Birmingham

and David

(2011) Live-streaming: Will football fans continue to be more law abiding than music fans? Sport in Society 14(1): 69–80. DOI: 10.1080/17430437.2011.530011

Black

(2001) Decentring regulation: Understanding the role of regulation and self-regulation in a “post-regulatory” world. Current Legal Problems 54(1): 103.

Blei

and Jordan

(2003) Latent dirichlet allocation. Journal of Machine Learning Research 3(Jan): 993–1022.

Boroughf

(2015) The next great YouTube: Improving content ID to foster creativity, cooperation and fair compensation. Albany Law Journal of Science & Technology 25(1): 95.

Bridy

(2016) Copyright’s digital deputies: DMCA-plus enforcement by internet intermediaries. In: Rothchild J (ed.) Research Handbook on Electronic Commerce Law. Edward Elgar, pp. 185–208.

Bridy

(forthcoming) Addressing infringement: Developments in the US and the DNS. In: Frosio G (ed.) The Oxford Handbook of Online Intermediary Liability. London: Oxford University Press. Available at: https://ssrn.com/abstract=3264879 (accessed 15 April 2020)

10.

Buckland

and Gey

(1994) The relationship between recall and precision. Journal of the American Society for Information Science 45(1): 12–19. DOI: 10.1002/(SICI)1097-4571(199401)45:1<12::AID-ASI2>3.0.CO;2-L

11.

Burgess

(2015) From ‘broadcast yourself’ to ‘follow your interests’: Making over social media. International Journal of Cultural Studies 18(3): 281–285.

12.

Burgess

and Green

(2018) Youtube: Online Video and Participatory Culture. 2nd ed. Digital Media and Society. Cambridge: Polity Press.

13.

Burk

(2003) Anti-circumvention misuse. IEEE Technology and Society Magazine 22(2): 40–47.

14.

Citron

(2007) Technological due process. Washington University Law Review 85: 1249.

15.

Devlin

Chang

M-W

Lee

, et al. (2018) BERT: Pre-training of deep bidirectional transformers for language understanding. Available at: https://arxiv.org/abs/1810.04805v1 (accessed 2 March 2019).

16.

Diakopoulos

(2015) Algorithmic accountability: Journalistic investigation of computational power structures. Digital Journalism 3(3): 398–415.

17.

Dootson

and Suzor

(2015) Game of clones and the Australia tax: Divergent views about copyright business models and the willingness of Australian consumers to infringe. The. University of New South Wales Law Journal 38: 206–329.

18.

Elkin-Koren

(2014) After twenty years: Revisiting copyright liability of online intermediaries. In: Frankel S and Gervais DJ (eds) The Evolution and Equilibrium of Copyright in the Digital Age. Cambridge: Cambridge University Press, pp. 29–51.

19.

Erickson

and Kretschmer

(2018) “This video is unavailable”: Analyzing copyright takedown of user-generated content on YouTube. Journal of Intellectual Property, Information Technology and Electronic Commerce 9(1): 75–89. http://www.jipitec.eu/issues/jipitec-9-1-2018/4680

20.

European Commission (2018) Proposal for a Regulation of the European Parliament and of the Council on preventing the dissemination of terrorist content online. 2018/0331 (COD), 12 September. Available at: https://ec.europa.eu/commission/sites/beta-political/files/soteu2018-preventing-terrorist-content-online-regulation-640_en.pdf (accessed 15 April 2020)

21.

European Union Parliament (2018) Amendments adopted by the European Parliament on 12 September 2018 ( ) on the proposal for a directive of the European Parliament and of the Council on copyright in the Digital Single Market COM(2016)059-C8-0383/2016-2016/0280(COD).

22.

Fiala

and Husovec

(2018) Using experimental evidence to design optimal notice and takedown process. TILEC Discussion Paper DP2018-028.

23.

Garrett

(2016) Comments of The Professional Sports Organizations in the Matter of Section 512 Study before the Copyright Office Library of Congress. 2015–2017, Washington, DC.

24.

Gillespie

(2007) Wired Shut Copyright and the Shape of Digital Culture. Cambridge, MA: MIT Press.

25.

Gillespie

(2018) Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions that Shape Social Media. 1st ed. New Haven, CT: Yale University Press.

26.

Google (2016) How Google Fights Piracy. Available at: https://drive.google.com/file/d/0BwxyRPFduTN2TmpGajJ6TnRLaDA/view (accessed 15 April 2020).

27.

Gray

(2020) Google Rules: The History and Future of Copyright under the Influence of Google. New York, NY: Oxford University Press.

28.

Heller

(1998) The tragedy of the anticommons: Property in the transition from Marx to markets. Harvard Law Review 111(3): 621–688. DOI: 10.2307/1342203

29.

Hull

(2010) Sports leagues’ new social media policies: Enforcement under copyright law and state law. The Columbia Journal of Law & the Arts 34: 457.

30.

Jacques

Garstka

Hviid

, et al. (2018) An empirical study of the use of automated anti-piracy systems and their consequences for cultural diversity. Script-Ed 15(2): 277–312. DOI: 10.2966/scrip.150218.277

31.

Jones

(2017) By throwing fans and writers in Twitter jail, sports leagues are abusing the law. The Guardian, 13 April. Available at: https://www.theguardian.com/sport/2017/apr/13/twitter-sports-leagues-rules-journalist-account-suspensions (accessed 19 March 2019).

32.

Kohl

(2013) Google: The rise and rise of online intermediaries in the governance of the internet and beyond (part 2). International Journal of Law and Information Technology 21: 187.

33.

McSherry

(2014) Lawrence Lessig settles fair use lawsuit over phoenix music snippets. In: eff.org. Available at: https://www.eff.org/press/releases/lawrence-lessig-settles-fair-use-lawsuit-over-phoenix-music-snippets (accessed 15 April 2020)

34.

Matsui

(2016) Does it have to be a copyright infringement: Live game streaming and copyright. Texas Intellectual Property Law Journal 24: 215.

35.

Mellis

(2007) Internet piracy of live sports telecasts. Marquette Sports Law Review 18: 259.

36.

Mou

Meng

Yan

, et al. (2016) How transferable are neural networks in NLP applications? arXiv:1603.06111 [cs]. Available at: http://arxiv.org/abs/1603.06111 (accessed 10 April 2019).

37.

Nas

(2004) The Multatuli project: ISP notice & take down. In: 4th international system administration and network engineering conference, Amsterdam, Netherlands, September 2004.

38.

Pariser

(2016) Comments of the Motion Picture Association of America before the Library of Congress United States Copyright Office in the Matter of Requests for Comments on United States Copyright Office Section 512 Study. Docket No. 2015-7, 1 April.

39.

Pasquale

(2015) The Black Box Society. Cambridge, MA: Harvard University Press.

40.

Patry

41.

Pedregosa

Varoquaux

Gramfort

, et al. (2011) Scikit-learn: Machine learning in python. Journal of Machine Learning Research 12: 2825–2830.

42.

Perel

and Elkin-Koren

(2016) Accountability in algorithmic copyright enforcement. Stanford Technology Law Review 19: 473–532.

43.

Perel

and Elkin-Koren

(2017) Black box tinkering: Beyond disclosure in algorithmic enforcement. Florida Law Review 69(1): 181.

44.

Reda

(2019) Upload filters. Available at: https://juliareda.eu/eu-copyright-reform/censorship-machines/ (accessed 11 April 2019).

45.

Seng

(2014) The state of the discordant union: An empirical analysis of DMCA takedown notices. Virginia Journal of Law and Technology 18: 369.

46.

Seng

(2015) ‘Who Watches the Watchmen?’ An Empirical Analysis of Errors in DMCA Takedown Notices. ID 2563202, SSRN Scholarly Paper, 23 January. Rochester, NY: Social Science Research Network. Available at: http://papers.ssrn.com/abstract=2563202 (accessed 25 May 2016).

47.

Sokolova

and Lapalme

(2009) A systematic analysis of performance measures for classification tasks. Information Processing & Management 45(4): 427–437. DOI: 10.1016/j.ipm.2009.03.002

48.

Steyvers

and Griffiths

(2007) Probabilistic topic models. Handbook of Latent Semantic Analysis 427(7): 424–440.

49.

Suzor

(n.d.) Understanding content moderation systems: New methods to understand internet governance at scale, over time, and across platforms. In: Whalen R (ed.) Computational Legal Studies: The Promise and Challenge of Data-Driven Legal Research. Cheltenham: Edward Elgar Publishing, pp. 1–19. Available at: https://eprints.qut.edu.au/129464/ (accessed 15 April 2020).

50.

Suzor

(2019) Lawless: The Secret Rules That Govern Our Digital Lives. Cambridge: Cambridge University Press.

51.

Suzor

West

Quodling

, et al. (2019) What do we mean when we talk about transparency? Toward meaningful transparency in commercial content moderation. International Journal of Communication 13: 18.

52.

Taylor

(2015) Video games, fair use and the internet: The plight of the let’s play. University of Illinois Journal of Law, Technology & Policy 1: [i]272.

53.

Taylor

(2018) Watch Me Play: Twitch and the Rise of Game Live Streaming. Princeton, NJ: Princeton University Press.

54.

Tehranian

(2011) Infringement Nation : Copyright 2.0 and You. New York, NY: Oxford University Press. Available at: https://trove.nla.gov.au/work/38120261 (accessed 18 April 2019).

55.

Tushnet

(2014) All of this has happened before and all of this will happen again: Innovation in copyright licensing. Berkeley Technology Law Journal 29(3): 1447.

56.

Urban

Karaganis

and Schofield

(2017) Notice and Takedown in Everyday Practice. 22 March. Available at: https://papers.ssrn.com/abstract=2755628 (accessed 18 September 2017).

57.

Urban

and Quilter

(2006) Efficient process or ‘Chilling effects’? Takedown notices under section 512 of the digital millennium copyright act. Santa Clara Computer and High Technology Law Journal 22: 621.

58.

Van der Sar

(2018) Movie Company Demands €200,000 From YouTube Over Pirated Film. Available at: https://torrentfreak.com/movie-company-demands-e200000-from-youtube-over-pirated-film-181129/ (accessed 19 March 2019).

59.

Vaswani

Shazeer

Parmar

, et al. (2017) Attention is all you need. In: Guyon I, Luxburg UV, Bengio S, et al. (eds) Advances in Neural Information Processing Systems 30. New York, NY: Curran Associates, Inc., pp. 5998–6008. Available at: http://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf (accessed 10 April 2019).

60.

Wang

(2015) Fair use or copyright infringement? Deadspin and SB Nation get tossed off Twitter for NFL GIFs. In: Nieman Lab. Available at: http://www.niemanlab.org/2015/10/fair-use-or-copyright-infringement-deadspin-and-sb-nation-get-tossed-off-twitter-for-nfl-gifs/ (accessed 19 March 2019).

61.

(2007) Tolerated use. Columbia Journal of Law & Arts 31: 617.

62.

Yafit

L-A

(2013) Copyright lawmaking and public choice: From legislative battles to private ordering. Harvard Journal of Law & Technology 27: 203.

63.

YouTube (n.d.) Policies on harmful or dangerous content. Available at: https://support.google.com/youtube/answer/2801964?hl=en-GB (accessed 19 April 2019).

64.

Zarsky

(2016) The trouble with algorithmic decisions: An analytic road map to examine efficiency and fairness in automated and opaque decision making. Ziewitz M (ed.). Science, Technology, & Human Values 41(1): 118–132. DOI: 10.1177/0162243915605575

65.

Zimmerman

(2014) Copyright and social media: A tale of legislative abdication. Pace Law Review 35(1): 260.

Playing with machines: Using machine learning to understand automated copyright enforcement at scale

Abstract

Keywords

Background

Interrogating content moderation at scale

Methodology

Data collection

Topic selection

Classification

Results and discussion

Full movies

Gameplay

Sports

Hacks

Limitations

Conclusion

Footnotes

Acknowledgements

Declaration of conflicting interests

Funding

ORCID iD

Notes

References