Abstract
The legitimacy of public broadcasting has been under pressure for several decades. Attempts to repair and restore this legitimacy have led to an intensification of accountability instruments and measures. These instruments and measures tend to focus on consumption figures or cost-benefit analyses, requiring new ways of capturing public value. This article argues that, given the media’s persistent role as an interpreter and multiplier of news via traditional distribution channels and diverse new platforms providing information and opinions, professional follow-up communication is a relevant source for the examination of broadcasters' contributions to public debate. The essay explores how the analysis of newspapers via computational methods can be used by public broadcasters to reflect on and demonstrate their role in public debates in contributing a diversity of topics and viewpoints.
The purpose of public service media (PSM) programming, as famously articulated by BBC founder John Reith, is to inform, educate and entertain, or, in more contemporary wording, “public broadcasters should supply broad and varied programming that is informative, invites public debate and caters to all segments of the population” (Costera Meijer, 2005, p. 29). Though this public remit is anchored in European and national media legislation, it is difficult, if not impossible, to assess public broadcasters’ performance on the basis of these rather general values. However, a culture of accountability emerged in the 80 s and 90 s in EU countries that embraced a neoliberal approach to governing public institutions (Power, 1997; Shore & Wright, 2015). Accountability instruments were introduced in PSM that focused particularly on measurable, ‘objective’ performance indicators such as market share and audience reach. Recently, Mazzucato et al. (2020, pp. 40–45) proposed that the BBC needed to capture the full value it creates, which would require moving beyond simply direct value metrics such as consumption figures and cost-benefit analyses. What is needed is the development of metrics and indicators that reflect social aims rather than economic objectives (see also Couldry et al., 2016; Rogers, 2018; Van den Bulck, 2015).
In this paper, we introduce a model that can capture PSM’s contribution to public debates through the topic modelling of newspaper coverage. To begin, we explore the notion of public values in relation to PSM. We then discuss the ‘legitimacy crisis’ in PSM, which has fuelled the rise of accountability instruments and measures. However, many of the existing instruments are primarily oriented towards objective quantitative measures and fail to capture and assess the creation of public value. We go on to argue that the ways that PSM’s programming are picked up and discussed in the daily press can serve a robust indicator of public broadcasters’ involvement in public debate about societal issues. Inspired by research into everyday follow-up communication (Nuernbergk, 2014; Porten-Cheé, 2017), we examine mentions of Dutch TV and radio programming in professional follow-up communication in the national press during the 2017–2018 broadcast season to explore our method and to provide research-based evidence of PSM’s contribution to public debates. This offers the opportunity to reflect upon the societal relevance of PSM beyond the conventional quantitative metrics of broadcasters’ market shares and the audience recognition of individual programmes. In conclusion, we suggest that research into professional follow-up communication can help public broadcasters to reflect on and demonstrate their role and contribution to democratic societies.
This paper focuses on the European PSM context and the Dutch context in particular. The question of how to account for public value, however, is highly relevant to public-facing institutions globally. Exploring the unique Dutch PSM context, we can demonstrate the value of our methodology: the system – in which the NPO governs a series of broadcasting associations – allows for comparison of the singular impact and role of the various broadcasters with regard to public debate. More specifically, the proposed methodology enables the exploration of (a) the topical diversity of broadcasting associations’ contribution to public debates; (b) the question whether they drive public debates or simply react to topics widely covered in the press; (c) their visibility in debates; and (d) their perceived role in these debates. Through this methodology we can reflect on whether – and if so how – public broadcasters help foster diverse topics and contribute to the awareness of diverse viewpoints. Although the methodology was developed in response to a question regarding legitimacy posed to us by a Dutch PSM organization, we find it can also be usefully applied to understand other institutions that participate and shape public debate.
The Public Values Framework
In 1995 Mark H. Moore put forward public value as an approach to public sector management. This idea was his response to concerns about New Public Management (NPM), which had emerged during Thatcherism in the United Kingdom and had been imported to many countries worldwide by the end of the twentieth century. The different conceptions of NPM are bound by “a deep respect for the use of market discipline in governance” (Bozeman, 2007, p. 76). Here public managers are agents enacting the formal mandates of public enterprises. Moore (1995, p. 17) criticized the mentality of public sector managers as akin to those of bureaucrats or managers in the sphere of private enterprise. Rather than innovating and expanding value they helped maintain the status quo. Public managers should instead, he argued seek to create public value: “we should evaluate the efforts of public sector managers not in the economic marketplace of individual consumers but in the political marketplace of citizens and the collective decisions of representative democratic institutions” (Moore, 1995, p. 31).
One example of an institution that has adopted this public value approach is the BBC. To legitimize its operations and guide its practice, the broadcaster embraced the public value doctrine (Collins, 2007) in Building Public Value: Renewing the BBC for a Digital World (2004). This policy document set out the scope of public value in its practical implications. Here public value was defined as “a measure of its contribution to the quality of life in the UK” (BBC, 2004, p. 29) and audiences were addressed not as consumers but as members of society as a whole. The document also suggested that objective methods of measurement and assessment in evaluating public value could be developed. Scholars have highlighted the importance of public value within the context of PSM in its capacity to bring “society and public institutions closer” together, reflecting the ambition that the public conceive of public services as their own (Ibarra, 2015, p. 150). The BBC’s approach to public value has set a benchmark for public organizations around the world (Mazzucato et al., 2020, p. 13).
Consultants in the private sector have argued that the BBC’s use of the public value framework is nothing more than a rhetorical strategy (Elstein, 2004; Oakley et al., 2006). Similarly, Oakley et al. (2006, p. 7) conclude that the broadcaster has used public value inconsistently and opportunistically; since its adoption the BBC’s fundamental modus operandi and its relation to its audience have not significantly changed. In the BBC’s defence, Alford and O’Flynn (2009) counter that these critics have misinterpreted public value by casting it primarily as a performance measurement framework, insufficiently recognizing that value can be measured not only in economic terms but in other ways as well.
Mazzucato and Ryan-Collins (2019) have analysed how public management theories have failed to tackle the issue of market failure, an underlying assumption of New Public Management. According to the logic of market failure, if public institutions do more than fix market failures, they crowd out private actors in the market and slow down innovation (Mazzucato & Ryan-Collins, 2019). Mazzucato and Ryan-Collins find such reasoning as well in the UK government’s White Paper on BBC Charter renewal in 2016. Mazzucato (2019) invites public institutions to take leadership roles so as to create industry value rather than simply fix market failures. To do so would require moving beyond static metrics such as consumption figures or cost-benefit analyses and to develop new ways of capturing public value (Mazzucato et al., 2020, p. 41). This argument aligns with critiques of existing performance measures that are oriented towards economic value (see also Couldry et al., 2016; Rogers, 2018).
The Legitimacy Crisis and the Need to Measure Public Value Creation
Technological developments, the emergence of commercial television and neoliberal media policy fostered by EU legislation have all contributed to PSM losing “their monopoly, legitimacy and ‘self-evident’ support from governments” (Van den Bulck, 2015, p. 78). The European Member States recognize that PSM cannot be taken for granted; their existence needs to be justified (Ibarra et al., 2015, p. 2). Attempts to repair and rebuild the eroded legitimacy of PSM have intensified advocacy for accountability instruments and measures (Shore & Wright, 2015; Van den Bulck, 2015).
Democracy & PSM, a 2019 report by the European Broadcasting Union (2019), uses market share, funding levels and the results of a public opinion survey on perceived political pressure on public broadcasters as proxies for a strong PSM. Interestingly, this report considers neither the quality and diversity of programming and their information content nor the contributions to public debate made by PSM. National procedures for public value tests are also marked by these blind spots (Gransow, 2018; Moe, 2010). Under the terms of European media legislation defined by the Amsterdam Protocol in 1997 and specified in the EC’s Broadcasting Communication of 2009, new services by public broadcasters must undergo ‘Public Value Tests’ to demonstrate that they not only serve the public interest but at the same time do not disrupt competition in the so-called free market (Brevini, 2013; Donders & Moe, 2012). Instead of focusing on PSM’s mission, these tests make public broadcasting first and foremost accountable to the market (Van den Bulck & Moe, 2012). Thus these institutions are “torn between evidencing market value, a concept strongly embedded in commercial logic where success depends on achieving sufficient popularity, and embodying its mandate as a not-for-profit institution with values that are in principle contrary to that logic” (Lowe & Martin, 2013, p. 20).
Performance benchmarks based on audience reach are caught in somewhat of a double-bind. As Ibarra et al. (2015) point out, “if PSM simply emulate commercial providers, then they risk losing their special status, their unique selling point; if, however, they differentiate themselves to the extent that their audience share declines, then they risk losing public and political support” (p. 5). After all, PSM, to exert an impact, must reach ‘the public’ at large. Audience ratings as a metric, however, entails two obvious caveats. First, reaching a single person who is actually in a position to make meaningful change can produce greater impact than simply ‘reaching’ millions of others who have no such potential efficacy. Second, PSM are expected to cater to an array of diverse social groups, including young people and members of ethnic minorities.
However, as Van den Bulck (2015) rightly remarks, none of the existing accountability arrangements solve the question of how PSM (and other media for that matter) can finally come to render account to their main stakeholder: the audience or public. Being made accountable to the market (through ex ante tests) and its consumers (through performance benchmarks based on audience reach) does not provide any guarantee for being made accountable to the citizenry and thus for promoting democracy. (pp. 83–84)
With the increasing individualization and growing diversity in the Netherlands since the 1960s, the ‘depillarization’ (Semetko, 1998) process, accelerated through the advent of commercial television in the late 1980s and the World Wide Web in the late 1990s, has brought about a dramatic reduction in the membership size of the broadcasting associations, calling the participatory rationale of the unique Dutch system into question. The membership-based system has been cracking under mounting criticism and the broadcasters, for their part, have sought new accountability instruments and measures to address the issues of pluralism and diversity. Are they, in fact, stimulating public debate, fostering diverse topics and contributing to an awareness of diverse viewpoints? In what follows we propose a methodology that charts the contribution of the Dutch broadcasting associations to public debate. First, however, we propose that newspapers provide a useful indicator of the broadcasters’ contribution to public debate.
Professional Follow-up Communication as Indicator
Traditionally, the national press has been considered a part of the public sphere: it constitutes a structured space for a society-wide exchange of information and opinion that mediate among citizens, collective actors and the political system as a whole (Gerhards & Neidhardt, 1993). Newspapers are often used in research as indicators of public debate (Wevers, 2017, p. 31). As Michael Schudson (1995) writes: When media offer the public an item of news, they confer upon it public legitimacy. They bring it into a common public forum where it can be discussed by a general audience. They not only distribute the report of an event or announcement to a large group, they amplify it. An event or speech or document in one location becomes within a day, or within hours, or instantaneously, available to millions of people all over a region or country or the world. This has enormous effects. (p. 19)
Given the media’s persistent role as an interpreter and multiplier of the news via traditional distribution channels and diverse new platforms, professional ‘follow-up communication’ is a relevant source for the examination of broadcasters’ contribution to public debate. We follow here the approach of research on follow-up communication that analyses how TV and radio programming is mentioned and discussed in everyday communication (Porten-Cheé, 2017). While this tradition, as a branch of media effects research, is exclusively dedicated to everyday communication on the part of audience members and excludes professional communication, we regard press coverage of TV and radio programmes as a professional form of follow-up communication. From our vantage, this particular form of follow-up communication points to the societal meaningfulness of a given TV or radio programme according to the professional routines of the press.
Our contention is that professional follow-up communication in newspapers is a meaningful indicator of public debate. It serves to complement rather than replace existing indicators. However, as the media landscape continues to evolve and transform, other media sources may be considered more relevant indicators of public debate. It is significant then that the types of analyses we propose are useful beyond follow-up communication in newspapers. These analyses can easily be transposed to online news websites (e.g. theguardian.com, bbc.co.uk etc.), discussions on social media platforms (e.g. Facebook, Reddit, Twitter, etc.), or even the closed captioning of radio and television.
With increasing digitization of newspaper editions and archives, professional follow-up communication has become ever more accessible in digital format. Researchers in the humanities have increasingly advocated the use of quantitative methods and digital tools to explore large collections of texts, a practice also referred to as “distant reading” (Moretti, 2013). In contrast to the close reading performed on only a few texts, researchers can now, with the help of computer software, aggregate and analyse thousands of texts.
Later in this paper we will compare the number of mentions of programmes in newspapers to the number of viewers. This comparison demonstrates that when quantitatively exploring mentions in newspapers, we are measuring something rather different than audience ratings or popularity for that matter. The comparison was made among programme types, allowing us to characterize professional follow-up communication in newspapers about television and radio programming according to preferred genres.
Modelling Broadcasters’ Contribution to Public Debate
Analysis of follow-up communication through a computational lens can be a valuable means to assess PSM’s societal contributions. One fruitful computational technique here is topic modelling, or more specifically Latent Dirichlet Allocation (Blei et al., 2003). LDA is an unsupervised learning algorithm that allows for the automatic identification of so-called topics within a large volume of text. Topics are represented as multinomial distributions over all the words in the corpus under consideration. That is: each word has a score associated with each topic, indicating the chance that a given word occurs within a given topic. Sorting words with the highest probability – for example: football, tennis, hockey – then allows a representative topic label (sports) to be identified. The texts fed to the model are, in turn, characterized by a distribution over all the topics. With topic modelling it is important to engage the most probable words that build these topics in order to determine whether they are actually meaningful, as they may not be coherent or stable (Schmidt, 2012).
To illustrate the types of analysis that can be carried out using topic modelling, we will expand upon the methodology we developed for a research project commissioned by the Dutch Catholic-Protestant Broadcasting Association covering the broadcast season 2017–2018. We pursued two questions. First, in what topical contexts are PSMs’ programmes frequently discussed? Second, what role does PSM play in public debates? To find answers we began by gathering one year’s worth of articles from nine national newspapers and three selected magazines. The articles had been published between 1 September 2017 and 31 August 2018 (one broadcasting season). The corpus contained more than 263.000 articles. To identify mentions of PSM programmes in these articles, we composed a list of programme titles by PSM that aired during the same period using data from the Dutch viewer audience measurement service (SKO). In drawing up the list, we removed programmes which were aired as reruns. Moreover, we included sport programmes but labelled them separately so that they could be filtered from analysis in a later stage, and we clustered all individual broadcasts of sport events (ice skating and ski jumping from the Olympics) as a single televised sport event (the Olympic Games).
After compiling the list, we searched the newspaper corpus for occurrences of those titles. Although the titles of most programmes are distinctive enough that they wouldn’t be found in other textual contexts, some titles cannot be detected through a simple search. We identified roughly four types of non-distinctive titles:
titles containing common adjective/nouns; shows or movies that have been adapted from a different media context (e.g. book-to-film adaptations); talk shows or radio programmes named after the presenter; events also broadcast on television/radio.
Additionally, sometimes only the programme’s abbreviated title is mentioned. To detect non-distinctive or variational titles, we guided our search queries via a set of rules, e.g. a reference to ‘the voice’ counts when the text mentions the word ‘programme’ or ‘NBC’. Most rules can be written in such a fashion, with the exception of talk shows and radio programmes named after the presenter. These titles require the manual classification of sentences, which can be used to train a machine-learning algorithm (more specifically: a linear SVM), on the basis of context words, to automatically distinguish between programme titles and proper names.
To train our topic model, we used the scikit-learn implementation (Pedregosa et al., 2011) from LDA with batch learning. Before training the model, the corpus was prepared and filtered to optimize the output. We lemmatized all the words in the corpus using Frog (Van den Bosch et al., 2007), removed stop words using a limited stop word list and all words occurring in fewer than 50 articles or in more than 70% of all articles. Because LDA supposes a fixed number of topics, we specified this beforehand and thus the choice is somewhat arbitrary. In the sciences it is common to evaluate the quality of the themes on the basis of different quantitative parameters, but doing so doesn’t necessarily lead to more coherent or easier-to-interpret topics (Chang et al., 2009). We compared and evaluated models trained on varying numbers of topics (we chose 25, 50, 75, 100, 150, 200 and 250) manually and decided that the 200-topic model resulted in the most relevant and interpretable topics.
As stated, each topic in LDA is represented not as a label but as a distribution over all the words in the corpus, and the words with the highest probability of occurring can be used to label the topics manually. The topics were interpreted and labelled by three researchers. We found 13 of the 200 topics to be uninterpretable, many of them having auxiliary verbs as the words most frequently associated with them. Furthermore, seven topics consisted of a rather unusual combination of two or more subjects, such as Catalan Independence and Poland. This outcome was most likely due to our decision not to remove proper nouns, which added some degree of noise to patterns of co-occurring words – if two articles mention different men named Dave, for example, they algorithmically seem more closely related.
Newspaper Mentions Versus Viewers
Earlier we provided that professional follow-up communication in newspapers is distinct from ratings and audience appreciation and could be a useful indicator of public debate. To support the idea we compared the number of mentions of programmes in newspapers to the number of viewers. The scatterplot on the left side of Figure 1 shows the correlation between the number of mentions each television show received and its average number of viewers. As is evident, the number of viewers is a rather weak predictor for mentions, with an
The Relation Between the Number of Mentions and the Number of Viewers Specified by the Genre of the Programme.
Mentions of the programmes tagged in the articles enable us to ask questions about the types of programmes that are generally popular in newspaper articles. The right side of Figure 1, showing the relative share of each genre in the total number of mentions (dark grey) and viewers (light grey), partly provides an explanation for this discrepancy. 1 Whereas entertainment programmes are among the programmes with the highest shares of total viewer numbers, their share in the total number of mentions in the newspaper corpus is relatively small. Programmes labelled as news/actualities, on the other hand, have a relatively small viewer share but the highest share in the number of mentions.
To conclude, the number of mentions enables us to measure what one could call ‘newspaper exposure’ or professional appreciation, as opposed to ratings and audience appreciation. Of course, programmes focused on news/actualities are not necessarily valued more in newspaper articles. After all, it seems obvious that a programme focused on newsworthy topics is more relevant to a medium that is inherently aimed at reporting current events than, say, a long-running television quiz show would be. But the number of mentions does indicate that these programmes reach their audiences not only directly but also indirectly via newspapers – through professional follow-up communication.
The Contribution of Broadcaster Associations to Public Debate
Having briefly characterized professional follow-up communication in newspapers, we now introduce and discuss two types of analyses that we conducted, aimed at assessing the role of the Dutch broadcasting associations in public debates: (1) analysis of distinctive topics, which identified the most distinctive topics mentioning PSM programming as compared to the topics of all newspaper articles in our dataset; and (2) quantitative and qualitative timeline analysis, which illustrates the development of certain topics over time and the role played by PSM programmes in public debates.
Analysing Their Most Distinctive Topics
Whereas if we analyse the mentions alone we discover which types of programmes are frequently mentioned in newspaper articles, when we combine those mentions with the topics outputted by our LDA model we can examine the thematic contexts in which these mentions of PSM programmes occur. A first step is to analyse the most distinctive topics of the articles mentioning PSM programmes. We define the distinctiveness of a topic as the relative proportion of the topics in articles mentioning PSM programmes (reference corpus) weighted against all other articles (target corpus). In accordance with other research on comparing and calculating distinctive topics of subsets of corpora, we divide the average proportion of the topics of the reference corpus by that of the target corpus (Oelke et al., 2014). Analysis of the most distinctive topics of articles mentioning PSM programmes provides an initial exploration of the topical tendencies of PSM and their topical diversity in contribution to public debates.
Table 1 shows the ten most distinctive topics for the articles mentioning PSM programmes. The four most distinctive topics are all words concerned with radio and television programmes. This is not surprising, since one would expect that articles that mention PSM programmes generally use more words revolving around radio and television than articles that do not mention PSM programmes. The four remaining topics are more interesting in that they deviate more from expectations. They fall into two categories. First, articles mentioning PSM programmes pay significantly greater attention to culture and the arts, indicated by the topics Theatre and Classical Music. Second, PSM programmes are often discussed in the context of Dutch government and the royal house, as is indicated by the topics Dutch politics and Dutch Royal Family.
The Most Distinctive Topics for Articles Mentioning PSM Programmes Compared to the Reference Corpus.
When we switch perspectives and look at the most distinctive topics of the articles not mentioning PSM programmes, we can identify those topics that are structurally absent in discussions about PSM programmes (see Table 2). These topics can roughly be cast into three categories. First, we see topics indicating structural segments of newspapers, like recipes (Cooking), Weather, and Puzzles. Second, there are four topics revolving around the economy and finance: Stock exchange, Economy, Investments and cryptocurrencies, and Banking and finance. The absence of PSM programmes indicating structural segments of newspapers is easy to explain, since one does not expect PSM programmes to be mentioned in crossword puzzles. The absence of mentions in the coverage of topics regarding the economy and finance could indicate structural blind spots in PSM programming. Lacunae can also be found in the third category of topics: foreign politics, indicated by the topics German politics and Catalan Independence and Poland. The lack of foreign politics here contrasts strikingly with the prominence of local topics seen in Table 1, suggesting that PSM programmes focus on national politics, which public broadcasters’ own correspondents cover thoroughly. In contrast, the absence of mentions of PSM programmes within topical contexts such as the economy, finance and foreign politics might also indicate that professional journalists refer to other sources than television and radio programmes when covering these topics, and particularly to the news reports of the press agencies.
The Most Distinctive Topics of the Reference Corpus Compared to the Articles PSM Programmes Are Mentioned in.
Driving or Reacting to Topics: A Quantitative Timeline Analysis
Another form of analysis we propose is focused on the development over time of certain topics. More specifically, we sought to disclose whether PSM topics follow proportional spikes in the newspaper corpus and, if not, whether these topics were discussed earlier or later in the context of programmes. The underlying question here is whether public broadcasters help drive public debates – or whether they are simply reacting to topics widely covered in the press.
As an example, we examined the development of the topic labelled #MeToo – referring to the movement aimed at speaking out against sexual abuse. The topic’s fluctuations and newsworthiness make it interesting to drill down into: it allows for comparison of the temporal (drive vs. react), the quantitative (size of contribution) and the qualitative contribution (role) of the broadcasting associations to public debates. As shown in Figure 2, the topic primarily revolves around words such as ‘sexual’, ‘abuse’, ‘victim’ and ‘#MeToo’, but the names of specific persons associated with the #MeToo-movement, Harvey Weinstein and – in the Dutch case – Jelle Brandt Corstius (who disclosed himself as a victim of sexual abuse), are also in the word cloud.
Word Cloud of the Topic Labelled #MeToo.
Figure 3 shows the development of the topic #MeToo over a yearlong period. Here we divided the articles according to the eleven public broadcasting associations and report the topic distribution and development over time as they pertain to each of the associations. The plot second from bottom illustrates the topic’s development in articles mentioning PSM programmes by all public broadcasting associations, and the bottom plot traces its development in all newspaper articles.
Yearlong Development of the Topic ‘Sexual Abuse’ in Articles Mentioning PSM Programming; the Values Calculated Monthly, the Inbetween Values Are Interpolated Using Pchip and the Broadcasters Are Sorted by Average Proportion.
The results show that while the average proportion of the topic #MeToo is, overall, considerably lower in all articles compared to the entirety of articles mentioning PSM programmes (and therefore appears more flattened out in the figure), the proportion of the topic in all articles shows a clear increase between September and November of 2017. The first allegations (setting aside earlier rumours) against Weinstein came out in early October, and the hashtag #MeToo took flight in the middle of October, leading other victims of sexual abuse to come forward under its banner. These events are reflected in the proportion of the topic #MeToo in the newspaper corpus, which spiked in November 2017, achieving a proportion almost triple what it had been in September 2017. After November, the attention given the topic slowly faded, though there continued to be some sporadically renewed interest in it.
Looking at the timelines of the topic #MeToo in Dutch newspapers (plot first from bottom), it is evident that the yearlong proportion of the topic in articles mentioning PSM programmes (plot second from bottom) follows roughly the same trend as in the complete newspaper corpus. However, whereas the proportion of #MeToo in all newspaper articles decreased 39% between November and December, it does so by 73% in articles mentioning PSM programmes. This suggests a more short-term attention span in professional follow-up communication.
Furthermore, breaking down the proportion by public broadcasting associations shows that while PSM mentions in overall articles follow the proportional spikes of the newspapers, the individual broadcasting associations show no such pattern. In fact, the proportion of the topic #MeToo in most subsets of articles does not follow the same trend that we see in the complete set of articles. In articles discussing programmes by PowNed and EO we see a peak appearing earlier, in October 2017, indicating an early relevance for their programming about the topic, whereas the proportion of #MeToo in articles discussing programmes by both VPRO and HUMAN peaks much later, in February 2018. We should note, though, that the proportion in articles discussing programmes aired on the three largest public broadcasting associations – BNNVARA, AVROTROS and KRO-NCRV – all peak in November 2018, similar to what we find in the newspaper articles. Given that these associations also receive the most mentions, their contribution weights more heavily in the total average than those of other broadcasting associations. Finally, the share of #MeToo in articles discussing BNNVARA’s programmes is by far the highest among all associations – more than twice that of all PSM articles in November.
Characterizing Their Role in Public Debates: A Qualitative Timeline Analysis
The quantitative timeline analysis helps to gauge the visibility of particular public broadcasters in the debate. From our quantitative analysis it might be tempting to conclude that BNNVARA played a fairly prominent role in the public debate on sexual abuse compared to other broadcasting associations. However, to stop here would be to overlook how that topic is discussed in relation to their programming. Only by diving into the underlying articles is it possible to reflect on the particular role played by the broadcasters in discussing sexual abuse.
In working one’s way around the vast number of articles, a relatively effective strategy to combine the more ‘distant’ quantitative approach as represented in Figure 3 with close reading is to take a specific subset of articles – here, all articles mentioning BNNVARA within a limited time period (November 2017) – and then to sort these articles according to the proportion of the topic one is interested in, in this case #MeToo. The qualitative contribution of BNNVARA aligns more with entertainment values than with ideals of information and critical discussion. Most of the mentions in BNNVARA’s programmes within the context of sexual abuse are centred on the aforementioned case of the journalist and author Jelle Brandt Corstius, who on one of BNNVARA’s talk shows accused his former producer Gijs van Dam of sexual assaulting him, followed by a live public denial from his alleged abuser on another BNNVARA talk show. While it can be argued that providing a platform for speaking out against sexual abuse is to some extent aligned with the aims of the #MeToo movement, BNNVARA’s role in the press coverage was limited to the platform it offered, with most mentions referring to what Jelle Brandt Corstius had said on De Wereld Draait Door, the most-watched Dutch talk show, and how the accused producer had responded on BNNVARA’s late-night talk show Pauw. Typically the articles covering these sensational events contain no guidance as to how such a sensitive topic should be discussed and do not attempt to provide a sensible interpretation of the case. The tenor of the coverage led many professional television commentators to argue that the back-and-forth allegations between the two parties in the dispute belong in court and should not be televised.
In contrast, the qualitative analysis of KRO-NCRV mentions yields a picture that shows a very different role played in the debate on sexual abuse – despite the broadcaster having a substantially smaller share in the topic. During the peak of the #MeToo-movement, most mentions to KRO-NCRV’s programmes refer to an investigative journalism television programme that contained allegations of sexual abuse from dozens of (relatively unknown) models. Whereas the debate had initially focused mainly on celebrities, the newspapers discussed KRO-NCRV’s contribution as a first step in lowering the threshold for people to air accusations and allowing relatively unknown people to come out with their stories. The close qualitative reading of newspaper coverage thus allows for insights into the reception of programmes and their contribution to public debates, as perceived, interpreted and proliferated by the press.
Conclusions
In this paper we responded to the fact that current accountability measures are directed towards the market and consumers but not the public at large. There thus needs to be instruments that reflect PSM’s fulfilment of their public remit, which includes support for citizenship. We have proposed a methodology so as to provide just such an instrument. Our contention is that professional follow-up communication in newspapers provides meaningful and robust data for evaluating broadcasters’ societal role and their contributions to public debate.
Using computational methods we sought to illustrate how the role of public broadcasting in the public sphere might be evaluated based on professional follow-up communication. First, in terms of diversity, we have shown how topic modelling might be a useful way to explore the range of topics that public broadcasters are associated with (and what blind spots occur) in newspapers. For the Dutch system we could identify how the broadcasting associations each contributed to the pluralism of the PSM system as a whole. Second, we have shown that temporally plotting topics makes it possible to reflect on whether broadcasters drive or simply follow the debates that emerge in the press. However, as explained above, the quantitatively significant association of a broadcaster with a particular topic does not reveal how the broadcaster’s programmes and the related press coverage treat the topic in question. Therefore, we lastly added the qualitative component of close reading of articles, which allowed us to characterize the role played by public broadcasting associations and their programmes within the development and interpretations of specific topics over time.
While the proposed method is robust and useful in many respects, we must be aware of the potential biases of professional follow-up communication. Follow-up communication here is bound by what counts as newsworthy by journalists and their professional routines. It might be said that journalists, as members of “communities of practice” (Meltzer & Martik, 2017), are caught in their own professional bubble and do not cover all diverse groups and positions (or do so unequally) within a wider society. Moreover, our working assumption here is that national newspapers are and will remain significant proxies for the topics of public debate for the foreseeable future. However, as mentioned earlier, the proposed analyses can be appropriated for other media sources as well.
The metrics used for evaluating an institution should correspond to their distinct goals and strategies and tailored for specific industries. Momentarily consumption figures and cost-benefit analyses are used to evaluate PSM, which tends to overlook their public value creation. We encourage the production of instruments that invite reflection on the objective that represents the core values of public facing institutions. Here repurposing existing metric scores and modes of engagement on social media could create useful proxies for evaluation (see Rogers, 2013). To realize a multidimensional approach, that captures the full breadth of their impact, would require combining (existing) metrics and instruments.
Within the context of debates about the legitimacy of PSM and the public value they create, the types of analysis we have discussed here are not only methodologically interesting but also generate results and insights that are instructive and are relevant politically. Recently, for the measurement of public value, some useful avenues for further development have been identified (see Mazzucato et al., 2020). Our suggestion is that public-facing institutions can use research into mentions of their programmes in professional follow-up communication to review their role in public debates.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article is based on a study that was commissioned by the Catholic-Protestant broadcasting association KRO-NCRV in the Netherlands.
Note
Author Biographies
