Data-driven news work culture: Reconciling tensions in epistemic values and practices of news journalism

Abstract

This study investigates the epistemological implications of the appropriation of audience analytics in a data-driven news culture. Focussing on two central aspects of epistemology, epistemic value and epistemic practices, we ask two overall questions (1) How are audience metrics balanced and reconciled in relation to other standards in the justification of news as valuable knowledge? How are different practices of research and presentation, truth-seeking and truth-telling, prioritized in a news organization marked as a data-driven news work culture? The study presents a case study of a Scandinavian legacy news publisher that has pursued the embracing of a data-driven news work culture. It is based on a qualitative multi-method approach. The findings show how metrics are used as a superior standard in deciding on the epistemic value of news. This is expressed in strategies, guidelines and discussions in the newsroom, and put into practice in coaching, evaluations and rewarding of the performance of individual journalists. In the everyday news production, metrics are reconciled in relation to independent standards in journalism, related to the claims of news journalism to provide relevant and verified public knowledge about current events. Moreover, the study shows how the embracement of metrics radicalizes the focus on presentation, packaging and timing in the optimization of news material and in the valuing of professional practices. Efforts in research and truth seeking are more seldom explicitly valued. The work of fulfilling reasonable truth claims is mainly taken for granted.

Introduction

Data has become increasingly central to how contemporary organizations operate, and practitioners and academics have suggested that data is the new oil, required to remain competitive. Organizations with proprietary platforms have tremendous possibilities for developing analytics infrastructures that collect and analyse usage patterns. Beer (2018) studied the data analytics industry and identified a data analytic imaginary where analytics and metrics are envisioned as being speedy, accessible, revealing, prophetic and smart. Such visions influence how their customers appropriate data into their organizations and practices. The power of analytics ultimately comes down to what it can reveal, and how it is being used to inform and guide journalists’ priorities in their news work (Carlson, 2018; Steensen and Westlund, 2021; Tandoc, 2019). Previous research has analysed the impact of audience analytics and metrics on the understanding of audiences (Anderson, 2011; Nelson, 2021; Zamith, 2015); the organization and institutionalization of distinctive roles in the newsroom (Ferrer-Conill and Tandoc, 2018); editorial decision-making and news judgement (Anderson, 2011; Napoli, 2011; Vu, 2014); and tensions related to journalistic values and qualities (Costera Meijer, 2012; Usher, 2012). Much research has focused on market logics associated with analytics in journalism, but less research has focused on epistemology. In advancing the concept of measurable journalism, Carlson (2018: 412) argues that a fundamental issue for future research concerns the implications of such development when it comes to the role of journalism in providing important and reliable knowledge about the world.

This article suggests there is a need for further research into the implications of the appropriation of data-driven news work, for the values and practices characterizing news journalism as knowledge-producing processes and institutions claiming epistemic authority in society (Carlson, 2017; Ekström and Westlund, 2019a). The study investigates two central aspects of the epistemology of news journalism (Ekström, 2002): (1) how journalists understand and justify news as valuable knowledge for the public (defined as the epistemic value of news); (2) how journalists apply and prioritize practices of truth seeking and truth telling (defined as the epistemic practices of journalism). More specifically, we explore tensions in the epistemology of a data-driven news journalism and the reconcilement of such tensions within a news organization.

The article presents a qualitative multi-method newsroom study, of a Scandinavian legacy news publisher that has pursued the embracing of a data-driven news work culture. They have strategically prioritized developing proprietary platforms for publishing (website and mobile applications) as well as audience analytics and advertising. The news publisher uses both proprietary audience analytics (systems and tools that they own and control) and non-proprietary audience analytics (offered by a third party with whom they share money and data) to produce and analyse metrics, including, but not limited to, such measuring patterns in real time, and across time, demographics and titles. Referring to this as a data-driven news work culture, we recognize the ways in which the news organization prioritizes and gives status to values and practices in line with the data-driven strategies.

Theoretical approach and research questions

The epistemic value of news

Social institutions whose main task is to produce knowledge within a particular domain typically claim significance concerning the value of this knowledge in society. News journalism promises news as a form of knowledge of certain value for the public. We refer to this as the epistemic value of news. This is implied and argued in meta-discourses of news journalism. Moreover, what is understood as valuable public knowledge is indicated in the standards in relation to which decisions and priorities are justified within news journalism. The epistemic value of news is not to be confused with the concept of news value that defines the specific criteria (timeliness, negativity, unexpectedness, superlativeness, etc.) applied in the selection and construction of news.

The role of news journalism in offering citizens important and reliable information is central to normative theories of the professionalization and democratic significance of journalism. Professional judgements are understood as elements of a journalism that claims relative autonomy and self-regulation in working for a public good (Örnebring, 2017; Waldenström et al., 2019). Autonomy essentially refers to how the standards applied in assessing what counts as valuable to inform about are decided from within, internally, amongst the journalists. This does not mean that news journalism defines what is valuable completely independently of market values or audience interests. Professionalization, however, implies a certain autonomy in judgements and expertise, which is a prerequisite for news to have epistemic qualities different from other sorts of information.

Audience analytics producing metrics has emerged as another form of value system in news journalism. The development represents a continuation of the attempts in journalism to quantify audiences and adjust the production of news to maximize reach, reading and engagement, justified with respect to both readers’ interest and commercial values (Carlson, 2018; Zamith, 2018). Amid steep declines in advertising and increasing prioritization of reader revenues, news publishers utilize analytics to guide news work with the aim of stimulating conversion into paid subscriptions (O’Brien et al., 2020; Wadbring and Bergström, 2021).

The development of an increasingly data-driven news work has prompted critical discussions, indicating tensions between value systems. The greater impact of metrics has been described as a shift from journalists’ perceptions about what audiences need to what data shows that audiences want (Ferrer-Conill and Tandoc, 2018; Zamith, 2015). The relationship between independent journalistic standards and the reliance on audience data is more complex. As Nelson (2021: 152–154) argues, in a news journalism that embraces increasingly sophisticated methods for learning about what audiences want, journalists will continue to depend on their own professional judgements of how to best serve the public. Moreover, depending on how information about audiences are used, the actual operationalization of what audiences want and value in news journalism will be more or less compatible with independent journalistic standards for the quality of news (Costera Meijer, 2012, 2021;Neilson and Gibson, 2021; Zamith, 2015: 143). A strong focus on hits and ratings is in potential conflict with the claims of journalism to produce news that is decisive for a prosperous public. However, as Costera Meijer (2012) shows, a careful listening to audiences’ experiences of news indicates that their valuing of truthful news, representing experiences, concerns and issues in society, is essentially compatible with core values of professional journalism (Costera Meijer, 2012).

This study focuses on how different standards of epistemic value are prioritized and reconciled in a data-driven news culture. While such a news culture intentionally gives precedence to metrics as the main value system, this does not mean that metrics have entirely replaced other standards in making journalistic judgements. We argue that a key characteristic of the data-driven news culture is the embracing and learning from audience analytics and the metrics produced, but in combination with a recurring activation of other values in making journalistic judgements. This parallel existence is critical since news publishers must carefully navigate in order to publish a steady flow of news that audiences want and are willing to pay for, while preserving journalistic authority and credibility by investing significant epistemic efforts into investigative reporting, the coverage of areas understood as important, and so forth.

With regard to the epistemic value of news, we ask (RQ1): How are audience metrics balanced and reconciled in relation to other standards in the newsroom? More specifically, we study (1) how journalists’ understandings of news as valuable knowledge are articulated in discussions in the newsroom, and (2) how standards are enacted in practices of professional judgements, when journalists decide and justify what is valuable to produce and publish.

The epistemic practices of news journalism

Like other knowledge-producing activities (science, public inquiries), news journalism is organized (1) to know about the world and (2) to present information for intended recipients. (Information is conceptually interrelated with, yet distinct from, knowledge, but in this article we will use these terms interchangeably.) We refer to these epistemic practices as research (including truth seeking) and presentation (including truth telling), both performed through various activities and routines (Ekström and Westlund, 2019a). News journalism typically relies on gathering and processing of information: access and evaluation of sources (practices of research, to know about), as well as styles of writing and talking, visualizations and editing into various formats (practices of presentation). News journalists claim authority as ‘credible spokespersons of “real-life” events’ (Zelizer, 1992 in Carlson 2017: 4), in what and how they know of a world beyond people’s everyday experiences, as well as in their audience-friendly and truthful presentation of news. News is information produced for an audience. While powerful news may exist without much or any independent research, the presentational aspects are embedded in all phases of news production (Ekström 2002).

The embracement of data-driven news work may have various consequences for the epistemic practices of news production. Audience analytics provides metrics in relation to which the news production and the job performances of individual journalists can be evaluated and improved. Depending on how metrics are used, they will have different consequences for how the news organization and individual journalists prioritize between efforts in research, presentation and packaging of news – efforts of truth seeking and truth telling.

With regard to the epistemic practices we ask (RQ 2): How are different practices of research, presentation and packaging prioritized in a news organization marked as a data-driven news work culture? More specifically, we examine priorities in evaluations and guidance of reporters, as well as in the everyday work of reporters and editors.

Research about data-driven news work, epistemic values and practices

Pioneering studies indicate an impact of audience metrics on how journalists justify the value of news (e.g. Anderson, 2011; Usher, 2012). The professional autonomy in news judgements tends to decrease, yet research also indicates a persistence in professional values (Anderson, 2011). Comparing two online newspapers, Anderson (2011) observed differences in the use and understanding of metrics, related to management strategies and the strength of a professional journalistic culture. Usher (2012) studied a newspaper’s transition to online first, and increased focus on metrics. Journalists expressed concerns that the particular public values (humanistic, thoughtful, not fast breaking) and the identity of the newspaper would be threatened.

Many studies have followed. Zamith (2015: 144) suggests that the proliferation of audience analytics may have contrastive implications for the civic values of journalism. A maximizing of output may result in news vital to a public discourse being demoted, but information about the audience may also help journalism to prioritize news of public interest. Audience analytics potentially offers more useful information about what audiences want to read than traditional industry surveys and focus groups carried out to tap into news consumption (Steensen and Westlund, 2021). Analytics is associated with a market orientation, and studies suggest that businesspeople have taken a strong position in defining and developing the goals and uses of such technology (Slaček Brlek et al., 2017). Studying the role of analytics companies in the transformation of news production in the US, Belair-Gagnon and Holton (2018: 505) conclude that these actors tend to ‘foster profit-oriented norms and values in newsrooms’. We find that the market orientation and acceptance of data-driven practices, in the above-mentioned studies and beyond, forms the emerging norm. Among recent exceptions though, found that the Belgian journalists studied remain hesitant or even fearful towards data-driven news practices. Altogether, these studies witness to emergence of a data-driven news culture.

Extensive research provides evidence of the impact of audience metrics on news judgement, decision-making, and practices primarily related to the presentation and packaging of news (Ferrer-Conill and Tandoc, 2018; Napoli, 2011; Zamith, 2015). Metrics inform editorial decisions, such as how journalists prioritize their leads for stories (Chua and Duffy, 2019), as well as how algorithms and online editors make priorities for the exposure of different news articles (Tandoc, 2019; Zamith, 2018). Audience-oriented editors, responsible for the navigation and incorporation of metrics, are increasingly central to news work (Ferrer-Conill and Tandoc, 2018), extending the work of editors in conducting continuous follow-ups, refining headings and reorganizing the site to make articles more or less prominent (Vu, 2014). However, the priorities between the epistemic practices of research (truth seeking) and presentation (truth telling) have not been thoroughly studied amid analytics and metrics.

The case: A legacy news publisher marked by a data-driven news work culture

This article reports on one of the local and online-first news outlets managed by MittMedia, a large legacy news media organization in Sweden focussing on local and regional news. For the sake of anonymity, we will refer to it as the ‘Swedish Local Post’ (SLP). We carried out substantial fieldwork in 2018, when MittMedia was known nationally for its efforts towards developing its proprietary news sites, mobile applications, as well as associated technological infrastructures such as proprietary audience analytics. MittMedia was also known for its explicit strategy of embracing audience analytics and metrics in news work. They embrace data to guide their epistemic practices, while rarely engaging in data journalism (e.g. analysing and visualizing data in the news).

At the time of our study, MittMedia was using a soft paywall (i.e. some news was free and some required payment). MittMedia was struggling financially, and following general evaluations and more specific tests in a few markets, they implemented a hard paywall on all their news sites and mobile applications. In 2019, MittMedia was purchased by the Bonnier Corporation, and thus became incorporated into the creation of the largest news publisher in Sweden. The media consortium uses centralized processes and economies of scale, helping to achieve efficiency in the organization amid their struggles to survive financially with the ongoing displacement of advertising going to the platform companies (most notably Facebook and Google). In this regard, they work similarly to many other large local/regional news publishing companies across the world. In 2020, International News Media Association (INMA) selected MittMedia as the global winner for the category ‘Best use of data analytics or research’. This is a testament to their substantial developments of proprietary audience analytics structures, put into use in everyday news work. Consequently, this case study represents a much-advanced data-driven news work culture in which strategic priorities for taking control over audience analytics and metrics by developing proprietary systems make an important feature.

One key aspect of differentiation has to do with their change in key metrics for their KPI (key performance indicator) in 2018, shifting from the quantitative and problematic metric of unique visitors (i.e. devices) to an internally developed metric of ‘completed reads’ that combines multiple metrics. Completed reads occur when ‘time spent’ scrolling down to the bottom of the article is reasonable. Completed reads are important to MittMedia, striving towards succeeding in publishing news materials that their readers actually read, as this affects whether they form digital news habits, which in turn is strongly correlated to whether they initiate/continue an online news subscription. Non-completed reads, when a user clicks on and opens an article, but then quickly moves on, are evaluated in terms of the news article (and the authoring journalist) failing to live up to the expectations of the reader.

Method and material

This study applies a qualitative multi-method approach. In 2018, three researchers spent approximately 3 weeks in the newsroom collecting data from different actors and activities. The researchers were granted access to the heart of the newsroom, enabling observations of, and interactions with, the live desk, desk editors, news reporters and editorial managers. Managers and journalists were very generous and transparent in giving access to the newsroom activities and in sharing information. The data collection combined observations and recordings of editorial meetings and everyday news work (inside and beyond the newsrooms), including access to internal communication via Slack channels. We conducted interviews with individual journalists, editors and managers, and collected documents on policies and internal guidelines, as well as specific test project by news desk. We followed closely the processing of a large amount of news. We used a method of situated interviewing, where the journalists were asked questions while doing news work.

In investigating the epistemological implications of audience metrics, we have focused on two organizational units inside SLP: (1) the news desk (a centrally placed team of online- and audience-oriented editors that monitors information intake as well as editing and promoting news materials); and (2) the reporter work (individual journalists on beats engaged in news production). The data material on the news desk specifically includes observations and interviews with five ‘live reporters’ (the in-house name for reporters responsible for intake, breaking news and 15-min reporting), five desk editors and the desk manager, their internal meetings as well as data from experimental ‘test weeks’ initiated to make assessments of their news work and their data-driven approach. For 14 days, a test was conducted in the newsroom where all news was published without any subsequent adjustments by the desk editors. After the 2-week test, the news organization carefully evaluated all the published news and noted failures, weaknesses and what should have been revised by the news editors in a normal situation. We were given access to the results of the desk editors’ reviews and notes. This gives us unique information about the standards applied in evaluating the quality of the news and what is considered to be important improvements. In total, the material includes the desk editors’ comments on 150 published news items.

The data on the reporter work includes recordings and notes from observations and interviews with nine news reporters, copies of source material and screenshots documenting their news in process, recordings and notes from meetings in which the news reports were discussed with the news manager and editors, and recordings from reporters’ source interviews. The data also comprises recordings from three coaching conversations between the news manager and individual reporters. The method for coaching was used as part of the overall 2018 implementation of the new KPI focussing on completed reads.

The analysis is theoretically guided and empirically grounded. The researchers have independently and systematically gone through the extensive empirical data, making notes about what is related to epistemic value and epistemic practices (see above). In the next step, the notes have been discussed and integrated into the most central aspects presented in the article. In this analysis, we have taken advantage of the fact that all three researchers have actively participated in the field work. Our explicit purpose has been to develop a focused analysis marked by high level of abstraction, in which we use selected data material and quotes for illustration.

Analysis

The analysis is structured by the two divisions of the news work: the news desk and the reporter work, each advancing findings for our two research questions.

The news desk

RQ 1: How are audience metrics balanced and reconciled in relation to other standards in the newsroom?

The data-driven news work culture at the news desk witnesses the coexistence of standards for defining the epistemic value of news, metrics and other journalistic standards decided from within. Metrics is handed the leading shirt guiding the work, yet occasionally other standards are invoked in justifying decisions.

The live reporters and editors are strongly committed to metrics in their daily news work, largely relying on audience preferences in deciding what is worth publishing. This is evident from our interviews as well as recorded discussions in the news room. In one of our interviews, a live reporter says ‘the audience is our boss’ and explains that audiences decide what they want to read. The reporter adds a rhetorical question: ‘What else would it be?’ At a meeting at the desk (with live reporters, editors and the desk manager) a similar understanding was expressed as follows: ‘It is the readers who decide what is readable and not’. ‘We should not teach the readers what they want’. ‘We must understand that it is the readers we work for’. The journalists describe this as an insight that has emerged after the news organization decided to strategically develop data-driven news work. At the meeting, they joked about the old-fashioned idea that the news publisher should claim a particular identity and decide on news in relation to this. However, although metrics are embraced as a way to learn about and satisfy the readers, the development of the news organization is also recognized as potentially controversial. The manager said ‘We have decided that we should work data-driven and then we have to take the discussion’.

The journalists’ references to audience metrics as a principal indicator of the value of news reflect what we interpret as two understandings of audiences. On the one hand, audiences are referred to as ‘customers’. The live reporters emphasize that their use of metrics is necessary for their survival. One of the live reporters says that they constantly have to ask themselves, ‘What sells?’ On the other hand, metrics is recognized as a reliable indicator of public interest – a system better at adapting the news to what people are interested in. Valuable news is news that has a commercial value and that people want to read and know about. These values tend to coincide when the journalists reflect on the news work and the benefits of a data-driven approach.

The internally developed metrics of completed reads is the main measure of valuable publishing, but there are situations in which the editors treat the news as valuable even though the metrics are poor. Journalists seem to rely on what we label values from within. This applies to investigations in which they have invested an extraordinary amount of reporter work as well as what they define and present as today’s top story. What constitutes a good investigative report is partly evaluated based on the audience data. However, as one of the editors says, they often keep such reports longer than the data justifies, because they ‘want their major investigations to be seen’. Investigative reports are prioritized on the site also when there is limited reading. In our interpretation, this is ultimately about presenting a valuable journalism that the newspaper claims to represent. A similar balancing is described when it comes to the day’s top story. As an editor states, they usually keep this story at the top of the site for quite some time, ‘although it obviously does not draw that much’. In addition, she continues: ‘It’s still like today’s news’.

In our interpretation, these examples indicate a claiming of epistemic values (the values of investigative reporting and the news media as an institution of today’s local news) not simply reducible to audience output. The importance of the particular identity of the news media is further indicated when editors downgrade news that still has high figures but is not considered reasonable to keep, as it is no longer news. One desk editor answers the question of whether or not she ever goes against what the KPI tells her: ‘Yeah, definitely. It can be, you know, sometimes a car accident goes well even though it was 13 h ago it happened, and that is just not reasonable! (Laughing) A big fire can go well for such a long time even though the fire was over a long time ago and there was no danger’. Thus, although metrics has become a prime standard in deciding what is valuable to inform about, there are situations in which metrics are temporarily overridden in favour of fundamental epistemic claims of investigative reporting, novelty and immediacy in news journalism.

RQ 2: How are different practices of research, presentation and packaging prioritized in a news organization marked as a data-driven news work culture?

The desk editors have a central role in a data-driven news organization. They have a particular responsibility for navigating and taking advantage of audience analytics and metrics in organizing the news on the site and sharpening the packaging of the news. It is all about optimizing the output of a news item including the ‘right ingredients’, as one news manager puts it, presenting it in an attractive way and offering it in a timely manner. When desk editors describe their work in our interviews, they emphasize their responsibilities to draw readers to their proprietary site and mobile application and to achieve a maximum of output. They also describe their work as more ‘selling’ compared to the work of the news reporters. By default setting, MittMedia algorithms and not human desk editors govern the visibility of articles published on their proprietary platforms. However, when on duty (day and evening time) the desk editors also choose to deactivate the algorithms for the prime spaces on their news site, and manually work with how specific news articles are being featured. In a situated interview following this manual work, a desk editor explains to us how she thinks when placing a news item on the site:

“I usually do some kind of judgment what I think in terms of news evaluation, what space it should have, and that is kind of grounded in how I conceive of what usually goes. And how does that correlate with that? Then maybe I test and put something here, to notice that doesn’t work at all, then maybe I replace it further down a bit.”

Evident in our observations, this manual work is highly marked by the information that the metrics offer, and can also function corrective to the manual work if the editor in fact was wrong concerning if many would read a certain piece. Our observations show how the desk editors feed in by editing articles produced by the reporters when they identify ways to improve attractiveness. The research behind the news, the truth seeking and fact checking seem to be mainly taken for granted as a task performed by the reporters. Based on their role in the processing of news, their expected skills and access to real-time data, the editors have a mandate to revise the news produced by the reporters, without checking with the reporters unless they find reasons to do so. They revise headings and lead paragraphs, change puff images, and add links and metadata if missing. In one of our interviews, an editor, who also has experience working as a reporter, talks about the approach to headlines in these different roles. She says, that in her role as an editor, she is prepared to go further in simplifying and highlighting the controversial. As a reporter, she is more concerned that the headline matches the content of the text. However, she also believes that these differences are increasingly blurred when the reporters also become ‘better at thinking about what sells’.

Timing is another crucial component of data-driven work, as part of the optimizing practices. Previous research has stressed how journalists prioritize being fast to fulfil imagined expectations from their readership, and also to be ahead of their competitors (Ekström et al., 2021). The findings from this study indicates another important approach to time, namely timing. The journalists systematically prioritize timing, making strategic decisions about when to publish news materials. When pushing forward news articles for publishing in their editorial content management system the journalists insert a date and time for when the news article should be released. With a calendar view interface, news editors get an overview of their news flows. Strategic decisions and adaptations to this schedule are made on a daily basis, catering to news consumption routines known through metrics, but also working towards always providing something new to their readers, yet not too much new information all at once as some news articles may then overshadow others. An example of this is when the news manager at a morning meeting brings up a job that did not produce the metrics expected: ‘The robbery at the grocery store: very good headline, good puff image, it should have drawn way more, but it was published during the game, right? (…) That could be an explanation’.

At the time of our study, a test was conducted in the newsroom. For 2 weeks, news was published without being modified/improved by the desk editors. The desk editors were instructed to review published news and note potential improvements with regard to how they normally would have edited the articles. The results of the test weeks, reveal what is treated as important when news quality is evaluated by the unit of the organization that has a central position in terms of the monitoring of audience metrics.

The desk editors’ reviews of the news mainly noted mistakes and possible improvements in the presentation and packaging of news. There are comments about: (1) missing meta data; (2) headlines being too vague, too long, not sufficiently pointed, containing words that do not ‘trigger a reader’, mentioning places when this is supposed to reduce reading, failing to include quotes and draw on emotions that affect reading; (3) images incorrectly edited, boring, insipid and not in harmony with the headline.

In this test, which corresponded to the routines of the desk editors, they hardly paid attention to improvements related to knowing about the events reported or possible problems related to truth claims. Exceptions included a few cases that stated that the headline promised something that was not covered in the news text and notes on information in the news that appeared to be somewhat incomplete. Otherwise, no notes were made related to truth seeking and truth telling. It may be that the desk editors could not see any improvements in these practices or they just assumed that the news reporters would professionally handle this. Ultimately, the test reflects a data-driven news culture where reporters largely work individually in producing their articles, and publish these themselves via the editorial Content Management System (CMS). The editors assessing their outputs do not pay much attention to epistemic claims and research, but instead focus on the packaging of news in developing the quality of news with the benefit of audience metrics.

The work of the news reporter

RQ 1: How are audience metrics balanced and reconciled in relation to other standards in the newsroom?

Tensions between internal standards and the embracement of audience metrics are clearly manifested in the work of the news reporter. The organization of the reporters’ work maintains a system of beat reporting that contributes to journalistic expertise in subject areas. The beat system reflects an understanding of news as public knowledge covering areas of societal relevance. Within the beats, the reporters, in our interpretation, have a certain autonomy to assess what is valued to inform about independently of audience metrics. They perform professional roles, such as holding people in power accountable, without justifying this based on assumed audience output. The professional autonomy is however restricted and subordinated to the standards of audience metrics. This is illustrated in this quote from a reporter explaining how she chooses what to write about: ‘It must have very high news value, you have to believe that this will be very clicked and very read ... you should know that it appeals to many’.

The precedence of metrics in deciding the value of news is most evident in the coaching of reporters, a method introduced to serve the development of a ‘customer-oriented’, ‘commercially driven’ and ‘data-driven’ news production (internal document with scripted guidelines). The method comprises regular discussions in relation to ongoing news reporting and the news manager’s scheduled coaching meetings with individual reporters. Here we focus on the latter. These include an initial longer conversation (about 40 min), followed up with shorter conversations (about 15 min). The relations between business, metrics and news production are clearly indicated in the following answers to the question ‘why coaching?’ presented in the internal guidelines: ‘We need more plus customers’ (i.e. subscribers); ‘We need customers who stay with us’; ‘We have to do what our audience wants’; ‘We always have to be ready to tweak’. More specifically, the method for coaching emphasizes the responsibility of individual reporters to improve their news work in accordance with audience metrics. The manager explains the coaching to one of the reporters as a learning process enabling them to more quickly change the content to be read and to ‘become more independent as a reporter, knowing what customers want’.

The main part of the coaching is a discussion on the news produced by the reporter over the recent period. Based on key metrics (completed reads, as well as conversion into paying subscribers), the high- and low-performing news articles are discussed in greater detail. This includes retrospective validation and prospective guidance. The conversations and evaluations give an insight into how audience metrics are reconciled in relation to other standards and understandings of news as valuable knowledge (RQ 1), but also how practices of research and presentation are recognized and prioritized (RQ 2, next section).

The reporter’s decisions in the news work are mainly evaluated in relation to metrics. This refers to the decision to report on a news event as well as to the workload invested in the news report. The news manager emphasizes the importance of calculating and considering work efforts in relation to outcomes (metrics) in each individual news report. Metrics are used to improve and ‘spend time on what really gives something’. They discuss whether additional interviews in a given news report could have created increased reading, thereby justifying such an effort. In their everyday work, reporters also consider the epistemic efforts, that is, what work journalists must invest to fulfil the truth claims (Ekström et al., 2021). However, in the coaching, truth seems mainly to be taken for granted when efforts are calculated in relation to output. In our interpretation, the standard of justification is the expected outcome in terms of reading and buying rather than a professional idea of the quality of news.

The coaching conversations show how metrics are prioritized but also reconciled in relation to other standards of justification. Here is an example. They discuss a news report with quite weak metrics about a company in a local community. The manager asks the reporter to comment on the outcome. He says that he is still happy with the news because he expected ‘that this will not give anything’. He considered not doing anything about it but decided to do ‘a short thing’, and he explained: ‘It was more so that we should have certain things included’. The news manager agrees: ‘[W]e can do things that we know in advance ... that this may not go well but we do it because it is included in our reporting that we keep track of them’. Obviously, they invoke another standard of justification. The reporter justifies some news as a ‘duty thing’. In our interpretation, they maintain an idea of a news media representing a public value not necessarily reflected in metrics, although the latter is prioritized. The news manager says that it may be OK with news not expected to give high numbers, but only occasionally, and if they are simple and quick to produce. In this conversation, the duty things are not discussed further. They shift to the main topic of the coaching: how the news can be refined to produce better outcomes.

RQ 2: How are different practices of research, presentation and packaging prioritized in a news organization marked as a data-driven news work culture?

Research is central to the everyday work of the news reporters. Their work follows traditional routines of daily news reporting. They use their networks and sources to stay informed within their areas of responsibility. Information is collected and evaluated and interviews are conducted. They value the credibility of sources and check facts. These epistemic practices, and the reporters’ professional competences in truth seeking and truth telling, seem to be very much taken for granted when improvements in the news production are discussed in the newsroom.

The coaching of news reporters focuses mainly on presentation and what makes the news attractive. Headlines are treated as most important to maximize the output. Improvements in lexical choices, syntax and rhythm are discussed. Avoiding disclosing the where and who in news headlines is a strategy designed to create curiosity. In some cases, the location is supposed to be what attracts the audience in the first place. Moreover, they pay attention to good and bad puff images and reflect on various aspects of a text that may explain why many have not read the entire article. The development of audience analytics in the news organization, to include completed reads, is reflected in these evaluations. It is not only about what gets the reader interested, but also what makes them inclined to spend time reading the entire text. The news manager explains completed reads indicates a good reading experience. By scrolling down the reader gets suggestions for reading and can click on the next article, and it increases exposure to advertising.

Reporters consider the efforts invested in the research phase in relation to the norms reinforced in the data-driven news culture. For example, a reporter is considering finding out more facts for a news report through another interview, but tells us that she chooses not to do so both for reasons of time and because more facts risk making the article too boring. Of course, such considerations are central to news production in general. However, these are also the decisions that reporters are expected to improve with the help of metrics. The justification of input (time in research) in relation to output (audience reading and paying) emphasized in the coaching is reflected in the reporters’ work.

The news reporters we have studied are well aware of the expectations that the news must be designed to attract readers. They prioritize the packaging of news to increase the reading of their articles. Here is an example. A reporter is working on a news item about initiatives to reduce electricity consumption in a municipality. He has conducted interviews and collected a lot of facts and is working on the news angle and the headline. He tells the researcher that this is the most difficult task: finding out what to highlight to attract the readers. The most important thing, he says, is that the headline and the text arouse readers’ interest. He shows a draft of the headline and points out that this needs to be improved to be ‘more heated’. Again, the refinement of headlines is an everyday practice in traditional news production. However, in this data-driven news culture the reporters’ responsibilities are emphasized and internalized through the regular evaluations of the performance of individual reporters.

In the processing of the news reports, the journalists carefully balance attractiveness and truth telling. Boring facts are excluded and quotes in headlines are tweaked. The norms of truth telling set limits on the polishing of news to increase output in reading. The news reporters are careful about the facts. No one who knows the facts should be able to blame the reporter for being wrong. Some of the reporters discuss their draft news with the news manager. In one example, the manager argues that the headline is too bureaucratic and should be more exciting. The manager suggests simplifications and tweaks. The reporter says that the suggested wording is not sufficiently accurate. They continue to discuss possible refinements and play with the limits of what they can say without deviating from their perception of the truth. Another example of this balancing act is when we were conducting a situated interview with a reporter writing an article after an interview with the police about a robbery in town. She is struggling with getting the interview correctly represented in the text as she wants to include content that she knows will catch reader interest:

“You know she said that ‘it is all crazy’, but how can I include that? (…) That is a quote I would like to have, I feel now. But how shall I formulate that and be correct in how I posed the question. But still get to include the answer?”

Our final example illustrates how attraction and truth telling are balanced in a news work relying on the practices of packaging as being most important. The news is about criticism of the conditions in some schools in the city. The reporter has completed the draft article. It is handed over to the news manager who now discusses how to improve the news with an editor for about half an hour. They focus on a particular wording in the suggested headline, describing the conditions in schools as a ‘mess’. They would like something more alarming and try alternative wordings. The alternatives they think of, however, suggest there are fights in the schools and this is not what the article is about. They start searching for synonyms online. They can’t find an alternative and instead suggest an intensification of the headline indicating the high frequency of the ‘mess’. Then they move to the puff image and similar challenges in terms of representation emerge. They cannot find archive images that indicate an alarming situation without being misleading. The editor instead creates a montage of several images.

Conclusion

The aim of this article is to simultaneously contribute to two areas in (digital) journalism studies. First, it contributes to research into quantification of journalism through analytics and metrics (Carlson, 2018; Tandoc, 2019; Zamith, 2018), one of the more well-researched areas in the field in recent years (Steensen and Westlund, 2021). Second, the study contributes to emerging research into the epistemologies of digital journalism (Ekström and Westlund, 2019b; Ekström et al., 2020; Matheson and Wahl-Jorgensen, 2020). The article offers a multi-method case study focussing on epistemological implications associated with an ongoing transformation into a data-driven news work culture. We investigated two key aspects of the epistemology of news journalism: the epistemic value and the epistemic practices of journalism.

The overall finding concerns two critical implications of the substantial appropriation and embracement of audience analytics and metrics, in the context of commercially driven news production: (1) the use of metrics as a superior standard in deciding on the epistemic value of news; (2) a dominant focus on practices of packaging in the optimization of news, partly at the expense of research and truth seeking.

The study shows how a data-driven approach, and the metrics used, are justified in the newsroom as a powerful way to improve journalism based on the principal idea that what audiences want ultimately decides the epistemic (and the commercial) value of the news. This principal idea of what should be valued – and the interrelated downgrading of other standards – is expressed in overall strategies, guidelines and discussions in the newsroom, and put into practice by developing a sophisticated metric and use as KPI, followed up in coaching, evaluations and rewarding of the performance of individual journalists. Amid what is at stake for journalism, we want to stress that the shift to completed reads, as the KPI guiding this data-driven news culture, prioritizes valuable journalism rather than simply subsuming into market logics. Importantly, in the everyday news production, metrics are reconciled in relation to independent standards in journalism, related to the claims of news journalism to provide relevant and verified public knowledge about current events. Hence, as regards the epistemic value of news, the question is not whether journalism should provide audiences with what they need or what they want (Ferrer-Conill and Tandoc, 2018), but rather how various standards are balanced in the practice of data-driven news work. What is ultimately at stake is the autonomy of news journalists in assessing what is important to publish relatively independent of expected outcomes in metrics.

The study further shows how the embracement of metrics radicalizes the focus on presentation and packaging in the everyday news production. Journalists primarily use metrics to guide their optimization of news materials, involving them carefully in selecting pictures, altering headlines and focussing on their timing when it comes to publishing news materials, etc. Desk editors’ competence and efforts in optimizing the news are highly valued in the organization, and reporters are expected to develop professional practices in learning from metrics in their everyday news production. Efforts in research and truth seeking are more seldom explicitly valued, and reporters are expected to carefully consider such efforts in relation to expected outcomes in metrics. It is not that accuracy and truth claims are ignored by the reporters, but they are mainly taken for granted in evaluations and discussions on how to improve the news work. In the reporters’ work, a professional autonomy is maintained in the selection of news topics and traditional practices of research and fact checking. This autonomy, however, seems to decrease because of stricter requirements for measurable output.

This case study of a data-driven news culture reveals important long-term implications for the epistemic authority of news journalism, and the claims of providing truthful information on actual events. The study reaffirms research showing that the development of analytics promises important resources for news journalism to learn about audiences’ behaviours and preferences. However, this organization also illustrates how the operationalization and implementation of metrics tend to reduce the value of the news to what metrics indicates that the audiences value. Research indicates that metrics, even in the advanced forms, give a limited and even misleading picture of audience engagement and what the audience actually values in the news (Costera Meijer, 2012; Steensen et al., 2020). Ultimately, while data-driven news work cultures can learn much from their audiences, and seek to optimize their news production accordingly, there are certainly limitations.

We call for future research exploring more in-depth why and how news workers override metrics and instead are guided by journalistic values and newsworthiness. Moreover, for a news journalism that claims to tell the truth, truth seeking must also be valued in the practices of news reporting. We call for future research into the development and/or appropriation of analytics in news work, focussing on the complexities involved in analysing how tensions in the epistemology of news journalism are reconciled. We argue that future research into how publisher’s prioritize reader revenue and develop proprietary analytics infrastructures has become increasingly important amid platforms having displaced advertising revenues, and also because of drastic changes as to how third-party platform and tech companies (e.g. Apple and Google) may gather, analyse and share data with partners. We call for research into professional authority and epistemic practices associated with emerging data-driven news cultures, and the future ongoing developments of proprietary analytics and more customized metrics.