Abstract
In a digital media environment where content distribution is shaped by technology companies’ algorithms and user behaviors, news organizations try to post content that can prompt user engagement in forms such as comments, shares, and likes or reactions. This study employs a content analysis of 1,600 messages and analyses of engagement metrics for 157,962 messages to examine to what extent and how Facebook messages of US and Israeli news organizations differ in the engagement modes they generate: commenting versus sharing versus liking/reacting. Drawing on the participation paradigm in audience research, news value theory, and literature on engagement enhancers, the study shows that certain content characteristics are associated with each of the examined engagement modes in more than one country while other content characteristics are associated with particular modes, but not with all of them. It offers a nuanced understanding of user interaction with news-related content and helps think about content units as more engaging or less engaging than others, or as engaging in different ways.
Keywords
As news organizations try to reach and attract audiences in the digital media environment, they post content on multiple platforms. These include not only platforms they own, but also platforms non-proprietary to the news media, such as Facebook (Westlund & Ekström, 2018), which draw a substantial number of people (e.g., Newman et al., 2021). On the latter platforms, news organizations may choose what to post and how to construct it, but the content distribution is shaped by both technology companies’ algorithms and user behaviors. On Facebook, for example, content generating greater user engagement in the forms of comments, shares, and likes or other reactions seems to gain greater visibility in the News Feed and may reach more people (e.g., Bromwich & Haag, 2018). Thus, when deciding what to post, news organizations also try to assess what may generate user engagement (Tenenboim, 2022), broadly understood here as digital ways of interacting with content.
This study focuses on these ways of interaction and the content characteristics that evoke them in news organizations’ messages. Industry discourse about user engagement has tended to lump together different ways of interaction (e.g., McLachlan & Newberry, 2021), creating an impression that content can simply be more engaging or less engaging. However, as argued in this article, commenting, sharing, and liking or reacting are distinct in the degree of expression they allow users. As such, these ways of interaction—or modes of engagement—may be associated with different content characteristics. Research focusing on news has typically examined relationships between certain content characteristics and a particular engagement mode, offering important insights into shareability (e.g., Valenzuela et al., 2017) and commenting enhancing (e.g., Weber, 2014). Some studies have also shown that certain modes of engagement on news sites were linked to different types of stories (Boczkowski & Mitchelstein, 2013; Tenenboim & Cohen, 2015). Yet, more attention is needed to possible relationships between different types of content characteristics and different modes of engagement beyond news sites to gain a nuanced understanding of user engagement and its enhancers. Such understanding would allow a better assessment of what drives distribution and discussion of content, as well as reaction to it. It may also play a role in decisions on how to construct and present content. This study aims at gaining nuanced understanding by comparatively examining which content characteristics (e.g., topics, news values, emotional triggers, evaluative aspects, and structural features) of news-related messages on Facebook are associated with commenting, which with sharing, and which with liking/reacting. News-related messages are understood here as posts that contain news or address an issue in the news. They may present an item such as a news report, analysis, opinion piece, or magazine feature.
The study investigates to what extent and how news organizations’ messages on Facebook differ in the engagement modes they generate. It employs a content analysis of 1,600 messages and an analysis of engagement metrics for 157,962 messages posted by 16 news organizations in the United States and Israel (eight in each country). I focus on commenting, sharing, and liking/reacting not only because their amount plays a role in determining what digital content people will see, but also because they can be seen as manifestations of a newer style of citizenship that is rooted in self-actualization through social expression (Bennett et al., 2011). Differently put, commenting, sharing, and liking/reacting can be seen as civic acts (even if they are used for dark purposes; see Quandt, 2018). In investigating their relationships with content characteristics, I draw on the participation paradigm in audience research that asks how people engage with, accede to or contest media (Livingstone, 2013), news value theory that looks at what makes content newsworthy (e.g., Galtung & Ruge, 1965; Harcup & O’Neill, 2017), and literature on engagement enhancers (e.g., Amit-Danhi & Shifman, 2022; Berger & Milkman, 2012).
Based on an integration of insights from these bodies of literature and a comprehensive empirical investigation, this study makes a twofold theoretical contribution: suggesting that content units can be understood not only as more engaging or less engaging than others, but also as engaging in different ways (e.g., prompting shares but not comments); and revealing what makes news-related messages engaging in these ways. The study thereby advances the understanding of user behaviors and can inform news organizations in their attempts to attract audiences in the digital media environment.
Engagement and Its Importance
Amid financial challenges of the news industry (Chyi & Tenenboim, 2017) in a media environment dominated by technology companies (Bell & Owen, 2017), news organizations have put greater emphasis on ways to engage audiences. Engagement has become a news industry buzzword and is also used in scholarly work, though there is no consensus on its meaning. Engagement in the context of media is a complex and multidimensional concept (e.g., Steensen et al., 2020). It has also appeared as part of different combinations of words, including audience engagement (Broersma, 2019), user engagement (Oh et al., 2018), and news engagement (Ha et al., 2018).
Focusing on audience members or users, it is possible to understand engagement as “the cognitive, emotional, or affective experiences that users have with media content or brands” (Broersma, 2019, p. 1). Engagement can also be understood as a continuum (Ksiazek et al., 2016; Oh et al., 2018). For example, Ksiazek et al. (2016) proposed a continuum ranging from mere exposure to interactivity, where “more (quantity) and better (quality) ways to interact with content and with other users indicate deeper engagement” (p. 505).
Centering on news-related content, news engagement can be seen as “the involvement in news content for either personal or social purposes, which can be indicated by the effort made in obtaining and utilizing the news content among the audience” (Ha et al., 2018, p. 720). The present study draws on this definition while investigating user involvement particularly at the distribution and interpretation stages of the news production process (Domingo et al., 2008), the two final stages at which journalists’ control seems to be limited compared with the stages of deciding what news is and how to construct it. Engagement at these stages is perceived here as digital ways of interacting with news-related content—for example, sharing, commenting, and liking/reacting.
Such engagement can be important for both business and democratic reasons. On the business side, engagement may help increase audience reach and brand loyalty and function as a form of “free labor” (Jönsson & Örnebring, 2011, p. 129) in content distribution. On the democratic side, user engagement offers potential for public deliberation and civic empowerment (Manosevitch & Tenenboim, 2017), although it can have dark aspects, such as spreading disinformation and promoting hate speech (e.g., Quandt, 2018).
Yet, commenting, sharing, and liking/reacting can be seen as manifestations of actualizing citizenship (Bennett, 2008) or engaged citizenship (Dalton, 2016)—styles of citizenship that are rooted in self-actualization through social expression (Bennett et al., 2011). These styles are based on a diminished sense of obligation to the government. They differ from dutiful citizenship (Bennett, 2008) or duty-based citizenship (Dalton, 2016), which draw on the notion that citizenship is a matter of duty and obligation and manifest in voting in elections, participating in government-based activities, and belonging to a political party, among other things. Actualizing citizens have a high sense of individual purpose, see voting as less meaningful than more personally defined acts—for example, voluntarism and consumerism, and favor loose networks of community action that are “often established or sustained through friendships and peer relations and thin social ties maintained by interactive information technologies” (Bennett, 2008, p. 14). In the digital environment, actualizing citizens may produce content and share it over peer networks that tie personal identity to engagement (Bennett et al., 2011). For instance, engagement may be through involvement in online discussions, including political discussions (Dalton, 2016). Citizens do not have to be either dutiful or actualizing/engaged but can combine elements from both the older and newer models.
Based on these models of citizenship, forms of user engagement in the digital media environment, including commenting, sharing, and liking/reacting, can be regarded as acts of civic engagement. Although the three forms may involve little effort or commitment and may be perceived as doing little to meaningfully promote a cause (see Morozov, 2009, about “slacktivism” and Christensen, 2011 for examination of the “slacktivism” criticism), they are seen in this study as civic acts that can matter. The subsection below explains how.
Commenting, Sharing, and Liking
Online comment sections provide a space for individuals to have their say, learn the positions of others and interact with one another (e.g., Tenenboim & Cohen, 2015). According to a national survey in the United States (Stroud et al., 2016), 55% of respondents (aged 18 or older) left an online comment, and 77.9% read comments at some point. While some news organizations disabled comments for their content due to hate speech, incivility, and misinformation (Nelson et al., 2021), commenting is a widely offered avenue for engagement (Stroud et al., 2016). Audience members can comment on various issues in the news, though they are particularly likely to comment on political and social issues (Stroud et al., 2016; Tenenboim & Cohen, 2015).
Citizen discussions about politics and public affairs have long been perceived as vital for a healthy democracy (e.g., Barber, 1984; Dewey, 1939), as they can help gain political knowledge (Eveland & Thomson, 2006) and better understand important issues. Research has also suggested that political expression on social media may drive or at least be linked to offline engagement in democratic life (Boulianne, 2019; Shah, 2016). For example, those who politically express themselves may be more likely to participate in street marches (see Boulianne, 2019). Yet, exposure to uncivil comments may have negative effects, such as an increase in readers’ perception that the commented-on content is biased (Anderson et al., 2018; see Masullo et al., 2021, for additional effects).
Sharing and liking/reacting are other important forms of user engagement. Social media users can share with their friends and followers content pertaining to issues of public concern and indicate that they like a content unit. Focusing on social media news users in the United States, Pew’s survey found that the proportion of users who often or sometimes shared/reposted and “liked” news stories was 49% (sharing/reposting) and 58% (liking), respectively (Mitchell et al., 2016).
The act of sharing allows people to play a role in determining what will reach other people and what the latter will know. When people “share” a piece of content, they increase the chance that their online friends and followers will be exposed to this piece. Indeed, according to research, sharing has become central to the way people experience the news (e.g., Hermida et al., 2012). As with commenting, sharing can also be a form of political expression that drives offline engagement (e.g., Gil de Zúñiga et al., 2014). On the negative side, sharing allows the spread of falsehoods, and they may be more shareable than the truth (e.g., Vosoughi et al., 2018).
User liking/reacting has various purposes, such as indicating receipt of a message, showing interest, or expressing support (see Sumner et al., 2018 for elaboration on the Like button as a multifunctional tool). It may also be a habitual or automatic response to attention-grabbing content (Alhabash et al., 2019). Yet, user liking/reacting can be seen as a civic act in several ways. First, by “liking” (or recommending) a content unit pertaining to a public matter or figure, users can explicitly express their support, showing to others that they see the message or person in a favorable way. Facebook users can also use other emojis, such as a crying face, to express how they feel about messages (Zerback & Wirz, 2021). Second, by pressing or clicking “Like” users may play a role in determining what will command public attention. As Dvir-Gvirsman (2019) has shown, popularity cues such as “likes” can influence people’s attention allocation and news selection. Content units that are more “liked” may generate more attention. Furthermore, by rewarding certain types of messages through “likes,” users may encourage others to produce such messages (see Muddiman & Stroud’s findings about user recommendations, 2017).
Although comments, shares, and likes/reactions are commonly used words when referring to engagement modes, the functions they represent have different names or appear differently on different platforms. For instance, on Twitter, comments are replies, and shares are retweets. Yet, this study focuses on Facebook, a dominant and highly popular platform worldwide (e.g., Newman et al., 2021). On Facebook, users are offered to comment, share, and like/react.
Linking Content Characteristics to Engagement Modes
In view of the importance and the distinct nature of commenting, sharing, and liking, this study seeks to identify to what extent and how news-related messages differ in the engagement modes they generate. The investigation is conducted within the participation paradigm in audience research that asks what participation modes are afforded to people in the media environment and how people “engage with, accede to, negotiate or contest” media (Livingstone, 2013, p. 6).
To understand people’s engagement with media, this study performs an analysis of engagement metrics and an analysis of content with which people engage. The analyses build on Tenenboim and Cohen (2015), who showed how news stories differed in engagement modes they generated on a news site, including clicking that is typically not visible to other users, and commenting that may be seen by others. Commenting, sharing, and liking/reacting can all be visible, but they are distinct in the degree of expression they allow users. Comments are open-ended in that they allow users to use their own words (Manosevitch, 2011). Likes/reactions are closed-ended in that they are selected from existing options (pressing or clicking a thumbs-up sign or another emoji) and do not include users’ own words. Shares allow users to add their own expression to the reposted content, though content can merely be reposted without additions. Considering the distinct nature of commenting, sharing, and liking/reacting, it is plausible that different news-related messages engage users in different ways. To investigate this on the popular platform Facebook, I first examine the following based on Tenenboim and Cohen’s (2015) approach:
RQ1. To what extent do news organizations’ messages on Facebook differ in the modes of engagement they generate?
RQ1a. To what extent do different engagement metrics correlate?
RQ1b. What is the degree of overlap among the most commented-on, shared, and liked/reacted messages?
Subsequently, I examine which content characteristics of messages are associated with commenting, which with sharing, and which with liking/reacting. This is an important investigation because it can help understand what drives different acts of engagement.
Drawing on news values theory (e.g., Galtung & Ruge, 1965; Harcup & O’Neill, 2017) and literature on engagement enhancers (e.g., Amit-Danhi & Shifman, 2022; Berger & Milkman, 2012), I identified five types of content-related characteristics that may be associated with engagement modes: news topics (e.g., politics and health), news values or factors (e.g., a prominent figure and a surprise), evaluative aspects (e.g., an overall tone and expressing criticism), triggers of emotions (e.g., evoking anger or sadness), and structural features (e.g., including a video, or writing messages of different lengths). Research has typically focused on relationships between some of these characteristics and a particular engagement mode (e.g., Karnowski et al., 2021; Valenzuela et al., 2017; Weber, 2014), but more attention is needed to possible relationships between different types of content-related characteristics and different engagement modes to gain a nuanced understanding of user engagement and its enhancers. In a media environment where different engagement modes are offered, used, and possibly contribute to the visibility of content, such understanding is important for a better assessment of what makes content spreadable, talked about, and reactive. It may also inform decisions on content production.
Focusing on the above-mentioned content-related characteristics, this study examines the following through an investigation of Facebook messages in two countries—the United States and Israel—to identify which patterns may transcend national boundaries:
RQ2. How do news organizations’ messages on Facebook differ in the modes of engagement they generate?
RQ2a. Which content characteristics of messages are associated with different modes of engagement?
RQ2b. Do the content characteristics associated with different modes of engagement differ across cultural/national contexts?
Although the literature on relationships between some characteristics (e.g., an overall tone and a presence of a video) and different engagement modes is limited or inconsistent for formulating hypotheses, the study offers hypotheses regarding content characteristics that have been theorized or yielded more consistent findings.
In making news judgments, journalists prioritize content about certain topics, events, or people (e.g., Galtung & Ruge, 1965). For example, journalists may see public affairs, including politics, economics, and international topics, as particularly important and may pay more attention to such affairs than audience members do (Boczkowski & Mitchelstein, 2013). Journalists also tend to prioritize content that contains certain factors, such as a conflict or surprising element. According to news value theory, stories generally satisfy one or more of certain requirements if they are to be selected for publication. These requirements can be regarded as news factors or news values (Galtung & Ruge, 1965; Harcup & O’Neill, 2017).
Similarly, audience members may be more likely to consume or interact with news items that focus on certain topics (e.g., Boczkowski & Mitchelstein, 2013; Tenenboim & Cohen, 2015) or contain particular news factors (e.g., Karnowski et al., 2021; Trilling et al., 2017; Ziegele et al., 2018). Focusing on news topics, although stories on public affairs are not necessarily highly clicked-on compared with other stories, for example, entertainment, sports, crime, and weather, the former may be highly commented-on (Boczkowski & Mitchelstein, 2013). Tenenboim and Cohen (2015) suggested that commenting plays a role in constructing social/group identity through discussions of political issues. Politics in democratic contexts involves disagreements and rivalries, and news users may post comments to show where they stand or to try to convince others that they are right (Tenenboim & Cohen, 2015). An empirical investigation found that news-related messages focusing on politics/government were linked to a high volume of comments posted on Facebook pages of Swedish newspapers (Larsson, 2018), yet further investigation is needed in additional national/cultural and organizational contexts. However, other topics may be associated with sharing. As content with a high practical utility is more likely to be transmitted by news users (e.g., via email, Berger & Milkman, 2012; Boczkowski & Mitchelstein, 2013), it is plausible that news-related content about health will be highly shared on social media. Larsson’s study of Swedish newspapers’ Facebook pages (2018) indeed found that health issues were linked to a high volume of shares, but further investigation is needed in other contexts. For news topics (H1), I hypothesize:
H1a. The news topic politics/government is positively associated with commenting.
H1b. The news topic health is positively associated with sharing.
In constructing social/group identity, news users can take advantage of commenting to take sides and express views. As such, commenting can be a particularly useful engagement mode when a news story involves a controversy or conflict (Boczkowski & Mitchelstein, 2013; Tenenboim & Cohen, 2015). An experiment also linked a high level of controversy in an article to commenting on it (Ziegele et al., 2018). Another key news factor in understanding to what content users are drawn is a surprising element in a story. Such an element may pique users’ interest as it defies previous expectations (Tenenboim & Cohen, 2015). It stands to reason that when consuming a surprising content, at least some users would share the surprise with others. Indeed, highly emailed news items were identified as having a bizarre character (Boczkowski & Mitchelstein, 2013) or evoking surprise (Berger & Milkman, 2012). In addition, a study found that students were more likely to share unexpected news on Twitter (Rudat et al., 2014). Yet, for both surprise and conflict, further investigation of engagement on social media is needed. For news factors (H2), this study proposes:
H2a. The news factor conflict is positively associated with commenting.
H2b. The news factor surprise is positively associated with sharing.
Despite the importance of news topics and factors, they may not be enough to entice a high volume of user engagement. In the current media environment, the boundaries between professional journalists and everyday citizens as well as between public and private are more blurred, giving rise to a more subjective or emotional expression (e.g., Papacharissi & Oliveira, 2012; Wahl-Jorgensen, 2016). Although subjectivity and emotionality are not new in journalism, they are particularly prominent and may be a vital force in enabling newer forms of engagement on social media (Wahl-Jorgensen, 2016).
Therefore, this study examines relationships between triggers of emotion and engagement modes. These triggers can be understood as content elements, such as a visual or combination of words that may evoke an emotion. A key emotion that can be evoked is anger, characterized by high arousal, that is, a high state of mobilization. This state may increase action-related behaviors, such as the transmission of content (Berger & Milkman, 2012). Furthermore, based on the assumption that individuals’ emotional responses are the result of appraising how significant a situation, event, or object is to them, social media users may respond through an emoji (for instance, an angry face) to content that triggers corresponding emotional appraisals (e.g., anger appraisals) (Zerback & Wirz, 2021). Berger and Milkman (2012) found that anger-evoking news stories were more likely to be emailed, and studies of political actors’ Facebook pages linked anger-evoking content to sharing (e.g., Amit-Danhi & Shifman, 2022) and reacting (e.g., Zerback & Wirz, 2021). Extending their logic to news organizations’ content on Facebook, this study hypothesizes:
H3. A trigger of anger is positively associated with sharing and reacting.
Berger and Milkman (2012) also suggested that content evoking sadness, a low-arousal emotion, was less likely to be emailed. Yet, Zerback and Wirz (2021) found a positive association between sadness-eliciting political posts and the frequency of sharing them on Facebook. A different study found that sad imagery did not influence message processing, despite the hypothesis that sad imagery would increase message engagement via depth of processing (Schultz et al., 2018). The relationships between a trigger of sadness and engagement modes, as well as between a trigger of anxiety (Berger & Milkman, 2012) and engagement modes, are examined as part of RQ2. Evaluative aspects, such as the tone of a post, are also investigated. Positively valenced news may contribute to engagement (Berger & Milkman, 2012; Trilling et al., 2017), although user comments with features of discussion that convey a disrespectful tone may be rewarded with upvotes under certain conditions (Shmargad et al., 2022). However, the tone of political actors’ posts was not associated with liking in the Hungarian context (Bene, 2017). Further investigation of the possible role of tone in eliciting different engagement modes is needed, particularly in the context of news-related content on social media.
Taken together, examining RQ1, RQ2, and H1–H3 allows assessing if different posts by news organizations engage users in different ways.
Method
This study is based on a content analysis of news-related posts on Facebook, as well as on analyses of engagement metrics. Posts by 16 news organizations in two countries—the United States and Israel—were examined to identify possible patterns across national boundaries. Despite differences between the United States and Israel, these countries share commonalities relevant to this study. First, Facebook is widely used in both countries (Israeli Internet Association, 2022; Pew, 2021). Second, both have democratic features (e.g., electoral process and pluralism; Economist Intelligence Unit, 2020), which can be linked to people’s ability to engage with news in the digital media environment (Tenenboim & Kligler-Vilenchik, 2020). Third, the media systems in the United States and Israel are similar in their market orientation (e.g., Neiger and Tenenboim-Weinblatt, 2016), which can manifest in news publishers’ competition for user attention (see Stroud, 2017 about attention as a valuable resource) and attempts to engage audiences. At the same time, the two countries differ in ways that may manifest in the production of content and engagement with it. For example, the Israeli culture is marked by “straight talk” that values simplicity of expression and explicitness of purpose (Katriel, 1986), unlike the US culture. In addition, focusing on journalistic cultures, Israeli journalists are generally more interventionist in their role perception, or more likely to perceive the pursuit of missions and promotion of certain values as important (e.g., Hanitzsch et al., 2019). This perception may be reflected in news publishers’ expression style on social media (Tenenboim, 2022).
For each country, the study focuses on eight for-profit organizations that are prominent in different markets in terms of circulation, traffic, or perceived status of the outlets they produce—newspapers or digital-native outlets in both of which written texts play a major role in storytelling. The selected outlets have Facebook pages and divergent political leanings. The US outlets include The New York Times, USA TODAY, Los Angeles Times, The Dallas Morning News, South Florida Sun-Sentinel, Vox, Slate, and The Daily Caller. The first five outlets are major daily newspapers that were among the 100 most circulated newspapers in the United States (Editor & Publisher, 2018). The last three outlets are prominent publishers of news-related content in the US digital media landscape and were estimated to draw millions of visitors per month (Website IQ, 2020). The Israeli outlets include Haaretz, Yedioth Ahronoth, Maariv, Israel Hayom, Makor Rishon, ynet. Walla, and Srugim. The first five outlets are among the major daily newspapers in Israel (Verthaim, 2018), and the last three outlets are digital-native news outlets. Walla and ynet are popular news websites in Israel, 1 and Srugim gained popularity mainly among religious or conservative audience (see Similarweb, 2021; Website IQ, 2020 for metrics and ranking).
Data Collection and Sampling
Data about posts that appeared on the main Facebook page of each news organization were collected, focusing on posts from 1 July 2018 to 30 June 2019—a 1-year period. A post is understood in this study as a content unit that can contain a written text, audio-visual elements (such as a video or photo), and/or a link. Data were collected through CrowdTangle, a social media monitoring tool owned and operated by Facebook (CrowdTangle Team, 2020). The data included the posts’ content, links to the posts, time of posting, the number of likes the Facebook page had at the time of posting, and engagement metrics (the numbers of comments, shares, likes, and other reactions) for each post that was published in the examined period. 2 The data also indicated if a post was sponsored, that is, if the content was an ad in the form of a post that someone paid to publish on a news organization’s page. Sponsored posts were excluded from the data.
In total, data about 157,962 posts were left—115,207 posts from eight US organizations and 42,755 posts from eight Israeli organizations. For US news organizations, the number of posts in a given period (the second half of 2018 or the first half of 2019) ranged from 4,668 (for The Dallas Morning News in 2019) to 11,093 (for The New York Times in 2018). For Israeli news organizations, the number of posts ranged from 312 (for Srugim in 2018) to 4,961 (for Maariv in 2018).
These data were used to examine the extent to which news organizations’ posts differ in the modes of engagement they generate (RQ1). To examine associations between content characteristics of posts and modes of engagement (RQ2 and H1–H3), a content analysis was carried out on a sample of 1,600 posts. To sample these posts, two steps were taken. First, a constructed week sampling was used: “Sample dates are stratified by day of the week to account for systematic variation due to day of week” (Riffe et al., 1993, p. 133). Such variation can occur as routines of some news organizations and newsmakers are related to the day of the week (e.g., Jang & Pasek, 2015). 3 Based on the approach that for online news content at least two constructed weeks are needed in a 6-month period (Hester & Dougall, 2007), 4 constructed weeks were created—2 for the past 6 months of 2018 and 2 for the first 6 months of 2019. The total number of posts published on the days of the constructed weeks was 12,191: 8,969 posts by the US organizations and 3,222 posts by the Israeli organizations. Second, systematic random sampling was used to select 800 US posts and 800 Israeli posts. The systematic random sampling was conducted across all the news outlets within a country. For each country, different intervals and random starting points were used.
Coding and Variables
The unit of analysis was a Facebook post, and coders were instructed to address only the post’s content as described above and not the comments attached to it. The posts were coded holistically, considering the different parts that were visible (e.g., a text presenting a story, and the image and title of the embedded story as they would appear in a user’s Facebook feed). 4 They were coded by the author and three trained native speakers of English or Hebrew based on a detailed codebook. Inter-coder reliability was examined for 300 of the posts (18.75%) using Krippendorff’s alpha. The coefficient alpha for the variables included in the analysis ranged from .708 to 1.000. The coefficient was lower than .700 for three variables (frequency, awe, and humor), which, as a result, were disqualified and excluded from the analysis.
The content characteristics included in the study are presented in Table 1, which also show Krippendorff’s alpha coefficients for measuring inter-coder reliability (see Appendix A for Cohen’s kappa coefficients as well). Detailed frequencies of the variables appear in Appendix B.
Examined Content Characteristics.
The statistical procedures for analyses are addressed in the “Findings” section.
Findings
To find the extent to which news organizations’ messages on Facebook differ in the engagement modes they generate (RQ1), the following were examined for each news organization: Spearman correlations between engagement metrics (RQ1a) and percentages of overlap between highly engaging messages (RQ1b).
The examined engagement metrics were positively correlated. The correlations were relatively strong, though not necessarily extremely strong. Specifically, the correlations between comments and shares and between comments and likes were lower than .800. They ranged from .376 to .797, although 38 of 64 coefficients were above .700 (For a given pair of engagement metrics, such as that of comments and shares, 32 coefficients were obtained: 16 news organizations × two time periods for each organization). The correlations between shares and likes ranged from .533 to .837, with 18 of 32 coefficients lower than .800. The correlations between comments and reactions (likes + other emojis) and between shares and reactions were relatively high. They ranged from .486 to .883, with 35 of 64 coefficients lower than .800. The correlations for each news organization appear in Appendix C.
Stable patterns were found among highly engaging posts, as can be seen in Figure 1. The percentages indicate that although groups of highly engaging posts can contain the same posts (e.g., posts can be both highly commented-on and highly shared), these groups also differ from one another to a notable degree.
Percentage of overlap between highly engaging posts of US and Israeli news organizations.
For example, the degree of overlap between the top 10% of commented-on posts and the top 10% of shared posts of a given organization ranged from about 39% to 60%. Put differently, between 40% (100–60) and 61% (100–39) of the most commented-on posts were not the same as the most shared posts. Focusing on the top 5% of engaging posts, about 45% to 70% of the most commented-on posts differed from the most shared posts. The percentages of overlap for each news organization, as well as a summary of the percentages, appear in Appendix C.
To find how news organizations’ messages on Facebook differ in the engagement modes they generate (RQ2), negative binomial regression analyses were performed as they are suitable for over-dispersed count data (over-dispersion means that the conditional variance is greater than the conditional mean). Such analyses have been used for predicting sharing on social media (Valenzuela et al., 2017) and participation in an article’s comments section on a news site (Weber, 2014). As with Valenzuela and his colleagues, I report incidence-rate ratios (IRRs), showing changes in incidence rate for a one unit increase in the predictor variable. To understand which content characteristics are associated with different engagement modes (RQ2a) in different cultural/national contexts (RQ2b), bivariate associations between a given content characteristic and a given mode of engagement are first examined, while controlling for the number of likes a Facebook page had at the time of posting. Later, multivariate models including the different content characteristics (predictors) are offered.
Table 2 shows how a given topic is related to each of the examined engagement modes (after controlling for the number of likes a Facebook page had) for the general sample (n = 1,600), US sample (n = 800) and Israeli sample (n = 800). Whereas an IRR greater than one indicates a positive effect, an IRR smaller than one indicates a negative effect.
Relationships Between Topics and Engagement Modes for News Organizations’ Posts (Negative Binomial Regressions).
Note. An empty cell indicates that no regression was performed due to a system’s error in computing the inverse LOG link function, the log likelihood, the gradient, or the Hessian matrix in the iterative process for at least one of the records.
p < .05; **p < .01; ***p < .001.
For politics/government, which was the most prevalent topic (e.g., 26.6% of the posts in the general sample), all the IRRs were greater than one (p < .001 or p < .01). Focusing on the general sample, the IRR was particularly high when commenting was the outcome variable: 2.264. This IRR suggests that posts where their main topic was politics/government were commented-on 126.4% more often than posts that their main topic was not politics/government. Posts about politics/government were shared 112.3% more often than other posts. Furthermore, the former posts were liked 65.2% more often than other posts. The positive associations between politics/government and engagement modes were found both in the United States and Israel.
The table also shows positive associations between health and sharing. Focusing on the general sample, posts that their main topic was health were shared 135.4% more often than posts that their main topics was not health (p < .001). In the US sample, posts about health were shared 307.4% more often than non-health posts (p < .001). Yet, in the Israeli sample, the association between health and sharing was not statistically significant.
H1a posited that politics/government would be positively associated with commenting. It is supported, though politics/government was also positively associated with sharing and other engagement modes. H1b posited that health would be positively associated with sharing. As health was linked to a high volume of sharing in the United States but not in Israel, the hypothesis is partially supported.
Table 3 focuses on news factors, showing that conflict, surprise, prominent figure, and reference to persons were positively associated with each of the examined engagement modes. For example, focusing on the general sample, posts that mentioned or referred to a conflict were commented-on 134.8% more often than non-conflict posts. The former posts were shared 126.1% more often than the latter posts. Posts containing an element of surprise or something unusual were shared 190.9% more often than other posts. The former posts were commented-on 84.2% more often than the latter posts. H2a posited that conflict would be positively associated with commenting, while H2b predicted that surprise would be positively associated with sharing. Both hypotheses were confirmed, though conflict was positively associated with sharing as well, and surprise was also positively linked to commenting.
Relationships Between News Factors and Engagement Modes for Media Outlets’ Posts (Negative Binomial Regressions).
p < .05; **p < .01; ***p < .001.
Table 4 focuses on emotional triggers, showing that anger was positively associated with commenting, sharing, and reacting, but not with liking. For instance, focusing on the general sample, posts evoking anger were shared 174.3% more often than posts that did not evoke anger. The former posts were reacted-to 65.7% more often than the latter posts. These patterns were found in both the US and Israeli samples. As H3 posited that anger would be positively associated with sharing and reacting, the hypothesis is supported. Sadness was also positively associated with sharing and reacting in both the United States and Israel, and anxiety was positively associated with sharing in the United States.
Relationships Between Emotions and Engagement Modes for News Organizations’ Posts (Negative Binomial Regressions).
Note. The reported emotions are the ones studied by Berger & Milkman (2012). Another emotion they studied—awe—is not presented here because the inter-coder reliability for this variable was relatively low. In addition, the frequency of awe in the sample was particularly low.
p < .05; **p < .01; ***p < .001.
Table 5 focuses on evaluative aspects of posts, showing that criticism was positively associated with each of the examined engagement modes; positive tone was positively associated with liking; and emotional language was positively associated with commenting. 5 In Israel, emotional language was also positively associated with sharing, liking, and reacting.
Relationships Between Evaluative Aspects and Engagement Modes for Media Outlets’ Posts (Negative Binomial Regressions).
Note. Posts were coded as containing evaluative language if this language was unattributed. Posts were coded as containing criticism or emotional language whether these were attributed or not.
p < .05; **p < .01; ***p < .001.
Table 6 focuses on structural features of posts, showing that the presence of a video was positively associated with commenting and sharing (also with liking and reacting in Israel). 6 The presence of a photo was negatively associated with commenting (also with other engagement modes in Israel). However, these findings should be interpreted with caution because the vast majority of the examined posts included a photo (this seemed to be the standard in constructing a post). The presence of an embedded link yielded certain negative associations with engagement modes, though the vast majority of the examined posts in the United States included an embedded link (this seemed to be the standard). 7 A follow-up analysis suggests that among posts containing an embedded link, posts linking to a news report were positively associated with each engagement mode.
Relationships Between Structural Features and Engagement Modes for Media Outlets’ Posts (Negative Binomial Regressions).
Note. Embedded link to a news report was examined among 1,108 posts in the general sample (those that included an embedded link), 746 posts in the US subsample, and 362 in the Israeli subsample.
p < .05; **p < .01; ***p < .001.
After separately analyzing associations between a given content characteristic and engagement modes, multivariate models with different content characteristics were examined to predict engagement modes. The examined predictors were most of the content characteristics in Tables 2 to 6. Content characteristics were excluded if they belonged to one of the following categories: not yielding any statistically significant association when bivariate associations were examined (the topic society/welfare); appearing in almost all of the examined posts in the general sample or in one of the countries (e.g., the presence of a photo); addressing an aspect that is also represented by another variable (posts identified as a “human interest story,” for example, tended to include the news factor “reference to persons”); and not generating a result due to a system’s error when bivariate associations were examined. This resulted in the inclusion of 24 content characteristics in the multivariate models while controlling for the number of likes a Facebook page had at the time of posting, as well as the country (United States—yes/no) and organization (newspaper—yes/no). An analysis of tolerance and variance inflation factor scores revealed no multicollinearity issues among the independent variables. 8 Table 7 presents these models for the general sample (n = 1,600).
Predicting Modes of Engagement With News Organizations’ Posts on Facebook: Multivariate Negative Binomial Models.
Note. A mixed-model negative binomial analysis has no direct analog to R2 (Coxe et al., 2009).
p < .05; **p < .01; ***p < .001.
Two key patterns emerged from the analysis. First, certain content characteristics, such as a prominent figure and a surprise, were statistically significant predictors of each engagement mode. Second, certain content characteristics were statistically significant predictors of one engagement mode or more, but not all engagement modes. For example, health predicted sharing, but not other engagement modes, and a positive tone predicted liking/reacting and sharing, but not commenting.
Focusing on particular engagement modes, some of the strongest predictors of commenting were a video and anger, and some of the strongest predictors of sharing were a video and surprise. A strong predictor of liking was a positive tone, and a strong predictor of reacting was a surprise.
Discussion
This study investigated to what extent and how news organizations’ posts on Facebook differ in the engagement modes they generate. Based on analyses of posts and engagement metrics, I propose three key principles. First, although highly commented-on, highly shared, and highly liked or reacted-to posts can overlap, they do not fully overlap. Each group of posts can differ from one another to a notable degree. Second, certain content characteristics of posts are linked to each of the examined engagement modes—commenting, sharing, or liking/reacting—in more than one cultural/national context. Third, certain content characteristics are linked to one or more modes of engagement, but not to all of them. These principles help think about content units as more engaging or less engaging than others, or as engaging in different ways.
The study makes a twofold theoretical contribution: suggesting to understand content units as stated above and revealing what can make news-related messages engaging in different ways. Below, I discuss key content characteristics that were linked to engagement modes.
With respect to news topics, this study shows that news organizations’ posts focusing on politics/government can be not only highly commented-on, but also highly shared and liked/reacted. The finding about commenting is in line with research showing that although stories on public affairs are not necessarily highly clicked-on compared to other stories, the former may prompt user comments (e.g., Boczkowski & Mitchelstein, 2013). Indeed, commenting can play a role in constructing a social or group identity through discussions of political issues (Tenenboim & Cohen, 2015). The present study suggests that the construction of such identity may also be done through sharing and liking/reacting-to news-related content about politics, at least in the United States and Israel. It is also possible that at least some of the engagement is organized or encouraged by political actors or their supporters/opposers. However, different engagement patterns were found regarding health: Posts about this topic were more likely to be shared than non-health posts, particularly in the United States (in line with Larson’s study in the Swedish context, 2018), but health posts were not linked to a high volume of comments and likes/reactions. A possible explanation is that news content deemed useful or meaningful tends to be circulated by users, though such content may not be enough for prompting other engagement modes.
Yet, a reference to a prominent figure, such as a president or an actor, and a surprising element or something unusual, are news factors that can entice different engagement modes, including commenting, sharing, and liking/commenting. Although the notion that news factors can contribute not only to perceived newsworthiness but also to engagement is not new (e.g., Trilling et al., 2017; Weber, 2014), this study adds to news values theory by identifying which factors are key to eliciting different engagement modes.
Whereas the news factor conflict was positively associated with each engagement mode when bivariate associations were examined, the associations were not statistically significant in the models with other content characteristics. In the model for commenting, the association between conflict and commenting was marginally significant. That said, stories about politics may involve controversial issues or conflicts. This may partly explain the results about conflict in the model with other predictors.
Emotional triggers seem to play a particularly important role in engaging users with news-related content. Triggers of anger in the content under study were linked to a high volume of shares and reactions, in line with previous research (e.g., sharing political content, Amit-Danhi & Shifman, 2022; emailing news, Berger & Milkman, 2012; reacting to political content, Zerback & Wirz, 2021). Yet, such triggers were also positively associated with commenting. The findings may suggest that as anger is characterized by a state of heightened activation (e.g., Berger & Milkman, 2012), it may also lead to user engagement in an open-ended manner, that is, writing comments. However, whereas triggers of sadness were positively associated with shares and likes/reactions, they were not linked to a high volume of comments. Berger and Milkman (2012) suggested that because sadness is a low-arousal emotion, it was less likely to prompt emailing of sad stories to other people. But the present study offers that the dynamic of sharing sad news posts on Facebook may be different. It is possible that users share such posts because they perceive them as relevant to people’s experiences or as impactful (e.g., Zerback & Wirz, 2021). Yet, perceptions of relevance or impact may not be enough for eliciting comment writing.
The findings also show the importance of evaluative aspects, such as the tone of a post. Although the Like button can be understood as a multifunctional tool (Alhabash et al., 2019; Sumner et al., 2018), the relationship between a positive tone and a high volume of likes may suggest that by liking news-related content on Facebook, users express their support. The presence of evaluative language beyond the fact of an occurrence or issue did not predict engagement in the models with other content characteristics. But the examination of bivariate relationships showed cross-national/cultural differences: Posts with evaluative language were negatively associated with commenting and sharing in the United States but positively associated with liking/reacting in Israel. These findings may reflect cultural differences related to the expression style used in these countries (e.g., Katriel, 1986). More generally, despite cross-national differences regarding some other content characteristics (not only evaluative aspects), multiple key engagement patterns transcended national boundaries.
One concern of the participation paradigm in audience research (Livingstone, 2013) is how people engage with media. Focusing on visible engagement modes with news-related messages on Facebook, this study portrays a complex and nuanced picture: When messages contain particular content characteristics, such as a surprise or a prominent figure, they may entice users to comment on, share, or like/react to the messages. However, when messages include other content characteristics, particular engagement modes are more likely. For example, users may share content with a high practical utility or content that evokes a negative emotion, but they may comment on content that triggers a specific negative emotion: anger.
The complex and nuanced picture offered in this study does not only enrich audience research under the participation paradigm and literature on engagement enhancers, but it can also inform news organizations that seek to increase the visibility of their Facebook posts in an attempt to reach and attract audiences. News organizations, at least in the examined democratic contexts, would do well not to shy away from posting about political issues. They could also post more health-related messages—not only during a global health crisis (though the analyzed data in this study were collected before the recent pandemic). Posting on social media reliable content about political and health issues can also be an important public service, particularly at a time when falsehoods are rampant in the digital media environment (e.g., Vosoughi et al., 2018). In constructing messages, news organizations could accentuate certain content characteristics, such as a surprising element, when appropriate. However, as triggers of anger may prompt user comments, including uncivil ones, news organizations would do well to carefully moderate comment threads attached to messages with such triggers.
The study focused on messages posted by US and Israeli news organizations on Facebook. Future research would do well to examine additional social media platforms and cultural/national contexts. It would also do well to consider additional factors in predicting different engagement modes. For example, a future cross-national study may examine different engagement modes with news-related messages about politics in countries with varying degrees of political polarization. User engagement is a complex and multidimensional phenomenon, but the findings of this study and future research can contribute to a better understanding of its behavioral dimensions.
Supplemental Material
sj-docx-1-sms-10.1177_20563051221130282 – Supplemental material for Comments, Shares, or Likes: What Makes News Posts Engaging in Different Ways
Supplemental material, sj-docx-1-sms-10.1177_20563051221130282 for Comments, Shares, or Likes: What Makes News Posts Engaging in Different Ways by Ori Tenenboim in Social Media + Society
Footnotes
Acknowledgements
The author thanks the coders for their diligent work. The author also expresses his gratitude to Stephen Reese, Talia Stroud, Iris Chyi, Gina Masullo, and Tom Johnson for their support and advice.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding from the School of Journalism and Media at the University of Texas at Austin was used to pay coders.
Supplemental Material
Supplemental material for this article is available online.
Notes
Author Biography
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
