Abstract
Civil and argumentative public discussions are considered crucial for functioning democracies. Among other factors, the quality of user discussions of political issues on news sites depends on prevalent discussion norms. We integrate injunctive and descriptive norms into news comment research, assuming that the degree of salience of the respective norm influences the commenting behavior. Furthermore, we discuss how technical affordances such as default comment sorting determine the comment visibility and thus the salience of norms. Using data from a content analysis of 8162 comments on eight German news sites, we investigate how the two norm types influence deliberative forms of commenting. The results show that different types of salient injunctive and descriptive norms promote norm-compliant commenting. Furthermore, the default comment sorting can determine which comments are more or less salient. The results underline the importance of distinguishing different norm types in analyzing the quality of user comments in comment sections.
Keywords
News sites are among the most important online sources for people to keep up with the news (Newman et al., 2019). Most news sites offer their users the possibility to discuss the journalistic articles or selected topics. A lot of news organizations have moved their discussions to social media platforms (Stroud et al., 2016). In Germany, about half of the news sites also provide comment sections on their websites (Strippel and Paasch-Colberg, 2020). We understand comment sections as technical infrastructures embedded within news sites, which provide the opportunity for users to comment on the journalistic reporting, either in the direct environment of a news article or in a separate forum which links discussions about a certain question to specific news articles (Ksiazek and Springer, 2020). From the viewpoint of democracy theory, they perform two vital functions for a strong public sphere: first, they provide citizens with information about political discussions within the political center, which is a central requirement in all theories of democracy (Ferree et al., 2002). Second, they enable citizens to participate in the public debate with a high potential visibility (Manosevitch and Tenenboim, 2017). These outcomes follow the requirements of the more demanding participatory or discursive approaches (Ferree et al., 2002). In Germany, 14% of the population comment on the news (Newman et al., 2019) and up to 67% read other peoples’ comments (Springer et al., 2015). From a strategic perspective, news organizations also pursue economic interests in offering and improving comment sections, hoping to make their users visit the site regularly and increase brand loyalty (Manosevitch and Tenenboim, 2017). At the same time, they try to keep the resource-intensive task of managing comment sections low (Chen and Pain, 2016).
The normative quality is a crucial factor for comment sections (Sukumaran et al., 2011). Previous research offers diverging perspectives on possible effects of norm-violating comments. Coe et al. (2014) found that uncivil contributions and missing arguments can negatively impact the perceived legitimacy of arguments, speakers, and democratic discourse in general. However, Rossini (2019) questions whether incivility alone can be made responsible for destructive discussion outcomes and recommends a broader understanding of acceptable speech. Scholars have identified different influences on the quality of user comments, such as design features (Wright and Street, 2007), topics, or commenting policies (Ksiazek, 2018). In addition, enforcing norms in comment sections can foster respect and interaction in the discussion (Wright and Street, 2007). Previous research has found mixed results regarding moderators’ or users’ influence on the quality of discussions (Stroud et al., 2015; Sukumaran et al., 2011). An integrated analysis of different norm types, their salience, and norm-influencing technical affordances has been missing. Based on the focus theory of normative conduct (Cialdini et al., 1990), this article addresses these research gaps: How do discussion norms become salient for subsequent users? How do they affect subsequent commenting? To what extent do technical affordances moderate the influence of norm salience on deliberative commenting?
Discussion norms in comment sections
Previous research has debated whether user discussions vitalize or threaten democratic debate. Positive aspects are an increased access to public debate, which can promote the diversity of viewpoints and backgrounds (Graham and Wright, 2015). Diversity is seen as an important feature of democratic decision-making (Price et al., 2006). Negative aspects include that citizens do not necessarily follow democratic discussion norms, but engage in uncivil or destructive discussions (Coe et al., 2014). At a societal level, these discussions might lead to fragmentation or balkanization (Sunstein, 2002). Previous research has shown that user comment sections are neither entirely beneficial nor hostile. The quality of the participation varies, but a substantive part of the users engages in civil, constructive and argumentative discussion (Coe et al., 2014; Freelon, 2015). Furthermore, the provision of good arguments can evoke disagreement and reduce the potential of fragmentation (Marzinkowski and Engelmann, 2021). These results have implications for journalism’s role in the public debate. It is no longer reduced to providing information and initiating discussions, but can also provide the conditions for good discourse. As hosts of these discussions, journalists are responsible for the content published on their websites and for avoiding negative repercussions for their journalistic work (Diakopoulos and Naaman, 2011). This can be achieved through the definition of discussion norms, consistent moderation practices, and technical infrastructures that facilitate good discourse (Freelon, 2015; Wise et al., 2006). A lot of journalists express skepticism of their role as moderators, due to disinterest, negative expectations about the quality of the discussions or fear of incompatibility with the journalistic objectivity norm (Chen and Pain, 2016; Diakopoulos and Naaman, 2011). Other studies found that journalists are increasingly seeing active engagement as an opportunity to improve the discussion and as part of their tasks (Binns, 2012; Chen and Pain, 2016).
To understand the role of discussion norms in the process of participating in user discussions, we turn to a psychological explanation of the operating principle of norms. From this perspective, norms refer to perceptions and expectations of a group or the society regarding appropriate and/or accepted behavior in a given situation (Turner, 1991). They are considered “to be implicit, conditionally followed, and motivated by external (vs internal) enforcement” (Farrow et al., 2017: 2) in certain situations or environments (Cialdini et al., 1990). Norms are an integral part of communication processes because they can guide behavior (Stroud et al., 2015; Sukumaran et al., 2011). Regarding online discussions, a frequently applied normative concept is public deliberation (e.g. Stroud et al., 2015). It emphasizes compliance with certain norms as a prerequisite for constructive and democratic discussions (Graham, 2009). Two central norms are rationality, which demands a reasoned and fact-based argumentation, and civility, which emphasizes respect for the arguments of the other side (Graham, 2009). 1 Whether these should be the only acceptable forms of political speech has been questioned (Bächtiger and Pedrini 2010; Rossini, 2019). However, these norms are considered beneficial for the public discourse (Cappella et al., 2002) and are often required by news organizations in their guidelines. Whether rationality and civility are practiced in comment sections influences recipients’ cognitions, emotions, and behavior. Uncivil comments promote negative perceptions of journalistic quality (Prochazka et al., 2018), reduce political trust and political self-efficacy (Borah, 2013), and increase aggressive emotions (Chen and Lu, 2017). Justifications in news media increase recipients’ argument repertoires (van der Wurff et al., 2019) and increase user participation (Marzinkowski and Engelmann, 2018). Therefore, the question arises to what extent these deliberative norms are established and enforced in comment sections.
Norm types in online commenting
The focus theory of normative conduct makes two core assumptions. The first suggests a more precise norm definition by distinguishing injunctive and descriptive norms (Cialdini et al., 1990). Injunctive norms describe behavior most people morally (dis)approve (Ought-norm), and descriptive norms refer to how most people typically behave (Is-norm). This distinction has become an integral part of the norm concept. Nevertheless, in the context of online discussions, studies have concentrated on descriptive norms (e.g. Stroud et al., 2015; Sukumaran et al., 2011). The second assumption states that neither of the two norm types is primarily more influential, but their respective temporal salience determines their behavioral influence. This means that the norm stands out from the context and attracts a person’s attention focus (Cialdini et al., 1990). A norm always has to be made salient by the respective situational context to be effective (Cialdini et al., 1990), but it does not have to be learned anew in each situation. Norm salience can activate already existing knowledge structures or learned schemata, for example, how one ought to behave in a discussion. Previous research on norms in online discussions has not sufficiently included the salience aspect. In the few experimental studies, salience was only implicitly tested due to the stimulus (Sukumaran et al., 2011). Cialdini et al. (1990) define salience primarily from the perspective of individuals. For analyzing user discussions, a concretization between external and internal salience is indispensable. The former concerns the extent to which an object stands out from other items or becomes visible in a situation. In contrast, the internal salience of an object refers to its personal relevance for the recipient (Kiousis, 2004). We will thus focus on external salience. In comment sections, external salience is evoked by the content and tone of users’ and community managers’ comments and the visibility created by the discussion architecture (Stroud et al., 2015; Sukumaran et al., 2011). However, studying the norm salience in real-life user comments bears a problem. Whereas experimental studies can control what subjects see, this cannot be reconstructed for real-life discussions. So far, there are almost no results on which comments users read. Especially in longer discussion threads, they probably do not read every comment (Springer et al., 2015). Users might have different perceptions of the current norm based on the discussion parts they read. Therefore, the normative salience should be determined individually through the comments that were highly visible for a user before writing a comment (Treem et al., 2020). These comments are most likely those that were published directly before the specific comment being analyzed, which we call preceding comments. We consider preceding comments a reliable predictor for salient norms and consequently as the relevant norm-setting comments.
Salience of injunctive deliberative norms
Collective injunctive norms of comment sections are defined through news organizations’ commenting policies (Ksiazek, 2018) or journalists’ and community managers’ moderation practices. If users promote norms that contradict those of the news organizations, their comments will most likely be sanctioned (Ksiazek and Springer, 2020). Journalistic moderation practices are especially effective in establishing and changing norms (Stroud et al., 2015). Moderation is defined as any kind of institutional engagement aimed at regulating processes or contents of online discussions (Paasch-Colberg et al., 2020). This encompasses sanctioning forms, such as deleting or changing comments, and engaging forms, such as participating in the discussion. The sanctioning forms can be further differentiated: Pre-moderation denotes the deletion of norm-violating comments before their publication on the website. Post-moderation affects comments that are already published (Reich, 2011). It can be carried out invisibly, by removing comments from the thread, or visibly, by replacing comments with a reference that the comment was deleted and, possibly, why. News sites employ a diversity of these moderation strategies (Chen and Pain, 2016; Paasch-Colberg et al., 2020; Reich, 2011). Empirical findings show that subsequently deleting uncivil comments leads to higher civility levels among users (Ksiazek, 2018). Furthermore, research has found that positive effects on the amount and quality of user comments can especially expected from visible moderation (Wise et al., 2006). However, users can become demotivated when they do not accept the moderators’ decisions (Wright and Street, 2007).
Pre- and post-moderation make norms salient to varying degrees. In pre-moderation, the norm violation is not perceptible so that injunctive norms do not become salient in the discussion. In visible post-moderation users can perceive the enforcement of the injunctive norms (Treem et al., 2020). Thus, visible post-moderation can make users feel responsible for their comments, create social control through public rewards or sanctions, and therefore lead to behavioral adjustments. Thus, we assume: The more injunctive deliberative norms are salient to users in discussion threads, the more they are likely to comply with these norms (H1).
Salience of descriptive deliberative norms
Descriptive norms in comment sections indicate which behavior is likely to be accepted in the discussion. For example, deliberative criteria in user comments (e.g. respect, verifiable justifications) can be perceived as prevalent descriptive collective norms. Previous findings on the usage of these criteria (Graham, 2009) and their influence on users (Sukumaran et al., 2011) can be interpreted in this light.
Descriptive deliberative norms in comment sections can promote users’ direct or indirect learning of norms. Direct learning results from personal experience (e.g. positive or negative feedback), indirect learning from observing other peoples’ comments. Experiments confirm the influence of descriptive norms using examples of individual littering behavior (Cialdini et al., 1990) or thoughtful (Sukumaran et al., 2011) or uncivil commenting (Chen and Lu, 2017). Different from injunctive norms, the descriptive norm can promote deliberative as well as non-deliberative norms. This is because the norm is created by the behavior of the majority of users, who do not necessarily follow the rules defined in the commenting policies or theory (Chen and Lu, 2017; Coe et al., 2014). We therefore assume: The more descriptive (non-)deliberative norms are salient to users in a discussion thread, the more they are likely to comply with these norms (H2).
We have argued that the degree of external salience of descriptive norms depends on the preceding comments. However, it is still unclear how many comments the norm-setting process is based on, because there is a lack of studies regarding the volume of comments users read on average. Eye-tracking studies on search engine use show that most users rely on the given ranking and pay most attention to the first three results before clicking on a link 2 (Kessler and Engelmann, 2019; Lorigo et al., 2006; Pan et al., 2007). The first three results provide an impression of the relevant content, which makes users confident in choosing a website. This assumption is supported by Cialdini et al. (1990). They found that until a certain extent, more norm-setting cues lead to an increased norm-consistent behavior, until it peaks at eight cues. When applied to online discussions, we assume that users show a similar behavior: they form a first impression about the current descriptive norm after a certain number of preceding comments. The size of this number determines the salience of the norm: in smaller groups of norm-setting comments, the difference between norm salience and norm violation might not be easily recognizable, because the groups do not differ enough in size. In larger reference groups, a majority for norm conformation is based on a higher number of comments. Here, the potential influence of single norm-violating comments is smaller. In the form of a contrast effect, their deviation might even make the norm promoted by the bigger majority more visible (Cialdini et al., 1990). Therefore, we assume, the higher the number of norm-setting comments considered as visible to users, the stronger the relation between salient descriptive norms and subsequent compliance with these norms (H3).
Salience of descriptive deliberative norms and technical affordances
The salience of descriptive norms in comment sections is determined by technical affordances (Evans et al., 2017). An important technical affordance is the comment sorting (Young et al., 2021), because it determines the order of the comments. On news sites, comments are usually sorted chronologically (Strippel and Paasch-Colberg, 2020). Although the sorting can be changed individually, search engine research shows that users normally do not change the presented order (Salmerón et al., 2013). When sorted by oldest, all users see the same comments at the top of the thread, because new comments are added at the bottom. Consequently, the first comments of a thread have a high visibility and could serve as norm-setting. When sorted by newest, new comments are always added at the top of the thread, so that the composition of the first comments varies between users (Lampe and Resnick, 2004). Sorting by oldest comments ensures greater interindividually consistent and comparable visibility, because the first comments are always on top of the thread. They can provide a first impression of the current descriptive norm in addition to the preceding comments. This way, they are expected to exert an additional norm-setting influence on subsequent comments. This additional effect should not exist on news sites with default sorting by newest due to the dynamically changing comment order over time. We assume: the first comments of a thread have an additional descriptive norm-setting effect on (non-)deliberative commenting on news sites with default sorting “by oldest comment” (H4).
Methods
To test the hypotheses, we conducted a manual content analysis of 8162 user comments on eight news sites.
Sampling of news sites
Newspapers and magazines and their online outlets are important for the German news landscape, with the increasing importance of online information sources (Hölig et al., 2021). For our sample, we selected from the 100 most wide-reaching journalistic German news sites (IVW, 2017) because we assume that wide-reaching news sites have high user participation levels. News sites were only selected if they provided comment sections, that is, the opportunity for users to discuss directly on the respective news site. The default thread visualization of the comments had to be a chronological order. This resulted in a sample of eight news sites (see Table 1). They represent a broad range of reporting styles, editorial lines, technical affordances, and moderation techniques. Moderation is strongly regulated in Germany. The German Network Enforcement Act (NetzDG) demands platform owners to delete hateful and illegal comments within 24 hours (Hölig et al., 2021). Even before this act was passed, editors had expressed a high sensitivity for hateful comments that had resulted in strict moderation regimes (Reich, 2011). In our sample, six sites employ pre-moderation and two post-moderation (see Table 1). Three practice visible moderation, with one of them naming reasons for deleting comments (see examples in Supplemental Appendix C2).
Default visualization and definition of norm-setting comments per news site.
Online reach was measured as the average in million visits (online + mobile) of each website over the entire period under investigation, from November 2015 to September 2017. Based on data provided by the German Audit Bureau of Circulation (IVW, 2017). Editorial lines according to Begenat (2016) and euro I topics (2018).
Sampling of topics
To select relevant topics, we collected a full sample of all online news articles published on the eight selected news sites with a RSS feed crawler. We chose two public interest topics: first, the debate on an upper limit for refugees in 2015/2016. It represented a conflict between the conservative governing parties CDU and CSU. Overall, this conflict was very emotional, because it included expressions of fear and anger about perceived injustices and stories about refugees’ personal destinies. Due to the long-term news coverage and the high numbers of comments on this topic, we selected articles from days on which at least three news sites reported on an event related to the discussion on an upper limit (e.g. a statement of a politician or a party congress where the issue was debated). This resulted in six investigation days between 20 November 2015 and 24 November 2016. The second topic was the debate on pension reforms in the 2017 federal election campaign. The topic was mainly debated between the government and the opposition. The debate was less emotional than the first topic, because it focused on technical details and outcomes of different pension models. Furthermore, it covered a wider range of thematic aspects. Participations rates were lower than for the first topic, so that all articles on this topic published between 23 November 2016 and 14 September 2017 were considered. 3 Across both topics, 159 threads were included in the sample. 4 The journalistic articles with complete discussion threads were saved as PDF files after moderation had occurred.
Sampling of comments
The selection of comments depended on the type of thread visualization employed by the websites. On news sites with hierarchical discussion views, only first-level comments were included. These comments do not reply to other comments and are often displayed at the first level, furthest to the left. In contrast, reply comments, generated by using a reply button, are displayed directly beneath the comment they replied to, but shifted to the right, so that they can be perceived as replies. The first-level comments are more salient than their reply comments due to their permanent visibility and visual cues (e.g. larger font or avatars). Reply comments are often displayed with a smaller font or avatar and folded in by default. Furthermore, reply comments can be written later than following first-level comments. Thus, in a hierarchical discussion view, we can only be certain that users who have submitted first-level comments have seen the preceding first-level comments. For users who submitted reply comments, it is difficult to reconstruct the visible comments. In comment sections with a linear discussion view, all comments are sorted chronologically, without considering the function of reply comments. Here, the distinction between first-level and reply comments is not relevant, because they are displayed at the same level. Therefore, all comments were included. 5 Theoretically, this way of selecting norm-setting comments follows the assumptions of salience of injunctive and descriptive norms. Methodologically, our strategy considers the specifics of all sampled news sites. Our sample consisted of 3087 user comments for the migration and 5075 for the pension topic (for detailed information see Supplemental Appendix A1). The comments were coded by six experienced coders, who were trained for several months for each of the two issues. In this course, we intensively discussed the recognition of civility and justification.
Variables
The coded variables will be explained in the following sections. A summary can be found in Supplemental Appendix A2.
Degree of salience of injunctive norms
For each comment in our dataset, we coded whether it had been subjected to a visible form of post-moderation, that is, deleted, shortened or commented on by a moderator (PA = 1; K−α = 1). We calculated two measures for the degree of salience of the injunctive norm for each comment: (1) an individual measure for each comment, which expresses how many of its three preceding comments were visibly moderated and (2) a global measure for all comments of the same thread, which expresses how many of the first three comments of the thread were visibly moderated. Both variables were standardized on a scale from 0 to 1. In 23 of the 40 discussions of the three websites that employ visible post-moderation, visibly moderated comments were detected. On all news sites, on average, 1% of the three preceding comments and 3% of the first three comments of a thread were visibly moderated (for information on all news sites see Supplemental Appendix A3).
Degree of salience of descriptive norms
Two deliberative quality indicators were coded for each comment in our sample. The justification norm was measured in a two-step process. First, we coded whether a comment contained a justification. Second, for those comments containing a justification, we differentiated between (1) objectively verifiable justifications, such as quoted sources, facts or statistics, which means that we did not measure the accuracy of a given justification, but whether it would be possible to verify it and (2) subjective justifications, such as opinions, hypotheses or personal experiences (Stroud et al., 2015; presence of a justification: PA = 0.81; K−α = .62; type of justification: PA = 0.87; K−α = .92). Objectively verifiable justifications were chosen as indicators for the justification norm because they are considered a desired form of reasoning according to deliberation theory (Stroud et al., 2015). Furthermore, the provision of evidence is a common demand for user discussions. 6 We coded civility as the absence of a derogatory statement toward discussion participants or third parties or their statements. The derogation could refer to their character, position, esteem, and so on (Graham, 2009; Ruiz et al., 2011; PA = 0.92; K−α = .57). Proverbs in general do not necessarily imply incivility in the deliberative sense. Derogatory statements are considered uncivil, but are not necessarily forbidden by the editorial policies. Severe norm violations (e.g. inciting or discriminating comments) had already been deleted by moderation. 7 All variables showed a pairwise agreement (PA) above the critical value of .70 (Frey et al., 2000). For the presence of a justification and civility, Krippendorff’s α values were lower, which can be explained by the highly skewed distribution of these nominal variables. Here, the pairwise agreement can be considered more suitable (Feng, 2014). The frequencies of the civil and justification norm can be found in Supplemental Appendix A4.
We calculated four different types of variables measuring the degree of salience, separately for each descriptive norm: (1) individually for each comment, as the share of norm-complying comments in its three preceding comments; (2) globally for all comments in the same thread, as the share of norm-conforming comments in the first three comments of the thread; (3) individually for each comment, measuring whether its preceding comment was norm-conforming (no/yes); and (4) individually for each comment, as the share of norm-complying comments in its five preceding comments. 8 All variables are standardized on a scale from 0 to 1. For the mean shares of the different operationalizations of the descriptive norms see Supplemental Appendix A5.
Dependent variables
The dependent variables are based on the same coding as the descriptive norms: whether a comment (1) contains an objectively verifiable justification and (2) is free from a derogatory statement. As described above, we assume that the preceding 1–5 comments of a comment resp. the first three comments of a thread are norm-setting (for descriptive statistics, see Supplemental Appendix A5).
Analysis strategy
We ran one-level logistic regressions 9 for explaining justified and civil commenting. Both dichotomous dependent variables are located at the comment level and indicate whether the deliberative norm was fulfilled or not. The models include the different variants of the independent variables explained above. Some are dichotomous, and some are continuous on a scale from 0 to 1. To compare the effect on the dependent variables, we interpret the odds ratios (ORs) as increases of the probability that the respective deliberative norm will be present. As measures of model fit, we use the likelihood ratio test and Nagelkerke’s pseudo-R2 (Menard, 2000). In each model, we included the other descriptive norm as a control variable because deliberative norms are correlated (Coe et al., 2014). In addition, we controlled for the topic, because deliberative commenting might also be influenced by the general tone within a topic, due to differences in involvement, emotionality or fact-relatedness (Ksiazek, 2018).
Results
In H1, we assumed that the more salient the injunctive deliberative norms on news sites to users, the more likely they lead to norm-compliant commenting. Higher salience was indicated through visible deletions within the preceding three comments. The visible deletion of norm-deviating user comments increases the civility in the thread (OR = 6.50, p < .01), but has no significant effect on justified commenting (OR = 1.45, n.s.). H1 is partially confirmed for the descriptive civility norm, but not for justification.
As assumed in H2, the salient descriptive justification norm in the three preceding comments increases the likelihood of norm-compliant justified commenting in subsequent comments (OR = 2.38, p < .001). This also applies to the descriptive civility norm, which has a positive effect on norm-compliant civil commenting (OR = 2.92, p < .001). H2 is confirmed for justified and civil commenting (see Supplemental Appendix B1 for the regression models).
In H3, we assumed that the higher the number of norm-setting comments, the stronger the relation between salient descriptive norms and subsequent compliance. Thus, we compare the descriptive justification and civility norm consisting of three preceding comments with indicators for the descriptive norm based on one and five preceding comments. The results show that justified commenting becomes more likely with a descriptive justification norm consisting of five preceding comments (OR = 3.76, p < .001) than with a norm consisting of one (OR = 1.55, p < .001) or three (OR = 2.38, p < .001) preceding comments. Similarly, the descriptive civility norm consisting of five comments is shown to increase civil commenting by a factor of 4.39 (p < .001), while the norm consisting of one comment increases civil commenting by a factor of 1.62 (p < .001) and consisting of three comments by factor of 2.92 (p < .001; Supplemental Appendices B2 and B3). 10
In H4, we assume that there is an additional descriptive norm-setting effect of the first comments of a thread on news sites with default sorting by oldest comments. First, we turn to the descriptive justification norm. On news sites with default sorting by oldest comments, the first three comments in a thread have an effect on subsequent justified commenting (OR = 1.35; p < .05), which becomes marginal when the preceding three comments are added to the model (OR = 1.27; p < .10; Supplemental Appendix B4). Second, we look at the descriptive civility norm. On news sites with default sorting by oldest comments, the first three comments in a thread also have an additional effect on subsequent civil commenting (OR = 1.89; p < .05), which becomes marginal when adding the preceding three comments into the model (OR = 1.74; p < .10; Supplemental Appendix B5). In detail, the norm-setting effect of the first three comments can be found on spiegel.de (OR = 2.26; p < .05; Supplemental Appendix B6). All in all, we find partial evidence in support of an influence of the default sorting according to H4 for both norms. This is supported by a comparison of both types of default sorting: the first three comments do not have an additional effect on subsequent comments on news sites with default sorting by newest comments, only on those with default sorting by oldest comments (Appendices B4 and B5, models 4 and 6).
Discussion
Our results suggest that justified and civil commenting can partially be explained by visible moderation of norm-deviating user comments (injunctive norms) and strongly by previous comments (descriptive norms). The effect of the descriptive norm increases with the number of comments that are considered norm-setting. The default sorting partially influences the norm-setting process, because the first comments of a thread have an additional effect on subsequent commenting on some websites.
Regarding the effect of the injunctive norm on civil commenting, we conclude that users adapt to the norms that are visibly promoted by moderation. Why they do so cannot be derived from our results. Possible motivations are the avoidance of sanctions or the willingness to learn about the prevalent norms to become part of the community. Thus, we assume that users observe the moderation practices over time (Stroud et al., 2015). However, this might not apply to all users, because regulative moderation can also generate negative reactions among users (Ksiazek and Springer, 2020). As we only found a positive effect on civil commenting, we assume that users primarily associate moderation with civility and less with justification. 11 Incivility dominates the discussion about comment sections (Coe et al., 2014) and is frequently perceived in discussions (Diakopoulos and Naaman, 2011). Especially in controversial topics such as migration and welfare, users might expect a lot of uncivil comments to be deleted by moderation (Diakopoulos and Naaman, 2011).
The significant effects of the descriptive civility and justification norm indicate that people look for orientation in the discussion before they write a comment. There are at least two explanations: first, users might be influenced by social proof, that is, a social pressure that causes people to copy the behavior of the people they observe (Cialdini, 1993). They aim to reduce uncertainty or discover similarities with others. Accordingly, users probably observe several comments or users. Which and how many comments users read before they write a comment can vary. Our results indicate a recency effect that is caused by the comments that directly precede a comment. Second, descriptive norm compliance may be motivated by the expectation of positive or negative sanctions from others, for example, in the form of (dis)likes or replies (Neubaum and Krämer, 2018).
Our findings also suggest that the absolute number of comments that promote a certain norm influences the norm-setting process. This is because in a larger context of norm-setting comments, more cues about the prevalent norm are available for subsequent users. We assume that users employ a heuristic processing of information which helps identifying appropriate behavior (Cialdini et al., 1990). Furthermore, norm-deviant comments can make the prevalent norm more salient, because they remind users of possible violations (Cialdini et al., 1990). In the case of one norm-setting comment, a deviant comment is not possible. With five norm-setting comments, the difference between deviant and non-deviant comments can be more easily perceived than with three norm-setting comments. Our findings do not show a saturation effect, where additional comments belonging to the prevalent norm would not further strengthen the influence of the descriptive norm (Cialdini et al., 1990). We assume that this effect might come into place with a higher number of norm-setting comments.
Regarding the default comment sorting, we found no generalizable effect, but a tendency and differences between the news sites. There are several reasons for the missing effect. First, the preceding comments are more visible and thus more salient. Our results indicate that first and preceding comments often promote different norms. In these cases, the preceding comments rule out the first comments. Second, there might be user-specific differences in the way the sorting affordance is used. Thus, the first comments of the thread might have been less salient to users than expected. Third, specifics of the news sites might be the reason for the mixed results. We assume that on spiegel.de, the first comments had an additive effect because they often corresponded with preceding comments, which can be a result of consistent pre-moderation. The first comments of a thread could have a priming effect on users and lead to an activation of schemata about the dominant norm, which is then reinforced by the preceding comments.
For journalistic moderation, our results add up to previous research that concluded that active moderation can improve the discussions on news sites and thus promote more civil and constructive discussions (Wise et al., 2006). Considering the influence of deliberative norms, news sites should spend some moderation effort to ensure that the desirable descriptive norms are salient throughout the whole thread. If they do not want to spend the necessary resources, they can choose a default sorting by oldest comments and make sure that the first comments are norm-compliant, because they can partly encourage others to norm compliance and improve the perceptions of the journalistic quality of the article (Prochazka et al., 2018). However, due to the strong effect of the preceding comments, they will leave to chance which norms are set for subsequent users and thus how the discussions continue. Depending on the moderation strategy, users with different motivations will feel encouraged to participate in the discussion (Springer et al., 2015), which can increase the visibility of desirable or non-desirable descriptive norms. Especially for users who identify with the community, a consistent moderation is desirable because it can help them internalize the norms and increase the motivation to avoid sanctions.
From the normative deliberation perspective, it is very desirable that users promoting deliberative norms can have an impact on the following discussion. The theory stresses that the participation of an active citizenry is essential for the acceptance of political decisions and thus for the functioning of the democratic process (Ruiz et al., 2011). Our results can be regarded from two directions: non-deliberative comments encourage others to violate deliberative norms, deliberative comments to adhere to them. Thus, users play a crucial role for the tone of the discussion (Ruiz et al., 2011). This is especially relevant on news sites, where the discussions are held in the direct environment of journalistic reporting. Civil and constructive debates can thus have beneficial effects for both the information process and the active engagement, because they can increase mutual respect and prevent the public debate from becoming fragmented (Graham, 2009; Marzinkowski and Engelmann, 2021). Future studies should extend this analysis to other deliberative norms (e.g. reciprocity or topic coherence; Graham, 2009).
Limitations
Our results are subject to limitations. Regarding discussion norms, the number of cases with visible moderation was very low. This implies that our data do not allow for a direct comparison of injunctive and descriptive norms. Focus theory argues that the situational salience of a norm determines how much impact it can exert on following users (Cialdini et al., 1990). In our sample, the descriptive norm had a higher probability of coming into effect. In addition, the interpretation is complicated because the reasons for moderation are not displayed on most news sites. Which norms news organizations prioritize depends on their strategic goals but also on moderators’ interpretation of norms. Both limitations should be addressed in research designs that can ensure a higher variance in these variables, for example, experimental designs which use visibly moderated comments as stimuli. Our findings on incivility are also limited by the fact that on news sites with pre-moderation, the full extent of incivility is not publicly visible, because comments with severe violations are not published. Civility was present at a high level, while justifications were less frequent. Our results could be affected by a ceiling effect in the case of civility and a floor effect in the case of justification.
Regarding user-specific influences, our method did not account for users’ reading activities. This can only be resolved in experimental settings. So far, there has been little research on users’ perceptions and their contribution to the development of descriptive norms across different discussion designs (Buder et al., 2015). Also, focus theory does not consider the possibility that loyal and frequent users might start to internalize norms and might therefore be less influenced by the situational norm salience. Thus, parts of our results could also be explained by established norms in certain communities. Our design could also not account for different motivations and perceptions users have of comments and social norms (Springer et al., 2015). Also, we could not consider whether users had used the technical affordance that allows us to change the sorting order, which also determines the degree to which comments are visible. To consider these insights, future studies should combine content analyses with survey elements.
Regarding the platform characteristics, the studied news sites differ not only in comment sorting, but also in moderation strategies and thread structures, so that differences in deliberative commenting could also originate from these factors. Our decision to exclude reply comments in hierarchical views implies that we cannot generalize our findings on reply comments and cannot control for their potential norm-setting influence on first-level comments. Furthermore, we could not control which comments had been visible to users due to pre-moderation. Future studies should use different types of data collection that can point out exactly which comments are visible to users in the moment of writing a comment. Our study did not include another sorting logic where the most popular comments are displayed on top. This sorting logic is applied on most social media platforms, which are also relevant spaces for discussion norms. There, the visibility of comments is influenced by users’ ratings and the point in time someone joins the discussion (Buder et al., 2015). Thus, it is even more difficult to reconstruct which comments were visible for subsequent users than in the case of chronological sorting. Our results thus cannot be replicated in these discussion environments. For news organizations employing relevance sorting, our results imply that they should carefully consider which comments they want to display as most relevant.
Methodologically, the relationship between publicly visible post-moderation and commenting behavior is not causal but should be understood as a correlation, because the deletion of comments through post-moderation happens with a delay.
Contributions of this study
This study makes substantial theoretical and methodological contributions to news comment research. From a socio-psychological perspective, the influences of specific norm types on discussion norms have rarely been investigated (Sukumaran et al., 2011). Our approach goes beyond most studies, because it links socio-psychological norm research with normative deliberation research (Stroud et al., 2015) and research on technical affordances (Evans et al., 2017). First, the distinction between two norm types allows integrating research on moderation and group norms into news comment research. Moreover, we considered the impact of external salience of specific forms of comment presentation and group norms on those who are involved in the discussion. Thus, our research does not normatively demand deliberative quality or empirically describe the degree quality, but specifies conditions for a fulfillment of single deliberative criteria.
Second, the content analysis confirmed previous experimental results (Chen and Lu, 2017; Sukumaran et al., 2011). Thus, this study establishes the external validity of experimental findings with a different method.
Third, our results provide indications for news organizations on how to engage in their user discussions. Visible moderation showed to have a positive impact on the civility of following comments, because other users are reminded of the discussion norms and perceive that they are enforced. In addition, the provision of reasons for deleting comments can improve the transparency of the rules and the reasons for moderation.
Supplemental Material
sj-pdf-1-nms-10.1177_14614448211068104 – Supplemental material for Salient deliberative norm types in comment sections on news sites
Supplemental material, sj-pdf-1-nms-10.1177_14614448211068104 for Salient deliberative norm types in comment sections on news sites by Ines Engelmann, Hanna Marzinkowski and Klara Langmann in New Media & Society
Footnotes
Acknowledgements
We would like to thank Lisa Bock, Charlotte Leikert, Kristin Limmer, Katharina Regneri, Tim Stottmeier, Laura Wodara, and Patrick Zerrer for their various assistance in data collection and transformation.
Authors’ note
All authors have agreed to the submission. The article is not currently being considered for publication by any other print or electronic journal.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) under Grant No. EN 1117/1-1.
Supplemental material
Supplemental material for this article is available online.
Notes
Author biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
