To whom do people reply in comment sections? Effects of attitude (un)congeniality,age,confidence,and knowledge

Abstract

Phenomena like echo chambers and societal polarization have often been linked to an individual preference for like-minded information (selective exposure). This view has been challenged recently: behavior on comment sections in online forums suggests the opposite dynamic, with users more likely to reply to attitudinally uncongenial content. Three experimental studies (total N = 1524) explore boundary conditions of this uncongeniality bias by measuring participants’ tendency to reply to comments on climate science. Studies 1 (student sample) and 2 (non-student sample) replicate the uncongeniality bias. However, Study 3 (representative for age and gender in Germany) yielded a surprising congeniality bias (a preference for replying to congenial comments) which was driven by participants with higher age. Further results across studies suggest that high confidence increases engagement (general likelihood to reply) but reduces antagonism (likelihood to reply to uncongenial content). Conversely, high knowledge reduces engagement but increases the uncongeniality bias.

Keywords

Climate change confidence knowledge selective exposure uncongeniality bias

Theoretical background

Birds of a feather flock together—this famous proverb captures the tendency to surround oneself with like-minded people and content. A rich literature in the social sciences indeed suggests that humans show a preference to expose themselves to information that is in line with their attitudes (congeniality bias; Hart et al., 2009) or to interact with like-minded others (homophily; McPherson et al., 2001). Moreover, this preference for like-mindedness has been linked to toxic phenomena that can be observed on social media, such as echo chambers (Sunstein, 2007) or filter bubbles (Pariser, 2011). It has also been discussed that mutual validation among like-minded others gives rise to polarization (Slater, 2007). Finally, some authors have suggested that attitudinally congenial information is more likely to be believed (Taber and Lodge, 2006), less likely to be scrutinized (Edwards and Smith, 1996), and more likely to be shared (Johnson et al., 2020), even if the information is (known to be) false (Traberg and Van Der Linden, 2022). In sum, many problematic societal phenomena are explained by a congeniality narrative—the tendency to want more of the same.

Festinger (1957) proposed that people avoid dissonant (i.e. challenging) information, and one countermeasure to do so is to surround oneself with consonant (i.e. congenial) information. This idea led to the development of the selective exposure paradigm which has quickly become the hallmark of the congeniality narrative. In a typical selective exposure experiment, a participant is confronted with attitudinally congenial and uncongenial pieces of information and has to choose one or more pieces for further perusal. A meta-analysis has revealed a medium-sized effect known as the congeniality bias (Hart et al., 2009): participants are more likely to select information that is congenial (vs uncongenial) with their attitudes.

While the congeniality narrative (and the selective exposure paradigm at its heart) certainly is well-established, its focus is on citizens as consumers of information whose only observable activity is the choice made among several sources. However, social media environments provide additional affordances: to “like” content, share content, to voice one’s opinion, and to reply to others. It is underresearched whether more interactive forms of social media communication also adhere to principles of the congeniality narrative.

Consider online discussion forums. Most forums (e.g. on news platform comment sections) are structurally equivalent to the setup of a typical selective exposure experiment with users being exposed to congenial and uncongenial content. The congeniality narrative would probably assume that forum users will mutually express their agreement with like-minded others. However, empirical evidence suggests that these discussions frequently lead to heated debates characterized by disagreement and negativity (Efstratiou et al., 2023; Lyu, 2023; Marchal, 2022). While this shows that disagreement (when it occurs) is associated with negative emotions, the question to which content users are likely to reply to in the first place has rarely been addressed. It stands to reason that users might even be more likely to reply to uncongenial (vs congenial) others—an uncongeniality bias of selective response that complements the congeniality bias of selective exposure.

Recent research (Buder et al., 2023) indeed provides evidence for the existence of such an uncongeniality bias. Three experiments used a setup that was otherwise identical to a selective exposure study: participants were confronted with congenial and uncongenial comments and had to make a selection. For instance, when participants should indicate which comment they would like to read more about, a congeniality bias emerged. However, all three studies included a selective response condition in which participants had to indicate their inclination to reply to a comment, and this consistently yielded an uncongeniality bias showing a preference to reply to uncongenial comments. Further analyses indicated that this bias is associated with unfavorable assessments toward the comment that participants replied to; that the uncongeniality bias was more pronounced when participants held a majority view in the discussion; and that larger uncongeniality bias was associated with attitude polarization.

The present research builds on these findings in two respects. First, previous studies were conducted on the same controversial topic (alternative medicine), used material that read and sounded like typical comments on discussion forums (e.g. expressing subjective views), and were conducted with convenience samples of local university students. The present research addresses these shortcomings by testing the generalizability of the uncongeniality bias with a different topic (climate change), different types of contributions (sharing of facts rather than subjective evidence), and different samples. Second, we explored further moderators of the uncongeniality bias. Our prior work has uncovered that attitudinal differences predict the tendency to reply. By extension, it is conceivable that the strength with which an attitude is held may additionally moderate the uncongeniality bias. Attitude strength is a multiply defined and complex construct consisting of various facets (Howe and Krosnick, 2017). We focused on two of those facets as these are also related and relatable to metacognition, a field that is of interest to us (Fischer et al., 2019): topic knowledge and confidence in one’s knowledge, both of which are positively correlated with attitude strength (Howe and Krosnick, 2017).

The concept of confidence is frequently investigated in the field of opinion dynamics. For instance, Hegselmann and Krause (2002) have developed computational models according to which higher confidence thresholds increase the likelihood to interact with others. Based on this notion, our pre-registered hypotheses predicted that having high confidence in one’s knowledge should increase the likelihood to reply. Moreover, we predicted that high confidence increases the uncongeniality bias as it involves publicly disagreeing with an uncongenial comment which requires certainty in the correctness of one’s beliefs. Several lines of research suggest that certainty and confidence determine how people engage with uncongenial information. For instance, high-attitude certainty is associated with resistance to persuasive attacks (Tormala and Rucker, 2007), and emotional conviction has been linked to increased amounts of refutational thought (Edwards and Smith, 1996). It has also been shown that high confidence is associated with approaching (vs avoiding) uncongenial information (Hart et al., 2009). More specifically, high-attitude certainty was found to increase attempts to persuade others to vote for one’s preferred candidate in an upcoming election (Visser et al., 2003). Finally, it has been reported that high certainty in the face of opposition elicits aggression against others which can foreshadow fanaticism (Gollwitzer et al., 2022).

The relation between knowledge and uncongeniality is much less clear. Several findings suggest that high knowledge in a domain is associated with increased social activity, such as value expression (Visser et al., 2016), frequency of posting online comments (Kim et al., 2021), or social influence in public opinion formation (Moussaïd et al., 2013). However, increased activity does not predict whether the response will be attitudinally selective. Extant studies have yielded mixed findings: while Taber and Lodge (2006) reported that high political knowledge increases the congeniality bias, no such relations were found in more recent studies (Jang, 2014). Therefore, we did not predict (but investigated) whether knowledge is moderating the uncongeniality bias, resulting in the following hypotheses.¹

H1. Participants indicate a stronger tendency to reply to an attitudinally uncongenial comment than to an attitudinally congenial comment (uncongeniality bias).

H2. Confidence increases the general tendency to reply (main effect of confidence).

H3. Confidence moderates the effect of attitude stance on the tendency to reply in a way that higher confidence leads to a more pronounced uncongeniality bias whereas lower confidence dampens the uncongeniality bias (interaction between attitudinal distance and confidence).

These hypotheses were tested in three studies, with new material, new comment types, and different samples. The present studies used the contested topic of climate change as a domain. We also changed the tone of conversation for the present studies. While discussion comments in the previous studies were characterized by subjectivity (use of first-person-pronouns, emotionally charged style), the present studies focused on content that was formulated in a matter-of-fact, scientific tone. This constitutes a conservative test for the selective response approach, as it is conceivable that participants will only reply to content when it expresses subjectivity. Study 1 tested the uncongeniality bias with a similar sample as the previous studies (convenience sample of University students); Study 2 then used the same setup, but tested a sample that explicitly excluded current students; and Study 3 explored selective response in a quota-matched sample (for gender and age) of the German population.

Data availability statement

All data, materials, and analysis code (in R) for all three studies are publicly available under: https://doi.org/10.17605/OSF.IO/BKQ4N. All participants provided informed consent. Methods were approved by the ethical review board of the Leibniz-Institut für Wissensmedien (IWM).

Study 1

Study 1 was conducted with several goals in mind. First, we sought to replicate the uncongeniality bias (H1) from previous studies, but with different material (climate change) and different style of comments (scientific tone). Second, we investigated the role of knowledge (exploratory) and confidence in one’s knowledge (pre-registered) as factors that increase the general likelihood to respond (H2) as well as the likelihood to preferentially respond to uncongenial comments (uncongeniality bias, H3).

Method

All hypotheses were pre-registered under https://aspredicted.org/HCW_9KD. Additional analyses are indicated as exploratory.

Participants

The online platforms Prolific and SONA were used to recruit participants. In line with our previous studies, participants consisted of a student sample. The total number of participants who completed the survey was N = 213. After excluding participants who did not complete the questionnaires and did not give their consent to be a part of the study N = 209 participants remained. Furthermore, participants were excluded from the analysis if one of the following statements applied: (1) A standard deviation of zero in the responses to the belief questionnaire (as this would be an indicator of mindless “click-through” behavior). (2) A combined belief score of zero in the first belief questionnaire (as it would be impossible in this case to define comments as congenial or uncongenial). In total, n = 5 participants were excluded. All analyses were performed with the remaining N = 204 participants (109 male, 93 female, 2 diverse). Mean age of participants was M = 23.60 years (SD = 6.18 years).

In terms of highest level of education, participant breakdown was as follows: 0.49% without any degree, 2.45% with an apprenticeship, 55.39% with college level diploma (Abitur), 26.47% with a Bachelor, and 12.74% with an even higher degree.

Procedure

First, participants were informed about the study and had to agree to the requirements (being above 18 years, fluent in German) to further proceed. Afterwards, demographic data were collected, including age, gender, and education level.

The main study was completed in the following order: First, participants’ beliefs about climate change were assessed. Second, participants were asked to fill out a climate change knowledge questionnaire and indicate their confidence in their answer for each knowledge item. Third, 12 comments on climate change were presented twice: (1) to familiarize participants with the range and style of comments, all comments were presented on a single page with 2 minutes for reading; (2) comments were then presented consecutively (randomized), and participants should indicate the tendency to reply to the comment,² serving as the key dependent variable. After completing their ratings for all 12 comments, participants’ beliefs about climate change were assessed again. In a final step, participants had to give their consent on whether they agreed that their data will be evaluated for the study.

Measures

To measure climate change knowledge, 15 statements were taken from Sundblad et al. (2009). Eight statements were true and seven statements were false (see Supplementary Table S1).

To measure confidence in climate change knowledge participants were asked to indicate how confident they were in their answer for each knowledge item. Participants could answer on a 6-point scale ranging from 1 to 6. Each point was labeled (in percentage) from “guessed – 50%” to “absolutely sure – 100%.”

To measure beliefs in climate change participants had to answer five questions tapping into the conviction whether climate change is caused by humans, whether humans are responsible for increasing temperatures, whether climate change will have negative consequences in the near future, whether it is possible that climate change is caused by humans, and whether climate change is risky. Responses on each item ranged from −8 to +8 on 17-point Likert-type scales (later recoded into values from 1 to 17).

To measure uncongeniality bias participants were presented with 12 short comments (~51 words) about climate change (see Supplementary Table S2), all formulated in a non-personal, scientifically sound, and matter-of-fact tone. The comments are based on material from previous studies (Said et al., 2021) and were rated as pro, con, or neutral in a pre-test by N = 50 participants on a 17-point scale ranging from −8 to +8. Comments were then classified based on those ratings as either con (−8 to −2), neutral/ambiguous (−1, 0, +1), or pro (+2, to +8).

To measure tendency to reply, a participant’s response to the following question (translated) was measured: “How much would you like to reply to this comment?” The response options were on a 6-point Likert-type scale ranging from 1 = not at all to 6 = totally.

Analysis

Participants’ knowledge accuracy was calculated by averaging over participants’ responses to the knowledge items; the average of confidence responses and responses to the belief questions yielded a confidence score and a belief score, respectively.

Comments about climate change that were defined as neutral were excluded for the main analysis.

The climate change beliefs of participants (dichotomized as pro or con) and the comments about climate change (pro or con) allowed us to determine the conflict that a given participant is likely to have experienced when reading a given comment. Conflict thus was dichotomized as zero for congenial comments and one for uncongenial comments.

All analyses were performed using R Statistical Software (v 3.6.3; R Core Team, 2020). We calculated linear mixed-effects models using the R-package lme4 (Bates et al., 2015). Linear mixed-effects models were fitted with individual participants and comments (pro vs con) as random effects. For the models, crossed random effects were assumed (Baayen et al., 2008). Specifically, the following model syntax was used for fitting linear mixed-effects:

\begin{matrix} Baseline model : answer ~ conflict + (1 | subject . id) + (1 | comment) . \\ Full model : answer ~ conflict * confidence + conflict * knowledge \\ + (1 | subject . id) + (1 | comment) . \end{matrix}

Results

Descriptive results

Means, standard deviations, and correlations of the main variables are displayed in Supplementary Table S3.

Uncongeniality bias

Using a hierarchical regression approach, results from linear mixed-effect models showed in line with H1 that participants indicated a stronger tendency to reply to an attitudinally uncongenial comment than to a congenial comment (see Table 1). The model with conflict showed a significantly better fit than the baseline model: χ²(1, N = 204) = 63.22, p < .001.

Table 1.

Relationship between conflict and participants’ tendency to reply (Study 1).

Fixed Effects	answer			Answer
Fixed Effects	Estimates	CI	p	Estimates	CI	p
(Intercept)	3.15	2.91 − 3.40	<0.001	3.25	2.98 − 3.52	<0.001
Conflict				0.20	0.04 − 0.36	0.016
Random Effects
σ²	1.87	1.87		1.87	1.87
τ₀₀	0.75_subject.id	0.73_subject.id		0.75_subject.id	0.73_subject.id
τ₀₀	0.02_post	0.11_post		0.02_post	0.11_post
ICC	0.29	0.31		0.29	0.31
N	204 _subject.id			204_subject.id
N	8 _post			8 _post
Observations	1632			1632
Marginal R²/conditional R²	0.004/0.304			0.000/0.311

σ² = variance, τ₀₀ = by-subject random intercept standard deviation, ICC = intraclass correlation coefficient, R² = coefficient of determination. The marginal R² (only fixed effects) and the conditional R² (fixed and random effects) are calculated based on Nakagawa et al. (2017).

Numbers in bold-face express significance at p < .05.

Effects of confidence, knowledge, and interactions with conflict.³

Linear mixed-effect models show a positive main effect of confidence on the tendency to reply. In line with H2, participants were more likely to reply to comments when they felt confidence in their knowledge about climate change. However, in contrast to H3, the uncongeniality bias was not more pronounced for highly confident participants. Exploratory analyses on knowledge indicate a negative main effect of knowledge indicating that participants with high knowledge about climate change were less likely to formulate replies (see Table 2). This effect was qualified by an interaction between conflict and knowledge. Participants with high prior knowledge were exhibiting an uncongeniality bias, but participants with low prior knowledge were particularly likely to respond to attitudinally congenial comments (Figure 1). The full model with interactions showed a significantly better fit than the model with conflict only, χ²(4, N = 204) = 29.10, p < .001.

Table 2.

Main effects and interactions of conflict, confidence, and knowledge on the tendency to reply (Study 1).

Fixed Effects	answer			answer			answer
Fixed Effects	Estim.	CI	P	Estim.	CI	P	Estim.	CI	p
(Intercept)	3.25	3.12 − 3.39	<0.001	3.15	2.91 − 3.40	<0.001	3.12	2.90 − 3.34	<0.001
Conflict				0.20	0.04 − 0.36	0.016	0.26	0.10 − 0.42	0.002
Confidence							0.30	0.13 − 0.47	0.001
Knowledge							-2.42	-3.70 – -1.15	<0.001
Conflict × confidence							0.00	-0.15 – 0.16	0.962
Conflict × knowledge							2.05	0.87 − 3.23	0.001
Random Effects
σ²			1.98		1.87			1.86
τ₀₀			0.72 subject.id		0.73 subject.id			0.66 subject.id
τ₀₀			0.72 subject.id		0.07 post			0.05 post
ICC			0.27		0.30			0.28
N			204 subject.id		204 subject.id			204 subject.id
N			204 subject.id		8 post			8 post
Observations			1632		1632			1632
Marginal R²/conditional R²			0.000 / 0.266		0.004 / 0.304			0.041 / 0.307

Numbers in bold-face express significance at p < .05.

Figure 1.

Interaction between conflict and knowledge (Study 1).

Discussion

The key result of Study 1 is that the uncongeniality bias of selective response that was reported in earlier studies (Buder et al., 2023; Buttliere & Buder, 2017) could be replicated, using a different domain (climate change instead of alternative medicine) and a different tone of discussion (unemotional and matter-of-fact instead of personal and emotional). Moreover, Study 1 investigated the role of confidence as a marker of attitude strength that might moderate the uncongeniality bias. While we found that highly confident participants were more likely to write replies in general, they were not more likely to express disagreement with uncongenial others. Conversely, we found that participants with high knowledge were less likely to reply overall but exhibited an uncongeniality bias.

Overall, Study 1 extends our knowledge about selective response and the uncongeniality bias. We found that the uncongeniality bias can be replicated in a different context, and we also yielded first evidence that knowledge about a topic (rather than confidence about that knowledge) is an important moderator of the uncongeniality bias. To further extend our findings, Study 2 attempted to replicate results from Study 1, but on a different sample.

Study 2

So far, the uncongeniality bias has only been reported for relatively young and educated samples of university students. Arguing with dissenting views is a skill that might be more prevalent among educated participants in academia, therefore reporting an uncongeniality bias in a non-student sample would provide a generalization of our previous findings. Therefore, Study 2 attempted to replicate our findings in a (convenience) sample that was explicitly comprised non-students.

Hypotheses were identical to Study 1: we expected a general uncongeniality bias (H1), a main effect of confidence (H2) as well as an interaction between conflict and confidence, suggesting that high confidence will increase the uncongeniality bias (H3).

Method

All hypotheses were pre-registered under https://aspredicted.org/8RQ_KWM. Additional analyses are indicated as exploratory.

Participants

The online platform Prolific was used to recruit the participants for study 2. To address the limitation in study 1 participants consisted of a non-student sample. Sample size for the second study was calculated based on our previous study using a data simulation of mixed model effects, resulting in a sample size of N = 230. To account for data exclusions and participant dropout, we aimed at N = 250 participants. All 250 participants gave their consent for data processing and only one participant had to be excluded based on pre-registered exclusion criteria, resulting in N = 249 (135 male, 109 female, 4 diverse, 1 undisclosed). Mean age of participants was M = 33.63 years (SD = 11.01 years).

In terms of highest level of education, participant breakdown was as follows: 0.80% without any degree, 11.65% with an apprenticeship, 21.69% with college level diploma (Abitur), 25.70% with a Bachelor, and 39.36% with an even higher degree (0.40% did not disclose).

Procedure

Procedure for Study 2 was identical to Study 1 with one exception: the knowledge test was presented at the beginning of the study for half of the participants and at the end for the other half of participants. Even though there was a difference regarding performance in the knowledge test between those who received the knowledge test at the beginning of the study and those who received it at the end, M_pr_e = 0.79, SD_pre = 0.11; M_post = 0.75, SD_post = 0.14, t(234.05) = 2.71, p = .007, we decided to collapse the data into a single analysis (to avoid being underpowered).

Measures and analyses

Measures and analyses were identical to those in Study 1.

Results

Descriptive results

Means, standard deviations, and correlations of the main variables are displayed in Supplementary Table S4.

Uncongeniality bias

Employing a hierarchical regression, results of linear mixed-effect models showed that participants indicated a stronger tendency to reply to uncongenial (vs congenial) comments, replicating the results from Study 1 for a non-student sample and confirming H1 (see Table 3). The model including conflict showed a significantly better fit than the baseline model: χ²(1, N = 249) = 4.30, p = .038.

Table 3.

Relationship between conflict and participants’ tendency to reply (Study 2).

Fixed Effects	answer			answer
Fixed Effects	Estimates	CI	p	Estimates	CI	p
(Intercept)	3.21	3.04 − 3.37	<0.001	3.10	2.94 − 3.26	<0.001
Conflict				0.21	0.04 − 0.38	0.015
Random Effects
σ²	2.05			2.05
τ₀₀	0.65 subject.id			0.65 subject.id
τ₀₀	0.03 post			0.01 post
ICC	0.25			0.24
N	249 subject.id			249 subject.id
N	8 post			8 post
Observations	1992			1992
Marginal R² / Conditional R²	0.000 / 0.249			0.006 / 0.248

Numbers in bold-face express significance at p < .05.

Effects of confidence, knowledge, and interactions with conflict

Using linear mixed-effect models, neither H2 nor H3 could be confirmed. Higher confidence did not increase the likelihood to reply to comments (H2), nor specifically increased the likelihood to reply to uncongenial comments (H3). In contrast, exploratory analyses revealed similar results to Study 1 with respect to knowledge. The full model (Table 4, rightmost column) showed a significantly better fit than the model with conflict only, χ²(4, N = 249) = 26.07, p < .001. Again, participants with high climate change knowledge were less likely to reply in general but simultaneously exhibited a stronger uncongeniality bias than those with low knowledge (who were actually exhibiting a congeniality bias; Figure 2).

Table 4.

Main effects and interactions of conflict, confidence, and knowledge on the tendency to reply (Study 2).

Fixed Effects	answer			Answer			answer
Fixed Effects	Estim.	CI	p	Estim.	CI	p	Estim.	CI	p
(Intercept)	3.21	3.09 − 3.32	<0.001	3.10	2.94 − 3.26	<0.001	3.08	2.93 − 3.24	<0.001
conflict				0.21	0.04 − 0.38	0.015	0.24	0.08 − 0.41	0.004
Confidence							0.13	−0.02 − 0.29	0.087
Knowledge							−1.72	−2.80 – −0.65	0.002
Conflict × confidence							−0.00	−0.15 – 0.14	0.978
Conflict × knowledge							2.36	1.35 − 3.37	<0.001
Random effects
σ²	2.07			2.05			2.03
τ₀₀	0.65 subject.id			0.65 subject.id			0.64 subject.id
τ₀₀	0.65 subject.id			0.01 post			0.01 post
ICC	0.24			0.24			0.24
N	249 subject.id			249 subject.id			249 subject.id
N	249 subject.id			8 post			8 post
Observations	1992			1992			1992
Marginal R² / Conditional R²	0.000 / 0.238			0.004 / 0.246			0.019 / 0.258

Numbers in bold-face express significance at p < .05.

Figure 2.

Interaction between conflict and knowledge (Study 2).

Discussion

The main goal of Study 2 was to demonstrate the existence of an uncongeniality bias for a non-student sample. Again, we found that participants were more likely to indicate a tendency to reply to uncongenial (vs congenial) comments, thus replicating the general uncongeniality bias reported in previous work (Buder et al., 2023). We also investigated the role that confidence and knowledge played in moderating this uncongeniality bias. Looking at interaction effects only, we (again) did not find that higher confidence in climate change knowledge was associated with a tendency to reply to uncongenial comments (contrary to H3). Apparently, confidence does not spill over into a tendency to express disagreement with uncongenial views, thus having no effect on discussion behavior. In contrast, it seems that (high) climate change knowledge increased the uncongeniality bias.

Study 2 helps to further elucidate boundary conditions of an uncongeniality bias. Together with Study 1, it provides evidence that the uncongeniality bias is small but robust. The role of confidence is only partially in line with our predictions. Confidence might increase the tendency to reply (though this was not significant in Study 2), but it does not increase the tendency to selectively respond to uncongenial comments. Conversely, knowledge has an antagonizing role in making people more likely to express disagreement with others. To further disentangle these effects, and in order to further explore the generalizability of the uncongeniality bias, Study 3 was conducted on a larger, quota-matched sample (for age and gender) of the German population.

Study 3

Study 3 was identical in setup to Study 2. The only difference is the sample under investigation, moving from a convenience sample to a large, quota-matched sample of the German population that is representative with respect to age and gender. For consistency, we kept the pre-registered hypotheses from Studies 1 and 2 according to which we expected an uncongeniality bias (H1), a main effect of confidence on the tendency to reply (H2), and an interaction between confidence and conflict (H3).

Method

All hypotheses were pre-registered under https://aspredicted.org/HCH_67Q. Additional analyses are indicated as exploratory.

Participants

For the third study, the online platform respondi was used to recruit the participants. The sample was representative for the German population in terms of age and gender. In total, N = 1121 participants were recruited. After excluding participants who did not complete the questionnaires and did not give their consent to be a part of the study N = 1086 participants remained. 20 more participants were excluded based on pre-registered criteria, thus yielding N = 1066 participants (531 male, 531 female, 4 diverse). Mean age of participants was M = 46.2 years (SD = 15.49 years). In terms of highest level of education, participant breakdown was as follows: 0.94% without any degree, 37.99% with an apprenticeship, 22.98% with college level diploma (Abitur), 13.60% with a Bachelor, and 20.45% with an even higher degree (0.47% did not disclose).

Procedure

Procedure for Study 3 was identical to Study 2.

Measures and analysis

Measures and analysis were identical to those in Study 2.

Results

Descriptive results

Means, standard deviations, and correlations of the main variables are displayed in Supplemental Table S5.

Uncongeniality bias

Results of linear mixed-effect models showed that contrary to H1 participants indicated a stronger tendency to reply to congenial than uncongenial comments, thus showing an unexpected congeniality bias in selective response (Table 5). The model including conflict showed a significantly better fit than the baseline model: χ²(1, N = 1066) = 61.85, p < .001.

Table 5.

Relationship between conflict and participants’ tendency to reply (Study 3).

Fixed effects	answer			answer
Fixed effects	Estimates	CI	p	Estimates	CI	p
(Intercept)	2.68	2.56 − 2.79	<0.001	2.84	2.72 − 2.97	<0.001
Conflict				−0.33	−0.42 – −0.25	<0.001
Random effects
σ²	1.36			1.35
τ₀₀	1.51 subject.id			1.51 subject.id
τ₀₀	0.02 post			0.06 post
ICC	0.53			0.54
N	1066 subject.id			1066 subject.id
N	8 post			8 post
Observations	8528			8528
Marginal R² / Conditional R²	0.000 / 0.529			0.010 / 0.536

Numbers in bold-face express significance at p < .05.

Effects of confidence, knowledge, and interactions with conflict

Results of linear mixed-effect models show in line with H2 that higher confidence was associated with an increased tendency to reply. This time, results yielded an interaction between conflict and confidence, however, the effect was not in line with H3. We found that participants with high confidence in their climate change knowledge were even less likely to reply to uncongenial and conflicting comments than participants with low confidence (Figure 3). An exploratory analysis of the role of knowledge showed that higher knowledge was related to a lower tendency to reply overall, a finding that is fully consistent with Studies 1 and 2. Finally, knowledge did not moderate the impact of conflict on the tendency to reply (see Table 6). The full model including interactions showed a significantly better fit than the model with conflict only, χ²(4, N = 1066) = 131.33, p < .001.

Figure 3.

Interaction between conflict and confidence (Study 3).

Table 6.

Main effects and interactions of conflict, confidence, and knowledge on the tendency to reply (Study 3).

Fixed Effects	answer			answer			answer
Fixed Effects	Estim.	CI	p	Estim.	CI	p	Estim.	CI	p
(Intercept)	2.68	2.60 − 2.75	<0.001	2.84	2.72 − 2.97	<0.001	2.84	2.71 − 2.96	<0.001
Conflict				−0.33	−0.42 – −0.25	<0.001	−0.32	−0.40 – −0.23	<0.001
Confidence							0.37	0.31 − 0.44	<0.001
Knowledge							−1.56	−2.13 – −1.00	<0.001
conflict × confidence							−0.14	−0.18 – −0.09	<0.001
conflict × knowledge							0.29	−0.08 – 0.67	0.126
Random effects
σ²	1.37			1.35			1.34
τ₀₀	1.51 subject.id			1.51 subject.id			1.38 subject.id
τ₀₀	1.51 subject.id			0.02 post			0.02 post
ICC	0.52			0.53			0.51
N	1066 subject.id			1066 subject.id			1066 subject.id
N	1066 subject.id			8 post			8 post
Observations	8528			8528			8528
Marginal R² / Conditional R²	0.000 / 0.523			0.010 / 0.536			0.059 / 0.538

Numbers in bold-face express significance at p < .05.

Further exploratory analyses

While Studies 1 and 2 showed the same uncongeniality bias that has been reported in previous work (Buder et al., 2023), Study 3 showed an overall congeniality bias in a large, quota-matched sample. Therefore, exploratory analyses looked into differences between the samples. One potential candidate was education level. However, while education level was interacting with conflict across all three studies, the effects were highly inconclusive (e.g. higher education was associated with reduced antagonism in Study 2 and increased antagonism in Study 3; see Supplementary Table S6a and S6b). The other major difference between the three studies was the average age of participants (M_Study1 = 23.60 years, M_Study2 = 33.63 years, M_Study3 = 46.2 years). To assess whether age does play a moderating role, we included it in the linear mixed-effect model analysis. Results show that age linearly affected whether or not participants were displaying an uncongeniality bias or congeniality bias. While younger participants indeed exhibited a small uncongeniality bias, this was overshadowed by a strong congeniality bias for older participants. However, testing for a quadratic effect yielded a significantly better fit, χ²(2, N = 1066) = 13.83, p < .001 (Table 7). As shown in Figure 4, an uncongeniality bias occurred in participants 20–30 years of age while both younger and older participants exhibited a congeniality bias. We also ran a corresponding re-analysis of Study 2 which showed that linear (but not quadratic) age interacted with conflict (Supplementary Table S7). Therefore, we tend to interpret and discuss our findings with regard to linear age. Moreover, age did not additionally moderate the relationships between conflict, confidence, and knowledge in Studies 2 or 3 (Supplementary Tables 8a and 8b).

Figure 4.

Interaction between conflict and age.

Table 7.

Relationship between conflict and tendency to reply, with age as moderator variable (Study 3).

	answer			answer
Fixed effects
Predictors	Estimates	CI	p	Estimates	CI	p
(Intercept)	2.84	2.71 − 2.96	<0.001	2.90	2.75 − 3.06	<0.001
Conflict	−0.32	−0.41 – −0.24	<0.001	−0.42	−0.52 – −0.32	<0.001
c age	0.01	0.01 − 0.02	<0.001	0.01	0.01 − 0.02	<0.001
conflict × c age	−0.01	−0.02 – −0.01	<0.001	−0.01	−0.02 – −0.01	<0.001
c age^2				−0.00	−0.00 – 0.00	0.142
conflict × c age^2				0.00	0.00 − 0.00	<0.001
Random effects
σ²	1.33			1.33
τ₀₀	1.51 subject.id			1.51 subject.id
τ₀₀	0.02 post			0.02 post
ICC	0.53			0.53
N	1066 subject.id			1066 subject.id
N	8 post			8 post
Observations	8528			8528
Marginal R² / conditional R²	0.015 / 0.541			0.015 / 0.542

Numbers in bold-face express significance at p < .05.

Discussion

Study 3 sought to investigate the tendency to reply to comments with a quota-matched sample (age and gender) of the German population. In contrast to both previous work as well as Studies 1 and 2, this was the first time that we found an overall preference of participants to reply to congenial rather than uncongenial comments. Further analyses of both Studies 2 and 3 showed that the different effects across studies could most consistently be attributed to linear age. While younger participants displayed a small uncongeniality bias this was outweighed by a pronounced congeniality bias in older participants. Study 3 also investigated the role of confidence and knowledge on the tendency to reply. Consistent with the patterns of Studies 1 and 2, being confident in one’s climate change knowledge increased the likelihood to reply. In contrast to previous studies, however, confidence in Study 3 was associated with selective response behavior. Highly confident participants were particularly likely to reply to low-conflict (congenial) comments, indicating a relatively defensive stance. As to the role of knowledge, Study 3 again found that knowledge reduces the tendency to reply overall. However, knowledge did not affect the tendency to selectively reply to congenial or uncongenial comments.

General discussion

A vast body of research has investigated the question of how individuals consume information on social media, often guided by findings from the selective exposure literature. However, there has been much less work on how individuals express information on social media. Using a similar experimental design as in the selective exposure paradigm recent work (Buder et al., 2023) has revealed that individuals preferentially expose themselves to congenial information but also preferentially reply to uncongenial information (uncongeniality bias). The present work builds on these recent findings and further explored conditions under which the uncongeniality bias can (or cannot) be found.

Results indicate that even for a different topic and a different tone of discussion the uncongeniality bias of selective response could be replicated (though with small effects) in studies on convenience samples of students and non-students. Strikingly, however, when investigating the most representative sample, the uncongeniality bias did not only disappear but actually reversed: participants were more likely to respond to congenial than to uncongenial comments. On one hand, this shows the limitations of many social science studies to rely on convenience samples of university students. On the other hand, results of Study 3 should not be taken to invalidate the uncongeniality bias in toto. As the key difference among the three samples was the average age (but not average level of education) of participants, it appears that an uncongeniality bias can be observed mostly among younger participants whereas older participants exhibit a congeniality bias of selective response. Several explanations may explain the age effects. First, higher age is linked to lower usage of social media (Perrin, 2015). It could be argued that the affordances of social media (consumption as well as expression) are more salient among younger participants in a way that they are more inclined to engage with different opinions. Second, the topic under investigation might also lead to different kinds of selective response. While climate change certainly affects entire societies, a meta-analysis has been reported that beliefs in anthropogenic climate change are negatively correlated with age (Hornsey et al., 2016). It could be the case that younger participants are more engaged with the topic of climate change, making them more likely to express disagreement with uncongenial views. Third, an association of higher age with congeniality would be fully in line with findings from lifespan psychology. According to socioemotional selectivity theory (Carstensen et al., 1999), older age is associated with a stronger focus on positive emotional encounters whereas younger age can be linked to a stronger desire to acquire new information. In this light, engaging with uncongenial others should be more rewarding with younger age, but more aversive with older age.

A second major goal of the present research was to investigate the moderating role of markers of attitude strength on selective response, in particular, the confidence that participants had in their knowledge about climate change as well as the amount of knowledge itself. Based on prior work (e.g. Tormala and Rucker, 2007), we hypothesized that confidence would not only increase the tendency to reply overall, but also specifically to reply to uncongenial comments. Results of the three studies were relatively uniform. Indeed, all studies showed that high confidence increased the tendency to reply (though not significantly so in Study 2), suggesting a higher amount of engagement. However, none of the three studies found that confidence increased the tendency to selectively reply to uncongenial comments; in fact, Study 3 even showed that high confidence increased the tendency to reply to congenial (vs uncongenial) comments. This finding is somewhat puzzling, and at present, we can only speculate about this effect. High confidence has often been linked with openness in a way that individuals are more likely to read uncongenial views, and a meta-analysis on selective exposure confirmed this effect (Hart et al., 2009). However, it would also be conceivable that highly confident persons see no need to further expose themselves to or engage with uncongenial views as they do not expect to gain new insights from those views. Further research is needed to explore these effects.

Interestingly, the findings for the role of knowledge provide evidence for the exact opposite. In all three studies, we find that having high climate change knowledge decreased the willingness to interact with others. However, particularly the data from Studies 1 and 2 suggest that possessing high knowledge about climate change made participants more antagonistic (i.e. increased the uncongeniality bias). The interesting disassociation between confidence (increasing engagement, reducing antagonism) and knowledge (reducing engagement, increasing antagonism) certainly requires further exploration in future studies.

The three studies contribute to our understanding of behavior in social media environments. While the majority of work in this field argues that this behavior is driven by congeniality and like-mindedness, there is an increasing literature suggesting that uncongeniality and conflict may also play an important role: uncongenial information cannot be avoided (Wojcieszak and Mutz, 2009) nor will it be avoided (Fletcher and Nielsen, 2017); disagreeing with others has been linked to the experience of cognitive dissonance (Matz and Wood, 2005) or perceived norm violations (Hsu et al., 2021) which may lead to replies that use offensive speech (Lyu, 2023); there is also evidence that experiencing uncongenial information may play a role in attitude polarization (Bail et al., 2018). Our studies add to this recent trend by suggesting that certain online venues (e.g. comment sections on news web pages) are better characterized by selective response than selective exposure. Moreover, our studies explore the role of knowledge and confidence in shaping patterns of selective response. Of course, selective response and behavior in actual discussion forums are also dependent on other factors, such as the (potentially biased) perception of social norms which is typically shaped by patterns of “likes” or “dislikes” of forum comments (Luzsa and Mayr, 2021). In a broader sense, patterns of selective response can be viewed through the lens of the Theory of Planned Behavior (Ajzen, 1991) which holds that behavioral intentions are shaped by attitudes (in our case, uncongenial attitudes), subjective norms, and perceived behavioral control.

The present research comes with a number of limitations. First, on a conceptual level, we focused on knowledge and confidence as markers of attitude strength (Howe and Krosnick, 2017), and one may argue that these facets are less defining for attitude strength than other markers like attitude importance (Boninger et al., 1995) or moral conviction (Skitka et al., 2021). We opted for knowledge and confidence as this would in principle have allowed us to investigate our data through the lens of metacognition and signal detection theory (Fischer et al., 2019), an approach that would have made our present investigation overfraught. Second, our measure of confidence was referring to confidence about the correctness of one’s knowledge. This is different from confidence about one’s attitude, and looking into the latter variable (also termed attitude certainty, Howe and Krosnick, 2017) might have yielded different results and must be open for future investigations. Third, like in most of our previous studies, we did not have participants formulate actual replies and rather measured the intention to reply. Fourth, our intention was to extend the generalizability of the uncongeniality bias. To this end, we took a very conservative stance by using comments as material that lack the personal (and often emotional) flavor of actual comments on social media. This would have been unproblematic if we would have yielded the hypothesized uncongeniality bias in all studies. That we found a congeniality bias (particularly among older participants) in Study 3 thus leaves the question unresolved if that pattern would have replicated with personally and emotionally loaded comments that are more typical of actual forum discussion.

In spite of these shortcomings, we believe that further investigation of selective response as a complement to selective exposure is a worthwhile pursuit for future research. Explorations of selective response are still in a nascent stage, and there are many more boundary conditions to be explored. They can make a valuable contribution in better understanding how people express their views (rather than consume information) on social media.

Supplemental Material

sj-docx-1-nms-10.1177_14614448241247214 – Supplemental material for To whom do people reply in comment sections? Effects of attitude (un)congeniality, age, confidence, and knowledge

Supplemental material, sj-docx-1-nms-10.1177_14614448241247214 for To whom do people reply in comment sections? Effects of attitude (un)congeniality, age, confidence, and knowledge by Jürgen Buder and Nadia Said in New Media & Society

Footnotes

Acknowledgements

The authors thank Nicole Antes for supporting the development of the climate change comments.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jürgen Buder

Supplemental material

Supplemental material for this article is available online.

Notes

Author biographies

Jürgen Buder is a researcher at the Leibniz-Institut für Wissensmedien in Tübingen. His current key research interests are toxic online phenomena as well as psychological investigations of emergent technologies (e.g. AI).

Nadia Said is a postdoc at the Applied Cognitive Psychology research group, University of Tübingen. Her research is centered around understanding the (meta-)cognitive underpinnings of belief polarization as well as climate change action, processing and dissemination of (mis)information, and perceptions of and interaction with artificial intelligence.

References

Ajzen

(1991) The theory of planned behavior. Organizational Behavior and Human Decision Processes 50(2): 179–211.

Baayen

Davidson

Bates

(2008) Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language 59(4): 390–412.

Bail

Argyle

Brown

, et al. (2018) Exposure to opposing views on social media can increase political polarization. Proceedings of the National Academy of Sciences 115(37): 9216–9221.

Bates

Maechler

Bolker

, et al. (2015) Fitting linear mixed-effects models using lme4. Journal of Statistical Software 67(1): 1–48.

Boninger

Krosnick

Berent

, et al. (1995) The causes and consequences of attitude importance. In: Petty

Krosnick

(eds) Attitude Strength: Antecedents and Consequences. New York: Psychology Press, pp. 159–189.

Buder

Zimmermann

Buttliere

, et al. (2023) Online interaction turns the congeniality bias into an uncongeniality bias. Psychological Science 34(10): 1055–1068.

Buttliere

Buder

(2017) Reading more vs. writing back: situation affordances drive reactions to conflicting information on the internet. Computers in Human Behavior 74: 330–336.

Carstensen

Isaacowitz

Charles

(1999) Taking time seriously: a theory of socioemotional selectivity. American Psychologist 54(3): 165–181.

Edwards

Smith

(1996) A disconfirmation bias in the evaluation of arguments. Journal of Personality and Social Psychology 71(1): 5–24.

10.

Efstratiou

Blackburn

Caulfield

, et al. (2023) Non-polar opposites: analyzing the relationship between echo chambers and hostile intergroup interactions on Reddit. Proceedings of the International AAAI Conference on Web and Social Media 17(1): 197–208.

11.

Festinger

(1957) A Theory of Cognitive Dissonance. Stanford, CA: Stanford University Press.

12.

Fischer

Amelung

Said

(2019) The accuracy of German citizens’ confidence in their climate change knowledge. Nature Climate Change 9(10): 776–780.

13.

Fletcher

Nielsen

(2017) Are news audiences increasingly fragmented? A cross-national comparative analysis of cross-platform news audience fragmentation and duplication. Journal of Communication 67(4): 476–498.

14.

Gollwitzer

Olcaysoy Okten

Pizarro

, et al. (2022) Discordant knowing: a social cognitive structure underlying fanaticism. Journal of Experimental Psychology: General 151(11): 2846–2878.

15.

Hart

Albarracín

Eagly

, et al. (2009) Feeling validated versus being correct: a meta-analysis of selective exposure to information. Psychological Bulletin 135(4): 555–588.

16.

Hegselmann

Krause

(2002) Opinion dynamics and bounded confidence: models, analysis and simulation. Journal of Artificial Societies and Social Simulation 5(3).

17.

Hornsey

Harris

Bain

, et al. (2016) Meta-analyses of the determinants and outcomes of belief in climate change. Nature Climate Change 6(6): 622–626.

18.

Howe

Krosnick

(2017) Attitude strength. Annual Review of Psychology 68: 327–351.

19.

Hsu

Niiya

Thelwall

, et al. (2021) Social media users produce more affect that supports cultural values, but are more influenced by affect that violates cultural values. Journal of Personality and Social Psychology 121(5): 969–983.

20.

Jang

(2014) Seeking congruency or incongruency online? Examining selective exposure to four controversial science issues. Science Communication 36(2): 143–167.

21.

Johnson

Neo

Heijnen

, et al. (2020) Issues, involvement, and influence: effects of selective exposure and sharing on polarization and participation. Computers in Human Behavior 104: 106155.

22.

Kim

Guess

Nyhan

, et al. (2021) The distorting prism of social media: how self-selection and exposure to incivility fuel online comment toxicity. Journal of Communication 71(6): 922–946.

23.

Luzsa

Mayr

(2021) False consensus in the echo chamber: exposure to favorably biased social media news feeds leads to increased perception of public support for own opinions. Cyberpsychology: Journal of Psychosocial Research on Cyberspace 15(1): 3.

24.

Lyu

(2023) Cross-cutting interaction, inter-party hostility, and partisan identity: analysis of offensive speech in social media. New Media & Society. Epub ahead of print 23 June. DOI: 10.1177/14614448231180654.

25.

McPherson

Smith- Lovin

Cook

(2001) Birds of a feather: homophily in social networks. Annual Review of Sociology 27(1): 415–444.

26.

Marchal

(2022) “Be nice or leave me alone”: an intergroup perspective on affective polarization in online political discussions. Communication Research 49(3): 376–398.

27.

Matz

Wood

(2005) Cognitive dissonance in groups: the consequences of disagreement. Journal of Personality and Social Psychology 88(1): 22–37.

28.

Moussaïd

Kämmer

Analytis

, et al. (2013) Social influence and the collective dynamics of opinion formation. PLoS One 8(11): e78433.

29.

Nakagawa

Johnson

Schielzeth

(2017) The coefficient of determination R2 and intra-class correlation coefficient from generalized linear mixed-effects models revisited and expanded. Journal of the Royal Society Interface 14(134): 20170213.

30.

Pariser

(2011) The Filter Bubble: What the Internet Is Hiding from You. New York: Penguin Books.

31.

Perrin

(2015) Social networking usage: 2005-2015. Available at: http://www.pewinternet.org/2015/10/08/2015/Social-Networking-Usage-2005-2015/ (accessed 30 October 2023).

32.

R Core Team (2020) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Available at: https://www.R-project.org/

33.

Said

Fischer

Anders

(2021) Contested science: individuals with higher metacognitive insight into interpretation of evidence are less likely to polarize. Psychonomic Bulletin & Review 29: 668–680.

34.

Skitka

Hanson

Morgan

, et al. (2021) The psychology of moral conviction. Annual Review of Psychology 72: 347–366.

35.

Slater

(2007) Reinforcing spirals: the mutual influence of media selectivity and media effects and their impact on individual behavior and social identity. Communication Theory 17(3): 281–303.

36.

Sundblad

Biel

Gärling

(2009) Knowledge and confidence in knowledge about climate change among experts, journalists, politicians, and laypersons. Environment and Behavior 41(2): 281–302.

37.

Sunstein

(2007) Republic.com 2.0. Princeton, NJ: Princeton University Press.

38.

Taber

Lodge

(2006) Motivated skepticism in the evaluation of political beliefs. American Journal of Political Science 50(3): 755–769.

39.

Tormala

Rucker

(2007) Attitude certainty: a review of past findings and emerging perspectives. Social and Personality Psychology Compass 1(1): 469–492.

40.

Traberg

Van Der Linden

(2022) Birds of a feather are persuaded together: perceived source credibility mediates the effect of political bias on misinformation susceptibility. Personality and Individual Differences 185: 111269.

41.

Visser

Krosnick

Norris

(2016) Attitude importance and attitude-relevant knowledge. In: Krosnick

Chiang

ICA

Stark

(eds) Political Psychology: New Explorations. Routledge, pp. 203–245.

42.

Visser

Krosnick

Simmons

(2003) Distinguishing the cognitive and behavioral consequences of attitude importance and certainty: a new approach to testing the common-factor hypothesis. Journal of Experimental Social Psychology 39(2): 118–141.

43.

Wojcieszak

Mutz

(2009) Online groups and political discourse: do online discussion spaces facilitate exposure to political disagreement? Journal of Communication 59(1): 40–56.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB

To whom do people reply in comment sections? Effects of attitude (un)congeniality,age,confidence,and knowledge

Abstract

Keywords

Theoretical background

Data availability statement

Study 1

Method

Participants

Procedure

Measures

Analysis

Results

Descriptive results

Uncongeniality bias

Effects of confidence, knowledge, and interactions with conflict. 3

Discussion

Study 2

Method

Participants

Procedure

Measures and analyses

Results

Descriptive results

Uncongeniality bias

Effects of confidence, knowledge, and interactions with conflict

Discussion

Study 3

Method

Participants

Procedure

Measures and analysis

Results

Descriptive results

Uncongeniality bias

Effects of confidence, knowledge, and interactions with conflict

Further exploratory analyses

Discussion

General discussion

Supplemental Material

sj-docx-1-nms-10.1177_14614448241247214 – Supplemental material for To whom do people reply in comment sections? Effects of attitude (un)congeniality, age, confidence, and knowledge

Footnotes

Acknowledgements

Funding

ORCID iD

Supplemental material

Notes

Author biographies

References

Supplementary Material

Effects of confidence, knowledge, and interactions with conflict.³