User information processing mechanisms for news quality judgment conformity to professional standards: Comparing combinations of news content/formal cue processing

Abstract

It is more difficult to judge news quality on digital news platforms because editorial cues (such as size and placement of news articles that signal the quality of articles in the traditional news environment) are far less obvious. Without these editorial cues, how do users process news cues to judge news quality in conformity to professional standards? Relying on dual information processing literature, this study investigates five combinations of news content/formal cue processing to identify user information processing mechanisms for news quality judgment conformity to professional standards. A total of 88 news articles were evaluated by 3547 survey respondents and two professional editors. Based on the partial-least-squares structural equation modeling, we found that the joint functioning of content/formal cue processing better explains news quality judgment conformity than other combinations (such as the independent functioning of each cue processing and the biased functioning of content cue processing affected by formal cue processing). The large, negative effect of joint functioning suggests that the less the respondents relied on formal cues, the greater they achieved news quality judgment conformity as they elaborated more on content cues. Elaboration on a given article’s believability/depth as a content cue and heuristics regarding its number of quotes as a formal cue had greater impact on judgment conformity. These results imply how the elaborative and heuristic routes of news processing interact and in what ways news cues can be processed to identify quality news which is necessary for democratic decision making.

Keywords

Digital news news quality content/formal cues judgment professional standards dual information processing

News distribution, consumption, and even production in the digital news environment are being increasingly influenced by the news choices of the public (Ferrer-Conill and Tandoc, 2018; Welbers et al., 2016)¹. Under this circumstance, the gap between journalists/editors and the public, especially in terms of judging news quality, deserves more scholarly attention. Here, the judgment of news quality refers to assessing the extent to which news articles accomplish quality journalism through the observation of journalistic norms. Because political deliberation is based on news consumption, it is a concern if the public assess low-quality news articles to be of high quality as democratic decisions based on such articles could have a detrimental impact on society. In this regard, it is important to identify the factors that contribute to the public’s news quality judgment in conformity to professional standards. In particular, users on digital news platforms have to judge news article quality based on fewer cues compared with the traditional print news environment, where news articles are bundled with the clearly marked brand name of a particular news organization and news articles’ layout, position, and size in the print-version newspaper reflect editorial importance. For example, news articles regarded as important are placed on the front page and in larger size (Feeley et al., 2016: 495; Maier, 2010: 14). In the absence of such editorial cues, content and form of an individual news article have become primary cues for digital news platform users to judge news quality. However, little is known about how the processing of these cues contributes to user news quality judgment conformity to professional standards.

The aim of the present study is to identify how each individual’s processing of content cues contributes to one’s news quality judgment in conformity to professional standards and how the content cue processing interacts with formal cue processing. Content cues indicate what is extracted based on the comprehension of a news article content (e.g. how objective the news story is), whereas formal cues indicate what is explicitly revealed in a news article that renders it in the form of a news article (e.g. whether the news article contains a byline). Our investigation is based on text analysis of news articles, rating of professional editors, and survey results of news users in the context of South Korea, where the digital news environment comprises a well-equipped digital infrastructure and a majority (79.8%) of the nation consuming news online (Korea Press Foundation, 2021) (see Online Appendix 1). Our previous studies based on this context found that South Korean users count for normative journalistic values when judging news quality (Choi et al., 2021) and their judgment of news quality resembles that of professionals (Choi, 2021) (see Online Appendix 2). Developing from these studies, we explore the information processing mechanisms that enhance judgment of news quality in conformity to professional standards by addressing the following questions (see Online Appen dix 3): Would the processing of content cues increase the likelihood of judgment conformity to professional standards? During this process, what role do formal cues play? Do both cues have positive effects on judgment conformity? Would formal cue processing influence content cue processing and would the bias created as a result influence judgment conformity? Or is there any interaction between these two? Which cues have a greater explanatory power?

To address these questions, we apply a dual information processing model. A number of different labels have been applied to this model, as follows: central/peripheral (Petty and Cacioppo, 1986), systematic/heuristic (Chaiken, 1980), rational/experiential (Epstein, 1994), reflective/reflexive (Lieberman, 2003), and reflective/impulsive (Strack and Deutsch, 2004). The former of these dual terms (i.e. central, systematic, rational, reflective) have commonly inferred effortful, reflective, elaborative, and deliberative processing, while the latter (i.e. peripheral, heuristic, experiential, reflexive, impulsive) rapid and automatic processing (Evans, 2008). In this vein, content cues embedded in news articles may require more time and cognitive resources to process compared to formal cues. Based on the framework of the dual processing model, we can presume that elaborative processing may enhance judgment conformity in line with professional standards; thus, news article content cue processing may better explain judgment conformity, but this presumption has not been tested. This study may contribute to identifying the underlying mechanism of news quality judgment conformity to professional standards, and furthermore, provide implications for the dual information processing literature revealing how the two information processing routes interact.

Literature review

News quality judgment conformity to professional standards

Journalism researchers have used the notion of news quality interchangeably with news excellence, quality journalism, and good journalism, without clearly defining what it means. By considering the conceptual accounts of journalism literature, it can be assumed that news articles adhering to journalistic norms such as objectivity, diversity, accuracy, and depth are of high quality (Gladney et al., 2007; Kovach and Rosenstiel, 2007; Sambrook, 2012; Steele, 2008) and that quality news addresses matters of general political and social significance (Zaller, 1999). In empirical studies of news consumption literature, researchers operationalized news evaluation as an assessment of how objective, informative, and well-written a news article is (Kim, 2015; Lee and Kim, 2016). Both streams of literature have regarded journalistic norms as primary constituents of quality journalism. Journalistic norms have also served as a basis for the public to judge news quality. Several studies have shown that users value journalism that promotes deeper understanding of local concerns (Costera Meijer & Bijleveld, 2016) and, in some cases, value journalistic norms even more than editors do (Gladney, 1996). Indeed, journalistic values had a greater predictive power of user-rated news quality compared to the linguistic/formal attributes of news articles and user demographics (Choi et al., 2021). Accordingly, we define news quality judgment as assessing the extent to which news articles that address politically and socially significant matters accomplish quality journalism by observing journalistic norms (see Online Appendix 4).

News quality judgment is multi-faceted. The fact that people assess a news article as high quality is no guarantee that they consistently regard its content as objective, in-depth, diverse, fact-based, believable, readable, and less sensational. Moreover, where one person may regard a given article as high quality because its story is considered objective, another may regard it as high quality because it appears to be well-written. Even though both individuals rate the same article as high quality, the underlying reason behind that judgment differs. Thus, although journalistic norms may form the notion of news quality, we can neither assume positive correlations among those norms nor treat them as a definitive set of characteristics of quality news. Thus, instead of imposing a researcher’s yardstick for news quality on users by creating a second-hand scale, we focus on users’ news quality judgment per se and its conformity to professional standards.

Professional editors are news gatekeepers who decide which articles are published, as well as being quality controllers who judge what constitutes worthwhile news (Singer, 2010). In their editorial decisions, they weigh audience interest, organizational interest, and journalistic principles (Duffy, 2021). These editorial decisions are reflected in the placement of news articles and column size in newspapers (Feeley et al., 2016: 495; Maier, 2010: 14). News articles placed on the front pages of newspapers in larger columns signal greater editorial importance and represent the editor’s judgment on their quality. Indeed, the front page of newspapers has provided a quality barometer in journalism studies (Kim and Chung, 2017); furthermore, the quality scores of front-page news articles have been found to be very high in empirical investigations (e.g. over 4.44 out of 5-point scale in Zaller (1999: 10)). The editorial importance therefore largely (if not entirely) reflects news quality from a professional standpoint.

If user news quality judgments conform to professional standards, then how would each person utilize cues embedded in news article content and form? Relevant studies have shown that users distinguish the quality of given news articles in a manner similar to researchers’ quality manipulation (Urban and Schweiger, 2014) and to editorial decisions (Choi, 2021). However, the information processing mechanism that underlies news quality judgment conformity is still not known. Previous studies that have examined information processing in the realm of news consumption have identified the factors that influence systematic/heuristic processing of presidential election news (Kaye and Johnson, 2021), news elaboration (Lee and Kim, 2016), news verification (Ping Yu, 2021), and news persuasiveness (Heinbach et al., 2018). Our investigation advances this stream of studies by examining the dual information processing of news quality judgment conformity to professional standards.

Combinatorial functioning of content/formal cues for news quality judgment conformity

According to the dual information processing model, elaborative processing requires greater motivation and ability than heuristic processing. This is because people have finite cognitive resources and therefore only tend to elaborate on information when they are motivated and/or capable of doing so. Relatedly, previous studies have operationalized motivation as issue involvement and ability as knowledge (Carpenter, 2015; Heinbach et al., 2018; Lee and Kim, 2016). As postulated by dual processing models, greater involvement in and knowledge of public affairs issues leads individuals to elaboratively process public affairs news (Lee and Kim, 2016). Thus, it seems logical to presume that content cue processing that requires a higher level of elaboration would contribute to greater judgment conformity than formal cue processing.

However, several empirical findings in the field of psychology have suggested the opposite. Research has indicated that experts’ decision-making is intuitively provoked rather than based on analytical reasoning (Reyna, 2004). In Trumbo (1999), heuristic processing accounts for most of people’s relevant information processing when making risk judgments and is associated with lower risk judgments than elaborative processing (Trumbo, 1999). This result runs counter to the old belief that rational and systematic thinking is important for reducing unnecessary over-action against risk. Moreover, heuristic processing has been shown to contribute to accurate decisions. An experiment examining how people distinguish good jams from bad jams revealed that analytical reasoning had a detrimental effect on making correct judgments (Wilson and Schooler, 1991). These findings indicate that the superiority of elaborative processing is not necessarily guaranteed. When a situation requires judgment for which one has previously trained or when the complexity of a task that demands judgment is low, heuristic processing may produce more accurate judgments than elaborative processing.

While users are not trained to judge news quality, they must make such judgments on a daily basis—either consciously or unconsciously. The complexity of such judgments is greater than, for example, judging the taste quality of jams (Wilson and Schooler, 1991), since accurately assessing news quality requires cognitive resources such as reading and understanding news articles. The information necessary to elaborate on news article quality derives from the articles’ content, but users’ characteristics may affect how they process information. For instance, the extent to which they have high ideological strength and/or perceive the press as credible may influence their news processing (Choi, 2021; Kaye and Johnson, 2021) (see Online Appendix 5). By considering these distinctions in news quality judgments (compared to the previous findings discussed in the preceding paragraph), we need to consider several combinations of content/formal cue processing to explain judgment conformity.

Chaiken and Maheswaran (1994) identified three possible combinations of elaborative and heuristic processing: attenuative, additive, and biased functioning. Elaborative processing had a strong positive effect on attitude formation when heuristics contradicted the information used for such processing, whereas heuristic processing had a statistically non-significant effect in the negative direction (attenuative functioning). By contrast, when no contradiction existed between heuristics and the information used for elaborative processing, both forms of processing had a positive impact on attitude formation (additive functioning). When information was ambiguous (e.g. a balanced mix of strong and weak arguments), heuristics influenced elaborative processing, and this biased elaborative processing influenced attitudes (biased functioning) (see Online Appendix 6). While Chaiken and Maheswaran proposed these three functions based on their investigation of attitude formation, we extend these findings by identifying how diverse combinations of content/formal cue processing may influence the news quality judgment conformity.

In addition to these three functions, we propose two more potential functions for investigation. The first is an adverse-attenuative functioning that exhibits a valence opposite to that of attenuative functioning. Past studies have clearly demonstrated that elaborative processing contributes to longer-lasting attitude formation than heuristic processing. As such, in the context of attitude formation, the effects of elaborative and heuristic processing combine as positive-negative (attenuative functioning) or positive-positive (additive functioning). However, in the context of judgment, several studies have shown that elaborative processing does not necessarily lead to more accurate judgment than heuristic processing (Pelham and Neter, 1995; Reyna, 2004; Wilson and Schooler, 1991). Thus, we test for the possibility of negative elaborative processing and positive heuristic processing effects.

The second potential function we introduce is a joint effect between elaborative processing and heuristic processing. Chaiken and Maheswaran (1994) did not examine the potential interaction between these two forms of processing—attenuative functioning and additive functioning are based on the independency between elaborative processing and heuristic processing. If these processing routes not only co-occur but are also interdependent, the question of how one type of processing moderates the other needs to be explored. Walther et al. (2018) found the statistically significant effect of interaction between systematic message cue and heuristic cue on perceived health advice quality, which indicates the possibility of the joint effect. When the level of heuristic processing changes, how does the relationship between elaborative processing and the likelihood of accurate judgment change? Taken together, in the context of news quality judgment conformity, we examine the five combinatorial functions of content/formal cues (see Figure 1).

Figure 1.

Potential effects of five combinations of content/formal cue processing on news quality judgment conformity to professional standards.

Previous studies addressing news consumption have mostly focused exclusively on news heuristics (e.g. Chung, 2017; Lee and Kim, 2016; Sundar et al., 2007). Among them, Lee and Kim (2016) found that people with higher levels of involvement tended to recall more message-relevant points when articles contained both text and graphs. This finding indicates that the processing of modal cues can influence news elaboration, enabling us to infer that biased functioning likely contributes to greater news quality judgment conformity to professional standards.

Another presumption warrants consideration. Studies that took a dual processing approach in a news context have posited the terms “elaborative processing” and “selective scanning,” which slightly differ from the two forms of processing set by the traditional dual processing model. Elaborative processing involves making connections between new information and existing knowledge, while selective scanning involves skimming and choosing information based on personal relevance, importance, and interest (Eveland and Dunwoody, 2002). Eveland and Dunwoody found a positive association between elaborative processing and content knowledge and a negative association between selective scanning and knowledge. Using the same conception, Ping Yu (2021) found that elaborative processing has a positive effect on news verification behaviors (such as checking authorship and news currency) contrary to the non-significant impact of selective scanning. These findings suggest that attenuative functioning may explain the likelihood of greater news quality judgment conformity.

Although the studies discussed in the preceding two paragraphs tested the impact of heuristics only or used different conceptions of the dual routes, their findings suggest that, among the five functions shown in Figure 1, biased or attenuative functioning of content/formal cues is more likely to enhance news quality judgment conformity. These presumptions are rather weak in their current form because researchers have not yet tested them in conjunction with the possibility of other combinations of elaborative and heuristic processing or in the context of news quality judgment conformity. Thus, we compare the explanatory power of the five functions and identify which functions contribute to judgment conformity to professional standards by addressing the following question:

RQ1. Which combination of content/formal cue processing better explains news quality judgment conformity to professional standards?

Constituents of content/formal cues

Based on our survey of journalism/news consumption studies (Bogart, 2004; Gladney, 1996; Gladney et al., 2007; Kovach and Rosenstiel, 2007; Sallot et al., 1998; Sambrook, 2012; Steele, 2008) and our statistical analysis (explained in detail in the methodology section), we regard seven journalistic values as content cues that may affect news quality judgment conformity. These values concern the extent to which a given news story is in-depth (depth), addresses diverse perspectives (diversity), is based on facts (factuality), is objective (objectivity), appeals to emotion (sensationalism), is trust-worthy (believability), and is easy to read (readability). Empirical research has identified comprehensibility (Urban and Schweiger, 2014) and believability (Choi et al., 2021) as the most important predictor of user news quality judgments. To make quality judgments that conform to professional standards, these values as content cues may still be more important than other values. People who regard a given article as highly comprehensible or believable may judge its quality more accurately than those who process it differently. We test this presumption by addressing the following question:

RQ2. Which cue is more important among other content cues (i.e. depth, diversity, factuality, objectivity, sensationalism, believability, and readability) in contributing to news quality judgment conformity to professional standards?

In addition to journalistic values as content cues, formal cues can influence news quality judgment conformity. Researchers have found that information about journalists’ expertise in bylines affects news quality assessments (Choi et al., 2021). The presence of authors’ names, expertise, quotes, and/or statistics have also been shown to affect attitude formation (Rains and Karmikel, 2009) and message/Web site credibility (Chen and Chaiken, 1999; Hong, 2006; Meyer et al., 2010; Sundar, 1998). Several studies have found that graphs and figures or other materials that increase the visual appeal and perceived vividness of a given topic also serve as heuristics (Cyr et al., 2018; Lee and Kim, 2016; Ryu and Kim, 2015) and that longer messages or messages with more arguments are usually perceived to be stronger (Chen and Chaiken, 1999). In addition, a positive association existed between referencing sources of information and Web site credibility (Hong, 2006). Building on these findings, we regard length, expertise, authorship, quotes, numbers, graphics, and referencing sources as formal cues that may affect news quality judgment conformity and investigate the following question:

RQ3. Which cue is more important among other formal cues (i.e. length, expertise, authorship, quotes, numbers, graphics, and referencing sources) in contributing to news quality judgment conformity to professional standards?

Method

Data description

A survey was administered between November 16 and December 18, 2018, with the assistance of Hankook Research (a South Korean market and public opinion research company).² South Koreans aged 20 + years who consumed news at least 3 days a week were eligible to take part. We excluded those with experience working for news organizations or digital news platforms. The information provided to the respondents included news article headlines, main texts, bylines, and publication dates and times. To avoid the possibility of news quality judgments being biased by news brand image perceptions, we did not include brand names with the information.

We used 1500 existing news articles published between August 2017 and August 2018 on Naver.com (the most popular search engine and digital news platform in South Korea). We recruited two editors (see Online Appendix 7) and asked them to rate the editorial importance of these articles, based on their headlines and editorial placement/size. The recruited editors rank-ordered the editorial importance of the given articles without any instructions provided, solely relying on their own editorial expertise. They had 3 weeks to complete the task and were paid to do so.

To achieve greater rigor, we included in our final dataset the articles that were almost unanimously ranked highest or lowest in terms of editorial importance coded by the recruited editors. Among the 1500 articles, the editors made almost identical decisions regarding 88 articles (Spearman rank correlation rho = 0.876, p < .001), placing them nearly unanimously in the first or last ranks, even though they worked for news brands with contrary news coverage (general vs economic/finance-focused) and contrary editorial stances (politically conservative vs liberal). These 88 news articles were published by 21 different news brands encompassing major/minor news organizations as well as general/specialized news coverage and covered political, economic/financial, and social issues. Survey respondents each assessed journalistic values and news quality for one of the 88 articles. A total of 3547 respondents were included in our final dataset, after eliminating four outliers whom we detected by regressing survey responses on our outcome variable.

Regarding news articles’ formal cues, we implemented natural language processing to analyze the collected text data of each article and detected morphemes, numerical expressions, quotations, unidentified sources, bylines, and graphics for each article.

Measurement

News quality judgment conformity

We calculated the absolute difference between respondents’ news quality evaluation scores and editorial news importance scores. The former ranged from 0 to 10, whereas the latter scored either 0 when the article was ranked last or 10 when ranked first by the recruited editors. These perfect editorial importance scores at both ends served as the basic criterion for identifying the extent to which the survey respondents’ news quality judgment scores departed from professional standards. The larger the absolute difference between the two, the greater the respondent’s news quality judgment deviated from professional standards. To facilitate interpretation, we subtracted the absolute differences from their maximum values so that higher values indicated greater conformity to professional standards (M = 5.946, SD = 1.974, min = 0, max = 10).

Content cues and formal cues

We regarded seven journalistic values—depth, diversity, factuality, objectivity, sensationalism (reverse-coded), believability, and readability—as content cues. By implementing exploratory factor analyses with the preliminary survey data, these seven values were extracted from 14 journalistic values (impartiality, diversity, informativeness, objectivity, comprehensiveness, depth, originality, readability, comprehensibility, clarity, factuality, interpretivism, sensationalism, and believability) drawn from previous journalism studies. We then conducted an additional confirmatory factor analysis with the seven journalistic values. In our main survey, we asked respondents to assess each journalistic value in each news article on a seven-point scale (1 = strongly disagree to 7 = strongly agree).

The formal cues discussed in the literature review include a news article’s length, authorship, expertise, graphics, quotes, numbers, and referencing sources. We measured length, numbers, and quotes by counting the number of morphemes in each article’s main text (not including the headline text) (M = 734.75, SD = 255.921), the number of numerical expressions (M = 33.557, SD = 24.610), and the number of both direct and indirect quotes (M = 4.261, SD = 3.015), respectively. For referencing sources, we reverse-coded for unidentified sources (e.g. “according to an anonymous source,” “citing an unnamed person”) where 1 indicated the absence of unidentified sources (67 articles (76.1%)) and 0 indicated their presence. For expertise, we coded articles that indicated the journalist(s)’ expertise (e.g. legal journalist, medical journalist) as 1 (8 articles (9.1%)) and articles that did not as 0. We counted the number of journalists (M = 1.511, SD = 0.711) and the number of graphs and figures (M = 1.011, SD = 0.809) for authorship and graphics, respectively. We divided the numbers of numerical expressions, quotes, journalists, and graphics by the number of morphemes, as the former four variables tend to correlate with article length.

Control variables

We controlled for issue involvement and knowledge, which are known to influence dual information processing (Chen and Chaiken, 1999; Lee and Kim, 2016; Petty and Cacioppo, 1986; Scholten et al., 2007), and for press credibility, which is related to the heuristic processing of news articles (Choi and Lim, 2019; Sundar et al., 2007). We also controlled for political ideology, ideological strength, and political discussion frequency, as they can influence the accuracy of quality judgment of public affairs news (Gunther and Schmitt, 2004; Vallone et al., 1985). In addition, news media use, news genre, and demographics were controlled in our research model (Chung, 2017; Lee and Kim, 2016; Ping Yu, 2021; Price and Zaller, 1993) (see Online Appendix 8).

Analysis

Using the aforementioned variables, we implemented partial least squares structural equation modeling (PLS-SEM) (Hair et al., 2017)—rather than the typical SEM. While SEM estimates the covariance matrix for a sample dataset, PLS-SEM explains the variance in a dependent variable. Compared to covariance-based SEM, PLS-SEM performs more efficiently with complex models, has greater statistical power, and does not have any distributional assumptions concerning data (Garson, 2016). Furthermore, PLS-SEM does not exhibit identification problems with single-item constructs and measurement models composed of both reflective and formative indicators, as is the case with our model. We implemented PLS-SEM with SmartPLS software (Ringle et al., 2015).

Results

To address RQ1, we examined the five combinations of content/formal cue processing by statistically testing them with three models: an independent model (attenuative/additive/adverse-attenuative functions), a biased model, and a joint model. Details about the model assumption check, model specification, and indicator drop-outs are provided in Online Appendix 9.

The “independent model” (featuring separate paths from content/formal cues to news quality judgment conformity) accounted for 28.3% of the variance in news quality judgment conformity to professional standards. The standardized root mean square residual (SRMR) was 0.041, indicating an excellent model fit. The path coefficients of content cue and formal cue processing toward news quality judgment conformity were 0.496 (p < .001) and −0.172 (p < .001), respectively. This result supports attenuative functioning (thus rejecting the additive/adverse-attenuative functions), which implies that content cue processing contributes to judgment conformity while formal cue processing is detrimental to it.

The adjusted R² explaining the variance in judgment conformity of the “biased model” (featuring a path from formal cue processing to content cue processing, and then to news quality judgment conformity to professional standards) was 25.4%, and its SRMR was 0.043, indicating a slightly lower explanatory power and model fit than the independent model. The path from formal cue processing to content cue processing was not statistically significant (−0.031, p = .281), whereas the path from content cue processing to the judgment conformity variable was statistically significant (0.497, p < .001).

Finally, the “joint model” (featuring the interaction term between content/formal cue processing) explained 32.3% of the variance (higher than the two preceding models) in judgment conformity. Its SRMR was 0.041. When we compared the three models’ corrected Akaikes information criterion (AICc), which is more accurate than AIC scores, we found that the joint model (2179.366) had a lower AICc score than the other two models (2381.204 for the independent model and 2522.601 for the biased model). This result suggests that the joint model better explained news quality judgment conformity.

Figure 2 shows the PLS-SEM results of our final model (i.e. joint model). The interaction between content/formal cue processing had a statistically significant, negative effect on judgment conformity (−0.197, p < .001). The relationship between content cue processing and judgment conformity was stronger with less consideration of formal cues (see Online Appendix 10). The less the respondents relied on formal cues, the greater they achieved news quality judgment conformity as they elaborated more on content cues. The effect size of the interaction term on news quality judgment conformity was large, as indicated by the 0.059 f² value.³

Figure 2.

PLS-SEM result explaining news quality judgment conformity to professional standards.

With regard to RQ2, believability was more important among content cues, followed by depth, readability, diversity, objectivity, and factuality. Regarding RQ3, quotes were more important among formal cues, followed by numerical expressions and journalist expertise. The number of journalists did not have a statistically significant effect.

We also conducted significance tests for the total effects (i.e. the sum of direct and indirect effects) as well as path coefficients (see Online Appendix 11). Among all the exogenous variables, including the control variables, the effect size of the interaction term on judgment conformity was greater than any other variables’ sum of direct and indirect effects on judgment conformity, except for the content cues nested in the term. The joint effect of content/formal cue processing exerted a relatively strong influence on judgment conformity, one in a reinforcing manner and the other in an attenuative manner. The drop-out of formal cue processing from our final model lowered R² (25.4%) and substantially increased AICc (2521.696), implying that content cue processing alone does not better explain news quality judgment conformity.

Discussion and conclusion

Users have become more involved in the digital news ecosystem, exerting influence not only on news distribution and consumption but also on news production (Ferrer-Conill and Tandoc, 2018; Welbers et al., 2016). Under this circumstance, the news gap between journalists/editors and users has gained attention. Previous findings implied that the gap in terms of news quality judgment between these two is not large as expected (Choi, 2021; Urban and Schweiger, 2014)—users judge news article quality in a manner comparable to journalists/editors. Then, the next step is to determine what information processing contributes to user news quality judgment in conformity to professional standards. The fact that users are left to judge news quality with fewer cues on digital news platforms compared to the traditional news environment, which had clear editorial cues (e.g. the size and placement of articles), adds significance to this investigation.

By returning to the dual information processing literature, we explained this mechanism with the processing of content cues and formal cues (the two information types embedded in news articles) by testing the five combinations of these two. We found that the joint functioning of content/formal cue processing has a stronger explanatory power than the other possible combinations. According to Chaiken and Maheswaran (1994), biased functioning occurs when information is ambiguous. Considering that this was not the case in our study, respondents might not have perceived the news articles provided for judgment to be ambiguous enough to stimulate heuristics based on formal cues. This seems reasonable, as news consumption is a daily task with which most people are familiar. Sometimes, for people with greater involvement, the graphs presented in the news articles along with the text supported their message elaboration (Lee and Kim, 2016). However, we found that graphics or other formal cues did not facilitate the perception of content cues for news quality judgment conformity.

Our findings show that the less the respondents relied on formal cues, the greater they achieved news quality judgment conformity as they elaborated more on content cues. As the negative sign of the interaction between content/formal cue processing indicates, one attenuated the other. However, this attenuation differed from Chaiken and Maheswaran’s (1994) attenuative functioning and the findings of Ping Yu (2021), as both studies assumed that elaborative processing and heuristic processing are independent. In our study, attenuation occurred in the form of one moderating the other. The interaction between content/formal cue processing had a large effect size.

While some previous studies have found that expertise judgments tend to rely on heuristic processing (Reyna, 2004), for untrained laypeople, as found in the present study, content cue processing plays a critical role in judgment conformity. Intuition or heuristics that more numerical expressions and more quotes signal higher news article quality did not positively affect news quality judgment conformity. Our finding that content cue processing played a stronger role in news quality judgment conformity might have stemmed from the fact that respondents had not previously received training related to news quality judgment and judging news quality accurately is not a low-complexity task (although it is difficult to describe it as otherwise, as people judge news quality either consciously or unconsciously by reading news articles in their daily lives). Although the path coefficient size of formal cue processing to judgment conformity was far smaller than that of content cue processing, we found formal cue processing to be not only theoretically but also empirically necessary to explain judgment conformity—the complete elimination of formal cue processing undermined our model’s performance.

All six journalistic values (i.e. believability, depth, diversity, factuality, objectivity, and readability) as content cues contributed to news quality judgment conformity to professional standards. In particular, the extent to which respondents assessed articles as believable and in-depth was found to be relatively more critical. Among formal cues, the number of quotes was relatively more important. Our finding that formal cue processing negatively contributed to judgment conformity suggests that the more quotes a given article cites, the less the respondents’ judgment of the article conforms with professional judgment. Respondents might have heuristically judged such an article as high quality based on its inclusion of many quotes without paying much attention to its content (Sundar, 1998). Another possibility cannot be ruled out—respondents could have been swayed by the type of quoted sources. For instance, articles citing government sources may have been regarded as higher quality than articles citing interest groups, or the political orientations of quoted sources could have influenced respondents’ judgment (Duncan et al., 2019). This possibility needs to be tested in follow-up research.

Journalist expertise and numerical expressions were statistically significant heuristics. In Meyer et al. (2010), when people perceived the news reporter as an expert, they regarded the given news article as credible. News stories written by specialist health journalists tended to have higher quality scores in the medical sphere compared to stories written by general journalists (Wilson et al., 2010). By observing the indication of journalist expertise in the byline, it is likely that respondents made a rough guess of news quality as if the given article is credible and of high-quality. Quantified expressions might also have led respondents to have a favorable perception of a given article. The inclusion of statistical information was positively associated with Web site credibility (Hong, 2006) and online news article credibility (Henke et al., 2020). In particular, Henke et al. found that numerical information increased the perceived accuracy of the article. Thus, quantified expressions might have made respondents speculate the article as if it is accurate, credible, and thus of high-quality.

This study is not without limitations. First, we did not provide our survey respondents information about the given news articles’ online social cues such as user comments and the numbers of likes, shares, and recommendations. We intended to focus on the respondents’ assessments of the articles’ content and form per se, rather than their reactions to factors that are exogenous to news articles. The inclusion of social indicators might have increased the negative contribution of heuristics to judgment conformity—previous research that found a significant effect of these social cues on news evaluation (Chung, 2017; Heinbach et al., 2018) support this speculation. Second, our research model did not include variables such as affects, need for cognition, and personalities, which may influence information processing (Carpenter, 2015; Nabi, 1999). The exclusion of these variables did not harm the validity of the study, because our main goal was to compare five combinations of content/formal cue processing. However, their inclusion might have increased our model’s explanatory power. Third, we did not ask respondents about their perceptions of formal cues, because, for instance, asking how many quotes they perceived a given article has might resulted in them elaborating on these heuristics—thus, distorting the result. Instead, we set a minimum time required to assess a given article to allow respondents to be exposed sufficiently to the formal features of the article. This follows the procedure of relevant literature that tested the dual information processing model in the context of news consumption (e.g. Lee and Kim, 2016). Fourth, it is our limitation that we did not examine the source of quotes besides the number of quotes. Lastly, our recruited editors did not assess the quality of the news articles in the same way as the survey respondents. Given the infeasibility of asking each editor to rate the quality of 1500 news articles from scratch, the editors ranked the news articles’ editorial importance based on the editorial decisions made by original editors who read and published the articles. This approach was taken considering the previous findings that have shown the positive association between editorial decisions and news quality (Kim and Chung, 2017; Zaller, 1999: 10). We also applied several methodological treatments to minimize any potential bias (see Online Appendix 12).

Our study compared five possible scenarios and empirically demonstrated that the joint functioning of content/formal cue processing has the best explanatory power. Because news is the basis for political deliberation and democratic decision making, judging news quality in an accurate manner is critical. The present findings demonstrate how to process cues extracted from news articles to render quality judgment closer to trained judgment. We hope this finding helps users to filter out low-quality news articles and, in turn, improves the quality of public discourse. While previous studies in communication research have found that dual information processing influences attitude formation/change, persuasion, and risk perceptions, explorations of the ways elaborative/heuristic processing routes contribute to judgment accuracy remain limited. Furthermore, relatively few have used this model to examine news consumption (see Online Appendix 13). We expect the present findings stimulate subsequent research to further substantiate the joint effect of positive content cue processing (or elaborative processing) and negative formal cue processing (or heuristic processing) on judgment conformity to professional standards (or judgment accuracy) in the context of news consumption (or communication studies) (see Online Appendix 14).

Supplemental Material

Supplemental Material - User information processing mechanisms for news quality judgment conformity to professional standards: Comparing combinations of news content/formal cue processing

Supplemental Material for User information processing mechanisms for news quality judgment conformity to professional standards: Comparing combinations of news content/formal cue processing by Sujin Choi in Journalism

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2018S1A5A2A03035181). In addition, the Institutional Review Board (IRB) at Khung Hee University approved this study as exempt from review (KHSIRB-18-069).

ORCID iD

Sujin Choi

Supplemental Material

Supplemental material for this article is available online

Notes

Author biography

Sujin Choi (PhD from the University of Texas at Austin in the US) is an associate professor in the Department of Media at Kyung Hee University in Seoul, Korea. She is interested in the influence of news algorithms on digital news production and consumption and furthermore on discussions in the public sphere. She also investigates the algorithmic quantification of human values and logics based on social science approaches and computational methods. Her work appears in journals such as Human Communication Research, Journal of Computer-Mediated Communication, New Media & Society, and Digital Journalism.

References

Bogart

(2004) Reflections on content quality in newspapers. Newspaper Research Journal 25(1): 40–53.

Carpenter

(2015) A meta-analysis of the ELM’s argument quality× processing type predictions. Human Communication Research 41(4): 501–534.

Chaiken

(1980) Heuristic versus systematic information processing and the use of source versus message cues in persuasion. Journal of Personality and Social Psychology 39(5): 752–766.

Chaiken

Maheswaran

(1994) Heuristic processing can bias systematic processing: effects of source credibility, argument ambiguity, and task importance on attitude judgment. Journal of Personality and Social Psychology 66(3): 460–473.

Chen

Chaiken

(1999) The heuristic-systematic model in its broader context. In: Chaiken

Trope

(eds), Dual-Process Theories in Social Psychology. New York: Guilford, 73–96.

Choi

(2021) News gap in a digital news environment: calibrating editorial importance from user-rated news quality and identifying user characteristics that close the news gap. New Media & Society 23(12): 3677–3701.

Choi

Lim

(2019) Determinant and consequence of online news authorship verification: blind news consumption creates press credibility. International Journal of Communication 13: 23.

Choi

Shin

Kang

(2021) Predicting audience-rated news quality: using survey, text mining, and neural network methods. Digital Journalism 9(1): 84–105.

Chung

(2017) Not just numbers: the role of social media metrics in online news evaluations. Computers in Human Behavior 75: 949–957.

10.

Costera Meijer

Bijleveld

(2016) Valuable journalism: measuring news quality from a user’s perspective. Journalism Studies 17(7): 827–839.

11.

Cyr

Head

Lim

, et al. (2018) Using the elaboration likelihood model to examine online persuasion through website design. Information & Management 55(7): 807–821.

12.

Duffy

(2021) Out of the shadows: the editor as a defining characteristic of journalism. Journalism 22(3): 634–649.

13.

Duncan

Culver

McLeod

, et al. (2019) Don’t quote me: effects of named, quoted, and partisan news sources. Journalism Practice 13(9): 1128–1146.

14.

Epstein

(1994) Integration of the cognitive and the psychodynamic unconscious. American Psychologist 49(8): 709.

15.

Evans

JSt BT

(2008) Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology 59: 255–278.

16.

Eveland

Jr Dunwoody

(2002) An investigation of elaboration and selective scanning as mediators of learning from the web versus print. Journal of Broadcasting & Electronic Media 46(1): 34–53.

17.

Feeley

O’Mally

Covert

(2016) A content analysis of organ donation stories printed in US Newspapers: application of newsworthiness. Health Communication 31(4): 495–503.

18.

Ferrer-Conill

Tandoc

Jr (2018) The audience-oriented editor: making sense of the audience in the newsroom. Digital Journalism 6(4): 436–453.

19.

Garson

(2016) Partial Least Squares: Regression and Structural Equation Models. Asheboro, NC: Statistical Associates Blue Book Series, Statistical Associates Publishing.

20.

Gladney

(1996) How editors and readers rank and rate the importance of eighteen traditional standards of newspaper excellence. Journalism & Mass Communication Quarterly 73(2): 319–331.

21.

Gladney

Shapiro

Castaldo

(2007) Online editors rate web news quality criteria. Newspaper Research Journal 28(1): 55–69.

22.

Gunther

Schmitt

(2004) Mapping boundaries of the hostile media effect. Journal of Communication 54(1): 55–70.

23.

Hair

Jr Hult

GTM

Ringle

, et al. (2017) A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM). Second edition. Thousand Oaks, California: Sage publications.

24.

Heinbach

Ziegele

Quiring

(2018) Sleeper effect from below: long-term effects of source credibility and user comments on the persuasiveness of news articles. New Media & Society 20(12): 4765–4786.

25.

Henke

Leissner

Möhring

(2020) How can journalists promote news credibility? Effects of evidences on trust and credibility. Journalism Practice 14(3): 299–318.

26.

Hong

(2006) The influence of structural and message features on Web site credibility. Journal of the American Society for Information Science and Technology 57: 114–127.

27.

Kaye

Johnson

(2021) Newstrusting” or “newsbusting?” heuristic and systematic information processing and trust in media. Atlantic Journal of Communication 29: 312–327.

28.

Kenny

(2016) Moderation. http://davidakenny.net/cm/moderation.htm

29.

Kim

(2015) Exploring the effects of source credibility and others’ comments on online news evaluation. Electronic News 9(3): 160–176.

30.

Kim

Chung

(2017) Anatomy of front pages: comparison between the New York Times and other US major metropolitan newspapers. International Journal of Communication 11: 949–966.

31.

Korea Press Foundation (2021) Media Users in Korea 2021. Seoul, Korea: Korea Press Foundation.

32.

Kovach

Rosenstiel

(2007) The Elements of Journalism: What Newspeople Should Know and the Public Should Expect. New York: Three Rivers Press.

33.

Lee

Kim

(2016) Effects of infographics on news elaboration, acquisition, and evaluation: prior knowledge and issue involvement as moderators. New Media & Society 18(8): 1579–1598.

34.

Lieberman

(2003) Reflexive and reflective judgment processes. In: Forgas

Williams

von Hippel

(eds), Social Judgments: Implicit and Explicit Processes. Cambridege: Cambridge University Press, 44–67.

35.

Maier

(2010) Newspapers offer more news than do major online sites. Newspaper Research Journal 31(1): 6–19.

36.

Meyer

Marchionni

Thorson

(2010) The journalist behind the news: credibility of straight, collaborative, opinionated, and blogged “news”. American Behavioral Scientist 54(2): 100–119.

37.

Nabi

(1999) A cognitive-functional model for the effects of discrete negative emotions on information processing, attitude change, and recall. Communication Theory 9(3): 292–320.

38.

Pelham

Neter

(1995) The effect of motivation of judgment depends on the difficulty of the judgment. Journal of Personality and Social Psychology 68(4): 581–594.

39.

Petty

Cacioppo

(1986) The elaboration likelihood model of persuasion. In: Berkowitz

(ed), Advances in Experimental Social Psychology, Volume 19. New York: Springer, 124–205.

40.

Ping Yu

(2021) How types of Facebook users approach news verification in the mobile media age: insights from the dual-information-processing model. Mass Communication & Society 24(2): 233–258.

41.

Price

Zaller

(1993) Who gets the news? Alternative measures of news reception and their implications for research. Public Opinion Quarterly 57(2): 133–164.

42.

Rains

Karmikel

(2009) Health information-seeking and perceptions of website credibility: examining web-use orientation, message characteristics, and structural features of websites. Computers in Human Behavior 25(2): 544–553.

43.

Reyna

(2004) How people make decisions that involve risk: a dual-processes approach. Current Directions in Psychological Science 13(2): 60–66.

44.

Ringle

Wende

Becker

(2015) SmartPLS 3. SmartPLS, Boenningstedt.

45.

Ryu

Kim

(2015) Testing the heuristic/systematic information-processing model (HSM) on the perception of risk after the Fukushima nuclear accidents. Journal of Risk Research 18(7): 840–859.

46.

Sallot

Steinfatt

Salwen

(1998) Journalists’ and public relations practitioners’ news values: perceptions and cross-perceptions. Journalism & Mass Communication Quarterly 75(2): 366–377.

47.

Sambrook

(2012) Delivering Trust: Impartiality and Objectivity in the Digital Age. Oxford: Reuters Institute for the Study of Journalism.

48.

Scholten

Van Knippenberg

Nijstad

, et al. (2007) Motivated information processing and group decision-making: effects of process accountability on information processing and decision quality. Journal of Experimental Social Psychology 43(4): 539–552.

49.

Singer

(2010) Quality control: Perceived effects of user-generated content on newsroom norms, values and routines. Journalism Practice 4(2): 127–142.

50.

Steele

(2008) Ethical values and quality control in the digital era. Nieman Reports 62(4): 57–58.

51.

Strack

Deutsch

(2004) Reflective and impulsive determinants of social behavior. Personality and Social Psychology Review 8(3): 220–247.

52.

Sundar

(1998) Effect of source attribution on perception of online news stories. Journalism & Mass Communication Quarterly 75(1): 55–68.

53.

Sundar

Knobloch-Westerwick

Hastall

(2007) News cues: information scent and cognitive heuristics. Journal of the American Society for Information Science and Technology 58(3): 366–378.

54.

Trumbo

(1999) Heuristic-systematic information processing and risk judgment. Risk Analysis 19(3): 391–400.

55.

Urban

Schweiger

(2014) News quality from the recipients’ perspective: investigating recipients’ ability to judge the normative quality of news. Journalism Studies 15(6): 821–840.

56.

Vallone

Ross

Lepper

(1985) The hostile media phenomenon: biased perception and perceptions of media bias in coverage of the Beirut massacre. Journal of Personality and Social Psychology 49(3): 577–585.

57.

Walther

Jang

Hanna Edwards

(2018) Evaluating health advice in a Web 2.0 environment: the impact of multiple user-generated factors on HIV advice perceptions. Health Communication 33(1): 57–67.

58.

Welbers

Van Atteveldt

Kleinnijenhuis

, et al. (2016) News selection criteria in the digital age: professional norms versus online audience metrics. Journalism 17(8): 1037–1053.

59.

Wilson

Schooler

(1991) Thinking too much: introspection can reduce the quality of preferences and decisions. Journal of Personality and Social Psychology 60(2): 181–192.

60.

Wilson

Robertson

McElduff

et al. (2010) Does it matter who writes medical news stories? PLoS Medicine 7(9).

61.

Zaller

(1999) Market Competition and News Quality. Atlanta, GA: Annual Meetings of the American Political Science Association. Paper for presentation at the 1999.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.23 MB