Abstract
Several theoretical approaches suggest that a promising approach to designing effective, tailored persuasive messages may be to draw on insights from an audience’s shared cognitive structure. Specifically, a cognitive-structural approach to message design would suggest that messages targeting central concepts in an audience’s shared cognitive structure will have stronger persuasive effects than messages targeting more peripheral concepts. The present investigation, however, failed to provide support for this approach. The results of four studies revealed that attitude and semantic networks each provided a different estimate of cognitive structure and made competing claims about whether this structure differs for Republicans and Democrats. However, neither structure successfully predicted which messages would be most effective. Instead, both Democrats and Republicans were persuaded by a wide range of arguments targeting both central and peripheral concepts. The results have implications for future work on the role of cognitive structures in persuasion and theory-driven message development.
Introduction
Although there is a wealth of research on many aspects of persuasive message design, there is, with a few exceptions (e.g., Hornik & Woolf, 1999), little theoretically grounded work on how to select which arguments to include in those messages. Researchers and practitioners are often left to make informed guesses as to which themes and arguments they think will be most effective with an audience rather than being able to rely on firm predictions (see Lee et al., 2016, for a review). Perhaps as a result, persuasive message design work commonly couples existing theoretical models with nonexperimental data—such as focus groups (Botta et al., 2008) or open-ended survey responses (Bai et al., 2009)—to draw conclusions about messages that would be effective without directly evaluating the effectiveness of such messages (for another example, see Noar et al., 2015).
To address this gap, Cruz et al. (2025) recently suggested that a cognitive-structural approach to message design might hold promise as a theory-driven technique for crafting persuasive messages. This approach follows in the tradition of previous research suggesting that an individual’s cognitive structure plays a role in how they are influenced—or not—by persuasive messaging (e.g., Gillham & Woelfel, 1977; Hamilton & Mineo, 1996; Lee et al., 2016; Woelfel, 1997). For instance, Barnett et al. (1976) and Serota et al. (1977) used findings on the structure of political beliefs among registered voters to identify promising message strategies for political candidates. Consistent with these approaches, a cognitive-structural approach would endeavor to optimize message effectiveness by evaluating someone’s cognitive structure to identify and target highly central concepts. Although these central concepts may be more difficult to influence than less central concepts (Dalege et al., 2016; Danowski, 1993; though cf. Brandt & Vallabha, 2023), they are expected to have the largest impact on individual judgments and behaviors (Dalege et al., 2017b; Smith & Parrott, 2012).
A preliminary study found some support for a cognitive-structural approach (Cruz et al., 2025) but also raised more theoretical questions. In particular, the findings did not provide a definitive answer about which method of mapping cognitive structure would be most useful for message design. Attitude networks (Dalege et al., 2016, 2019) and semantic networks (Danowski, 1993) provided different understandings of cognitive structure and identified different central concepts, but both made a mix of correct and incorrect predictions about the concepts that would be most predictive of behavior change. Furthermore, this preliminary study focused on relatively naturalistic changes in attitude and behavior over time rather than experimentally testing the effects of targeted messages. Accordingly, the more specific, concept-level predictions of this approach remain untested. The goal of the present investigation was to build on these preliminary findings with stronger empirical tests.
Theoretical Background
As noted above, cognitive-structural approaches to message design focus on the potential to design messages by identifying and targeting the most central concepts in an individual’s cognitive structure for a given issue. 1 Broadly, cognitive structure refers to the location of and linkages among cognitive elements, such as a message recipient’s system of beliefs and evaluations related to an attitude object, with more strongly linked cognitions serving as more central nodes in the network (Keating, 2018). The importance of cognitive structure for attitudes and behavior—particularly of central nodes in that structure—has been emphasized in a number of existing theoretical approaches, including recent work on attitude networks (such as the casual attitude network model, CAN; Dalege et al., 2016) and classic communication work on semantic networks (e.g., Danowski, 1993).
Existing work on attitude networks proposes that attitudes are composed of an interconnected set of cognitive elements (e.g., Woelfel, 1997) and evaluative reactions (e.g., Dalege et al., 2016, 2017b, 2019). These cognitive elements and evaluative reactions include concrete beliefs, affective responses, and mental representations of behavioral tendencies related to the attitude object. Work on automatic attitude activation, such as the hot cognition hypothesis (Lodge & Taber, 2005; Morris et al., 2003), has previously demonstrated that when one element in this structure is activated (e.g., by seeing it mentioned in a persuasive message), connected elements are automatically activated via spreading activation. Moreover, affective responses to activated elements occur preconsciously and cannot be suppressed, although they may be elaborated or overridden upon further deliberation (Lodge & Taber, 2005; see also Glaser et al., 2015).
Subsequent work on the CAN model (Dalege et al., 2016) and related work on indirect and lateral attitude change (e.g., Alvaro & Crano, 1997; Glaser et al., 2015) proposes that these connections have implications not just for activation, but persuasion as well. Specifically, these perspectives suggest the nodes are linked causally, such that shifts in one node generate shifts in connected nodes. In other words, just as activation of one element can activate related elements, change in one element can change related elements. This work also proposes that the centrality of an activated element is the most important determinant of how much change will occur, where more highly central nodes are defined as those that have greater direct influence on other nodes (i.e., strength), connect more pairs of other nodes (i.e., betweenness), or are more likely to spread direct or indirect influence throughout the network (i.e., closeness). Highly central nodes, compared to less central nodes, are also proposed to be ideal targets for persuasive messages because they are stronger predictors of attitudes and behaviors (Dalege et al., 2017b). Accordingly, employing an attitude network as a foundation for message design would entail designing persuasive messages that focus on highly central evaluative reactions in someone’s attitude network. As an example, a message recipient’s attitude network about bees might include “important for pollinating crops” and “helpful for maintaining our state’s natural beauty,” with the former node being higher in centrality and the latter being comparatively lower in centrality. In this instance, a message would theoretically be more persuasive if it emphasized the importance of bees for pollinating fruits and vegetables than if it emphasized that bees contribute to a neighborhood’s aesthetic value. 2
In semantic networks, in contrast, the nodes constitute interconnected concepts that people use to describe attitude objects (Danowski, 1993; Rice & Danowski, 1993; Smith & Parrott, 2012). These interconnected concepts are associated and associative words that people hold in memory (Calabrese et al., 2020; Doerfel, 1998). In these networks, more central words are defined as those with a greater number of other words leading into or out from them (i.e., indegree and outdegree centrality, respectively) or a greater ability to connect other pairs of words to one another (i.e., betweenness centrality) (Freeman, 1978). Messages including these highly central words, compared to less central words, are also proposed to trigger greater activation of the entire semantic network and thereby facilitate greater persuasion (Danowski, 1993). Accordingly, employing a semantic network as a foundation for message design would entail designing persuasive messages that focus on highly central words in the semantic network of descriptions of a focal issue. As an example, a message recipient’s semantic network about bees might include “sting” as a highly central node, but not include any nodes related to the concept of “biodiversity.” In such an instance, a message would theoretically be more persuasive if it emphasized that bees are safe to be around as long as they are not bothered than if it emphasized that bees help to improve local biodiversity.
Although both attitude and semantic networks emphasize the importance of the structure of message recipients’ thoughts and cognitions, they also differ in important ways. For one, attitude networks provide a sense of implicit cognitive structure, in that people do not actively generate the nodes or their connections. In other words, attitude networks describe a structure that exists despite the fact that people may be unaware of which cognitive elements are connected or how strongly (e.g., see Alvaro & Crano, 1997). Evidence consistent with the implicit nature of cognitive structure can be found in the literature on automatic attitude activation (Bargh et al., 1992; Lodge & Taber, 2005; Morris et al., 2003), which illustrates that spreading activation among related concepts occurs almost instantly (within 100–200 ms), more quickly than conscious reactions can be formed (which takes at least 500 ms). Given that spreading activation along this structure occurs automatically and preconsciously, it is also reasonable to expect that it may play an important role in persuasive outcomes. For instance, Lodge and Taber (2005) propose it may contribute to biased processing by automatically activating positive or negative reactions to the concepts in a message, even before they can be consciously processed.
Semantic networks, in contrast, provide a sense of relatively explicit cognitive structure, in that people generate the words that form these networks and deliberately connect different concepts to one another. Thus, though they may not be aware of the entire structure of the semantic network, they likely have a relatively clear conscious understanding of how different concepts connect. This structure may also play an important role in persuasive outcomes, because it reflects the related concepts that someone feels are most relevant or important when they consciously reflect on a given node. Conscious processing may also be particularly important for complex topics toward which people have ambivalent attitudes (i.e., both positive and negative evaluations), because these topics take longer for people to react to and lead to different patterns of activation than non-ambivalent attitudes (Bargh et al., 1992; Lodge & Taber, 2005).
Because these two methods of mapping cognitive structure are rarely, if ever, employed in the same study, however, it is not clear which one is a stronger determinant of persuasive effects. A preliminary study (Cruz et al., 2025) found that each network made a mix of correct and incorrect predictions about how cognitive structure would connect to behavior change, but additional research is clearly needed before firm conclusions can be drawn.
Individual and Shared Cognitive Structures
In addition to the challenge of reconciling differences that may emerge in how the attitude network and semantic network depict cognitive structure, there may be additional complexities in that cognitive structures may be shared to a greater or lesser extent across people (e.g., Converse, 1964; Costantini et al., 2019). For some people and for particular attitude objects, it seems logical to expect that cognitive structures are highly idiosyncratic. Someone’s knowledge and beliefs about their romantic partner of many years is likely to be organized in different ways than anyone else’s, even among others who know that person. Likewise, a leading expert on a given topic will likely have a much more complex, elaborate cognitive structure for that topic than the average person. On the other hand, there are many other attitude objects for which many people are likely to share a very similar cognitive structure. For example, the constraints of factual reality and educational standards mean that most U.S. high school students are exposed to a very similar set of information about many topics. Although some differences across people in how they organize their thinking about those topics is to be expected (e.g., due to differences in learning ability, motivation, previous and subsequent education and experience, and knowledge retention), we would expect the core features of that organization to be similar. This expectation is consistent with the fundamental assumption that education works by shaping and modifying cognitive structures (e.g., Ausubel, 1963; Ifenthaler et al., 2011). In many cases, it may therefore be feasible to develop a smaller set of message strategies tailored to an audience’s shared cognitive structure, rather than needing different messages for each individual.
This is not to say, however, that this shared structure may not also differ meaningfully for different audience segments (e.g., see Costantini et al., 2019). Especially when it comes to matters of public opinion, people who differ in their political identification may have diverging networks (e.g., Dalege et al., 2017b). Political ideology is arguably a special case of cognitive structure in that it entails an organization of concepts, beliefs, and attitudes that is shared among people within the same general political group (Converse, 1964; Keating & Bergan, 2017). Given that members of different political groups tend to expose themselves selectively to similar media sources (Stroud, 2010), find sources who share their political party to be more persuasive (Bolsen et al., 2019; Cruz & Carpenter, 2024), and associate more with one another than with members of the opposing political party (Boutyline & Willer, 2017; Huber & Malhotra, 2017), it is logical to expect them to develop a shared manner of organizing their thinking about many attitude objects. Research has also demonstrated the importance of tailoring messages to audience segments based on political identification (e.g., Feinberg & Willer, 2019; Nisbet & Mooney, 2007). Perhaps one reason why such message tailoring has been effective is that it has unwittingly taken advantage of differences in cognitive structure between political groups. If so, a structure-focused approach could have the power to simplify and enhance the process of designing and targeting messages for different political audiences.
The Present Investigation
To address these questions about cognitive structure and message design, we report the results of four studies designed to test core elements of a possible cognitive-structural approach to message design. Notably, we compared messages predicted to work against messages predicted not to work. This comparison provided a particularly stringent test of the theoretical framework by testing both the prediction that targeting central nodes would lead to statistically significant persuasive effects and the prediction that targeting peripheral nodes would not lead to statistically significant persuasive effects. Study 1 focused on identifying high- and low-centrality concepts in Democrats’ and Republicans’ attitude networks and semantic networks for a test case: bee conservation. These concepts then served as the basis for messages designed to test the framework’s predictions about message topic selection. Study 2 evaluated whether pro-conservation messages including the selected message topics varied in their perceived message effectiveness and perceived quality, in order to account for alternative explanations of any observed differences in persuasive effects. Finally, Study 3–4 tested the effects of the messages on conservation attitudes and behavioral intentions.
We focused on bee conservation as the topic for these studies for two main substantive reasons. First, there are sharp political divides on environmental issues in the U.S. (Cruz, 2017). Despite a wealth of research on how to design messages for members of different political parties, disagreements about the relative importance of the environment and the economy have also been increasing over time (Jones, 2023). Although public discourse and media coverage also often focus on climate change as the source of this disagreement, the party divide on environmental issues extends even to topics that might be assumed to evoke universal concern, including water pollution, air pollution, species loss, and destruction of rain forests (Brenan, 2023). Although research on human dimensions of pollinator conservation is relatively limited (Hall & Martins, 2020; Schultz, 2011), it is reasonable to expect that general attitudes toward bee conservation may likewise diverge along party lines.
Second, bee conservation is an issue where the types of messages that resonate with members of different parties seem likely to differ. Bees, and pollinators more generally, are closely associated with both economic and environmental benefits. Bees convey substantial economic benefits to farmers and growers by visiting the majority of flowering plants and global food crops (Klein et al. 2007; Rodger et al. 2021). In line with other research on message framing (e.g., Nisbet, 2009), there has thus been the (untested) assumption that utilitarian, economic-focused messages will be necessary to make the case for public support for pollinator conservation (Kusmanoff et al., 2020). This may especially be the case for Republicans, who are more likely to prioritize economic growth over environmental protection (Jones, 2023). Democrats, however, who are more likely to prioritize the environment, may be more receptive to arguments that emphasize the environmental value of bee conservation, such as its contribution to global biodiversity goals (e.g., the Global Diversity Framework; United Nations Convention on Biological Diversity, 2022).
In sum, because we expected members of different political groups to (1) have diverging attitudes toward bee conservation, (2) organize their thinking about the issue in systematically different ways, and therefore (3) find different types of messages persuasive, we found it to be an appealing issue for testing a cognitive-structural approach to message design.
Study 1: Node Identification for Message Topics
As the preceding discussion suggests, testing a cognitive-structural approach to message design requires a preliminary investigation of (1) which concepts are central to an audience’s shared cognitive structure for a given issue and (2) whether that structure (and, therefore, the central concepts) differs for different segments of the audience. Study 1 was designed to address these questions in the context of a statewide pollinator conservation effort. Formally, the research questions were:
RQ1AN: Which nodes are identified as central in the attitude networks for this issue?
RQ1SN: Which nodes are identified as central in the semantic networks for this issue?
RQ2AN: How do Republicans’ and Democrats’ attitude networks differ for this issue?
RQ2SN: How do Republicans’ and Democrats’ semantic networks differ for this issue?
Method
Sample
Participants were recruited online through CloudResearch Prime Panels in June of 2021 with the incentive that they would receive a small amount of compensation for their participation. A minimum sample size of 250 for each group was targeted, which was appropriate given the moderate size of the attitude networks to be estimated and our use of continuous measures (see Dalege et al., 2017a). Across all four studies, data collection was limited to adults in Pennsylvania (where most of the authors are located and where the pollinator conservation effort is taking place) to eliminate differences that might emerge across different states or countries. A total of 1018 eligible participants filled out the survey. Because the focus was specifically on identifying differences between Republicans and Democrats, however, any participants that did not indicate a party affiliation (n = 6) or identified as moderate Independents (n = 155) or members of some other party (n = 32) were excluded from the final sample. Given that prior research has suggested that Independent leaners (those who call themselves Independents but admit an ideological preference) act in many ways like partisans (Petrocik, 2009), we classified both identified Republicans and conservative-leaning Independents as Republicans (n = 414) and both identified Democrats and liberal-leaning Independents as Democrats (n = 411) for the purposes of the study (see OSF page 3 for full demographics). Across all four studies, several steps were taken to ensure data quality. CloudResearch’s Sentry system has built-in features for data quality protection, including bot detection, IP deduplication checks, and honesty verification. We also included attention check questions and reviewed participants’ responses to check for incoherent open-ended answers or instances of straight lining.
Procedure
Participants accessed the online survey through Qualtrics. After giving their informed consent, they responded to an open-ended prompt for the semantic network analysis and measures of each node in the attitude network. The set of nodes measured was designed to capture participants’ evaluative responses to bees (beliefs, attitudes, emotions) as comprehensively as possible. The selection of items drew on the expertise of a subject matter expert and on existing theoretical work on attitudes (e.g., Albarracín & Shavitt, 2018; Bagozzi et al., 1979; Oskamp & Schultz, 2014). All procedures were approved by the institutional review board at Penn State.
Measures
The prompt for the semantic network analysis was: “What do you know about bees? Please write down anything that comes to mind.” Response length was unrestricted.
To capture evaluative reactions, beliefs were measured with 10 items focused on perceptions of whether or not bees were important for different reasons (e.g., “for producing honey”), evaluations were measured with 12 items focused on general impressions of bees (e.g., 1 = boring, 7 = fascinating), and emotional responses were measured with seven items focused on how people might typically feel when seeing a bee outside (e.g., “happy,” “nervous”). Items were measured on 7-point Likert scales (1 = strongly disagree, 7 = strongly agree). The complete list of items across all four studies can be found on the project’s OSF page.
Analysis Procedure
Attitude Network Analysis
We applied a Gaussian model to estimate the attitude network using partial correlation coefficients among the continuous measures of the evaluative reactions (Epskamp, 2016; Foygel & Drton, 2010). Specifically, we used the EBICglasso function in R (R Core Team, 2023) to generate Gaussian graphical models using the Bayesian Information Criterion (BIC) and estimate the network and node centrality. The network and node centrality measures were estimated separately for the Democrat and Republican subsamples. The node centrality measures included strength, betweenness, and closeness scores. Because of limited variability in the closeness scores for each node, however, we focused primarily on strength and betweenness scores (see Table 1). Statistically, strength scores represent the sum of edge values for a node, and betweenness scores represent the unweighted frequency with which a node lies along the shortest path between a pair of other nodes.
Study 1: Attitude Network Node Centrality Estimates.
Note. * = node selected for Study 2–4; bolded values indicate nodes with the largest centrality estimates within each subsample.
Semantic Network Analysis
To prepare the open-ended data for semantic network analysis, responses from Democrats and Republicans were divided into separate files and pre-processed using the package tm (Feinerer & Hornik, 2023) in R. This process involved removing punctuation, extra blank spaces, numbers, symbols, and stop words, including articles (e.g., the, an), pronouns (e.g., I, he, they), and “to be” verbs (e.g., am, is). The two files were then analyzed using WORDij 3.0 (Danowski, 2013) to generate the directional matrices of word co-occurrences that could be used to construct the semantic networks. The program parameters were set to exclude words and word pairs with fewer than three occurrences from the analysis; words were considered to co-occur if they fell within two words of one another in the text. The matrices were binarized to facilitate interpretation 4 , and then UCINET 6.721 (Borgatti et al., 2002) was used to estimate normalized indegree, outdegree, and betweenness centrality (see Freeman, 1978) for the different words (i.e., nodes) in each network.
Results
To answer RQ1AN and RQ2AN, the nodes with the highest betweenness and strength scores were identified in the attitude networks for the Democrat and Republican subsamples, respectively. As shown in Table 1, highly central nodes for the Republican subsample included evaluations of bees as wonderful, interesting, necessary, appealing, safe, valuable, and important; beliefs about the importance of bees for farmers; and happy emotional responses. Highly central nodes for the Democrat subsample included evaluations of bees as safe, necessary, and beautiful; beliefs about the importance of bees for honey production, wild plants and animals, biodiversity, and farmers; and afraid emotional responses. These findings suggest that there was some overlap in structure between Republicans and Democrats, but also some important differences in which concepts were most central.
To answer RQ1SN and RQ2SN, the nodes with the highest indegree, outdegree, and betweenness scores were identified in the semantic networks for the Democrat and Republican subsamples. Whereas the most central concepts differed between the two political parties in the attitude network analysis, this was not the case in the semantic network analysis. Aside from two key words from the prompt (bee, know), the most central terms for both Democrats and Republicans were honey, sting, pollination, flower (most often as part of the word pair pollination flower), and make (most often as part of the word pair make honey) (see Table 2).
Study 1: Semantic Network Node Centrality Estimates.
Note. In-degree, out-degree, and betweenness are normalized. Terms listed are the ten most central terms, across the three estimates, for each group.
Study 1 Discussion
Consistent with previous findings (Cruz et al., 2025), Study 1 results indicate that attitude networks and semantic networks provide two different views of cognitive structure. In the former, central nodes in both groups focused on perceptions of bees as (un)safe and useful, including evaluations of bees as necessary and beneficial for farmers. In contrast, although these two broad themes were also evident in the semantic networks—participants focused on how bees make honey, sting, and pollinate—the specific concepts or terms that were most central largely did not overlap. Pollination, for instance, was quite peripheral in the attitude networks despite its prominence in the semantic networks. Likewise, words like farmer, interesting, and biodiversity were absent from the semantic networks despite their prominence in the attitude networks.
The two types of networks also led to different conclusions about whether cognitive structures for this issue differ between Republicans and Democrats. Based on the attitude network, there did appear to be differences. In particular, the primary distinction seemed to be that Republicans’ attitude networks were organized around general positive-negative evaluations of bees (e.g., the extent to which bees are interesting, necessary, and valuable), whereas Democrats’ attitude networks were organized around specific beliefs about bees, especially their environmental value (e.g., the importance of bees for improving biodiversity and supporting wild plants and animals). Based on the semantic network, in contrast, no differences were observed between the two groups. Instead, there appeared to be consistency among people from different political orientations in their explicit organization of cognitions related to this particular attitude object, perhaps because bee conservation, unlike other environmental issues, is not a highly politicized topic. Overall, the results suggest that (1) attitude networks and semantic networks produce different expectations for persuasive message effects and, thus, different strategies for message design; and (2) the link between political identification and cognitive structure depends on which network is used to estimate that structure, such that attitude and semantic networks lead to different expectations about whether message topic selection should differ for Republicans and Democrats.
Study 2: Ruling Out Message Quality as an Alternative Explanation
To test the divergent strategies for message design indicated by the results of Study 1, we selected eight nodes (i.e., message topics) as subjects of further investigation in Study 2–4. The nodes were selected to facilitate a strong empirical test of predictions drawn from both the attitude network and semantic network results. In particular, we selected one topic central to both networks (safe, closely connected to the term sting), one topic central to neither network (natural beauty), one topic central only to the attitude network (necessary), and one topic central only to the semantic network (pollination). To further examine the divergent expectations about audience segmentation, we also selected two topics central to the attitude network for Republicans but not Democrats (interesting, wonderful) and two topics central to the attitude network for Democrats but not Republicans (biodiversity, wild plants and animals). Because the semantic network indicated no differences in structure between the two groups, these nodes also enabled an additional test of which network provided a stronger basis for message design.
Prior to testing the effects of messages targeting these topics, we sought to examine possible alternative explanations for any differential persuasive effects that might emerge. Specifically, we wanted to establish evidence that different patterns of effects could not be accounted for by reasons other than differences in the topics’ positions in cognitive structure. To do so, Study 2 focused mainly on assessing perceived message effectiveness (PME). PME is a measure of the degree to which a persuasive message will be positively evaluated by recipients regarding its ability to persuade (Dillard et al., 2007). Research has demonstrated the predictive validity of PME as an indicator of actual message effectiveness (e.g., Bigsby et al., 2013; Dillard & Ha, 2016). A systematic review also found that PME ratings constitute a plausible indicator of likely message effectiveness, successfully predicting the majority of outcomes examined (Noar et al., 2018). Accordingly, the first objective of Study 2 was to establish evidence that all eight messages developed based on the results of Study 1 exhibited parity in PME.
Establishing that the messages were perceived to be of similar quality was important for our purposes because we wanted to identify the extent to which messages would have different persuasive effects only because they were targeting topics in different positions in the cognitive structure. These differences in position would not be expected to affect PME because they are distinct from the argument’s overall quality; strong, logical, high-quality messages can be developed to target any node in an audience’s cognitive structure, whether central or peripheral. Furthermore, whereas people may be salient of what they find important or central to their decisions for some issues, this may not always be the case. As Alvaro and Crano (1997) note, people are often not aware of the connections between different concepts in their cognitive structure, especially when there are not logical links between them. Accordingly, people are unlikely to make a conscious connection between PME and a targeted concept’s position in cognitive structure. Differences due to structure would thus be expected to emerge even when two messages are of equally high quality.
Additionally, we examined source credibility for the intended source of all eight messages. Source credibility refers to participants’ perceptions of a message source’s expertise and trustworthiness (Hovland et al., 1953). Source credibility can impact message persuasiveness. For example, research indicates that messages from sources with high perceived source credibility are more likely to be accepted than messages from sources with low perceived credibility (Hovland & Weiss, 1951), and messages from credible sources have stronger effects on beliefs and subsequent behavior change (Lee & Stevens, 2022). Accordingly, the second objective was to establish that perceived credibility was high for the stated message source.
Method
Sample
Participants (N = 267) were recruited online in April–May 2023 using the same procedures as Study 1. The plurality identified as Democrats (42.7%), followed by Republicans (32.6%) and Independents (20.2%). See OSF page for additional demographics.
Procedures
The eight messages used in the study (see OSF page for materials) were created by the authors using facts and information suggested by entomology faculty and graduate students. To emulate a realistic situation in which participants would be likely to encounter bee conservation messages in everyday life, the messages were designed to look like Twitter posts 5 from the Center for Pollinator Research at Penn State, a well-known and recognizable university among Pennsylvania residents. Each message was accompanied by an image of a native bee with a graphic highlighting the node (message topic) targeted by the message (e.g., “bees are interesting”). Messages and images were reviewed by experts in entomology to verify accuracy of the factual information and appropriateness of the images. The length of the messages ranged from 44–47 words and 269–280 characters, to match the length allowed for actual Twitter posts.
Participants were told that they were being asked to help evaluate “some messages that may be used in a campaign to promote bee conservation” in Pennsylvania. They were then randomly assigned to see one of the eight possible messages and asked to fill out the measures of PME about it. This process was then repeated twice more, so that each participant evaluated a total of three randomly selected messages. After evaluating all three, they filled out the measures of perceived source credibility for the source (the Center for Pollinator Research). Participants were then given a chance to leave open-ended comments about the messages and asked for their demographic information. With the repeated measures design, power analyses (using G*Power; Faul et al., 2009) indicated that a sample size of 163 was needed to detect a small effect (f = .10) with moderately high power (.80). Thus, the obtained sample size (N = 267) was sufficient.
Measures
Perceived Message Effectiveness
Drawing on existing measures (e.g., Davis et al., 2013; Zhao et al., 2011), perceived message effectiveness was evaluated in two ways. First, perceived quality was measured with seven items (e.g., “this message is worth remembering,” “this message makes me think about helping bees”). Second, perceived impact was measured with six items. These items asked people to evaluate what effect the message would have on the average person (e.g., “this message would convince the average person to click the link,” “this message would convince the average person to do something to help bees”). All items were measured on 7-point Likert-type scales (1 = strongly disagree, 7 = strongly agree).
Source Credibility
Source credibility was measured using McCroskey and Teven’s (1999) scale. The scale is proposed to measure three factors: competence (e.g., “intelligent-unintelligent”), caring/goodwill (e.g., “insensitive-sensitive”), and trustworthiness (e.g., “honest-dishonest”). Each factor is measured with six items on 7-point semantic differential scales.
Confirmatory Factor Analyses
The fit of the source credibility items to the proposed three-factor structure was examined with confirmatory factor analysis (CFA; Hunter & Gerbing, 1982) using the less R (Gerbing, 2023) and lavaan (Rosseel et al., 2023) packages in R. The lavaan package was also used to conduct a multilevel CFA (MCFA; Muthén, 1994) of the message quality and impact items. This approach was adopted because each subject responded to these scales for three different messages, creating a nested data structure and introducing substantial dependence in the data: for quality, ICC(1) = .68; for impact, ICC(1) = .79. After dropping invalid items, the simple CFA showed good fit of the data to a single source credibility factor, χ2 (14) = 54.35, p < .001, CFI = .97, SRMR = 0.04. The MCFA also indicated excellent fit of the PME data to a two-factor model, χ2 (128) = 393.21, p < .001, CFI = .96, SRMR (within) = 0.04, SRMR (between) = 0.03. Retained items were averaged as composite measures of source credibility (α = .92) and perceived message quality (α = .94) and impact (α = .95). For additional details and factor loadings, see OSF page.
Results
Message Comparisons
To examine whether the eight messages differed in quality or perceived impact, we constructed a multilevel model (MLM) to account for the fact that participants’ responses to these measures were nested within messages. Then, we ran an unconditional means model to identify whether there was evidence that ratings clustered by message. High ICC(1) values would indicate evidence of clustering, suggesting that ratings of some messages differed from others in a systematic way. Low ICC(1) values, on the other hand, would suggest that such clustering did not occur, providing evidence of similarity across the different messages. The results revealed no evidence of clustering for either message quality or perceived impact, both ICC(1)s < 0.01. Accordingly, the MLM found almost no systematic variance in message ratings, indicating that the messages were evaluated equally highly in PME. One-sample t-tests also demonstrated that average ratings of quality (M = 5.66, SD = 1.06) and impact (M = 4.87, SD = 1.27) were substantially higher than the scale midpoint (4): for quality, t (798) = 44.41, p < .001, d = 3.10; for impact, t (798) = 19.46, p < .001, d = 1.39.
Although both groups rated the messages highly, however, independent-samples t-tests suggested that Democrats evaluated message quality, t (665) = 3.77, p < .001, r = .15, and perceived impact, t (666) = 2.25, p = .03, r = .09, somewhat more highly than did Republicans.
Source Credibility
Finally, a one-sample t-test demonstrated that average ratings of source credibility (M = 6.02, SD = 1.12) were substantially higher than the scale midpoint (4), t (266) = 29.45, p < .001, d = 3.71. An independent samples t-test indicated no statistically significant differences between Democrats and Republicans in their ratings of source credibility, t (221) = 1.79, p = .08, r = .12.
Study 2 Discussion
Overall, the results of Study 2 suggest that the messages developed based on the results of Study 1 were all evaluated positively on PME, albeit somewhat more so for Democrats than Republicans. Moreover, these evaluations did not differ systematically from message to message, indicating that all messages were evaluated in a similar manner regardless of where the targeted node was located in cognitive structure. Source credibility was also evaluated favorably. These results provided preliminary evidence that the messages could be expected to impact attitudes and behavior. Furthermore, they provided the foundation for a strong empirical test of the framework, as any differences in message effects that might emerge could not be accounted for by differences in overall message quality or perceptions of the source.
Study 3: Core Test of Identified Messages
Building on the results from Study 2, the aim of Study 3 was to test specific predictions about message effects based on the cognitive structures identified in Study 1. The first set of predictions focused on testing this cognitive-structural approach to message design without regard to between-group differences in structure. As noted above, this approach proposes that messages targeting more central nodes in the cognitive structures will lead to greater persuasive effects than messages targeting less central nodes. Given the Study 1 results, theorizing about attitude networks led to the prediction that:
H1AN: On average, messages targeting more central nodes in the attitude network (necessary, safe) will be more persuasive than messages targeting less central nodes (pollination, natural beauty).
Theorizing about semantic networks, on the other hand, led to the prediction that:
H1SN: On average, messages targeting more central nodes in the semantic network (safe, pollination) will be more persuasive than messages targeting less central nodes (necessary, natural beauty).
The diverging theoretical expectations also led to specific competing predictions:
H2AN: On average, a message targeting a node central only to the attitude network (necessary) will be more persuasive than a message targeting a node central only to the semantic network (pollination).
H2SN: On average, a message targeting a node central only to the semantic network (pollination) will be more persuasive than a message targeting a node central only to the attitude network (necessary).
We also derived another set of predictions focused on investigating whether differences in shared cognitive structure between Republicans and Democrats would lead to different message effects. Based on the semantic network, no differences were expected, because no differences emerged in the structure of the semantic network between the two groups. The attitude network, on the other hand, led to the following predictions:
H3AN: Messages targeting the interesting and wonderful nodes will be more persuasive for Republicans than Democrats.
H4AN: Messages targeting the biodiversity and wild plants and animals nodes will be more persuasive for Democrats than Republicans.
These hypotheses were all pre-registered (https://aspredicted.org/DGQ_48Z).
Method
Sample
Participants were recruited online in May–August 2023 using the same procedures as Study 1–2. The sample size needed to detect an average message effect (for persuasion, r = .18; Rains et al., 2018) with moderately high power (.80) using the planned contrasts needed to test the hypotheses (three groups) was estimated to be N = 303. With this in mind, CloudResearch was contracted to recruit N = 1000 participants at the pre-test at a projected 50% retention rate, for a planned post-test sample of N = 500. Unfortunately, however, recruitment underperformed projections, with 249 participants completing the post-test. Including leaners, the final sample was evenly split between Democrats (n = 102, 41.0%) and Republicans (n = 109, 43.8%). See OSF page for additional demographics for each subsample.
Procedure
At the pre-test, participants who gave their informed consent were told that the survey would focus on their thoughts and feelings about bees. The pre-test survey included demographic questions and the baseline measures of conservation attitudes and behavior. Approximately two weeks later, the same participants were recruited for the post-test. These participants were randomly assigned to one of the eight message conditions (i.e., necessary, safe, biodiversity, wild plants and animals, interesting, wonderful, pollination, or natural beauty). The messages were identical to those used in Study 2. After viewing the message, participants again filled out measures of conservation attitudes and behavior.
Measures
Bee-Focused Conservation Attitudes
Respondents were asked about their attitudes toward bee conservation with six items (e.g., “I think that bee conservation is a good idea,” “I am in favor of bee conservation”) measured on 7-point Likert-type scales (1 = strongly disagree, 7 = strongly agree). These items were identical at the pre- and post-test.
Bee-Focused Conservation Behavior
At the pre-test, respondents were asked whether or not they had taken any actions to support bees in the past. Seven behaviors (e.g., “I seek out information about bees,” “I spend time learning about bees”) were measured using 7-point Likert-type scales (1 = never, 7 = all of the time). The same items were used at the post-test but reworded to ask how likely they would be to engage in those behaviors in the future (1 = extremely unlikely, 7 = extremely likely).
Confirmatory Factor Analysis
Using the same procedures as in Study 2, a simple CFA was used to examine the fit of the pre-test and post-test attitude and behavior items to a two-factor structure. At the pre-test, good fit was obtained after removing invalid indicators from each scale, χ2 (51) = 310.73, p < .001, CFI = .97, SRMR = 0.04. Removing the same items also produced good fit at the post-test, χ2 (13) = 43.12, p < .001, CFI = .99, SRMR = 0.04. The remaining items were averaged to form a composite measure for each scale.
Results
Prior to examining the main hypotheses and research questions, we examined whether overall attitude and behavior change occurred as a result of message exposure. The results indicated that there was a ceiling effect for conservation attitudes. Both Democrats (M = 6.11, SD = 1.11) and Republicans (M = 6.16, SD = 0.96) already had strong positive attitudes toward bee conservation at the pre-test, so there was little room for the sample to change from the pre-test to the post-test, t (248) = 1.45, p = .15, r = .09. On the other hand, substantial and statistically significant changes were observed on the behavior measure, t (248) = 20.82, p < .001, r = .80. Democrats and Republicans both reported that they planned to engage more often in bee conservation behaviors at the post-test (D: M = 4.82, SD = 1.49; R: M = 4.70, SD = 1.48) than they had at the pre-test (D: M = 3.03, SD = 1.41; R: M = 3.00, SD = 1.46). A 2 (time points) x 2 (political groups) mixed ANOVA also indicated that the interaction between time and party identification was not significant for either attitudes, F (1, 209) = 1.22, p = .27, η2 = .01, or behavior, F (1, 209) = 0.26, p = .61, η2 < .01. Thus, both Democrats and Republicans were, on average, impacted equally strongly by message exposure. Given the ceiling effect for attitudes, the main analyses focused on change scores for behavior as the dependent variable of interest.
H1AN
To test H1AN, which focused on evaluating the overall predictions drawn from the attitude network, we used contrast coding to differentiate between the message conditions that were predicted to be most effective by the attitude network (necessary, safe = +1), those predicted to be least effective by the attitude network (pollination, natural beauty = –1), and the remaining conditions (= 0). Then, to see if the outcomes matched this predicted pattern, we conducted a one-way ANOVA using this variable, with behavior change as the dependent variable. Results indicated that the contrast code variable did not have a significant effect on behavior change, F (2, 246) = 0.16, p = .85. Thus, H1AN was not supported; there was no evidence to suggest that targeting more central nodes in the attitude network led to more behavior change than targeting less central nodes (MΔB = 1.80 vs. 1.81). 6
H1SN
To test H1SN, which focused on evaluating the overall predictions drawn from the semantic network, we repeated the procedure from H1AN, but with a different set of contrast codes to differentiate between the message conditions that were predicted to be most effective by the semantic network (safe, pollination = +1), those that were predicted to be least effective by the semantic network (necessary, natural beauty = –1), and the remaining conditions (= 0). Results indicated that the contrast code variable did not have a significant effect on behavior change, F (2, 246) = 0.61, p = .55. Thus, H1SN was not supported; there was no evidence to suggest that targeting more central nodes in the semantic network led to more behavior change than targeting less central nodes (MΔB = 1.69 vs. 1.92).
H2AN & H2SN
To test the competing predictions from the two networks—in which a node was predicted by one network and not the other—we repeated the procedure from the previous hypotheses, but with a different set of contrast codes to differentiate between the node predicted exclusively by the attitude network (necessary = +1), the node predicted exclusively by the semantic network (pollination = –1), and the remaining nodes (= 0). Results indicated that the extent of behavior change did not differ significantly across these messages, F (2, 246) = 0.29, p = .75. Thus, there was no evidence to suggest that either network outperformed the other (MΔB = 1.82 for necessary message, MΔB = 1.58 for the pollination message). Neither H2AN nor H2SN was supported.
H3AN & H4AN
To test whether the attitude network could successfully predict which messages were more effective for Democrats and which were more effective for Republicans, we used contrast coding to differentiate message conditions that were predicted to be more effective for Republicans (interesting, wonderful = +1), conditions that were predicted to be more effective for Democrats (biodiversity, wildlife = –1), and the remaining conditions (= 0). We then ran a 3 (message codes) x 2 (political parties) between-subjects ANOVA to examine whether the pattern of effects differed by political party. The interaction effect was non-significant, F (2, 205) = 0.36, p = .70. In sum, neither H3AN nor H4AN was supported.
Post-hoc Analysis: Individual Message Comparisons
Given that the pattern of effects across message conditions did not conform to any of the predictions derived from the semantic or attitude networks, a post-hoc 8 (message conditions) × 2 (political parties) between-groups ANOVA was conducted to examine whether any unexpected differences emerged. Overall, the effect of message condition on the amount of behavior change was marginally significant, F (7, 195) = 1.99, p = .06; but the interaction between condition and political party was not, F (1, 195) = 1.36, p = .23. Inspecting the marginal means suggested that the effect of message condition could primarily be accounted for by the fact that behavior change was greater in the wild plants and animals condition than in the other conditions (MΔB = 2.66 vs. 1.45–1.92). The fact that this message produced the greatest change among Democrats (MΔB = 2.87) was consistent with expectations given its centrality in their attitude network. The fact that it also produced the greatest change among Republicans, however (MΔB = 2.44), was unexpected.
Study 4: Replication and Extension
In Study 4, we aimed to replicate and extend the findings from Study 3. To address potential weaknesses of the sample in Study 3, we focused on recruiting a larger sample of individuals for Study 4. We also focused only on participants at each extreme of party identification in order to address the possibility that the negligible differences between Democrats and Republicans in Study 3 were due to the presence of moderates with only slight ideological preferences.
For this study, we also employed a post-test only design with a control group rather than a pre-test post-test design. A post-test only design had two advantages for the purposes of replication and extension. For one, it allowed us to avoid concerns that attrition would again lead to a lower sample size than desired. For another, it allowed us to rule out the possibility that testing effects could have accounted for the results. Public knowledge of bees is generally very low (Wilson et al., 2017), so we considered the possibility that merely being exposed to the survey questions could have been informative or educational for subjects. For instance, reading and responding to the statement, “Bees support many different wild plants and animals” may have been sufficient on its own to prompt people to think about bees’ environmental value, even before exposure to the persuasive messages. If exposure to these statements ultimately influenced participants’ cognitive structures (as education has the power to do; Ausubel, 1963), then that could account for the unexpected nature of the results. A post-test only design provided a way of examining the message effects without concerns about the possible effects of prior exposure to the survey.
Method
Sample
Participants (N = 655) were recruited online in February–March 2024 using the same procedures as Study 3. To ensure only individuals with strong party affiliations completed the survey, a screener question at the beginning of the survey was used to exclude those who identified as slight Democrat, neither, or slight Republican. The final sample was approximately evenly split between strong Democrats (49.8%, n = 326) and strong Republicans (50.2%, n = 329). See OSF page for demographics for each subsample.
As noted in previously, the sample size needed to detect an average message effect (for persuasion, r = .18; Rains et al., 2018) with moderately high power (.80) using the planned contrasts needed to test the hypotheses (three groups) was estimated to be N = 303. Thus, the obtained sample size of 655 was ample.
Procedure
Using a post-test only design, participants were randomly assigned to one of the eight message conditions established in Study 3 (i.e., necessary, safe, biodiversity, wild plants and animals, interesting, wonderful, pollination, or natural beauty) or to the control group. The control group was assigned to evaluate a message about a non-environmental topic, styled similarly to the other messages but with a generic message promoting kindness over a picture of a flower. After viewing the message, all participants completed demographic questions and measures of conservation attitudes, conservation behavior, and perceived message quality.
Measures
The measures of conservation attitudes and behavior were identical to those in Study 3. The measure of perceived message quality was identical to that used in Study 2. Using the same procedures as in the previous studies, a CFA was conducted to test the fit of the data on these measures to a three-factor structure. Good fit was obtained after dropping invalid items (two each from the attitude and behavior scales, three from the quality scale), χ2 (62) = 261.82, p < .001, CFI = .97, SRMR = 0.03. The remaining items were averaged to form a composite measure for each scale.
Results
Preliminary one-sample t-tests indicated that the average rating of quality (M = 5.40, SD = 1.21) was substantially higher than the scale midpoint (4), t (654) = 29.57, p < .001, d = 2.31. An independent samples t-test also suggested modest but statistically significant differences between Democrats (M = 5.50, SD = 1.20) and Republicans (M = 5.30, SD = 1.22) in their ratings of message quality, t (653) = 2.06, p = .04, r = .08.
Examining the descriptive statistics for the conservation attitude and behavior measures also suggested that the messages, overall, were impactful. Across the experimental message conditions, both Democrats and Republicans reported after message exposure that they intended to engage in bee-focused conservation behavior at levels within error of the means observed at the post-test for Study 3 participants (respectively, M = 4.58, SD = 1.54; M = 4.41, SD = 1.63), well above what was observed at the Study 3 pretest (Ms = 3.00–3.03). As before, conservation attitudes were also strongly positive for both Democrats (M = 5.97, SD = 1.13) and Republicans (M = 5.86, SD = 1.18).
Once again, however, there was remarkable parity of outcomes across the message conditions. Focusing on behavior, an 8 (conditions) x 2 (political parties) independent-groups ANOVA revealed no main effect of condition, F (7, 577) = 0.75, p = .63; no main effect of political party, F (1, 577) = 0.99, p = .32; and no interaction between party and condition, F (7, 577) = 0.42, p = .89. The same patterns were also observed for conservation attitudes. Thus, similar to what was found in Study 3, all of the messages appeared to be similarly effective in persuading people from both political parties of the importance of bee conservation, regardless of the centrality of the targeted node in either the attitude network or semantic network. 7
Unexpectedly, the control message also resulted in strong intentions to engage in bee-focused conservation behavior among both Democrats (M = 4.92, SD = 1.40) and Republicans (M = 4.54, SD = 1.29) as well as in highly positive conservation attitudes (respectively, M = 6.15, SD = 0.95; M = 6.05, SD = 0.90). In retrospect, this may have occurred because the message promoted a universalist or benevolent value (kindness) in conjunction with an image that evoked nature and the environment (a flower). Universalism, which refers to “understanding, appreciation, tolerance and protection for the welfare of all people and for nature,” and benevolence, which refers to “preservation and enhancement of the welfare of people with whom one is in frequent personal contact” (Schwartz et al., 2012, p. 664), have previously been identified as positive predictors of pro-environmental behavior (e.g., Katz-Gerro et al., 2017). Accordingly, priming participants to think about the importance of kindness may have prompted them to feel kindly toward bees as well. Regardless, although such an effect might have been predicted by traditional hierarchical approaches to attitude structure (e.g., Hunter et al., 1976), it would clearly not be predictable based on either attitude or semantic network structure.
Post-hoc Analysis: Node-level Shifts across Studies
As a final step to attempt to rule out possible extraneous factors that might have accounted for the apparent lack of association between cognitive structure and message outcomes, we conducted an additional post-hoc analyses to examine the possibility that cognitive structure diverged across samples. If the central nodes for the Study 1 sample differed from the central nodes for the Study 3 and 4 samples, that could explain why predictions based on Study 1 centrality data failed to find support in Study 3 and 4. To examine this possibility, we estimated the pre-test attitude network for Study 3 and computed correlations among those node-level centrality estimates and the estimates from Study 1. 8 Scores were generally stable for Republicans (betweenness scores: r = .52; tie strength: r = .50), but less so for Democrats (betweenness: r = –.21; tie strength: r = .36). These results suggest that cognitions related to this topic are not strongly crystallized—at least among members of the general public—meaning different samples likely do provide somewhat different estimates of cognitive structure. Still, the extent to which these differences could feasibly account for the null results is limited. Though some expectations would change (e.g., safe would no longer be a highly central node for both Republicans and Democrats), others would not (e.g., interesting would still be highly central for Republicans but not for Democrats). Furthermore, the estimated attitude network would still ultimately lead us to predict that different messages would differ in their impact, not that all messages would be equally effective.
General Discussion
Although there is ample research and theory on many aspects of persuasive message design, available guidance on how to choose which arguments to include in a message is more limited. To address this gap, Cruz et al. (2025) recently suggested that a cognitive-structural approach to message design could be a promising theoretically grounded technique for argument selection. However, as is the case with many examples of formative message design research (e.g., Bai et al., 2009), preliminary tests of this approach did not explicitly test message effects. Accordingly, the purpose of this investigation was to build on those preliminary findings by testing more specific predictions about message effectiveness drawn from insights about cognitive structure. We also sought to clarify whether semantic or attitude networks might provide a better foundation for designing messages for different audiences—in this case, different political groups.
Study 1 replicated previous findings that different estimates of cognitive structure suggest different approaches to message design. The semantic network suggested that effective messages, regardless of someone’s political affiliation, would emphasize bees’ importance for pollination and honey production and address common fears about bee stings. The attitude network, in contrast, suggested that effective messages would differ depending on political party, with messages tailored for Democrats focusing more on bees’ environmental value for biodiversity and wildlife and messages tailored for Republicans focusing more on bees’ general appeal as interesting and wonderful creatures. The results of Study 3 and 4, however, suggest that neither strategy outperformed the other. Contrary to expectations, all eight messages substantially increased intentions to engage in bee-focused conservation behavior among both Republicans and Democrats. Furthermore, although the message focused on bees’ benefits for wildlife was somewhat more effective than the other messages in Study 3, the same pattern was not identified in Study 4. Study 4 also suggested that targeting higher-order values may be just as effective in encouraging pro-environmental behavior as targeting nodes in the attitude network for a particular issue.
From an applied perspective, these findings are very encouraging. They suggest that the issue of bee conservation is viewed favorably by many people. In contrast to the political divide on many environmental issues (Cruz, 2017), bee conservation appears to have broad, bipartisan support. Furthermore, many different types of messages appear to be effective in encouraging people to act on their already positive attitudes toward this issue. This suggests that there are ample opportunities for practitioners in this area to have a positive impact. It also appears that these practitioners do not necessarily need to follow conventional recommendations that messages about environmental issues should stick to economic, utilitarian arguments to be effective, especially for conservative audiences (e.g., see Kusmanoff et al., 2020; Nisbet, 2009).
From a theoretical perspective, the findings are less encouraging. In general, the results suggest that evaluating an audience’s shared cognitive structure would be unlikely to enhance message design for this issue. In part, the apparent disconnect between cognitive structure and message effects could be attributed to the fact that careful message design appears to be less important for bee conservation than is often the case; all messages were effective, so choosing among them was a relatively trivial matter. Still, the results indicate that fundamental questions remain about how and to what extent cognitive structures can inform message design.
The more explicit cognitive structure for this issue, drawn from participants’ semantic network, indicated—consistent with the results—that Republicans and Democrats would not differ in their reactions to the messages. However, this structure was not able to identify which arguments would be most effective. The messages focusing on pollination and safety were no more effective than the other messages, and the most effective message in Study 3—about wild plants and animals—focused on a topic that did not appear in the semantic network at all. The implicit cognitive structure for this issue fared no better. Targeting central concepts in the attitude network was no more effective than targeting peripheral concepts, and there was no evidence that structural differences between Democrats and Republicans led to different message effects.
One potential explanation for our pattern of results is that the cognitive structures observed here, particularly the attitude network, were too diffusely connected for highly central nodes to play a meaningful role in how participants responded to the messages. For example, in Study 3, a post-hoc analysis showed that the average shortest path length (ASPL) was 17.59 for Republicans and 17.30 for Democrats. Given that higher ASPL values indicate lower connectivity (Dalege et al., 2017a, 2019), this suggests that the participants’ attitude networks were fairly sparse. Because change in cognitions is less likely to be inhibited in sparser networks (Dalege et al., 2019), this suggests that each of the targeted nodes was reasonably malleable. In other words, network connectivity may serve as a boundary condition for cognitive-structural approaches to message design. Future work should more carefully examine this possibility.
Another possibility is that indicators of cognitive structure aside from node centrality would be more predictive of message effectiveness. For example, the literature on automatic attitude activation (Bargh et al., 1992; Lodge & Taber, 2005; Morris et al., 2003) suggests that accessibility, which reflects how readily activation of one node will lead to activation of another node, may be a better indicator of which nodes will be best to target. If so, it is possible that all nodes targeted by our messages were equally accessible, which would then account for why they were all equally effective. We are doubtful that all these concepts were equally accessible, especially given that some of them were rarely if ever mentioned by subjects in their open-ended responses about bees. Still, we cannot rule out the possibility that response time measures that more directly examine relative accessibility would lead to different conclusions. Examining accessibility and centrality together in future studies would help address this possibility. Future studies might also consider the role of processing fluency for messages targeting central vs. peripheral (or more accessible vs. less accessible) nodes, which could also impact their persuasive effects (e.g., Song & Schwartz, 2008).
A final possibility is that each message may have impacted nodes beyond the one that was directly targeted. This could have occurred either automatically and preconsciously (Bargh et al., 1992; Lodge & Taber, 2005; Morris et al., 2003) or due to deliberate reasoning about the message’s relevance to other nodes (e.g., see Glaser et al., 2015). For instance, for participants reading the natural beauty message, the sentence “Bees make our state beautiful!” could have automatically activated the more general term “beautiful” in participants’ minds, drawing a preconscious connection to the node tapping evaluations of bees as beautiful creatures. Alternatively, participants could have reasoned that although it was not mentioned in the message, bees themselves might “contribute to the beauty of our gardens, parks, and natural landscapes” because they themselves are beautiful creatures. If, via either of these processes, the messages targeting peripheral nodes also impacted central nodes, then this could also explain why the messages were all similarly effective. If so, evaluating the potential for messages to impact off-target nodes may be an important additional consideration for message design.
Regardless, the findings make it clear that additional work is needed to understand when and how cognitive structures connect to persuasive effects. Our findings failed to support the assumptions that messages targeting central nodes are more persuasive than messages targeting peripheral nodes in the cognitive structure; that explicit or implicit structures in the form of a semantic network or attitude network may provide accurate expectations about which messages will be more persuasive; or that different political ideologies, as representations of different cognitive networks, will lead people to find different types of messages more or less persuasive. We are hopeful that additional work will advance our understanding, and that other researchers’ theoretical ideas will help to move this area forward. For instance, it is possible that message processing states and responses—such as elaboration, involvement, or need for cognition—may help to shed light on how persuasion is produced via cognitive structures.
Strengths and Limitations
This paper should be considered in light of its strengths and limitations. Regarding limitations, at least three are worth noting. First, as discussed above, participants’ networks for the issue of bee conservation were quite sparse. Future work would benefit from probing more ego-involving attitude objects characterized by denser networks, such as those related to politically charged topics (e.g., gun control) or tied closely to a particular group’s identity (e.g., bee conservation among beekeepers rather than the general public). Second, because our theoretical and methodological approach led us to focus on cognitive networks shared across target audience members, our estimated networks reflect members of different political affiliations in aggregate rather than individual-level information. It is possible that an individual's network would suggest different message strategies, though avoiding grouping would come at the cost of creating a much larger array of messages to accommodate larger numbers of individuals. From a practical perspective, it is also unrealistic to expect that campaign designers could craft and deliver individually targeted messages, even if that information was available. Still, a small-scale study that examined ways of constructing individual-level networks, such as semantic network analysis of larger corpuses of individuals’ writing about a topic (e.g., long-form essays or sets of social media posts) or free word association tasks (Tsai & Huang, 2002), could provide useful insights. Third, although we explored two different methods of mapping cognitive structure here, there are still many additional methods that could be explored. Comparing these methods to alternative procedures, such as mapping cognitive structure for more abstract concepts like “the environment,” could provide additional insights.
Regarding strengths, this set of studies offered a strong empirical test of whether identified message topics worked or not. Rather than relying only on the formative research in Study 1, we conducted an additional study to rule out alternative explanations for possible message effects (Study 2) and examined both messages that were predicted to work and those that were predicted to fail (Study 3–4). Formative research on message design sometimes draws conclusions or makes recommendations about messaging without testing those messages explicitly. Even when follow-up studies are conducted, they may focus on confirming that formative research has correctly identified messages that will be effective and neglect to test whether it has also correctly identified messages that will be ineffective (e.g., Parvanta et al., 2013). Our findings suggest that those types of procedures may be insufficient for developing effective messaging strategies. Despite the encouraging evidence for a cognitive-structural approach for message design uncovered in previous research (Cruz et al., 2025), the more precise empirical tests presented here suggest the need for additional research and refinement. Furthermore, the findings emphasize how critical it is to seek disconfirming evidence during the process of message design. If we had only included tests of whether or not messages predicted to be effective based on cognitive structure caused significant change, we would have concluded that the framework received support.
Conclusion
The series of studies presented here constitutes an initial empirical evaluation of a cognitive structure-focused approach to message design. Drawing on previous theoretical work, we anticipated that the most effective persuasive messages would be those that targeted highly central nodes within an audience’s implicit cognitive structure—in the form of their attitude network—or explicit cognitive structure—in the form of their semantic network. However, our findings ultimately failed to support these expectations. Future work can help clarify whether the results occurred because of a specific boundary condition for this approach, such as the sparsity of the target audience’s networks, and continue to advance our understanding of how cognitive structures impact persuasive message effects.
Footnotes
Acknowledgements
We would like to thank Christina Grozinger and Heather Frantz for their input and feedback on the message content and design.
Ethical Considerations
The studies reported in this manuscript were determined exempt from formal IRB review by the Office for Research Protections at the Pennsylvania State University (STUDY00017983, STUDY00022240).
Consent to Participate
Participants were provided with information about each study on the first page of the corresponding online Qualtrics survey. Participants indicated their informed consent by clicking a button and choosing to continue with the study.
Consent for Publication
Not applicable.
Funding
The authors disclosed receipt of the following financial support for the research reported in this article: This work was supported by a grant from the Communication, Science, and Society Initiative (CSSI), a partnership between Penn State’s Department of Communication Arts and Sciences and the Huck Institutes for the Life Sciences; and by the Commonwealth of Pennsylvania Department of Agriculture through Agriculture Resource Center funds awarded to the Center for Pollinator Research at The Pennsylvania State University.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statements
Data are available upon request to the corresponding author.
Open Practices
Data are available upon request to the corresponding author.