Sage Journals: Discover world-class research

Abstract

We investigate the effect of social media endorsements (likes, retweets, shares) on individuals’ policy preferences. In two pre-registered online experiments (N = 1,384), we exposed participants to non-neutral policy messages about the COVID-19 pandemic (emphasizing either public health or economic activity as a policy priority) while varying the level of endorsements of these messages. Our experimental treatment did not result in aggregate changes to policy views. However, our analysis indicates that active social media users did respond to the variation in engagement metrics. In particular, we find a strong positive treatment effect concentrated on a minority of individuals who correctly answered a factual manipulation check regarding the endorsements. Our results suggest that though only a fraction of individuals appear to pay conscious attention to endorsement metrics, they may be influenced by these social cues.

Keywords

social media social conformity political polarization COVID-19

Introduction

Social media has been hypothesized to have broad effects on politics (Zhuravskaya et al., 2020). However, the magnitude of these effects and the mechanisms through which they arise remain debated. This article studies how social media affects individuals’ policy preferences. In particular, we study endorsements, as evinced by common metrics of engagement: likes, ❤s, s, retweets, and shares. Social endorsements are a central feature of social media and are observed by billions of individuals around the world, thus even small effects may have important consequences for policy-making and political dynamics. Can the perceived support of social media messages affect how individuals evaluate policies?

To answer this question, we conducted two pre-registered online experimental studies in Europe (Ireland, n = 305, and Italy, n = 300) and the United States (n = 779) in the context of the COVID-19 pandemic and its policy trade-offs (public health vs. economic activity, Settele & Shupe, 2022).¹ The experiment allows us to isolate the effects of perceived support for policy choices in a controlled environment different from individuals’ own social media. We study endorsements, a specific but important feature of social media, but without conflating issues of social image, peer effects, or selective exposure. Instead, we exposed individuals to strangers’ tweets and endorsements, and examined their effects on individuals’ policy preferences in an anonymous survey. More specifically, we exposed participants to non-neutral policy messages about the COVID-19 pandemic, manipulated the perceived level of endorsements of these messages, and examined how this affected their policy attitudes.

We find no overall effects of endorsements on policy views. On aggregate, we estimate a precise zero effect of our treatment on the policy views of our participants. A factual manipulation check suggests that most individuals pay little attention to endorsement metrics, with only one-third of participants correctly answering a post-treatment question about these metrics.

One particular focus of our work studies the effects of endorsement metrics on active social media users, which we define as those who use Facebook or Twitter for at least 1 hr a day. These are individuals who are frequently exposed to engagement metrics and are thus likely to be sensitive and responsive to likes and retweets on social media posts. In addition, it is most important to understand how the behavior and attitudes of these individuals are affected by the online environment, as they are the ones who use the platforms most frequently.

We find that our experimental treatment shifted the policy views of active social media users by about 0.12 standard deviations, with the effect further concentrated on the minority of individuals who correctly answered the manipulation check. These results suggest that though only a small share of the population appears to pay conscious attention to likes or retweet metrics, they may be influenced by these social cues. These findings can have further implications for policy decision-making, since social media users are known to be more engaged with politics and can have a disproportionate influence on policy agendas (Barberá et al., 2019; Vaccari et al., 2015; Vaccari & Valeriani, 2021).

Our work is related to a growing literature on the relationship between social media and politics (Zhuravskaya et al., 2020). Social media has been shown to affect electoral outcomes (Fujiwara et al., 2021), legislative processes (Barberá et al., 2019), political knowledge (Munger et al., 2022), and protest participation (Enikolopov et al., 2020; Fergusson & Molina, 2019). However, there is still little understanding of how different features of social media affect behavior.

Studies have emphasized how social media exposes individuals to echo-chambers of predominantly like-minded information (Bakshy et al., 2015; Barberá, 2015; Halberstam & Knight, 2016) and amplifies political polarization (Allcott, Braghieri, et al., 2020; Gorodnichenko et al., 2021; Levy, 2021; Settle, 2018; Sunstein, 2018). Media concerns about the influence of social media on elections are also common,² yet many contend that these concerns may be overblown (Allcott & Gentzkow, 2017; Boxell et al., 2017; Eady et al., 2019; Gentzkow & Shapiro, 2011; Guess, 2021; Guess et al., 2020; Scharkow et al., 2020). As a way to sharpen our understanding of these issues, we study one precise mechanism, endorsements, through which social media may affect political dynamics.

Given that social pressure is known to shape behavior and views (Bursztyn & Jensen, 2017; Carlson & Settle, 2016; Cialdini & Goldstein, 2004), online social endorsements can be an important channel through which social media may affect policy preferences, especially in situations of evolving public opinion (Bursztyn et al., 2020; Casoria et al., 2021; Hensel et al., 2022). Furthermore, humans are known to rely on heuristics or mental shortcuts to make judgments and decisions (i.e., bounded rationality), especially in situations where information is scarce and uncertainty is high (Kahneman, 2003; Simon, 1955). In the context of our study, there was substantial uncertainty on how to balance minimizing the spread of COVID-19 while preserving economic activity. An opinion that is highly endorsed would appear to have a higher level of credibility (Luo et al., 2022; Shin et al., 2022), thus potentially making it more persuasive to the audience—in what is termed the endorsement heuristic (Hilligoss & Rieh, 2008). Relatedly, bandwagon heuristics, whereby individuals are likely to follow what others do can also explain these social dynamics (Banerjee, 1992; Bikhchandani et al., 1992; Metzger et al., 2010; Sundar, 2008). The use of heuristics, and thus the effectiveness of social media endorsements in shaping opinion, is expected to be prevalent in settings involving new issues where individuals are yet to form fixed opinions.

Related work has documented that online social endorsements and perceptions of support affect whether individuals select to read content (Anspach, 2017; Messing & Westwood, 2014), like or retweet messages (Alatas et al., 2019; Egebark & Ekström, 2018), and self-report voting (Bond et al., 2012, 2017), and can have broader implications for online political dissent (Morales, 2020). We contribute to this work by studying how the perceived endorsements attached to social media messages affect policy attitudes, and we do so in the context of the COVID-19 pandemic.

Background

Since we present results from studies conducted with nationally representative samples in Ireland, Italy, and the United States, here we briefly describe the social contexts at the time of our experimental intervention and data collection. Our surveys were conducted in July 2020, at the end of the first COVID-19 wave. Overall, there was substantial policy uncertainty and contentious debates regarding the trade-offs between public health and economic activity.

In Ireland, daily confirmed deaths per million had stabilized at 0.26 (Mathieu et al., 2020). The country was at the end of the first period of lockdown, which had a significant negative impact on the economy with unemployment rising up to 28% and gross domestic product (GDP) forecasted to decline by 10.5%.³ By the time our European data collection started on 8 July, cafés, restaurants, and non-essential retail outlets were allowed to open with social-distancing measures, though there were still restrictions on social gatherings.

Italy, having been severely affected early during the pandemic, was the first country to enact a nationwide lockdown on 9 March 2020. By early July, however, restrictions had gradually been eased and freedom of movement across regions had been restored, as COVID-19 deaths per million had also stabilized at 0.26 (Mathieu et al., 2020). Concerning the economic impact, by this date real GDP was forecasted to fall by over 11% in 2020.⁴

In the United States, lockdown policies varied across states with California being the first to issue a statewide stay-at-home order on 19 March, though by early April, about 90% of the US population were living under stay-at-home orders. By May 2020, the unemployment rate had grown to 14.7%, the highest since the Great Depression. While daily confirmed deaths per million was down to 1.55 in early July, by the time our US survey was sent out on 31 July, this metric had risen again to 3.27 (Mathieu et al., 2020). The number of confirmed cases had exceeded 3 million and many states postponed re-opening plans as case numbers rose.⁵

Experimental Design

Our main hypothesis is that the policy attitudes of participants are affected by social media metrics. As users conform to others’ preferences, social media affects policy attitudes by informing individuals about others’ views. Specifically, we hypothesize:

H1, Conformity: Individuals conform to views which appear more popular (as evinced by social media support metrics, that is, likes and retweets).⁶

We also report here our results from investigating additional sources of heterogeneity. First, we are particularly interested in studying whether social conformity effects arising from endorsement metrics are larger for active social media users, who are frequently exposed to, are more likely to pay attention to, and understand the significance of these metrics. Our definition of active social media users consists of those who use Facebook or Twitter for 1 hr or more each day (combined).⁷ Second, through the use of a factual manipulation check, we investigate the role of attention in our findings. In particular, if participants did not observe the endorsement metrics, we would not expect an effect to arise (i.e., this serves as a form of placebo check). We pre-registered these dimensions of potential heterogeneity along with a number of other variables, therefore, the results presented should be considered exploratory. These analyses help us understand the mechanisms through which the effects of endorsement metrics arise (or fail to do so).

Finally, we discuss two additional pre-registered hypotheses related to the order in which the messages appear (anchoring), and we present results on one other margin of heterogeneity, pre-treatment attitudes. This last margin of heterogeneity speaks to the literature on social media and political polarization. We find no strong patterns along other margins of heterogeneity; the estimates for all variables specified in our original pre-registration are shown in Appendix Figure A2.⁸

Implementation and Design

Our first survey was conducted using nationally representative samples in terms of age, gender, and region in Ireland (n = 305) and Italy (n = 300), and it was sent out on 8 July 2020. Our second survey was conducted in the United States, using a nationally representative sample in terms of age, gender, and census regions (n = 1,519), and was sent out on 31 July 2020. Both surveys were programmed in Qualtrics. The main analyses presented below pool the two surveys, but our main results are quantitatively similar when analyzing the samples separately.⁹ Unless otherwise noted, we follow the pre-analysis plans.

To recruit our sample, we contracted Dynata, formerly Research Now SSI, a survey company often used to recruit participants for research in social science (Krupnikov et al., 2021; Snowberg & Yariv, 2021). The company recruits panel members through various marketing channels and collects their sociodemographic information. To obtain a nationally representative sample, survey invitations are sent to potential respondents whose sociodemographic distributions match the one in the latest census data of the country (Bol et al., 2021). Respondents are rewarded for participating in surveys depending on the length and content of the survey. Data quality is ensured by identifying and, after checking, potentially removing random responding, illogical or inconsistent responding, overuse of item non-response (e.g., “don’t know”), and speeding (overly quick survey completion).¹⁰

We first measured participants’ pre-treatment policy attitudes using statements about COVID-19 policy responses, for example “The government’s highest priority should be saving as many lives as possible even if it means the economy will recover more slowly.”¹¹ Participants indicated their agreement with these statements on a 1 to 7 Likert-type scale. We standardized these responses and coded positive values as being pro-economy. In addition, we combined the questions into one index through principal components analysis.

We next randomized participants into one of three treatments: control, pro-economy, or pro-health.¹² In each treatment, participants are shown six tweets from strangers about COVID-19 policies, of which three are pro-economy and three are pro-health.¹³ In the control condition, all tweets have low endorsements (a low number of likes and retweets). In the pro-economy condition, the three pro-economy tweets are given high endorsements while the three pro-health tweets are given low endorsements. In the pro-health condition, the three pro-economy tweets are given low endorsements while the three pro-health tweets are given high endorsements.

The tweets were preceded by the following text:

The algorithms used on social media may sometimes present you with posts by complete strangers. You will now be shown 6 tweets. As if you were going through your own social media feed (eg Twitter or Facebook), please consider whether you would “like” or “retweet” each of the following 6 tweets.

Figure 1 shows an example of the experimental variation. The tweets were generated using https://www.tweetgen.com/ using the following input:

Text: We ran a search of COVID-19-related tweets on Twitter and selected six tweet messages, three pro-locking down (which we term “pro-health”) and three pro-opening up (which we term “pro-economy”).

Metrics: “Low” endorsement tweets have between 0 and 10 likes and 0 and 1 retweets. “High” endorsement tweets have between 50 and 100 likes and 10 and 20 retweets. While the metrics chosen for our “High” treatment may seem conservative, these numbers are more realistic for tweets by private persons (rather than famous people).¹⁴

User: The profile pictures are generated by an algorithm using the website https://thispersondoesnotexist.com/. No username is shown.¹⁵

Time: We randomly picked times and dates in the weeks before data collection.

An additional treatment dimension in the US sample exposed half the participants in each of the three above treatments to an attention prime prior to the six tweets. Participants were shown an unrelated tweet followed by three questions about the content, the timing of this tweet, and (importantly) the number of likes. We designed this manipulation to prime participants into paying careful attention to the subsequent six tweets and their endorsements, since absence of treatment effects can potentially be attributable to participants not noticing the metrics. We find that the prime did not reinforce the expected effect and in fact nullified the effect for social media users.¹⁶ However, this treatment also allowed us to assert that the absence of an effect for non–social media users is not due to them not looking at the endorsement metrics.¹⁷ Our analysis focuses on the non-primed group, n = 779 (out of 1,519) in the United States, and n = 605 in Europe.¹⁸

Figure 1.

Example of experimental variation.

After the six tweets, we elicited participants’ post-treatment attitudes using a different set of questions about COVID-19 policy responses. Participants stated their agreement on a 7-point Likert-type scale to a number of policies, such as “Prohibiting gatherings” and “Closing the borders.”¹⁹ We use the first principal component of these responses as an index measure of post-treatment policy attitudes, our main outcome variable. We again defined positive values as being more pro-economy.

After the post-treatment attitude questions, we conducted a factual manipulation check by asking participants,

Views about COVID-19 policy response can be roughly split into two: (1) Pro-health: prioritise the elimination of COVID-19 over economic activities, for example by extending lockdown measures despite economic costs. (2) Pro-economy: prioritise economic activities over the elimination of COVID-19, for example by opening up the economy despite risks of a second wave. Which of these two views had more likes in the 6 tweets shown earlier?

Participants selected from “pro-health,” “pro-economy,” “neither (both had about the same number of likes),” or “don’t know.” Participants could not go back to the previous screen to check the number of “likes” on the tweets.²⁰

Finally, we collected data on education, income, self-reported political ideology on a 0 to 10 left-right scale, party voted in the last election (or if they voted), experience of COVID-19, degree of stubbornness measured by the participants’ resistance to change (Oreg, 2003), media consumption, trust in the media and the government, and the frequency with which they discuss policy issues with family and friends (both on and outside of social media). We measured participants’ social media use by asking about time spent per day on the social media platforms Facebook and Twitter and define active social media users as those who spend more than 1 hr daily on Facebook or Twitter combined.²¹

Empirical Analysis and Results

Main Analysis

Key summary statistics are shown in Appendix Table A1, for the whole sample and split by social media use. Notably, active users are younger, hold a more right-wing ideology, and tend to support more pro-economy policies pre-treatment. They were also more likely to correctly answer the factual manipulation check at the end of the survey. The proportion of active users is highest in Ireland (30%) and lowest in the US Midwest (17%); in all regions, Facebook use is more common than Twitter.

We estimate the effect of social media endorsements on participants’ policy attitudes using ordinary least squares (OLS) as follows

\begin{array}{l} P o s t A t t i t u d e s_{i} = β_{0} + β_{1} T r e a t m e n t_{i} \\ + P r e A t t i t u d e {s^{'}}_{i} λ + {X^{'}}_{i} δ + ε_{i} \end{array}

(1)

The dependent variable $P o s t A t t i t u d e s_{i}$ is the standardized first principal component of the responses to the post-treatment policy questions, with a higher value representing a more pro-economy attitude. $T r e a t m e n t_{i}$ represents the assigned treatment and equals 1 for the pro-economy treatment, –1 for the pro-health treatment, and 0 otherwise. The coefficient of interest is $β_{1}$ , the effect of perceived endorsements on policy attitudes, which is expected to be positive. Hence, participants exposed to tweets where pro-economy views appear more popular are expected to show an increase in $P o s t A t t i t u d e s_{i}$ , while participants exposed to tweets showing popular pro-health views are expected to show a decrease in $P o s t A t t i t u d e s_{i}$ . $P r e A t t i t u d e s_{i}$ is taken from the responses to the pre-treatment policy questions. ${X^{'}}_{i}$ is a vector of control variables including gender (coded as a dummy for male), age, region (census regions for the United States, country for the European Union [EU] sample), household income (coded as the log of the midpoint of the interval specified by the subject), education (coded as a dummy for whether the subject has at least a 2 year college degree), and political ideology (self-reported response on a 0–10 left-right scale).²² We include country fixed effects for all of our pooled analyses below, and we use robust standard errors in all specifications.²³

We measure pre-treatment attitudes in two ways. First, we use the principal component of all the pre-treatment policy attitude questions (as pre-registered). Second, we use the question that has the highest correlation with the post-treatment attitude index to represent participants’ pre-treatment attitudes. In the US sample, the question used is “The government’s highest priority should be saving as many lives as possible even if it means the economy will recover more slowly. What do you think of this statement?.” In the European sample, the question used is “Sweden’s government has so far avoided implementing a lockdown in order to keep the economy going. What do you think of this policy?.” Although this latter approach differs from the pre-registered specification, we find that the correlation between pre-treatment and post-treatment attitudes is substantially higher when using this measure, potentially better capturing the policy dimension of interest and thus maximizing the statistical gains from our quasi-pre-post design (Clifford et al., 2021).²⁴ We present results using both measures.

We estimate Model 1 for the whole sample and for active social media users below. The results are shown in Figure 2. Estimates including a fully interacted model that tests for differences between the groups can be found in Appendix Table A2. We observe no overall treatment effect on participants’ policy attitudes. Our estimates suggest a precisely estimated zero effect of endorsements on policy attitudes for our entire sample (left panel). However, we find a differential treatment effect for active social media users (right panel) that suggests these users shift their policy attitudes by about 0.12 standard deviations in response to our treatment. That is, we find evidence consistent with our main hypothesis only for active social media users: endorsement metrics have a significant effect in persuading active users to shift their views in the intended direction.

Figure 2.

Treatment effects.

A factual manipulation check allows us to identify individuals who paid conscious attention to the endorsement counts and to study the extent to which the treatment effects are driven by them (Kane & Barabas, 2019). We asked participants—after they had submitted their policy preferences—about the relative levels of “likes” in the tweets they had seen. Overall, only 33.8% of participants answered the manipulation check correctly, with the rest answering incorrectly or “don’t know.”²⁵ The proportion of correct responders is higher in the group of active social media users than passive/non-users (38.7% vs. 32.1%, t test, p = .0225). Importantly, this post-treatment attention check is endogenous to the extent to which attention (or correct reporting) of the endorsement metrics may be selective (Montgomery et al., 2018).²⁶

To further explore these findings, we split our sample by both social media use and whether they answered this manipulation check correctly. The results are shown in Figure 3. Estimates are also presented in Appendix Table A6, and a fully interacted model that tests for differences between the groups can be found in Appendix Table A7. We observe that the treatment effect is concentrated on social media users who correctly answered the manipulation check. The coefficients are robust to the addition of controls, suggesting that selective attention is unlikely to explain these findings (Oster, 2019). Instead, the results suggest that a relatively small fraction of participants (about 10%) were sensitive to the social cues provided by the engagement metrics in our experiment; the treatment shifted their policy views by about 0.38 standard deviations (p value < .001).

Figure 3.

Treatment effects by social media use and manipulation check.

We also observe patterns suggestive of heterogeneous treatment effects for passive/non–social media users; but these are not robust to the addition of controls, revealing instead that these users may pay selective attention to social cues which match their policy attitudes. In particular, passive/non–social media users who correctly answered the manipulation check were more likely to hold views which aligned with the assigned treatment, while those who did not correctly answer the question were more likely to hold views which differed from their assigned treatment.

As further evidence of potential selection in these subsamples, regressing the treatment assignment on pre-treatment attitudes highlights that correctly answering the manipulation check is potentially endogenous (in Appendix Table A8). The patterns appear particularly stark for passive/non–social media users and suggest that they are more prone to selective attention. To evaluate the extent to which the heterogeneity in Figure 3 may be driven by selective attention, as a further robustness check, we test the sensitivity of our estimates to the addition of controls in a selection on unobservables framework (following Oster, 2019). Our results (shown in Figure 4) corroborate our reading of the results presented here, revealing that the estimates for passive/non–social media users are sensitive to unobservable selection, while those for active social media users are not.²⁷

Figure 4.

Selection-bias-adjusted treatment effects for participants with correct manipulation check.

The analyses presented here suggest that the (relatively small) subset of active social media users who tend to pay conscious attention to endorsement metrics are indeed influenced by these social cues. On the contrary, passive/non–social media users are more likely to notice endorsement metrics which reinforce their pre-existing attitudes, but they are on average not influenced by these metrics.

As two additional tests, in Appendix Table A9, we split our sample by countries and show that, though somewhat noisier, the patterns we documented are largely consistent across countries, with the largest effects concentrated on social media users who correctly answered the manipulation check (row 3, columns 4–6). We also find that our results are stronger when excluding the top and bottom 5% respondents in terms of study duration (i.e., those that were “rushing through” or taking too long to respond, see Appendix Figure A5 and Appendix Table A18). Finally, the results are shown separately for each post-treatment question in Appendix Figure A3, revealing a general shift in attitudes, and that our results are not driven by any particular question.

Additional Analyses

Anchoring

In addition to our main hypothesis on endorsement-driven social conformity, we pre-registered two additional hypotheses. First, the nature of the experiment allows us to evaluate the presence of anchoring, the idea that individuals may be disproportionately affected by the views that they first see (Furnham & Boo, 2011; Tversky & Kahneman, 1974). We hypothesize that this heuristic extends to social media settings in which individuals are exposed to different views.

H2, Anchoring: Individuals are anchored (or primed) by what they are first exposed to, so they tend to conform to the first views they observe.

Second, we explore the complementarity between our two hypotheses. In particular, we hypothesize that there are positive complementarities of the two treatments, such that higher endorsement metrics have a differential effect on attitudes when users are exposed to these views first.

H3, Complementarities: Both anchoring and popularity affect individuals’ policy views, and there are positive complementarities between the two: individuals first exposed to popular messages conform most to these views.

The order in which tweets were shown to participants was randomized, which allows us to evaluate our anchoring and complementarities hypotheses. In particular, this randomization allows us to evaluate whether the first post observed affects policy attitudes, by estimating the following model

\begin{array}{l} P o s t A t t i t u d e s_{i} = β_{0} + β_{1} F i r s t M e s s a g e E c o n_{i} \\ + P r e A t t i t u d e {s^{'}}_{i} λ + {X^{'}}_{i} δ + ε_{i} \end{array}

(2)

The indicator variable $F i r s t M e s s a g e E c o n_{i}$ equals 1 if participants were first exposed to a pro-economy message, and 0 otherwise. Our parameter of interest, $β_{1}$ , therefore captures the post-treatment attitudes of participants who were first exposed to a pro-economy message, relative to participants who were first exposed to a pro-health message. In other words, both groups are “treated,” and we estimate the differential treatment effect.

In addition, we investigate whether this anchoring effect of the first tweet, and our main treatment, are “complementary.” In particular, we estimate this model

\begin{array}{l} \begin{array}{l} P o s t A t t i t u d e s_{i} = β_{0} + β_{1} F i r s t M e s s a g e E c o n_{i} \\ + β_{2} F i r s t M e s s a g e H i g h_{i} \end{array} \\ \begin{array}{l} + β_{3} F i r s t M e s s a g e E c o n_{i} \\ \times F i r s t M e s s a g e H i g h_{i} \end{array} \\ + P r e A t t i t u d e {s^{'}}_{i} λ + {X^{'}}_{i} δ + ε_{i} \end{array}

(3)

The indicator variable $F i r s t M e s s a g e H i g h_{i}$ equals 1 if the first tweet was in the “high-popularity” category. Our parameter of interest, $β_{3}$ , captures the differential effect of anchoring when the first tweet also had these high social endorsement metrics.

Our results from these analyses are presented in Figure 5. We find weak and marginally significant anchoring effects that appear to be concentrated on passive/non–social media users. These findings on anchoring have therefore unclear policy implications. In particular, potential policy interventions from social media platforms hoping to exploit these results (e.g., by manipulating the top message shown in a news feed) may prove ineffective, since the effect appears to be only present in passive/non–social media users.

Figure 5.

Anchoring effects and complementarities.

In addition, we find no overall complementarity between the content of the first message (pro-econ vs. pro-health) and the engagement metrics, but (conditional on controls) we do find a complementarity for active social media users. This pattern suggests that, for this subset of participants, the content of the first tweet did matter when it had the high endorsement metrics. Although these results are more imprecisely estimated, they appear to confirm our previous findings and suggest once more that active social media users can be influenced by engagement metrics.

Political Polarization

Social media has commonly been associated with an increase in political polarization (Zhuravskaya et al., 2020). In concordance with these worries, active social media users in our survey were less likely to consider themselves politically moderate (Figure 6) and were less likely to hold (pre-treatment) moderate policy views with respect to COVID-19 (Appendix Figure A4). However, these patterns could well be the result of selection into social media use: individuals who hold more polar views tend to be more active on social media, perhaps as an outlet for their extreme opinions. Although recent work documents that deactivating Facebook can indeed reduce individuals’ political polarization (Allcott, Braghieri, et al., 2020), the extent to—and the precise mechanisms through—which social media causes polarization remains debated.

Figure 6.

Self-reported political ideology and social media use.

We explored whether our treatment varied with participants’ pre-treatment policy attitudes: Is the effect of endorsements larger for congenial views? This margin of heterogeneity was pre-registered among others, but we do not have enough statistical power to perform multiple hypothesis corrections. For this reason, the results from this section should be viewed as merely exploratory.

We investigate the heterogeneity of our main treatment by pre-treatment attitudes with the following empirical model

\begin{array}{l} P o s t A t t i t u d e s_{i} = β_{0} + β_{1} T r e a t m e n t_{i} + β_{2} P r e A t t i t u d e s_{i} \\ + β_{3} T r e a t m e n t_{i} \times P r e A t t i t u d e s_{i} + {X^{'}}_{i} δ + ε_{i} \end{array}

(4)

and do so specifically for active social media users.

The estimated marginal treatment effects from estimating Model 4 are shown in Figure 7. We find that our treatment differentially affects social media users depending on their pre-existing attitudes. A one standard deviation increase in pre-treatment attitudes leads to an additional change of between 0.11 and 0.16 standard deviations in policy attitudes (depending on how we measure pre-treatment attitudes). Put differently, individuals who held more extreme pre-treatment attitudes were more responsive to the treatment, and the treatment reinforced their pre-treatment attitudes. This pattern is particularly driven by individuals who held relatively more pro-economy views before the treatment. We also present models in which we include triple-interactions of treatment, active social media use, and pre-treatment attitudes (Appendix Table A12). The results suggest that public endorsement metrics may be a mechanism through which social media affects individuals’ policy preferences and could also contribute to polarization, and especially since individuals may be more likely to endorse congenial views (Garz et al., 2020).

Figure 7.

Heterogeneity by pre-treatment attitudes.

Discussion

We hypothesized that online endorsements could affect the formation of policy preferences. Our experiment, in contrast, revealed a precisely estimated null effect of perceived endorsements on our representative samples in the United States and Europe. Furthermore, we found that only about one-third of the experiment participants appeared to pay conscious attention to these engagement metrics. At the same time, we found suggestive evidence of large treatment effects concentrated on a small share of individuals: active social media users who did appear to pay attention to the endorsement metrics (about 10% of all participants). For these individuals, a higher sensitivity to popularity metrics means that the perceived popularity of (even) strangers’ opinion is enough to sway their attitude. Finally, given our conservative measure of “high” metrics, our results potentially underestimate the impact of endorsement for viral tweets or tweets by famous people, which may attract thousands or even more likes and retweets—this would be an interesting avenue for future research.

That the effects appeared concentrated on only a fraction of participants perhaps suggests that the broader effects of endorsement metrics on politics may be limited. However, social media dynamics could further propagate across society in different ways (Margetts et al., 2015; Tufekci, 2017). Social media engagement is also associated with other forms of political engagement; as such, these individuals could exert disproportionate influence in political processes (Barberá et al., 2019; Vaccari et al., 2015; Vaccari & Valeriani, 2021) and have a broad impact on public opinion (Centola et al., 2018). In our survey, active social media users are significantly more likely to (say they) have voted in the previous election. They also report more frequently discussing policy issues with friends or family members both on and outside of social media (Appendix Table A16).²⁸

Our micro-level study identifies endorsement metrics as one channel through which social media affects users’ policy attitudes. However, there is a trade-off between isolating a precise causal mechanism in a controlled setting versus external validity, and future work should aim to study this relationship in real social media settings. Improved understanding of these effects can inform social media platforms in the design of appropriate interventions to address issues of polarization, misinformation, and foreign influence in politics, among others (since platforms are unlikely to promote account deactivation, as in Allcott, Braghieri, et al., 2020). One further implication is that these social cues may reinforce the effects of selective exposure (as emphasized in Zhuravskaya et al., 2020). If individuals with more extreme preferences are also more likely to “like” content, then not only will social media algorithms expose users to more polarized opinions (Levy, 2021), but such content may also appear to have broader support. In particular, our results may underestimate the true effect of endorsement metrics on social media platforms where individuals are exposed to posts and endorsements by people they know and whom they choose to follow—whose opinions the individual likely agrees with—which, as our exploratory analysis suggests, may further contribute to political polarization. How these different features of social media interact to influence political views and the persistence of the effects remain an important avenue for future work. Finally, substantial uncertainty surrounding the topic of COVID-19 in the early stages of the pandemic likely made policy attitudes in this respect highly malleable. Future work should also examine whether endorsements can shape policy attitudes of social media users in deeper entrenched topics for which views are likely to be more rigid.

Footnotes

Appendix

Acknowledgements

We thank Anna Dreber Almenberg, Karen Arulsamy, Stefan Müller, and Johannes Wohlfart as well as seminar participants at UCD and Collegio Carlo Alberto for comments. The experiments were approved by University College Dublin Office of Research Ethics (reference numbers: HS-E-20-110-Samahita and HS-E-20-134-Samahita). Authors’ order has been randomized using the AEA Author Randomization Tool (reference: eTjbUATa_zKY), denoted by ⓡ. All errors are our own.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding from UCD, Collegio Carlo Alberto, and the Einaudi Institute for Economics and Finance is gratefully acknowledged.

ORCID iD

Margaret Samahita

Supplemental material

Supplemental material for this article is available at .

Notes

Author Biographies

Pierluigi Conzo (PhD, University of Rome “Tor Vergata”) is an Associate Professor of Economics at the University of Turin and a Research Fellow of the Collegio Carlo Alberto. His recent research interests include the analysis of the determinants (and the outcomes) of prosocial preferences as well as the socio-political effects of anti-immigration campaigns.

Laura K. Taylor (PhD/MA, University of Notre Dame) is an Associate Professor of Psychology at University College Dublin. Her research interests include constructive intergroup relations, particularly in conflict-affected settings.

Juan S. Morales (PhD, University of Toronto) is an Assistant Professor of Economics at Wilfrid Laurier University. His research interests include political economy, development economics, applied microeconomics, conflict, and media.

Margaret Samahita (PhD, Lund University) is an Assistant Professor of Economics at University College Dublin. Her research interests include behavioral economics, experimental economics, and political economy.

Andrea Gallice (PhD, European University Institute) is Associate Professor of Economics at the University of Turin and a Research Fellow of the Collegio Carlo Alberto. His research interests include political economy and the analysis of social status concerns and social pressures.

References

Alatas

Chandrasekhar

A. G.

Mobius

Olken

B. A.

Paladines

(2019). When celebrities speak: A nationwide Twitter experiment promoting vaccination in Indonesia [NBER working paper no. 25589]. https://www.nber.org/system/files/working_papers/w25589/w25589.pdf

Allcott

Boxell

Conway

Gentzkow

Thaler

Yang

D. Y.

(2020). Polarization and public health: Partisan differences in social distancing during the Coronavirus pandemic. Journal of Public Economics, 191, 104254.

Allcott

Braghieri

Eichmeyer

Gentzkow

(2020). The welfare effects of social media. American Economic Review, 110(3), 629–676.

Allcott

Gentzkow

(2017). Social media and fake news in the 2016 election. Journal of Economic Perspectives, 31(2), 211–236.

Anspach

N. M.

(2017). The new personal inﬂuence: How our Facebook friends inﬂuence the news we read. Political Communication, 34(4), 590–606.

Bakshy

Messing

Adamic

L. A.

(2015). Exposure to ideologically diverse news and opinion on Facebook. Science, 348(6239), 1130–1132.

Banerjee

Alsan

Breza

Chandrasekhar

A. G.

Chowdhury

Duﬂo

Goldsmith-Pinkham

Olken

B. A.

(2020). Messages on COVID-19 prevention in India increased symptoms reporting and adherence to preventive behaviors among 25 million recipients with similar effects on non-recipient members of their communities [NBER working paper no. 27496]. https://web.stanford.edu/~arungc/BABCCDGO.pdf

Banerjee

A. V.

(1992). A simple model of herd behavior. The Quarterly Journal of Economics, 107(3), 797–817.

Barberá

(2015). Birds of the same feather tweet together: Bayesian ideal point estimation using Twitter data. Political Analysis, 23(1), 76–91.

10.

Barberá

Casas

Nagler

Egan

P. J.

Bonneau

Jost

J. T.

Tucker

J. A.

(2019). Who leads? Who follows? Measuring issue attention and agenda setting by legislators and the mass public using social media data. American Political Science Review, 113(4), 883–901.

11.

Bikhchandani

Hirshleifer

Welch

(1992). A theory of fads, fashion, custom, and cultural change as informational cascades. Journal of Political Economy, 100(5), 992–1026.

12.

Bol

Giani

Blais

Loewen

P. J.

(2021). The effect of COVID-19 lockdowns on political support: Some good news for democracy? European Journal of Political Research, 60(2), 497–505.

13.

Bond

R. M.

Fariss

C. J.

Jones

J. J.

Kramer

A. D.

Marlow

Settle

J. E.

Fowler

J. H.

(2012). A 61-million-person experiment in social inﬂuence and political mobilization. Nature, 489(7415), 295–298.

14.

Bond

R. M.

Settle

J. E.

Fariss

C. J.

Jones

J. J.

Fowler

J. H.

(2017). Social endorsement cues and political participation. Political Communication, 34(2), 261–281.

15.

Boxell

Gentzkow

Shapiro

J. M.

(2017). Greater internet use is not associated with faster growth in political polarization among US demographic groups. Proceedings of the National Academy of Sciences, 114(40), 10612–10617.

16.

Bursztyn

Egorov

Fiorin

(2020). From extreme to mainstream: The erosion of social norms. American Economic Review, 110(11), 3522–3548.

17.

Bursztyn

Jensen

(2017). Social image and economic behavior in the ﬁeld: Identifying, understanding, and shaping social pressure. Annual Review of Economics, 9, 131–153.

18.

Carlson

T. N.

Settle

J. E.

(2016). Political chameleons: An exploration of conformity in political discussions. Political Behavior, 38(4), 817–859.

19.

Casoria

Galeotti

Villeval

M. C.

(2021). Perceived social norm and behavior quickly adjusted to legal changes during the COVID-19 pandemic. Journal of Economic Behavior & Organization, 190, 54–65.

20.

Centola

Becker

Brackbill

Baronchelli

(2018). Experimental evidence for tipping points in social convention. Science, 360(6393), 1116–1119.

21.

Cialdini

R. B.

Goldstein

N. J.

(2004). Social inﬂuence: Compliance and conformity. Annual Review of Psychology, 55, 591–621.

22.

Clifford

Sheagley

Piston

(2021). Increasing precision without altering treatment effects: Repeated measures designs in survey experiments. American Political Science Review, 115(3), 1048–1065.

23.

Eady

Nagler

Guess

Zilinsky

Tucker

J. A.

(2019). How many people live in political bubbles on social media? Evidence from linked survey and Twitter data. SAGE Open, 9(1), 2158244019832705.

24.

Egebark

Ekström

(2018). Liking what others “Like”: Using Facebook to identify determinants of conformity. Experimental Economics, 21(4), 793–814.

25.

Enikolopov

Makarin

Petrova

(2020). Social media and protest participation: Evidence from Russia. Econometrica, 88(4), 1479–1514.

26.

Fergusson

Molina

(2019). Facebook causes protests [Documento CEDE no. 41]. https://thedocs.worldbank.org/en/doc/f73fcca90d36718b63430887535b1eb6-0050022021/original/Facebook-Causes-Protests.pdf

27.

Fujiwara

Müller

Schwarz

(2021). The effect of social media on elections: Evidence from the United States [NBER working paper no. 28849]. https://www.princeton.edu/~fujiwara/papers/SocialMediaAndElections.pdf

28.

Furnham

Boo

H. C.

(2011). A literature review of the anchoring effect. The Journal of Socio-Economics, 40(1), 35–42.

29.

Garz

Sörensen

Stone

D. F.

(2020). Partisan selective engagement: Evidence from Facebook. Journal of Economic Behavior & Organization, 177, 91–108.

30.

Gentzkow

Shapiro

J. M.

(2011). Ideological segregation online and ofﬂine. The Quarterly Journal of Economics, 126(4), 1799–1839.

31.

Gorodnichenko

Pham

Talavera

(2021). Social media, sentiment and public opinions: Evidence from #Brexit and #USElection. European Economic Review, 136, 103772.

32.

Guess

A. M.

(2021). (Almost) everything in moderation: New evidence on Americans’ online media diets. American Journal of Political Science, 65(4), 1007–1022.

33.

Guess

A. M.

Nyhan

Reiﬂer

(2020). Exposure to untrustworthy websites in the 2016 US election. Nature Human Behavior, 4(5), 472–480.

34.

Halberstam

Knight

(2016). Homophily, group size, and the diffusion of political information in social networks: Evidence from Twitter. Journal of Public Economics, 143, 73–88.

35.

Hensel

Witte

Caria

A. S.

Fetzer

Fiorin

Götz

F. M.

Gomez

Haushofer

Ivchenko

Kraft-Todd

(2022). Global behaviors, perceptions, and the emergence of social norms at the onset of the COVID-19 pandemic. Journal of Economic Behavior & Organization, 193, 473–496.

36.

Hilligoss

Rieh

S. Y.

(2008). Developing a unifying framework of credibility assessment: Construct, heuristics, and interaction in context. Information Processing & Management, 44(4), 1467–1484.

37.

T. Q.

Nie

Alpizar

Carlsson

Nam

P. K.

(2022). Celebrity endorsement in promoting pro-environmental behavior. Journal of Economic Behavior & Organization, 198, 68–86.

38.

Iyengar

Hahn

K. S.

Krosnick

J. A.

Walker

(2008). Selective exposure to campaign communication: The role of anticipated agreement and issue public membership. The Journal of Politics, 70(1), 186–200.

39.

Kahneman

(2003). A perspective on judgment and choice: Mapping bounded rationality. American Psychologist, 58(9), 697–720.

40.

Kane

J. V.

Barabas

(2019). No harm in checking: Using factual manipulation checks to assess attentiveness in experiments. American Journal of Political Science, 63(1), 234–249.

41.

Krupnikov

Nam

H. H.

Style

(2021). Convenience samples in political science experiments. In Druckman

J. N.

Green

D. P.

(Eds.), Advances in experimental political science (pp. 165–183). Cambridge University Press.

42.

Levy

(2021). Social media, news consumption, and polarization: Evidence from a ﬁeld experiment. American Economic Review, 111(3), 831–870.

43.

Luo

Hancock

J. T.

Markowitz

D. M.

(2022). Credibility perceptions and detection accuracy of fake news headlines on social media: Effects of truth-bias and endorsement cues. Communication Research, 49(2), 171–195.

44.

Margetts

John

Hale

Yasseri

(2015). Political turbulence: How social media shape collective action. Princeton University Press.

45.

Mathieu

Ritchie

Rodés-Guirao

Appel

Giattino

Hasell

Macdonald

Dattani

Beltekian

Ortiz-Ospina

Roser

(2020). Coronavirus pandemic (COVID-19). Our World in Data. https://ourworldindata.org/coronavirus

46.

Messing

Westwood

S. J.

(2014). Selective exposure in the age of social media: Endorsements trump partisan source afﬁliation when selecting news online. Communication Research, 41(8), 1042–1063.

47.

Metzger

M. J.

Flanagin

A. J.

Medders

R. B.

(2010). Social and heuristic approaches to credibility evaluation online. Journal of Communication, 60(3), 413–439.

48.

Montgomery

J. M.

Nyhan

Torres

(2018). How conditioning on posttreatment variables can ruin your experiment and what to do about it. American Journal of Political Science, 62(3), 760–775.

49.

Morales

J. S.

(2020). Perceived popularity and online political dissent: Evidence from Twitter in Venezuela. The International Journal of Press/Politics, 25(1), 5–27.

50.

Munger

Egan

P. J.

Nagler

Ronen

Tucker

(2022). Political knowledge and misinformation in the era of social media: Evidence from the 2015 UK election. British Journal of Political Science, 52(1), 107–127.

51.

Oreg

(2003). Resistance to change: Developing an individual differences measure. Journal of Applied Psychology, 88(4), 680.

52.

Oster

(2019). Unobservable selection and coefﬁcient stability: Theory and evidence. Journal of Business & Economic Statistics, 37(2), 187–204.

53.

Zheng

Chen

G. M.

(2018). Twitter versus Facebook: Comparing incivility, impoliteness, and deliberative attributes. New Media & Society, 20(9), 3400–3419.

54.

Ross

Greene

House

(1977). The “false consensus effect”: An egocentric bias in social perception and attribution processes. Journal of Experimental Social Psychology, 13(3), 279–301.

55.

Scharkow

Mangold

Stier

Breuer

(2020). How social network sites and other online intermediaries increase exposure to news. Proceedings of the National Academy of Sciences, 117(6), 2761–2763.

56.

Settele

Shupe

(2022). Lives or livelihoods? Perceived trade-offs and policy views. The Economic Journal, 132(643), 1150–1178.

57.

Settle

J. E.

(2018). Frenemies: How social media polarizes America. Cambridge University Press.

58.

Shin

Wang

Y.-T.

(2022). Twitter and endorsed (fake) news: The inﬂuence of endorsement by strong ties, celebrities, and a user majority on credibility of fake news during the COVID-19 pandemic. International Journal of Communication, 16, 2573–2595.

59.

Simon

H. A.

(1955). A behavioral model of rational choice. The Quarterly Journal of Economics, 69(1), 99–118.

60.

Snowberg

Yariv

(2021). Testing the waters: Behavior across participant pools. American Economic Review, 111(2), 687–719.

61.

Sundar

S. S.

(2008). The MAIN model: A heuristic approach to understanding technology effects on credibility. In Metzger

Flanagin

(Eds.), Digital media, youth, and credibility (pp. 73–100). MIT Press.

62.

Sunstein

C. R.

(2018). #Republic: Divided democracy in the age of social media. Princeton University Press.

63.

Tufekci

(2017). Twitter and tear gas: The power and fragility of networked protest. Yale University Press.

64.

Tversky

Kahneman

(1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.

65.

Vaccari

Valeriani

(2021). Outside the bubble: Social media and political participation in Western democracies. Oxford University Press.

66.

Vaccari

Valeriani

Barberá

Bonneau

Jost

J. T.

Nagler

Tucker

J. A.

(2015). Political expression and action on social media: Exploring the relationship between lower- and higher-threshold political activities among Twitter users in Italy. Journal of Computer-Mediated Communication, 20(2), 221–239.

67.

Wang

Morey

A. C.

Srivastava

(2014). Motivated selective attention during political ad processing: The dynamic interplay between emotional ad content and candidate evaluation. Communication Research, 41(1), 119–156.

68.

Zhuravskaya

Petrova

Enikolopov

(2020). Political effects of the internet and social media. Annual Review of Economics, 12, 415–438.

Can ❤s Change Minds? Social Media Endorsements and Policy Preferences

Abstract

Keywords

Introduction

Background

Experimental Design

Implementation and Design

Empirical Analysis and Results

Main Analysis

Additional Analyses

Anchoring

Political Polarization

Discussion

Footnotes

Appendix

Acknowledgements

Declaration of conflicting interests

Funding

ORCID iD

Supplemental material

Notes

Author Biographies

References