The bias of Twitter as an agenda-setter on COVID-19: An empirical research using log data and survey data in Japan

Abstract

This study investigates the bias of Twitter as an agenda-setter during COVID-19. Specifically, we analyze the agenda-setting function of Twitter (Study 1) and characteristics of information disseminators on Twitter, agenda-builders (Study 2), related to the COVID-19 pandemic. In Study 1, we examined rank correlations between the media agendas on COVID-19 and public agendas. The results indicated that Twitter agendas resonate with those who have liberal tendencies. In Study 2, we used data from the Internet survey to identify the political attitudes of agenda-builders who tweet or retweet on COVID-19. The results of the model analyses indicated that people with liberal tendencies, motivated by their political attitude, created original tweets, and some of those tweets were then retweeted by flaming-oriented people driven by a sense of justice. This seems to be how information about COVID-19 spreads on Twitter in Japan.

Keywords

Agenda-builders agenda-setting COVID-19 Japan public opinion Twitter

Introduction

The aim of this article is to identify the biases of Twitter as an agenda-setting medium. Over a year since the outbreak of COVID-19 hit Japan, varying information has been spread through social media, as well as mass media such as newspapers and television. According to a survey conducted by Japan’s Ministry of Internal Affairs and Communications (n = 2000), 22.6% of respondents said that Twitter was the information medium through which they had seen the news about COVID-19, which is the highest percentage among social media outlets surveyed (Ministry of Internal Affairs and Communications, 2020). In addition, given that information on Twitter has sometimes been discussed in the Diet, it can influence public policy. Thus, the presence of Twitter as a news medium on COVID-19 is not small. The biases found on Twitter are important factors for the decision-making of people and governments. This article provides insights into Twitter’s biases through analyses of its agenda-setting function, which lead to a deep understanding of the publicity of Twitter.

Although the method employed in agenda-setting research has revealed the relationship between media and people, it has only revealed the relationship at an aggregate level. Therefore, to examine the bias of Twitter, clarifying who the agenda-builders are is imperative. While information is edited by professional editors in traditional mass media, information is transmitted and spread by ordinary users on Twitter. In other words, netizens become the agenda-builders. Therefore, we conduct a survey on netizens.

This article examines the relationship between the Twitter agenda and the public agenda in Study 1, and clarifies the characteristics of those who spread information about COVID-19, the agenda-builders, in Study 2.

The significance of this study is as follows: agenda-setting studies focusing on social media are still few and they only focus on intermedia. Thus, there is a need to explore the relationship between the actual public agenda and that of social media. In addition, the originality of this research lies in that it combines log data and survey data to reveal both the agenda-setting function and the agenda-builders, simultaneously. By these methods, this research contributes to the development of social media agenda-setting theory.

Literature review

This study provides evidence on Twitter’s agenda-setting function from two aspects. First, the relationships between Twitter agendas and the public agendas. Second, the characteristics of Twitter users as the agenda-builders. In the following, we derive theoretical frameworks by reviewing agenda-setting research and research on information dissemination on social media.

Agenda-setting research has accumulated more than 400 empirical studies, demonstrating that the agendas reported in mass media can influence the public agendas (McCombs, 2014). Agenda-setting research has developed by examining the rank correlation between agendas in mass media, such as newspapers and television, and public agendas. In recent years, empirical studies in Asia have also been published. Wu and Guo (2020) examined media communication during the Taiwanese presidential election campaign and explored agenda-setting between newspaper and television agendas and public agendas. They revealed the existence of the third-level agenda-setting and an opportunity for selective exposure. Cheng and Chan (2015) studied the relationship between newspaper agendas and public agendas, citing social movements in Hong Kong, and found the third-level agenda-setting function. Cheng (2016) further confirmed its robustness in the case of political scandals. Although agenda-setting studies have been published in China and Taiwan in recent years, no new studies have been published in Japan for at least a decade.

In the Western area, social media has come into the scope of agenda-setting research. Researchers have examined the relationship between information from mass media accounts on Twitter and tweets of the general public (McGregor & Vargo, 2017; Vargo et al., 2014; Vargo & Guo, 2017) and the intermedia agenda-setting function between newspapers and Twitter (Ceron, 2014). In the background of such research, there is a view that sees Twitter as public opinion (Vargo, 2011). However, there are limitations in viewing Twitter as public opinion. It has been pointed out that content data on Twitter cannot be used as an accurate representative sample (Salganik, 2017). Some empirical studies in Japan have also demonstrated that online political discourses do not always reflect public opinions (Tanaka & Hamaya, 2019; Toriumi, 2020). Therefore, to validate Twitter’s agenda-setting power, we need to conduct a survey and use data from real people. Thus, in Study 1, we consider Twitter as a kind of medium and examine the relationship between agendas on Twitter and actual public agendas from survey data.

Previous research has not studied the characteristics of agenda-builders. However, to reveal the bias of Twitter as an agenda-setter, it is necessary to reveal the tendency of agenda-builders as well. What are the characteristics of information diffusion on the social media? There is a certain amount of research on information dissemination in social media. For example, the sharing of anger plays an important role in the formation of networks on Weibo (Fan et al., 2014). In addition, people with feelings of anger tend to be more likely to participate in political discussions on social media (Wollebæk et al., 2019). Based on these studies, it is expected that agenda-builders are spreading information in the Twitter-sphere based on negative motives. Therefore, the term bias in this article refers to the possibility that public opinion in the Twitter-sphere deviates from actual public opinion, motivated by negative emotions.

Based on the above, this article sets up the theoretical frameworks that information on Twitter is more negatively biased than actual public opinion. From this perspective, this article develops the following two studies to demonstrate the bias of Twitter’s agenda-setting function. Study 1 identifies how the agenda-setting function of Twitter works, and Study 2 clarifies the role of Twitter as an agenda-setter by identifying the characteristics of information spreaders as agenda-builders. The research questions (RQs) are as follows. We can identify biases in the formation of public space by Twitter by answering the following RQs:

RQ1. Is there a political bias in Twitter’s agenda-setting function?

RQ2. What are the biases of those who have tweeted or retweeted on COVID-19?

Study 1

Agenda-setting functions of media

Research on the agenda-setting function of media has evolved from a first-level function that focuses on the relative importance among major issues (e.g., diplomacy, economy, and other large topics) to a second-level function that focuses on the attributes of specific issues and to a third-level function that focuses on the people’s cognition of network among issues. In 2020, COVID-19 has been particularly important as a major issue, so this study focuses on the second-level agenda-setting function, attributes of COVID-19. We employed the second-level method rather than the third-level method for two reasons. First, various issues have been presented in relation to COVID-19. The major focus in Japan has been on what should be prioritized, specifically, promoting the economy or controlling the infection. Therefore, the difference in relative importance between these issues is pertinent when measuring media effects. Second, Twitter is a short microblog with a 140-character limit. Usually, only one issue can be conveyed in a single tweet. Therefore, the third-level method, which forces one to exclude tweets that only mention one issue, will result in considerable missing data.

There are debates regarding the scope of the second-level agenda-setting function (Takeshita, 2006). One of them limits the agenda-setting function to the analysis of relative importance among sub-issues of a particular issue (Takeshita & Mikami, 1995), and another extends the agenda-setting function to the framing function of “how to think” about a particular issue (McCombs et al., 2000). While assuming that there is a continuum between these two positions, we based our study on the former debate. As described later, we divided issues related to COVID-19 into 20 categories using content analysis on Twitter. Of these, the majority of the categories merely point out the situation, such as “The number of infected people” and “The impact on corporate profits (such as bankruptcy or closure of businesses).” Judging the importance of these agendas may be relative to other agendas.

However, some agendas inevitably involve framing. For example, “Support for the needy and increase in the number of suicides” includes the political attitude that people who have fallen into hardship due to COVID-19 should be supported. Above all, “The overall attitude and policy of the government in dealing with COVID-19 is not good” occupied the top spot on the Twitter agenda. Tweets that totally criticize the government and the ruling party, such as criticizing the inappropriate behaviors of government officials, are classified into this category. This category clearly contains political attitudes that are critical of the government and the ruling party. Judgments of the importance of these agendas are involved in framing, whether people agree or disagree with the issues. In our study, the analysis is limited to the relative importance of these categories. The effect of framing, which involves directing people’s attitudes about an issue, is stronger than that of agenda-setting function, people’s recognition of an issue. Therefore, a one-shot survey is limited in its measurement. If we want to strictly measure the framing effect, a randomized controlled experiment would be necessary. Of course, this is a matter of degree, but we limit our analysis to people’s agenda recognition based on data limitation. Specifically, we ranked the importance of each category from the content data of Twitter (Twitter agendas) and an Internet survey (public agendas), and analyzed correlations between Twitter agendas and public agendas.

Hypothesis

Research results have been unclear on whether the Twitter agenda is more associated with political liberals or conservatives. Vargo et al. (2014) and McGregor and Vargo (2017) comprehensively collected Twitter log data mentioning election-related topics during the 2012 US presidential election, and then classified official media accounts on Twitter by partisan attitudes, and examined their relationships with tweets by ordinary users. The dependent variable in Vargo et al. (2014) was the agenda network (third-level agenda-setting) in the tweets of ordinary users categorized as Obama supporters and Romney supporters. The result indicated that Romney supporters were more strongly associated with information originating from partisan media accounts. By contrast, the dependent variable in McGregor and Vargo (2017) was the major agenda (first-level agenda-setting) in tweets of ordinary users, categorized into three categories according to the number of tweets during the election period. The result indicated that the correlation between liberal media tweets and high-frequency users was the highest. Since the two studies differed not only in the dependent variable but also in the method of analysis, it is difficult to obtain a unified view. Therefore, the hypothesis is derived from theoretical considerations.

As mentioned above, in the social media sphere, negative emotions such as anger drive the dissemination of information. Also, the negative bias of Twitter has been pointed out in studies outside of the political context, such as Hennig-Thurau et al. (2015), and there is sufficient theoretical accumulation. Furthermore, considering this in the context of COVID-19, anger, or critical comments about various countermeasures taken by the Japanese government are likely to spread on social media. Currently, Japan’s ruling party is relatively conservative. Thus, to be anti-government implies that one is liberal.

H1. The Twitter agenda tends to resonate with liberal people.

Method

We conducted the analysis in Study 1 using Spearman’s rank correlation coefficient of the media agendas and public agendas, following McCombs and Shaw (1972).

For the analysis, it was necessary to identify the media agendas and the public agendas, and this task was conducted as follows. The data used in this study consisted of content data from Japanese Twitter and an Internet survey. First, we identified the agendas on Twitter. The content analysis period is from 16 September 2020, when the new government took office in Japan, to 12 December 2020, which was the time when the “third wave” of the COVID-19 pandemic hit Japan. During this period, netizens watched the government’s policies with a keen eye as the number of infected people increased.

We used the Twitter API to obtain all the tweets that included the word “corona (in Japanese)” during the periods. As we view Twitter as a medium, it is inappropriate to equate a tweet that has not been shared at all with a tweet that has been shared a lot. Twitter is designed to make frequently shared tweets more likely to be seen by people, so we can view the issues shared a lot as relatively emphasized Twitter agendas. Therefore, we conducted a content analysis on tweets retweeted more than 1000 times. In this way, we would also minimize the impact of bots.

We then categorized tweets using the method of the grounded theory approach and created the top 20 categories based on frequencies mentioned (Table 1). The method is as follows. Summaries of each data are transcribed onto an Excel sheet. After a certain amount of data are collected, they are grouped according to the content, and names are given to each group. Next, new data are transcribed, and those that fit into the existing groups are labeled as the existing group name. If not, the data are withheld from labeling. When a certain amount of data are collected, the grouping process is performed again, and new groups are created as necessary. These processes were repeated to check all the data. All the tweet data were coded according to these categories. Coding was conducted by the author and an assistant majoring in sociology. The Cohen’s Kappa coefficient for inter-coder reliability was .81. In this way, the top 20 groups were created. Of the tweets analyzed, those that did not fit into any of the 20 categories were excluded because their mention numbers were too small to be considered as agendas. We also excluded tweets that contain the word “corona” but are not related to COVID-19, such as “I want to do something when the corona is converged” and using the word “corona” in a facetious manner.

Table 1.

Twitter agendas.

		Twitter
		2 months		1 month		2 weeks		1 week
		Rate	Rank	Rate	Rank	Rate	Rank	Rate	Rank
1	Public policies to promote the economy (such as “GoTo Travel”, “GoTo Eat”)	7.9%	5	8.9%	4	10.4%	3	15.1%	3
2	Information on the number of infected people (including cluster outbreaks) in Japan and abroad	14.5%	2	13.8%	3	7.9%	4	8.9%	4
3	Vaccine development and securing and provision of vaccines by the government	1.4%	18	0.8%	18.5	1.2%	17.5	2.2%	10
4	Difficulties in the medical field and strengthening of medical resources	11.3%	3	14.1%	2	18.3%	2	16.2%	2
5	The impact on corporate profits (such as bankruptcy and closure of businesses)	2.2%	15	1.6%	15	2.1%	13	2.2%	10
6	Infection control measures by the government and companies (such as cluster measures, telework promotion and short-term business)	2.9%	12	3.0%	9.5	1.5%	16	1.7%	13
7	Deterioration of the employment situation (including a decrease in salaries)	1.8%	16	0.8%	18.5	0.6%	19	0.0%	19.5
8	Hosting of the Tokyo Olympics	1.5%	17	1.3%	16.5	1.2%	17.5	0.6%	18
9	Impacts on the macroeconomy (such as stock prices, GDP and consumption)	0.3%	20	0.3%	20	0.0%	20	0.0%	19.5
10	Transportation of people to and from overseas (such as business and tourism)	3.5%	10	4.6%	6	4.0%	8	3.4%	8
11	Infection prevention (such as virus infectivity, situations where people are susceptible to infection and effectiveness of masks)	8.0%	4	7.3%	5	4.9%	5	5.6%	5
12	Impacts on school and university education (including entrance examination and tuition fees)	1.2%	19	1.3%	16.5	1.8%	14.5	1.1%	16
13	Support for the needy and increase in the number of suicides	2.7%	13	2.5%	13	3.0%	11	1.7%	13
14	Cultural events (such as live music concerts)	4.1%	8	2.7%	11.5	1.8%	14.5	1.1%	16
15	Expand PCR test	2.4%	14	2.2%	14	2.4%	12	1.1%	16
16	Overall attitude and policy of the government in dealing with COVID-19 is not good	17.0%	1	21.0%	1	22.9%	1	26.8%	1
17	Celebrities infected (such as entertainers, athletes and politicians)	5.9%	6	4.4%	7	4.6%	6.5	2.2%	10
18	Campaigns of businesses (such as restaurants) that are doing their best even during the COVID-19 pandemic	4.0%	9	2.7%	11.5	3.4%	9.5	3.9%	7
19	Symptoms (including sequelae) of new coronaviruses	4.2%	7	3.7%	8	4.6%	6.5	4.5%	6
20	Way of the mass media (such as TV and newspapers) report.	3.1%	11	3.0%	9.5	3.4%	9.5	1.7%	13
	Number of tweets	910		630		328		179

GDP: gross domestic product; PCR: polymerase chain reaction; COVID-19: coronavirus disease.

There is no settled theory on how long the agenda-setting effect works. This is especially true for Twitter, where there is a lack of accumulated research. Therefore, to clarify the duration the agenda-setting function works, agenda-ranking tables were created for 2 months, 1 month, 2 weeks, and 1 week before the survey. The result is presented in Table 1 This is the ranking of agendas of Twitter.

The survey data used in this article were obtained through an Internet survey in Japan. The survey was conducted from 10 December 2020, at approximately 22:00 JST to 13 December 2020, at approximately 14:00 JST. The questionnaire was distributed to 23,659 individuals who registered with a research panel through Cross Marketing Inc., an Internet research company in Japan, and its affiliates. We obtained 1000 samples from respondents aged 15–64 years, living in Tokyo, whose distribution of gender and age were assigned according to the population distribution of Tokyo (as of January 2020).

The survey asked the following questions regarding the 20 items in Table 1:

Topics regarding COVID-19 have become important topics in Japan today. How much interest do you have in the following topics with regard to COVID-19? Please indicate your level of interest on a scale ranging from 1 to 10. (1 = Not at all interested, 10 = Very interested).

The respondents who gave the same answers to all items in this question were excluded from the analysis. As a result, the sample size for the analysis was 903. Statistics based on gender and age are listed in Appendix 1. The total score for each item, multiplied by the score and its frequency, is presented in Table 2. This is the ranking of the public agendas.

Table 2.

Public agendas (n = 903).

	Public agendas
	frequency × score	Rank
Public policies to promote the economy (such as “GoTo Travel”, “GoTo Eat”)	5405	9
Information on the number of infected people (including cluster outbreaks) in Japan and abroad	5582	5
Vaccine development and securing and provision of vaccines by the government	5857	2
Difficulties in the medical field and strengthening of medical resources	6003	1
The impact on corporate profits (such as bankruptcy and closure of businesses)	5451	8
Infection control measures by the government and companies (such as cluster measures, telework promotion, and short-term business)	5384	10
Deterioration of the employment situation (including a decrease in salaries)	5557	6
Hosting of the Tokyo Olympics	4479	19
Impacts on the macroeconomy (such as stock prices, GDP, and consumption)	5,159	13
Transportation of people to and from overseas (such as business and tourism)	4975	16
Infection prevention (such as virus infectivity, situations where people are susceptible to infection and effectiveness of masks)	5,839	3
Impacts on school and university education (including entrance examination and tuition fees)	4,693	17
Support for the needy and increase in the number of suicides	5,158	14
Cultural events (such as live music concerts)	4,562	18
Expand PCR test	5328	11
Overall attitude and policy of the government in dealing with COVID-19 is not good	5470	7
Celebrities infected (such as entertainers, athletes, and politicians)	3804	20
Campaigns of businesses (such as restaurants) that are doing their best even during the COVID-19 pandemic	5231	12
Symptoms (including sequelae) of new coronaviruses	5827	4
Way of the mass media (such as TV and newspapers) report	5139	15

GDP: gross domestic product; PCR: polymerase chain reaction; COVID-19: coronavirus disease.

Spearman’s rank correlation coefficient can only tell us the correlation of each ranking, but it cannot identify factors that affect people’s cognition of agendas. In other words, we cannot exclude the possibility that agendas happen to be correlated due to factors other than the media exposure. Therefore, we conduct analyses by groups through creating groups according to the status of media uses, to corroborate the relationship between people’s agendas and media agendas. It is important to note that tendencies to use particular media are also associated with other media uses. In this survey, the respondents were asked about the time spent on 11 types of media (detailed in Appendix 2). If we simply use Twitter browsing time as a variable, we cannot eliminate the influence of other media outlets. Therefore, we conducted the following operation:

Twitter browsing orientation = \frac{Twitter browsing time}{Time spent on all media outlets}

The result of these formulas is the relative proportions of Twitter in daily media consumption. Next, we created groups with a Twitter browsing orientation more than the median and the median or less. In addition, we created groups with a liberal tendency more than the median and the median or less. We then created ranking lists of public agendas for each group. The method of measuring the liberal tendency is described in detail in the relevant section of Study 2.

Through the above process, we created five agenda-ranking tables for the entire sample, for groups with high and low Twitter browsing orientation and for a liberal group and a conservative group. For each of these, we examined the correlation with the agenda-ranking tables for Twitter by periods.

Results

Table 3 summarizes the results. The first thing to note is that Twitter agendas are only correlated with the public agenda rankings for a 1-week term. This result suggests Twitter does not have a relatively long-term agenda-setting function.

Table 3.

Rank correlation coefficients between Twitter agendas and public agendas.

	Twitter
	2 months	1 month	2 weeks	1 week
Public (entire) (n = 903)				.540*
Public (entire) (n = 903)				0.014
Twitter browsing orientation (high) (n = 444)				.554*
Twitter browsing orientation (high) (n = 444)				0.011
Twitter browsing orientation (low) (n = 445)				.516*
Twitter browsing orientation (low) (n = 445)				0.020
Liberal tendencies (high) (n = 414)				.546*
Liberal tendencies (high) (n = 414)				0.013
Liberal tendencies (low) (n = 489)				.488*
Liberal tendencies (low) (n = 489)				0.029

The top row of each column shows a Spearman’s rank correlation coefficient, and the bottom row shows p-values. Only values with p-values less than 5% are listed.

p < .05.

In terms of media uses, the correlation coefficient with the Twitter agendas (1 week) is larger for the higher Twitter browsing orientation group than for the lower one. With regard to political attitudes, the correlation coefficient with the Twitter agenda (1 week) was approximately .058 points larger for the group with high liberal tendencies than that for the group with low liberal tendencies.

Discussion

Since Spearman’s rank correlation coefficient reveals only a simple bivariate correlation, it is difficult to demonstrate whether the results reflect the characteristics of the media or is influenced by other factors. Thus, the analyses by the groups can be useful. Since the correlation coefficients with the Twitter agendas are higher for the higher Twitter browsing orientation group than for the lower one, we can at least assume that the higher the proportion of time spent browsing Twitter in daily life, the more people’s agendas resonate with the Twitter agendas. This implies that there is a certain degree of influence of the media on the public agendas.

In terms of political attitudes, the Twitter agendas were relatively strongly correlated with the agendas of the group with higher liberal tendencies. This is consistent with the results of the Twitter log data analysis in two aspects. First, the most popular Twitter agenda throughout the entire period was criticism of the government and the ruling party. The second is the evaluation for the “Go To Travel” program,¹ which was a key policy of the Japanese government. In contrast to the newspapers which tend to be neutral in their reporting, of the tweets surveyed that mentioned the policy, 79.2% (n = 77) were critical of the policy. Based on the study’s survey data, a negative correlation was observed between liberal tendencies and the degree of approval for the government (r = −.336, p < .001). Therefore, it is possible that there is a liberal bias in the discourses on Twitter. This point will be further analyzed in Study 2 through analysis of the agenda-builders.

Study 2

The analysis by the rank correlation is limited at the aggregate level. Therefore, in Study 2, we clarify the biases of Twitter as a medium by exploring the characteristics of the builders of the Twitter agendas, the information disseminators.

Hypotheses

Generally, when people are asked whether they agree or disagree with political issues, the distribution of opinions is close to a normal distribution. Tanaka and Hamaya (2019) calculated the distribution of opinions on the revision of Article 9 of the Japanese Constitutional Law based on the number of online postings. They found that “strongly disagree” or “strongly agree” was the most common, that is, the distribution of opinions was valley-shaped. This result implied the more people had strong opinions, the more they tended to post them on the Internet. In the context of agenda-setting research, McGregor and Vargo (2017) found that high-frequency tweeters were more strongly correlated with the agendas of partisan media accounts. Therefore, the following hypothesis is set:

H2-1. Politically extreme people are more likely to (a) tweet or (b) retweet on COVID-19.

As mentioned earlier, it is not clear from relevant studies whether the Twitter agendas are more associated with liberals or conservatives (McGregor & Vargo, 2017; Vargo et al., 2014). Therefore, we reference the following two aspects of the results of Study 1 in this article. (1) The correlation coefficient with the Twitter agendas is larger for the group with higher liberal tendencies. (2) The top Twitter agenda throughout the entire period is criticism of the government and the ruling party. Based on these two points, we formulate the following hypothesis:

H2-2. People with liberal tendencies are more likely to (a) tweet or (b) retweet on COVID-19.

In addition, this study examines motivations of those who tweet or retweet on COVID-19. As mentioned earlier, given that there are many posts critical of the government on Twitter, such a situation resembles a “Flaming” phenomenon. Yamaguchi (2017) conducted an empirical analysis on the motivations of people who participated in flaming. The results revealed that 60%–70% of those who participated in flaming wrote on the Internet out of a sense of justice, such as “I participate in flaming because I couldn’t forgive.” In addition, the model analysis revealed that those who believed that “flaming makes society better,” which Yamaguchi (2017) labeled as “justice type,” were more likely to participate in flaming. To put this in the context of this study, people who criticize the government on Twitter are motivated by a sense of justice to make society better by condemning the government’s failures. Therefore, we formulate the following hypothesis:

H2-3. People who view “flaming” positively are more likely to (a) tweet or (b) retweet on COVID-19.

Method

First, we looked at the descriptive statistics of how many people have tweeted or retweeted on COVID-19. Second, to test the aforementioned hypotheses, we conducted binomial logit model analyses.

The dependent variables are “whether or not individuals have tweeted on COVID-19 in the past two to three months” and “whether or not individuals have retweeted on COVID-19 in the past two to three months.” For each of these, the variable was set to “0” for “not at all” and “1” for the others in Tables 5 and 6. The explanatory variables were created as follows.

Strength of political orientations and liberal tendencies

For political attitudes, we followed Tanaka and Hamaya’s (2019) measurements and set the following 10 items. The responses were scored on a 7-point scale (1 = agree, 2 = moderately agree, 3 = slightly agree, 4 = neither agree nor disagree, 5 = slightly disagree, 6 = moderately agree, and 7 = disagree).

[1] Amend Article 9 of the Japanese Constitutional Law.

[2] Increase social security spending.

[3] Allow married couples to choose their own family name.

[4] Between economic growth and environmental protection, I want to give priority to environmental protection.

[5] Nuclear power plants should be abolished immediately.

[6] The government should guarantee jobs and income to some extent.

[7] Teach patriotism to children in schools.

[8] China’s invasion of the Japanese territorial sea should be eliminated even if military force is used.

[9] I think the ex-prime minister Abe’s administration was trying to take Japan back to the dark days before World War II.

[10] Between the interests of the nation as a whole and the interests of the individual, I give priority to the interests of the individual.

The responses for these items were examined using factor analysis (number of factors = 2, maximum likelihood method, and promax rotation). As a result, [3] and [10] were excluded because of low communality (.103 and .113, respectively). The responses for the remaining eight items were scored on a scale from 1 to 3 for liberal responses and −3 to −1 for conservative responses, with “neither agree nor disagree” being 0. Then, these responses were combined, and their average was the individuals’ liberal tendencies. In other words, if individual i’s answer to question j is ( $q_{i j}$ = −3, −2, −1 0, 1, 2, 3), liberal tendencies are expressed as $l i b_{i} = (\sum_{j = 1}^{8} q_{i j}) / 8$ . The strength of political orientation is expressed as $e x t_{i} = | l i b_{i} - l i b_{0} |$ . Since the strength of political orientation is the bias of one’s thinking toward either liberals or conservatives, the larger the absolute value of a person’s liberal tendencies lib, the more extreme that person’s orientation. Therefore, we can take the absolute value as an indicator of the strength. The value of lib₀ on the right side is the median value, and the value of ext_i becomes larger as the opinions become more extreme, so ext_i is an indicator of the strength of political orientation. This method is called the “folding method” and is often used as an indicator of an individual’s political orientation (Barberá, 2015; Lee et al., 2014).

Attitudes toward flaming

The following two measurements were created based on those used in Yamaguchi (2017) for attitudes toward flaming. For each of them, we asked a question on a seven-point scale ranging from “agree” to “disagree” and scored them by inverting the numbers corresponding to their responses:

What do you think about “flaming?” Please choose one that is closest to your opinion.

Flaming is a phenomenon in which the Internet is flooded with critical comments about what a person or company has said or done.

[1] It is good for society because it exposes corporations’ injustice and celebrities’ anti-social behavior (such as discriminatory remarks).

[2] It improves society by punishing the misbehavior of ordinary people who do not have morals.

Cronbach’s alpha of the responses to these two items was .737, which was sufficient, so we combined the two items to create a variable that measures the flaming orientation.

Control variables

For control variables, we used basic demographic attributes (gender, age, education, and annual household income), Twitter browsing orientation (defined in Study 1), interest in COVID-19, interest in politics, approval degree for the government, and attitude toward COVID-19 and society. It is easy to assume that higher Twitter browsing orientation leads to tweeting or retweeting. Similarly, for interest in COVID-19, it is assumed that the higher the interest, the higher the probability of tweeting or retweeting. The same situation also applies to interest in politics. Therefore, these factors should be controlled. Furthermore, the approval degree for the current government and liberal tendencies are negatively correlated (r = −.336, p < .001). Based on Study 1, it is possible that Twitter has become the main platform for spreading dissatisfaction with the current government. In other words, it is possible that people who are negative toward the current government in the first place are also developing criticisms of the government on COVID-19-related topics. Therefore, it is necessary to set the approval degree for the current government as a control variable. In addition, given that the period in which the survey was conducted was in the midst of the “third wave” of the COVID-19 outbreak in Japan and that Tokyo recorded the highest number of infections (12 December: 620 people), it is easy to assume that information behavior would change as a result. Therefore, the attitude toward COVID-19 and society should also be controlled. The measurement is as follows: “A: It is important to maintain employment and the economy even in the face of COVID-19.” and “B: It is important to reduce and refrain from various activities to prevent the spread of infection.” In this survey, we asked respondents to indicate which of the two options they thought was closest to their opinion on the seven-point scale (the number closer to B is larger).

Basic statistics

The basic statistics of the explanatory and control variables are listed in Table 4.

Table 4.

Basic statistics of explanatory variables and control variables.

	Frequency	Min	Max	Median	M	SD
Gender(male = 0; female = 1)	903	0	1	0	0.50	0.50
Age	903	15	64	41	40.72	13.13
Dummy of a university graduate	903	0	1	1	0.70	0.46
Annual household income	903	0	2250	550	676.88	460.06
Twitter browsing orientation	889	0	0.803	0.013	0.055	0.100
Interest in COVID-19	903	1	7	5	5.18	1.37
Interest in politics	903	1	7	5	5.02	1.50
Approval degree for the government	903	1	10	4	4.15	2.12
Attitude to COVID-19 and society	903	1	7	4	3.90	1.73
Strength of political orientations	903	0	3.13	0.38	0.61	0.60
Liberal tendencies	903	−3	3	0.13	0.12	0.85
Flaming orientation	903	1	7	4	3.76	1.32

SD: standard deviation; COVID-19: coronavirus disease.

Results

In this survey, we asked respondents how many tweets and retweets on the topic of COVID-19 they had posted per month on average over the past 2–3 months. The results are presented in Tables 5 and 6. About 25.8% of Twitter users tweeted on COVID-19. Similarly, 24.0% of Twitter users retweeted on COVID-19. We also conducted an analysis based on the number of tweets and retweets by multiplying the average number of tweets and retweets of each choice by their frequencies. The results revealed that 2.1% of users generated 54.3% of COVID-19-related tweets, and 3.1% generated 62.3% of the retweets. This means that more than half of the COVID-19-related discourses on Twitter were generated by a small number of users, so it can be expected that there are some biases in the information shared on Twitter.

Table 5.

Frequency of tweets related to COVID-19 (in the past 2–3 months).

	n = 430 (Twitter users)	n = 903 (total sample)
Not at all	74.2%	87.0%
Once a month	6.5%	3.3%
2–3 times per month	8.8%	4.7%
4–9 times per month	3.7%	1.8%
10–19 times per month	2.8%	1.3%
20–29 times per month	1.9%	0.9%
30–59 times per month	1.2%	0.6%
60–99 times per month	0.7%	0.3%
100–199 times per month	0%	0%
200 or more times per month	0.2%	0.1%

COVID-19: coronavirus disease.

n = 430 on the left side of the table refers to the number of people who had registered on Twitter in the past 2–3 months.

Table 6.

Frequency of retweets related to COVID-19 (in the past 2–3 months).

	n = 430 (Twitter users)	n = 903 (Total sample)
Not at all	76.0%	88.0%
Once a month	5.8%	2.9%
2–3 times per month	7.2%	3.7%
4–9 times per month	3.0%	1.6%
10–19 times per month	3.3%	1.6%
20–29 times per month	1.6%	0.9%
30–59 times per month	1.6%	0.8%
60–99 times per month	0.9%	0.4%
100–199 times per month	0.5%	0.2%
200 or more times per month	0%	0%

COVID-19: coronavirus disease. n = 430 on the left side of the table refers to the number of people who had registered on Twitter in the past 2–3 months.

Analyses by model

Based on the above, the model used in this study is defined as follows, with individuals as denoted by i:

l o g i t [P (T w e e t_{i} = 1)] = \log (\frac{P [T w e e t_{i}]}{1 - P [T w e e t_{i}]})

\begin{array}{l} = α + β_{1} C h a r a c t r i s t i c s_{i} + β_{2} C o n t r o l_{i} \\ + γ_{1} P o l i t i c a l_{i} + γ_{2} F l a m i n g_{i} \end{array}

The model is a logit model. All variables on the right side are standardized, and whether to retweet or not is formulated by converting the left side into $l o g i t [P (R T_{i} = 1)]$ . Each symbol refers to the following.

$T w e e t_{i}$ : A dummy variable indicating whether or not i has tweeted on COVID-19-related topics. The value is 1 if i tweeted and 0 otherwise.

$P (T w e e t_{i} = 1)$ : Probability of being a $T w e e t_{i}$

$C h a r a c t r i s t i c s_{i}$ : An attribute vector of i. Specifically, it is a vector with four variables: gender, age, education, and annual household income.

$C o n t r o l_{i}$ : A vector on the control variables of i. Specifically, a vector with five variables: Twitter browsing orientation, interest in COVID-19, interest in politics, approval degree for the current government, and attitude toward COVID-19 and society.

$P o l i t i c a l_{i}$ : A vector on the political attitudes of i. Specifically, it is a vector with two variables: strength of political orientations and liberal tendencies.

$F l a m i n g_{i}$ : A vector on the attitude toward flaming of i.

Tables 7 and 8 present models in which all these variables are fed. As for whether or not individuals tweeted (Model 1), looking at the control variables, there were negative correlations for gender (a dummy variable of female) and age and positive correlations for Twitter browsing orientation and interest in COVID-19. In particular, the standardized regression coefficient for Twitter browsing orientation was large (Exp(B) = 2.455, p < .001), so this had relatively large explanatory power for whether people tweet or not. As for the explanatory variables, while there was a significant positive correlation for liberal tendencies (Exp(B) = 1.276, p = .045), there was no significant correlation for the strength of political orientation and the flaming orientation. Therefore, H2-2(a) was supported, whereas H2-1(a) and H2-3(a) were not supported.

Table 7.

The result of a binomial logit model analysis (Model 1).

Dependent variable: tweet
	Exp(B) (standardized)	SE	p value	95% CI
Gender	.739*	.121	.012	[.583, .936]
Age	.713**	.128	.008	[.554, .917]
Dummy of a university graduate	1.168	.119	.192	[.925, 1.474]
Annual household income	.960	.115	.726	[.767, 1.203]
Twitter browsing orientation	2.455**	.114	<.001	[1.965, 3.068]
Interest in COVID-19	1.518**	.141	.003	[1.151, 2.002]
Interest in politics	.973	.137	.839	[.744, 1.272]
Approval degree for the government	1.224	.123	.102	[.961, 1.558]
Attitude to COVID-19 and society	1.025	.124	.844	[.803, 1.307]
Strength of political orientations	1.026	.115	.824	[.819, 1.284]
Liberal tendencies	1.276*	.122	.045	[1.005, 1.621]
Flaming orientation	1.244	.117	.063	[.988, 1.565]
Constant	.100**	.133	<.001
Nagelkerke’s R² = .286
	n = 889

SE: standard error; CI: confidence interval; COVID-19: coronavirus disease.

p < .05. **p < .01.

Table 8.

The result of a binomial logit model analysis (Model 2).

Dependent variable: retweet
	Exp(B) (standardized)	SE	p value	95% CI
Gender	.852	.124	.195	[.668, 1.086]
Age	.732*	.134	.020	[.563, .952]
Dummy of a university graduate	1.174	.123	.193	[.922, 1.495]
Annual household income	.916	.121	.469	[.723, 1.161]
Twitter browsing orientation	2.459**	.113	<.001	[1.969, 3.071]
Interest in COVID-19	1.314	.144	.057	[.992, 1.740]
Interest in politics	1.115	.146	.454	[.838, 1.485]
Approval degree for the government	1.021	.126	.868	[.797, 1.308]
Attitude to COVID-19 and society	1.059	.127	.653	[.825, 1.359]
Strength of political orientations	1.102	.116	.400	[.879, 1.382]
Liberal tendencies	1.217	.122	.107	[.959, 1.546]
Flaming orientation	1.397**	.121	.006	[1.102, 1.771]
Constant	.091	.137	<.001
Nagelkerke’s R² = .283
	n = 889

SE: standard error; CI: confidence interval; COVID-19: coronavirus disease.

p < .05. **p < .01.

As for whether or not individuals retweeted (Model 2), there was a negative correlation with age and Twitter browsing orientation had a positive correlation (Exp(B) = 2.459, p < .001). As for the explanatory variables, there was a significant positive correlation with the flaming orientation (Exp(B) = 1.397, p = .006), whereas there was no significant correlation with the strength of political orientation and liberal tendencies. Therefore, H2-3(b) was supported, whereas H2-1(b) and H2-2(b) were not supported. Notably, analyses were conducted using the SPSS software version 26 and replicated using the R version 4.0.3.

Discussion

For liberal tendencies, there was a significant positive correlation only in Model 1. However, for the flaming orientation, there was a significant positive correlation only in Model 2. Moreover, the standardized regression coefficients of the flaming orientation in Model 2 were relatively large, which indicated a relatively large impact on the decision-making process of retweeting COVID-19-related topics.

These results suggest two implications. First, the result that being liberal connotes that a person is more likely to tweet is consistent with the results of Study 1. The agenda regarding COVID-19 on Twitter resonated with liberals. Therefore, it is natural that tweeters, as the builders of agendas, tend to be liberal. Second, there is a difference in nature between tweeting and retweeting. This matter can only be discussed hypothetically because there is no related research. The psychological cost of retweeting is considered relatively lower than that of tweeting because retweeting is a reaction to original tweets. Unlike tweeting, which requires the creation of original text, retweeting can be done in one click. Therefore, the decision-making process to retweet is considered to be less burdensome than the decision to tweet. Based on this nature of retweeting and the fact that the measurements used in this study equated flaming with seeking justice, it is possible to see that people with liberal tendencies, motivated by their political attitude, create original tweets, and some of those tweets are then retweeted by flaming-oriented people driven by a sense of justice. This seems to be how information about COVID-19 spreads on Twitter. The reason for this may be that the motivation for tweeting or retweeting is the negative sentiment, and since the current administration is conservative, liberal people who are against the government have become active in the Twitter-sphere in response.

By contrast, the correlation with the strength of political orientation, which was significant in previous studies, was not significant. This also could be attributed to the differences in the topics addressed. The topic covered in Tanaka and Hamaya (2019) was “whether or not people agree or disagree with the revision of Article 9 of the Japanese Constitutional Law,” which is a topic somewhat distant from the daily lives of citizens. In addition, the topics covered by McGregor and Vargo (2017), such as economy, environment, and diplomacy, were highly abstract. In comparison, COVID-19, the subject of this study, is an infectious disease that directly affects the health and life of individuals, a familiar topic in the daily lives of citizens. Therefore, it is possible that a person need not have a strong political orientation to disseminate or spread information on COVID-19.

Conclusions and theoretical implications

In Study 1, we found that Twitter agendas resonated with individuals who had liberal tendencies. Although this issue has been the subject of previous agenda-setting studies by intermedia, the results have been inconsistent. By combining Twitter log data analysis and survey research, this study reveals the biases of Twitter as a medium. In Japan, because the ruling party is relatively conservative, liberals who oppose it constitute the discourse in the Twitter-sphere. In other words, the negative bias on Twitter manifested itself as an attitude of criticism of the conservative government. Study 2 suggested the situation of the Twitter-sphere regarding COVID-19 where individuals with liberal tendencies generate tweets, and some of those tweets are then retweeted by flaming-oriented people driven by a sense of justice.

Study 1 and Study 2 confirmed that there is a liberal bias in the discourse on Twitter regarding COVID-19 in Japan, which is currently run by a conservative government, and sentiments against them are shaping Twitter agendas.

We have progressed relevant literature on media’s agenda-setting from two aspects. First, we combined log data analysis with survey data analysis to reveal the relationship between social media agendas and actual public agendas. Second, we uncovered biases in the Twitter-sphere by simultaneously conducting an agenda-setting study and an agenda-builders study. This study also reinforces the fact that there is selective exposure in the context of agenda-setting research, as shown by Wu and Guo (2020). In other words, the phenomenon that liberal people build agendas and liberal people are influenced more by Twitter agendas can be considered as a kind of selective exposure. These findings advance the possibilities of agenda-setting theory by revealing not only that social media “has or does not have” an agenda-setting function for public opinion, but also “what kind of agenda-setting function it has,” and clarifying the biases of the Twitter-sphere.

Moreover, this study makes a substantial contribution to the analysis of political communication because it clarifies the mechanism of Twitter bias in Japan from the perspective of agenda-setting theory. This is a major development in communication research as not many studies have clarified the structure of such biases.

Furthermore, this study may serve as a reference for people to read the information on Twitter. In Japan, Twitter is often used for social movements. Often, there are bursts of critical comments about public policies. How we should deal with such phenomena? This study provides suggestions thereof. In other words, Twitter has a liberal bias and a tendency to flame and does not necessarily reflect public opinion. It is vital that people respond calmly to information on Twitter.

Limitation and directions for future research

This article has some limitations. Since this study is based on a one-shot survey, it is not possible to clarify the causal relationship between variables. In other words, the model cannot reveal the direction of causality between Twitter agendas and public agendas or the direction of causality between liberal tendencies and tweeting behavior. In the future, it is necessary to identify causal relationships using time-series data. For example, it may be beneficial to use computational social science methods to capture changes in the trends of discourses in the Twitter-sphere, and use time-series data to clarify how people’s perceptions and attitudes toward agendas change in response. This means that to get a more rigorous understanding of the agenda-setting function, we need to conduct multiple surveys and combine them with social media log data.

This study was also limited in that it used data only from Japan. It is recommended that the anti-government bias of the Twitter-sphere revealed in this study be tested for its robustness using data from Western countries.

If information contact on Twitter is selective in nature, it may not make sense to identify a general Twitter agenda and question people’s perceptions of it. However, this study has shown that, at least in the Twitter-sphere in Japan, liberal discourses are predominant. The reason for this is that, as previous studies have shown, Twitter has a negativity bias and the current government in Japan is conservative. In other words, liberal opinions predominate as negative opinions against conservatism. Just finding that Twitter’s negativity bias works in a political context can be a major theoretical contribution to social media information diffusion. The presence of Twitter as a medium in political communication is not minor. A clarification of the bias of Twitter will support good political communication. This study is significant as its etude.

Footnotes

Appendix 1

Appendix 2 Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Grant-in-Aid for JSPS (Japan Society for the Promotion of Science) Fellows Grant Number 19J22028. In addition, Mr. Hiroki Deguchi (DeNA Co., Ltd.) offered help in using the Twitter API. We would like to express our deepest gratitude to them.

ORCID iD

Tsukasa Tanihara

Notes

Author biography

Tsukasa Tanihara is a Ph.D. Student at the Graduate School of Human Relations, Keio University, and Special Research Fellow for JSPS, the Japan Society for the Promotion of Science. After working for the Ministry of Education, Culture, Sports, Science, and Technology, he moved to the research field in 2017. He got his Master’s degree in Sociology at Ritsumeikan University (Japan). He has been studying media effects from the perspective of social psychology and history. He has published some academic articles about the history of magazines and cinema in Japan.

References

Barberá

(2015, September 3–6). How social media reduces mass political polarization: Evidence from Germany, Spain, and the U.S. [Paper presentation]. 2015 American Political Science Association Conference, San Francisco, CA, United States. http://pablobarbera.com/static/barbera_polarization_APSA.pdf

Ceron

(2014, August 28–31). Twitter and the traditional media: Who is the real agenda setter? [Paper presentation]. 2014 American Political Science Association Conference, Washington, DC, United States. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2454310

Cheng

(2016). The third-level agenda-setting study: An examination of media, implicit, and explicit public agendas in China. Asian Journal of Communication, 26(4), 319–332.

Cheng

Chan

(2015). The third level of agenda setting in contemporary China: Tracking descriptions of moral and national education in media coverage and people’s minds. International Journal of Communication, 9, 1090–1107.

Fan

Zhao

Chen

(2014). Anger is more influential than Joy: Sentiment correlation in Weibo. PLOS ONE, 9(10), Article e110184. https://doi.org/10.1371/journal.pone.0110184

Hennig-Thurau

Wiertz

Feldhaus

(2015). Does Twitter matter? The impact of microblogging word of mouth on consumers’ adoption of new movies. Journal of the Academy of Marketing Science, 43(3), 375–394. https://doi.org/10.1007/s11747-014-0388-3

Lee

J. K.

Choi

Kim

(2014). Social media, network heterogeneity, and opinion polarization. Journal of Communication, 64(4), 702–722. https://doi.org/10.1111/jcom.12077

McCombs

M. E.

(2014). Setting the agenda: The mass media and public opinion (2nd ed.). Polity Press.

McCombs

M. E.

Lopez-Escobar

Llamas

J. P.

(2000). Setting the agenda of attributes in the 1996 Spanish general election. Journal of Communication, 50(2), 77–92. https://doi.org/10.1111/j.1460-2466.2000.tb02842.x

10.

McCombs

M. E.

Shaw

D. L.

(1972). The agenda-setting function of mass media. Public Opinion Quarterly, 36(2), 176–187. http://www.jstor.org/stable/2747787

11.

McGregor

S. C.

Vargo

C. J.

(2017). Election-related talk and agenda-setting effects on Twitter: A big data analysis of salience transfer at different levels of user participation. The Agenda Setting Journal, 1(1), 44–62. https://doi.org/10.1075/asj.1.1.05mcg

12.

Ministry of Internal Affairs and Communications. (2020). Shingata coronavirus ni kansuru jyouhouryuutsu chousa [An information distribution survey on new coronavirus infections]. https://www.soumu.go.jp/menu_news/s-news/01kiban18_01000082.html

13.

Salganik

M. J.

(2017). Bit by bit: Social research in the digital age. Princeton University Press.

14.

Takeshita

(2006). Current critical problems in agenda-setting research. International Journal of Public Opinion Research, 18(3), 275–296. https://doi.org/10.1093/ijpor/edh104

15.

Takeshita

Mikami

(1995). How did the mass media influence the voters’ choice in the 1993 general election in Japan? A study of agenda-setting. Keio Communication Review, 17(3), 27–41.

16.

Tanaka

Hamaya

(2019). Netto ha syakai wo bundan shinai [The Internet does not polarize society]. KADOKAWA.

17.

Toriumi

(2020, July 8). 2020 tochijisen de koike tochiji heno ouen message ga Twitter jyou ni hotondo nkatta ken [There were almost no messages of support for Governor Koike on Twitter in the 2020 governorship race]. Yahoo! News. https://news.yahoo.co.jp/byline/toriumifujio/20200708-00186958/

18.

Vargo

C. J.

(2011, August 10–13). Twitter as public salience: An agenda-setting analysis [Paper presentation]. AEJMC Annual Conference, St. Louis, MO, United States. https://www.academia.edu/17048587/

19.

Vargo

C. J.

Guo

(2017). Networks, big data, and intermedia agenda-setting: An analysis of traditional, partisan, and emerging online U.S. news. Journalism & Mass Communication Quarterly, 94(4), 1031–1055. https://doi.org/10.1177/1077699016679976

20.

Vargo

C. J.

Guo

McCombs

M. E.

Shaw

D. L.

(2014). Network issue agendas on Twitter during the 2012 U.S. Presidential Election: Network issue agendas on Twitter. Journal of Communication, 64(2), 296–316. https://doi.org/10.1111/jcom.12089

21.

Wollebæk

Karlsen

Steen-Johnsen

Enjolras

(2019). Anger, fear, and echo chambers: The emotional basis for online behavior. Social Media + Society, 5(2), 1–14. https://doi.org/10.1177/2056305119829859

22.

H. D.

Guo

(2020). Beyond salience transmission: Linking agenda networks between media and voters. Communication Research, 47(7), 1010–1033. https://doi.org/10.1177/0093650217697765

23.

Yamaguchi

(2017). Enjyou ni kakikomu douki no jissyou bunnseki [An empirical analysis of flaming participants’ motives]. InfoCom Review, 69, 61–74.