Abstract
With an increasing number of people, especially adolescents, using more private online platforms, such as WhatsApp, for news, an important question for democracy is whether such platforms can facilitate learning about politics and current events. In this study, we examine adolescents’ affective (emotions, feelings), behavioral (actions and behavioral intentions), and cognitive (political knowledge) responses to interpersonal political discussion on WhatsApp. We conducted a preregistered field experiment at six secondary schools in the Netherlands (N = 230). We assigned respondents with strong ties to a WhatsApp group. For seven days, respondents received a link to an online political news item on a daily basis; and (1) either had to read or (2) read and discuss it. The results indicate that interpersonal discussion evokes stronger positive emotions and feelings, as well as issue-specific knowledge. In addition, elaboration on the content of political discussion was positively related to issue-specific knowledge. In this way, instant messaging apps may serve as a resource for engaging adolescents with politics and current events.
Keywords
Sharing and discussing news appears to be on the rise within instant messaging apps, and WhatsApp in particular (Newman et al. 2019). With more than one billion active users in over 180 countries, WhatsApp use for news has almost tripled since 2014, and hence, overtaken Twitter. Instant messaging apps (e.g., WhatsApp, iMessage, Facebook Messenger, WeChat, and Telegram) give users control over who can see the content they share, and, more importantly, create a trusting sharing environment for exchanging and discussing news content, and political views in particular (Newman et al. 2019). On the one hand, instant messaging apps may knit people’s interpersonal networks close together (see, e.g., Valkenburg and Peter 2009) and foster interpersonal (political) discussion (Valeriani and Vaccari 2018). There is substantive evidence that stronger networks tend to be more agreeable ones (Mutz 2006), which have been demonstrated to promote participation (McClurg 2006). On the other hand, instant messaging apps constrain possibilities for linking communities to wider spheres (Swart et al. 2018b). In this regard, instant messaging apps might limit the diversity of news sources and political opinions that users are exposed to—in line with selective exposure theory (Brundidge 2010)—raising questions about the democratic value of connecting through such apps (Thorson 2014).
Although WhatsApp is increasingly popular for exchanging and discussing news, there are only a limited number of studies examining the use of WhatsApp for news consumption (see, e.g., Dodds 2019; Kligler-Vilenchik 2019; Malka et al. 2015; Masip et al. 2018; Swart et al. 2018a), yet there is no experimental evidence for the causal effects of WhatsApp use for news. We have little knowledge about what news content is shared on instant messaging apps; and, even less knowledge about how it is discussed. In this study, we will therefore focus on the question: What are the affective, behavioral, and cognitive responses to interpersonal political discussion on instant messaging apps (i.e., WhatsApp)?
To move forward, we conducted a novel field experiment at six secondary schools in the Netherlands to examine affective (i.e., emotions and feelings), behavioral (i.e., actions and behavioral intention), and cognitive (i.e., political knowledge) responses to interpersonal political discussion on WhatsApp. We particularly focus on young news consumers as they often turn to WhatsApp and other instant messaging apps for their news (Newman et al. 2019), and their political interest is developing (see, e.g., Neundorf et al. 2013). As adolescents tend to be less involved in politics and current events compared with older adults (see, e.g., Neundorf et al. 2013), instant messaging apps may serve as a resource for engaging adolescents with politics and current events.
By doing so, we contribute to the literature in two important ways. First, we aim to shed light on the role of instant messaging apps in adolescents’ news consumption patterns. Since people are informed through social networks, or, more precisely, through their friends and acquaintances on social networks, people can get exposed to news accidentally, regardless of their will to get informed about political and current events (Fletcher and Nielsen 2018). We build upon previous studies that examined news exposure on “semi-public” online platforms (e.g., Facebook, Twitter), by focusing on instant messaging apps providing a more intimate and controlled environment. This is extremely relevant as recent political events, such as the 2018 presidential elections in Brazil (Baptista et al. 2019), have already raised serious concerns about WhatsApp—playing a crucial role in the spread of misinformation. Second, existing research on the relationship between WhatsApp and interpersonal political discussion relied on cross-sectional data (Valeriani and Vaccari 2018) or longitudinal survey data (Gil de Zúñiga et al. 2019; Valenzuela et al. 2019). By conducting an experimental study among adolescents, we offer an empirical exploration of the democratic implications of the increasing popularity of instant messaging apps, such as political interest and political learning. In other words, we aim to provide a better understanding of the causal effects of interpersonal political discussion on the development of political interest and political knowledge among adolescents.
Interpersonal Political Discussion on Instant Messaging Apps
In recent years, the role of instant messaging apps has become increasingly important in news consumption (Newman et al. 2019), especially among adolescents. Instant messaging apps enable users to send and receive not only text, but also multiple other formats (e.g., images, videos, voice recordings, documents) to either a single person or a group of individuals via their mobile devices. These apps enable conversations in relatively more intimate, closed environments compared with social media platforms such as Facebook or Twitter. As a result, with 9.5 million daily users, WhatsApp is the most used messaging service in the Netherlands. For many, instant messaging apps have become a fixed component of their daily media repertoire (Newman et al. 2019).
As a result, instant messaging apps are increasingly used for interpersonal political discussion, which has been defined as “episodes of political conversation […] that take place between the non-elite members of a political community” (Schmitt-Beck 2008: 341). By providing a relatively private and controlled environment for political conversations (Valeriani and Vaccari 2018), instant messaging apps have changed the ways in which people talk about politics. Instant messaging apps facilitate the exchange of information by enabling users to create their own communities (Swart et al. 2018a), and enable users to discuss content related to politics and current events in a more intimate and private setting compared with social media (Masip et al. 2018).
Previous studies have already shown how people use “semi-public” online platforms to share and discuss news, for example, on Facebook and Twitter (see, e.g., Kalsnes and Larsson 2018; Trilling et al. 2017). Sharing a link to an online news item on such platforms requires only a minimum of effort, however is—depending on privacy settings and characteristics of the site—capable of reaching a large to even unlimited audience compared with instant messaging apps. As a result, people could be concerned that sharing or expressing their views on controversial issues in more public settings may damage their relationships with some members of their online social networks (Vaccari et al. 2016). These people may be more at ease talking about politics and current events in spaces that are more private, closed, and intimate, such as those enabled by instant messaging apps (Valeriani and Vaccari 2018).
WhatsApp in particular has become a primary network for sharing and discussing news (e.g., Brazil: 53%, Malaysia: 50%, and South Africa: 49%; Newman et al. 2019). WhatsApp enables users to share political content through links to online news items, images of newspaper articles, or videos of television programs. Users cannot only access news anywhere and anytime through their phones, but also almost immediately start discussing about what they have just read. Besides, WhatsApp can have potential benefits for democracy, as citizens are able to engage in vibrant political talk and discussion across political ideologies (Kligler-Vilenchik 2019). But so far, only a limited number of studies examined the use of WhatsApp for news consumption (see, e.g., Dodds 2019; Malka et al. 2015; Masip et al. 2018; Swart et al. 2018a). More recently, scholars have focused on understanding political discussion on WhatsApp by using observational and cross-sectional data (Valeriani and Vaccari 2018) or longitudinal survey data (Gil de Zúñiga et al. 2019; Valenzuela et al. 2019). Yet, there is no experimental evidence for the causal mechanisms underlying WhatsApp use for news.
Affective, Behavioral, and Cognitive Responses to Interpersonal Political Discussion on Instant Messaging Apps
As instant messaging apps provide a relatively private and controlled environment for interpersonal political discussion, it is increasingly difficult for researchers to study such apps. We have therefore little to no information about how news is consumed on instant messaging apps. To move forward, we aim to examine the effects of using WhatsApp for discussing political news.
It is important to know more about the role of instant messaging apps in the field of political communication, as interpersonal discussion about political issues has a number of potential benefits for public life. Discussing politics for example allows citizens to gain knowledge and share views, and even increases political participation (Shah et al. 2007). In a recent two-wave panel study, Gil de Zúñiga et al. (2019) found that interpersonal political discussion via WhatsApp seems to foster conventional forms of participation (e.g., voting, contacting elected officials) and activism. People who engage more frequently in political talk and political messaging are more likely to participate in the political process. Interestingly, this positive effect of WhatsApp discussion on participation seems to be stronger for younger than for older respondents.
Besides positive implications, instant messaging apps could have a potentially negative impact on democracy. Recent accounts have highlighted the potential for social media to polarize political discussions—increasing the salience of partisan news (Anspach 2017). The theory of selective exposure argues that citizens in their decisions on media use tend to choose news content that matches their political and ideological positions (Brundidge 2010). To explore the consequences of using messaging apps on democratic citizenship, Valenzuela et al. (2019) conducted a two-wave panel survey focusing on information sharing practices of WhatsApp users. While Valenzuela et al. (2019) found that WhatsApp use is a strong predictor of engagement in protest and other forms of political participation, they did not find a clear-cut relationship between WhatsApp use and issue position extremity.
In our study, we examine the implications of interpersonal political discussion via WhatsApp of adolescents along the lines of the following distinction: affective, behavioral, and cognitive responses. Adolescents would particularly benefit from interpersonal political discussion on instant messaging apps. They tend to be less involved in politics and current events compared with older adults (see, e.g., Neundorf et al. 2013). This means that there is a great amount of untapped potential for them to be engaged citizens. Instant messaging apps may serve as a resource for tapping into this potential by fostering interpersonal political discussion. Indeed, Shah et al. (2001) and Kwak et al. (2006) found that online platforms are a more useful resource for civic and political engagement among younger than older adults. Adolescents may not only use their mobile devices to express political views and opinions, but also to obtain additional political information.
Affective responses
First, we turn to the affective responses (e.g., emotions, feelings) to interpersonal political discussion on instant messaging apps. Emotions and feelings play an important role in the formation, expression, as well as the mobilization of public opinion. Prior work has highlighted the role of emotions—among other processes—in political opinion formation (Kühne et al. 2011; Lecheler et al. 2013), citizens’ attitude toward political issues (e.g., immigration; Wirz et al. 2018) and political participation (Valentino et al. 2011).
To date, few studies have examined the link between affective responses and interpersonal political discussion in social networks. Borrowing from affective intelligence theory, Parsons (2010) argues that interpersonal communication may activate various emotions—depending on whether the information shared is consistent with or contrary to an individual’s predispositions. Interpersonal discussion within homogeneous social networks may elicit feelings of enthusiasm, hope, and/or pride, reinforced by political talk with like-minded others (Huckfeldt et al. 2006). Conversely, the psychological discomfort of political discussion (e.g., through disagreement) within heterogeneous social networks may activate negative feelings, such as anxiety, fear, and/or anger (Mutz 2006). In a more recent study, Valenzuela and Bachmann (2015) found that political disagreement in informal political discussions can also be associated with positive feelings, such as pride.
Turning to the role instant messaging apps, Waterloo et al. (2018) examined the norms of expressing emotions in social networks. They found that the expression of emotions (i.e., anger, sadness, disappointment, worry, joy, and pride) was found to be most appropriate for WhatsApp relative to three other more public platforms, namely Facebook, Twitter, and Instagram. This indicates that more private spaces, such as instant messaging apps, allows for looser norms of emotion expression. Thus, given the consistent findings in the research literature, we propose the following hypothesis:
Behavioral responses
In contrast to affect, which focuses on emotions and feelings, behavior refers to actions as well as behavioral intentions triggered by interpersonal political discussion. It can be argued that participating in political discussions leads to an increase in exposure to a variety of different political viewpoints and arguments. As a consequence, exposure to these different opinions can motivate people to develop more informed views and advance their own political opinions, thus affecting their interest in politics and the conversation (Torcal and Maldonado 2014). Political interest has proven to be one of the most essential factors behind political engagement, participation, and other types of political behavior, characterized as “the best indicator of the possession of awareness, competence, information, and knowledge about politics” (Torcal and Maldonado 2014: 680). Studies suggest that citizens’ interest in politics develops during adolescence and stabilizes by the age of twenty (see, e.g., Prior 2010). As adolescents show higher levels of technological expertise regarding mobile use and, more importantly, have more online friends and belong to more WhatsApp groups (Chan 2018), instant messaging apps could in turn provide new and interesting ways for young citizens to connect with politics and current events (Gil de Zúñiga et al. 2019).
Besides, based on a uses-and-gratifications perspective, previous studies found that people who frequently engage in online political discussion are more internally motivated to actively seek out political information (Scheufele 2002; Yamamoto et al. 2015); so-called communicatory or interpersonal utility. It refers to people’s motivation to use news media to acquire information that they can describe to others or use as arguments during future discussions with others (Scheufele 2002). People can anticipate talking to others about political news by seeking out additional information, for example, to counter opposing viewpoints to strengthen their own argument. All in all, it is expected that interpersonal disagreement could drive information seeking/interest in information about politics (Lyons et al. 2016). We therefore hypothesize the following:
Cognitive responses
Finally, we turn our attention to the importance of interpersonal discussion of news and politics in the process of political learning. Political knowledge has a number of potential benefits for public life, for example, related to voting behavior (e.g., EU issue voting; De Vries et al. 2011) and public opinion formation (see, e.g., Tillman 2012). In addition, Wells and Dudash (2007) argue that young citizens’ political knowledge leads to increased feelings of political efficacy and participation (Wells and Dudash 2007).
In turn, existing research suggests that political discussion helps increase political knowledge (see, e.g., Eveland and Schmitt 2015; Schmitt-BeckLup 2013; Valenzuela et al. 2019). The differential gains model, for example, predicts, “People who process news content more carefully by talking it over with others are also more likely to extract relevant pieces of political knowledge” (Scheufele 2002: 51). In other words, it can be theorized that political discussion helps people to think about political and current events, relate personal experiences to politics, and reconsider issue stances (Scheufele 2002). Besides, during a political discussion, group members can provide information that individual members did not know before (Eveland and Schmitt 2015). In this way, additional exposure to political news enhances individual memory. Interpersonal political discussion also provides an opportunity for repetition and rehearsal of information from memory. As individuals engage in discussion of news, they practice retrieval, repeat the information, and see others do the same (Hirst and Echterhoff 2012). All in all, political discussions are able to consolidate the corresponding information in long-term memory and, eventually, foster knowledge-building processes. We therefore predict that interpersonal political discussion is important in the process of political learning on instant messaging apps. In this study, we are moving beyond overall factual knowledge to issue-specific factual knowledge, as exposure to specific news content could be a better predictor of political learning than exposure to general news content (Eveland and Schmitt 2015). We formulate the following hypothesis:
In addition, we extend our focus on the relationship between political talk and political knowledge. According to Eveland (2004), discussion could influence knowledge through at least three processes. First, the exposure explanation suggests that “individuals glean information from their discussion partner in much the same way that they would gain information from the news media directly” (p. 179). In this way, interpersonal political discussion in instant messaging apps is only an additional opportunity for exposure to information of interest (in addition to, or independent of, exposure to news media). Second, the anticipatory elaboration explanation suggests that the “expectation of an impending discussion is an internal motivation that then increases cognitive elaboration on news content” (p. 180). In this way, increased elaboration could not merely occur during exposure, but also any time before the actual discussion takes place. Finally, the discussion-generated elaboration explanation suggests that “the act of engaging in discussion forces meaningful information processing and thus increases learning due to an influence on information processing during discussion” (p. 180). The findings of Eveland (2004) suggest that the direct relationship between discussion and knowledge may be mediated through motivations and information processing behaviors. Although significant research has demonstrated the role of interpersonal political discussion in learning outcomes, relatively few studies have addressed this process in an online environment. Therefore, we formulate the following research question:
Method
We conducted a field experiment with pupils at six secondary schools in the Netherlands to examine interpersonal political discussion on WhatsApp. 2
Procedure, Respondents, and Design
After a pilot study, we recruited respondents via secondary schools located in various areas in the Netherlands. We contacted secondary schools in advance to explain and promote the study. After obtaining consent of the school’s principals, we visited graduate classes to explain the study. Pupils were invited to participate in the study “WhatsApp with Politics?!” (the University approved the study under IRB protocol number 2019-PCJ-10527). They received an information letter and/or e-mail via the school announcing the project’s goals, planning, and execution. After the respondents gave consent, we started the experiment. In consultation with their school, pupils got a reward for their participation.
Respondents
We collected data in the period from September 5 to November 15, 2019, at six secondary schools in the Netherlands. Our sample consisted of 232 pupils within the mid and highest levels of Dutch secondary education. In total, 230 respondents completed the entire experimental procedure (99.14%), 54.35 percent were female, mean age was 16.69. In the Netherlands, VWO is the highest level of secondary education (duration 6 years), and HAVO is the level below that (duration 5 years). About 22.61 percent of the respondents were in the fifth grade of HAVO, 43.04 percent in the fifth grade of VWO, and 33.91 percent in the sixth grade of VWO. 3 About 47.83 percent of the respondents uses WhatsApp more than ten times per day, and 23.91 percent six to ten times per day. On average, respondents are not often exposed to news by newspapers (M = 1.53 days per week, SD = 1.28; on a scale from 0 days per week to 7 days per week), television (M = 3.26 days per week, SD = 2.00), radio (M = 2.78 days per week, SD = 2.01), websites of newspapers (M = 2.94 days per week, SD = 2.25), or online news websites (M = 3.12 days per week, SD = 2.17). Instead, respondents often use social media to find news (M = 5.66 days per week, SD = 2.65).
The experiment included three different stages: a pretest, a WhatsApp group conversation, and a post-test. We discuss each of these stages in more detail below.
Pretest
In the first stage, respondents filled out a short online survey. The survey contained questions about demographics, media use, political interest, political participation, political orientation, news interest, social norms, political discussion, need for cognition, and WhatsApp use. We also asked their mobile phone numbers.
We assigned classmates with strong ties to the same WhatsApp group for two reasons. First, tie strength is a strong predictor of interpersonal political discussion (Eveland and Schmitt 2015). Second, due to ethical considerations, we aim to minimize the chance of adolescents—participating in our study—feeling harmed, intimidated, or emotionally hurt as much as possible. To do so, respondents were asked to list the classmates that they have most contact with (in school, outside school, and/or online). They listed at least two and could not list more than eight classmates.
WhatsApp group conversation
Based on the provided information, we created small groups of close friends. These groups were randomly assigned to either the control group or the treatment group (see Figure 1). In the posttest, we asked respondents to indicate on a 7-point scale whether they agree or disagree with the following statement: “Of all my classmates, I have most contact with the members of the WhatsApp conversation I was part of” (M = 4.58, SD = 2.06).
Flow diagram of the procedure.
Detecting communities
To create groups of strong ties, we iterated through each of the respondents’ list of strong ties to create links for each social relationship (e.g., pupil X—pupil Y, pupil X—pupil Z). We used a collection of network analysis tools in R (igraph; Csardi and Nepusz 2006) to form a directed graph (i.e., edges in the graph having an associated direction). Next, we detected groups consisting of densely connected pupils based on edge betweenness (Girvan and Newman 2002).
The WhatsApp group included (1) the researcher, (2) respondents who are relationally close to each other, that is, strong ties. In total, there were sixty-one WhatsApp groups, which consisted of three to six respondents. 4 Fay et al. (2000) namely argue that three or four, at the most five, is the optimal group size for interpersonal conversations. In our study, most WhatsApp groups included three (n = 27), four (n = 20), or five (n = 9) respondents. However, dealing with a varying number of respondents per class and lists of tie strength, we were sometimes required to create groups of six respondents (n = 4).
Respondents were part of a WhatsApp group for seven days. During these days, the researcher shared links to online news items at random time points during the day, but never during school hours or at night between 20.30 and 7.00. The experiment had a between-subjects design. We randomly assigned small groups of respondents to one of the two conditions. In the control group, respondents were asked to read the article, 5 whereas in the treatment group we promoted interpersonal discussion.
Posttest
One day after the experiment ended, respondents received another short online survey. The survey contained question about the affective (i.e., emotions, feelings), behavioral (e.g., actions and behavioral intentions related to media use, issue-specific interest), and cognitive (e.g., issue-specific knowledge) responses to the WhatsApp group, as well as questions about social norms, trust, and tie strength.
See Figure 1 for a flow diagram of the procedure.
Stimulus Material
We used links to Dutch online news items of the NOS (i.e., Nederlandse Omroep Stichting, Dutch Broadcasting Foundation) as stimulus material, as it is the most trusted news brand in the Netherlands (Newman et al. 2019). Furthermore, we used the NOS to avoid to some extent familiarity bias, because this news outlet is less popular among youngsters in the Netherlands. This makes prior exposure to the stimulus material less likely. By using two items, we examined whether respondents trust the NOS as a news outlet (from strongly disagree to strongly agree, 7-point scale): “NOS publishes reliable and objective news” (M = 4.68, SD = 1.35), and “One can trust NOS” (M = 4.96, SD = 1.35).
In our study, we strongly depended on the latest news. We shared political news items covering, for example, the reception of asylum seekers, teacher shortages, civilian service for adolescents, Prinsjesdag, a manifesto for generation Y, and women quotas in corporate boardrooms.
Measures
Independent variable
The independent variable of this experiment is interpersonal political discussion (vs. reading). It is the process of making sense of the political information respondents gain through the WhatsApp group conversations. Interpersonal political discussion is coded 1 if the respondent was part of a WhatsApp group in which news items have been discussed (Ngroups = 32, Nrespondents = 128), and 0 if the respondent was part of a WhatsApp group in which news items have merely been read (Ngroups = 29, Nrespondents = 102).
Dependent variables
We used the following set of dependent variables:
Affective responses: We measured affective responses by asking respondents to indicate whether their participation in the WhatsApp group conversation made them feel curious, angry, happy, excited, fearful, and sad (from definitely not to definitely, 7-point scale). We performed a factor analysis to uncover the underlying structure of our variables. We report the total amount of variance in the variables explained by the common factor. We combined negative responses (eigenvalue = 2.24, explained variance = 59.7%, α = .82, M = 1.73, SD = 1.00), and positive responses (eigenvalue = 1.45, explained variance = 54.5%, α =.86, M = 3.91, SD = 1.21). Besides, respondents indicated whether they think the WhatsApp group conversation was interesting, boring, uncomfortable, and relevant (from definitely not to definitely, 7-point scale). Again, we combined positive responses (α = .82). We were not able to combine negative responses (α = .37).
Behavioral responses: To examine the behavioral responses to interpersonal political discussion on instant messaging apps, we asked respondents to indicate whether their participation makes them more inclined to (a) “visit online news websites,” (b) “search issue-specific information,” (c) “discuss the news items with their family and friends,” (d) “share a news item on WhatsApp,” (e) “talk about a news item in a personal conversation on WhatsApp,” (f) “talk about a news item in a group conversation on WhatsApp” (from strongly disagree to strongly agree, 7-point scale). A mean score was calculated (range = 1–7) and used to measure behavioral change (eigenvalue = 3.25, explained variance = 91.1%, α =.85, M = 3.15, SD = 1.21). In addition, we measured issue-specific interest for every news item by asking respondents to indicate whether their participation in the WhatsApp group conversation increased their issue-specific interest (from strongly disagree to strongly agree, 7-point scale). A mean score was calculated (range = 1–7) and used to measure issue-specific interest (eigenvalue = 2.31, explained variance = 76.7%, α = .77, M = 3.69, SD = 1.11).
Cognitive responses: Finally, we examined issue-specific knowledge by asking two multiple-choice (i.e., three answer options and a don’t know option) questions per news item. We operationalized issue-specific knowledge by counting the number of correct answers (maximum 14 points; M = 6.91, SD = 1.91). We started with one practice question: “Who is the current Prime Minister of the Netherlands?.” We included a timer for each question.
Additional variables
We included a set of additional control variables—that have been shown in previous literature to have an impact on our dependent variables (see, e.g., Kenski and Stroud 2006)—in the pretest.
Sociodemographics: We included a variety of sociodemographic variables, such as age, gender, and education.
Political interest: As individuals who are interested in politics are likely to consume more news and be more knowledgeable politically (Kenski and Stroud 2006), we asked respondents to rate their degree of interest in local politics, national politics, European politics, and international politics (from not interested to extremely interested, 7-point scale). A mean score was calculated (range = 1–7) and used to measure political interest (eigenvalue = 2.06, α =.81, M = 3.54, SD = 1.36).
Need for cognition: Need for cognition was measured using five items. We asked respondents the extent of agreement on a 7-point scale: For example, “I prefer to solve more complex problems instead of simple ones,” and “I like to have responsibility for handling situations that require a lot of thinking” (see, e.g., Tian 2011). A mean score was calculated (range = 1–7) and used to measure need for cognition (eigenvalue = 2.78, explained variance = 59.1%, α =.87, M = 3.15, SD = 1.21).
Self-generated elaboration1: Using Python, we excluded all researcher’s messages and WhatsApp notifications (e.g., “Messages to this chat and calls are now secured with end-to-end encryption”). For every participant, we calculated the total number of contributed words (Mreading = 22.40, SDreading = 22.51, Mdiscussing = 309.91, SDdiscussing = 167.51).
Conversation-partner generated information1: For every respondent, we calculated the total number of words they were exposed to. For every respondent, we subtracted the total number of words in their WhatsApp conversation by the number of their own contributed words (Mreading = 58.48, SDreading = 45.16, Mdiscussing = 983.22, SDdiscussing = 468.57).
Analysis
In line with our preanalysis plan, we used a multilevel method to explore the affective, behavioral, and cognitive responses to interpersonal political discussion on WhatsApp. Due to the nested structure of our data, with respondents nested within WhatsApp groups, a multilevel method was used to control for the dependence of the observations. In addition, a multilevel approach enables us to understand group-level factors related to interpersonal political discussion beyond individual-level predictors.
To do so, we first estimated a so called “empty” model for every independent variable to determine the intraclass correlation (ICC): the fraction of total variation in the data that is accounted for by between-group variation. The greater the correlation among units within a group (the bigger the ICC), the greater the impact of the standard error. We examined whether individual differences or cross-level interactions are responsible for the majority of the variance. A high ICC (>.10) indicates we need multilevel model analyses. As the ICC in our study varies from.02 to.22, we have decided to use a multilevel method. 6
We examined whether respondents in the treatment group (i.e., interpersonal political discussion; vs. reading) indicate stronger affective responses: positive emotions (i.e., curious, happy, excited), negative emotions (i.e., angry, fearful, sad), positive feelings (i.e., interesting, relevant), and negative feelings (i.e., boring, uncomfortable). Next, we examined whether respondents in the treatment group (i.e., interpersonal political discussion; vs. reading) indicate stronger behavioral responses: whether their participation makes them more inclined to, for example, search issue-specific information, discuss news items with their family and friends. And, whether their participation in the WhatsApp conversation increased their issue-specific interest. Finally, we explored whether respondents in the treatment group (i.e., interpersonal political discussion; vs. reading) indicate a higher score on issue-specific political knowledge (cognitive responses).
Supplemental Appendix A reports a check of the randomizations.
Results
In total, 230 respondents completed the entire experimental procedure (99.14 percent). 7
Affective Responses
First, we examined whether interpersonal political discussions on instant messaging apps are associated with stronger affective responses. We conducted multilevel regression analyses in which we included several sociodemographic variables such as age, gender, and education. The results are shown as estimated marginal means (predictive margins) in Figure 2 and regression coefficients in Table 1. The bars in Figure 2 represent marginal means by condition, after controlling for age, gender, and education. In this way, the mean response has been adjusted for any other variables in our model.
Affective responses to interpersonal political discussion (predictive margins and SEs).
Multilevel Models to Predict Affective Responses.
Note. ICC = intraclass correlation; AIC = Akaike information criterion; BIC = Bayesian information criterion. Testing for different schools yields identical results. Values in bold are significant.
*p < .05, **p < .01, ***p < .001.
Multilevel Models to Predict Behavioral and Cognitive Responses.
Note. ICC = intraclass correlation; AIC = Akaike information criterion; BIC = Bayesian information criterion. Testing for different schools yields identical results. Values in bold are significant.
*p < .05, **p < .01, ***p < .001.
As shown in Table 1, we found that respondents in the treatment condition showed more positive emotions toward the group conversation (i.e., happy, excited, curious; b = .40, p < .05), as well as more positive feelings toward the group conversation (i.e., interesting, relevant; b = .63, p < .01). The results however demonstrate no effects for negative emotions (i.e., angry, fearful, and sad; b = –.15, p = .32), or negative feelings (i.e., boring; b = –.05, p = .77, uncomfortable; b = –.14, p = .43). In other words, our results show that respondents participating in interpersonal political discussion on WhatsApp evaluate the group conversation not only as more interesting and relevant, but also show more positive emotions (e.g., happy, excited, curious), thereby partially supporting H1.
Behavioral Responses
Second, we examined whether interpersonal political discussions in instant messaging apps are associated with stronger behavioral responses (see Figure 3). We used a multilevel method in which we also included control variables such as age, gender, and education as well as several additional control variables that might have an impact on our dependent variable (political interest, need for cognition). Again, the estimated marginal means are shown in Figure 3. The bars represent marginal means by condition, after controlling for age, gender, and education, as well as need for cognition and political interest. The regression coefficients are shown in Table 2. We found no significant effects for behavioral changes (b = .09, p = .60), such as visiting online news items, searching issue-specific information, sharing or discussing news items in a personal or group conversation on WhatsApp. Furthermore, we found that respondents in the treatment condition did not indicate more issue-specific interest in the news topics that have been discussed in the group conversation (b = –.10, p =.54). In other words, participation in interpersonal political discussion on WhatsApp did not indicate behavioral changes. We found no support for H2.
Behavioral and cognitive responses to interpersonal political discussion (predictive margins and SEs).
In addition, we explored whether certain individual characteristics might have an impact on behavioral responses to interpersonal political discussion on WhatsApp. Interestingly, both political interest (b = .18, p < .01) and need for cognition (b = .23, p < .001) have a positive and significant link with behavioral change. This indicates that both politically interested adolescents and adolescents who engage in and enjoy thinking are more likely to use WhatsApp for news in the future (e.g., share a news item on WhatsApp, talk about a news item on WhatsApp). Furthermore, we found that the relationship of political interest with issue-specific interest is positive and significant (b = .23, p < .05), indicating that politically interested adolescents are more interested in the topics that they read and/or discussed on WhatsApp.
Cognitive Responses
Next, we examined whether participating in interpersonal political discussions in instant messaging apps is positively associated with issue-specific knowledge. Again, we conducted multilevel regression analyses in which we also included our additional control variables (political interest, need for cognition) as well as sociodemographic variables (age, gender, and education) that might have an impact on our dependent variable. The estimated marginal means are shown in Figure 3. The bars represent marginal means by condition, after controlling for need for cognition, political interest, as well as age, gender, and education. Several findings are noteworthy. As shown in Table 2, the results demonstrate that respondents in the treatment condition did have more knowledge about the news topics that have been discussed in the group conversation (b = .66, p < .05). In other words, our results show that participation in interpersonal political discussion on WhatsApp leads to more issue-specific knowledge, thereby supporting H3.
In addition, we explored whether respondents in the control condition answered “I don’t know” more often compared with respondents in the treatment condition. As shown in Table 2, the results demonstrate that respondents in the treatment condition did not answer “I don’t know” more often (b = –.18, p = .46) compared with respondents in the control condition. In other words, the results indicate that respondents who merely read the online news item did not answer “I don’t know” more often compared with respondents who participated in interpersonal political discussion on WhatsApp. Interestingly, we found the relationship of need for cognition with answering “I don’t know” is negative and significant (b = –.20, p < .05). This suggests that adolescents who engage in and enjoy thinking are less likely to answer “I don’t know” to knowledge questions compared with those with a lower need for cognition.
Finally, we examined whether self-generated elaboration (i.e., the discussion-generated elaboration explanation) or conversation-partner generated information (i.e., the exposure explanation) are positively associated with issue-specific knowledge. We conducted a multilevel regression analysis merely for respondents in the treatment group (Ngroups = 32, Nrespondents = 128). As shown in Table 3, we found that self-generated elaboration was a significant predictor of issue-specific knowledge (b = .002, p < .05). This means that participants who actively contribute to a political discussion have more issue-specific knowledge. Interestingly, we found that the effect of conversation-partner generated information on issue-specific knowledge is negative and significant (b = –.001, p < .05). Thus, respondents who actively participate in political discussion, measured as the number of words contributed to the group conversation, have more issue-specific knowledge. Though, as indicated by the negative and significant effect of conversation-partner generated information on issue-specific knowledge, respondents might get distracted by the number of words contributed to the conversation by other members of the WhatsApp group.
Multilevel Models to Predict Issue-Specific Knowledge.
Note. ICC = intraclass correlation; AIC = Akaike information criterion. BIC = Bayesian information criterion. Testing for different schools yields identical results. Values in bold are significant.
*p < .05, **p < .01, ***p < .001.
Robustness check
To examine the robustness of the effects, we also conducted multilevel regression analyses predicting affective, behavioral, and cognitive responses adding perceived tie strength as a control variable. As previously described, respondents were asked in the posttest whether they agree or disagree (7-point scale) with the following statement: “Of all my classmates, I have most contact with the members of the WhatsApp conversation I was part of.” Importantly, the results are very similar to the ones found in the previous analyses (results not shown here). Specifically, after removing the outliers, we still find robust effects for H1 and H3. In addition, we found a positive significant relationship of tie strength with positive emotions (i.e., happy, excited, curious; b = .10, p < .05). Unsurprisingly, this indicates that participation in a WhatsApp group with very strong ties made participants feel more curious, happy, and excited compared with participation in a WhatsApp group with somewhat less strong ties. We found a comparable link for positive feelings (i.e., interesting, relevant; b = .09, p < .05)—suggesting that increased tie strength is positively associated with evaluating participation in the WhatsApp group as more interesting and relevant.
Conclusion and Discussion
This study was set out to uncover the impact of the use of instant messages apps, such as WhatsApp, in the political communication field. Research on the use of instant messaging apps is nearly nonexistent as engagement occurs behind closed doors. By using a novel field experiment, this study adds to existing literature by empirically exploring the democratic implications of instant messaging apps.
Turning to the effects of interpersonal political discussion, the findings from our field experiment suggest that, as indicated by the affective intelligence theory (Parsons 2010), interpersonal political discussion with classmates on WhatsApp has the potential to elicit positive emotions and feelings. Interesting to note, though, is that this effect was not found for negative emotions and feelings (e.g., anger, sadness, boredom). Apparently, the discussion evoked more positive emotions and feelings than negative ones. This finding partly dovetails with previous work by Waterloo et al. (2018). WhatsApp and other instant messaging apps particularly facilitate discussions among friends and friends—usually characterized by intimacy, respect, and mutual regard (Kenny 1994)—which might result in ideologically more homogeneous networks. Huckfeldt et al. (2006) argues that interpersonal political discussion within homogeneous social networks may particularly elicit feelings of enthusiasm, hope, and/or pride, reinforced by political discussion with like-minded others. Our findings indeed suggest that participation in a WhatsApp group with very strong ties made participants feel more curious, happy, and excited compared with participation in a WhatsApp group with somewhat less strong ties. We did not find an effect for negative emotions and feelings. An explanation for this might be that the psychological discomfort of political discussion (e.g., through disagreement) may activate negative feelings, such as anxiety, fear, and/or anger within heterogeneous social networks (Mutz 2006). In our study, we have particularly facilitated discussions among groups of strong ties, possibly resulting in less heterogeneous groups. Future work should further elaborate on the role of heterogeneous viewpoints in interpersonal political discussion on instant messaging apps.
We also examined the importance of interpersonal political discussion of news and politics in the process of political learning. In line with recent work by Valenzuela et al. (2019), our findings indicate that political discussion on WhatsApp helps increase issue-specific knowledge. Interpersonal political discussion helps people to think about political and current events, relate personal experiences to politics, and reconsider issue stances (i.e., the differential gains model; Scheufele 2002). In addition, we have extended the focus on cognition relating to political discussion to bring us closer to understanding the relationship between discussion and issue-specific knowledge. The self-generated elaboration, operationalized as the number of words contributed to the WhatsApp group conversation, was positively associated with issue-specific knowledge. This is in line with the so-called discussion-generated elaboration explanation (Eveland 2004), which suggests that “the act of engaging in discussion forces meaningful information processing and thus increases learning due to an influence on information processing during discussion” (p. 180). In this way, interpersonal political discussion in instant messaging apps could provide an opportunity for repetition and rehearsal of information from memory, enable to consolidate the corresponding information in long-term memory and, eventually, fostering attitude-building as well as knowledge-building processes. Another explanation for the relationships between interpersonal political discussion and issue-specific knowledge might be the so-called anticipatory elaboration explanation (Eveland 2004): “the expectation of an impending discussion is an internal motivation that then increases cognitive elaboration on news content” (p. 180). Respondents in the experimental condition might have expected to engage in discussion of a political news item and therefore they invested more heavily in processing the information upon first being exposed to it because they want to be prepared to engage in later discussion of this information (Eveland 2004). Recently, Valenzuela et al. (2019) already found that WhatsApp use for news could exert a significant influence on political learning, as well as on protesting and other expressive forms of political participation—especially over the course of the election campaign. It is of great importance to further explore the role of WhatsApp in major political events.
With respect to the other effects, the results show, however, no significant effects for behavioral responses to interpersonal political discussion. There was no support for the hypotheses predicting that interpersonal political discussion on WhatsApp would have a positive effect on political interest and behavior (e.g., seeking additional issue-specific information, sharing or discussing a news item on WhatsApp). A possible explanation for this finding, based on models of social influence, is that the WhatsApp groups in our study consisted of respondents with mostly agreeable viewpoints. In this way, respondents were not necessarily exposed to a variety of different political viewpoints and arguments—limiting the motivation to develop more informed views and advance their own political opinions, and thereby their interest in politics (Torcal and Maldonado 2014). In addition, respondents were less likely to, for example, seek additional information as they did not have to counter opposing viewpoints to strengthen their own arguments (Lyons et al. 2016). Or they might simply not have additional resources to urge others to engage in political behavior. Future research, of course, needs to confirm this possibility.
Setting up this research design involved one key limitation. As we conducted a field experiment, we tried to stage the WhatsApp group conversations in a natural realistic manner. As political discussion takes place with family and friends rather than acquaintances (Eveland and Schmitt 2015), we created groups of strong ties. This simulated a real-world scenario. However, for data collection purposes and to monitor the experimental conditions, the researcher was also part of the WhatsApp group conversation (i.e., not involved in the discussion), and consequently, respondents could respond differently in a real world and private conversation with family and friends. As such, respondents who would normally avoid or merely read online political discussions could have given very different responses from naturally active discussants. Dahlberg (2006) however argues that infrequent posters on online political websites are not necessarily unengaged or nonparticipating. They may be as committed and reflexive as are frequent posters. While it would be interesting to study the role of certain individual characteristics (i.e., political interest, but also agreeableness, extraversion; Gerber et al. 2012) in naturally occurring political discussion, it would be extremely difficult to achieve this goal as it would require a study that is both ecologically valid and ethical. Alternatively, future research designs could attempt to obtain insights in actual news content shared on WhatsApp.
Despite this limitation, our study clearly advances existing research on this topic. Possibly the greatest advantage of this study was our ability to grasp better how we can link discussion patterns to learning outcomes. In doing so, the study has several important implications. As adolescents tend to be less involved in politics and current events compared with older adults (see, e.g., Neundorf et al. 2013), we explore how instant messaging apps could serve as a resource for engaging adolescents with politics and current events. Besides, we offer an empirical exploration of the democratic implications of instant messaging apps. Broadening this perspective, and to reiterate a point already made, recent major political events, such as the 2018 presidential elections in Brazil (Baptista et al. 2019), have already raised serious concerns about WhatsApp—playing a crucial role in the spread of misinformation. While this study adds to a still quite limited number of studies investigating the causal effects of discussing political news on instant messaging apps, more research is needed to fully understand the risks and opportunities of such platforms. This holds particularly true for the affective and cognitive responses to political discussion. Weeks (2015) has already indicated that emotions (e.g., anger, anxiety) play an important role in how citizens respond to uncorrected political misinformation. Besides, Valenzuela et al. (2019) found that knowledgeable citizens may nevertheless spread misinformation as a way to justify their own attitudes. In this way, political knowledge is insufficient to prevent misinformation spreading. Future work should elaborate on whether, and if so, how political discussion on WhatsApp and other instant messaging apps increases citizens’ ability to detect dis- and misinformation.
Furthermore, there is a need for further research on the potential for instant messaging apps to polarize political discussions. Instant messaging apps may knit people’s interpersonal networks close together (see, e.g., Valkenburg and Peter 2009), and foster interpersonal (political) discussion (Valeriani and Vaccari 2018), but on the other hand constrain possibilities for linking communities to wider spheres (Swart et al. 2018b). In this way, instant messaging apps might limit the diversity of news sources and political opinions that users are exposed to, raising questions about the democratic value of connecting through such apps. It would then be worthwhile to explore the content specific features of news content shared on WhatsApp, and the role of different news outlets, particularly if they are known for their different political leanings (e.g., their conservative or liberal views).
All in all, this study has provided a strong set of findings, relevant to the social and political implications of instant messaging apps. By conducting a field experiment using WhatsApp, these findings not only update and advance earlier research about interpersonal political discussion, but also provide a further understanding of the role of instant messaging apps in our current media ecosystem.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
ORCID iDs
Susan A. M. Vermeer https://orcid.org/0000-0002-9829-8057 Damian Trilling 
Supplemental Material
Supplemental material for this article is available online.