Voting and civic engagement: Results from an online field experiment

Abstract

How does voting interact with civic engagement outside the electoral process? An online field experiment on more than 140,000 registered voters in San Francisco yielded two main results. Subjects who voted in the 2016 primary elections were nearly three times more likely to open a survey from a nonprofit organization than those who did not vote in the primary election. However, explicitly priming voter identity and gratitude made all subjects far less likely to engage in this form of civic participation.

Introduction

Citizens have two main types of actions in their political toolkits: electoral and non-electoral. Recent work at the macro-level has observed substitution between these two categories over time (Dalton, 2008), while individual-level analysis has suggested a positive correlation (Oser, 2016). However, the causal relationship between these two forms of participation remains unclear. Does engaging in one type cause more or less of the other type? As a practical matter, can organizations leverage prior activity in one category to induce the other?

Nonprofit organizations provide a useful context for studying causal relationships within the political toolkit (Han, 2016). These organizations can facilitate electoral participation through voter mobilization efforts (LeRoux and Krawczyk, 2014). In addition, nonprofits often experiment with different kinds of appeals to induce a wide range of non-electoral actions (Han, 2009). Furthermore, some of the most effective appeals establish a relational context by linking current requests to prior behavior (Han, 2016).

This study leverages a sample of more than 140,000 registered voters to better understand who participates in modern forms of civic engagement and what kinds of appeals are most effective. The results of an online field experiment yielded two main results. Subjects who voted in the 2016 primary elections were nearly three times more likely to engage with a survey distributed by a civic organization than those who did not vote in the primary election. However, explicitly priming voter identity and gratitude made all subjects far less likely to engage in this form of civic participation. These results may offer some practical wisdom to nonprofit organizations and improve our understanding of how voters manage their political toolkits.

Experimental design

A field experiment was conducted in collaboration with a pro-housing civic organization in San Francisco, California. The organization designed a survey to measure public opinion about issues affecting the community and wanted to maximize participation. The sample for the experiment included 140,189 registered voters who provided their email address during the registration process. This design did not control for information spillovers between multiple units in the same household (Coppock, 2014), although interference should bias against significant treatment effects.

The survey was distributed via email using a fully crossed 2 × 2 between-subjects factorial design. The voter identity manipulation occurred in the subject line of the email: “SF [voter/resident], what are your priorities for the city?” This manipulation was inspired by prior research on the effects of priming voter identity (Bryan et al., 2011), although it does not rely on subtle linguistic cues (Gerber et al., 2016). The voter gratitude manipulation occurred in the first line in the body of the email: treated subjects were thanked for being a registered voter in San Francisco (Panagopoulos, 2011), while subjects in the control condition were again asked what was important to them. Crucially, this control condition did not include any additional language that was not already contained by the treatment condition.

The outcome variable measured whether the subject clicked the link embedded in the email to open the survey. Since survey responses were anonymous, it was not possible to match this click rate with survey completion rates. However, it is expected that treatment assignment had no effect on the probability of completing the survey given that the subject had already clicked to open the survey.

Results

The voter file contained information about subjects’ party identification, age, and whether they voted in the 2016 primary election that occurred two months prior to the experiment. ¹ Figure 1 explores which of these characteristics were associated with higher click rates. The most striking result is that subjects who voted in the primary election were nearly three times as likely to open the survey than non-voters. This result may reflect the fact that participatory citizens tend to embrace both non-institutionalized and electoral-oriented activities (Oser, 2016), or that voters are more likely to contribute to public goods (Bolsen et al., 2014).

Figure 1.

Predictors of online civic engagement. Each line represents the partial correlation between a particular demographic factor and the likelihood of opening the survey, along with 95% confidence intervals. These point estimates were derived from a logistic regression of the outcome variable on all covariates and condition fixed effects.

Figure 2 reports the results associated with the experimental manipulations. Surprisingly, subjects in all three treatment conditions were about 40% less likely to open the survey than subjects in the control condition (p = 0.000). ² This result is even more remarkable given the relatively low baseline, as floor effects should have biased against finding significant treatment effects. Floor effects may, however, explain why the identity and gratitude manipulations did not appear to have an additive effect on click rates.

Figure 2.

Treatment effects on online civic engagement. Each bar represents the proportion of subjects in each condition who clicked on the survey link, along with 95% confidence intervals.

What explains this decrease in civic engagement? Since the manipulations conflated voter identity and gratitude with politics more generally, one explanation could be that subjects are simply averse to politics. This mechanism seems unlikely since all subjects are registered voters who volunteered their email addresses during the registration process. Furthermore, if the main effect was driven by an aversion to politics, then this effect should be smaller for subjects with higher levels of interest in politics. Instead, treatment effects were significant and just as large for subjects who voted in the primary election as those who did not vote. In fact, the voter identity treatment effect was slightly larger for primary voters.

A separate but related mechanism is voter fatigue. The experiment was conducted three months before a general election in a state that include several propositions on the ballot. Therefore, treated subjects may have exhausted their capacity to engage in electoral politics and potentially misconstrued the email as one about the general election. This mechanism is consistent with the larger treatment effect observed among primary voters.

Alternatively, the priming of past voting behavior may have caused a boost in self-concept that subsequently enabled treated subjects to feel “licensed” to dismiss additional requests to take costly civic action (Merritt et al., 2010). This mechanism would also be consistent with larger treatment effects among primary voters, although licensing should apply only to the voter gratitude treatment. These data cannot fully distinguish between the voter fatigue and licensing mechanisms because self-concept was not measured directly.

Discussion

This paper explored whether voting influences civic engagement outside the electoral process. The results of an online field experiment on more than 140,000 registered voters in San Francisco yielded two main results. First, subjects who voted in the recent primary election were far more likely to open a survey from a nonprofit organization than those who did not vote in the primary election. However, at least one attempt to explicitly leverage prior voting behavior had negative unintended consequences. Priming voter identity and gratitude made subjects far less likely to engage in this form of non-electoral participation.

The results of this experiment may offer practical wisdom to nonprofit organizations looking to improve the effectiveness of their appeals. Of course, these data were generated by a unique sample of registered voters at a particular time in the electoral cycle. In addition, interacting with a survey distributed by a nonprofit organization is hardly representative of all forms of non-electoral civic engagement. Therefore, additional research is needed to evaluate the generalizability of these results. Future work should continue to explore the conditions under which citizens construe the various items in their political toolkits as complements or substitutes.