What if neither randomized control trials nor public voting records are available in a get-out-the-vote field experiment?

Abstract

When American political scientists conduct get-out-the-vote (GOTV) field experiments, randomized control trials and public voting records are the two pillars of these experiments. However, what if neither is available in other countries? For instance, Japanese election commissions are averse to both. To tackle this problem, I propose a second-best solution. I collaborated with the election commission of a municipality in Japan. The election commission sent direct mail (DM) to all 18-year-olds in the municipality and disclosed voting records at the week-of-birth level. Finally, I estimated the treatment effect of the DM on voter turnout using a regression discontinuity design, where I compared the turnout of 18-year-olds with that of 19-year-olds or 17-year-olds. I cannot find any statistically significant evidence showing that the DM increased 18-year-old voters’ turnout. I hope that my setup enables GOTV field experiments in many countries outside the United States so that we can infer the causal effects of GOTV tactics in various contexts.

Introduction

When American political scientists conduct get-out-the-vote (GOTV) field experiments, randomized control trials (RCTs) and public voting records are the two pillars of these experiments (Green and Gerber 2019). However, what if neither is available in other countries? For instance, Japanese election commissions are averse to both. To them, treating some citizens but not others is unfair, and disclosing whether a citizen votes violates his or her privacy. In many countries, vote validation is impossible for legal, administrative, and economic reasons (Achen and Blais 2016, 206, Selb and Munzert 2013, 189).

Here is my second-best solution. I collaborated with the election commission of a municipality in Japan. The election commission sent direct mail (DM) to all 18-year-olds in the municipality and disclosed voting records at the week-of-birth level. Finally, I estimated the treatment effect of the DM on voter turnout using a regression discontinuity design (RDD), where I compared the turnout of 18-year-olds with that of 19-year-olds or 17-year-olds. The election commission was fine with this experiment because boosting the youngest voters’ turnout was already an important goal and aggregating voting records at the week-of-birth level protected citizens’ privacy.

Increasing youth voter turnout is critical to the democracy of today and the future (Holbein and Hillygus 2020). The younger citizens are, the less likely they are to vote, and this is true across countries and over time (Fieldhouse et al., 2007; Highton and Wolfinger 2001; Wolfinger and Rosenstone 1980). Since voting is habitual (Gerber et al., 2003; Plutzer 2002), a high turnout of teenage voters leaves a “footprint” such that the cohort tends to go to polling stations for decades to come (Franklin 2004).

Design

In the few weeks leading up to the House of Councilors election on July 10, 2022, I conducted an experiment in Toshima City, Tokyo Metropolitan Area, in collaboration with the municipality election commission. ¹ The subjects of the experiment were all citizens who were eligible (and thus automatically registered) in the city as of the election day and were born between June 3, 2002 and July 11, 2004 (inclusive, N = 3, 319). The treated group was composed of citizens who were 18 years old as of June 1, 2022 (born between June 3, 2003, and June 2, 2004, N _T = 1, 417). ² The minimum age for voter eligibility is 18. There are two control groups. Control Group 1 was composed of citizens who were 19 years old as of June 1, 2022 (born between June 3, 2002, and June 2, 2003, N_C1 = 1, 753). Control Group 2 was composed of citizens who were 17 years old as of June 1, 2022, but turned 18 by July 10 (born between June 3 and July 11, 2004, N_C2 = 149).

As the treatment, the election commission sent three pieces of GOTV DM (postcards) to every citizen in the treated group; however, no action was taken for the control groups. The first, second, and third pieces of DM were sent out 19, 11, and 5 days before election day, respectively. ³ Early voting began 17 days before election day. These postcards reminded citizens of the election, the dates to vote, and the number of places to vote. In addition, they asked recipients to record the date (and time) when they planned to vote (1st and 3rd DM cards) and/or the place where they planned to vote (2nd and 3rd DM cards). Scholars have established that more DM leads to higher voter turnout (up to four pieces of DM), that commitment devices work, and that mail sent by a voter registrar is effective (Green and Gerber 2019, 72–73).

After the election, the election commission disclosed citizens’ voting records aggregated at the week-of-birth level. The unit of observation is the week of birth w. I normalize w so that Week 0 corresponds to the week of June 3 to 9, 2003. Accordingly, the data cover Week −52 to Week 56. For each week of birth w, I obtain the number of eligible citizens (N _w ) and the number of voters (V _w ).

To conduct RDD analyses, I expand the disclosed data to make the dataset for analysis. The unit of analysis is the citizen i. For Week w, I make a subdataset composed of V _w units where the running variable is X _i = w + 0.5 and the outcome variable is Y _i = 1 and N _w − V _w units where X _i = w + 0.5 and Y _i = 0. I repeat this procedure for all weeks and then combine all subdatasets to obtain the main dataset.

I have two cutoff points (K). In the case of Cutoff 1, K₁ ≡ 0, I use Weeks −52 to 51 and compare 18-year-olds $\left(K_1<X_i<{\overline{X}}_1\equiv 52\right)$ with 19-year-olds $({\underset{¯}{X}}_1\equiv -52<X_i<K_1)$ . In the case of Cutoff 2, K₂ ≡ 52, I use Weeks 0 to 56 and compare 18-year-olds $({\underset{¯}{X}}_2\equiv 0<X_i<K_2)$ with 17-year-olds $\left(K_2<X_i<{\overline{X}}_2\equiv 57\right)$ .

Political scientists have long exploited the date of birth as a running variable for RDD in studying youth voter turnout (Cepaluni and Hidalgo 2016; Holbein and Hillygus 2016; Meredith 2009). ⁴ It is highly unlikely that the date of birth was manipulated to impact the treatment of babies 19 years later. Therefore, my setup satisfies the assumptions for RDD. ⁵

Result

Figure 1 presents the RD plots for Cutoffs 1 (left panel) and 2 (right panel). ⁶ In each panel, the horizontal axis indicates week of birth, the vertical axis represents voter turnout, the vertical line corresponds to the cutoff point, each dot shows the voter turnout corresponding to each week of birth, and the curves express global polynomial fits. In each panel, unexpectedly, the treated group (the right side of the left panel and the left side of the right panel) seem to have lower voter turnout around the cutoff than the corresponding control group.

Figure 1.

RD Plot. (a) Cutoff 1: 19-year-olds vs. 18-year-olds. (b) Cutoff 2: 18-year-olds vs. 17-year-olds.

To formally determine whether the impression is retained, I conduct a local linear regression to estimate the treatment effect at each cutoff point. ⁷ For Cutoffs 1 and 2, the conventional local-polynomial RD estimates are −0.15 and −0.08, while the robust bias-corrected 95% confidence intervals are (−0.35, 0.04) and (−0.42, 0.19) (the p-values are 0.11 and 0.47), respectively. Neither estimate is significantly different from zero at the 5% level. ⁸

While the present analyses are based on the continuity-based approach, I conduct the local randomization approach in Supplementary Material (SM). In essence, the implication remains the same; the treatment, if ever, decreased voter turnout, although nonsignificantly. In the SM, I also present robustness checks that support the finding, along with evidence implying that my results probably do not stem from the recipients being too tired of the repetitive GOTV messages.

Conclusion

The essential ingredients of GOTV field experiments, RCTs and public voting records, are not available in some countries, such as Japan. As a remedy, I propose a new second-best experimental design: treating the youngest citizens, obtaining voter turnout at the week of birth level, and estimating the treatment effect by RDD. I implemented the design in collaboration with the Election Commission of Toshima City, where we sent the three postcards in a row within two weeks before the 2022 Japanese upper house election.

I cannot find any statistically significant evidence showing that my three pieces of DM increased 18-year-old voters’ turnout. In fact, it is more likely that the DM decreased the voter turnout, although nonsignificantly.

I hope that my setup enables GOTV field experiments in many countries outside the United States so that we can infer the causal effects of GOTV tactics in various contexts.

Supplemental Material

Supplemental Material - What if neither randomized control trials nor public voting records are available in a get-out-the-vote field experiment?

Supplemental Material for What if neither randomized control trials nor public voting records are available in a get-out-the-vote field experiment? by Kentaro Fukumoto in Research & Politics.