Does the election closeness message increase voter turnout? Evidence from a field experiment in Japan

Abstract

The calculus of voting indicates that individuals are more likely to vote when they believe their vote could affect the outcome. However, most field experiments examining the effect of perceived election closeness have found no significant effect. This study investigates the causal effect of communicating election closeness through a field experiment conducted in Ashiya, Japan. Japanese city council elections are an ideal setting for this analysis because they are often decided by narrow margins. The results of a field experiment conducted during the 2023 Ashiya City Council election demonstrate that conveying information about past close elections does not significantly increase voter turnout. A post-election survey conducted to explore the mechanisms underlying this result indicated that most voters paid little attention to the content of the postcards distributed during the campaign. These findings indicate that messages emphasizing electoral closeness are insufficient to increase voter turnout. More broadly, the positive association between electoral closeness and turnout found in observational studies may reflect candidates’ mobilization efforts rather than the voters’ perception of closeness. This study also suggests that field experiments using direct mail may underestimate causal effects because voters are likely to overlook the postcards they receive.

Introduction

Does the message that one’s own vote could determine the outcome of an election increase voter turnout? According to the calculus of voting theory developed by Riker and Ordeshook (1968), voters choose to vote when the expected utility of voting exceeds the cost of participation. Previous studies have emphasized civic duty and voting costs as key factors (e.g. Kostelka and Blais, 2021; Kostelka et al., 2023), given that the impact of a single vote on the election outcome is typically regarded as negligible. However, when electoral competition is intense, the probability that a single vote may alter the election outcome increases, regardless of the size of the district. This suggests that communicating the closeness of an election may have a significant impact on voter turnout. Indeed, empirical studies based on observational data have shown a robust positive association between the closeness of electoral competition at the district level and voter turnout (e.g. Arceneaux and Nickerson, 2009; Blais, 2000; Cancela and Geys, 2016; Geys, 2006; Wattenberg, 2002).

An unresolved issue in previous studies on the effect of perceived election closeness on voter turnout is that most experimental studies have failed to find a significant effect. Although observational studies have shown a positive association between electoral competitiveness and voter turnout, these studies face an inherent limitation: they cannot identify whether the observed correlation is attributable to voters’ perceptions of electoral closeness or to strategic mobilization efforts by candidates in competitive districts. Thus, recent studies have sought to estimate the causal effect of perceived election closeness on voter turnout through field experiments. However, the findings from these studies remain mixed. Some experimental studies have observed statistically significant increases in turnout (e.g. Biggers et al., 2024; Bursztyn et al., 2023), while most studies have found no significant effect (e.g. Bhatti et al., 2018; Enos and Fowler, 2014; Gerber and Green, 2000; Gerber et al., 2020).

This study contributes to the existing literature by rigorously examining the causal effect of communicating electoral closeness on voter participation through a field experiment conducted in Ashiya, a municipality in Japan. Japanese municipal elections offer several advantages for estimating this effect. First, most municipal assembly elections in Japan are highly competitive (Horiuchi, 2005), as assembly members are elected under the single non-transferable vote system (SNTV) in districts with large magnitudes. In the case of Ashiya, there were two city assembly elections in the past decade in which the margin of victory was within ten votes, including one decided by a single vote. Second, despite high electoral competitiveness, this fact is not widely recognized among the electorate, as the Japanese mass media rarely highlight the outcomes of local assembly elections. In other words, experiments conducted in the context of Japanese municipal elections are less susceptible to concerns about pretreatment bias (Linos and Twist, 2018), as voters are generally unaware of how competitive these elections actually are. These points enhance the validity of the experimental findings.

The results of the field experiment conducted during the 2023 Ashiya City Council election indicate that messages emphasizing electoral closeness do not significantly increase voter turnout, even in the context of Japanese municipal elections, which are well suited for detecting such effects. To examine the causal effect of the message that one’s vote could determine the election outcome, I collaborated with the Ashiya City Election Management Committee to distribute postcards conveying the fact that two elections in Ashiya were decided by fewer than 10 votes in the past decade. This was part of an election awareness initiative during the 2023 Ashiya City Council election. This study analyzed both aggregate-level and individual-level data to estimate the causal effect of distributing postcards. However, in both analyses, the estimated values of the effects of the experimental intervention on turnout were statistically insignificant.

A further contribution of this study lies in its analysis of why the intervention had no significant effect, made possible by a follow-up mail survey conducted after the experiment. While previous studies indicate that communicating election closeness does not significantly increase voter turnout, the reasons for these null results often remain unclear. This follow-up survey, which included questions assessing respondents’ awareness of the postcards, helps to unpack the mechanism behind the experimental results. The follow-up survey results suggested that many voters failed to notice the postcards and that even among those who did, many could not remember the content. Accordingly, postcard distribution likely had a negligible effect on voter turnout. These findings highlight the need to carefully determine whether the null result was due to the ineffectiveness of the intervention itself or to the fact that most voters did not pay attention to it due to the methodological limitations of the intervention.

Literature review

Previous studies on the effect of perceived election closeness

Scholars have argued that communicating the importance of a single vote’s value to voters is a significant way to address declining voter turnout. Concerns that the tendency to abstain increases when people perceive their vote’s impact on election outcomes to be minimal were expressed as early as the 18th century. This impact is represented as the probability (P) term in the calculus of voting theory by Riker and Ordeshook (1968), positioned as an important factor influencing decisions regarding voter participation. This probability not only embodies a belief in one’s political efficacy but also varies depending on institutional environments; for example, Riker and Ordeshook (1968) formulate P as 1 divided by the number of eligible voters in the electoral district. Furthermore, previous studies have suggested that election closeness is also a significant determinant of P, as it reaches the maximum value (1) when the vote shares between candidates are completely equal.

Prior to the rise of experimental studies in political science fields, the primary approach to examining the effect of perceived election closeness on voter turnout was through observational studies. The argument that voters’ perception of their vote could change the election outcome encourages their participation is primarily based on findings from these observational studies. One of the most robust pieces of evidence from meta-analyses on voter turnout (Cancela and Geys, 2016; Geys, 2006) is a significant correlation between election closeness and turnout rates. As the competitiveness between candidates increases, the probability that one vote can influence the election outcome rises. Thus, researchers have considered the correlation between turnout rate and election closeness as evidence supporting the theoretical assumption that a high estimation of one’s vote value is crucial for voter participation. ¹

In contrast to findings from observational studies, some experimental studies ² have demonstrated that informing voters about election closeness does not necessarily encourage their participation. Gerber and Green (2000) is one of the earliest studies to examine the causal effect of perceived election closeness on voter turnout through a large-scale field experiment in the United States. This study reported that conveying information on election closeness via door-to-door communication increased voter turnout by 12.1 percentage points compared to the control condition. However, it was also reported that such a significant effect was not detected when the information was conveyed via phone calls and direct mail. Bennion (2005) examined the effect of perceived election closeness through a field experiment and found no significant difference in voter turnout between the treatment group and a group that received a message on civic duty. Enos and Fowler (2014) focused on the result of the Massachusetts House of Representatives 6th Worcester District election in 2010, which resulted in a tie. From the results of a field experiment conducted during the subsequent special election, their study concluded that information about election closeness did not encourage voter participation on average. Gerber et al. (2020) also suggest that the effect size of perceived election closeness on the probability of voter participation is less than 0.1 percentage points, based on both survey and field experiments.

The inconsistency between observational and experimental studies is not particularly surprising. Whether at the individual- or the aggregate-level analysis, various observable and unobservable confounding factors influence the correlations observed in observational studies. For example, the difference in the mobilization efforts of candidates in competitive versus non-competitive districts is one of these confounding factors. Yeandle (2024) demonstrates, from a natural experiment using movers, that the experience of mobilization by political actors increases in more competitive districts. If the correlation between election closeness and voter turnout observed in many previous studies is driven by confounders, the causal effect of perceived election closeness on voter turnout will either be statistically non-significant or inevitably small, even if significant.

A recent study emphasized issues related to the experimental design. Biggers et al. (2024) point out that the ambiguity of experimental design in previous studies has obscured the causal effect of perceived election closeness, as these messages influence factors beyond just the perception of electoral competitiveness, such as the perceived importance of the election itself or beliefs about the candidates. To address this issue, the study conducted a novel field experiment across seven US states, specifically designed to isolate and test the causal effect of election closeness. The participants were randomly assigned to one of the four groups (placebo, election reminder, non-competitive message, and competitive message). The analysis of mean differences in turnout rates between these groups revealed that the group informed of a relatively small margin (350 votes) had a 1.6 percentage point higher turnout compared to that informed of a relatively large margin (2500 votes). Moreover, when the difference in turnout rates between these groups was estimated more rigorously, the gap increased to approximately 5 percentage points. In addition, the study reported that the mean difference in turnout rates between the placebo group and the group informed of the small margin was approximately 3 percentage points.

Limitations

Field experiments have the advantage of addressing not only omitted variable bias and endogeneity but also producing results with high external validity, as they are conducted in real-world settings. They are widely regarded as the “gold standard” for testing causal relationships; thus, prior studies have employed them to examine the causal effect of perceived election closeness on voter turnout. However, these studies still face unresolved issues that have yet to be fully addressed. Addressing these challenges will further advance our understanding of the impact of communicating the value of one’s vote.

The first issue is that the design of the messages used in previous field experiments has not been sufficient to accurately examine the impact of communicating election closeness on voter turnout. Biggers et al. (2024) used the figure of “350 votes” as an experimental stimulus to make voters believe that their vote could potentially change the election outcome. However, the theoretical basis for setting this 350-vote margin as the experimental treatment remains unclear; thus, there are doubts about its effectiveness. Theoretically, voters can be certain that their vote will change the election outcome only when the candidates have the same number of votes. From this perspective, the 350-vote margin is too large for voters to believe that their vote could alter the election outcome. Regardless of whether the margin is 2500 or 350 votes, the likelihood that a single vote will change the outcome is substantially low. Nevertheless, this study demonstrates that this treatment has a statistically significant effect on increasing voter turnout compared to other messages. This suggests that participants may have interpreted the “350-vote margin” as an experimental stimulus that is distinct from enhancing the perceived efficacy of their vote.

Focusing on Japanese municipal council elections has several advantages for addressing these issues. General municipal assembly elections in Japan often result in very close outcomes because of the institutional characteristics of the SNTV system. Japanese municipal council elections, including those in Ashiya, are ideal settings for rigorously testing the causal relationship between election closeness and voter turnout. In addition, a notable feature of Japanese municipal council elections is that many voters have little awareness of the election outcomes. The results of general municipal assembly elections in Japan usually do not attract much attention from voters, unlike national or mayoral elections, as they are rarely reported in detail by the mass media. ³ This suggests that many voters may be unaware that the outcomes of municipal council elections are often determined by only a few votes. In other words, pretreatment bias is unlikely to be a concern in the case of general municipal council elections in Japan. This characteristic also represents an advantage of using the Japanese case.

The second issue is the lack of analysis regarding the mechanisms through which experimental results are generated. Many studies have shown that informing voters about election closeness does not significantly increase voter turnout. However, there has been insufficient discussion about why such null results occur. As noted above, Biggers et al. (2024) point out certain ambiguities in the experimental designs of existing studies; however, this does not explain the reasons underlying the null results. This study analyzes the mechanisms behind the experimental outcomes by conducting a follow-up mail survey after the field experiment. This enables more rigorous identification of whether the absence of significant effects is attributable to the intervention itself or to flaws in the experimental design.

The third issue is the lack of examination of generalizability in non-US contexts. As Bhatti et al. (2018) point out, most field experimental studies on voter turnout have been conducted in the United States. This US bias exists primarily because of the ease of accessing public voter records for academic research. However, this bias also creates a problem in that the transportability of the experimental results remains unclear. This study contributes to examining the generalizability of findings from prior experimental studies to other contexts. In general, election committees in Japanese municipalities are reluctant to disclose individual-level voter records to researchers because of privacy concerns. However, fortunately, the Ashiya City Election Management Committee permitted us to access individual voter records and use them for academic research. ⁴ This enabled us to conduct the first field experiment in Japan using individual-level voter records to examine the causal effect of communicating election closeness on voter turnout in a non-US context.

Experimental design

Data

This study examines the causal effect of election closeness on voter turnout through both aggregate- and individual-level analyses. As explained in Online Supplemental Appendix A.2, random allocation in this experiment was conducted at the town level, rather than the individual level. Due to this town-level allocation, the study combines analyses at both town and individual levels to examine the effect of perceived election closeness.

To obtain individual-level information on voter participation records and calculate turnout rates by towns, ⁵ I obtained permission from the Ashiya City Election Management Committee to access the voting records of the 2019 Ashiya City Council election and the 2022 Upper House election prior to the 2023 Ashiya City Council election. These data were used to explore pairs of towns with similar voter turnout rates in the 2023 Ashiya City Council election.

While electoral systems vary across Japanese local governments, municipal assembly elections are generally held under an SNTV system, and the Ashiya City Council election is no exception. In general, local assemblies in Japan lack the authority to formulate or propose budgets; thus, media attention tends to concentrate on mayors. However, because local assemblies are able to reject proposed budgets, electoral outcomes have important implications for a municipality’s policy direction. In fact, the Ashiya City Council has repeatedly rejected proposals concerning urban redevelopment projects around Ashiya Station since 2020. In this context, the turnout in the 2023 Ashiya City Council election was 55.1%, exceeding the level observed in the 2019 election (48.7%).

Although using data that included all participation records would be ideal, it was not feasible to manually transcribe the information of nearly 80,000 individuals recorded in thick booklets by hand within a short period. ⁶ Thus, for the individual-level analysis, the author randomly selected 1500 individuals from the voting records of the 2023 Ashiya City Council election and created an individual-level dataset including the following: voting participation status (election day voting/early voting/absentee voting/abstention) in the 2023 Ashiya City Council election, ⁷ and past voting experience in the 2023 Hyogo Prefectural Assembly election.

Randomization

This study employed matched-pair randomization to enhance the efficiency of average treatment effect estimation. This method provides the highest estimation efficiency of the average treatment effect among various randomization techniques (Bai, 2022). In this study, 25 pairs of towns with similar predicted turnout rates for the 2023 Ashiya City Council election were constructed using turnout data from both the 2019 Ashiya City Council election and the 2022 Upper House election. ⁸ One town in each pair was randomly assigned to the treatment and the other to the placebo group. ⁹

The procedure to create these 25 pairs of towns was as follows: First, 50 towns were randomly selected ¹⁰ from the 60 comprising Ashiya City and 25 pairs were randomly created. Second, the absolute differences in turnout rates of the 2019 Ashiya City Council election and the 2022 Upper House election were calculated for each pair, and their sum was calculated. Third, this process was repeated 10 million times, and the set of pairs ¹¹ with the smallest total differences was selected.

Experimental stimuli

The postcards serving as experimental stimuli were distributed to residents of towns assigned to either the treatment or the placebo condition, with one postcard sent per household using Town Plus, a postcard distribution service provided by Japan Post. Although somewhat more expensive compared to postcard delivery services offered by other private companies, ¹² Town Plus has an advantage of being able to deliver postcards to almost all households in the allocated towns, including those in high-security apartments. Furthermore, Town Plus allows distribution to targeted households without requiring the exact names and addresses of the residents.

For the field experiment, two types of postcards were created. One was distributed to the household under the treatment condition, and the other was distributed to the household under the placebo condition. The following information was included in both types of postcards: (1) the voting day for the Ashiya mayoral and city council election was 23 April, (2) the early voting period was from 17 to 22 April, and (3) information about early voting locations. In addition, the treatment postcard included a quiz on past electoral outcomes in Ashiya; options and answers with explanatory notes were presented on the postcard. However, the postcard acting as a placebo included a picture of a ballot box instead of this information.

The postcard distributed to the households in the treatment condition contained a quiz about the frequency of close election outcomes in Ashiya’s mayoral and city council elections since 2000. Specifically, the question posed was as follows: “Since 2000, how many times has the margin between the elected candidate and the runner-up in Ashiya City mayoral or city council elections been within 10 votes?” The following two options were provided: (1) Given Ashiya City’s electorate of 80,000, such narrow margins must have occurred 0 times; or (2) Could there have been two or three instances where the outcome was determined by fewer than 10 votes? The correct answer (2) was indicated below the options, accompanied by supplementary information. The actual images of the postcards are shown in the Online Supplemental Appendix A.1.

Estimation

Town-level analysis

The formula for the average treatment effect ( $\widehat{\tau }$ ) of communicating election closeness information in the aggregate-level data analysis is estimated as follows

$$\widehat{\tau }=\frac{1}{N}\sum _{j=1}^N\left(Turnou{t}_j^{Treat}-Turnou{t}_j^{Placebo}\right).$$

(1)

where $Turnout{T}_j^{Treat}$ is the turnout rate for town assigned treatment group in pair $j$ , and $Turnout{T}_j^{Placebo}$ is the turnout rate for town assigned placebo group in pair $j$ . Here, $j$ indexes matched pairs; each $j$ represents a single matched pair comprising one treated unit and one placebo unit. $N$ represents the total number of matched pairs. In addition, based on the Neyman variance estimator (Fukumoto, 2022), the standard error of the effect ( $\widehat{SE}\left(\widehat{\tau }\right)$ ) is estimated as follows

$$\widehat{SE}\left(\widehat{\tau }\right)=\sqrt{\frac{1}{N(N-1)}\sum _{j=1}^J{\left(\left(Turnou{t}_j^{Treat}-Turnou{t}_j^{Placebo}\right)-\widehat{\tau }\right)}^2.}$$

(2)

Individual-level analysis

Previous studies have suggested that the causal effect of the pivotal information tends to be larger for individuals with a weaker propensity to vote compared to those with a stronger propensity (Bhatti et al., 2018; Enos and Fowler, 2014). However, the aggregate-level analysis was limited by the small sample size, which made it difficult to examine how the effect varies according to individuals’ prior voting behavior. Thus, to clarify this relationship with past voting behavior, this study conducted an analysis using individual-level data. I estimated the interaction effect between the experimental intervention and past voting behavior using a linear probability model, ¹³ specified as follows

$$Vot{e}_{ij}=\alpha +{\beta }_{1}Trea{t}_{ij}+{\beta }_{2}Votepre{f}_{ij}+{\beta }_{3}Trea{t}_{ij}\times Votepre{f}_{ij}+{\mathit{Z}}_{ij}\gamma +{\epsilon }_{ij}.$$

(3)

The dependent variable $Vot{e}_{ij}$ is a binary variable equal to 1 if individual $i$ in town $j$ voted in the 2023 Ashiya City Council election, and 0 otherwise. $Trea{t}_{ij}$ is a treatment indicator equal to 1 if individual $i$ is assigned to the treatment condition and 0 if individual $i$ is assigned to the placebo condition. ${\beta }_1$ represents the average treatment effect conditional on the included covariates. $Votepre{f}_{ij}$ is a binary variable indicating whether individual $i$ voted in the 2023 Hyogo Prefectural Assembly election, held 10 days before the Ashiya City Council election, with ${\beta }_2$ denoting the association between prior turnout and subsequent participation (1 = voted, 0 = abstained). $Trea{t}_{ij}\times Votepre{f}_{ij}$ is an interaction variable to identify heterogeneous treatment effects by prior voting behavior, and ${\beta }_3$ represents the differential effect for individuals with a prior voting record. Finally, $Z_{ij}$ denotes a vector of additional control variables, $\gamma$ is the corresponding coefficient vector, and ${\epsilon }_{ij}$ is an error term.

Results

Town-level analysis

Since the postcards designed to communicate electoral closeness were assigned at the town level, I began by examining the results based on town-level data. Figure 1 presents the estimated average treatment effect (ATE) for turnout using the aggregate-level data. The circles indicate the point estimates of the ATE, and the horizontal bars represent the corresponding 95% and 99% confidence intervals. The vertical dashed line indicates a null effect (zero). When the confidence interval crosses this line, the estimated effect is not statistically significant at the corresponding level.

Figure 1.

Results of the town-level data analysis.

This figure shows that communicating electoral closeness does not significantly affect voter turnout. The point estimate of the ATE is near zero ( $\widehat{\tau }$  = −0.057, $\widehat{SE}$  = 0.670), and its 95% and 99% confidence intervals overlap with the dashed line at zero. While the matched-pair design substantially improved estimation efficiency, this study still failed to detect a statistically significant effect of communicating electoral closeness on voter turnout.

Individual-level analysis

To examine whether the treatment effect varied according to prior voting records, I estimated interaction models using individual-level data. The estimated results from the individual-level analyses are summarized in Table 1. This table presents the point estimates, cluster-robust standard errors, and p-values for the coefficients of the variables included in the statistical models.

Table 1.

Results of the individual-level data analysis.

	Coef.	Std. Err.	p
Treat	–0.0400	0.0348	0.2586
Sex	0.0141	0.0197	0.4803
Age	0.0023	0.0006	<0.001
Votepref	0.5281	0.0269	<0.001
Treat × Votepref	0.0240	0.0494	0.6298
N		1,273

The estimated result in Table 1 indicated that there is no significant difference in the effect of the experimental intervention, regardless of whether individuals voted in past elections. Examining the estimation results of the model including the interaction term, the coefficient of the interaction term is 0.024 (standard error = 0.049), and it is not statistically significant. This result indicates that the experimental treatment did not have a statistically significant effect on turnout, even among individuals who had not voted in previous local elections. Consistent with the aggregate-level analysis, the individual-level analysis also yielded a null causal effect of communicating election closeness on voter turnout.

Follow-up analysis

Data and methods

A consistent finding revealed through both aggregate and individual-level analyses is that the experimental intervention had little effect on increasing voting participation, even though this study was designed to detect the effect of perception of election closeness. To reveal the reason for the null effects, this study conducted further analyses using the follow-up survey.

The follow-up survey targeting 1500 individuals, who were selected for the individual-level analysis, was conducted from 25 June to 31 July 2024. The valid sample size was 703 (response rate: 46.9%). This survey included questions regarding respondents’ perceptions of the postcard and their awareness of close elections in Ashiya. Both aspects relate to the underlying mechanisms generating the experimental results.

First, this study analyzes the difference in postcard recognition percentage between individuals living in towns where the postcard was distributed (treatment and placebo groups) and those where it was not (non-distribution group). As mentioned above, the question regarding postcard recognition was included in the follow-up survey. However, interpreting the descriptive statistics of this question as a direct measure of postcard recognition is problematic, as it likely includes systematic errors that overestimate the recognition. To mitigate this bias, this study utilizes information on postcard perception from individuals residing in towns where the postcard was not distributed. By comparing postcard recognition rates between voters in towns with and without postcard distribution, this study is able to rigorously examine the degree of postcard recognition. If there is little difference in recognition between these three groups, it would suggest that many voters either did not see the distributed postcard or paid little attention to its content if they did encounter it.

Second, this study analyzes the extent of knowledge about past local election outcomes in Ashiya among three groups: the treatment, placebo, and no-distribution groups. It was assumed that most Ashiya citizens are unaware of local election outcomes, given that mass media rarely report on these results. Thus, the researcher predicted that individuals living in towns where the postcard (as treatment stimulus) was not distributed would be largely unaware that the Ashiya City Council election, among others, was decided by a margin of less than 10 votes. In addition, if the analysis of postcard recognition shows a significant difference between groups, the differences in knowledge about election outcomes would also vary between groups. Conversely, if there were no significant differences in knowledge about election outcomes between groups, it would suggest that many voters did not pay attention to the content of the postcard, even when it was distributed.

Results of follow-up survey analysis

The follow-up mail survey conducted after the field experiment included the following question to assess the recognition of the postcard:

During the 2023 city council and mayoral elections in Ashiya, Ashiya City distributed postcards as a part of an election awareness initiative with academic researchers. Did you recognize this postcard that was distributed as a part of the election awareness initiative?

The response options of this question are as follows: “1: I recognize it and remember the contents of the postcard,” “2: I have a slight memory of it but do not remember the contents,” “3: I am not directly aware of it but have heard about it from someone,” and “4: I am completely unaware of it, and have no memory of seeing or hearing about it.” In the following analysis, the proportion of respondents who chose option 1 was used as the recognition rate of the postcard.

In this field experiment, the postcard was distributed to nearly all households in the towns assigned to the treatment and the placebo groups; in contrast, no such postcard was distributed to households in the control group. If it is assumed that respondents saw and remembered the content of the postcard, the theoretical recognition rate would be 1 for those in the treatment and the placebo groups and 0 for those in the control group. Figure 2 shows the results of actual postcard recognition rates across the experimental groups. The recognition rates were 8.36%, 8.42%, and 4.26% in the treatment group (N = 311), the placebo group (N = 297), and the non-distribution group (N = 94), respectively. Although the recognition rates in the treatment and the placebo groups were relatively and significantly higher than those of the non-distribution group, the difference in these rates was approximately 4 percentage points. Moreover, the figure indicated that despite the widespread distribution of the postcard to all households in the treatment and the placebo groups, the vast majority of respondents were not aware of the postcards.

Figure 2.

The degree of postcard recognition among groups.

The mail survey also investigated whether there were significant differences in the level of knowledge regarding the outcomes of past Ashiya City Council elections. The following question related to past election outcomes in Ashiya was included in this survey: “In the past two decades, how many times do you think the number of votes between the winner and the runner-up in Ashiya City Council or Ashiya mayoral elections was within 10 votes?” The options are “1: 0 times,” “2: 1–2 times,” “3: 3 or more times,” and “4: I do not know.” As indicated on the postcard distributed to the households in the treatment group, the correct option is 2.

It could be assumed that if respondents had read the content of the postcard, the rate of choosing the correct option in the treatment group would be significantly higher than that in the other groups. However, the results of this survey revealed no significant differences in correct response rates between the groups (14.5% in the treatment group, 13.5% in the placebo group, and 12.8% in the non-distribution group). As shown in Figure 2, most respondents did not notice the postcards. In addition, this result suggests that it is highly likely that election closeness does not have a strong positive effect on voter turnout, even when it is communicated. Furthermore, the overall correct response rate among Ashiya residents was less than 20%. This indicates that the null results are attributable to Ashiya residents’ already being aware that the outcome of the city council election can be decided by just a few votes.

Conclusion

This study examined the causal effect of communicating election closeness on voter turnout through a field experiment. In Japanese municipal elections, it is common for outcomes to be decided by a small margin of votes; however, many voters are unaware of this fact. By leveraging this characteristic and improving the design of the postcard to inform election closeness, this study provided robust findings on the causal relationship between communicating election closeness and voter turnout. The findings from the field experiment demonstrated that there was no significant difference in voter turnout between the placebo and the treatment groups. Previous studies suggested that the effect would be relatively larger among individuals with weaker voting propensities; however, this study did not find such a tendency.

A key contribution of this study is the use of a post-experiment follow-up survey to clarify the mechanism underlying the null results. While many studies have estimated treatment effects via postcards or direct mail, voters do not always notice them, let alone carefully read them. Indeed, the results of the follow-up survey suggested that an overwhelming majority of recipients may have overlooked the postcards. This finding highlights a critical methodological limitation of direct mail distribution: it may underestimate the average causal effect size. Previous studies examining the effects of get-out-the-vote (GOTV) on voter turnout through the distribution of postcards tend to show that the effects are statistically insignificant or substantially small, even when statistically significant. However, findings that rely on flyers or postcards should be interpreted with caution, recognizing the potential for downward bias in the estimates.

Another contribution of this study is the finding that communicating election closeness may not increase voter turnout. It cannot be ruled out that a statistically significant treatment effect went undetected because an overwhelming number of voters did not notice the postcards or failed to remember their content. Despite this limitation, the follow-up survey showed that some voters noticed the postcards; nevertheless, both aggregate- and individual-level analyses yielded statistically insignificant treatment effects. More importantly, this study produced null results, even though Ashiya was selected as a suitable case for testing the causal effect of election closeness. These findings suggest that informing voters that an election is close is unlikely to increase turnout; this suggests that the correlation between electoral closeness and turnout found in observational studies could be interpreted as reflecting candidates’ mobilization efforts.

However, this does not necessarily imply that all field experiments using direct mail delivery produce statistically insignificant results. For example, when scholars examine the effects of social pressure messages with unusual content that may evoke strong discomfort, it is less likely that people will fail to notice a postcard or remember its content. In this sense, the extent to which postcard content is recognized depends on the nature of the message used in the treatment. Nevertheless, it is important to note that the postcard delivery method may still entail a risk of underestimating the causal effects. To ensure that the information is conveyed clearly and accurately, phone calls or face-to-face contact may be more appropriate than postcards.

Supplemental Material

sj-docx-1-psw-10.1177_14789299261445001 – Supplemental material for Does the election closeness message increase voter turnout? Evidence from a field experiment in Japan

Supplemental material, sj-docx-1-psw-10.1177_14789299261445001 for Does the election closeness message increase voter turnout? Evidence from a field experiment in Japan by Masahiro Zenkyo in Political Studies Review