Abstract
Civic organisations and progressive campaigns regard digital advertising as an essential method to register to vote low-participation groups, such as ethnic minorities, young voters and frequent home movers like private-sector tenants. Digital strategies appear to be promising in countries like the UK, where the registration process can be completed online, usually in less than 5 minutes, using a web link in the advert. But are typical digital campaigns effective in registering voters? To find out, we provide evidence from three randomised controlled trials: two conducted with advocacy organisations and the third run by the research team, carried out in two types of UK elections (general and local) and assigned either at the aggregate (Study 1 and Study 2) or individual (Study 3) level. Despite wide reach and relatively high rates of engagement, we find that the digital ad campaigns trialed across three studies did not affect under-registered groups’ voter registrations. These null findings raise questions about commonly-used digital advertising strategies to register marginalised groups. They are consistent with other studies that report either null or minimal effects of digital ads on other types of political behaviour.
Introduction
Voter registration and turnout among ethnic-minority citizens, young people and frequent residential movers, like private-sector renters, remain low (Fieldhouse et al., 2021b). These groups of citizens are therefore obvious targets for non-partisan campaigns that aim to increase electoral participation. They are also targeted by progressive campaigns who want to change the composition of the electorate in their favour (Broockman and Kalla, 2020; Foos and John, 2018). Hence, how can non-partisan and partisan campaigns effectively register people who are difficult to contact with conventional voter registration methods, such as door-to-door canvassing and direct mail? Social media ads appear to be promising because most people are active on social media. Moreover, in many jurisdictions, such as the United Kingdom and forty US-States, the registration process may be completed online, often taking less than 5 min. Since the act of registration does not require as much time and effort as voting (which usually happens offline), digital campaigns could be more effective at voter registration than directly mobilising registered voters to turn out at the polls. However, the research record so far has not been promising. Some studies show that social media advertising campaigns produce relatively small positive effects on voter turnout (Bond et al., 2012); they find that treatment effects are conditional (Haenschen and Jennings, 2019) or, more frequently, null (Aggarwal et al., 2023; Coppock et al., 2022; Haenschen, 2022). There is also mixed evidence on whether digital ad campaigns can affect party vote shares (Aggarwal et al., 2023; Coppock et al., 2022; Hager, 2019). Typical voter mobilisation campaigns often deploy a strategy of raising awareness about the efficacy of the vote, employing ‘cognitive mobilisation’ messages such as ‘Your Vote Matters’ or ‘Don’t Miss Out’. However, there is scepticism about whether such cognitive mobilisation messages actually work (Hersh, 2020; Holbein and Hillygus, 2020). Behavioural messages, delivered via other modes, have been shown to be more promising (Mann and Bryant, 2020; Mann and Sinclair, 2014). In contrast to voter turnout, there is also very limited experimental evidence on the impact of these campaigns on voter registration and the subsequent link to turnout. But despite these concerns, the lack of robust evidence on the effectiveness of digital ads for voter registration means that many campaigns use social media ads to try and register voters, often with messages that might not be ideally suited for the task.
We provide evidence from three digital trials that show that social media campaigns may be ineffective at registering groups of under-registered voters, studied at two separate UK elections. What distinguishes our approach is that we culminate findings from trials that have similarities in design and were applied within the same geographic context. The studies were also conducted by different organisations in different elections, which gives confidence that our results are not due to one particular messenger or a specific electoral contest: the first messenger was an advocacy organisation in a general election; the second is a researcher-led campaign conducted in a local election; and the third campaign was run by an advocacy organisation also at a local election. The studies also vary at the level of assignment, with the first and second study targeting postcode sectors, and the third study assigned at the individual level. They also try out different types of mobilisation messages, either cognitive or behavioural, implemented on a variety of social media platforms. We cannot rule out that other combinations of these features in the UK context might work, but the variation and consistency of three null results from three trials in our experiments suggest that digital ads have no meaningful effect on voter registrations.
The scarcity of evidence on the effects of digital ads on electoral registration
Despite the increasing prominence of digital methods in election campaigns, and the heavy financial investments that modern campaigns devote online (Fowler et al., 2020; Jungherr et al., 2020), randomised controlled trials evaluating the impacts of digital media adverts are still rare, and do not focus on voter registration (Fowler et al., 2020). Moreover, they are limited to a few social media platforms, such as Facebook and X (formerly Twitter), and neglect others, like Instagram and Snapchat, social media outlets used in our study, which are increasingly popular. An early randomised trial on Facebook using digital adverts in the USA increased turnout by around 0.5 percentage points (Bond et al., 2012), with a subsequent study targeted at millennial voters only effective in competitive districts (Haenschen and Jennings, 2019). Another US study revealed a zero average treatment effect on turnout, though a positive impact where message, audience and electoral context are congruent (Haenschen, 2022). Null effects on turnout have also been reported based on experiments, where ads were mostly meant to persuade voters (Aggarwal et al., 2023; Coppock et al., 2022; Hager, 2019). Coppock et al. (2022) find minimal effects of digital ads on Democrat vote share in the US context. Beyond turnout and vote shares, studies show null effects of public or semi-public Facebook and Twitter posts on online and offline political activism (Coppock et al., 2015; Foos et al., 2020). Even major changes to the Facebook online experience, such as changes to media feed algorithms, do not impact turnout (Guess et al., 2023), polarisation, or political knowledge (Nyhan et al., 2023). Experimental studies of voter registration have so far mostly relied on direct mail and door-to-door canvassing. Face-to-face canvassing has been shown to have larger positive effects in the region of 2.2 percentage points (Braconnier et al., 2017; Nickerson, 2015), but was less feasible during the Covid-19 pandemic. Direct mail and postcards have also been employed as effective means of voter registration (John et al., 2015; Mann and Bryant, 2020). Other registration experiments done by email and text message explicitly targeted at low-registration groups have recorded divergent findings (Bennion and Nickerson, 2018; Cheng-Matsuno et al., 2023; Kölle et al., 2019; Nickerson, 2007). Some of the most promising interventions take place within the school context. Studies using classroom presentations in colleges and high schools show substantively large increases in youth voter registration (Addonizio, 2011; Bennion and Nickerson, 2016), but they are difficult to scale up.
Study designs
We present three trials to test the impact of digital ads on voter registration. 1 Voter registration in the UK may be done online, requiring only basic information, such as name, address, nationality, date of birth, National Insurance Number (NI number) and email address. The process takes approximately 5 minutes to complete in one session. This digital registration process closely resembles the procedure used in the 40 US states and DC that allow for digital voter registration. The three trials share a common context which is elections happening in the UK between 2019 and 2021, and delivered by two NGOs and the research team. As in many democracies, campaigning in the UK increasingly uses social media, whether done by political parties, advocacy groups or even traditional governmental organisations concerned with voter registration and turnout (Dommett, 2021). One of the trials was a pragmatic intervention, done with an advocacy group using their own campaign materials. The other two followed a common research design to the trials and interventions, reflecting recent work about the most effective interventions. Campaigns targeting low-registration voters, such as young people, tend to assume that they need to be mobilised by making them aware of the general importance of politics and their role in it. According to this reasoning, common mobilisation campaigns, such as ‘Rock the Vote’, are run based on the assumption that a main cause of low turnout is disengagement from politics. But there are good reasons to question the validity of the assumption that these voters lack the motivation to vote (Holbein and Hillygus, 2020). Tasks like registration are perceived to be costly and many citizens, such as young people, experience difficulty navigating the process (Holbein and Hillygus, 2020). As Holbein and Hillygus (2020: 33) write, ‘For new voters, the registration requirement, in particular, is recognised as especially burdensome – it typically must be completed by a certain deadline, it must be updated with every change in address’. Based on many metrics, interest and engagement with politics have increased in recent decades (Dalton, 2007). Many people also have a civic orientation. If people are already cognitively mobilised, a cognitive mobilisation campaign would be bound to fail, especially when citizens do not have the skills to complete a task and go through a bureaucratic process, such as voter registration. It may hence be better to help them take practical steps to register, that is, encouraging them to plan, process information and then prepare to convert an intention into action. While typically campaigns focus on cognitive mobilisation (Ahmed, 2019), messages that build on insights from behavioural science, such as plan-making (Nickerson and Rogers, 2010) might therefore be more effective.
Study 1: Social media ads from a civic organisation
We worked with a civic organisation, randomly assigning a part of their well-organised campaign to test if social media ads are effective at registering young people to vote in the UK 2019 General Election. 2 , 3 We assigned 879 postcode sectors 4 located within 40 UK parliamentary constituencies to two groups: one control, and one treatment group that received voter registration ads for 7 days directly prior to the registration deadline from the organisation via Instagram and Snapchat. Typical example ads used in the trial by the organisation are displayed in Figure C.1 in the Appendix. All ads contained a direct link (e.g., via swipe-up on Instagram) to the UK Government’s voter registration website (https://www.gov.uk/register-to-vote). Ads were targeted at young people aged between 18 and 35 years. The registration messages appeared in postcode sectors assigned to the treatment group in the week before the voter registration deadline on 26 November. To avoid spillovers, they were displayed in the mornings and evenings only. The organisation also provided data on the successful placement of ads in each postcode sector, as well as spending, impressions, and engagement metrics at the campaign level.
Post-election, we obtained de-identified voter registration data from public registers in the 40 constituencies included in the experimental sample. We matched this data with their experimental assignment through the postcode column (ensuring individual-level de-identification, with the smallest unit being the postcode). The crucial question remains whether the social media clicks translated into actual voter registration. In Appendix Section C.2, we address how we handle non-reporting postcode sectors, which were missing due to being located outside the experimental sample. Table C.11 presents these results. As expected, there are no significant differences in whether control and treatment sectors report voter registration numbers, indicating that missingness is unlikely to be a result of the treatment. To ensure the validity of our experimental design, we conducted balance checks in Appendix Table A.7. These checks demonstrate that census covariates are balanced across treatment and control sectors. For further insights, descriptive statistics of the covariates are provided in Appendix Table A.2, while descriptive statistics of the outcome variables can be found in Appendix Table A.1.
Study 2: Large-scale trial of social media ads
This online field experiment was conducted in the context of the 2021 English local elections, using Instagram, Facebook and Snapchat ads created by the research team with help of a designer. The experimental sample comprised 1981 postcode sectors located in 69 local authorities. We followed three criteria to select these postcode sectors: sector size, mean age and share of BAME (Black, Asian and minority ethnic) residents. 5 The 1981 postcode sectors were block-randomly assigned with equal probabilities to one treatment group or a pure control group. The assignment was stratified by region and postcode sector size. The treatment was a 10-days-long digital ad campaign on Instagram, Facebook and Snapchat that ran in postcode sectors assigned to treatment. The ad campaign included a bundle of three social media ads that built on the following behavioural themes, 1) follow-through 2) anti-sludge and 3) dynamic social norms. A follow-through type aims to nudge individuals by strengthening their sense of grit to tackle obstacles. An anti-sludge type aims to vary the perception about the costs of voting. A dynamic norm type of message appeals to social pressure when given information about what other individuals in the same community are doing. The ads can be found in Appendix Section D.1.
In this trial, we exclusively focused on individuals residing within the postcode sectors by applying a resident filter. After the election, we collected voter registration data with help from the Electoral Commission, and matched them to our experimental assignment via the postcode sector column. Note that we were only able to obtain digitised data on registrations that also applied for a postal vote. While this is an important limitation, given the context of the Covid-19 pandemic, it is less severe than would have been the case in other periods. We discuss deviations from the PAP in detail in Appendix section D.3. We show non-reporting postcode sectors in Table D.17. As expected given random assignment, we find no significant differences in whether control and treatment sectors report voter registration numbers, indicating that missingness is unlikely to have occurred as a function of the treatment. Balance checks are displayed in Table A.7 in the Appendix and show that census covariates are balanced across treatment and control sectors. Descriptive statistics of the covariates can be found in Appendix Table A.4. Descriptive statistics of the outcome variable can be found in Appendix Table A.3.
Study 3: Social media ads from an issue advocacy organisation
During the 2021 English local elections, we conducted a field experiment to examine the impact of digital ads and SMS text messages on voter registrations among members and sympathisers of an advocacy organisation. The digital experiment was a collaborative effort between the advocacy organisation and the researchers. The Facebook campaign specifically targeted individuals using the emails provided by the organisation at the individual-level. Subjects had opted in to contact by the organisation. The sample comprised 9290 individuals. Sample 1 encompassed 7174 participants who shared both email addresses and phone numbers, while Sample 2 encompassed 2116 participants with email addresses only. Random assignment for Sample 1 was based on block (by county/city) and cluster (by household) to two factors: Factor 1 determined the mode of contact (Facebook ads, SMS or pure control group), and Factor 2 indicated the option of a contact number in the Facebook ads or SMS to aid with voter registration. Subjects in Sample 2 were randomly assigned to three conditions: pure control, Facebook ads and Facebook adverts plus a callback option. In this paper, we focus on presenting and evaluating the effect of Facebook adverts only, as text messages are not within the scope of this study. 6 Subjects in the Facebook treatment group were located in nine different counties, in larger cities. The distribution of subjects is displayed in Table A.8, which suggests that the density of the treatment per location was not very high.
The Facebook campaign conveyed three distinct messages centred around the themes of 1) follow-through (days 1-3), 2) anti-sludge (days 4-6) and 3) social norms (days 7-9). These themes employed behavioural approaches to voter registration, recognising that individuals might already be motivated to register, but may require support or encouragement to navigate the process effectively. The ads used in the campaign can be found in Appendix Section E.1. Similar to Study 2, the same ads were used, but for individual-level targeting, an additional option of ‘callback’ was included to offer assistance to subjects in the registration process, if requested. Two volunteers from the organisation were assigned to distinct cities to provide assistance. The targeting strategy involved using first and last names, along with email addresses. Over the course of a 9-day period, individuals were repeatedly targeted, leading to the total number of impressions. Registration and turnout data were collected at the local council premises. We manually matched geographic location, names and last name, which we had been supplied by the advocacy organisation, to the electoral registers. Descriptive statistics can be found in Tables A.5 and A.6.
Campaign statistics.
Note: Number of successfully targeted individuals in Study 3 is estimated based N reach/N ads.
Results
In all studies, we estimate the Intent-to-Treat (ITT) effect using linear regression with heteroskedasticity-consistent (HC2) standard errors:
where Y represents the population-scaled proportion of registered voters per postcode sector, ranging between 0 and 1, in Study 1 and Study 2, and a binary registration indicator in Study 3. RegistrationAd indicates whether a sector (Study 1 and 2) or an individual (Study 3) was assigned to receive voter registration ads (Snapchat and Instagram in Study 1; Snapchat, Instagram and Facebook in Study 2, Facebook in Study 3). C stands for fixed effects for constituency (Study 1), region-postcode sector size (Study 2) and county (Study 3). We present both the estimated ITTs of all three trials and the combined treatment effect based on fixed effects meta-analysis in Figure 1.
7
Comparison of the three trials. Notes: The meta-analysis coefficient is based on a fixed effects model, which results in the combined treatment effect, weighted by the precision of the studies.
Recall that Study 1 was a campaign designed and conducted by a civic organisation using cognitive messages, assigned at the postcode sector level. Extended results are displayed in Table C.9. Study 2 was conducted by the research team, assigned at the postcode sector level and used behavioural messages. The extended results can be found in Table D.16. Lastly, Study 3 is based on the advocacy organisation’s campaign, assigned at the individual level and used behavioural messages. We present the extended results of Study 3 in Table E.20. All study-level treatment effects can be interpreted as differences-in -proportions versus the control group. Study 1 and 2 coefficients are derived from covariate-adjusted full models, to reduce sampling variability. In Study 1, the dependent variable scales the absolute number of young people registered by population per postcode sector and in Study 2 it scales the absolute number of postal voters registered by population per postcode sector. In Study 3 the coefficient is also derived from the covariate-adjusted model, reporting a covariate-adjusted difference-in-proportions to allow for parallel interpretation.
Across the three trials, we observe null effects of social media ads on voter registrations. The estimated Intent-to-Treat effects across trials are −0.9 percentage points (Study 1), −2 percentage points (Study 2) and 0.1 percentage points (Study 3). None of the estimated effects are substantively large and positive, or significantly different from zero. The estimated combined treatment effect is −0.8 percentage points, with a 95% confidence interval ranging from −2.5 to 0.9 percentage points. That means that 0.9 percentage points is the upper bound of the estimated 95% confidence interval. Further details and robustness checks for each study can be found in the respective Appendix sections, C, D and E. Unsurprisingly, given the consistent null effects on registrations, in Study 3, where we were able to collect validated turnout data, we also find null effects on turnout (see Table E.22). Moreover, as pre-registered, we report heterogeneous treatment effects of the digital ad campaigns in Studies 1 and 2. Tables C.15 and C.19 show that we do not find any heterogeneous effects conditional on mean age or the mean share of BAME residents in a postcode sector.
Conclusion
This study provides a sobering picture of consistent null effects on voter registrations obtained from three typical social media ad campaigns, evaluated with randomised trials and aimed at under-registered groups. We used Facebook but also other popular social media platforms, Instagram and Snapchat. The latter have so far received little attention from researchers conducting randomised campaign trials. Our results show that social media-based cognitive and behavioural mobilisation messages aimed at ethnic minorities, private-sector tenants and young people, were ineffective at increasing voter registrations.
Even with these important null findings, it is important to stress the limitations to these studies. Our estimates using aggregate data are inevitably noisy given targeting for Study 1 and Study 2 were carried out at the postcode sector level rather than at the household level and that outcome data for Study 2 was only available for those who registered for a postal vote. We also failed to obtain pre-treatment outcome data which might have helped to reduce sampling variability. But it is important to note that we also obtained a null effect from our individual-level Study 3, where we matched targeted individuals to their registration and turnout records and assigned digital ads by household. Even with household level targeting, which as-good-as rules out large spillovers between treated and untreated households, we still estimate the effect of the digital ads to be zero. Taken all together, our meta-analysis estimates an upper bound of the confidence interval of +1 percentage point. Given that voter registration as an outcome should be easier to affect via digital ads than turnout or vote choice since it can be done online in less than 5 minutes, being able to rule out medium to large effects on voter registration rates is an informative finding.
The busy campaign environment may have also contributed to the null effects we observe because ads compete for attention with other campaign messages, and the environment might be saturated (Kalla and Broockman, 2017). What would speak against such an interpretation is that results are consistently null, even in local elections, where the online space was less saturated with ads than in the 2019 General Election. Moreover, the digital ad campaigns received a relatively large number of online impressions and clicks, and would likely have been classified as ‘successful’ campaigns, based on commonly-measured digital soft outcomes. We acknowledge that we did not target just unregistered voters meaning that many would have already registered, but this is also typical of GOTV interventions, which usually target a mix of individuals with different turnout probabilities (Green and Gerber, 2019). The populations we target reflect the difficulty of identifying non-registered voters, which campaigns and researchers commonly face. Finally, in the aggregate trials we also cannot rule out spillover effects that might disguise a larger treatment effect. But we did all we could to minimise spillovers, for instance by targeting voters at their home address and timing ads outside daytime hours when people were less likely to be travelling.
So, did the campaigns fail because of the medium, the message, the context or a combination of the above? While this question is impossible to answer conclusively, the coordinated experimental designs of our studies provide variation along important dimensions. First, while all trials were conducted in Britain in a period of political upheaval (Fieldhouse et al., 2021a), there is important variation in context – one trial was conducted in the 2019 General Election, the other two in the 2021 local elections. While the 2019 trial was conducted before the Covid-19 pandemic, the 2021 trials were conducted during the Covid-19 pandemic, with social distancing rules still in place in the UK and a greater focus on postal voting. Moreover, while we cannot rule out that the underlying theoretical strategies aimed at cognitive and behavioural mobilisation could have been implemented more effectively via digital ads, we worked together with two different outside groups, and used focus groups and A/B testing to trial the messages before fielding them in large-scale digital trials. What we find is that social media ads, no matter whether they use cognitive or behavioural mobilisation messages, did not translate into higher registration rates. The results cannot tell us whether the underlying theories are correct, given the variety of ways one can think of presenting and delivering such messages in offline and online spaces. Studies that evaluate other forms of behavioural messaging, using methods other than digital ads have produced more positive results (Holbein and Hillygus, 2020), which would point to the method of delivery as an important factor. While our trials focus on voter registration as the main outcome, the results are consistent with those obtained from trials that evaluated the effects of digital ads on turnout in a different country context, the United States (Aggarwal et al., 2023).
With all these caveats, we still believe that the findings based on the three coordinated trials reported here provide important evidence that digital ads did not result in detectable effects. The causal evidence we provide raises questions about whether trying to increase the electoral participation of under-registered voters via social media campaigns is a promising electoral strategy. Process-wise, registrations should be easier to increase than turnout, since individuals can complete the registration process online in one go. That leaves us with the question about the role of the target population, young people, ethnic-minority voters and private-sector renters. These populations are the natural target populations for digital registration campaigns in Britain because they contain the largest shares of non-registered individuals, and they are hard-to-reach offline. Moreover, we do not find any evidence of heterogeneous effects by age or share of ethnic-minority voters in any of the trials. While these results are likely underpowered to detect small differences in effect sizes conditional on demographic covariates, we doubt that heterogeneous effects could explain our findings. While the results of these trials are sobering, social media platforms will likely remain one medium of choice for many organisations that attempt to register voters. Given that even very small effects could scale on social media, the effects of digital ads remain an important topic to be studied via large-scale trials and meta-analyses. We hope that we have contributed three data points to that effort.
Supplemental Material
Supplemental Material - Null effects of social media ads on voter registration: Three digital field experiments
Supplemental Material for Null effects of social media ads on voter registration: Three digital field experiments by Asli Unan, Peter John, Florian Foos and Vanessa Cheng-Matsuno in Research & Politics
Footnotes
Acknowledgements
We are indebted to the two organisations we worked with and their staff, and to the UK Democracy Fund (Joseph Rowntree Reform Trust and partners) for funding. We would like to thank the Electoral Commission and Local Authorities for sharing data for Studies 2 and 3, and the Liberal Democrats Insights and Data Team, specifically Josh Townsley, for sharing de-identified register data with us for Study 1. We are grateful to Theresa Bischof, Peter Bristow, Tyler Johnson, Adam Keyworth, Sara Luxmoore, Len Metson, Yasmina O’Sullivan, Olea Rugumayo, and Johannes Rosenbusch for valuable research assistance. We would also like to thank Jessica Kennedy and Ali Goldsworthy for their encouragement and feedback, the participants of the 2019 Nuffield CESS Social Media Workshop, particularly Ray Duch, Horacio Larreguy, and David Nickerson, the participants of the 2020 APSA panel ‘Field and Natural Experiments on Democratic Participation’, the participants of the Yale American & Comparative Political Behavior Workshop and the University of Vienna’s Political Science Seminar, for excellent comments. The replication files are available via the R&P dataverse (Unan et al., 2023).
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Joseph Rowntree Reform Trust and the UK Democracy Fund.
Supplemental Material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
