Restricted accessResearch articleFirst published online 2025-1
How much is it about the money? Using facebook to promote a web survey during the COVID-19 pandemic and estimating average response-returns on ad-investments
The use of ads posted on online social networks has become a popular strategy to recruit survey participants. This paper quantifies the returns from ad-investments in terms of the number of responses. We use as a case study a survey about living and working conditions that was promoted in Spain via Facebook from March 2020 to October 2021. Results show that the effect of each additional euro invested in Facebooks ads daily generated between 2 and 2.8 additional completed surveys per day. These results are stable and consistent throughout the period we study. Nonetheless, we find that the number of survey responses tended to be significantly higher when the COVID-19 restrictions imposed were tighter. Our findings are relevant and have practical implications for future survey research.
CornesseC.BlomA. G.DutwinD.KrosnickJ. A.de LeeuwE. D.LegleyeS.PasekJ.PennayD.PhillipsB.SakshaugJ. W.StruminskayaB.WenzA. (2020). A review of conceptual approaches and empirical evidence on probability and nonprobability sample survey research. Journal of Survey Statistics and Methodology, 8(1), 4–36. https://doi.org/10.1093/jssam/smz041
5.
CouperM. P. (2000). Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64(4), 464–494. https://doi.org/10.1086/318641
6.
CouperM. P.MillerP. v. (2008). Web survey methods: Introduction. Public Opinion Quarterly, 72(5), 831–835. https://doi.org/10.1093/poq/nfn066
7.
DaikelerJ.BosnjakM.ManfredaK. L. (2020). Web versus other survey modes: An updated and extended meta-analysis comparing response rates. Journal of Survey Statistics and Methodology, 8(3), 513–539. https://doi.org/10.1093/jssam/smz008
8.
De HertP.PapakonstantinouV.MalgieriG.BeslayL.SanchezI. (2018). The right to data portability in the GDPR: Towards user-centric interoperability of digital services. Computer Law & Security Review, 34(2), 193–203. https://doi.org/10.1016/J.CLSR.2017.10.003
9.
de LeeuwE. D. (2008). Choosing the method of data collection. In de LeeuwE. D.HoxJ. J.DillmanD. A. (Eds.), International handbook of survey methodology (pp. 113–135). Taylor & Francis Group.
10.
de PedrazaP.VicenteM. R. (2021). Are Spaniards happier when the bars are open? Using life satisfaction to evaluate COVID-19 non-pharmaceutical interventions (NPIs). International Journal of Environmental Research and Public Health, 18(19), 2021. https://doi.org/10.3390/ijerph181910056
11.
de PedrazaP.GuziM.TijdensK. (2020). Life satisfaction of employees, labour market tightness and matching efficiency. International Journal of Manpower, 42, 341–355. https://doi.org/10.1108/IJM-07-2019-0323
12.
de PedrazaP.TijdensK.Muñoz De BustilloR.SteinmetzS.DeR. M. (2010). A Spanish continuous volunteer web survey: Sample bias, weighting and efficiency. Revista Española de Investigaciones Sociológicas, 131, 109–130. https://www.jstor.org/stable/25746569.
13.
de PedrazaP.VollbrachtI. (2023). General theory of data, artificial intelligence and governance. Humanities and Social Sciences Communications, 10(1), 607. https://doi.org/10.1057/s41599-023-02096-w
14.
DodgeN.ChapmanR. (2018). Investigating recruitment and completion mode biases in online and door to door electronic surveys. International Journal of Social Research Methodology, 21(2), 149–163. https://doi.org/10.1080/13645579.2017.1336596
15.
DuffyB.SmithK.TerhanianG.BremerJ. (2005). Comparing data from online and face-to-face surveys. International Journal of Market Research, 47(6), 615–639. https://doi.org/10.1177/147078530504700602
FanW.YanZ. (2010). Factors affecting response rates of the web survey: A systematic review. Computers in Human Behavior, 26(2), 132–139. https://doi.org/10.1016/j.chb.2009.10.015
21.
Ferri-GarcíaR.RuedaM. del M. (2020). Propensity score adjustment using machine learning classification algorithms to control selection bias in online surveys. PLOS ONE, 15(4), 1–19. https://doi.org/10.1371/journal.pone.0231500
22.
GöritzA. S.ReinholdN.BatinicB. (2000). Online panels. In BatinitcB.ReipsU. D.BosnjakM.WernerA. (Eds.), Online social sciences (pp. 29–51). Hogrefe Publishing.
23.
GreenH.FernandezR.MacPhailC. (2021). Social media as a platform for recruitment to a national survey during the COVID-19 pandemic: Feasibility and cost analysis. JMIR Formative Research, 5(7), 2021. https://doi.org/10.2196/28656
24.
GrowA.PerrottaD.del FavaE.CimentadaJ.RampazzoF.Gil-ClavelS.ZagheniE. (2020). Addressing public health emergencies via Facebook surveys: Advantages, challenges, and practical considerations. Journal of Medical Internet Research, 22(12), 2021. https://doi.org/10.2196/20653
25.
GuL. L.SkierkowskiD.FlorinP.FriendK.YeY. (2016). Facebook, Twitter, & Qr codes: An exploratory trial examining the feasibility of social media mechanisms for sample recruitment. Computers in Human Behavior, 60, 86–96. https://doi.org/10.1016/J.CHB.2016.02.006
HeadB. F.DeanE.FlaniganT.SwicegoodJ.KeatingM. D. (2016). Advertising for cognitive interviews: A comparison of Facebook, craigslist, and snowball recruiting. Social Science Computer Review, 34(3), 360–377. https://doi.org/10.1177/0894439315578240
28.
INE (Spanish National Statistical Office). (2020). Labor Force survey. https://www.ine.es
29.
INE (Spanish National Statistical Office). (2023). Survey on ICT usage by households and individuals. https://www.ine.es
30.
LannelliL.GigliettoF.RossiL.ZurovacE. (2020). Facebook digital traces for survey research: Assessing the efficiency and effectiveness of a Facebook ad–based procedure for recruiting online survey respondents in niche and difficult-to-reach populations. Social Science Computer Review, 38(4), 462–476. https://doi.org/10.1177/0894439318816638
31.
LeeS.ValliantR. (2009). Estimation for volunteer panel web surveys using propensity score adjustment and calibration adjustment. Sociological Methods Research, 37(3), 319–343. https://doi.org/10.1177/0049124108329643
32.
LehdonvirtaV.OksanenA.RäsänenP.BlankG. (2021). Social media, web, and panel surveys: Using non-probability samples in social and policy research. Policy and Internet, 13(1), 134–155. https://doi.org/10.1002/poi3.238
33.
LindgrenE.MarkstedtE.MartinssonJ.AndreassonM. (2020). Invitation timing and participation rates in online panels: Findings from two survey experiments. Social Science Computer Review, 38(2), 225–244. https://doi.org/10.1177/0894439318810387
34.
LoosveldtG.SonckN. (2008). An evaluation of the weighting procedures for an online access panel survey. Survey Research Methods, 2(2), 93–105. https://doi.org/10.18148/SRM/2008.V2I2.82
35.
ManfredaK. L.BosnjakM.BerzelakJ.HaasI.VehovarV. (2008). Web surveys versus other survey modes: A meta-analysis comparing response rates. International Journal of Market Research, 50(1), 79–104. https://doi.org/10.1177/147078530805000107
36.
NeundorfA.ÖztürkA. (2021). Recruiting research participants through Facebook: Assessing Facebook advertisement tools.
37.
PedersenM. J.NielsenC. V. (2016). Improving survey response rates in online panels: Effects of low-cost incentives and cost-free text appeal Interventions. Social Science Computer Review, 34(2), 229–243. https://doi.org/10.1177/0894439314563916
38.
PerrottaD.GrowA.RampazzoF.CimentadaJ.del FavaE.Gil-ClavelS.ZagheniE. (2021). Behaviours and attitudes in response to the COVID-19 pandemic: Insights from a cross-national Facebook survey. EPJ Data Science, 10(17), 1–13. https://doi.org/10.1140/epjds/s13688-021-00270-1
39.
ReipsU.-D. (2006). Web-based methods. In EidM.DienerE. (Eds.), Handbook of multimethod measurement in psychology (pp. 73–85). American Psychological Association.
40.
RevillaM.CornilleauA.CousteauxA. S.LegleyeS.de PedrazaP. (2016). What is the gain in a probability-based online panel of providing internet access to sampling units who previously had No access?Social Science Computer Review, 34(4), 479–496. https://doi.org/10.1177/0894439315590206
41.
RinkenS.DomínguezJ. A.TrujilloM.LafuenteR.SotomayorR.Serrano-Del-rosalR. (2020). Combined mobile-phone and social-media sampling for web survey on social effects of COVID-19 in Spain. Survey Research Methods, 14(2), 165–170. https://doi.org/10.18148/srm/2020.v14i2.7733
42.
RosenbaumP. R.RubinD. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41
43.
SchneiderD.HarknettK. (2022). What’s to like? Facebook as a tool for survey data collection. Sociological Methods and Research, 51(1), 108–140. https://doi.org/10.1177/0049124119882477
44.
SchonlauM.van SoestA.KapteynA.CouperM. (2009). Selection bias in web surveys and the use of propensity scores. Sociological Methods & Research, 37(3), 291–318. https://doi.org/10.1177/0049124108327128
45.
ShaverL. G.KhawerA.YiY.Aubrey-BasslerK.EtchegaryH.RoebothanB.AsghariS.WangP. P. (2019). Using Facebook advertising to recruit representative samples: Feasibility assessment of a cross-sectional survey. Journal of Medical Internet Research, 21(8), 21. https://doi.org/10.2196/14021
46.
ShevchenkoY.HuberN.ReipsU.-D. (2023). Psychological well-being during the COVID-19 pandemic: Combining a web survey with experience sampling methodology. PLOS ONE, 18(3), e0282649. https://doi.org/10.1371/journal.pone.0282649
47.
SillsS. J.SongC. (2002). Innovations in survey research: An application of web-based surveys. Social Science Computer Review, 20(22), 22–30. https://doi.org/10.1177/089443930202000103
UN. (2014). A world that counts: Mobilising the data revolution for sustainable development. Independent Experts Advisory Group on Data Revolution for Sustainable Development. www.undatarevolution.org
50.
WageIndicator Foundation. (2020). Living and working in Coronavirus times survey. In WageIndicator foundation.
51.
WitteveenD.de PedrazaP. (2021). The roles of general health and COVID-19 proximity in contact tracing app usage: Cross-sectional survey study. In: JMIR Public Health and Surveillance(Vol, 7, Issue 8). JMIR Publications Inc. https://doi.org/10.2196/27892
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
