Restricted accessResearch articleFirst published online 2024-4
Effectiveness of WhatsApp for Measuring Migration in Follow-Up Phone Surveys. Lessons from a Mode Experiment in Two Low-Income Countries during COVID Contact Restrictions
Phone surveys have increasingly become important data collection tools in developing countries, particularly in the context of sudden contact restrictions due to the COVID-19 pandemic. So far, there is limited evidence regarding the potential of the messenger service WhatsApp for remote data collection despite its large global coverage and expanding membership. WhatsApp may offer advantages in terms of reducing panel attrition and cutting survey costs. WhatsApp may offer additional benefits to migration scholars interested in cross-border migration behavior which is notoriously difficult to measure using conventional face-to-face surveys. In this field experiment, we compared the response rates between WhatsApp and interactive voice response (IVR) modes using a sample of 8446 contacts in Senegal and Guinea. At 12%, WhatsApp survey response rates were nearly eight percentage points lower than IVR survey response rates. However, WhatsApp offers higher survey completion rates, substantially lower costs and does not introduce more sample selection bias compared to IVR. We discuss the potential of WhatsApp surveys in low-income contexts and provide practical recommendations for field implementation.
AlencarA. (2019). Digital place-making practices and daily struggles of Venezuelan refugees in Brazil. In SmetsK.LeursK.GeorgiouM.WittebornS.GajjalaR. (Eds.), The SAGE handbook of media and migration. Sage.
2.
AnsolabehereS.SchaffnerB. F. (2014). Does survey mode still matter? Findings from a 2010 multi-mode comparison. Political Analysis, 22(3), 285–303. https://doi.org/10.1093/pan/mpt025
3.
BallivianA.AzevedoJ. P.DurbinW. (2015). Using mobile phones for high-frequency data collection. In ToninelliD.PinterR.de PedrazaP. (Eds.), Mobile research methods: Opportunities and challenges of mobile research methodologies (pp. 21–39). Ubiquity Press. https://doi.org/10.5334/bar.c
4.
BarbosaS.MilanS. (2019). Do not harm in private chat apps: Ethical issues for research on and with WhatsApp. Westminster Papers in Communication and Culture, 14(1).
5.
BeaucheminC.SchoumakerB. (2016). Micro methods: Longitudinal surveys and analyses. In WhiteM. J. (Ed.), International handbook of migration and population distribution (6th ed.). Springer.
6.
Bia-ZafinikamiaM.TjadenJ.GninafonH. (2020). The impact of mobile cinema events on potential migrants in Guinea. Impact Evaluation Report. International Organization for MigrationInternational Organization for Migration.
7.
BilsborrowR. E.HugoG.OberaiA. S. (1997). International migration statistics: Guidelines for improving data collection systems (1. Publ). ILO.
8.
ChenJ.NeoP. (2019). Texting the waters: An assessment of focus groups conducted via the WhatsApp smartphone messaging application. Methodological Innovations, 12(3), 205979911988427. https://doi.org/10.1177/2059799119884276
9.
ChristopherA.Di BattistaF.MusilloB. (2021). Switching to or starting mobile phone surveys: External guidance document.
10.
DabalenA. (2016). Mobile phone panel Surveys in developing countries: A practical Guide for microdata collection (Online-Ausg). World Bank E-Library ArchiveThe World Bank. https://doi.org/10.1596/978-1-4648-0904-0
11.
DaikelerJ.BošnjakM.Lozar ManfredaK. (2020). Web versus other survey modes: An updated and extended meta-analysis comparing response rates. Journal of Survey Statistics and Methodology, 8(3), 513–539. https://doi.org/10.1093/jssam/smz008
12.
DekkerR.EngbersenG.KlaverJ.VonkH. (2018). Smart refugees: How Syrian asylum migrants use social media information in migration decision-making. Social Media + Society, 4(1), 205630511876443. https://doi.org/10.1177/2056305118764439
13.
DemombynesG.GubbinsP.RomeoA. (2013). Challenges and opportunities of mobile phone-based data collection: Evidence from South Sudan. World Bank Policy Research Working. (6321). https://ssrn.com/abstract=2202683
14.
DillonB. (2012). Using mobile phones to collect panel data in developing countries. Journal of International Development, 24(4), 518–527. https://doi.org/10.1002/jid.1771
15.
DunschF.TjadenJ.QuivigerW. (2019). Migrants as messengers: The impact of peer-to-peer communication on potential migrants in Senegal. Impact evaluation report. International Organization for Migration (IOM): International Organization for Migration (IOM).
16.
FawcettJ. T.ArnoldF. (1987). The role of surveys in the study of international migration: An appraisal. International Migration Review, 21(4), 1523–1540. https://doi.org/10.1177/019791838702100427
FirchowP.Mac GintyR. (2020). Including hard-to-access populations using mobile phone surveys and participatory indicators. Sociological Methods & Research, 49(1), 133–160. https://doi.org/10.1177/0049124117729702
19.
GarlickR.OrkinK.QuinnS. (2020). Call me maybe: Experimental evidence on frequency and medium effects in microenterprise surveys. The World Bank Economic Review, 34(2), 418–443. https://doi.org/10.1093/wber/lhz021
20.
GibsonD. G.PereiraA.FarrenkopfB. A.LabriqueA. B.PariyoG. W.HyderA. A.[AdnanA. (2017). Mobile phone surveys for collecting population-level estimates in low- and middle-income countries: A literature review. Journal of Medical Internet Research, 19(5), Article e139. https://doi.org/10.2196/jmir.7428
21.
GibsonD. G.WosuA. C.PariyoG. W.AhmedS.AliJ.LabriqueA. B.HyderA. A.KhanI. A.RutebemberwaE.FloraM. S. (2019). Effect of airtime incentives on response and cooperation rates in non-communicable disease interactive voice response surveys: Randomised controlled trials in Bangladesh and Uganda. BMJ Global Health, 4(5), Article e001604. https://doi.org/10.1136/bmjgh-2019-001604
22.
GreenleafA. R.GadiagaA.ChoiY.GuiellaG.TurkeS.BattleN.AhmedS.MoreauC. (2020). Automated and interviewer-administered mobile phone surveys in Burkina Faso: Sociodemographic differences among female mobile phone survey respondents and nonrespondents. JMIR MHealth and UHealth, 8(7), Article e17891. https://doi.org/10.2196/17891
23.
GreenleafA. R.GibsonD. G.KhattarC.LabriqueA. B.AlainB.PariyoG. W.GeorgeW. (2017). Building the evidence base for remote data collection in low- and middle-income countries: Comparing reliability and accuracy across survey modalities. Journal of Medical Internet Research, 19(5), Article e140. https://doi.org/10.2196/jmir.7331
24.
GruchyT. deVeareyJ.OpitiC.MlotshwaL.ManjiK.HanefeldJ. (2021). Research on the move: Exploring WhatsApp as a tool for understanding the intersections between migration, mobility, health and gender in South Africa. Globalization and Health, 17(1), 71. https://doi.org/10.1186/s12992-021-00727-y
25.
GummerT.SchmiedebergC.BujardM.ChristmannP.HankK.KunzT.LuckD.NeyerF. J. (2020). The impact of Covid-19 on fieldwork efforts and planning in pairfam and FReDA-GGS. Advance online publicationhttps://doi.org/10.18148/srm/2020.v14i2.7740
26.
JacobsenJ.KühneS. (2021). Using a mobile app when surveying highly mobile populations: Panel attrition, consent, and interviewer effects in a survey of refugees. Sage. https://doi.org/10.1177/0894439320985250
27.
JervenM.JohnstonD. (2015). Statistical tragedy in Africa? Evaluating the data base for African economic development. The Journal of Development Studies, 51(2), 111–115. https://doi.org/10.1080/00220388.2014.968141
28.
KaufmannK. (2020). Mobile methods: Doing migration research with the help of smartphones. In SmetsK.LeursK.GeorgiouM.WittebornS.GajjalaR. (Eds.), The SAGE handbook of media and migration (pp. 167–179). SAGE Publications Ltd. https://doi.org/10.4135/9781526476982.n22
29.
LaiS.Erbach-SchoenbergE. zuPezzuloC.RuktanonchaiN. W.SorichettaA.SteeleJ.LiT.DooleyC. A.TatemA. J. (2019). Exploring the use of mobile phone data for national migration statistics. Palgrave Communications, 5(1), 34. https://doi.org/10.1057/s41599-019-0242-9
30.
LauC. Q.CronbergA.MarksL.AmayaA. (2019). In search of the optimal mode for mobile phone surveys in developing countries. A comparison of IVR, SMS, and CATI in Nigeria. Survey Research Methods, 13(3), 305–318. https://doi.org/10.18148/srm/2019.v13i3.7375
31.
LeeH.KimS.CouperM. P.WooY. (2019). Experimental comparison of PC web, smartphone web, and telephone surveys in the new technology era. Social Science Computer Review, 37(2), 234–247. https://doi.org/10.1177/0894439318756867
32.
L'EngleK.SefaE.AdimazoyaE. A.YarteyE.LenziR.TarpoC.Heward-MillsN. L.LewK.AmpehY. (2018). Survey research with a random digit dial national mobile phone sample in Ghana: Methods and sample quality. PloS One, 13(1), Article e0190902. https://doi.org/10.1371/journal.pone.0190902
33.
LeoB.MorelloR.MellonJ.PeixotoT.DavenportS. T. (2015). Do mobile phone surveys work in poor Countries?SSRN Electronic Journal. Advance online publicationhttps://doi.org/10.2139/ssrn.2623097
34.
LiuM.-M.CreightonM. J.RiosmenaF.Baizán Mun OzP. (2016). Prospects for the comparative study of international migration using quasi-longitudinal micro-data. Demographic Research, 35(1), 745–782. https://doi.org/10.4054/DemRes.2016.35.26
LynnP.KaminskaO. (2013). The impact of mobile phone on survey measurement error. The Public Opinion Quarterly, 77(2), 586–605. http://www.jstor.org/stable/24545755
37.
MaffioliE. M. (2020). Collecting data during an epidemic: A novel mobile phone research method. Journal of International Development, 32(8), 1231–1255. https://doi.org/10.1002/jid.3515
38.
ManjiK.HanefeldJ.VeareyJ.WallsH.Dd DruchyT. (2021). Using WhatsApp messenger for health systems research: A scoping review of available literature. Health Policy and Planning, 36(5), 594–605. https://doi.org/10.1093/heapol/czab024
PhadnisR.WickramasingheC.ZevallosJ. C.DavlinS.KumarapeliV.LeaV.LeeJ.PereraU.SolorzanoF. X.VasconezJ. F.VásconezJ. F. (2021). Leveraging mobile phone surveys during the COVID-19 pandemic in Ecuador and Sri Lanka: Methods, timeline and findings. PloS One, 16(4), Article e0250171. https://doi.org/10.1371/journal.pone.0250171
SchusterC.Perez BritoC. (2011). Cutting costs, boosting quality and collecting data real-time : Lessons from a cell phone-based beneficiary survey to strengthen Guatemala’s conditional cash transfer program (No. 166). World Bank.
43.
StecklovG.WeinrebA.CarlettoC. (2018). Can incentives improve survey data quality in developing countries? Results from a field experiment in India. Journal of the Royal Statistical Society: Series a, 181(4), 1033–1056. https://doi.org/10.1111/rssa.12333
44.
SugieN. F. (2018). Utilizing smartphones to study disadvantaged and hard-to-reach groups. Sociological Methods & Research, 47(3), 458–491. https://doi.org/10.1177/0049124115626176
TjadenJ.DunschF. (2021). The effect of peer-to-peer risk information on potential migrants – evidence from a randomized controlled trial in Senegal. World Development, 145(1), 105488. https://doi.org/10.1016/j.worlddev.2021.105488
48.
TjadenJ.GninafonH. (2022). Raising awareness about the risk of irregular migration - quasi-experimental evidence from Guinea. Population and Development Review. (Online First).
49.
UNDP. (2018). WhatsApp surveying guide: Lessons learned from two qualitative WhatsApp surveys in Lebanon. New York City: United Nations Development Programme.
50.
UNHCR. (2020). Data collection in times of physical distancing.
51.
VehovarV.BerzelakN.Lozar ManfredaK. (2010). Mobile phones in an environment of competing survey modes: Applying metric for evaluation of costs and errors. Social Science Computer Review, 28(3), 303–318. https://doi.org/10.1177/0894439309353035
52.
WillekensF.MasseyD. S.RaymerJ.BeaucheminC. (2016). International migration under the microscope. Science, 352(6288), 897–899. https://doi.org/10.1126/science.aaf6545
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
