Characteristics of Respondents to Web-Based or Traditional Interviews in Mixed-Mode Surveys. Evidence from the Italian Permanent Population Census

Abstract

In order to provide useful tools for researchers in the design of actions to promote participation in web surveys, it is key to study the characteristics that define the profile of a “web respondent”, so that specific interventions can be planned. In this contribution, which draws on data collected during the 2019 housing population census in Italy, we define the set of familial and geographical characteristics that correspond to a greater probability that the interviewed household will choose to respond online, by estimating a multilevel model. The profile of a “computer-assisted web interview household” (CAWI-H) is then defined, on the basis of the structural characteristics of this population. Moreover, the geographical distribution of households is studied according to their distance from the CAWI-H profile. The results show that households that are more distant from the CAWI-H profile have characteristics that correspond to segments of the population generally affected by economic and social fragility; they are mainly elderly, foreigners, residents in small towns, and people with a low level of education. It is to these households in particular that survey designers can address specific actions that can enhance their willingness to participate in web surveys.

Keywords

Mixed-mode surveys respondent profiling multilevel models computer assisted web interview (CAWI)

References

Antoun, C., M.P. Couper, and F G. Conrad. 2017. “Effects of Mobile versus PC Web on Survey Response Quality: A Crossover Experiment in a Probability Web Panel.” Public Opinion Quarterly 81: 280–306. DOI: https://doi.org/10.1093/poq/nfw088.

Bandilla, W., M.P. Couper, and L. Kaczmirek. 2014. “The Effectiveness of Mailed Invitations for Web Surveys and the Representativeness of Mixed-Mode versus Internet-only.” Survey Practice 7(4). DOI: https://doi.org/10.29115/SP-2014-0020.

Benassi, F., and A. Naccarato. 2017. “Households in Potential Economic Distress. A Geographically Weighted Regression Model for Italy, 2001–2011.” Spatial Statistics 21: 362–376. DOI: https://doi.org/10.1016/j.spasta.2017.03.002.

Benzeval, M., J. Burton, T.F. Crossley, P. Fisher, C. Gardiner, A. Ja¨ckle, and J. Moore. 2021. “High Frequency Online Data Collection in an Annual Household Panel Study: Some Evidence on Bias Prevention and Bias Adjustment.” Understanding Society Working Paper Series 03: 1–26. Available at: https://www.understandingsociety.ac.uk/sites/default/files/downloads/working-papers/2021-03.pdf (accessed September 2021).

Bethlehem, I., F. Cobben, and B. Schouten. 2011. Handbook of Nonresponse in Household Surveys. New York: Wiley.

Bianchi, A., S. Biffignandi, and P. Lynn. 2017. “Web-face-to-face Mixed-mode Design in a Longitudinal Survey: Effects on Participation Rate, Sample Composition, and Costs.” Journal of Official Statistics 33(2): 385–408. DOI: http://dx.doi.org/10.1515/JOS-2017-0019.

Biemer P.P. 2010. “Total Survey Error: Design, Implementation, and Evaluation.” Public Opinion Quarterly 74: 817–848. DOI: https://doi.org/10.1093/poq/nfq058.

Biffignandi, S. and J.G. Bethlehem. 2012. “Web Surveys: Methodological Problems and Research Perspectives.” In Advanced Statistical Methods for the Analysis of Large Data sets, edited by A. Di Ciaccio, M. Coli, and J.M.A. Ibanez: 363–374. Berlin: Springer.

Biffignandi S., and J. Bethlehem. 2021 Handbook on Web Surveys. Second edition. New York: Wiley.

10.

Biffignandi, S. and M. Pratesi. 2002. “Modelling the Respondents’ Profile in a Web Survey on Firms in Italy.” In Developments in Social Science Methodology, edited by A. Ferligoj and A. Mrvar: 171–185. Ljubljana: FDV.

11.

Brick, J.M., and R. Tourangeau. 2017. “Responsive Survey Designs for Reducing Nonresponse Bias.” Journal of Official Statistics 33(3): 735–752. DOI: https://doi.org/10.1515/jos-2017-0034.

12.

Cellini R., and G. Torrisi. 2014. “Regional Resilience in Italy: A Very Long-run Analysis.” Regional Studies 48(11): 1779 – 1796. DOI: https://doi.org/10.1080/00343404.2013.861058.

13.

Chieppa, A., G. Gallo, V. Tomeo, F. Borrelli, and S. di Domenico. 2018. “Knowledge Discovery for Inferring the Usually Resident Population from Administrative Registers.” Mathematical Population Studies 26(2): 96–102. DOI: https://doi.org/10.1080/08898480.2017.1418114.

14.

Ciapanna, E. and G. Roma. 2020. Connected Italy. Rome: Bank of Italy. (Questioni di Economia e Finanza. Occasional Papers No. 573). Available at: https://www.bancaditalia.it/pubblicazioni/qef/2020-0573/QEF_573_20.pdf?language_id=1 (accessed September 2021).

15.

Citro, C.F. 2014. “From Multiple Modes for Surveys to Multiple Data Sources for Estimates”. Survey Methodology 40(2): 137–161. Available at: https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2014002/article/14128-eng.pdf?st=x9gGl05C (accessed September 2021).

16.

Cobben, F., and J.G. Bethlehem. 2013. Web Panels for Official Statistics. The Hague: Statistics Netherland. (Discussion Paper 201307). Available at: https://www.cbs.nl 〉 2013 〉 2013-07-x10-pub (accessed September 2021).

17.

Couper M.P. 2017. “New Developments in Survey Data Collection.” Annual Review of Sociology 43: 121–145. DOI: https://doi.org/10.1146/annurev-soc-060116-053613.

18.

Cracolici M.F., M. Cuffaro and P. Nijkamp. 2007. “Geographical Distribution of Unemployment: An Analysis of Provincial Differences in Italy.” Growth and Change 38(4): 649–670. DOI: https://doi.org/10.1111/j.1468-2257.2007.00391.x.

19.

Crescenzi, F. 2015. “Changing census by combining administrative sources and sample surveys”. In Proceedings of the 60th ISI World Statistics Congress, 26–31 July 2015, Rio de Janeiro, Brazil. Available at: https://2015.isiproceedings.org/Files/IPS022-P4-S.pdf (accessed September 2021).

20.

D’Ambrosio C. 2003. “Household Characteristics and the Distribution of Income in Italy: An Application of Social Distance Measures.” The Review of Income and Wealth 47(1): 43–64. DOI: https://doi.org/10.1111/1475-4991.00003.

21.

De Clercq M., J. Buysse, and M. D’Haese. 2020. The state of Rural Digitalisation in Europe. Desira. Available at: https://desira2020.eu/wp-content/uploads/2020/11/Brie-fing_Digitalisation-in-Europe.pdf (accessed September 2021).

22.

De Leeuw, E.D. 2005. “To Mix or Not to Mix Data Collection Modes in Surveys”. Journal of Official Statistics 21(2): 233–255.

23.

De Leeuw E.D. 2018. “Mixed Mode: Past, Present, and Future.” Survey Research Methods 12(2): 75–89. DOI: doi:10.18148/srm/2018.v12i2.7402.

24.

De Leeuw, E.D., and N. Berzelak. 2016. “Survey Mode or Survey Modes?” In The Sage Handbook of Survey Methodology, edited by C. Wolf, D. Joye, T.W. Smith, and Y. Fu, 142–156. London: Sage Publications.

25.

De Leeuw, E.D., J. Hox, and D. Dillman. 2008. International Handbook of Survey Methodology. New York: Routledge.

26.

De Leeuw E.D., Z.T. Suzer-Gurtekin, and J.J. Hox. 2019. “The Design and Implementation of Mixed-mode Surveys.” In Advances in Comparative Survey Methods: Multinational, Multiregional, and Multicultural contexts (3MC), edited by P. Johnson, B.E. Pennell, I.A.L. Stoop, and B. Dorer: 387–408. New York: Wiley.

27.

Dillman, D.A., J.D. Smyth, and L.M. Christian. 2014. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. Fourth edition. New York: Wiley.

28.

Durrant, G.B., and F. Steele. 2009. “Multilevel Modelling of Refusal and Non-contact in Household Survey Evidence from Six UK Government Surveys.” Journal of the Royal Statistical Society: Series A 172(2): 361–381. DOI: https://doi.org/10.1111/j.1467-985X.2008.00565.x.

29.

European Union 2017. Regulation of the European Parliament as Regards the Territorial Typologies 2017/2391. Official Journal of the European Union. Available at: https://eurlex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32017R2391 (accessed September 2021).

30.

Eurostat 2020. Digital Economy and Society Statistics – Households and Individuals. Luxemburg. Available at: https://ec.europa.eu/eurostat/statistics-explained/index.php/-Digital_economy_and_society_statistics_-_households_and_individuals (accessed September 2021).

31.

Eurostat 2021. Applying the Degree of Urbanisation. A Methodological Manual to Define Cities, Towns and Rural Areas for International Comparisons. 2021 edition. Eurostat: Luxemburg.

32.

Goldstein, H. 2010. Multilevel Statistical Models. Fourth Edition. New York: Wiley.

33.

Hargittai, E. 2002.“Second-Level Digital Divide: Differences in People's online skills.” First Monday 7(4). DOI: https://doi.org/10.5210/fm.v7i4.942.

34.

Hox J.J., E.D. de Leeuw, and E.A.O. Zijlmans. 2015. Measurement Equivalence in Mixed Mode Surveys. Frontiers in Psychology 6: 87. DOI: https://doi.org/10.3389/fpsyg.2015.00087.

35.

Istat 2018. “Preliminary Experimental Results on the Italian Population and Housing Census Estimation Methods.” In Proceedings of the Twentieth Conference of European Statisticians Group of Experts on Population and Housing Censuses, 26–28 September, 2018. Geneva: United Nations Economic Commission for Europe. Available at: https://unece.org/fileadmin/DAM/stats/documents/ece/ces/ge.41/2018/Meeting-Geneva-Sept/ECE_CES_GE.41_2018_20-1811953E.pdf (accessed September 2021).

36.

Istat 2019. Cittadini e ICT. Statistiche report. Rome: Istat. Available at: https://www.istat.it/it/archivio/236920 (accessed September 2021).

37.

Jaccard, p. 1908. “Nouvelles recherches sur la distribution florale.” Bulletin de la Société Vaudoise des Sciences Naturelles 44(163): 223–270.

38.

Kappelhof, J.W.S. 2015. “Face-to-Face or Sequential Mixed-Mode Survey Among Non-Western Minorities in the Netherlands: The effect of Differential Survey Designs on the Possibility of Nonresponse Bias.” Journal of Official Statistics, 31(1): 1–3. DOI: https://doi.org/10.1515/jos-2015-0001.

39.

Keusch F., B. Struminskaya, C. Antoun, M.P. Couper, F. Kreuter. 2019. “Willingness to Participate in Passive Mobile Data Collection.” Public Opinion Quarterly 83: 210–235. DOI: https://doi.org/10.1093/poq/nfz007.

40.

Luiten A., J. Hox and E.D. de Leeuw. 2020. Survey Nonresponse Trends and Fieldwork Effort in the 21st Century: Results of an International Study across Countries and Surveys. Journal of Official Statistics 36(3): 469–487. DOI: https://doi.org/10.2478/-jos-2020-0025.

41.

Maddala, G.S. and K. Lahiri. 2010. Introduction to Econometrics. Fourth Edition. New York: Wiley.

42.

Maslovskaya, O., G.B. Durrant, P.W.F. Smith and A. Villar. 2019. “What are the Characteristics of Respondents using Different Devices in Mixed-device Online Surveys? Evidence from Six UK Surveys.” International Statistical Review 87(2): 326–346. DOI: https://doi.org/10.1111/insr.12311.

43.

McCormick, W.P., N.I. Lyons, and K. Hutcheson. 1992. “Distributional Properties of Jaccard's Index of Similarity.” Communications in Statistics – Theory and Methods 21(1): 51–68. DOI: https://doi.org/10.1080/03610929208830764.

44.

Messer, B.L., and D.A. Dillman. 2011. “Surveying the general public over the internet using address-based sampling and mail contact procedures.” Public Opinion Quarterly, 75(3), 429–457. DOI: https://doi.org/10.1093/poq/nfr021.

45.

Mohorko A., E. de Leeuw, and J. Hox. 2013. “Internet Coverage and Coverage Bias in Europe: Developments Across Countries and Over Time.” Journal of Official Statistics 29(4): 609–622. DOI: https://doi.org/10.2478/jos-2013-0042.

46.

Pratesi, M., K. Lozar Manfreda, S. Biffignandi, and V. Vehovar. 2004. “List-based Web Surveys: Quality, Timeliness and Nonresponse in the Steps of the Participation Flow.” Journal of Official Statistics 20(3): 451–465. Available at: https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/list-based-web-surveys-quality-timeliness-and-nonresponse-in-the-steps-of-the-participation-flow.pdf.

47.

Rados, J. 2021. “Broadband Access in Rural and Mountainous Areas.” The European Parliament Intergroup on Rural, Mountainous and Remote Areas and Smart Villages. Available at: https://www.smart-rural-intergroup.eu/broadband-access-in-rural-and-mountainous-areas-in-the-eu/ (accessed September 2021).

48.

Reynaud C., S. Miccoli, F. Benassi, A. Naccarato, and L. Salvati. 2020. “Unravelling a Demographic ‘Mosaic’: Spatial Patterns and Contextual Factors of Depopulation in Italian Municipalities, 1981–2011.” Ecological Indicators 115: 106356. DOI: https://doi.org/10.1016/j.ecolind.2020.106356.

49.

Righi P., P.D. Falorsi, S. Daddi, E. Fiorello, P. Massoli, and M.D. Terribili. 2021. “Optimal Sampling for the Population Coverage Survey of the New Italian Register Based Census.” Journal of Official Statistics 37(3): 655–671. DOI: https://doi.org/10.2478/jos-2021-0029.

50.

Rivero, M.S., M.C.R. Rangel, and J.M.S. Martín. 2019. “Geotourist Profile Identification using Binary Logit Modeling: Application to the Villuercas-Ibores-Jara Geopark (Spain).” Geoheritage 11: 1399–1412. DOI: https://doi.org/10.1007/s12371-019-00384-z.

51.

Rizzato, F. 2020. “Broadband Snapshots: Italy”. Opensignal. Available at https://www.opensignal.com/2020/09/04/broadband-snapshots-italy (accessed September 2021).

52.

Santarelli, E., and F. Cottone. 2009. “Leaving Home, Family Support and Intergenerational Ties in Italy: Some Regional Differences.” Demographic Research 21(1): 1–22. DOI: https://doi.org/10.4054/DemRes.2009.21.1.

53.

Scherpenzeel, A.C. 2011. “Data Collection in a Probability-based Internet Panel: How the LISS Panel Was Built and How It Can Be Used.” Bulletin of Sociological Methodology 109(1): 56–61. DOI: https://doi.org/10.1177/0759106310387713.

54.

Scherpenzeel, A.C. and J.G. Bethlehem. 2011. “How Representative are Online Panels?Problems of Coverage and Selection and Possible Solutions.” In Social and Behavioural Research and the Internet: Advances in Applied Methods and Research Strategies, edited by J.W.M. Das, P. Ester, and L. Kaczmirek: 105-132. New York: Routledge.

55.

Tourangeau R. 2017. Presidential Address: Paradoxes of Nonresponse. Public Opinion Quarterly 81: 803–817. DOI: https://doi.org/10.1093/poq/nfx031.

56.

UNECE (United Nations Economic Commission for Europe). 2018. Guidelines on the Use of Registers and Administrative Data for Population and Housing Censuses. ECE/CES/STAT/2018/4. New York: United Nations. Available at: https://unece.org/-guidelines-use-registers-and-administrative-data-population-and-housing-censuses-0 (accessed September 2021).

57.

Wooldridge J.M. 2012. Introductory Econometrics: A Modern Approach. Fifth Edition. Boston: Cengage Learning.