Do Initial Respondents Differ From Callback Respondents? Lessons From a Mobile CATI Survey

Abstract

Mobile phones are increasingly being used to conduct mobile computer-assisted telephone interview (CATI) surveys and some say it is just a question of time before they replace fixed phones. Although mobile phones allow their users to be called anywhere and at any time, a 100% response rate (RR) is unlikely to be achieved with a single call attempt. Callbacks are costly and delay fieldwork completion, but they are worth the effort when better overall RRs as well as those for specific subgroups of the population are obtained. Using data collected on a nationwide mobile CATI survey, this article investigates the differences between initial and callback respondents. Findings reveal that although the majority of interviews were achieved with a single call attempt, the initial sample differed from the callback sample in terms of the age and residence of respondents. Additionally, callback respondents were more likely to be interviewed outside home.

Keywords

mobile phones CATI surveys callbacks two-level ordered logit model

Get full access to this article

View all access options for this article.

References

American Association for Public Opinion Research. (2008). AAPOR cell phone task force: Guidelines and considerations for survey researchers when planning and conducting RDD and other telephone surveys in the US with respondents reached via cell phone numbers. Retrieved from http://www.aapor.org/uploads/Final_AAPOR_Cell_Phone_TF_report_041208.pdf

American Association for Public Opinion Research. (2010). Cell phone task force report. Retrieved from http://www.aapor.org/Cell_Phone_Task_Force_Report.htm

Autoridade Nacional das Comunicações. (2014). O Sector das Comunicações ‘14. Retrieved from http://www.anacom.pt/streaming/Sector_Comunicacoes_2014.pdf?contentId=1358849&field=ATTACHED_FILE

Battaglia

Khare

Frankel

Murray

Buckley

Peritz

(2007). Response rates: How have they changed and where are they headed? In Lepkowski

Tucker

Brick

de Leeuw

Japec

Lavrakas

Link

Sangster

(Eds.), Advances in telephone survey methodology (pp. 529–560). Hoboken, NJ: Wiley.

Biemer

Link

(2008). Evaluating and modeling early cooperator effects in RDD surveys. In Lepkowski

Tucker

Brick

de Leeuw

Japec

Lavrakas

Link

Sangster

(Eds.), Advances in telephone survey methodology (pp. 587–618). Hoboken, NJ: Wiley.

Boase

Ling

(2013). Measuring mobile phone use: Self-report versus log data. Journal of Computer-Mediated Communication, 18, 508–519.

Brick

Dipko

Presser

Tucker

Yuan

(2007). Cell phone survey feasibility in the US: Sampling and calling cell numbers versus landline numbers. Public Opinion Quarterly, 71, 23–29.

Brick

Edwards

Lee

(2007). Sampling telephone numbers and adults, interview length, and weighting in the California Health Interview Survey Cell phone pilot study. Public Opinion Quarterly, 71, 793–813.

Carley-Baxter

Peytchev

Black

(2010). Comparison of cell phone and landline surveys: A design perspective. Fields Methods, 22, 3–15.

10.

Couper

(2013). Is the sky falling? New technology, changing media, and the future of surveys. Survey Research Methods, 7, 145–156.

11.

Cunningham

Martin

Brick

(2003). An experiment in call scheduling. Paper presented at the Annual Meeting of the American Statistical Association, San Francisco, CA, August 3–7. Retrieved from http://www.amstat.org/sections/srms/proceedings/y2003/Files/JSM2003-000306.pdf

12.

Durrant

Steele

(2009). Multilevel modelling of refusal and noncontact nonresponse in household surveys: Evidence from six UK Government surveys. Journal of the Royal Statistical Society, Series A, 172, 361–381.

13.

European Society for Opinion and Marketing Research. (2011). ESOMAR Guideline for conducting survey research via mobile phone. Retrieved from http://www.esomar.org

14.

European Commission. (2004). How Europeans spend their time: Everyday like of women and men. Retrieved May 2014, from http://epp.eurostat.ec.europa.eu/cache/ITY_OFFPUB/KS-58-04-998/EN/KS-58-04-998-EN.PDF

15.

European Commission. (2008). Flash Eurobarometer 241: Information Society as seen by EU citizens. Brussels, Belgium: Author.

16.

European Commission. (2014). Special Eurobarometer 414: E-Communications Household Survey and Telecom single market survey roaming results. Brussels, Belgium: Author.

17.

Goyder

Leiper

(1985). The decline in survey response: A social values interpretation. Sociology, 19, 55–71.

18.

Groves

Couper

(1998). Nonresponse in household interview surveys. New York, NY: Wiley.

19.

Groves

Fowler

Jr. Couper

Lepkowski

Singer

Tourangeau

(2004). Survey methodology. New York: Wiley.

20.

Häder

(2012). Data quality in telephone surveys via mobile and landline phone. In Häder

Häder

Kühne

(Eds.), Telephone surveys in Europe (pp. 247–264). New York, NY: Springer.

21.

Hansen

(2008). CATI sample management systems. In Lepkowski

Tucker

Brick

de Leeuw

Japec

Lavrakas

Link

Sangster

(Eds.), Advances in telephone survey methodology (pp. 340–358). Hoboken, NJ: Wiley.

22.

Holbrook

Krosnick

Pfent

(2008). The causes and consequences of response rates in surveys by the News Media and Government Contractor Survey Research Firms. In Lepkowski

Tucker

Brick

de Leeuw

Japec

Lavrakas

Link

Sangster

(Eds.), Advances in telephone survey methodology (pp. 499–528). Hoboken, NJ: Wiley.

23.

Johnson

Cho

(2004). Understanding nonresponse mechanisms in telephone surveys. In Proceedings of the Section on Survey Research Methods (pp. 4952–4959). Alexandria, VA: American Statistical Association. Retrieved May 2016, from https://www.amstat.org/sections/srms/Proceedings/y2004/files/Jsm2004-000246.pdf

24.

Kristal

White

Davis

Corycell

Raghunathan

Kinne

Lin

(1993). Effects of enhanced calling efforts on response rates, estimates of health behavior, and costs in a telephone health survey using random-digit dialing. Public Health Reports, 108, 372–379.

25.

Kühne

Häder

(2012). Telephone surveys via landline and mobile phones: Mode effects and response quality. In Häder

Häder

Kühne

(Eds.), Telephone surveys in Europe (pp. 229–246). New York, NY: Springer.

26.

Kuusela

Simpanen

(2002). Effects of mobile phones on telephone survey practices and results. Paper presented at the International Conference on Intelligent Computing, Copenhagen, Denmark.

27.

Lavrakas

Tompson

Benford

Fleury

(2010). Investigating data quality in cell phone surveying. Paper presented at the 65th annual conference of the American Association for Public Opinion Research, Chicago, IL.

28.

Link

Battaglia

Frankel

Osborn

Mokdad

(2007). Reaching the U.S. cell phone generation: Comparison of cell phone survey results with an ongoing landline telephone survey. Public Opinion Quarterly, 71, 814–839.

29.

Merkle

Bauman

Lavrakas

(1993). The impact of callbacks on survey estimates in an annual RDD survey. In Proceedings of the section on survey research methods (pp. 1070–1075). Alexandria, VA: American Statistical Association. Retrieved May 2016, from http://www.amstat.org/sections/srms/Proceedings/papers/1993_183.pdf

30.

Montaquila

Brick

Hagedorn

Kennedy

Keeter

(2008). Aspects of nonresponse bias in RDD telephone surveys. In Lepkowski

Tucker

Brick

de Leeuw

Japec

Lavrakas

Link

Sangster

(Eds.), Advances in telephone survey methodology (pp. 561–586). Hoboken, NJ: Wiley.

31.

Ofcom. (2013). Mobile phone usage: Attitudes towards mobile phone functions including reception. Retrieved October 2014, from http://stakeholders.ofcom.org.uk/binaries/consultations/mobile-voice-data-experience/annexes/usage.pdf

32.

Potthoff

Manton

Woodbury

(1993). Correcting for nonavailability bias in surveys by weighting based on number of callbacks. Journal of the American Statistical Association, 88, 1197–1207.

33.

Poynter

Williams

York

(2014). The handbook of mobile market research. Chichester, England: Wiley.

34.

Qayad

Chowdhury

Balluz

(2010). Respondent differences and length of data collection in the Behavioral Risk Factor Surveillance System. Survey Methodology, 36, 223–227.

35.

Raudenbush

Bryk

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.

36.

Reimer

Roth

Montgomery

(2012). Optimizing call patterns for landline and cell phone surveys. In Proceedings of the section on survey research methods (pp. 4648–4660). Alexandria, VA: American Statistical Association.

37.

Roy

Vanheuverzwyn

(2002). Mobile phone in sample surveys. Paper presented at the International Conference on Intelligent Computing, Copenhagen, Denmark.

38.

Shaiko

Dwyre

O’Gorman

Stonecash

Vike

(1991). Pre-election political polling and the non-response bias issue. International Journal of Public Opinion Research, 3, 86–99.

39.

Snijders

Bosker

(2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). London, England: Sage.

40.

Statistics Portugal. (2013). Survey on the use of ICTs by families. Portugal: Author.

41.

Steeh

Piekarski

(2008). Accommodating new technologies: Mobile and VoIP communication. In Lepkowski

Tucker

Brick

de Leeuw

Japec

Lavrakas

Link

Sangster

(Eds.), Advances in telephone survey methodology (pp. 423–446). Hoboken, NJ: Wiley.

42.

Traugott

(1987). The importance of persistence in respondent selection for preelection surveys. Public Opinion Quarterly, 51, 48–57.

43.

Vehovar

Belak

Batagelj

Cikic

(2004). Mobile phone surveys: The Slovenian case study. Metodoloski zvezki, 1, 1–19.

44.

Vicente

(2015). The best times to call in a mobile phone survey. International Journal of Market Research, 56, 1–16. doi:10.2501/IJMR-2014-047

45.

Vicente

Lopes

(2015). When should I call you? An analysis of differences in demographics and responses according to respondents’ location in a mobile CATI survey. Social Science Computer Review, 33, 766–778. doi:10.1177/0894439314560696

46.

Vicente

Reis

(2009). Telephone surveys using mobile phones: An analysis of response rates, survey procedures and respondents’ characteristics. Australasian Journal of Market and Social Research, 17, 49–56.

47.

Vicente

Reis

Santos

(2009). Using mobile phones for survey research: A comparison with fixed phones. International Journal of Market Research, 51, 613–633.

48.

Weidman

(2010). Do characteristics of RDD survey respondents differ according to difficulty of obtaining response? In Proceedings of the section on survey research methods (pp. 3956–3965). Alexandria, VA: American Statistical Association. Retrieved May 2016, from https://www.amstat.org/sections/srms/proceedings/y2010/Files/308040_59913.pdf

49.

Yuan

Allen

Brick

J. M.

Dipko

Presser

Tucker

(2005). Surveying households on cell phones—Results and lessons. In Proceedings of the section on survey research methods. Alexandria, VA: American Statistical Association. Retrieved May 2016, from http://www.bls.gov/ore/pdf/st050280.pdf

50.

ZuWallack

(2009). Piloting data collection via cell phones: Results, experiences and lessons learned. Fields Methods, 21, 388–406.