Sage Journals: Discover world-class research

Abstract

This study examines the effect of telephone survey dialing patterns on lab productivity and survey responses. Using an original data set of paradata from 2010 to 2017 and a machine learning technique for variable selection, we find that early and late afternoon shifts are as productive as late evening shifts for both landline and cellphone Random Digit Dialing (RDD) samples. Also, early weekdays are more productive than the weekend for the cellphone RDD samples. Most importantly, time of the interview affects survey responses; therefore, survey practitioners and scholars should be cognizant of this effect when scheduling calls.

Get full access to this article

View all access options for this article.

References

American Association for Public Opinion Research (AAPOR). 2010. New considerations for survey researchers when planning and conducting RDD telephone surveys in the US with respondents reached via cell phone numbers. Deerfield, IL: American Association for Public Opinion Research.

Blumberg

S. J.

Luke

J. V.

. 2008. Wireless substitution: Early release of estimates from the national health interview survey, July–December 2007. National Center for Health Statistics. https://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless200805.htm (accessed February 3, 2020).

Blumberg

S. J.

Luke

J. V.

. 2018. Wireless substitution: Early release of estimates from the national health interview survey, July–December 2017. National Center for Health Statistics. https://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201806.pdf (accessed February 3, 2020).

Breiman

2001. Random forests. Machine Learning 45:532.

Brick

J. M.

Allen

Cunningham

Maklan

. 1996. Outcomes of a calling protocol in a telephone survey. In Proceedings of the survey research methods section of the American Statistical Association, 142–49. http://www.asasrms.org/Proceedings/y1996f.html (accessed February 3, 2020).

Brick

J. M.

Brick

P. D.

Dipko

Presser

Tucker

Yuan

. 2007. Cell phone survey feasibility in the US: Sampling and calling cell numbers versus landline numbers. Public Opinion Quarterly 71:23–39.

Brick

J. M.

Dipko

Presser

Tucker

Yuan

. 2006. Nonresponse bias in a dual frame sample of cell and landline numbers. International Journal of Public Opinion Quarterly 70:780–93.

Brick

J. M.

Edwards

W. S.

Lee

. 2007. Sampling telephone numbers and adults, interview length, and weighting in the California Health Interview Survey Cell Phone Pilot Study. Public Opinion Quarterly 71:793–813.

Buskirk

T. D.

2018. Surveying the forests and sampling the trees: An overview of classification and regression trees and random forests with applications in survey research. Survey Practice 11:1–15.

10.

Buskirk

T. D.

Kirchner

Eck

Signorino

C. S.

. 2018. An introduction to machine learning methods for survey researchers. Survey Practice 11:1–13.

11.

Carley-Baxter

L. R.

Peytchev

Black

M. C.

. 2010. Comparison of cell phone and landline surveys: A design perspective. Field Methods 22:3–15.

12.

Durrant

G. B.

Groves

R. M.

Staetsky

Steele

. 2010. Effects of interviewer attitudes and behaviors on refusal in household surveys. Public Opinion Quarterly 74:1–36.

13.

Dutwin

Lavrakas

. 2016. Trends in telephone outcomes, 2008–2015. Survey Practice 9:1–9.

14.

Ehlen

. 2007. Cellular-only substitution in the United States as lifestyle adoption: Implications for telephone survey coverage. Public Opinion Quarterly 71:717–33.

15.

Greenberg

B. S.

Stokes

S. L.

. 1990. Developing an optimal call scheduling strategy for a telephone survey. Journal of Official Statistics 6:421–35.

16.

Groves

R. M.

Couper

M. P.

. 1998. Nonresponse in household interview surveys. New York: John Wiley.

17.

Guterbock

Benson

Lavrakas

. 2018. The changing costs of random digital dial cell phone and landline interviewing. Survey Practice 11:3168.

18.

Guterbock

Lavrakas

P. J.

Tompson

T. N.

ZuWallack

. 2011. Cost and productivity ratios in dual-frame RDD telephone surveys. Survey Practice 4:3047.

19.

Hansen

S. E.

2007. CATI sample management systems. In Advances in telephone survey methodology, eds. de Leeuw

Japec

Lavrakas

P. J.

Link

M. W.

Sangster

R. L.

, 340–58. Hoboken, NJ: John Wiley.

20.

Hastie

Friedman

Tibshirani

. 2009. The elements of statistical learning. New York: Springer.

21.

Holbrook

Krosnick

J. A.

Pfent

. 2007. The causes and consequences of response rates in surveys by the news media and government contractor survey research firms. In Advances in telephone survey methodology, eds. de Leeuw

Japec

Lavrakas

P. J.

Link

M. W.

Sangster

R. L.

, 499–528. Hoboken, NJ: John Wiley.

22.

Keeter

Kennedy

Clark

Tompson

Mokrzycki

. 2007. What’s missing from national landline RDD surveys? The impact of the growing cell-only population. Public Opinion Quarterly 71:772–92.

23.

Kennedy

2007. Evaluating the effects of screening for telephone service in dual frame RDD surveys. Public Opinion Quarterly 71:750–71.

24.

Kerin

R. A.

Peterson

R. A.

. 1983. Scheduling telephone interviews: Lessons from 250,000 dialings. Journal of Advertising Research 23:41–47.

25.

Kulka

R. A.

Weeks

M. F.

. 1988. Toward the development of optimal calling protocols for telephone surveys: A conditional probabilities approach. Journal of Official Statistics 4:319–32.

26.

Laflamme

2008. Data collection research using paradata at statistics Canada. In Proceedings from the 2008 Statistics Canada International Symposium on Methodological Issues. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.496.1920&rep=rep1&type=pdf (accessed February 3, 2020).

27.

Lavrakas

P. J.

Benson

Blumberg

Buskirk

Cervantes

I. F.

Christian

Dutwin

, et al. 2017. The future of US general population telephone survey research. AAPOR Task Force Report. https://www.aapor.org/Education-Resources/Reports/The-Future-Of-U-S-General-Population-Telephone-Sur.aspx (accessed February 3, 2020).

28.

Lavrakas

P. J.

Shuttles

C. D.

Steeh

Fienberg

. 2007. The state of surveying cell phone numbers in the United States: 2007 and beyond. Public Opinion Quarterly 71:840–54.

29.

Link

M. W.

Battaglia

M. P.

Frankel

M. R.

Osborn

Mokdad

. 2007. Reaching the U.S. cell phone generation: Comparison of cell phone survey results with an ongoing landline telephone survey. Public Opinion Quarterly 71:814–39.

30.

Massey

J. T.

1996. Optimum calling patterns for random digit dialed telephone surveys. In Proceedings of the Survey Research Methods Section of the American Statistical Association, 485–90. http://www.asasrms.org/Proceedings/papers/1996_081.pdf (accessed February 3, 2020).

31.

Montgomery

Zeng

Khare

Wooten

. 2011. A comparison of cell and land-line dialing patterns. Paper presented at the 65th Annual Conference of the American Association for Public Opinion Research , Phoenix, AZ, July 20.

32.

Reimer

Roth

Montgomery

. 2012. Optimizing call patterns for landline and cell phone surveys. In Proceedings, American Statistical Association, Annual Meeting, vol. 2012, p. 4648. NIH Public Access.

33.

Srinath

K. P.

Battaglia

M. P.

Cardoni

Crawford

Snyder

Wright

R. A.

. 2001. Balancing cost and mean squared error in RDD telephone surveys: The National Immunization Survey. In Proceedings of the Annual Meeting of the American Statistical Association. https://www.cdc.gov/nchs/data/nis/data_collection/srinath2001b.pdf (accessed February 3, 2020).

34.

Stec

J. A.

Lavrakas

P. J.

Shuttles

C. W.

. 2004. Gaining efficiencies in scheduling call-backs in large RDD national surveys. In Proceedings of the Survey Research Methods Section of the American Statistical Association, 4430–37. http://www.asasrms.org/Proceedings/y2004/files/Jsm2004-000680.pdf (accessed February 3, 2020).

35.

Steeh

Buskirk

T. D.

Callegaro

. 2007. Using text messages in U.S. mobile phone surveys. Field Methods 19:59–75.

36.

Steeh

Piekarski

. 2007. Accommodating new technologies: Mobile and VOIP communication. In Advances in telephone survey methodology, eds. de Leeuw

Japec

Lavrakas

P. J.

Link

M. W.

Sangster

R. L.

, 423–46. Hoboken, NJ: John Wiley.

37.

Tucker

Brick

J. M.

Meekins

. 2007. Household telephone service and usage patterns in the United States in 2004: Implications for telephone samples. Public Opinion Quarterly 71:3–22.

38.

Vicente

2017. Exploring fieldwork effects in a mobile CATI survey. International Journal of Market Research 59:57–76.

39.

Vicente

Lopes

. 2015. When should I call you? An analysis of differences in demographics and responses according to respondents’ location in a mobile CATI survey. Social Science Computer Review 33:766–78.

40.

Vicente

Reis

. 2009. Telephone surveys using mobile phones: An analysis of response rates, survey procedures and respondents’ characteristics. Australasian Journal of Market and Social Research 2:49–56.

41.

Wagner

2013. Adaptive contact strategies in a telephone and face-to-face surveys. Survey Research Methods 7:45–55.

42.

Ward

C. D.

Reimer

Khare

Black

. 2013. Home is where the cooperation is: The association between interview location and cooperation among cell-phone users. JSM Proceedings 8:1–12.

43.

Weeks

M. F.

Jones

B. L.

Folsom

R. E.

Jr Benrud

C. H.

. 1980. Optimal times to contact sample households. Public Opinion Quarterly 44:101–14.

44.

Weeks

M. F.

Kulka

R. A.

Pierson

S. A.

. 1987. Optimal call scheduling for a telephone survey. Public Opinion Quarterly 51:540–49.

45.

Zuwallack

2009. Piloting data collection via cell phones: Results, experiences, and lessons learned. Field Methods 21:388–406.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.67 MB

Telephone Survey Calling Patterns,Productivity,Survey Responses,and Their Effect on Measuring Public Opinion

Abstract

Get full access to this article

References

Supplementary Material