Abstract
With declining response rates and challenges of using RDD sampling for telephone surveys, collecting data from address-based samples has become more attractive. Two approaches are doing telephone interviews at telephone numbers matched to addresses and asking those at sampled addresses to call into an Interactive Voice Response (IVR) system to answer questions. This study used in-person interviewing to evaluate the effects of nonresponse and problems matching telephone numbers when telephone and IVR were used as the initial modes of data collection. The survey questions were selected from major US federal surveys covering a variety of topics. Both nonresponse and, for telephone, inability to find matches result in important nonresponse error for nearly half the measures across all topics, even after adjustments to fit the known demographic characteristics of the residents. Producing credible estimates requires using supplemental data collection strategies to reduce error from nonresponse.
Keywords
References
1.
AAPOR. 2016. “Standard Definitions Final Dispositions of Case Codes and Outcome Rates for Surveys (Revised 2016).” American Association for Public Opinion Research. Available at: https://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf (accessed September 2019).
2.
AAPOR Task Force. 2017. “The Future Of U.S. General Population Telephone Survey Research.” 2017. Available at: https://www.aapor.org/Education-Resources/Reports/The-Future-Of-U-S-General-Population-Telephone-Sur.aspx (accessed September 2019).
3.
American Community Survey (ACS) Available at: https://www.census.gov/programs-surveys/acs/ (accessed June 2020).
4.
Behavioral Risk Factor Surveillance Survey (BRFSS). Available at: https://www.cdc.gov/brfss/index.html (accessed June 2020).
5.
Blumberg, S.J. and J.V. Luke. 2016. “Wireless Substitution: Early Release of Estimates From the National Health Interview Survey, July-December 2016.” Available at: https://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201705.pdf (accessed September 2019).
6.
Current Population Survey (CPS). Available at: https://www.census.gov/programs-surveys/cps.html (accessed June 2020).
7.
Curtin, R., S. Presser, and E. Singer. 2005. “Changes in Telephone Survey Nonresponse over the Past Quarter Century.” Public Opinion Quarterly 69(1):87–98. DOI: https://doi.org/10.1093/poq/nfi002.
8.
De Leeuw, E.D., D.A. Dillman, and J.J. Hox. 2008. “Mixed-Mode Surveys: When and Why.” In International Handbook of Survey Methodology , edited by Edith D. de Leeuw, Joop J. Hox, and Don A. Dillman, 299–311. New York: Erlbaum. Also European Association of Methodology. Available at: https://www.eam-online.org.
9.
De Leeuw, E.D. and W. de Heer. 2002. “Trends in Household Survey Nonresponse: A Longitudinal and International Comparison.” In Survey Nonresponse , edited by Groves Robert M., Don A. Dillman, John L. Eltinge, and Roderick J.A. Little: 41–54. New York: Wiley.
10.
Dillman, D.A., J.D. Smyth, and L.M. Christian. 2014. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method . Hoboken, NJ: John Wiley & Sons., Inc.
11.
Groves, R.M., F.J. Fowler, M.P. Couper, J.M. Lepkowski, E. Singer, and R. Tourangeau. 2009. Survey Methodology . Hoboken, NJ: John Wiley and Sons.
12.
Groves, R.M. 2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys: What Do We Know about the Linkage between Nonresponse Rates and Nonresponse Bias?” Public Opinion Quarterly 70(5):646–675. DOI: https://doi.org/10.1093/poq/nfl033.
13.
Groves, R.M., P.P. Biemer, L.E. Lyberg, J.T. Massey, W.L. Nicholls, and J. Waksberg, eds. 1988. Telephone Survey Methodology . New York, NY: John Wiley & Sons.
14.
Groves, R.M. and E. Peytcheva. 2008. “The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis.” Public Opinion Quarterly 72(2):167 – 189. DOI: https://doi.org/10.1093/poq/nfn011.
15.
Groves, R.M. and R.L. Kahn. 1979. “Surveys by Telephone; a National Comparison with Personal Interviews.” New York, N.Y.: Academic Press.
16.
Hansen, M.H. and W.N. Hurwitz. 1946. “The Problem of Non-Response in Sample Surveys.” Journal of the American Statistical Association , 41(236):517–529. DOI: https://doi.org/10.1080./01621459.1946.10501894.
17.
Health Information National Trends Survey (HINTS). Available at: https://hints.cancer.-gov/ (accessed june 2020).
18.
Keeter, S., C. Kennedy, M. Dimock, J. Best, and P. Craighill. 2006. “Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey.” Public Opinion Quarterly 70(5):759–779. DOI: https://doi.org/10.1093/poq/nfl035.
19.
Keeter, S., C. Miller, A. Kohut, R.M. Groves, and S. Presser. 2000. “Consequences of Reducing Nonresponse in a National Telephone Survey.” Public Opinion Quarterly 64(2):125–148. DOI: https://doi.org/10.1086/317759.
20.
Kohut, A., S. Keeter, C. Doherty, M. Dimock, and L. Christian. 2012. “Assessing the Representativeness of Public Opinion Surveys.” Pew Research Center, Washington, DC . Available at: http://www.people-press.org/2012/05/15/assessing-the-representativeness-of-public-opinion-surveys/ (accessed September 2019).
21.
National Health Interview Survey (NHIS). Available at: https://www.cdc.gov/nchs/nhis/index.htm (accessed June 2020).
22.
National Crime Victimization Survey (NCVS). Available at: https://www.census.gov/programs-surveys/ncvs.html (accessed June 2020).
23.
Tourangeau, R. and T.J. Plewes. 2013. Nonresponse in Social Science Surveys: A Research Agenda – National Research Council, Division of Behavioral and Social Sciences and Education, Committee on National Statistics, Panel on a Research Agenda for the Future of Social Science Data Collection . Washington, DC: The National Academies Press.
24.
Waksberg, J. 1978. “Sampling Methods for Random Digit Dialing.” Journal of the American Statistical Association 73(361):40 – 46. DOI: https://doi.org/10.1080/01621459.1978.10479995.