Random digit dialed telephone surveys are facing two serious problems undermining probability-based inference and creating a potential for bias in survey estimates: declining response rates and declining coverage of the landline telephone frame. Optimum survey designs need to focus reduction techniques on errors that cannot be addressed through statistical adjustment. This requires (a) separating and estimating the relative magnitude of different error sources and (b) evaluating the degree to which each error source can be statistically adjusted. In this study, the authors found significant differences in means both for nonrespondents and for the eligible population excluded from the landline frame, which are also in opposite directions. Differences were also found for element variances and associations, which can affect survey results but are rarely examined. Adjustments were somewhat effective in decreasing both sources of bias, although addressing at least one through data collection led to less bias in the adjusted estimates.
American Association for Public Opinion Research (AAPOR).2008. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. Lenexa, KS: AAPOR.
2.
Belinfante, Alexander.1997. ‘‘Telephone Subscribership in the United States.’’Washington, DC: Federal Communications Commission.
3.
Blumberg, Stephen J. and Julian V. Luke.2007a. ‘‘Coverage Bias in Traditional Telephone Surveys of Low-income and Young Adults.’’Public Opinion Quarterly71:734-49.
4.
Blumberg, Stephen J. and Julian V. Luke.2007b. ‘‘Wireless Substitution: Early Release of Estimates Based on Data From the National Health Interview Survey, January-June 2007.’’ Retrieved January 10, 2008 (http://www.cdc.gov/nchs/nhis.htm).
5.
Blumberg, Stephen J. and Julian V. Luke.2007c. ‘‘Wireless Substitution: Early Release of Estimates Based on Data From the National Health Interview Survey, July-December 2006.’’ Retrieved December 10, 2007 (http://www.cdc.gov/nchs/nhis.htm).
6.
Blumberg, Stephen J. and Julian V. Luke.2008. ‘‘Wireless Substitution: Early Release of Estimates Based on Data From the National Health Interview Survey, January-June 2008.’’ Retrieved December 17, 2008 (http://www.cdc.gov/nchs/nhis.htm).
7.
Boyle, John, Michael Bucuvalas, Linda Piekarski, and Andy Weiss.2009. ‘‘ Zero Banks: Coverage Error in List Assisted RDD Samples.’’Survey Practice . Retrieved November 16, 2010 (http://surveypractice.org/2009/01/29/zero-banks-coverage-error-in-list-assisted-rdd-samples/ ).
8.
Brick, J. Michael, Sarah Dipko, Stanley Presser, Clyde Tucker, and Yangyang Yuan.2006. ‘‘Nonresponse Bias in a Dual Fame Sample of Cell and Landline Number.’’Public Opinion Quarterly70:780-93.
9.
Brick, J. Michael, Joseph Waksberg, Dale Kulp, and Amy Starer.1995. ‘‘Bias in List-assisted Telephone Samples.’’ Public Opinion Quarterly59:218-35.
10.
Carley-Baxter, Lisa, Michele Lynberg Black, and Susan Twiddy.2007. ‘‘The Impact of Incentives on Survey Participation and Reports of Intimate Partner and Sexual Violence.’’Presented at the American Association for Public Opinion Research annual conference, May 17, Anaheim, CA .
11.
Carley-Baxter, Lisa R., Andy Peytchev, and Michele Lynberg Black.2010. ‘‘Comparison of Cell Phone and Landline Surveys: A Design Perspective.’’Field Methods22:3-15.
12.
Centers for Disease Control and Prevention.1999. ‘‘1999 Behavioral Risk Factor Surveillance System Summary Data Quality Report.’’Atlanta, GA: National Center for Chronic Disease Prevention and Health Promotion.
13.
Centers for Disease Control and Prevention.2007. ‘‘2006 Behavioral Risk Factor Surveillance System Summary Data Quality Report.’’Atlanta, GA: National Center for Chronic Disease Prevention and Health Promotion.
14.
Curtin, Richard, Stanley Presser, and Eleanor Singer.2000. ‘‘The Effects of Response Rate Changes on the Index of Consumer Sentiment.’’ Public Opinion Quarterly64:413-28.
15.
Curtin, Richard, Stanley Presser, and Eleanor Singer.2005. ‘‘Changes in Telephone Survey Nonresponse Over the Past Quarter Century.’’ Public Opinion Quarterly69:87-98.
16.
de Leeuw, Edith and Wim de Heer.2002. ‘‘Trends in Household Survey Nonresponse: A Longitudinal and International Comparison.’’ Pp. 41-54 in Survey Nonresponse, edited by R. Groves, D. Dillman, J. Eltinge, and R. J. A. Little.New York: John Wiley.
17.
Fahimi, Mansour, Dale Kulp, and J. Michael Brick .2008. ‘‘Bias in List-assisted 100-series RDD Sampling.’’Survey Practice. Retrieved November 16, 2010 (http://surveypractice.org/2008/09/25/bias-in-list-assisted-100-series-rdd-sampling/ ).
18.
Folsom, Ralph E. and Avi C. Singh.2000. ‘‘The Generalized Exponential Model for Sampling Weight Calibration for Extreme Values, Nonresponse, and Poststratification.’’ Pp. 598-603 in Joint Statistical Meetings of the American Statistical Association. Alexandria, VA: American Statistical Association.
19.
Groves, Robert M.2006. ‘‘Nonresponse Rates and Nonresponse Bias in Household Surveys.’’Public Opinion Quarterly70:646-75.
20.
Groves, Robert M. and Lars E. Lyberg.1988. ‘‘An Overview of Nonresponse Issues in Telephone Surveys.’’ Pp. 191-211 in Telephone Survey Methodology, edited by R. M. Groves, P. P. Biemer, L. E. Lyberg, J. T. Massey, W. L. Nicholls II, and J. Waksberg, NY: John Wiley.
21.
Groves, Robert M., Stanley Presser, and Sarah Dipko.2004. ‘‘The Role of Topic Interest in Survey Participation Decisions.’’Public Opinion Quarterly68:2-31.
22.
Hartley, Herman O.1962. ‘‘Multiple Frame Surveys.’’ Pp. 203-206 in American Statistical Association, Section on Social Statistics. Alexandria, VA: American Statistical Association.
23.
Izrael, David, David C. Hoaglin, and Michael P. Battaglia .2004. ‘‘To Rake or Not to Rake Is Not the Question Anymore With the Enhanced Raking Macro.’’Presented at the 29th Annual SAS Users Group International Conference, May 9-12, 2004.
24.
Keeter, Scott, Catherine Miller, Andrew Kohut, Robert M. Groves, and Stanley Presser .2000. ‘‘Consequences of Reducing Nonresponse in a National Telephone Survey.’’Public Opinion Quarterly64:125-48.
25.
Kuusela, Vesa, Vasja Vehovar, and Mario Callegaro .2007. ‘‘Mobile Phones’ Influence on Telephone Surveys.’’ Pp. 87-112 in Advances in Telephone Survey Methodology, edited by J. M. Lepkowski, C. Tucker, J. M. Brick, E. D. D. Leeuw, L., Japec, P. J. Lavrakas, M. W. Link, and R. L. Sangster. New York: John Wiley.
26.
Lepkowski, James M. and Mick P. Couper.2002. ‘‘Nonresponse in the Second Wave of Longitudinal Household Surveys.’’ Pp. 259-72 in Survey Nonresponse , edited by R. Groves, D. Dillman, J. Eltinge, and R. J. A. Little.New York: John Wiley.
27.
Lin, I-Fen and Nora Cate Schaeffer.1995. ‘‘Using Survey Participants to Estimate the Impact of Nonparticipation.’’ Public Opinion Quarterly59:236-58.
28.
Lynberg Black, Michele, Lisa Carley-Baxter, and Susan Twiddy.2007. ‘‘Does the Introductory Context Affect Reporting of Victimization and Perpetration in a National Survey of Intimate Partner Violence or Sexual Violence?’’Presented at the American Association for Public Opinion Research annual conference, May 17, Anaheim, CA.
29.
McCarty, Christopher, Mark House, Jeffrey Harman, and Scott Richards.2006. ‘‘Effort in Phone Survey Response Rates: The Effects of Vendor and Client-controlled Factors.’’Field Methods18:172-88.
30.
National Center for Health Statistics.2007 . ‘‘National Immunization Survey: A User’s Guide for the 2006 Public-use Data File.’’Hyattsville, MD: National Center for Health Statistics .
31.
Raghunathan, Trivellore E., James M. Lepkowski, John Van Hoewyk, and Peter Solenberger .2001. ‘‘A Multivariate Technique for Multiply Imputing Missing Values Using a Sequence of Regression Models.’’Survey Methodology27:85-95.
32.
Singer, Eleanor.2002. ‘‘The Use of Incentives to Reduce Nonresponse in Household Surveys.’’ Pp. 163-77 in Survey Nonresponse , edited by R. M. Groves, D. A. Dillman, J. L. Eltinge , and R. J. A. Little.New York: John Wiley.
33.
Singer, Eleanor, John Van Hoewyk, and Mary P. Maher.2000. ‘‘Experiments With Incentives in Telephone Surveys.’’Public Opinion Quarterly64:171-88.
34.
Tjaden, Patricia and Nancy Thoennes.2000. Extent, Nature, and Consequences of Intimate Partner Violence: Findings From the National Violence Against Women Survey. Report NCJ 181867 , National Institute of Justice and the Centers for Disease Control and Prevention.
35.
Tjaden, Patricia and Nancy Thoennes.2006. Extent, Nature, and Consequences of Rape Victimization: Findings From the National Violence Against Women Survey. Report NCJ210346, National Institute of Justice and the Centers for Disease Control and Prevention.
36.
Tobin, J.1958. ‘‘Estimation of Relationships for Limited Dependent Variables.’’ Econometrica26:24-36.
37.
Tucker, N. Clyde, J. Michael Brick, and Brian Meekins.2007. ‘‘Household Telephone Service and Usage Patterns in the United States in 2004: Implications for Telephone Samples.’’Public Opinion Quarterly71:3-22.
38.
Waksberg, Joseph.1978. ‘‘Sampling Methods for Random Digit Dialing .’’ Journal of the American Statistical Association73:40-46.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
