Many face-to-face surveys use field staff to create lists of housing units from which samples are selected. However, housing unit listing is vulnerable to errors of undercoverage: Some housing units are missed and have no chance to be selected. Such errors are not routinely measured and documented in survey reports. This study jointly investigates the rate of undercoverage, the correlates of undercoverage, and the bias in survey data due to undercoverage in listed housing unit frames. Working with the National Survey of Family Growth, we estimate an undercoverage rate for traditional listing efforts of 13.6 percent. We find that multiunit status, rural areas, and map difficulties strongly correlate with undercoverage. We find significant bias in estimates of variables such as birth control use, pregnancies, and income. The results have important implications for users of data from surveys based on traditionally listed housing unit frames.
AxinnW. G.LinkC. F.GrovesR. M.. 2011. “Responsive Survey Design, Demographic Data Collection, and Models of Demographic Behavior.”Demography48:1127–49.
2.
BarrettD. F.BeaghenM.SmithD.BurchamJ.. 2002. “Census 2000 Housing Unit Coverage Study.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 146–51. Alexandria, VA: American Statistical Association.
3.
BenderR.LangeS.. 2001. “Adjusting for Multiple Testing—When and How?”Journal of Clinical Epidemiology54:343–49.
4.
BotmanS. L.1987. “Coverage Implications of a Poststratification Adjustment on Estimates from a Household Survey.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 395–400. Alexandria, VA: American Statistical Association.
5.
BoydH. W.WestfallR.. 1955. “Interviewers as a Source of Error in Surveys.”Journal of Marketing19:311–24.
6.
Casas-CorderoC.KreuterF.WangY.BabeyS.. 2013. “Assessing the Measurement Error Properties of Interviewer Observations of Neigh-bourhood Characteristics.”Journal of the Royal Statistical Society, Series A176:227–49.
7.
Central Co-ordinating Team. 2010. “European Social Survey Round 4 2008/2009.”Final Activity Report ESS4e03.0, City University, London, England.
8.
ChildersD. R.1992. “The 1990 Housing Unit Coverage Study.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 506–11. Alexandria, VA: American Statistical Association.
9.
ChildersD. R.1993. “Coverage of Housing in the 1990 Decennial Census.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 635–40. Alexandria, VA: American Statistical Association.
10.
ChromyJ. R.BowmanK. R.CrumpC. J.PackerL. E.PenneM. A.. 1999. “Population Coverage in the National Household Survey on Drug Abuse.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 576–80. Alexandria, VA: American Statistical Association.
11.
CloggC.MassagliM.EliasonS.. 1989. “Population Undercount and Social Science Research.”Social Indicators Research21:559–98.
12.
ColemanJ. S.1994. “A Rational Choice Perspective on Economic Sociology.” Pp. 166–80 in The Handbook of Economic Sociology, edited bySmelserN. J.SwedbergR.. Thousand Oaks, CA: Russel Sage Foundation.
13.
CookP.1985. “The Case of the Missing Victims: Gunshot Woundings in the National Crime Survey.”Journal of Quantitative Criminology1:91–102.
14.
CouperM. P.TourangeauR.MarvinT.. 2009. “Taking the Audio Out of Audio-CASI.”Public Opinion Quarterly73:281–303.
15.
EckmanS.2013. “Do Different Listers Make the Same Housing Unit Frame? Variability in Housing Unit Listing.”Journal of Official Statistics29:249–59.
16.
EckmanS.EnglishN.. 2012. “Creating Housing Unit Frames from Address Databases: Geocoding Precision and Net Coverage Rates.”Field Methods24:399–408.
17.
EckmanS.KreuterF.. 2011. “Confirmation Bias in Housing Unit Listing.”Public Opinion Quarterly75:139–50.
18.
EckmanS.O’MuircheartaighC.. 2011. “Performance of the Half–Open Interval Missed Housing Unit Procedure.”Survey Research Methods5:125–31.
19.
EggersF. J.MoumenF.. 2011. “American Housing Survey Components of Inventory Change: 2007-2009.”Technical report, U.S. Department of Housing & Urban Development Office of Policy Development & Research, Washington, DC.
20.
FayR. E.1989. “An Analysis of Within Household Undercoverage in the Current Population Survey.” In Proceedings of the Bureau of the Census Annual Research Conference, pp. 156–75. Alexandria, VA: Bureau of the Census, Department of Commerce.
21.
FeinD. J.1990. “Racial and Ethnic Differences in U.S. Census Omission Rates.”Demography27:285–302.
22.
FinkR.1963. “Interviewer Training and Supervision in a Survey of Laos.”International Social Science Journal15:21–34.
23.
GrovesR. M.2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.”Public Opinion Quarterly70:646–75.
24.
GrovesR. M.HeeringaS. G.. 2006. “Responsive Design for Household Surveys: Tools for Actively Controlling Survey Errors and Costs.”Journal of the Royal Statistical Society: Series A (Statistics in Society)169:439–57.
25.
GrovesR. M.2009. “Planning and Development of the Continuous National Survey of Family Growth.”Vital Health Statistics1:1–58.
26.
HainerP.1987. “A Brief and Qualitative Anthropological Study Exploring the Reasons for Census Coverage Error among Low Income Black Households.”Report Prepared under Contract with the Census Bureau, Washington, DC.
27.
HainerP.HinesC.MartinE.ShapiroG. M.. 1988. “Research on Improving Coverage in Household Surveys.” In Proceedings of the Fourth Annual Census Bureau Research Conference, U.S. Census Bureau, Washington, DC, pp. 513-39.
28.
HarterR.EckmanS.EnglishN.O’MuircheartaighC.. 2010. “Applied Sampling for Large-scale Multi-stage Area Probability Designs.” Pp. 169–97 in Handbook of Survey Research, edited byMarsdenP. V.WrightJ., 2nd ed. Castle Hill, Australia: Emerald.
29.
HawkesW.1986. “Census Data Quality: A User’s View.”Journal of Official Statistics2:531–44.
30.
HeeringaS. G.WestB. T.BerglundP. A.. 2010. Applied Survey Data Analysis. Ann Arbor, MI: Chapman & Hall/CRC.
31.
HerzogT. N.ScheurenF. J.WinklerW. E.. 2007. Data Quality and Record Linkage Techniques. New York: Springer.
32.
HorriganM.MooreW.PedlowS.WolterK.. 1999. “Undercoverage in a Large National Screening Survey for Youths.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 570–75.
JowellR.and the Central Co-ordinating Team. 2003. “European Social Survey 2002/2003.”Technical Report ESS1e06.1, City University, London, England.
35.
JowellR.and the Central Co-ordinating Team. 2005. “European Social Survey Round 2 2004/2005.”Technical Report ESS2e03.1, City University, London, England.
36.
JowellR. and the Central Co-ordinating Team. 2007. “European Social Survey Round 3 2006/2007.”Technical Report ESS3e03.2, City University, London, England.
37.
JoynerK.PetersH.HynesK.SikoraA.TaberJ.RendallM.. 2012. “The Quality of Male Fertility Data in Major U.S. Surveys.”Demography49:101–24.
38.
JudkinsD.DiGaetanoR.ChuA.ShapiroG.. 1999. “Coverage in Screening Surveys at Westat.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 581–86. Alexandria, VA: American Statistical Association.
39.
KennickellA. B.2000. “Asymmetric Information, Interviewer Behavior, and Unit Nonresponse.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 238–43. Alexandria, VA: American Statistical Association.
40.
KennickellA. B.2003. “Reordering the Darkness: Application of Effort and Unit Nonresponse in the Survey of Consumer Finances.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 2119–26. Alexandria, VA: American Statistical Association.
41.
KishL.1965. Survey Sampling. New York: John Wiley.
42.
KishL.HessI.. 1958. “On Noncoverage of Sample Dwellings.”Journal of the American Statistical Association53:509–24.
43.
KreuterF.ValliantR.. 2007. “A Survey on Survey Statistics: What Is Done and Can be Done in Stata.”Stata Journal7:1–21.
44.
LepkowskiJ. M.MosherW. D.DavisK.GrovesR. M.HoewykJ. V.. 2010. “The 2006–2010 National Survey of Family Growth: Sample Design and Analysis of a Continuous Survey.”Vital Health Statistics2:150.
45.
ManheimerD.HymanH.. 1949. “Interviewer Performance in Area Sampling.”Public Opinion Quarterly13:83–92.
46.
MartinE.1981. “A Twist on the Heisenberg Principle: Or, How Crime Affects Its Measurement.”Social Indicators Research9:197–223.
47.
MartinE.2007. “Strength of Attachment: Survey Coverage of People with Tenuous Ties to Residences.”Demography44:427–40.
48.
MartinezG.DanielsK.ChandraA.. 2012. “Fertility of Men and Woman Aged 15-44 Years in the United States: National Survey of Family Growth, 2006–2010.”Vital Health Statistics1.
49.
MeierF.MooreT.. 1999. “The Effect of Screening on Coverage in the National Health Interview Study.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 566–69. Alexandria, VA: American Statistical Association.
50.
MontaquilaJ. M.MohadjerL.WaksbergJ.KhareM.. 1996. “A Detailed Look at Coverage in the Third National Health and Nutrition Examination Survey.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 532–37. Alexandria, VA: American Statistical Association.
51.
MortonK. B.HunterS. R.ChromyJ. R.MartinP. C.. 2006. “Population Coverage in the National Survey of Drug Use and Health.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 3441–46. Alexandria, VA: American Statistical Association.
52.
MortonK. B.MartinP. C.ChromyJ. R.FosterM.HirschE. L.. 2010. “2009 National Survey on Drug Use and Health Sample Design Report.”Technical report, Substance Abuse and Mental Health Services Administration, Rockville, MD.
53.
O’MuircheartaighC. A.EckmanS. A.WeissC.. 2002. “Traditional and Enhanced Field Listing for Probability Sampling.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 2563–67. Alexandria, VA: American Statistical Association.
54.
O’MuircheartaighC.EnglishN.. 2011. “Eliminating Invisible Boundary Problems in Area Samples, with an Application to the National Children’s Study (NCS).” In Presentation at the American Association for Public Opinion Research Conference. Alexandria, VA: American Statistical Association.
55.
O’MuircheartaighC. A.EnglishE. M.EckmanS. A.. 2007. “Predicting the Relative Quality of Alternative Sampling Frames.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 551–74. Alexandria, VA: American Statistical Association.
56.
O’MuircheartaighC. A.EnglishE. M.EckmanS. A.UpchurchH.GarciaE.LepkowskiJ.. 2006. “Validating a Sampling Revolution: Benchmarking Address Lists against Traditional Listing.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 4189–96. Alexandria, VA: American Statistical Association.
57.
PearsonJ. M.2003. “Quality and Coverage of Listings for Area Sampling.” In Proceedings of the Section on Government Statistics, American Statistical Association. Alexandria, VA: American Statistical Association.
58.
PeytchevA.PeytchevaE.GrovesR. M.. 2010. “Measurement Error, Unit Nonresponse, and Self-reports of Abortion Experiences.”Public Opinion Quarterly74:319–27.
59.
PhilipsonT.LawlessT.. 1997. “Multiple-output Agency Incentives in Data Production: Experimental Evidence.”European Economic Review41:961–70.
60.
SchnellR.2008. “Avoiding Problems of Traditional Sampling Strategies for Household Surveys in Germany: Some New Suggestions.”Technical report, Deutsches Institut für Wirtschaftsforschung, Germany.
61.
ShapiroG.DiffendalG.CantorD.. 1993. “Survey Undercoverage: Major Causes and New Estimates of Magnitude.” In Paper delivered at the US Bureau of the Census Annual Research Conference, Washington, DC.
62.
ShapiroG. M.KostanichD.. 1988. “High Response Error and Poor Coverage are Severely Hurting the Value of Household Survey Data.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 443–48. Alexandria, VA: American Statistical Association.
63.
SingerE.2011. “Toward a Benefit-cost Theory of Survey Participation: Evidence, Further Tests, and Implications.”Journal of Official Statistics27:379–92.
64.
ThompsonG.TurmelleC.. 2004. “Classification of Address Register Coverage Rates: A Field Study.” In Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 4477–84. Alexandria, VA: American Statistical Association.
65.
TourangeauR.KreuterF.EckmanS.. 2012. “Motivated Underreporting in Screening Interviews.”Public Opinion Quarterly76:453–69.
66.
TourangeauR.ShapiroG.KearneyA.ErnstL.. 1997. “Who Lives Here? Survey Undercoverage and Household Roster Questions.”Journal of Official Statistics13:1–18.
67.
U.S. Census Bureau. 1993. “Programs to Improve Coverage in the 1990 Census.”Technical Report. 1990 CPH-E-3, Washington, DC.
68.
U.S. Census Bureau. 2006. “Technical Paper 66: Design and Methodology, Current Population Survey.”Technical Report, Washington, DC.
69.
ValentineC.ValentineB.. 1971. “Missing Men: A Comparative Methodological Study of Underenumeration and Related Problems.”Report prepared under contract for the US Census Bureau, Washington, DC.
70.
ZandbergenP. A.2009. “Geocoding Quality and Implications for Spatial Analysis.”Geography Compass3:647–80.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
