Survey research is still confronted by a trend of increasing nonresponse rates. In this context, several methodological advances have been made to stimulate participation and avoid bias. Yet, despite the growing number of tools and methods to deal with nonresponse, little is known about whether nonresponse biases show similar trends as nonresponse rates and what mechanisms (if any) drive changes in bias. Our article focuses on biases in cohort distributions in the U.S. and German general social surveys from 1980 to 2012 as one of the key variables in the social sciences. To supplement our cross-national comparison of these trends, we decompose changes into within-cohort change (WCC) and between-cohort change. We find that biases in cohort distributions have remained relatively stable and at a relatively low level in both countries. Furthermore, WCC (i.e., survey climate) accounts for the major part of the change in nonresponse bias.
American Association for Public Opinion Research. 2015. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 8th Ed. Oakbrook Terrace, IL: American Association for Public Opinion Research.
2.
AtrosticBarbara K.BatesNancyBurtGeraldineSilbersteinAdriana. 2001. “Nonresponse in US Government Household Surveys: Consistent Measures, Recent Trends, and New Insights.” Journal of Official Statistics17:209–26.
3.
BethlehemJelke G.2002. “Weighting Nonresponse Adjustments Based on Auxiliary Information.” Pp. 275–88 in Survey Nonresponse, edited by GrovesR. M.DillmanD. A.EltingeJ. L.LittleR. J. A.. New York: John Wiley.
4.
BlohmMichaelKochAchim. 2013. “Respondent Incentives in a National Face-to-face Survey.” Methoden, Daten, Analysen7:89–122.
5.
BreenRichard. 2004. “Conclusions.” Pp. 383–410 in Social Mobility in Europe, edited by BreenR.. Oxford, UK: Oxford University Press.
6.
BreenRichardLuijkxRuud. 2004. “Social Mobility in Europe between 1970 and 2000.” Pp. 37–76 in Social Mobility in Europe, edited by BreenR.. Oxford, UK: Oxford University Press.
7.
BrickJ. MichaelDipkoSarahPresserStanleyTuckerClydeYuanYangyang. 2006. “Nonresponse Bias in a Dual Frame Sample of Cell and Landline Numbers.” Public Opinion Quarterly70:780–93.
8.
BrickJ. MichaelWilliamsDouglas. 2013. “Explaining Rising Nonresponse Rates in Cross-sectional Surveys.” The Annals of the American Academy of Political and Social Science645:36–59.
9.
CurtinRichardPresserStanleySingerEleanor. 2005. “Changes in Telephone Survey Nonresponse over the Past Quarter Century.” Public Opinion Quarterly69:87–98.
10.
Das GuptaPrithwis. 1978. “A General Method of Decomposing a Difference between Two Rates into Several Components.” Demography15:99–112.
11.
DavidovEldad. 2011. “Nationalism and Constructive Patriotism: A Longitudinal Test of Comparability in 22 Countries with the ISSP.” International Journal of Public Opinion Research23:88–103.
12.
de LeeuwEdith. 2005. “To Mix or Not to Mix Data Collection Modes in Surveys.” Journal of Official Statistics21:233–55.
13.
de LeeuwEdithHeerWim de. 2002. “Trends in Household Survey Nonresponse: A Longitudinal and International Comparison.” Pp. 41–54 in Survey Nonresponse, edited by GrovesR. M.DillmanD. A.EltingeJ. L.LittleR. J. A.. New York: John Wiley.
14.
DillmanDon A.PhelpsGlennTortoraRobertSwiftKarenKohrellJulieBerckJodiMesserBenjamin L.. 2009. “Response Rate and Measurement Differences in Mixed-mode Surveys Using Mail, Telephone, Interactive Voice Response (IVR) and the Internet.” Social Science Research38:1–18.
15.
DillmanDon A.SmythJolene D.ChristianLeah M.. 2014. Internet, Phone, Mail, and Mixed-mode Surveys: The Tailored Design Method. Hoboken, NJ: John Wiley.
16.
DuncanOtis D.DuncanBeverly. 1955. “A Methodological Analysis of Segregation Indexes.” American Sociological Review20:210–17.
17.
FirebaughGlenn. 1990. “Replacement Effects, Cohort and Otherwise: Response to Rodgers.” Pp. 439–46 in Sociological Methodology, edited by CloggC. C.. Oxford, UK: Basil Blackwell.
GrovesRobert M.2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly70:646–75.
21.
GrovesRobert M.CouperMick P.. 1998. Nonresponse in Household Interview Surveys. New York: John Wiley.
22.
GrovesRobert M.HeeringaSteven G.. 2006. “Responsive Design for Household Surveys: Tools for Actively Controlling Survey Errors and Costs.” Journal of the Royal Statistical Society169:439–57.
23.
GrovesRobert M.PeytchevaEmilia. 2008. “The Impact of Nonresponse Rates on Nonresponse Bias.” Public Opinion Quarterly72:167–89.
24.
GrovesRobert M.SingerEleanorCorningAmy. 2000. “Leverage-saliency Theory of Survey Participation.” Public Opinion Quarterly61:299–308.
25.
HäderMichael. 2015. Empirische Sozialforschung. Eine Einführung. Wiesbaden, Germany: Springer VS.
26.
Harris-KojetinBrianTuckerClyde. 1999. “Exploring the Relation of Economic and Political Conditions with Refusal Rates to a Government Survey.” Journal of Official Statistics15:167–84.
27.
KaltonGrahamFlores-CervantesIsmael. 2003. “Weighting Methods.” Journal of Official Statistics19:81–97.
28.
KeeterScottMillerCarolynKohutAndrewGrovesRobert M.PresserStanley. 2000. “Consequences of Reducing Nonresponse in a National Telephone Survey.” Public Opinion Quarterly64:125–48.
29.
KitagawaEvelyn M.1955. “Components of a Difference between Two Rates.” Journal of the American Statistical Association50:1168–94.
30.
KreuterFrauke. 2013. “Facing the Nonresponse Challenge.” The Annals of the American Academy of Political and Social Science645:23–35.
KreuterFraukeOlsonKristenWagnerJamesYanTingEzzati-RiceTrena M.Casas-CorderoCarolinaLemayMichaelPeytchevAndyGrovesRobert M.RaghunathanTrivellore E.. 2010. “Using Proxy Measures and Other Correlates of Survey Outcomes to Adjust for Non-response: Examples from Multiple Surveys.” Journal of the Royal Statistical Society: Series A173:389–407.
33.
LinkMichael W.OldendickRobert W.. 1999. “Call Screening: Is It Really a Problem for Survey Research?” Public Opinion Quarterly63:577–89.
34.
MeulemanBartDavidovEldadBillietJaak. 2009. “Changing Attitudes toward Immigration in Europe, 2002-2007. A Dynamic Group Conflict Theory Approach.” Social Science Research38:352–65.
35.
PeytchevAndy. 2013. “Consequences of Survey Nonresponse.” The Annals of the American Academy of Political and Social Science645:88–111.
36.
PeytchevAndyRileySarahRosenJeffreyMurphyJoeLindbladMark. 2010. “Reduction of Nonresponse Bias in Surveys through Case Prioritization.” Survey Research Methods4:21–29.
37.
PeytchevaEmiliaGrovesRobert M.. 2009. “Using Variation in Response Rates of Demographic Subgroups as Evidence of Nonresponse Bias in Survey Estimates.” Journal of Official Statistics25:193–201.
38.
PforrKlausBlohmMichaelBlomAnnelies G.ErdelBarbaraFeldererBarbaraFräßdorfMathisHajekKristinHelmschrottSusanneKleinertCorinnaKochAchimKriegerUlrichKrohMartinMartinSilkeSaßenrothDeniseSchmiedebergClaudiaTrüdingerEva-MariaRammstedtBeatrice. 2015. “Are Incentive Effects on Response Rates and Nonresponse Bias in Large-scale, Face-to-face Surveys Generalizable to Germany? Evidence from Ten Experiments.” Public Opinion Quarterly79:740–68.
39.
PresserStanleyMcCullochSusan. 2011. “The Growth of Survey Research in the United States: Government-Sponsored Surveys, 1984–2004.” Social Science Research40:1019–24.
40.
RevillaMelanie. 2010. “Quality in Unimode and Mixed-mode Designs: A Multitrait-multimethod Approach.” Survey Research Methods4:151–64.
41.
RodgersWillard L.1990. “Interpreting the Components of Time Trends.” Pp. 421–38 in Sociological Methodology, edited by CloggC. C.. Oxford, UK: Basil Blackwell.
42.
RyderNorman B.1965. “The Cohort as a Concept in the Study of Social Change.” American Sociological Review30:843–61.
43.
SchoutenBarryCobbenFannieBethlehemJelke. 2009. “Indicators for the Representativeness of Survey Response.” Survey Methodology35:101–13.
44.
SingerEleanorMathiowetzNancy A.CouperMick P.. 1993. “The Impact of Privacy and Confidentiality Concerns on Survey Participation the Case of the 1990 US Census.” Public Opinion Quarterly57:465–82.
45.
SingerEleanorPresserStanley. 2008. “Privacy, Confidentiality, and Respondent Burden as Factors in Telephone Survey Nonresponse.” Pp. 449–70 in Advances in Telephone Survey Methodology, edited by LepkowskiJ. M.TuckerC.BrickJ. M.de LeeuwE.JapecL.LavrakasP. J.LinkM. W.SangsterR. L.. New York: John Wiley.
46.
SingerEleanorYeCong. 2013. “The Use and Effect of Incentives in Surveys.” The Annals of the American Academy of Political and Social Science645:112–41.
47.
SmithTom W.MarsdenPeter V.HoutMichael. 2015. General Social Surveys, 1972-2014: Cumulative Codebook. Chicago, IL: National Opinion Research Center.
48.
SteehCharlotteKirgisNicoleCannonBrianDeWittJeff. 2001. “Are They Really as Bad as They Seem? Nonresponse Rates at the End of the Twentieth Century.” Journal of Official Statistics17:227–48.
49.
StephensonC. Bruce. 1978. Weighting the General Social Surveys for Bias Related to Household Size. Chicago, IL: NORC.
50.
StoopInekeBillietJaakKochAchimFitzgeraldRory. 2010. Improving Survey Response. Lessons Learned from the European Social Survey. Chichester, UK: John Wiley.
51.
TourangeauRoger. 2004. “Survey Research and Societal Change.” Annual Review of Psychology55:775–801.
52.
YangYangLandKenneth C.. 2006. “A Mixed Models Approach to the Age-period-cohort Analysis of Repeated Cross-section Surveys: Trends in Verbal Test Scores.” Sociological Methodology36:75–97.
53.
YangYangLandKenneth C.. 2013. Age-period-cohort Analysis. New Models, Methods, and Empirical Applications. Boca Raton, FL: CRC Press.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
