Abstract
The U.S. Consumer Expenditure Interview Survey asks many filter questions to identify the items that households purchase. Each reported purchase triggers follow-up questions about the amount spent and other details. We test the hypothesis that respondents learn how the questionnaire is structured and underreport purchases in later waves to reduce the length of the interview. We analyze data from 10,416 four-wave respondents over two years of data collection. We find no evidence of decreasing data quality over time; instead, panel respondents tend to give higher quality responses in later waves. The results also hold for a larger set of two-wave respondents.
References
1.
Allaire, J.J., J. Horner, Y. Xie, V. Marti, and N. Porte. 2019. markdown: Render Markdown with the C Library ’Sundown’ . R package version 1.1. Available at: https://CRAN.R-project.org/package=markdown (accessed November 2020).
2.
Angel, S., R. Heuberger, and N. Lamei. 2017. “Differences Between Household Income from Surveys and Registers and How These Affect the Poverty Headcount: Evidence from the Austrian SILC.” Social Indicators Research, 138(2):575–603. DOI: https://doi.org/10.1007/s11205-017-1672-7.
3.
Bach, R.L. 2021. “A Methodological Framework for the Analysis of Panel Conditioning Effects.” In Measurement Error in Longitudinal Data . In Measurement Error in Longitudinal Data , edited by A. Cernat and J. Sakshaug. Oxford University Press, Oxford, UK: 19–41.
4.
Bach, R.L., and S. Eckman. 2018. “Motivated Misreporting in Web Panels.” Journal of Survey Statistics and Methodology, 6(3):418–430. DOI: https://doi.org/10.1093/jssam/smx030.
5.
Bach, R.L., and S. Eckman. 2019. “Participating in a Panel Survey Changes Respondents’ Labour Market Behaviour.” Journal of the Royal Statistical Society: Series A (Statistics in Society), 182(1):263–281. DOI: https://doi.org/10.1111/rssa.12367.
6.
Bach, R., and S. Eckman. 2020. “Rotation Group Bias in Reporting of Household Purchases in the U.S. Consumer Expenditure Survey.” Economics Letters . DOI: https://doi.org/10.1016/j.econlet.2019.108889.
7.
Bach, R.L., S. Eckman, and J. Daikeler. 2020. “Misreporting Among Reluctant Respondents.” Journal of Survey Statistics and Methodology , 8(3):566–588. DOI: https://doi.org/10.1093/jssam/smz013.
8.
Bailar, B.A. 1975. “The Effects of Rotation Group Bias on Estimates from Panel Surveys.” Journal of the American Statistical Association , 70(349):23. DOI: https://doi.org/10.2307/2285370.
9.
Bee, A., B.D. Meyer, and J.X. Sullivan. 2015. “The Validity of Consumption Data. Are the Consumer Expenditure Interview and Diary Surveys Informative?” In Improving the Measurement of Consumer Expenditure s, edited by C.D. Carroll, T.F. Crossley, and J. Sabelhaus: 204–240. University of Chicago Press, Chicago.
10.
Bosley, J.M. Dashen, and J. Fox. 1999. “When Should We Ask Follow-up Questions About Items in Lists?” In Proceedings of the Section on Survey Research Methods, American Statistical Association . Available at. http://www.asasrms.org/Proceedings/papers/1999_131.pdf (accessed November 2020).
11.
Cohen, S.B., and V.L. Burt. 1985. “Data Collection Frequency Effect in the National Medical Care Expenditure Survey.” Journal of Economic and Social Measurement , 13(2):125–151.
12.
Daikeler, J., R.L. Bach, H. Silber, and S. Eckman. 2020. “Motivated Misreporting in Smartphone Surveys.” Social Science Computer Review. DOI: https://doi.org/10.1177/0894439319900936.
13.
Duan, N., M. Alegria, G. Canino, T. McGuire, and D. Takeuchi. 2007. “Survey Conditioning in Self-Reported Mental Health Service Use: Randomized Comparison of Alternative Instrument Formats.” Health Research and Educational Trust , 42(2):890–907. DOI: https://doi.org/10.1111/j.1475-6773.2006.00618.x.
14.
Eckman, S. 2020. “Underreporting of Purchases in the U.S. Consumer Expenditure Survey.” Available at: https://osf.io/e3sgz/ (accessed November 2020).
15.
Eckman, S., and F. Kreuter. 2018. “Misreporting to Looping Questions in Surveys: Recall, Motivation and Burden.” Survey Research Methods , 12(1):59–74. DOI: https://doi.org/10.18148/srm/2018.v12i1.7168.
16.
Eckman, S., F. Kreuter, A. Kirchner, A. Jäckle, R. Tourangeau, and S. Presser. 2014. “Assessing the Mechanisms of Misreporting to Filter Questions in Surveys.” Public Opinion Quarterly, 78(3):721–733. DOI: https://doi.org/10.1093/poq/nfu030.
17.
Edgar, J. 2010. “Respondent Record Use in U.S. Consumer Expenditure Survey.” In Presented at the American Association for Public Opinion Research Conference, May 13–16, 2010, Chicago, Illinois, U.S.A. Available at: https://www.bls.gov/cex/research_papers/pdf/Respondent-Record-Use-in-the-US-Consumer-Expenditure-Survey-Presentation.pdf (accessed November 2020).
18.
Edgar, J., and J. Gonzalez. 2009. “Correlates of Data Quality in the Consumer Expenditure Quarterly Interview Survey.” In Proceedings of the Survey Research Methods Section of the American Statistical Association August 1–6, 2009, Washington D.C., U.S.A. Available at: https://www.bls.gov/osmr/research-papers/2009/pdf/st090140.pdf (accessed November 2020).
19.
Fricker, S, B. Kopp, L. Tan, and R. Tourangeau. 2015. “A Review of Measurement Error Assessment in a U.S. Household Consumer Expenditure Survey.” Journal of Survey Statistics and Methodology, 3(1):67–88. DOI: https://doi.org/10.1093/jssam/smu025.
20.
Halpern-Manners, A., and J.R. Warren. 2012. “Panel Conditioning in Longitudinal Studies: Evidence From Labor Force Items in the Current Population Survey.” Demography , 49(4):1499–1519. DOI: https://doi.org/10.1007/s13524-012-0124-x.
21.
Halpern-Manners, A., J.R. Warren, and F. Torche. 2016. “Panel Conditioning in the General Social Survey.” Sociological Methods and Research , 46(1): 103–124. DOI: https://doi.org/10.1177/0049124114532445.
22.
Hansen, M.H., W.N. Hurwitz, H. Nisselson, and J. Steinberg. 1955. “The Redesign of the Census Current Population Survey.” Journal of the American Statistical Association , 50(271):701–719. DOI: https://doi.org/10.2307/2281161.
23.
Harrell Jr, F.E. 2020. with contributions from C. Dupont, and many others. Hmisc: Harrell Miscellaneous. R package version 4.4-1. Available at: https://CRAN.R-project.org/package=Hmisc (accessed November 2020).
24.
Hirsch, B.T., and J.V. Winters. 2016. “Rotation Group Bias in Measures of Multiple Job Holding.” Economic Letters, 147:160–163. DOI: https://doi.org/10.1016/j.econlet.2016.08.039.
25.
Huntington-Klein, N. 2020. vtable: Variable Table for Variable Documentation . R package version 1.2.1. Available at: https://CRAN.R-project.org/package=vtable (accessed November 2020).
26.
Jensen, P.S., H.K. Watanabe, and J.E. Richters. 1999. “Who’s Up First? Testing for Order Effects in Structured Interviews Using a Counterbalanced Experimental Design.” Journal of Abnormal Child Psychology , 27:439–445. DOI: https://doi.org/10.1023/A:1021927909027.
27.
Kessler, R.C., H.-U. Wittchen, J.M. Abelson, K. McGonagle, N. Schwarz, K.S. Kendler, B. Knäuper, and S. Zhao. 1998. “Methodological Studies of the Composite International Diagnostic Interview (CIDI) in the US National Comorbidity Survey (NCS).” International Journal of Methods in Psychiatric Research , 7(1):33–55. DOI: https://doi.org/10.1002/mpr.33.
28.
Kreuter, F., S. McCulloch, S. Presser, and R. Tourangeau. 2011. “The Effects of Asking Filter Questions in Interleafed versus Grouped Format.” Sociological Methods and Research , 40(88):88–104. DOI: https://doi.org/10.1177/0049124110392342.
29.
Krueger, A.B., A. Mas, and X. Niu. 2017. “The Evolution of Rotation Group Bias: Will the Real Unemployment Rate Please Stand Up?” Review of Economics and Statistics , 99(2):258–264. DOI: https://doi.org/10.1162/REST_a_00630.
30.
Lehnen, R.G., and A.J. Reiss. 1978. “Response Effects in the National Crime Survey.” Victomology, 3:110–160.
31.
Little, R.J.A. 1988. “Missing-Data Adjustments in Large Surveys.” Journal of Business and Economic Statistics , 6(3):287. DOI: https://doi.org/10.2307/1391878.
32.
Long, J.A. 2020. jtools: Analysis and Presentation of Social Scientific Data . R package version 2.1.0. Available at: https://cran.r-project.org/package=jtools (accessed November 2020).
33.
Mathiowetz, N.A., and T.J. Lair. 1994. “Getting Better? Change or Error in the Measurement of Functional Limitations.” Journal of Economic and Social Measurement, 20(3):237–262. DOI: https://doi.org/10.3233/JEM-1994-20305.
34.
McBride, B. 2013. “Examining Changes in Filter Question (FQ) Reporting in the Consumer Expenditure Quarterly Interview Survey.” In Proceedings of the Survey Research Methods Section of the American Statistical Association. Boston, Massachusetts, U.S.A.: 4304–4316. Available at: http://www.asasrms.org/Proceedings/y2013/files/400269_500735.pdf (accessed November 2020).
35.
Nancarrow, C., and T. Cartwright. 2007. “Online access panels and tracking research: The conditioning issue.” International Journal of Market Research , 49(5):573–594. DOI: https://doi.org/10.1177/147078530704900505.
36.
National Research Council. 2013. Measuring What We Spend: Toward a New Consumer Expenditure Survey . The National Academies Press, Washington, D.C.
37.
Neter, J., and J. Waksberg. 1964. “Conditioning effects from Repeated Household Interviews.” Journal of Marketing , 28(2):51–56. DOI: https://doi.org/10.1177/002224296402800211.
38.
Parker, J.A., N.S. Souleles, and C.D. Carroll. 2014. “In benefits of panel data in consumer expenditure surveys.” In Improving the Measurement of Consumer Expenditures : 75–99. University of Chicago Press. DOI: https://doi.org/10.7208/chicago/9780226194714.003.0004.
39.
R Core Team. 2020. R: A Language and Environment for Statistical Computing . R Foundation for Statistical Computing, Vienna, Austria. Available at: https://www.R-project.org/ (accessed November 2020).
40.
Schonlau, M., and V. Toepoel. 2015. “Straightlining in Web Survey Panels Over Time.” Survey Research Methods, 9(2):125–137. DOI: https://doi.org/10.18148/srm/2015.v9i2.6128.
41.
Shields, J., and N. To. 2005. “Learning To Say No: Conditioned Underreporting in an Expenditure Survey.” In Proceedings of the Survey Research Methods Section of the American Statistical Association. Available at: http://www.asasrms.org/Proceedings/y2005/files/JSM2005-000432.pdf (accessed November 2020).
42.
Silberstein, A.R. 1990. “First Wave Effects in the U.S. Consumer Expenditure Interview Survey.” Survey Methodology, 16(2):293–304.
43.
Silberstein, A.R., and C.A. Jacobs. 1989. “Symptoms of Repeated Interview Effects in the Consumer Interview Survey.” In Panel Surveys , edited by D. Kasprzyk, G.J. Duncan, G. Kalton, and M.P. Singh, 289–303. Wiley, New York.
44.
Solon, G. 1986. “Effects of Rotation Group Bias on Estimation of Unemployment.” Journal of Business and Economic Statistics , 4(1):105–109. DOI: https://doi.org/10.1080/07350015.1986.10509499.
45.
Struminskaya, B. 2016. “Respondent Conditioning in Online Panel Surveys. Results of Two Field Experiments.” Social Science Computer Review , 34(1):95–115. DOI: https://doi.org/10.1177/0894439315574022.
46.
Sun, H., R. Tourangeau, and S. Presser. 2018. “Panel Effects: Do the Reports of Panel Respondents Get Better or Worse over Time?” Journal of Survey Statistics and Methodology , 7(4):572–588. DOI: https://doi.org/10.1093/jssam/smy021.
47.
U.S. Bureau of Labor Statistics. 2016. Consumer Expenditures and Income: Handbook of Methods. Available at. https://www.bls.gov/opub/hom/cex/pdf/cex.pdf (accessed November 2020).
48.
U.S. Bureau of Labor Statistics. 2020. Household Survey Response Rates. Available at: https://www.bls.gov/osmr/response-rates/household-survey-response-rates.htm (accessed November 2020).
49.
Van Buuren, S., and K. Groothuis-Oudshoorn. 2011. “mice: Multivariate Imputation by Chained Equations in R.” Journal of Statistical Software, 45(3):1–67. DOI: https://doi.org/10.18637/jss.v045.i03.
50.
Venables, W.N., and B.D. Ripley. 2002. Modern Applied Statistics with S . Springer, New York, fourth edition. Available at: http://www.stats.ox.ac.uk/pub/MASS4 (accessed November 2020).
51.
Waring, E., M. Quinn, A. McNamara, E. Arino de la Rubia, H. Zhu, and S. Ellis. 2020. “skimr: Compact and Flexible Summaries of Data , 2020. R package version 2.1.2. Available at: https://CRAN.R-project.org/package=skimr (accessed November 2020).
52.
Waterton, J., and D. Lievesley. 1989. “Evidence of Conditioning Effects in the British Social Attitudes Panel.” In Panel Surveys , edited by D. Kasprzyk, G.J. Duncan, G. Kalton, and M.P. Singh, 319–339. Wiley, New York.
53.
Wickham, H., M. Averick, J. Bryan, W. Chang, L. D’Agostino McGowan, R. François, G. Grolemund, A. Hayes, L. Henry, J. Hester, M. Kuhn, T. Lin Pedersen, E. Miller, S. Milton Bache, K. Müller, J. Ooms, D. Robinson, D. Paige Seidel, V. Spinu, K. Takahashi, D. Vaughan, C. Wilke, K. Woo, and H. Yutani. 2019. “Welcome to the Tidyverse.” Journal of Open Source Software , 4(43): 1686. DOI: https://doi.org/10.21105/joss.01686.
54.
Wilson, T., and S. Abdirizak. 2017. “Statistical Examination of Rounding Tendencies in the Consumer Expenditure Interview Survey.” In Proceedings of the American Statistical Association July 29–August 3, 2017, Baltimore, Maryland, U.S.A. 894–907. Available at: https://www.bls.gov/osmr/research-papers/2017/pdf/st170160.pdf (accessed November 2020).
55.
Yan, T., and K. Copeland. 2010. “Panel Conditioning in the Consumer Expenditure Quarterly Interview Survey.” In Proceedings of the Survey Research Methods Section of the American Statistical Association, May 13–16, 2010, Chicago, Illinois, U.S.A. Available at: http://www.asasrms.org/Proceedings/y2010/Files/307812_59394.pdf (accessed November 2020).
56.
Yan, T., and S. Eckman. 2012. “Panel Conditioning: Change in True Value versus Change in Self-Report.” In Proceedings of the Survey Research Methods Section of the American Statistical Association, July 28–August 2, 2012, San Diego, California, U.S.A. Available at: http://www.asasrms.org/Proceedings/y2012/Files/306203_76099.pdf (accessed November 2020).