Abstract
The Current Population Survey (CPS) is the source of official US labor force statistics. The wording of the CPS employment questions may not always cue respondents to include informal work in their responses, especially when providing proxy reports about other household members. In a survey experiment conducted using a sample of Amazon Mechanical Turk respondents, additional probing identified a substantial amount of informal work activity not captured by the CPS employment questions, both among those with no employment and among those categorized as employed based on answers to the CPS questions. Among respondents providing a proxy report for another household member, the share identifying additional work was systematically greater among those receiving a detailed probe that offered examples of types of informal work than among those receiving a simpler global probe. Similar differences between the effects of the detailed and the global probe were observed when respondents answered for themselves only among those who had already reported multiple jobs. The findings suggest that additional probing could improve estimates of employment and multiple job holding in the CPS and other household surveys, but that the nature of the probe is likely to be important.
References
1.
Abraham, K.G., J.C. Haltiwanger, K. Sandusky, and J.R. Spletzer. 2018. “Measuring the Gig Economy: Current Knowledge and Open Issues.” National Bureau of Economic Research Working Paper No. 24950. August. Doi: http://dx.doi.org/10.3386/w24950.
2.
Abraham, K.G. and S.N. Houseman. 2018. “Making Ends Meet: The Role of Informal Work in Supplementing Americans’ Income.” Upjohn Institute, unpublished working paper. December. Available at: https://www.aeaweb.org/conference/2019/preliminary/paper/QreAaS2h (accessed July 2019).
3.
Allard, M.D. and A.E. Polivka. 2018. “Measuring Labor Market Activity Today: Are the Words Work and Job Too Limiting for Surveys?” Monthly Labor Review . November. Available at: https://www.bls.gov/opub/mlr/2018/article/pdf/measuring-labor-market-activity-today.htm (accessed April 2019).
4.
Belli, R.F., N. Schwarz, E. Singer, and J. Talarico. 2000. “Decomposition Can Harm the Accuracy of Behavioural Frequency Reports.” Applied Cognitive Psychology 14: 295–308. Doi: http://dx.doi.org/10.1002/1099-0720(200007/08)14:4<295::AID-ACP646>3.0.CO;2-1.
5.
Blair, E. and S. Burton. 1987. “Cognitive Processes Used by Survey Respondents to Answer Behavioral Frequency Questions.” Journal of Consumer Research 14(2): 280–288. Doi: https://doi.org/10.1086/209112.
6.
Board of Governors of the Federal Reserve System. 2017. Report on the Economic Well-Being of U.S. Households in 2016 . Washington, D.C: Board of Governors of the Federal Reserve System. Available at: https://www.federalreserve.gov/publications/files/2016-report-economic-well-being-us-households-201705.pdf (accessed May 2019).
7.
Bower, G.H. and S.G. Gilligan. 1979. “Remembering Information Related to One’s Self.” Journal of Research in Personality 13: 420–432. Doi: https://doi.org/10.1016/0092-6566(79)90005-9.
8.
Bracha, A. and M.A. Burke. 2018. “The Ups and Downs of the Gig Economy, 2015–2017.” Federal Reserve Bank of Boston Working Paper 18–12. October. Available at: https://www.bostonfed.org/publications/research-department-working-paper/2018/the-ups-and-downs-of-the-gig-economy-2015-2017.aspx (accessed May 2019).
9.
Bracha, A. and M. A. Burke. 2019. “How Big is the Gig?” Federal Reserve Bank of Boston, unpublished working paper. January.
10.
Brown, N.R., L.J. Rips, and S.K. Shevell. 1985. “The Subjective Dates of Natural Events in Very-Long-Term Memory.” Cognitive Psychology 17(2): 139 – 177. Doi: https://doi.org/10.1016/0010-0285(85)90006-4.
11.
Bureau of Labor Statistics. Undated. “Labor Force Statistics from the Current Population Survey: Frequently Asked Questions.” Available at: https://www.bls.gov/cps/faq.htm (accessed June 2018).
12.
Bureau of Labor Statistics. 2018. “Contingent and Alternative Employment Arrangements, May 2017.” Available at: https://www.bls.gov/news.release/pdf/conemp.pdf (accessed June 2018).
13.
Cohany, S.R. 1996. “Workers in Alternative Employment Arrangements.” Monthly Labor Review. October: 31–45. Available at: https://www.bls.gov/mlr/1996/10/art4full.pdf (accessed May 2019).
14.
Dashen, M. 2000. “The Effects of Retention Intervals on Self- and Proxy Reports of Purchases.” Memory 8(3): 129–143. Doi: https://doi.org/10.1080/096582100387560.
15.
Fowler, F.J., Jr. 1992. “How Unclear Terms Affect Survey Data.” Public Opinion Quarterly 56(2): 218–231. Doi: https://doi.org/10.1086/269312.
16.
Katz, L.F. and A.B. Krueger. 2019a. “Understanding Trends in Alternative Work Arrangements in the United States.” NBER Working Paper No. 25425. Cambridge, MA: National Bureau of Economic Research. Doi: https://doi.org/10.3386/w25425.
17.
Katz, L.F. and A.B. Krueger. 2019b. “The Rise and Nature of Alternative Work Arrangements in the United States, 1995–2015.” ILR Review 72(2): 382–416. Doi: https://doi.org/10.1177%2F0019793918820008.
18.
Kojetin, B.A. and L.A. Miller. 1993. “The Intrahousehold Communications Study: Estimating the Accuracy of Proxy Responses at the Dyadic Level.” Paper presented at the 48th Annual Conference of the American Association for Public Opinion Research, St. Charles, Illinois. May. Available at: http://www.asasrms.org/Proceedings/papers/1993_188.pdf (accessed May 2019).
19.
Kuiper, N.A. and T.B. Rogers. 1979. “Encoding of Personal Information: Self-Other Differences.” Journal of Personality and Social Psychology 37(4): 499–514. Doi: http://dx.doi.org/10.1037/0022-3514.37.4.499.
20.
Martin, E. 2006. “Survey Questionnaire Construction.” U.S. Census Bureau Research Report Series, Survey Methodology #2006-13. Available at: https://www.census.gov/srd/papers/pdf/rsm2006-13.pdf. (accessed May 2019).
21.
Martin, E. and A.E. Polivka. 1995. “Diagnostics for Redesigning Survey Questionnaires: Measuring Work in the Current Population Survey.” Public Opinion Quarterly 59(4): 547–567. Doi: https://doi.org/10.1086/269493.
22.
Menon, G. 1997. “Are the Parts Better than the Whole? The Effects of Decompositional Questions on Judgments of Frequent Behaviors.” Journal of Marketing Research 34(3): 335–346. Doi: https://doi.org/10.1177%2F002224379703400303.
23.
Phillips, J.M., B.A. Bickart, and G. Menon. 2006. “Reporting About Others’ Behavior: The Role of Judgment Strategy, Knowledge, and Regularity.” September. Available at: https://ssrn.com/abstract=946247 (accessed April 2019).
24.
Polivka, A.E. 1996. “A Profile of Contingent Workers.” Monthly Labor Review . October: 10–21. Available at: https://www.bls.gov/opub/mlr/1996/article/profile-of-contingent-workers.htm (accessed May 2019).
25.
Robles, B. and M. McGee. 2016. Exploring Online and Offline Informal Work: Findings from the Enterprising and Informal Work Activities (EIWA) Survey . Washington: Board of Governors of the Federal Reserve System. Doi: https://doi.org/10.17016/FEDS.2016.089.
26.
Schober, M.F. and F.G. Conrad. 1997. “Does Conversational Interviewing Reduce Survey Measurement Error?” Public Opinion Quarterly 61(4): 576–602. Doi: https://doi.org/10.1086/297818.
27.
Schober, M.F., A.L. Suessbrick, and F.G. Conrad. 2018. “When Do Misunderstandings Matter? Evidence from Survey Interviews about Smoking.” Topics in Cognitive Science 10(2): 452–484. Doi: https://doi.org/10.1111/tops.12330.
28.
Schwarz, N. 1999. “Self-Reports: How the Questions Shape the Answers.” American Psychologist 54(2): 93–105. Doi: http://dx.doi.org/10.1037/0003-066X.54.2.93.
29.
Schwarz, N. and T. Wellens. 1997. “Cognitive Dynamics of Proxy Responding: The Diverging Perspectives of Actors and Observers.” Journal of Official Statistics 13(2): 159–179. Doi: https://doi.org/10.1.1.39.5355.
30.
Sudman, S. and N.M. Bradburn. 1973. “Effects of Time and Memory Factors on Response in Surveys.” Journal of the American Statistical Association 68: 805–815. Doi: https://doi.org/10.2307/2284504.
31.
Sudman, S., N.M. Bradburn, and N. Schwarz. 1996. Thinking About Answers: The Application of Cognitive Processes to Survey Methodology . San Francisco: Jossey-Bass.
32.
Tourangeau, R. 2000. “Remembering What Happened: Memory Errors and Survey Report.” In The Science of Self-Report: Implications for Research and Practice , edited by A.A. Stone, C.A. Bachrach, J.B. Jobe, H.S. Kurtzman, and V.S. Cain, 29–47. Mahwah, NJ: Lawrence Erlbaum Associates Inc.
33.
Tourangeau, R., F.G. Conrad, M.P. Couper, and C. Ye. 2014. “The Effects of Providing Examples in Survey Questions.” Public Opinion Quarterly 78(1): 100–125. Doi: https://doi.org/10.1093/poq/nft083.
34.
Tourangeau, R., L.J. Rips, and K. Rasinski. 2000. The Psychology of Survey Response . New York: Cambridge University Press.
35.
U.S. Census Bureau. 2006. Design and Methodology: Current Population Survey , Technical Paper No. 66. Washington DC. Available at: https://www.census.gov/prod/2006pubs/tp-66.pdf (accessed July 2019).