Researchers interested in studying the frequency of events or behaviors among a population must rely on count data provided by sampled individuals. Often, this involves a decision between live event counting, such as a behavioral diary, and recalled aggregate counts. Diaries are generally more accurate, but their greater cost and respondent burden generally yield less data. The choice of survey mode, therefore, involves a potential trade-off between bias and variance of estimators. We use a case study comparing inferences about payment instrument use based on different survey designs to illustrate this dilemma. We then use a simulation study to show how and under what conditions a hybrid survey design can improve efficiency of estimation, in terms of mean-squared error. Overall, our work suggests that such a hybrid design can have considerable benefits, as long as there is nontrivial overlap in the diary and recall samples.
AhmedNaeemBrzozowskiMatthewCrossleyThoms F.. 2010. “Measurement Errors in Recall Food Consumption Data.” Institute for Fiscal Studies Working Papers. Institute for Fiscal Studies, London, England.
2.
AndersenL. P.MikkelsenK. L.. 2008. “Recall of Occupational Injuries: A Comparison of Questionnaire and Diary Data.” Safety Science46:255–60.
3.
AndersonKevinBurfordOksanaEmmertonLynne. 2016. “Mobile Health Apps to Facilitate Self-care: A Qualitative Study of User Experiences.” PLoS ONE11:e0156164.
4.
AngrisaniMarcoKapteynArieSchuhScott. 2014. “Measuring Household Spending and Payment Habits: The Role of ‘Typical’ and ‘Specific’ Time Frames in Survey Questions.” Pp. 414–440 in Improving the Measurement of Consumer Expenditures, chapter 15, edited by CarrollChristopherCrossleyThomasSabelhausJohn. Cambridge, MA: NBER.
5.
BattistinErichPadulaMario. 2015. “Survey Instruments and the Reports of Consumption Expenditures: Evidence from the Consumer Expenditure Surveys.” Journal of the Royal Statistical Society, Series A179:559–81.
BlairEdwardBurtonScott. 1986. “Processes Used in the Formulation of Behavioral Frequency Reports in Surveys.” American Statistical Association Proceedings of the Section on Survey Methods, 481–87. Retrieved February 15, 2017 (http://www.asasrms.org/Proceedings/y1986f.html).
BoundJohnBrownCharlesMathiowetzNancy. 2001. “Measurement Error in Survey Data.” Pp. 3705-843 in Handbook of Econometrics. Vol. 5, edited by HeckmanJames J.LeamerEdward. Amsterdam, the Netherlands: Elsevier.
10.
BradburnNorman M.RipsLance J.ShevellSteven K.. 1987. “Answering Autobiographical Questions: The Impact of Memory and Inference on Surveys.” Science236:157–61.
11.
BRFSS. Various Years. “Behavioral Risk Factor Surveillance System.” Retrieved September 18, 2017 (http://www.cdc.gov/brfss/).
12.
BrzozowskiMatthewCrossleyThomas F.WinterJoachim K.. 2017. “A Comparison of Recall and Diary Food Expenditure Data.” Food Policy72:53–61.
13.
CES. Various Years. “Consumer Expenditure Survey.” Retrieved September 18, 2017 (http://www.bls.gov/cex/).
14.
ChatzitheochariStellaFisherKimberlyGilbertEmilyCalderwoodLisaHuskinsonTomClearyAndrewGershunyJonathan. 2018. “Using New Technologies for Time Diary Data Collection: Instrument Design and Data Quality Findings from a Mixed-mode Pilot Survey.” Social Indicators Research137:379–90.
15.
ClarkePhilip M.FiebigDenzil G.GerdthamUlf-G.. 2008. “Optimal Recall Length in Survey Design.” Journal of Health Economics27:1275–84.
16.
CrossleyThomasWinterJoachim. 2014. “Asking Households about Expenditures: What Have We Learned?” Pp. 23–50 in Improving the Measurement of Consumer Expenditures, chapter 1, edited by CarrollChristopherCrossleyThomasSabelhausJohn. Cambridge, MA: NBER.
17.
DCPC. Various Years. 2012. “Diary of Consumer Payment Choices.”
18.
DeatonAngusGroshMargaret. 2000. “Consumption.” Pp. 91–134 in Designing Household Survey Questionnaires for Developing Countries: Lessons from 15 Years of the Living Standards Measurement Study, chapter 5, edited by GroshMargaretGlewwePaul. Washington, DC: The World Bank.
19.
EisenhowerDonnaMathiowetzNancy A.MorgansteinDavid. 1991. “Recall Error: Sources and Bias Reduction Techniques.” Pp. 125–144 in Measurement Errors in Surveys, edited by BiermerPaul P.GrovesRobert M.LybergLars E.MathiowetzNancy A.SudmanSeymour. Hoboken, NJ: Wiley.
20.
GelmanAndrew. 2008. “Scaling Regression Inputs by Dividing by Two Standard Deviations.” Statistics in Medicine27:2865–73.
21.
GelmanAndrewRubinDonald. 1992. “Inference from Iterative Simulation Using Multiple Sequences.” Statistical Science7:457–511.
22.
GreavesStephenEllisonAdrianEllisonRichardRanceDeanStandenChrisRisselChrisCraneMelanie. 2015. “A Web-based Diary and Companion Smartphone App for Travel/activity Surveys.” Transportation Research Procedia11:297–310.
23.
GrovesRobert M.1989. Survey Errors and Survey Costs. New York: Wiley.
24.
GrovesRobert M.DillmanDon A.EltingeJohn L.LittleRoderick J. A.. 2001. Survey Nonresponse. New York: Wiley.
25.
HurdMichaelRohwedderSusann. 2009. “Methodological Innovations in Collecting Spending Data: The HRS Consumption and Activities Mail Survey.” Fiscal Studies30:435–59.
26.
JonkerNicoleKosseAnneke. 2009. “The Impact of Survey Design on Research Outcomes: A Case Study of Seven Pilots Measuring Cash Usage in the Netherlands.” DNB Working Paper 221. De Nederlandsche Bank, Amsterdam, the Netherlands.
27.
KemsleyWilliam F. F.NicholsonJ. L.. 1960. “Some Experiments in Methods of Conducting Consumer Expenditure Surveys.” Journal of the Royal Statistical Society, Series A123:307–28.
28.
MariniMargaret MooneySheltonBetth Anne. 1993. “Measing Household Work: Recent Experience in the United States.” Social Science Research22:361–82.
29.
McKenzieJohn. 1983. “The Accuracy of Telephone Call Data Collected by Diary Methods.” Journal of Marketing Research20:417–27.
30.
MeansBarbaraSwanGary E.JobeJared B.EspositoJames L.LoftusElizabeth F.. 1989. “Recall Strategies for Estimation of Smoking Levels in Health Surveys.” Paper presented at American Statistical Association Meetings, San Francisco, CA.
31.
MenonGeeta. 1994. “Judgments of Behavioral Frequencies: Memory Search and Retrieval Strategies.” Pp. 161–72 in Autobiographical Memory and the Validity of Retrospective Reports, edited by SchwarzNorbertSudmanSeymour. New York: Springer-Verlag.
32.
NeterJohnWaksbergJoseph. 1964. “A Study of Response Errors in Expenditure Data from Household Interviews.” Journal of the American Statistical Association59:18–55.
NSSO Expert Group on Sampling Errors. 2003. “Suitability of Different Reference Periods for Measuring Household Consumption: Result of a Pilot Study.” Economic and Political Weekly37:307–21.
35.
NusserSarah M.BeylerNicholas K.WelkGregory J.CarriquiryAlicia L.FullerWayne A.KingBenjamin M. N.. 2012. “Modeling Errors in Physical Activity Recall Data.” Journal of Physical Activity and Health9:56–67.
RockwoodTodd. 2015. “Assessing Physical Health.” Pp. 107–142 in Handbook of Health Survey Methods, chapter 5, edited by JohnsonTimothy P.. Hoboken, NJ: John Wiley.
SchmidtTobias. 2011. “Fatigue in Payment Diaries: Empirical Evidence from Germany.” Discussion Paper Series 1: Economic Studies No 11/2011. Deutsche Bundesbank, Frankfurt, Germany.
40.
ShephardR. J.2003. “Limits to the Measurement of Habitual Physical Activity by Questionnaires.” British Journal of Sports Medicine37:197–206.
41.
SiemieniakoDariusz. 2017. “The Consumer Diaries Research Method.” Pp. 53–66 in Formative Research in Social Marketing: Innovative Methods to Gain Consumer Insights, edited by KubackiKrzysztofRundle-ThieleSharyn. Singapore, Malaysia: Springer.
42.
SilbersteinAdriana R.ScottStuart. 1991. “Expenditure Diary Surveys and Their Associated Errors.” Pp. 303–326 in Measurement Errors in Surveys, edited by BiermerPaul P.GrovesRobert M.LybergLars E.MathiowetzNancy A.SudmanSeymour. Hoboken, NJ: Wiley.
43.
StrubeGerhard. 1987. “Answering Survey Questions: The Role of Memory.” Pp. 86–01 in Social Information Processing and Survey Methodology, edited by HipplerHans-J.SchwarzNorbertSudmanSeymour. New York: Springer-Verlag.
44.
SudmanSeymourBradburnNorman M.. 1973. “Effects of Time and Memory Factors on Response in Surveys.” Journal of the American Statistical Association68:805–15.
45.
SudmanSeymourBradburnNorman M.SchwarzNorbert. 1996. Thinking about Answers: The Application of Cognitive Processes to Survey Methodology. San Francisco, CA: Jossey-Bass.
46.
ThomasNealHarelOferLittleRoderick J. A.. 2016. “Analyzing Clinical Trial Outcomes Based on Incomplete Daily Diary Reports.” Statistical Medicine35:2894–906.
47.
TourangeauRogerYanTing. 2007. “Sensitive Questions in Surveys.” Psychological Bulletin133:859–83.
