Dependent interviewing is a method in which respondents are presented with their previously reported data during data collection to aid response formation, reduce response burden and improve data quality. This method has been widely studied in longitudinal household surveys and found to be effective at reducing response error. Although some establishment surveys have adopted its use, its effect on response burden and data quality has not been empirically studied with such rigor. Responding to an establishment survey often entails reporting of proprietary information, use of records, self-administered forms, input from multiple individuals, and changes in respondents over time. These factors may impact the effectiveness of dependent interviewing on response burden and data quality. Given the differences in the response process between establishment and household survey respondents, more research is needed. The U.S. Energy Information Administration (EIA) is attempting to fill some of those gaps in the research. EIA uses dependent interviewing in several surveys but has never evaluated its use. Recently, EIA conducted usability testing and respondent debriefings with respondents to examine the effect of dependent interviewing on response burden and data quality. We present findings from this research and discuss the implications for establishment surveys and future research.
WillimackDKSnijkersG. The business context and its implications for the survey response process. In: SnijkersGHaraldsenGJonesJWillimackDK (eds) Designing and conducting business surveys. Hoboken, NJ: John Wiley and Sons, 2013, pp.39–82.
2.
WillimackDKNicholsE. A hybrid response process model for business surveys. J Off Stat2010; 26: 3–24.
3.
MathiowetzNAMcGonagleKA. An assessment of the current state of dependent interviewing in household surveys. J Off Stat2000 Dec 1; 16: 401.
4.
RipsLJConradFGFrickerSS. Straightening the seam effect in panel surveys. Public Opin Q2003 Feb 1; 67: 522–554.
5.
TverskyAKahnemanD. Availability: a heuristic for judging frequency and probability. Cognit Psychol1973 Sep 1; 5: 207–232.
6.
JäckleA. Dependent interviewing: a framework and application to current research. In: LynnP (eds) Methodology of longitudinal surveys. Chichester, UK: John Wiley & Sons, 2009, pp.93–112.
7.
MurrayTSMichaudSEganM,et al.Invisible seams? The experience with the Canadian Labour Market Activity Survey. In: Proceedings of the US Bureau of the Census Annual Research Conference, Arlington, VA. Washington, DC: U.S. Census Bureau, 1991 Mar 17-20.
8.
LemaitreG. Dealing with the seam problem for the Survey of Labour and Income Dynamics. Ottawa, CN: Statistics Canada, 1992, SLID Research Paper Series 92–05.
9.
HillDH. The relative empirical validity of dependent and independent data collection in a panel survey. J Off Stat1994 Dec 1; 10: 359.
10.
HaleAMichaudS. Dependent interviewing: impact on recall and on labour market transitions. Ottawa, CN: Statistics Canada, 1995, SLID Research Paper Series No. 95-06.
11.
MooreJBatesNPascaleJ, et al.Use of dependent interviewing procedures to improve data quality in the measurement of change. Washington, DC: U.S. Census Bureau, 2006 Jan, Report No. Survey Methodology #2006-02.
12.
JäckleALynnP. Dependent interviewing and seam effects in work history data. J Off Stat2007; 23: 529–551.
13.
JäckleA. Measurement error and data collection methods: effects on estimates from event history data. Colchester, UK: University of Essex, 2008, ISER Working Paper Series 2008–13.
14.
BrüderlJCastiglioniLLudwigV, et al.Collecting event history data with a panel survey: combining an electronic event history calendar and dependent interviewing. Methods Data Analyses2017 Feb 7; 11: 22.
15.
LynnPJäckleAJenkinsSP,et al.The impact of questioning method on measurement error in panel survey measures of benefit receipt: evidence from a validation study. J R Stat Soc Ser A: Stat Soc2012 Jan; 175: 289–308.
16.
LugtigPJäckleA. Can I just check…? Effects of edit check questions on measurement error and survey estimates. J Off Stat2014 Mar 1; 30: 45–62.
17.
HoogendoornAW. A questionnaire design for dependent interviewing that addresses the problem of cognitive satisficing. J Off Stat2004; 20: 219–232.
18.
LynnPSalaE. Measuring change in employment characteristics: the effects of dependent interviewing. Int J Public Opin Res2006 Dec 1; 18: 500–509.
19.
KrosnickJAAlwinDF. An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opin Q1987 Jan 1; 51: 201–219.
20.
EggsJJäckleA. Dependent interviewing and sub-optimal responding. Surv Res Methods2015; 9: 15–29.
21.
ConradFGRipsLJFrickerSS. Seam effects in quantitative responses. J Off Stat2009 Jan 1; 25: 339.
22.
LugtigPLensvelt-MuldersGJ. Evaluating the effect of dependent interviewing on the quality of measures of change. Field Methods2014 May; 26: 172–190.
23.
EngstromCSinibaldiJ. Reducing burden in a web survey through dependent interviewing. J Surv Stat Methodol2024 Feb 1; 12: 60–79.
24.
BavdažMGiesenDČerneSK, et al.Response burden in official business surveys: measurement and reduction practices of national statistical institutes. J Off Stat2015 Dec 1; 31: 559–588.
25.
RidolfoHMoralesG. Use of dependent interviewing in federal establishment surveys. Washington, DC: U.S. Energy Information Administration, 2022 Jul.
26.
HaraldsenG. Quality issues in business surveys. In: SnijkersGHaraldsenGJonesJWillimackDK (eds) Designing and conducting business surveys. Hoboken, NJ: John Wiley & Sons, 2013, pp.83–126.
27.
MorrisonRLDillmanDAChristianLM. Questionnaire design guidelines for establishment surveys. J Off Stat2010; 26: 43–85.
28.
EdwardsWSCantorD. Toward a response model in establishment surveys. In: BiemerPPGrovesRMLybergLEMathiowetzNASudmanS (eds) Measurement errors in surveys. New York: John Wiley & Sons, 1991, pp.211–233.
29.
BaileyJT. Evaluation of historical data use in the 1992 August Yield Survey. Washington, DC: U.S. National Agricultural Statistics Service, 1993 Jan, Report No. SRB-93-02.
30.
HolmbergA. Pre-printing effects in official statistics: an experimental study. J Off Stat2004 Jun 1; 20: 341.
31.
RidolfoHEdgarJ. Let me tell you what you told me: dependent interviewing in establishment surveys. In: Proceedings of the American Association for Public Opinion Research Annual Conference, Hollywood, FL.American Statistical Association Survey Research and Methods Section, 2015, pp.4241–4255.
32.
ThomasE. Displaying previously reported data to respondents in the Quarterly Census of Employment and Wages Program at the Bureau of Labor Statistics. In: Paper presented at the Federal Committee on Statistical Methodology Research and Policy Conference, Washington, DC, 2021 Nov 2-4.
33.
RidolfoHPickK. 2020 Census of Agriculture CAWI Content Test using non-edited reported data (NERD) – findings and recommendations from usability testing. Washington, DC: U.S. National Agricultural Statistics Service, 2020 May.
34.
PaffordBV. The influence of using previous survey data in the 1986 April ISP Grain Stocks Survey. Washington, DC: U.S. National Agricultural Statistics Service, 1988 Apr, Report No. SRB-88-01.
35.
PhillipsJMMitraAKnappG, et al.The determinants of acquiescence to preprinted information on establishment survey instruments. In: Proceedings of the Joint Statistical Meetings, Orlando, FL. Alexandria, VA:American Statistical Association, 1995 Aug 13-17, pp.1169–1171.
36.
StanleyJSaferMA. “Last time you had 78, how many do you have now?” The effect of providing previous reports on current reports of cattle inventories. Washington, DC: U.S. National Agricultural Statistics Service, 1997.
37.
WhittardDRitchieFPhanV, et al.The perils of pre-filling: lessons from the UK’s annual survey of hours and earning microdata. Stat J IAOS2024 Aug 1; 39: 661–677.
38.
AbayomiEMaliszewskiSKreinerL,et al.They spoke, we listened: reducing respondent burden using previously reported data. In: Proceedings of the Joint Statistical Meetings, Vancouver, Canada. Alexandria, (US): American Statistical Association, 2018 Jul 28-Aug 2, pp.1595–1603.
39.
VittorianoLMCyBulskiKAPottsMS,et al.The ‘venting’ question: how open-ended comments can be used to improve establishment surveys. In: Paper presented at the VI International Conference of Establishment Surveys, 2021 Jun 14-17; online.
40.
RodhouseJOttK. Utilizing respondents’ previously reported data in a census of establishments: results from an experiment in the Census of Agriculture’s 2020 Content Test. In: Paper presented at the Federal Committee on Statistical Methodology Research & Policy Conference, Washington, DC, 2021 Nov 2-4.
41.
YoungLJRodhouseJBTernerZ,et al.Adopting previously reported data into the 2022 census of agriculture: lessons learned from the 2020 September agricultural survey. In: SnijkersGBavdazMBenderSJonesJMacFeelySSakshaugJWThompsonKJvan DeldenA (eds) Advances in business statistics, methods and data collection. Hoboken, NJ: John Wiley & Sons, 2023, pp.805–820.
42.
SpiegelmanMZottiA. Is this information correct? Assessing the burden and data quality tradeoffs of using extant data. In: Paper presented at the Federal Computer Assisted Survey Information Collection Workshop, 2022 Apr 5-6; online.
43.
SpiegelmanMOkonAThomasT,et al.Who works here? Rostering school staff with vendor-assisted lists. In: Paper presented at the American Association for Public Opinion Research Annual Conference, Toronto (CN), 2019 May 16-19.
44.
Al BaghalT. Last year your answer was… the impact of dependent interviewing wording and survey factors on reporting of change. Field Methods2017 Feb; 29: 61–78.
45.
JäckleAEckmanS. Is that still the same? Has that changed? On the accuracy of measuring change with dependent interviewing. J Surv Stat Methodol2020 Sep; 8: 706–725.
46.
JäckleAAl BaghalTEckmanS,et al.How to pop the question? Interviewer and respondent behaviours when measuring change with proactive dependent interviewing. In: LynnP (eds) Advances in longitudinal survey methodology. Chichester, UK: Wiley, 2021, pp.368–398.
47.
RodhouseJBOttK. Respondent perceptions of previously reported data. Surv Pract2022 Jun 30; 15: 1–19.
48.
NeterJWaksbergJ. A study of response errors in expenditures data from household interviews. J Am Stat Assoc1964 Mar 1; 59: 18–55.
49.
BraunVClarkeV. Using thematic analysis in psychology. Qual Res Psychol2006 Jan 1; 3: 77–101.
50.
RidolfoHMoralesG. Dependent interviewing in the Quarterly Survey of Industrial, Commercial, & Institutional Coal Users (EIA-3). Washington, DC: U.S. Energy Information Administration, 2024 Jun.
51.
RidolfoHMoralesG. Dependent interviewing in the Densified Biomass Fuel Report (EIA-63C). Washington, DC: U.S. Energy Information Administration, 2024 Jun.
52.
American Association for Public Opinion Research (AAPOR). Standard definitions: final dispositions of case codes and outcome rates for surveys. 10th edition. Alexandria, VA: American Association for Public Opinion Research, 2023.
