What's the chance? Interviewers' expectations of response in the 2010 SCF

Abstract

Many household surveys collect and maintain process paradata. Such information has particular appeal, because variations in the level or particular patterns of effort might bear on understanding unit nonresponse or other aspects of survey participation. But for such information to be useful, it is necessary to understand the process that generates them. For surveys that lack a highly structured contact protocol, choices of survey managers or interviewers determine whether and when a record is generated; if that choice is not neutral with respect to characteristics of respondents, then the process data may not be directly usable to study such topics as response propensity, unless there is some means of controlling for choice. For such surveys, one a priori plausible control is the subjective likelihood that a given case can be successfully completed. Effort might be thought to be most likely to be applied to cases that are most likely to be completed. This paper examines data from the 2010 Survey of Consumer Finances (SCF), on the subjective evaluation of the likelihood of case completion that interviewers were required to complete for each effort on each case. In general, the data suggest that interviewers cannot predict the outcome of cases sufficiently reliably or precisely to be systematically useful.