Respondent Debriefings Conducted by Experts: A Technique for Questionnaire Evaluation

Abstract

This article explores the use of expert respondent debriefings to evaluate the quality of survey data. In this case study, subject matter experts observed 169 interviews and conducted qualitative respondent debriefings on select cases in a field test of a census coverage survey. By comparing “true” residence status for each person as determined by the debriefing against the residence status obtained by the questionnaire alone, the authors determined whether the questionnaire was collecting accurate information. For the 473 people for whom survey data was available, the questionnaire failed only five times to assign the correct residence status code in a way that would have detrimentally affected coverage estimates. The respondent debriefing technique helped pinpoint specific problems in the questionnaire, as well as confirm that the questionnaire performed adequately in most cases. This article describes the expert respondent debriefing methodology that was used in a face-to-face interview setting and discusses how it could be adapted to telephone interviewing.