The Effect of Survey Mode on Socially Undesirable Responses to Open-ended Questions

Abstract

We estimate the difference in the chance of giving a socially undesirable response, one that violates social norms, by administration mode (online or in person) for a self-administered questionnaire (SAQ). We do this by evaluating open-ended responses to a photographic stimulus designed to generate undesirable responses. We test five variables: socially undesirable responses generally, blatant stereotyping, distaste, inappropriate descriptors, and the number of words in the response. The results show that, contrary to expectations, online SAQs are not more likely to produce socially undesirable responses generally, but online responses are less likely to contain blatant stereotyping. Responses from the online SAQs were longer, suggesting that respondents from the paper-and-pencil SAQs may have used stereotypes to bypass writing lengthy responses. The longer responses of online respondents enabled them to avoid stereotyping, thereby biasing their responses in an alternate way.