How long do respondents think online surveys should be? New evidence from two online panels in Germany

Abstract

In recent years, the number of surveys, especially online surveys, has increased dramatically. Due to the absence of interviewers in this survey mode (who can motivate the respondents to continue answering), some researchers and practitioners argue that online surveys should not be longer than 20 min. However, so far, there has been little research investigating how long respondents think that online surveys should or could be. In this study, we therefore asked respondents of two online panels in Germany (one probability-based panel and one nonprobability panel) about their opinions on the ideal and maximum lengths of surveys. We also investigated whether socio-demographic, personality-related, and survey-related variables were associated with the ideal and maximum lengths reported by respondents. Finally, we compared the stated and observed survey lengths to evaluate the extent to which respondents are able to accurately estimate survey length. Our results suggest that the ideal length of an online survey is between 10 and 15 min and the maximum length is between 20 and 28 min, depending on the measure of central tendency (mean or median) used and the panel. Moreover, we found significant effects of socio-demographics (gender, age, education, and number of persons in household), of personality traits, and survey-related questions (whether the respondents liked the survey, found it easy, and answered from a PC) on at least one of the dependent variables (ideal or maximum lengths). Finally, we found only small differences (less than two min) between stated and observed lengths.

Keywords

ideal length online panels questionnaire length survey duration web surveys

Get full access to this article

View all access options for this article.

References

Blom

A. G.

Fikel

Friedel

Höhne

J. K.

Krieger

Rettig

, & SFB 884 “Political Economy of Reforms.” (2020). German Internet Panel, wave 42 (July 2019). doi: 10.4232/1.13465

Crawford

S. D.

Couper

M. P.

Lamias

M. J.

(2001). Web surveys: Perceptions of burden. Social Science Computer Review, 19, 146–162. http://doi.org/10.1177/089443930101900202

Deutskens

de Ruyter

Wetzels

Oosterveld

(2004). Response rate and response quality of internet-based surveys: An experimental study. Marketing Letters, 15, 21–36. http://doi.org/10.1023/B:MARK.0000021968.86465.00

Galesic

Bosnjak

(2009). Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly, 73, 349–360. http://doi.org/10.1093/poq/nfp031

Marcus

Bosnjak

Lindner

Pilischenko

Schuetz

(2007). Compensating for low topic interest and long surveys: A field experiment on nonresponse in web surveys. Social Science Computer Review, 25, 372–383. http://doi.org/10.1177/0894439307297606

Rammstedt

Kemper

C. J.

Klein

M. C.

Beierlein

Kovaleva

(2013). A short scale for assessing the Big Five dimensions of personality. Methods, Data, Analyses, 7, 233–249. http://doi.org/10.12758/mda.2013.013

Revilla

Ochoa

(2017). Ideal and maximum length for a web survey. International Journal of Market Research, 59, 557–565. http://doi.org/10.2501/IJMR-2017-039

Schlosser

Höhne

J. K.

(2018). Embedded Client Side Paradata (ECSP). Zenodo. http://doi.org/10.5281/zenodo.1218941

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB