Several authors and software vendors advocate the benefits of auto forwarding in web surveys, but there is little empirical research on this approach. We experimentally tested automatic versus manual forwarding (MF) under different levels of cognitive effort. We manipulated information accessibility (IA; low vs. high) and consistency requirements (CRs; yes vs. no), along with auto forwarding (AF) versus MF in two studies conducted among students in Finland. We find that an AF survey takes less time to complete, but only for those completing a survey on personal computers or tablets; no time advantage is found for smartphone users. We also find that respondents in both AF and MF conditions return more often to items with higher cognitive burden (low IA or a CR). MF respondents change answers more often than AF respondents. AF appears to reduce straightlining slightly. We find no difference in response consistency between two behavioral items between AF and MF, but a slight advantage for AF for two attitude items. Finally, respondents reported more positive experiences with the AF version. Auto forwarding appears to be somewhat more efficient and easy to use but may decrease the quality of responses to cognitively demanding questions.
BethlehemJ.BiffignandiS. (2011). Handbook of web surveys. New Jersey: John Wiley & Sons, Inc.
2.
CallegaroM.Lozar ManfredaK.VehovarV. (2015). Web survey methodology. London: SAGE.
3.
CoreG. L. (1992). Cognitive phenomenology: Feelings and the construction of judgment. In MartinL.TesserA. (Eds.), The construction of social judgments (pp. 133–164). Hillsdale, NJ: Lawrence Erlbaum.
4.
CouperM. P. (2008). Designing effective web surveys. New York, NY: Cambridge University Press.
5.
CouperM. P.AntounC.MavletovaA. (2017). Mobile web surveys: A total survey error perspective. In BiemerP.EckmanS.EdwardsB.de LeeuwE.KreuterF.LybergL.TuckerC.WestB. (Eds.), Total survey error in practice (pp. 133–154). New York, NY: Wiley.
6.
CouperM. P.PetersonG. (2016). Why do web surveys take longer on smartphones?Social Science Computer Review, 35, 357–377.
7.
CouperM. P.TourangeauR.ConradF. G.CrawfordS. D. (2004). What they see is what we get: Response options for web surveys. Social Science Computer Review, 22, 111–127.
8.
CouperM. P.TourangeauR.ConradF. G.ZhangC. (2013). The design of grids in web surveys. Social Science Computer Review, 31, 322–345.
9.
de BruijneM.DasM.van SoestA.WijnantA. (2015, 7). Adapting grid questions on mobile devices. Paper presented at the European Survey Research Association conference, Reykjavik.
10.
de LeeuwE. D.HoxJ. J.KlauschT.RobertsA.de JonghA. (2012, May). Design of web questionnaires: Matrix questions or single question formats. Paper presented at the Annual Conference of the American Association for Public Opinion Research, Orlando, FL.
11.
FunkeF.ReipsU.-D. (2012). Why semantic differentials in web-based research should be made from visual analog scales and not from 5-point scales. Field Methods, 24, 310–327.
12.
GalesicM.TourangeauR.CouperM. P.ConradF. G. (2008). Eye-tracking data: New insights on response order effects and other cognitive shortcuts in survey responding. Public Opinion Quarterly, 72, 866–891.
13.
HammenK. (2010). The impact of visual and functional design elements in online survey research. Paper presented at the General Online Research Conference (GOR), Mannheim, Germany.
14.
HansonT. (2017, March). Adapting questionnaires for smartphones: An experiment on grid format questions. Paper presented at the General Online Research Conference, Berlin.
15.
HaysR. D.BodeR.RothrockN.RileyW.CellaD.GershonR. (2010). The impact of next and back buttons on time to complete and measurement reliability in computer-based surveys. Quality of Life Research, 19, 1181–1184.
16.
HeerweghD. (2003). Explaining response latencies and changing answers using client-side paradata from a web survey. Social Science Computer Review, 21, 360–373.
17.
HoffmanL.RovineM. J. (2007). Multilevel models for the experimental psychologist: Foundations and illustrative examples. Behavior Research Methods, 39, 101–117.
18.
HoxJ. (2002). Multilevel analysis. Techniques and applications. Mahwah, NJ: Lawrence Erlbaum.
19.
HöhneJ. K.SchlosserS.KrebsD. (2016, March). Analyzing cognitive burden of survey questions with paradata: A web survey experiment. Paper presented at the General Online Research Conference (GOR), Dresden.
20.
HorwitzR.TancretoJ. G.ZelenakM. F.DavisM. C. (2012). Use of paradata to assess the quality and functionality of the American Community Survey internet instrument (DSSD 2012 American Community Survey Memorandum Series #ACS12-MP-04). Washington, DC: U.S. Census Bureau. Retrieved fromhttp://www.census.gov/acs/www/Downloads/library/2012/2012_Horwitz_01.pdf
21.
HubertM.VandervierenE. (2008). An adjusted boxplot for skewed distributions. Computational Statistics and Data Analysis, 52, 5186–5201.
22.
KaczmirekL. (2009). Human-survey interaction. Usability and nonresponse in online surveys. Cologne, Germany: Herbert von Halem Verlag.
23.
KlauschT.de LeeuwE. D.HoxJ. J.RobertsA.de JonghA. (2012, March). Matrix vs. single question formats in web surveys: Results from a large scale experiment. Paper presented at the General Online Research Conference, Mannheim.
24.
KrishnamoorthyK.ThomsonJ. (2004). A more powerful test for comparing two Poisson means. Journal of Statistical Planning and Inference, 119, 23–35.
RatcliffR. (1993). Methods for dealing with reaction time outliers. Psychological Bulletin, 114, 510–532.
27.
RiversD. (2006, September). Web surveys for health measurement. Paper presented at Building Tomorrow’s Patient-Reported Outcome Measures: The Inaugural PROMIS Conference, Gaithersburg, MD, pp. 11–13.
28.
SchwarzN.BlessH. (1992). Constructing reality and its alternatives: An inclusion/exclusion model of assimilation and contrast effects in social judgment. In MartinL. L.TesserA. (Eds.), The construction of social judgments (pp. 217–245). Hillsdale, NJ: Lawrence Erlbaum.
29.
ShneidermanB. (1998). Designing the user interface: Strategies for effective human-computer interaction (3rd ed.). Boston, MA: Addison Wesley.
30.
SelkäläA.RonkainenS.AlasaarelaE. (2011). Features of the Z-scoring method in graphical two-dimensional web surveys: The case of ZEF. Quality & Quantity, 45, 609–621.
31.
SiminskiP. (2008). Order effects in batteries of questions. Quality and Quantity, 42, 477–490.
32.
SnijdersT. A. B.BoskerR. J. (2012). Multilevel analysis. An introduction to basic and advanced multilevel modeling. Thousand Oaks, CA: Sage.
33.
SternM. J. (2008). The use of client-side paradata in analyzing the effects of visual layout on changing responses in web surveys. Field Methods, 20, 377–398.
34.
StrackF. (1992). “Order effects” in survey research: Activation and information functions of preceding questions. In SchwarzN.SudmanS. (Eds.), Context effects in social and psychological research (pp. 23–34). New York, NY: Springer-Verlag.
35.
StrackF.SchwarzN.BlessH.KüblerA.WänkeM. (1993). Awareness of the influence as a determinant of assimilation versus contrast. European Journal of Social Psychology, 23, 53–62.
36.
SudmanS.BradburnN. M. (1986). Asking questions. San Francisco, CA: Jossey-Bass.
37.
SudmanS.BradburnN. M.SchwarzN. (1996). Thinking about answers. The application of cognitive processes to survey methodology. San Francisco, CA: Jossey-Bass.
38.
The American Association for Public Opinion Research. (2008). Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 5th edition. Lenexa, Kansas: AAPOR.
39.
TodorovA. (2000). Context effects in national health surveys. Effects of preceding questions on reporting serious difficulty seeing and legal blindness. Public Opinion Quarterly, 64, 65–76.
40.
ToepoelV.DasM.Van SoestA. (2009). Design of web questionnaires. The effects of the number of items per screen. Field Methods, 21, 200–213.
41.
TourangeauR.RipsL. J.RasinskiK. (2000). The psychology of survey response. Cambridge, MA: Cambridge University Press.
42.
TukeyJ. (1977). Exploratory Data Analysis. Reading, MA: Addison-Wesley.
43.
YanT.OlsonK. (2013). Analyzing paradata to investigate measurement error. In KreuterF. (Ed.), Improving surveys with paradata. Analytic uses of process information (pp. 73–95). New York, NY: Wiley.
