Why Semantic Differentials in Web-Based Research Should Be Made from Visual Analogue Scales and Not from 5-Point Scales

Abstract

In a web experiment, participants were randomly assigned to two semantic differentials either made from discrete 5-point ordinal rating scales or from continuous visual analogue scales (VASs) with 250 gradations. Respondents adjusted their ratings with VASs more often to maximize the precision of answers, which had a beneficial effect on data quality. No side effects like differences in means, higher dropout, more nonresponse, or higher response times were observed. Overall, the combination of semantic differentials and VASs results in a number of advantages. Potential for further research is discussed.

Keywords

visual analogue scale VAS Likert scale semantic differential web survey

Get full access to this article

View all access options for this article.

References

Averbuch

Katzper

. 2004. Assessment of visual analog versus categorical scale for measurement of osteoarthritis pain. Journal of Clinical Pharmacology 44:368–72.

Best

S. J.

Krueger

B. S.

. 2004. Internet data collection. Thousand Oaks, CA: SAGE.

Bosnjak

2001. Participation in non-restricted web surveys: A typology and explanatory model for item-nonresponse. In Dimensions of Internet science, eds. Reips

U.-D.

Bosnjak

, 193–207. Lengerich, Germany: Pabst Science Publishers.

Buchanan

Reips

U.-D.

. 2001. Platform-dependent biases in online research: Do Mac users really think different? In Perspectives on Internet research: Concepts and methods, eds. Jonas

K. J.

Breuer

Schauenburg

Boos

http://kiwi.uni-psych.gwdg.de/congress/gor-2001/contrib/buchanan-tom (accessed March 4, 2012).

Cork

R. C.

Isaac

Elsharydah

Saleemi

Zavisca

Lori

2004. A comparison of the verbal rating scale and the visual analog scale for pain assessment. The Internet Journal of Anesthesiology http://www.ispub.com/journal/the-internet-journal-of-anesthesiology/volume-8-number-1/a-comparison-of-the-verbal-rating-scale-and-the-visual-analog-scale-for-pain-assessment.html (accessed March 4, 2012).

Couper

M. P.

Tourangeau

Conrad

F. G.

Singer

. 2006. Evaluating the effectiveness of visual analog scales: A web experiment. Social Science Computer Review 24:227–45.

Couper

M. P.

Traugott

M. W.

Lamias

M. J.

. 2001. Web survey design and administration. Public Opinion Quarterly 65:230–53.

de Leeuw

E. D.

Hox

J. J.

Dillman

D. A.

, eds. 2008. International handbook of survey methodology. New York: Lawrence Erlbaum Associates.

Dillman

D. A.

Bowker

D. K.

. 2001. The web questionnaire challenge to survey methodologists. In Dimensions of Internet science, eds. Reips

U.-D.

Bosnjak

, 159–78. Lengerich, Germany: Pabst Science Publishers.

10.

Dillman

D. A.

Smyth

J. D.

Christian

L. M.

. 2009. Internet, mail, and mixed-mode surveys. Hoboken, NJ: Wiley.

11.

Flynn

van Schaik

van Wersh

. 2004. A comparison of multi-item Likert and visual analogue scales for the assessment of transactionally defined coping function. European Journal of Psychological Assessment 20:49–58.

12.

Fuchs

2008. Mobile web survey: A preliminary discussion of methodological implications. In Envisioning the future of survey interviews, eds. Conrad

Schober

, 77–94. New York: Wiley.

13.

Fuchs

Funke

. 2007. Multimedia web surveys: Results from a field experiment on the use of audio and video clips in web surveys. In The challenges of a changing world: Proceedings of the Fifth International Conference of the Association for Survey Computing, eds. Trotman

Burrell

Gerrard

Anderton

Basi

Couper

Morris

Birks

Johnson

A. J.

Baker

Rigg

Taylor

Westlake

, 63–80. Berkeley, CA: ASC.

14.

Funke

Reips

U.-D.

Thomas

R. K.

. 2011. Sliders for the smart: Type of rating scale on the web interacts with educational level. Social Science Computer Review 29:221–31.

15.

Galesic

Tourangeau

Couper

Conrad

. 2008. Eye-tracking data: New insights on response order effects and other signs of cognitive shortcuts in survey responding. Public Opinion Quarterly 72:892–913.

16.

Gerich

2007. Visual analogue scales for mode-independent measurement in self-administered questionnaires. Behavior Research Methods 39:985–92.

17.

Göritz

A. S.

2006. Incentives in web studies: Methodological issues and a review. International Journal of Internet Science 1:58–70.

18.

Groves

R. M.

Fowler

F. J.

Jr. Couper

M. P.

Lepkowski

J. M.

Singer

Tourangeau

. 2009. Survey methodology. 2nd ed. New York: Wiley.

19.

Hayes

M. H. S.

Patterson

D. G.

. 1921. Experimental development of the graphic rating method. Psychological Bulletin 18:98–99.

20.

Heerwegh

2003. Explaining response latencies and changing answers using client side paradata from a web survey. Social Science Computer Review 21:360–73.

21.

Hofmans

Theuns

. 2008. On the linearity of predefined and self-anchoring visual analogue scales. British Journal of Mathematical and Statistical Psychology 61:401–13.

22.

Honing

Reips

U.-D.

. 2008. web-based versus lab-based studies: A response to Kendall (2008). Empirical Musicology Review 3:73–77.

23.

Joinson

McKenna

Postmes

Reips

U.-D.

, eds. 2007. The Oxford handbook of Internet psychology. New York: Oxford University Press.

24.

Krosnick

J. A.

1991. Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5:213–36.

25.

Krosnick

J. A.

1999. Survey research. Annual Review of Psychology 50:537–67.

26.

Mangan

M. A.

Reips

U.-D.

. 2007. Sleep, sex, and the web: Surveying the difficult-to-reach clinical population suffering from sexsomnia. Behavior Research Methods 39:233–36.

27.

McReynold

Ludwig

. 1987. On the history of rating scales. Personality and Individual Differences 8:281–83.

28.

Myles

P. S.

Troedel

Boquest

Reeves

. 1999. The pain visual analog scale: Is it linear or nonlinear? Anesthesia & Analgesia 89:1517–20.

29.

Myles

P. S.

Urquhart

. 2005. The linearity of the visual analogue scale in patients with severe acute pain. Anaesthesia and Intensive Care 33:54–58.

30.

Osgood

C. E.

1952. The nature and measurement of meaning. Psychological Bulletin 49:197–237.

31.

Osgood

C. E.

Suci

G. J.

Tannenbaum

P. H.

. 1957. The measurement of meaning. Urbana: University of Illinois Press.

32.

Reips

U.-D.

2002. Standards for Internet-based experimenting. Experimental Psychology 49:243–56.

33.

Reips

U.-D.

2007. The methodology of Internet-based experiments. In The Oxford handbook of Internet psychology, eds. Joinson

A. A.

McKenna

Postmes

Reips

U.-D.

, 373–90. Oxford: Oxford University Press.

34.

Reips

U.-D.

2008. How Internet-mediated research changes science. In Psychological aspects of cyberspace: Theory, research, applications, ed. Barak

268–94. Cambridge, England: Cambridge University Press. http://gsb.haifa.ac.il/∼sheizaf/cyberpsych/12-Reips.pdf (accessed March 4, 2012).

35.

Reips

U.-D.

2010. Design and formatting in Internet-based research. In Advanced Internet methods in the behavioral sciences, eds. Gosling

Johnson

, 29–43. Washington, DC: American Psychological Association.

36.

Reips

U.-D.

Funke

. 2008. Interval-level measurement with visual analogue scales in Internet-based research: VAS Generator. Behavior Research Methods 40:699–704.

37.

Schwarz

1999. Self-reports: How the questions shape the answers. American Psychologist 54:93–105.

38.

Schwarz

Oyserman

. 2001. Asking questions about behavior: Cognition, communication, and questionnaire construction. American Journal of Evaluation 22:127–60.

39.

Smyth

J. D.

Dillman

D. A.

Christian

L. M.

Stern

M. J.

. 2006. Effects of using visual design principles to group response options in web surveys. International Journal of Internet Science 1:6–16.

40.

Stieger

Reips

U.-D.

. 2010. What are participants doing while filling in an online questionnaire: A paradata collection tool and an empirical study. Computers in Human Behavior 26:1488–95.

41.

Sudman

Bradburn

Schwarz

. 1996. Thinking about answers: The application of cognitive processes to survey methodology. San Francisco: Jossey-Bass.

42.

Tourangeau

Couper

M. P.

Conrad

F. G.

. 2004. Spacing, position, and order: Interpretative heuristics for visual features of survey questions. Public Opinion Quarterly 68:368–93.

43.

Tukey

J. W.

1977. Exploratory data analysis. Reading, MA: Addison-Wesley.