Using memes in online surveys to engage and motivate respondents

Abstract

This study explored the influence of Internet memes, specifically image macros of animals with motivational captions, on survey respondents’ engagement with the survey-taking experience and subsequent data quality. A web-based field experiment was conducted with online survey respondents from two sample sources, one crowdsourced, and one commercially managed online panel. Half of the respondents from each sample source were randomly selected to see the memes at various points throughout the survey; the other half did not. Direct and indirect measures of survey engagement and response quality were used to assess effectiveness of the memes. Quantitative results were inconclusive, with few significant differences found in measures of engagement and data quality between respondents in the meme or control condition in either sample source. However, qualitative open-ended comments from respondents who saw the memes in both sample groups revealed that memes provide respondents a fun break and relief from the cognitive burdens of answering online survey questions. In conclusion, memes represent a relatively inexpensive and easy way for survey researchers to connect with respondents and show appreciation for their time and effort.

Keywords

computer-mediated communication memes online survey panel samples response quality survey engagement

Get full access to this article

View all access options for this article.

References

Baghal

T. A.

Lynn

(2015). Using motivational statements in web-instrument design to reduce item-missing rates in a mixed-mode context. Public Opinion Quarterly, 79(2), 568–579.

Baker

S. J.

Blumberg

J. M.

Brick

M. P.

Couper

Courtright

J. M.

Dennis

Dillman

D. A.

Zahs

(2010). Research synthesis AAPOR report on online panels. Public Opinion Quarterly, 74(4), 711–781.

Baker-Prewitt

Miller

(2009). Beyond “trapping” the undesirable panelist: The use of red herrings to reduce satisficing. Presented at 2009 CASRO Panel Quality Conference, New Orleans, LA, United States. https://docplayer.net/38492373-Beyond-trapping-the-undesirable-panelist-the-use-of-red-herrings-to-reduce-satisficing.html

Bradburn

(1977). Respondent burden. In Health survey research methods: Research Proceedings from the 2nd Biennial Conference (PHS Publication No 79-3207, pp. 49–53). http://www.asasrms.org/Proceedings/papers/1978_007.pdf

Brüggen

Dholakia

U. M.

(2010). Determinants of participation and response effort in web panel surveys. Journal of Interactive Marketing, 24, 239–250.

Brüggen

Wetzels

de Ruyter

(2011). Individual differences in motivation to participate in online panels: The effect on response rate and response quality perceptions. International Journal of Market Research, 53(3), 369–390.

Callegaro

DiSogra

(2008). Computing response metrics for online panels. Public Opinion Quarterly, 72(5), 1008–1032.

Callegaro

Lozar Manfreda

Vehovar

(2015). Web survey methodology. SAGE.

Cechanowica

Gutwin

Brownell

Goodfellow

(2013). Effects of gamification on Participation and data quality in a real-world market research domain. In Gamification 2013 (pp. 58–65). ACN.

10.

Chien

Y.-T.

Chang

C.-Y.

(2012). Exploring the impact of animation-based questionnaire on conducting a web-based educational survey and its association with vividness of respondents’ visual images. British Journal of Educational Technology, 43(3), E81–E85.

11.

Cooke

(2011). Keynote: The engagement agenda. International Journal of Market Research, 53(1), 115–117.

12.

Couper

M. P.

Conrad

F. G.

Tourangeau

(2007). Visual context effects in web surveys. Public Opinion Quarterly, 71(4), 623–634.

13.

Couper

M. P.

Tourangeau

Kenyon

(2004). Picture this! Exploring visual effects in web surveys. Public Opinion Quarterly, 68(2), 255–266.

14.

Davison

(2012). The language of internet memes. In Mandiberg

(Ed.), The social media reader (pp. 120–136). New York University Press.

15.

Dillman

D. A.

Smyth

J. D.

Christian

L. M.

(2009). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). John Wiley & Sons.

16.

Dillman

D. A.

Tortora

R. D.

Conradt

Bowker

(1998). Influence of the plain vs fancy design on response rates for web surveys. Proceedings of the Survey Research Methods Section, American Statistical Association. www.websm.org/db/12/1862/rec/

17.

Dolnicar

Grün

Yanamandram

(2013). Dynamic, interactive survey questions can increase survey data quality. Journal of Travel & Tourism Marketing, 30(7), 690–699.

18.

Downes-Le Guin

Baker

Mechling

Ruyle

. (2012). Myths and realities of respondent engagement in online surveys. International Journal of Market Research, 54(5), 613–633.

19.

Ewing

(2012). Not just playing around. Quirk’s Marketing Research Review, 26(3), 30–34.

20.

Funke

(2016). A web experiment showing negative effects of slider scales compared to visual analogue scales and radio button scales. Social Science Computer Review, 34(2), 244–254.

21.

Galesic

Bosnjak

(2009). Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly, 73(2), 349–360.

22.

Goodman

J. K.

Paolacci

(2017). Crowdsourcing consumer research. Journal of Consumer Research, 44, 196–210.

23.

Groves

R. M.

Couper

M. P.

(1998). Nonresponse in household interview surveys. John Wiley.

24.

Groves

R. M.

Presser

Dipko

(2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68(1), 2–31.

25.

Grundlingh

(2018). Memes as speech acts. Social Semiotics, 28(2), 147–168.

26.

Gummer

Vogel

Kunz

Roßmann

(2020). Let’s put a smile on that scale: Findings from three web survey experiments. International Journal of Market Research, 62(1), 18–26.

27.

Hauser

D. J.

Schwarz

(2016). Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48(1), 400–407. https://doi.org/10.3758/s13428-015-0578-z

28.

Hulland

Miller

(2018). Keep on Turkin’? Journal of the Academy of Marketing Science, 46, 789–794. https://doi.org/10.1007/s11747-018-0587-4

29.

Hyman

M. R.

Sierra

J. J.

(2010). Adjusting self-reported attitudinal data for mischievous respondents. International Journal of Market Research, 54(1), 129–145.

30.

Kaminska

McCutcheon

A. L.

Billiet

(2010). Satisficing among reluctant respondents in a cross-national context. Public Opinion Quarterly, 74(5), 956–984.

31.

Kelman

H. C.

(1958). Compliance, identification, and internalization: Three processes of attitude change. Journal of Conflict Resolution, 2(1), 51–60.

32.

Kelman

H. C.

(1961). Processes of opinion change. Public Opinion Quarterly, 25(1), 57–78.

33.

Keusch

(2013). The role of topic interest and topic salience in online panel web surveys. International Journal of Market Research, 55(1), 59–80.

34.

Krosnick

J. A.

(1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 213–236.

35.

Krosnick

J. A.

Alwin

D. F.

(1987). An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly, 51(2), 201–219.

36.

Kostyk

Zhou

Hyman

M. R.

(2019). Using surveytainment to counter declining survey data quality. Journal of Business Research, 95, 211–219.

37.

Lipner

W. E.

(2007). The future of online market research. Journal of Advertising Research, 47(2), 142–146. https://doi.org/10.2501/S002184990707016x

38.

Litman

Robinson

Rosenzweig

(2015). The relationship between motivation, monetary compensation, and data quality among US- and India-based workers on Mechanical Turk. Behavior Research Methods, 47(2), 519–528. https://doi.org/10.3758/s13428-014-0483-x

39.

Lozar-Manfreda

Batagelj

Vehovar

(2002). Design of web survey questionnaires: Three basic experiments. Journal of Computer-Mediated Communication, 7(3), JCMC731. https://doi.org/10.1111/j.1083-6101.2002.tb00149.x

40.

Macer

Wilson

(2017). Observations from 12 years of an annual market research technology survey. International Journal of Market Research, 59(2), 173–198.

41.

Malhotra

Galletta

(2005). A multidimensional commitment model of volitional systems adoption and usage behavior. Journal of Applied Psychology, 69(3), 362–278.

42.

Meade

A. W.

Craig

S. B.

(2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455.

43.

Murray

Manrai

(2013). Memes, memetics and marketing: A state of the art review and a lifecycle model of meme management in advertising. In Moutinho

Bigne

Manrai

A. K.

(Eds.), The Routledge companion to the future of marketing (pp. 328–344). Routledge.

44.

Onanuga

(2020). Transnationalizing humour on social media: A linguistic analysis of ideology, identity, and didactics in. Robert Mugabe Quotes Memes. Digital Scholarship in the Humanities, 35(2), 399–416.

45.

Palan

Schitter

(2018). Prolific.ac—A subject pool for online experiments. Journal of Behavioral and Experimental Finance, 17, 22–27.

46.

Pass

L. J.

Dolnicar

Karlsson

(2018). Instructional manipulation checks: A longitudinal analysis with implications for MTurk. International Journal of Research in Marketing, 35, 258–269.

47.

Peer

Brandimarte

Samat

Acquisti

(2017). Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology, 70, 153–163.

48.

Puleston

(2011). Online research—Game on! A look at how gaming techniques can Transform your online research. In Birks

D. F.

Banks

Gerrad

Johnson

Khan

Macer

… Wills

. (Eds.), Shifting the boundaries of research (pp. 20–50). Association for Survey Computing.

49.

Puleston

(2012, January). Gamification 101: From theory to practice part I. Quirk’s E-Newsletter.

50.

Redline

C. D.

Dillman

D. A.

(2002). The influence of alternative visual designs on respondents’ performance with branching instructions in self-administered questionnaires. In Groves

R. M.

Dillman

D. A.

Eltinge

J. L.

Little

R. J. A.

(Eds.), Survey Nonresponse (pp. 179–193). Wiley.

51.

Roberts

Gilbert

Allum

Eisner

(2019). Satisficing in surveys: A systematic review of the literature. Public Opinion Quarterly, 83(3), 598–626.

52.

Roster

C. A.

Albaum

Smith

S. M.

(2017). Effect of topic sensitivity on online survey panelists’ motivation and data quality. Journal of Marketing Theory and Practice, 25(1), 1–16.

53.

Roster

C. A.

Lucianetti

Albaum

(2015). Exploring slider vs. categorical response formats in web-based surveys. Journal of Research Practice, 11(1), D1. http://jrp.icaap.org/index.php/jrp/article/view/509/413

54.

Shifman

(2014a). The cultural logic of photo-based meme genres. Journal of Visual Culture, 13(3), 340–358.

55.

Shifman

(2014b). Memes in digital culture. MIT Press.

56.

Sleep

Puleston

(2011). The game experiments: Researching how gaming techniques can be used to improve the quality of feedback from online research [Presented at Impact: Research Reloaded]. Esomar.

57.

Smith

S. M.

Roster

C. A.

Golden

L. L.

Albaum

G. S.

(2016). A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples. Journal of Business Research, 69, 3139–3148.

58.

Tourangeau

Couper

M. P.

Conrad

F. G.

(2004). Spacing, position, and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly, 71, 91–112.

59.

Tourangeau

Rips

L. J.

Rasinski

(2000). The psychology of survey response. Cambridge University Press.

60.

Vincente

Reis

(2010). Using questionnaire design to fight nonresponse bias in web surveys. Social Science Computer Review, 28(2), 251–267.

61.

Walker

R. W.

Cook

W. A.

(2013). You can’t put a price tag on a survey participant’s enjoyment: The latest findings from the ARF’s “Foundations of Quality” research. Journal of Advertising Research, 53(3), 254–257.

62.

Ward

M. K.

Meade

A. W.

(2018). Applying social psychology to prevent careless responding during online surveys. Applied Psychology: An International Review, 67(2), 231–263. https://doi.org/10.1111/apps.12118