This study explored the influence of Internet memes, specifically image macros of animals with motivational captions, on survey respondents’ engagement with the survey-taking experience and subsequent data quality. A web-based field experiment was conducted with online survey respondents from two sample sources, one crowdsourced, and one commercially managed online panel. Half of the respondents from each sample source were randomly selected to see the memes at various points throughout the survey; the other half did not. Direct and indirect measures of survey engagement and response quality were used to assess effectiveness of the memes. Quantitative results were inconclusive, with few significant differences found in measures of engagement and data quality between respondents in the meme or control condition in either sample source. However, qualitative open-ended comments from respondents who saw the memes in both sample groups revealed that memes provide respondents a fun break and relief from the cognitive burdens of answering online survey questions. In conclusion, memes represent a relatively inexpensive and easy way for survey researchers to connect with respondents and show appreciation for their time and effort.
BaghalT. A.LynnP. (2015). Using motivational statements in web-instrument design to reduce item-missing rates in a mixed-mode context. Public Opinion Quarterly, 79(2), 568–579.
2.
BakerS. J.BlumbergJ. M.BrickM. P.CouperM.CourtrightJ. M.DennisD.DillmanD. A.ZahsD. (2010). Research synthesis AAPOR report on online panels. Public Opinion Quarterly, 74(4), 711–781.
BradburnN. (1977). Respondent burden. In Health survey research methods: Research Proceedings from the 2nd Biennial Conference (PHS Publication No 79-3207, pp. 49–53). http://www.asasrms.org/Proceedings/papers/1978_007.pdf
5.
BrüggenE.DholakiaU. M. (2010). Determinants of participation and response effort in web panel surveys. Journal of Interactive Marketing, 24, 239–250.
6.
BrüggenE.WetzelsM.de RuyterK. (2011). Individual differences in motivation to participate in online panels: The effect on response rate and response quality perceptions. International Journal of Market Research, 53(3), 369–390.
7.
CallegaroM.DiSograC. (2008). Computing response metrics for online panels. Public Opinion Quarterly, 72(5), 1008–1032.
8.
CallegaroM.Lozar ManfredaK.VehovarV. (2015). Web survey methodology. SAGE.
9.
CechanowicaJ.GutwinC.BrownellB.GoodfellowL. (2013). Effects of gamification on Participation and data quality in a real-world market research domain. In Gamification 2013 (pp. 58–65). ACN.
10.
ChienY.-T.ChangC.-Y. (2012). Exploring the impact of animation-based questionnaire on conducting a web-based educational survey and its association with vividness of respondents’ visual images. British Journal of Educational Technology, 43(3), E81–E85.
11.
CookeM. (2011). Keynote: The engagement agenda. International Journal of Market Research, 53(1), 115–117.
12.
CouperM. P.ConradF. G.TourangeauR. (2007). Visual context effects in web surveys. Public Opinion Quarterly, 71(4), 623–634.
13.
CouperM. P.TourangeauR.KenyonK. (2004). Picture this! Exploring visual effects in web surveys. Public Opinion Quarterly, 68(2), 255–266.
14.
DavisonP. (2012). The language of internet memes. In MandibergM. (Ed.), The social media reader (pp. 120–136). New York University Press.
15.
DillmanD. A.SmythJ. D.ChristianL. M. (2009). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). John Wiley & Sons.
16.
DillmanD. A.TortoraR. D.ConradtJ.BowkerD. (1998). Influence of the plain vs fancy design on response rates for web surveys. Proceedings of the Survey Research Methods Section, American Statistical Association. www.websm.org/db/12/1862/rec/
17.
DolnicarS.GrünB.YanamandramV. (2013). Dynamic, interactive survey questions can increase survey data quality. Journal of Travel & Tourism Marketing, 30(7), 690–699.
18.
Downes-Le GuinT.BakerR.MechlingJ.RuyleE. (2012). Myths and realities of respondent engagement in online surveys. International Journal of Market Research, 54(5), 613–633.
19.
EwingT. (2012). Not just playing around. Quirk’s Marketing Research Review, 26(3), 30–34.
20.
FunkeF. (2016). A web experiment showing negative effects of slider scales compared to visual analogue scales and radio button scales. Social Science Computer Review, 34(2), 244–254.
21.
GalesicM.BosnjakM. (2009). Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly, 73(2), 349–360.
GrovesR. M.CouperM. P. (1998). Nonresponse in household interview surveys. John Wiley.
24.
GrovesR. M.PresserS.DipkoS. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68(1), 2–31.
25.
GrundlinghL. (2018). Memes as speech acts. Social Semiotics, 28(2), 147–168.
26.
GummerT.VogelV.KunzT.RoßmannJ. (2020). Let’s put a smile on that scale: Findings from three web survey experiments. International Journal of Market Research, 62(1), 18–26.
27.
HauserD. J.SchwarzN. (2016). Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48(1), 400–407. https://doi.org/10.3758/s13428-015-0578-z
HymanM. R.SierraJ. J. (2010). Adjusting self-reported attitudinal data for mischievous respondents. International Journal of Market Research, 54(1), 129–145.
30.
KaminskaO.McCutcheonA. L.BillietJ. (2010). Satisficing among reluctant respondents in a cross-national context. Public Opinion Quarterly, 74(5), 956–984.
31.
KelmanH. C. (1958). Compliance, identification, and internalization: Three processes of attitude change. Journal of Conflict Resolution, 2(1), 51–60.
32.
KelmanH. C. (1961). Processes of opinion change. Public Opinion Quarterly, 25(1), 57–78.
33.
KeuschF. (2013). The role of topic interest and topic salience in online panel web surveys. International Journal of Market Research, 55(1), 59–80.
34.
KrosnickJ. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 213–236.
35.
KrosnickJ. A.AlwinD. F. (1987). An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly, 51(2), 201–219.
36.
KostykA.ZhouW.HymanM. R. (2019). Using surveytainment to counter declining survey data quality. Journal of Business Research, 95, 211–219.
LitmanL.RobinsonJ.RosenzweigC. (2015). The relationship between motivation, monetary compensation, and data quality among US- and India-based workers on Mechanical Turk. Behavior Research Methods, 47(2), 519–528. https://doi.org/10.3758/s13428-014-0483-x
39.
Lozar-ManfredaK.BatageljZ.VehovarV. (2002). Design of web survey questionnaires: Three basic experiments. Journal of Computer-Mediated Communication, 7(3), JCMC731. https://doi.org/10.1111/j.1083-6101.2002.tb00149.x
40.
MacerT.WilsonS. (2017). Observations from 12 years of an annual market research technology survey. International Journal of Market Research, 59(2), 173–198.
41.
MalhotraY.GallettaD. (2005). A multidimensional commitment model of volitional systems adoption and usage behavior. Journal of Applied Psychology, 69(3), 362–278.
42.
MeadeA. W.CraigS. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455.
43.
MurrayN.ManraiA.ManraiL. (2013). Memes, memetics and marketing: A state of the art review and a lifecycle model of meme management in advertising. In MoutinhoL.BigneE.ManraiA. K. (Eds.), The Routledge companion to the future of marketing (pp. 328–344). Routledge.
44.
OnanugaP. (2020). Transnationalizing humour on social media: A linguistic analysis of ideology, identity, and didactics in. Robert Mugabe Quotes Memes. Digital Scholarship in the Humanities, 35(2), 399–416.
45.
PalanS.SchitterC. (2018). Prolific.ac—A subject pool for online experiments. Journal of Behavioral and Experimental Finance, 17, 22–27.
46.
PassL. J.DolnicarS.KarlssonL. (2018). Instructional manipulation checks: A longitudinal analysis with implications for MTurk. International Journal of Research in Marketing, 35, 258–269.
47.
PeerE.BrandimarteL.SamatS.AcquistiA. (2017). Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology, 70, 153–163.
48.
PulestonJ. (2011). Online research—Game on! A look at how gaming techniques can Transform your online research. In BirksD. F.BanksR.GerradL.JohnsonA.KhanR.MacerT.… WillsP. (Eds.), Shifting the boundaries of research (pp. 20–50). Association for Survey Computing.
49.
PulestonJ. (2012, January). Gamification 101: From theory to practice part I. Quirk’s E-Newsletter.
50.
RedlineC. D.DillmanD. A. (2002). The influence of alternative visual designs on respondents’ performance with branching instructions in self-administered questionnaires. In GrovesR. M.DillmanD. A.EltingeJ. L.LittleR. J. A. (Eds.), Survey Nonresponse (pp. 179–193). Wiley.
51.
RobertsC.GilbertE.AllumN.EisnerL. (2019). Satisficing in surveys: A systematic review of the literature. Public Opinion Quarterly, 83(3), 598–626.
52.
RosterC. A.AlbaumG.SmithS. M. (2017). Effect of topic sensitivity on online survey panelists’ motivation and data quality. Journal of Marketing Theory and Practice, 25(1), 1–16.
ShifmanL. (2014a). The cultural logic of photo-based meme genres. Journal of Visual Culture, 13(3), 340–358.
55.
ShifmanL. (2014b). Memes in digital culture. MIT Press.
56.
SleepD.PulestonJ. (2011). The game experiments: Researching how gaming techniques can be used to improve the quality of feedback from online research [Presented at Impact: Research Reloaded]. Esomar.
57.
SmithS. M.RosterC. A.GoldenL. L.AlbaumG. S. (2016). A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples. Journal of Business Research, 69, 3139–3148.
58.
TourangeauR.CouperM. P.ConradF. G. (2004). Spacing, position, and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly, 71, 91–112.
59.
TourangeauR.RipsL. J.RasinskiK. (2000). The psychology of survey response. Cambridge University Press.
60.
VincenteP.ReisE. (2010). Using questionnaire design to fight nonresponse bias in web surveys. Social Science Computer Review, 28(2), 251–267.
61.
WalkerR. W.CookW. A. (2013). You can’t put a price tag on a survey participant’s enjoyment: The latest findings from the ARF’s “Foundations of Quality” research. Journal of Advertising Research, 53(3), 254–257.
62.
WardM. K.MeadeA. W. (2018). Applying social psychology to prevent careless responding during online surveys. Applied Psychology: An International Review, 67(2), 231–263. https://doi.org/10.1111/apps.12118
