Revising a measure of anti-oppressive social media literacy through the use of cognitive interviewing

Abstract

Current theories and measures of social media literacy are politically neutral, which poses a problem as far-right influences permeate mainstream social media platforms. The theory of Anti-Oppressive Social Media Literacy describes three orientations social media users may display toward far-right content: endorsement of far-right content, ambivalence toward far-right content, or rejection of far-right content. Based on this theory, we created a survey measure in which participants were asked to rate their likelihood of responding in various ways to hypothetical social media posts that included far-right dog whistles. Our sample of 14 young adults aged 18–24 consisted of 57% Black participants, 71% full-time students, and 71% people who worked at least part-time. We conducted cognitive interviews to examine how participants interpreted these scenarios and their response items. Findings revealed the measure’s overall effectiveness and identified actionable revisions. This study can inform future measure development based on Anti-Oppressive Social Media Literacy.

Keywords

Anti-oppression cognitive interview measure development social media literacy young adults

Get full access to this article

View all access options for this article.

References

Al-Kire

Miller

Pasek

, et al. (2024) White by another name? Can anti-Christian bias claims serve as a racial dog whistle? Psychological Science 35(4): 415–434.

Alter

(2021) Talking with Tucker Carlson, the most powerful conservative in America. Time, 15 July. Available at: https://time.com/6080432/tucker-carlson-profile/

Arbeit

Burnham

SLF

de Four

, et al. (2020) Youth practitioners can counter fascism: What we know and what we need. Journal of Youth Development 15(5): 37–67.

Arbeit

Negrete

Berger

, et al. (2024) Antifascist praxis in developmental science: Possibilities for collective resistance to fascism. Child Development Perspectives 18(2): 73–81.

Barratt

(2006) The Barratt Simplified Measure of Social Status (BSMSS): measuring SES (Unpublished manuscript). http://socialclassoncampus.blogspot.com/2012/06/barratt-simplified-measure-of-social.html

Bhat

Klein

(2020) Covert hate speech: white nationalists and dog whistle communication on Twitter. In: Bouvier

Rosenbaum

(eds) Twitter, the Public Sphere, and the Chaos of Online Deliberation. Cham: Springer, pp. 151–172.

Block

(2022) Accusations of “grooming” are the latest political attack – with homophobic origins. NPR, 11 May. Available at: https://www.npr.org/2022/05/11/1096623939/accusations-grooming-political-attack-homophobic-origins

Bray

(2017) Antifa: The Anti-Fascist Handbook. Brooklyn, NY: Melville House.

Breland

(2019) Why are right-wing conspiracies so obsessed with pedophilia? Mother Jones, July/August. Available at: https://www.motherjones.com/politics/2019/07/why-are-right-wing-conspiracies-so-obsessed-with-pedophilia/

10.

Burley

(2017) Fascism Today: What It Is and How to End It. Chico, CA: AK Press.

11.

Burley

(ed.) (2022) No Pasarán! Antifascist Dispatches from a World in Crisis. AK Press.

12.

Burnham

SLF

Arbeit

Hilliard

(2022) The subtle spread of hateful memes: Examining engagement intentions among parents of adolescents. Social Media + Society 8(2): 1–14.

13.

Burnham

SLF

Arbeit

(2023) Social media literacy to confront far-right content: Saying “no” to neutrality. Human Development 67(3): 117–134.

14.

Campbell

(2018) Charlottesville isn’t playing the media’s “both sides” game anymore. Huffpost, 7 August. Available at: https://www.huffpost.com/entry/charlottesville-isnt-playing-the-medias-both-sides-game-anymore_n_5b63390ce4b0b15abaa0e48b

15.

Césaire

(2000 [1950]) Discourse on Colonialism (trans. Pinkham

). New York: NYU Press.

16.

Cohen

(2018) Exploring echo-systems: how algorithms shape immersive media environments. Journal of Media Literacy Education 10(2): 139–151.

17.

Corneau

Stergiopoulos

(2012) More than being against it: anti-racism and anti-oppression in mental health services. Transcultural Psychiatry 49(2): 261–282.

18.

Dalrymple

Burke

(1995) Anti-Oppressive Practice: Social Care and the Law. Maidenhead: Open University Press.

19.

Daniels

(2018) The algorithmic rise of the “Alt-Right.” Contexts 17(1): 60–65.

20.

Deci

Ryan

(1985) The general causality orientations scale: self-determination in personality. Journal of Research in Personality 19(2): 109–134.

21.

Dickson

(2022) The far right’s next target for a bullshit moral panic: antidepressants. Rolling Stone, 14 July. Available at: https://www.rollingstone.com/culture/culture-features/tucker-carlson-antidepressants-ssri-misinformation-moral-panic-1378712/

22.

Ebner

(2023) From margins to mainstream: how extremism has conquered the political middle. International Centre for Counter-Terrorism, 10 August. Available at: https://icct.nl/publication/margins-mainstream-how-extremism-has-conquered-political-middle

23.

Edison Hayden

(2021) ‘There’s nothing you can do’: the legacy of #pizzagate. Southern Poverty Law Center Hatewatch, 7 July. Available at: https://www.splcenter.org/hatewatch/2021/07/07/theres-nothing-you-can-do-legacy-pizzagate

24.

Ellison

Izadi

(2021) ‘Definitely not the results we want’: Facebook staff lamented “perverse incentives” for media. The Washington Post, 26 October. Available at: https://www.washingtonpost.com/business/2021/10/26/conservative-media-misinformation-facebook/

25.

Feng

(2024) YouTube’s algorithm more likely to recommend users right-wing and religious content, research finds. NBC News, 18 June. Available at: https://www.nbcnews.com/tech/tech-news/youtubes-algorithm-recommends-users-right-wing-religious-content-resea-rcna155478

26.

Field

Park

Theophilio

, et al. (2022) An analysis of emotions and the prominence of positivity in #BlackLivesMatter tweets. PNAS 119(35): e2205767119.

27.

Flores

(2022) The new world order: the historical origins of a dangerous modern conspiracy theory. Center on Terrorism, Extremism, and Counterterrorism at Middlebury Institute of International Studies, 30 May. Available at: https://www.middlebury.edu/institute/academics/centers-initiatives/ctec/ctec-publications/new-world-order-historical-origins-dangerous

28.

Gay & Lesbian Alliance Against Defamation (GLAAD) (2022) Meta and Twitter refuse action on “Libs of TikTok” posts as doctors and staff of Boston Children’s Hospital receive death threats. GLAAD. Available at: https://glaad.org/meta-and-twitter-refuse-action-libs-tiktok-posts-doctors-and-staff-boston-childrens/

29.

Gay & Lesbian Alliance Against Defamation (GLAAD) (2023) Moms for liberty uses false and inflammatory language to vilify LGBTQ people. GLAAD. Available at: https://glaad.org/moms-for-liberty-groomer-rhetoric-anti-lgbtq/

30.

Gonzalez

(2022) False QAnon conspiracies in politics: what to know as midterm elections approach. CNET, 30 July. Available at: https://www.cnet.com/news/misinformation/false-qanon-conspiracies-in-politics-what-to-be-aware-of-with-the-midterms-approaching/

31.

Gorcenski

(2018) The Pittsburgh shooting proves Trump emboldens racists – & social media empowers them. Refinery29, 28 October. Available at: https://www.refinery29.com/en-us/2018/10/215302/pittsburgh-shooting-reason-white-supremacists-trump

32.

Gorcenski

(2019) The Proud Boys: a republican party street gang. Political Research Associates, 28 February. Available at: https://politicalresearch.org/2019/02/28/proud-boys

33.

Graham

(2016) Inter-ideological mingling: white extremist ideology entering the mainstream on Twitter. Sociological Spectrum 36(1): 24–36.

34.

Gruzd

Wellman

Takhteyev

(2011) Imagining Twitter as an imagined community. American Behavioral Scientist 55(10): 1294–1318.

35.

Hayes

(2021) Laughing at mass death brings us closer to fascism. Truthout, 29 September. Available at: https://truthout.org/audio/laughing-at-mass-death-brings-us-closer-to-fascism/

36.

Hayes

(2022) Fascism has gone mainstream. Truthout, 9 September. Available at: https://truthout.org/audio/fascism-has-gone-mainstream/

37.

Hayes

Dixon

(2020) “We surrender nothing and no one”: a playbook for solidarity amid fascist terror. Truthout, 5 June. Available at: https://truthout.org/audio/we-surrender-nothing-and-no-one-a-playbook-for-solidarity-amid-fascist-terror

38.

Hayes

Kaba

(2023) Let This Radicalize You: Organizing and the Revolution of Reciprocal Care. Chicago, IL: Haymarket Books.

39.

Hoffman

Ware

Shapiro

(2020) Assessing the threat of incel violence. Studies in Conflict & Terrorism 43(7): 565–587.

40.

Hope

(2022) The Black antifascist tradition: a primer. In: Burley

(ed.) No Pasarán! [They Shall Not Pass!]: Antifascist Dispatches from a World in Crisis. Chico, CA: AK Press, pp. 63–87.

41.

Johnson

(2019) Ur-fascism and neo-fascism. The Journal of International Relations, Peace Studies, and Development 5(1): 2.

42.

Jones

(2021) How to manufacture a moral panic. New York Magazine, 11 July. Available at: https://nymag.com/intelligencer/2021/07/christopher-rufo-and-the-critical-race-theory-moral-panic.html

43.

Kahne

Middaugh

Lee

, et al. (2011) Youth online activity and exposure to diverse perspectives. New Media & Society 14(3): 492–512.

44.

Kaplan

(1972) On the ambivalence-indifference problem in attitude theory and measurement: A suggested modification of the semantic differential technique. Psychological Bulletin 77(5): 361–372.

45.

Katsambekis

(2023) Mainstreaming authoritarianism. The Political Quarterly 94(3): 428–436.

46.

Kelly

DiBranco

DeCook

(2021) Misogynist incels and male supremacism: overview and recommendations for addressing the threat of male supremacist violence. New America, 18 February. Available at: https://www.newamerica.org/political-reform/reports/misogynist-incels-and-male-supremacism/

47.

Lacy

(2022) Elon Musk’s growing purge of his Twitter critics – at the behest of the far right. The Intercept, 16 December. Available at: https://theintercept.com/2022/12/16/twitter-elon-musk-andy-ngo-its-going-down/

48.

Legate

Weinstein

Przybylski

(2019) Parenting strategies and adolescents’ cyberbullying behaviors: evidence from a preregistered study of parent–child dyads. Journal of Youth and Adolescence 48(2): 399–409.

49.

Lerner

(2018) Concepts and Theories of Human Development (4th ed.). New York: Routledge.

50.

Lerner

Hershberg

Hilliard

, et al. (2015) Concepts and theories in human development. In: Bornstein

Lamb

(eds) Developmental Science: An Advanced Textbook. 7th ed. New York: Psychology Press, pp. 3–42.

51.

Lieberman

Schroeder

(2020) Two social lives: how differences between online and offline interaction influence social outcomes. Current Opinion in Psychology 31: 16–21.

52.

McCluskey

(2022) How addictive social media algorithms could finally face a reckoning in 2022. Time, 4 January. Available at: https://time.com/6127981/addictive-algorithms-2022-facebook-instagram/

53.

Mackey

Lee

(2022) Left-wing voices are silenced on Twitter as far-right trolls advise Elon Musk. The Intercept, 29 November. Available at: https://theintercept.com/2022/11/29/elon-musk-twitter-andy-ngo-antifascist/

54.

Moffitt

(2022) How do mainstream parties ‘become’ mainstream, and pariah parties ‘become’ pariahs? Conceptualizing the processes of mainstreaming and pariahing in the labelling of political parties. Government and Opposition 57(3): 385–403.

55.

Mondon

(2025) Populism, public opinion, and the mainstreaming of the far right: the “immigration issue” and the construction of a reactionary “people.” Politics 45(1): 19–26.

56.

Muslim Justice League (2020) StopCVE now, May. Available at: https://muslimjusticeleague.org/cve/

57.

National Coordinator for Counterterrorism and Security (NCSS) (2024) Memes as an online weapon: an analysis of the use of memes by the far right. National Coordinator for Counterterrorism and Security: Ministry of Justice and Security, 1 August. Available at: https://english.nctv.nl/documents/publications/2024/08/01/far-right-memes-undermining-and-far-from-recognizable

58.

Nelson

(2022) How stochastic terrorism uses disgust to incite violence. Scientific American, 5 November. Available at: https://www.scientificamerican.com/article/how-stochastic-terrorism-uses-disgust-to-incite-violence/

59.

Newton

(2018) Why social media is friend to far-right politicians around the world. The Verge, 30 October. Available at: https://www.theverge.com/2018/10/30/18040510/stochastic-terrorism-democratic-recession-gab

60.

Ortutay

(2025) Meta rolls back hate speech rules as Zuckerberg cites “recent elections” as a catalyst. Associated Press, 8 January. Available at: https://apnews.com/article/meta-facebook-hate-speech-trump-immigrant-transgender-41191638cd7c720b950c05f9395a2b49

61.

Padilla

Leighton

(2017) Cognitive interviewing and think aloud methods. In: Zumbo

Hubley

(eds) Understanding and Investigating Response Processes in Validation Research. Cham: Springer, pp. 211–228.

62.

Padilla

Benítez

Castillo

(2013) Obtaining validity evidence by cognitive interviewing to interpret psychometric results. Methodology 9(3): 113–122.

63.

Pareso

(2021) Twitter’s own research shows it’s a megaphone for the right—but it’s complicated. Adweek, 22 October. Available at: https://www.adweek.com/morning-media-newsfeed/twitters-own-research-shows-its-a-megaphone-for-the-right-but-its-complicated/

64.

Rape, Abuse, & Incest National Network (RAINN) (2025) Get the facts about grooming. RAINN, 1 October. Available at: https://rainn.org/get-the-facts-about-sexual-violence/get-the-facts-about-grooming/

65.

Ray

Gibbons

(2021) Why are states banning critical race theory? Brookings, November. Available at: https://www.brookings.edu/articles/why-are-states-banning-critical-race-theory/

66.

Redfield

Conron

Tentindo

, et al. (2023) Prohibiting Gender-Affirming Medical Care for Youth. Los Angeles, CA: The Williams Institute, UCLA. Available at: https://williamsinstitute.law.ucla.edu/publications/bans-trans-youth-health-care/

67.

Roberts

(2022) Dealing with diversity in psychology: science and ideology. PsyArXiv [preprint]. DOI: 10.31234/osf.io/xk4yu.

68.

Romano

(2022) The right’s moral panic over “grooming” invokes age-old homophobia. Vox, 21 April. Available at: https://www.vox.com/culture/23025505/leftist-groomers-homophobia-satanic-panic-explained

69.

Romero

(2022) The Great Reset is not a conspiracy to force changes in economic systems. Politifact, 26 February. Available at: https://www.politifact.com/factchecks/2022/oct/11/liz-wheeler/great-reset-not-conspiracy-force-changes-economic-/

70.

Roth

(2021) Twitter reportedly suspended accounts by mistake after extremists abused new private media policy. The Verge, 4 December. Available at: https://www.theverge.com/2021/12/4/22817386/twitter-suspended-accounts-extremists

71.

Ruisch

Anderson

Inbar

, et al. (2020) A matter of taste: gustatory sensitivity predicts political ideology. Journal of Personality and Social Psychology: Personality Processes and Individual Differences 121(2): 394–409.

72.

Schmidt

(2021) Where America first went wrong on confederate monuments. Slate, 6 April. Available at: https://slate.com/news-and-politics/2021/04/charlottesville-confederate-monuments-america-civil-war-germany-world-war-ii.html

73.

Suyemoto

Erisman

Holowka

, et al. (2016) UMass Boston comprehensive demographic questionnaire, revised. As cited in Wadsworth LP, Morgan LP, Hayes-Skelton SA, Roemer L, & Suyemoto KL. Ways to boost your research rigor through increasing your cultural competence. The Behavior Therapist 39: 83–89.

74.

Swart

(2021) Experiencing algorithms: how young people understand, feel about, and engage with algorithmic news selection on social media. Social Media + Society 7(2): 1–11.

75.

Tech Transparency Project (2022) Facebook profits from white supremacist groups, 10 August. Available at: https://www.techtransparencyproject.org/articles/facebook-profits-white-supremacist-groups

76.

Timberg

Dwoskin

Albergotti

(2021) Inside Facebook, Jan. 6 violence fueled anger, regret over missed warning signs. The Washington Post, 21 October. Available at: https://www.washingtonpost.com/technology/2021/10/22/jan-6-capitol-riot-facebook/

77.

Tobin

Varner

Angwin

(2017) Facebook’s uneven enforcement of hate speech rules allows vile posts to stay up. Propublica, 28 December. Available at: https://www.propublica.org/article/facebook-enforcement-hate-speech-rules-mistakes

78.

Tourangeau

(2003) Cognitive aspects of survey measurement and mismeasurement. International Journal of Public Opinion Research 15: 3–7.

79.

Varghese

(2018) Twitter has purged left-wing accounts with no explanation. Wired, 10 October. Available at: https://www.wired.co.uk/article/twitter-political-account-ban-us-mid-term-elections

80.

Warzel

(2023) Twitter is a far-right social network. The Atlantic, 23 May. Available at: https://www.theatlantic.com/technology/archive/2023/05/elon-musk-ron-desantis-2024-twitter/674149/

81.

Willis

(2015) Analysis of the Cognitive Interview in Questionnaire Design. Oxford: Oxford University Press.

82.

Wutich

Beresford

Russell Bernard

(2025) Sample sizes for 10 types of qualitative data analysis: an integrative review, empirical guidance, and next steps. International Journal of Qualitative Methods 23: 16094069241296206.

83.

Yang

(2023) Twitter quietly changes its hateful conduct policy to remove standing protections for its transgender users. NBC News, 18 April. Available at: https://www.nbcnews.com/tech/twitter-changes-hateful-conduct-policy-rcna80338

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB