Clarifying Measurement Issues With the Purity Subscale of the Moral Foundations Questionnaire in Christian and Nonreligious Participants

Abstract

The moral foundation of Purity plays a prominent role in moral psychology. Previous studies have identified measurement issues with the standard operationalization of Purity (the Moral Foundations Questionnaire; MFQ) when comparing religious and nonreligious samples. The cause(s) and implications of such issues remain understudied. As the methodological backbone of much moral psychology research, the MFQ’s Purity subscale warrants further attention. In four samples (total N = 8,005), we identify measurement problems traceable to a single problematic item serving as a proxy for religious identification. Conservative estimates of the bias introduced by this item suggest that an average nonreligious participant’s Purity score are underestimated by at least 0.2 standard deviations, compared with an average Christian. Given Purity’s centrality to moral psychology, this bias may have influenced many findings. We offer recommendations for new and existing studies to avoid this bias at little cost. Data and analysis code are available at https://osf.io/ek98n/.

Keywords

morality religion measurement research methods

Get full access to this article

View all access options for this article.

References

Atari

Haidt

Graham

Koleva

S. P.

Stevens

S. T.

Dehghani

(2022). Morality beyond the WEIRD: How the nomological network of morality varies across cultures. PsyArXiv Preprint. https://doi.org/10.31234/osf.io/q6c9r

Chakroff

Young

L. L.

(2015). Harmful situations, impure people: An attribution asymmetry across moral domains. Cognition, 136, 30–37. https://doi.org/10.1016/j.cognition.2014.11.034

Chen

F. F.

(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. https://doi.org/10.1080/10705510701301834

Clifford

Iyengar

Cabeza

Sinnott-Armstrong

(2015). Moral foundations vignettes: A standardized stimulus database of scenarios based on moral foundations theory. Behavior Research Methods, 47(4), 1178–1198. https://doi.org/10.3758/s13428-014-0551-2

Crone

D. L.

(2022). Conceptual issues with the moral foundation of Purity: The case of religion. PsyArXiv Preprint. https://doi.org/10.31234/osf.io/3e8bv

Crone

D. L.

Bode

Murawski

Laham

S. M.

(2018). The socio-moral image database (SMID): A novel stimulus set for the study of social, moral and affective processes. PLOS ONE, 13(1), Article e0190954. https://doi.org/10.1371/journal.pone.0190954

Crone

D. L.

Laham

S. M.

(2015). Multiple moral foundations predict responses to sacrificial dilemmas. Personality and Individual Differences, 85(4), 60–65. https://doi.org/10.1016/j.paid.2015.04.041

Crone

D. L.

Laham

S. M.

(2022). Retracing the threads: The importance of the Purity foundation for predicting culture war attitudes is measurement-contingent. PsyArXiv Preprint. https://doi.org/10.31234/osf.io/578us

Crone

D. L.

Rhee

J. J.

Laham

S. M.

(2021). Developing brief versions of the Moral Foundations Vignettes using a genetic algorithm-based approach. Behavior Research Methods, 53, 1179–1187. https://doi.org/10.3758/s13428-020-01489-y

10.

Davis

D. E.

Dooley

M. T.

Hook

J. N.

Choe

McElroy

S. E.

(2017). The purity/sanctity subscale of the Moral Foundations Questionnaire does not work similarly for religious versus non-religious individuals. Psychology of Religion and Spirituality, 9(1), 124–130. https://doi.org/10.1037/rel0000057

11.

Davis

D. E.

Rice

Van Tongeren

D. R.

Hook

J. N.

DeBlaere

Worthington

E. L.

Choe

(2016). The moral foundations hypothesis does not replicate well in black samples. Journal of Personality and Social Psychology, 110(4), e23–e30. https://doi.org/10.1037/pspp0000056

12.

Franchin

Geipel

Hadjichristidis

Surian

(2019). Many moral buttons or just one? Evidence from emotional facial expressions. Cognition & Emotion, 33(5), 943–958. https://doi.org/10.1080/02699931.2018.1520078

13.

Frimer

J. A.

Tell

C. E.

Haidt

(2015). Liberals condemn sacrilege too: The harmless desecration of Cerro Torre. Social Psychological and Personality Science, 6(8), 878–886. https://doi.org/10.1177/1948550615597974

14.

Graham

Haidt

Koleva

S. P.

Motyl

Iyer

Wojcik

S. P.

Ditto

P. H.

(2013). Moral foundations theory: The pragmatic validity of moral pluralism. In Devine

Plant

(Eds.), Advances in experimental social psychology (Vol. 47, pp. 55–130). Academic Press. https://doi.org/10.1016/B978-0-12-407236-7.00002-4

15.

Graham

Haidt

Motyl

Meindl

Iskiwitch

Mooijman

(2018). Moral foundations theory: On the advantages of moral pluralism over moral monism. In Gray

Graham

(Eds.), The Atlas of moral psychology: Mapping good and evil in the mind (pp. 211–222). Guilford Press.

16.

Graham

Haidt

Nosek

B. A.

(2009a). Liberals and conservatives rely on different sets of moral foundations. Harvard Dataverse. https://doi.org/10.7910/DVN/SJTRBI

17.

Graham

Haidt

Nosek

B. A.

(2009b). Liberals and conservatives rely on different sets of moral foundations. Journal of Personality and Social Psychology, 96(5), 1029–1046. https://doi.org/10.1037/a0015141

18.

Graham

Nosek

B. A.

Haidt

Iyer

Koleva

S. P.

Ditto

P. H.

(2011). Mapping the moral domain. Journal of Personality and Social Psychology, 101(2), 366–385. https://doi.org/10.1037/a0021847

19.

Gray

Young

L. L.

Waytz

(2012). Mind perception is the essence of morality. Psychological Inquiry, 23(2), 101–124. https://doi.org/10.1080/1047840X.2012.651387

20.

Haidt

(2010). The purity foundation’s global influence. https://ymblog.jonathanhaidt.org/2010/05/the-purity-foundations-global-influence/

21.

Haidt

Joseph

C. M.

(2004). Intuitive ethics: How innately prepared intuitions generate culturally variable virtues. Daedalus, 133(4), 55–66. https://doi.org/10.1162/0011526042365555

22.

Haidt

Joseph

C. M.

(2011). How moral foundations theory succeeded in building on sand: A response to Suhler and Churchland. Journal of Cognitive Neuroscience, 23(9), 2117–2122. https://doi.org/10.1162/jocn.2011.21638

23.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

24.

Koleva

S. P.

Graham

Iyer

Ditto

P. H.

Haidt

(2012). Tracing the threads: How five moral concerns (especially Purity) help explain culture war attitudes. Journal of Research in Personality, 46(2), 184–194. https://doi.org/10.1016/j.jrp.2012.01.006

25.

Koleva

S. P.

Haidt

(2012). Let’s use Einstein’s safety razor, not Occam’s Swiss army knife or Occam’s chainsaw. Psychological Inquiry, 23(2), 175–178. https://doi.org/10.1080/1047840X.2012.667678

26.

Kollareth

Russell

J. A.

(2019). Disgust and the sacred: Do people react to violations of the sacred with the same emotion they react to something putrid? Emotion, 19(1), 37–52. https://doi.org/10.1037/emo0000412

27.

Rosseel

(2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.

28.

Royzman

E. B.

Atanasov

Landy

J. F.

Parks

Gepty

(2014). CAD or MAD? Anger (not disgust) as the predominant response to pathogen-free violations of the divinity code. Emotion, 14(5), 892–907. https://doi.org/10.1037/a0036829

29.

Schaller

Park

J. H.

(2011). The behavioral immune system (and why it matters). Current Directions in Psychological Science, 20(2), 99–103. https://doi.org/10.1177/0963721411402596

30.

Schein

Gray

(2018). The theory of dyadic morality: Reinventing moral judgment by redefining harm. Personality and Social Psychology Review, 22(1), 32–70. https://doi.org/10.1177/1088868317698288

31.

Schmitt

Ali

A. A.

(2015). The practical importance of measurement invariance. In Lance

C. E.

Vandenberg

R. J.

(Eds.), More statistical and methodological myths and urban legends (pp. 327–346). Routledge.

32.

Schmitt

Kuljanin

(2008). Measurement invariance: Review of practice and implications. Human Resource Management Review, 18(4), 210–222. https://doi.org/10.1016/j.hrmr.2008.03.003

33.

Suhler

C. L.

Churchland

P. S.

(2011). Can innate, modular “foundations” explain morality? Challenges for Haidt’s moral foundations theory. Journal of Cognitive Neuroscience, 23(9), 2103–2116. https://doi.org/10.1162/jocn.2011.21637

34.

Tamul

Elson

Ivory

Hotter

Lanier

Wolf

Martínez-Carrillo

(2020). Moral foundations’ methodological foundations: A systematic analysis of reliability in research using the moral foundations questionnaire. PsyArXiv Preprint. https://doi.org/10.31234/osf.io/shcgv

35.

Tilburt

J. C.

James

K. M.

Jenkins

S. M.

Antiel

R. M.

Curlin

F. A.

Rasinski

K. A.

(2013). “Righteous minds” in health care: Measurement and explanatory value of social intuitionism in accounting for the moral judgments in a sample of U.S. physicians. PLOS ONE, 8(9), Article e73379. https://doi.org/10.1371/journal.pone.0073379

36.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. https://doi.org/10.1177/109442810031002

37.

van de Schoot

Lugtig

Hox

(2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486–492. https://doi.org/10.1080/17405629.2012.686740

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.94 MB