The moral foundation of Purity plays a prominent role in moral psychology. Previous studies have identified measurement issues with the standard operationalization of Purity (the Moral Foundations Questionnaire; MFQ) when comparing religious and nonreligious samples. The cause(s) and implications of such issues remain understudied. As the methodological backbone of much moral psychology research, the MFQ’s Purity subscale warrants further attention. In four samples (total N = 8,005), we identify measurement problems traceable to a single problematic item serving as a proxy for religious identification. Conservative estimates of the bias introduced by this item suggest that an average nonreligious participant’s Purity score are underestimated by at least 0.2 standard deviations, compared with an average Christian. Given Purity’s centrality to moral psychology, this bias may have influenced many findings. We offer recommendations for new and existing studies to avoid this bias at little cost. Data and analysis code are available at https://osf.io/ek98n/.
AtariM.HaidtJ.GrahamJ.KolevaS. P.StevensS. T.DehghaniM. (2022). Morality beyond the WEIRD: How the nomological network of morality varies across cultures. PsyArXiv Preprint. https://doi.org/10.31234/osf.io/q6c9r
ChenF. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. https://doi.org/10.1080/10705510701301834
4.
CliffordS.IyengarV.CabezaR.Sinnott-ArmstrongW. (2015). Moral foundations vignettes: A standardized stimulus database of scenarios based on moral foundations theory. Behavior Research Methods, 47(4), 1178–1198. https://doi.org/10.3758/s13428-014-0551-2
5.
CroneD. L. (2022). Conceptual issues with the moral foundation of Purity: The case of religion. PsyArXiv Preprint. https://doi.org/10.31234/osf.io/3e8bv
6.
CroneD. L.BodeS.MurawskiC.LahamS. M. (2018). The socio-moral image database (SMID): A novel stimulus set for the study of social, moral and affective processes. PLOS ONE, 13(1), Article e0190954. https://doi.org/10.1371/journal.pone.0190954
7.
CroneD. L.LahamS. M. (2015). Multiple moral foundations predict responses to sacrificial dilemmas. Personality and Individual Differences, 85(4), 60–65. https://doi.org/10.1016/j.paid.2015.04.041
8.
CroneD. L.LahamS. M. (2022). Retracing the threads: The importance of the Purity foundation for predicting culture war attitudes is measurement-contingent. PsyArXiv Preprint. https://doi.org/10.31234/osf.io/578us
9.
CroneD. L.RheeJ. J.LahamS. M. (2021). Developing brief versions of the Moral Foundations Vignettes using a genetic algorithm-based approach. Behavior Research Methods, 53, 1179–1187. https://doi.org/10.3758/s13428-020-01489-y
10.
DavisD. E.DooleyM. T.HookJ. N.ChoeE.McElroyS. E. (2017). The purity/sanctity subscale of the Moral Foundations Questionnaire does not work similarly for religious versus non-religious individuals. Psychology of Religion and Spirituality, 9(1), 124–130. https://doi.org/10.1037/rel0000057
11.
DavisD. E.RiceK.Van TongerenD. R.HookJ. N.DeBlaereC.WorthingtonE. L.ChoeE. (2016). The moral foundations hypothesis does not replicate well in black samples. Journal of Personality and Social Psychology, 110(4), e23–e30. https://doi.org/10.1037/pspp0000056
12.
FranchinL.GeipelJ.HadjichristidisC.SurianL. (2019). Many moral buttons or just one? Evidence from emotional facial expressions. Cognition & Emotion, 33(5), 943–958. https://doi.org/10.1080/02699931.2018.1520078
13.
FrimerJ. A.TellC. E.HaidtJ. (2015). Liberals condemn sacrilege too: The harmless desecration of Cerro Torre. Social Psychological and Personality Science, 6(8), 878–886. https://doi.org/10.1177/1948550615597974
14.
GrahamJ.HaidtJ.KolevaS. P.MotylM.IyerR.WojcikS. P.DittoP. H. (2013). Moral foundations theory: The pragmatic validity of moral pluralism. In DevineP.PlantA. (Eds.), Advances in experimental social psychology (Vol. 47, pp. 55–130). Academic Press. https://doi.org/10.1016/B978-0-12-407236-7.00002-4
15.
GrahamJ.HaidtJ.MotylM.MeindlP.IskiwitchC.MooijmanM. (2018). Moral foundations theory: On the advantages of moral pluralism over moral monism. In GrayK.GrahamJ. (Eds.), The Atlas of moral psychology: Mapping good and evil in the mind (pp. 211–222). Guilford Press.
16.
GrahamJ.HaidtJ.NosekB. A. (2009a). Liberals and conservatives rely on different sets of moral foundations. Harvard Dataverse. https://doi.org/10.7910/DVN/SJTRBI
17.
GrahamJ.HaidtJ.NosekB. A. (2009b). Liberals and conservatives rely on different sets of moral foundations. Journal of Personality and Social Psychology, 96(5), 1029–1046. https://doi.org/10.1037/a0015141
18.
GrahamJ.NosekB. A.HaidtJ.IyerR.KolevaS. P.DittoP. H. (2011). Mapping the moral domain. Journal of Personality and Social Psychology, 101(2), 366–385. https://doi.org/10.1037/a0021847
HaidtJ.JosephC. M. (2004). Intuitive ethics: How innately prepared intuitions generate culturally variable virtues. Daedalus, 133(4), 55–66. https://doi.org/10.1162/0011526042365555
22.
HaidtJ.JosephC. M. (2011). How moral foundations theory succeeded in building on sand: A response to Suhler and Churchland. Journal of Cognitive Neuroscience, 23(9), 2117–2122. https://doi.org/10.1162/jocn.2011.21638
23.
HuL.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
24.
KolevaS. P.GrahamJ.IyerR.DittoP. H.HaidtJ. (2012). Tracing the threads: How five moral concerns (especially Purity) help explain culture war attitudes. Journal of Research in Personality, 46(2), 184–194. https://doi.org/10.1016/j.jrp.2012.01.006
25.
KolevaS. P.HaidtJ. (2012). Let’s use Einstein’s safety razor, not Occam’s Swiss army knife or Occam’s chainsaw. Psychological Inquiry, 23(2), 175–178. https://doi.org/10.1080/1047840X.2012.667678
26.
KollarethD.RussellJ. A. (2019). Disgust and the sacred: Do people react to violations of the sacred with the same emotion they react to something putrid?Emotion, 19(1), 37–52. https://doi.org/10.1037/emo0000412
27.
RosseelY. (2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.
28.
RoyzmanE. B.AtanasovP.LandyJ. F.ParksA.GeptyA. (2014). CAD or MAD? Anger (not disgust) as the predominant response to pathogen-free violations of the divinity code. Emotion, 14(5), 892–907. https://doi.org/10.1037/a0036829
29.
SchallerM.ParkJ. H. (2011). The behavioral immune system (and why it matters). Current Directions in Psychological Science, 20(2), 99–103. https://doi.org/10.1177/0963721411402596
30.
ScheinC.GrayK. (2018). The theory of dyadic morality: Reinventing moral judgment by redefining harm. Personality and Social Psychology Review, 22(1), 32–70. https://doi.org/10.1177/1088868317698288
31.
SchmittN.AliA. A. (2015). The practical importance of measurement invariance. In LanceC. E.VandenbergR. J. (Eds.), More statistical and methodological myths and urban legends (pp. 327–346). Routledge.
32.
SchmittN.KuljaninG. (2008). Measurement invariance: Review of practice and implications. Human Resource Management Review, 18(4), 210–222. https://doi.org/10.1016/j.hrmr.2008.03.003
33.
SuhlerC. L.ChurchlandP. S. (2011). Can innate, modular “foundations” explain morality? Challenges for Haidt’s moral foundations theory. Journal of Cognitive Neuroscience, 23(9), 2103–2116. https://doi.org/10.1162/jocn.2011.21637
34.
TamulD.ElsonM.IvoryJ.HotterJ.LanierM.WolfJ.Martínez-CarrilloN. (2020). Moral foundations’ methodological foundations: A systematic analysis of reliability in research using the moral foundations questionnaire. PsyArXiv Preprint. https://doi.org/10.31234/osf.io/shcgv
35.
TilburtJ. C.JamesK. M.JenkinsS. M.AntielR. M.CurlinF. A.RasinskiK. A. (2013). “Righteous minds” in health care: Measurement and explanatory value of social intuitionism in accounting for the moral judgments in a sample of U.S. physicians. PLOS ONE, 8(9), Article e73379. https://doi.org/10.1371/journal.pone.0073379
36.
VandenbergR. J.LanceC. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. https://doi.org/10.1177/109442810031002
37.
van de SchootR.LugtigP.HoxJ. (2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486–492. https://doi.org/10.1080/17405629.2012.686740
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
