Inconsistent or careless responding is a significant threat to the validity of self-reported personality data. Using archival samples of undergraduate and community participants, we developed an inconsistent responding scale using items that appear on both the 60- and 100-item versions of the HEXACO Personality Inventory-Revised—two widely used measures of the HEXACO model of personality trait structure. We identified pairs of correlated HEXACO items in Sample 1 and created a total inconsistent responding score by summing absolute differences between each item pair. The Brief Response Inconsistency Evaluation (BRIE) for the HEXACO effectively differentiated between genuine and randomly generated responses across samples. The BRIE also correlated as expected with other measures of careless responding and relevant personality traits (e.g., conscientiousness). Tentative cut scores for the BRIE that appear to provide a reasonable balance between sensitivity and specificity in Sample 1 were investigated. Future research should examine the BRIE with different populations and translations of the HEXACO inventories and further investigate the effectiveness of the recommended cut scores.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.
2.
AriasV. B.GarridoL. E.JenaroC.Martínez-MolinaA.AriasB. (2020). A little garbage in, lots of garbage out: Assessing the impact of careless responding in personality survey data. Behavior Research Methods, 52(6), 2489–2505.
3.
ArthurW.Jr.HagenE.GeorgeF.Jr. (2021). The lazy or dishonest respondent: Detection and prevention. Annual Review of Organizational Psychology and Organizational Behavior, 8, 105–137.
4.
AshtonM. C.LeeK. (2007). Empirical, theoretical, and practical advantages of the HEXACO model of personality structure. Personality and Social Psychology Review, 11, 150–166.
5.
AshtonM. C.LeeK. (2009). The HEXACO-60: A short measure of the major dimensions of personality. Journal of Personality Assessment, 91(4), 340–345. https://doi.org/10.1080/00223890902935878
6.
AshtonM. C.LeeK. (2020). Objections to the HEXACO model of personality structure—And why those objections fail. European Journal of Personality, 34(4), 492–510.
7.
AshtonM. C.LeeK.de VriesR. E. (2014). The HEXACO honesty-humility, agreeableness, and emotionality factors: A review of research and theory. Personality and Social Psychology Review, 18, 139–152.
8.
AshtonM. C.LeeK.PeruginiM.SzarotaP.de VriesR. E.Di BlasL.BoiesK.De RaadB. (2004). A six-factor structure of personality-descriptive adjectives: Solutions from psycholexical studies in seven languages. Journal of Personality and Social Psychology, 86, 356–366. https://doi.org/10.1037/0022-3514.86.2.356
9.
AustF.DiedenhofenB.UllrichS.MuschJ. (2013). Seriousness checks are useful to improve data validity in online research. Behavior Research Methods, 45, 527–535. https://doi.org/10.3758/s13428-012-0265-2
10.
BarendsA. J.de VriesR. E. (2019). Noncompliant responding: Comparing exclusion criteria in MTurk personality research to improve data quality. Personality and Individual Differences, 143, 84–89. https://doi.org/10.1016/j.paid.2019.02.015
Ben-PorathY. S.TellegenA. (2020). Minnesota Multiphasic Personality Inventory-3 (MMPI-3): Manual for administration, scoring, and interpretation. University of Minnesota Press.
13.
BowlingN. A.HuangJ. L.BraggC. B.KhazonS.LiuM.BlackmoreC. E. (2016). Who cares and who is careless? Insufficient effort responding as a reflection of respondent personality. Journal of Personality and Social Psychology, 111(2), 218–229.
14.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.
15.
CurranG. (2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4–19. https://doi.org/10.1016/j.jesp.2015.07.006
16.
DonnellanM. B.RakhshaniA. (2022). How does the number of response options impact the psychometric properties of the Rosenberg Self-Esteem Scale?Assessment. Advance online publication. https://doi.org/10.1177/10731911221119532
17.
DrislaneL. E.PatrickC. J. (2017). Integrative alternative conceptions of psychopathic personality: A latent variable model of triarchic psychopathy constructs. Journal of Personality Disorders, 31, 110–132. http://dx.doi.org.ezproxy.shsu.edu/10.1521/pedi201630240
18.
GarciaL. F.AlujaA.RossierJ.OstendorfF.GlicksohnJ.OumarB.BellajT.RuchW.WangW.KoviZ.ScigalaD.CekrlijaD.StiversDiBlasL.ValdiviaM.Ben JemaaS.AtitsogbeK. A.HansenneM. (2021). Exploring the stability of HEXACO-60 structure and the association of gender, age, and social position with personality traits across 18 countries. Journal of Personality, 90(2), 256–276.
19.
GoldammerP.AnnenH.StöckliL.JonasK. (2020). Careless responding in questionnaire measures: Detection, impact, and remedies. The Leadership Quarterly, 31(4), 101384.
20.
HandelR. W.Ben-PorathY. S.TellegenA.ArcherR. P.(2010). Psychometric functioning of the MMPI-2-RF VRIN-r and TRIN-r scales with varying degrees of randomness, acquiescence, and counter-acquiescence. Psychological Assessment, 22(1), 87–95. https://doi.org/10.1037/a0017061
21.
KeeleyJ. W.WebbC.PetersonD.RoussinL.FlanaganE. H. (2016). Development of a response inconsistency scale for the Personality Inventory for DSM–5. Journal of Personality Assessment, 98, 351–359. https://doi.org/10.1080/00223891.2016.1158719
22.
KelleyS. E.EdensJ. F.DonnellanM. B.RuchenskyJ. R.WittE. A.McDermottB. E. (2016). Development and validation of an inconsistent responding scale for an abbreviated version of the Psychopathic Personality Inventory_Revised. Personality and Individual Differences, 91, 58–62. https://doi.org/10.1016/j.paid.2015.11.033
23.
KelleyS. E.van DongenJ. D. M.DonnellanM. B.EdensJ. F.EisenbarthH.FossatiA.HownerK.SommaA.SormanK. (2018). Examination of the triarchic procedure for inconsistent responding in six non-English language samples. Psychological Assessment, 30(5), 610–620. https://doi.org/10.1037/pas0000485
24.
KruegerR. F.DerringerJ.MarkonK. E.WatsonD.SkodolA. E. (2012). Initial construction of a maladaptive personality trait model and inventory for DSM-5. Psychological Medicine, 42, 1879–1890.
25.
LambrosA. M.MarcusD. K.NagelM. G. (2022). Construction and preliminary validation of an inconsistent responding scale for the HEXACO-100. Psychological Assessment. Advance online publication. https://doi.org/10.1037/pas0001203
LilienfeldS. O.WidowsM. R. (2005). Psychopathic Personality Inventory–Revised professional manual. Psychological Assessment Resources.
28.
MarjanovicZ.HoldenR.StruthersW.CribbieR.GreenglassE. (2015). The inter-item standard deviation (ISD): An index that discriminates between conscientious and random responders. Personality and Individual Differences, 84, 79–83.
29.
McGrathR. E.Hall-SimmondsA.GoldbergL. R. (2020). Are measures of character and personality distinct? Evidence from observed-score and true-score analyses. Assessment, 27(1), 117–135. https://doi.org/10.1177/1073191117738047
30.
McGrathR. E.MitchellM.KimB. H.HoughL. (2010). Evidence for response bias as a source of error variance in applied assessment. Psychological Bulletin, 136(3), 450–470.
31.
MeadeA. W.CraigS. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455. https://doi.org/10.1037/a0028085
32.
MillerJ. D.LamkinJ.Maples-KellerJ. L.LynamD. R. (2016). Viewing the triarchic model of psychopathy through general personality and expert-based lenses. Personality Disorders: Theory, Research, and Treatment, 7(3), 247–258. https://doi.org/10.1037/per0000155
33.
MoreyL. C. (2007). Personality Assessment Inventory: Professional manual (2nd ed.). Psychological Assessment Resources.
34.
MoshagenM.ThielmannI.HilbigB. E.ZettlerI. (2019). Meta-analytic investigations of the HEXACO Personality Inventory (Revised). Zeitschrift für Psychologie, 227, 186–194.
35.
MowleE. N.KelleyS. E.EdensJ. F.SmithS. T.WygantD. B.SelbornM. (2017). Development of an inconsistent responding scale for the Triarchic Psychopathy Measure. Psychological Assessment, 29(8), 990–1000. https://doi.org/10.1037/pas0000395
36.
NicholsA. L.EdlundJ. E. (2020). Why don’t we care more about carelessness? Understanding the causes and consequences of careless participants. International Journal of Social Research Methodology, 23(6), 625–638.
37.
NikolovaN.HendryM. C.DouglasK. S.EdensJ. F.LilienfeldS. O. (2012). The inconsistency of inconsistency scales: A comparison of two widely used measures. Behavioral Sciences and the Law, 30, 16–27.
38.
PatrickC. J. (2010). Operationalizing the triarchic conceptualization of psychopathy: Preliminary description of brief scales for assessment of boldness, meanness and disinhibition [Unpublished manual]. Department of Psychology, Florida State University.
39.
R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
40.
RothM.AltmannT. (2019). A multi-informant study of the influence of targets’ and perceivers’ social desirability on self-other agreement in ratings of the HEXACO personality dimensions. Journal of Research in Personality, 78, 138–147. https://doi-org.ezproxy.shsu.edu/10.1016/j.jrp.2018.11.008
41.
RuchenskyJ. R.DonnellanM. B.EdensJ. F. (2018). Development and initial validation of the HEXACO-triarchic scales. Psychological Assessment, 30, 1560–1566. https://doi.org/10.1037/pas0000609
42.
SchmittN.StultsD. M. (1985). Factors defined by negatively keyed items: The result of careless respondents?Applied Psychological Measurement, 9, 367–373.
43.
SilberH.RoßmannJ.GummerT. (2022). The issue of noncompliance in attention check questions: False positives in instructed response items. Field Methods, 34(4), 346–360.
44.
SleepC. E.WeissB.LynamD. R.MillerJ. D. (2019). An examination of the triarchic model of psychopathy’s nomological network: A meta-analytic review. Clinical Psychology Review, 71, 1–26. https://doi.org/10.1016/j.cpr.2019.04.005
45.
SotoC. J.JohnO. P.(2017). The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Personality and Social Psychology, 113, 117–143. https://doi.org/10.1037/pspp0000096
46.
ThielmannI.AkramiN.BabarovicT.BellochA.BerghR.ChirumboloA.ColovicP.de VriesR. E.DostalD.EgorovaM.GnisciA.HeydaschT.HilbigB. E.HsuK.IzdebskiP.LeoneL.MarcusB.MededovicJ.NagyJ.. . .LeeK. (2020). The HEXACO-100 across 16 languages: A large-scale test of measurement invariance. Journal of Personality Assessment, 102(5), 714–726.
47.
van DongenJ. D. M.DrislaneL. E.NijmanH.Soe-AgnieS. E.van MarleH. J. C. (2016). Further evidence for reliability and validity of the triarchic psychopathy measure in a forensic sample and a community sample. Journal of Psychopathology and Behavior Assessment, 39, 58–66. https://doi.org/10.1007/s10862-016-9567-5
48.
WardM. K.MeadeA. W. (2022). Dealing with careless responding in survey data: Prevention, identification, and recommended best practices. Annual Review of Psychology, 74, 577–596.
49.
WoodsC. M. (2006). Careless responding to reverse-worded items: Implications for confirmatory factor analysis. Journal of Psychopathology and Behavioral Assessment, 28(3), 186–191.
50.
YentesR. D.WilhelmF. (2021). Careless: Procedures for computing indices of careless responding (R package version 1.2.1). R Core Team.
51.
ZettlerI.ThielmannI.HilbigB. E.MoshagenM. (2020). The nomological net of the HEXACO model of personality: A large-scale meta-analytic investigation. Perspectives on Psychological Science, 15(3), 723–760.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
