Development of an Inconsistent Responding Scale for the HEXACO Personality Inventory-Revised

Abstract

Inconsistent or careless responding is a significant threat to the validity of self-reported personality data. Using archival samples of undergraduate and community participants, we developed an inconsistent responding scale using items that appear on both the 60- and 100-item versions of the HEXACO Personality Inventory-Revised—two widely used measures of the HEXACO model of personality trait structure. We identified pairs of correlated HEXACO items in Sample 1 and created a total inconsistent responding score by summing absolute differences between each item pair. The Brief Response Inconsistency Evaluation (BRIE) for the HEXACO effectively differentiated between genuine and randomly generated responses across samples. The BRIE also correlated as expected with other measures of careless responding and relevant personality traits (e.g., conscientiousness). Tentative cut scores for the BRIE that appear to provide a reasonable balance between sensitivity and specificity in Sample 1 were investigated. Future research should examine the BRIE with different populations and translations of the HEXACO inventories and further investigate the effectiveness of the recommended cut scores.

Keywords

inconsistent responding careless responding personality assessment HEXACO self-report

Get full access to this article

View all access options for this article.

References

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.

Arias

V. B.

Garrido

L. E.

Jenaro

Martínez-Molina

Arias

(2020). A little garbage in, lots of garbage out: Assessing the impact of careless responding in personality survey data. Behavior Research Methods, 52(6), 2489–2505.

Arthur

Jr. Hagen

George

Jr. (2021). The lazy or dishonest respondent: Detection and prevention. Annual Review of Organizational Psychology and Organizational Behavior, 8, 105–137.

Ashton

M. C.

Lee

(2007). Empirical, theoretical, and practical advantages of the HEXACO model of personality structure. Personality and Social Psychology Review, 11, 150–166.

Ashton

M. C.

Lee

(2009). The HEXACO-60: A short measure of the major dimensions of personality. Journal of Personality Assessment, 91(4), 340–345. https://doi.org/10.1080/00223890902935878

Ashton

M. C.

Lee

(2020). Objections to the HEXACO model of personality structure—And why those objections fail. European Journal of Personality, 34(4), 492–510.

Ashton

M. C.

Lee

de Vries

R. E.

(2014). The HEXACO honesty-humility, agreeableness, and emotionality factors: A review of research and theory. Personality and Social Psychology Review, 18, 139–152.

Ashton

M. C.

Lee

Perugini

Szarota

de Vries

R. E.

Di Blas

Boies

De Raad

(2004). A six-factor structure of personality-descriptive adjectives: Solutions from psycholexical studies in seven languages. Journal of Personality and Social Psychology, 86, 356–366. https://doi.org/10.1037/0022-3514.86.2.356

Aust

Diedenhofen

Ullrich

Musch

(2013). Seriousness checks are useful to improve data validity in online research. Behavior Research Methods, 45, 527–535. https://doi.org/10.3758/s13428-012-0265-2

10.

Barends

A. J.

de Vries

R. E.

(2019). Noncompliant responding: Comparing exclusion criteria in MTurk personality research to improve data quality. Personality and Individual Differences, 143, 84–89. https://doi.org/10.1016/j.paid.2019.02.015

11.

Beach

D. A.

(1989). Identifying the random responder. The Journal of Psychology: Interdisciplinary and Applied, 123(1), 101–103. https://doi.org/10.1080/00223980.1989.10542966

12.

Ben-Porath

Y. S.

Tellegen

(2020). Minnesota Multiphasic Personality Inventory-3 (MMPI-3): Manual for administration, scoring, and interpretation. University of Minnesota Press.

13.

Bowling

N. A.

Huang

J. L.

Bragg

C. B.

Khazon

Liu

Blackmore

C. E.

(2016). Who cares and who is careless? Insufficient effort responding as a reflection of respondent personality. Journal of Personality and Social Psychology, 111(2), 218–229.

14.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.

15.

Curran

(2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4–19. https://doi.org/10.1016/j.jesp.2015.07.006

16.

Donnellan

M. B.

Rakhshani

(2022). How does the number of response options impact the psychometric properties of the Rosenberg Self-Esteem Scale? Assessment. Advance online publication. https://doi.org/10.1177/10731911221119532

17.

Drislane

L. E.

Patrick

C. J.

(2017). Integrative alternative conceptions of psychopathic personality: A latent variable model of triarchic psychopathy constructs. Journal of Personality Disorders, 31, 110–132. http://dx.doi.org.ezproxy.shsu.edu/10.1521/pedi201630240

18.

Garcia

L. F.

Aluja

Rossier

Ostendorf

Glicksohn

Oumar

Bellaj

Ruch

Wang

Kovi

Scigala

Cekrlija

Stivers

Blas

Valdivia

Ben Jemaa

Atitsogbe

K. A.

Hansenne

(2021). Exploring the stability of HEXACO-60 structure and the association of gender, age, and social position with personality traits across 18 countries. Journal of Personality, 90(2), 256–276.

19.

Goldammer

Annen

Stöckli

Jonas

(2020). Careless responding in questionnaire measures: Detection, impact, and remedies. The Leadership Quarterly, 31(4), 101384.

20.

Handel

R. W.

Ben-Porath

Y. S.

Tellegen

Archer

R. P.

(2010). Psychometric functioning of the MMPI-2-RF VRIN-r and TRIN-r scales with varying degrees of randomness, acquiescence, and counter-acquiescence. Psychological Assessment, 22(1), 87–95. https://doi.org/10.1037/a0017061

21.

Keeley

J. W.

Webb

Peterson

Roussin

Flanagan

E. H.

(2016). Development of a response inconsistency scale for the Personality Inventory for DSM–5. Journal of Personality Assessment, 98, 351–359. https://doi.org/10.1080/00223891.2016.1158719

22.

Kelley

S. E.

Edens

J. F.

Donnellan

M. B.

Ruchensky

J. R.

Witt

E. A.

McDermott

B. E.

(2016). Development and validation of an inconsistent responding scale for an abbreviated version of the Psychopathic Personality Inventory_Revised. Personality and Individual Differences, 91, 58–62. https://doi.org/10.1016/j.paid.2015.11.033

23.

Kelley

S. E.

van Dongen

J. D. M.

Donnellan

M. B.

Edens

J. F.

Eisenbarth

Fossati

Howner

Somma

Sorman

(2018). Examination of the triarchic procedure for inconsistent responding in six non-English language samples. Psychological Assessment, 30(5), 610–620. https://doi.org/10.1037/pas0000485

24.

Krueger

R. F.

Derringer

Markon

K. E.

Watson

Skodol

A. E.

(2012). Initial construction of a maladaptive personality trait model and inventory for DSM-5. Psychological Medicine, 42, 1879–1890.

25.

Lambros

A. M.

Marcus

D. K.

Nagel

M. G.

(2022). Construction and preliminary validation of an inconsistent responding scale for the HEXACO-100. Psychological Assessment. Advance online publication. https://doi.org/10.1037/pas0001203

26.

Lee

Ashton

M. C.

(2018). Psychometric properties of the HEXACO-100. Assessment, 25(5), 543–556. https://doi.org/10.1177/1073191116659134

27.

Lilienfeld

S. O.

Widows

M. R.

(2005). Psychopathic Personality Inventory–Revised professional manual. Psychological Assessment Resources.

28.

Marjanovic

Holden

Struthers

Cribbie

Greenglass

(2015). The inter-item standard deviation (ISD): An index that discriminates between conscientious and random responders. Personality and Individual Differences, 84, 79–83.

29.

McGrath

R. E.

Hall-Simmonds

Goldberg

L. R.

(2020). Are measures of character and personality distinct? Evidence from observed-score and true-score analyses. Assessment, 27(1), 117–135. https://doi.org/10.1177/1073191117738047

30.

McGrath

R. E.

Mitchell

Kim

B. H.

Hough

(2010). Evidence for response bias as a source of error variance in applied assessment. Psychological Bulletin, 136(3), 450–470.

31.

Meade

A. W.

Craig

S. B.

(2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455. https://doi.org/10.1037/a0028085

32.

Miller

J. D.

Lamkin

Maples-Keller

J. L.

Lynam

D. R.

(2016). Viewing the triarchic model of psychopathy through general personality and expert-based lenses. Personality Disorders: Theory, Research, and Treatment, 7(3), 247–258. https://doi.org/10.1037/per0000155

33.

Morey

L. C.

(2007). Personality Assessment Inventory: Professional manual (2nd ed.). Psychological Assessment Resources.

34.

Moshagen

Thielmann

Hilbig

B. E.

Zettler

(2019). Meta-analytic investigations of the HEXACO Personality Inventory (Revised). Zeitschrift für Psychologie, 227, 186–194.

35.

Mowle

E. N.

Kelley

S. E.

Edens

J. F.

Smith

S. T.

Wygant

D. B.

Selborn

(2017). Development of an inconsistent responding scale for the Triarchic Psychopathy Measure. Psychological Assessment, 29(8), 990–1000. https://doi.org/10.1037/pas0000395

36.

Nichols

A. L.

Edlund

J. E.

(2020). Why don’t we care more about carelessness? Understanding the causes and consequences of careless participants. International Journal of Social Research Methodology, 23(6), 625–638.

37.

Nikolova

Hendry

M. C.

Douglas

K. S.

Edens

J. F.

Lilienfeld

S. O.

(2012). The inconsistency of inconsistency scales: A comparison of two widely used measures. Behavioral Sciences and the Law, 30, 16–27.

38.

Patrick

C. J.

(2010). Operationalizing the triarchic conceptualization of psychopathy: Preliminary description of brief scales for assessment of boldness, meanness and disinhibition [Unpublished manual]. Department of Psychology, Florida State University.

39.

R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

40.

Roth

Altmann

(2019). A multi-informant study of the influence of targets’ and perceivers’ social desirability on self-other agreement in ratings of the HEXACO personality dimensions. Journal of Research in Personality, 78, 138–147. https://doi-org.ezproxy.shsu.edu/10.1016/j.jrp.2018.11.008

41.

Ruchensky

J. R.

Donnellan

M. B.

Edens

J. F.

(2018). Development and initial validation of the HEXACO-triarchic scales. Psychological Assessment, 30, 1560–1566. https://doi.org/10.1037/pas0000609

42.

Schmitt

Stults

D. M.

(1985). Factors defined by negatively keyed items: The result of careless respondents? Applied Psychological Measurement, 9, 367–373.

43.

Silber

Roßmann

Gummer

(2022). The issue of noncompliance in attention check questions: False positives in instructed response items. Field Methods, 34(4), 346–360.

44.

Sleep

C. E.

Weiss

Lynam

D. R.

Miller

J. D.

(2019). An examination of the triarchic model of psychopathy’s nomological network: A meta-analytic review. Clinical Psychology Review, 71, 1–26. https://doi.org/10.1016/j.cpr.2019.04.005

45.

Soto

C. J.

John

O. P.

(2017). The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Personality and Social Psychology, 113, 117–143. https://doi.org/10.1037/pspp0000096

46.

Thielmann

Akrami

Babarovic

Belloch

Bergh

Chirumbolo

Colovic

de Vries

R. E.

Dostal

Egorova

Gnisci

Heydasch

Hilbig

B. E.

Hsu

Izdebski

Leone

Marcus

Mededovic

Nagy

. . .Lee

(2020). The HEXACO-100 across 16 languages: A large-scale test of measurement invariance. Journal of Personality Assessment, 102(5), 714–726.

47.

van Dongen

J. D. M.

Drislane

L. E.

Nijman

Soe-Agnie

S. E.

van Marle

H. J. C

. (2016). Further evidence for reliability and validity of the triarchic psychopathy measure in a forensic sample and a community sample. Journal of Psychopathology and Behavior Assessment, 39, 58–66. https://doi.org/10.1007/s10862-016-9567-5

48.

Ward

M. K.

Meade

A. W.

(2022). Dealing with careless responding in survey data: Prevention, identification, and recommended best practices. Annual Review of Psychology, 74, 577–596.

49.

Woods

C. M.

(2006). Careless responding to reverse-worded items: Implications for confirmatory factor analysis. Journal of Psychopathology and Behavioral Assessment, 28(3), 186–191.

50.

Yentes

R. D.

Wilhelm

(2021). Careless: Procedures for computing indices of careless responding (R package version 1.2.1). R Core Team.

51.

Zettler

Thielmann

Hilbig

B. E.

Moshagen

(2020). The nomological net of the HEXACO model of personality: A large-scale meta-analytic investigation. Perspectives on Psychological Science, 15(3), 723–760.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB