A variety of approaches have been presented for assessing desirable responding in self-report measures. Among them, the overclaiming technique asks respondents to rate their familiarity with a large set of real and nonexistent items (foils). The application of signal detection formulas to the endorsement rates of real items and foils yields indices of (a) knowledge accuracy and (b) knowledge bias. This overclaiming technique reflects both cognitive ability and personality. Here, we develop an alternative measurement model based on multidimensional item response theory (MIRT). We report three studies demonstrating this new model’s capacity to analyze overclaiming data. First, a simulation study illustrates that MIRT and signal detection theory yield comparable indices of accuracy and bias—although MIRT provides important additional information. Two empirical examples—one based on mathematical terms and one based on Chinese idioms—are then elaborated. Together, they demonstrate the utility of this new approach for group comparisons and item selection. The implications of this research are illustrated and discussed.
AckermanP. L.EllingsenV. J. (2014). Vocabulary overclaiming — a complete approach: Ability, personality, self-concept correlates, and gender differences. Intelligence, 46(1), 216–227. https://doi.org/10.1016/j.intell.2014.07.003
2.
AdamsR. J.WilsonM.WangW.-C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21(1), 1–23. http://doi.org/10.1177/0146621697211001
3.
AtirS.RosenzweigE.DunningD. (2015). When knowledge knows no bounds: Self-perceived expertise predicts claims of impossible knowledge. Psychological Science, 26(8), 1295–1303. https://doi.org/10.1177/0956797615588195
4.
BingM. N.KluemperD.DavisonH. K.TaylorS.NovicevicM. (2011). Overclaiming as a measure of faking. Organizational Behavior and Human Decision Processes, 116(1), 148–162. http://doi.org/10.1016/j.obhdp.2011.05.006
5.
BoltD. M.JohnsonT. R. (2009). Addressing score bias and differential item functioning due to individual differences in response style. Applied Psychological Measurement, 33(5), 335–352. https://doi.org/10.1177/0146621608329891
6.
ChenH.-F.JinK.-Y. (2022). The impact of item feature and response preference in a mixed-format design. Multivariate Behavioral Research, 57(2–3), 208–222. https://doi.org/10.1080/00273171.2020.1820308
7.
ClarianaM.CastellóA.CladellasR. (2016). Feeling of knowing and over-claiming in students from secondary school to university. Learning and Individual Differences, 49, 421–427. https://doi.org/10.1016/j.lindif.2016.05.008
8.
CronbachL. J. (1949). Essentials of psychological testing. Harper.
9.
DeCarloL. T. (2020). An item response model for true–false exams based on signal detection theory. Applied Psychological Measurement, 44(3), 234–248. https://doi.org/10.1177/0146621619843823
10.
DeCarloL. T. (2021). A signal detection model for multiple-choice exams. Applied Psychological Measurement, 45(6), 423–440. https://doi.org/10.1177/01466216211014599
11.
DouglasJ.CohenA. (2001). Nonparametric item response function estimation for assessing parametric model fit. Applied Psychological Measurement, 25(3), 234–243. https://doi.org/10.1177/01466210122032046
12.
DuboisP.PaulhusD. L. (2020). The VOCE: A measure of vocabulary over-claiming in English[manuscript under review]. University of British Columbia.
13.
DunlopP. D.BourdageJ. S.de VriesR. E.HilbigB. E.ZettlerI.LudekeS. G. (2017). Openness to (reporting) experiences that one never had: Overclaiming as an outcome of the knowledge accumulated through a proclivity for cognitive and aesthetic exploration. Journal of Personality and Social Psychology, 113(5), 810–834. https://doi.org/10.1037/pspp0000110
14.
DunlopP. D.BourdageJ. S.de VriesR. E.McNeillI. M.JorritsmaK.OrchardM.AustenT.BainesT.ChoeW.-K. (2020). Liar! Liar! (when stakes are higher): Understanding how the overclaiming technique can be used to measure faking in personnel selection. Journal of Applied Psychology, 105(8), 784–799. http://doi.org/10.1037/apl0000463
15.
FellC. B.KönigC. J.JungS.SorgD.ZieglerM. (2019). Are country level prevalences of rule violations associated with knowledge overclaiming among students?International Journal of Psychology, 54(1), 17–22. https://doi.org/10.1002/ijop.12441
16.
FoxJ. (1997). Applied regression analysis, linear models, and related methods. Sage.
17.
FurrR. M.FunderD. C. (2007). Behavioral observation. In RobinsR. W.FraleyR. C.KreugerR. F. (Eds.), Handbook of research methods in personality psychology (pp. 273–291). Guilford Press.
HoldenR. R.KronerD. G. (1992). The relative efficacy of differential response latencies for detecting faking on a self-report measure of psychopathology. Psychological Assessment, 4(2), 170–173. http://doi.org/10.1037/1040-3590.4.2.170
20.
HoldenR. R.KronerD. G.FekkenG. C.PophamS. M. (1992). A model of personality test item response dissimulation. Journal of Personality and Social Psychology, 63(2), 272–279. http://doi.org/10.1037/0022-3514.63.2.272
21.
HolmanR.GlasC. A. W. (2005). Modelling non‐ignorable missing‐data mechanisms with item response theory models. The British Journal of Mathematical and Statistical Psychology, 58(1), 1–17. http://doi.org/10.1111/j.2044-8317.2005.tb00312.x
22.
HülürG.WilhelmO.SchipolowskiS. (2011). Prediction of self-reported knowledge with over-claiming, fluid and crystallized intelligence and typical intellectual engagement. Learning and Individual Differences, 21(6), 742–746. https://doi.org/10.1016/j.lindif.2011.09.006
23.
JeonM.De BoeckP. (2016). A generalized item response tree model for psychological assessments. Behavior Research Methods, 48(3), 1070–1085. https://doi.org/10.3758/s13428-015-0631-y
24.
JinK.-Y.ChenH.-F.WangW.-C. (2018). Mixture item response models for inattentive responding behavior. Organizational Research Methods, 21(1), 197–225. https://doi.org/10.1177/1094428117725792
25.
JinK.-Y.WangW.-C. (2014). Generalized IRT models for extreme response style. Educational and Psychological Measurement, 74(1), 116–138. http://doi.org/10.1177/0013164413498876
26.
JonasK. G.MarkonK. E. (2019). Modeling response style using vignettes and person-specific item response theory. Applied Psychological Measurement, 43(1), 3–17. https://doi.org/10.1177/0146621618798663
27.
JonesD. N.NeriaA. L.HelmF. A.SahlanR. N.CarréJ. R. (2020). Religious overclaiming and support for religious aggression. Social Psychological and Personality Science, 11(7), 1011–1021. https://doi.org/10.1177/1948550620912880
28.
KamC.RisavyS. D.PerunovicW. E. (2015). Using over-claiming technique to probe social desirability ratings of personality items: A validity examination. Personality and Individual Differences, 74, 177–181. https://doi.org/10.1016/j.paid.2014.10.017
LiuL.CheungH. T. (2018). The partial productivity of schematic idioms in Chinese: Evidence from speakers’ judgment. Leukemia & Lymphoma, 19(1), 80–116. http://doi.org/10.1075/lali.00004
31.
LudekeS. G.MakranskyG. (2016). Does the over-claiming questionnaire measure overclaiming? Absent convergent validity in a large community sample. Psychological Assessment, 28(6), 765–774. http://doi.org/10.1037/pas0000211
32.
MacMillanN. A.CreelmanC. D. (2005). Detection theory: A user’s guide (2nd ed.). Erlbaum.
33.
MüllerS.MoshagenM. (2018). Overclaiming shares processes with the hindsight bias. Personality and Individual Differences, 134, 298–300. https://doi.org/10.1016/j.paid.2018.06.035
34.
MüllerS.MoshagenM. (2019). True virtue, self-presentation, or both?: A behavioral test of impression management and overclaiming. Psychological Assessment, 31(2), 181–191. https://doi.org/10.1037/pas0000657
OrlandoM.ThissenD. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24(1), 50–64. https://doi.org/10.1177/01466216000241003
39.
OrlandoM.ThissenD. (2003). Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27(4), 289–298. https://doi.org/10.1177/0146621603027004004
40.
PaulhusD. L. (1991). Measurement and control of response bias. In RobinsonJ. P.ShaverP. R.WrightsmanL. S. (Eds.), Measures of personality and social psychological attitudes (pp. 17–59). Academic Press.
41.
PaulhusD. L. (2011). Overclaiming on personality questionnaires. In ZieglerM.MacCannC.RobertsR. D. (Eds.), New perspectives on faking in personality assessment (pp. 151–164). Oxford University Press.
42.
PaulhusD. L. (2017). Socially desirable responding on self-reports. In Zeigler-HillV.ShackelfordT. (Eds.), Encyclopedia of personality and individual differences. Springer. http://doi.org/10.1007/978-3-319-28099-8_1349-1
43.
PaulhusD. L.DuboisP. J. (2014). Application of the overclaiming technique to scholastic assessment. Educational and Psychological Measurement, 74(6), 975–990. https://doi.org/10.1177/0013164414536184
PaulhusD. L.HarmsP. D.BruceM. N.LysyD. C. (2003). The over-claiming technique: Measuring self-enhancement independent of ability. Journal of Personality and Social Psychology, 84(4), 890–904. http://doi.org/10.1037/0022-3514.84.4.890
46.
PaulhusD. L.HoldenR. R. (2010). Measuring self-enhancement: From self-report to concrete behavior. In AgnewC. R.CarlstonD. E.GrazianoW. G.KellyJ. R. (Eds.), Then a miracle occurs: Focusing on behavior in social psychological theory and research (pp. 227–246). Oxford University Press.
47.
PaulhusD. L.VazireS. (2007). The self-report method. In RobinsR. W.FraleyR. C.KruegerR. (Eds.), Handbook of research methods in personality psychology (pp. 224–239). Guilford Press.
48.
PennycookG.RandD. G. (2018). Who falls for fake news? The roles of bullshit receptivity, overclaiming, familiarity, and analytic thinking. Journal of Personality, 88(2), 185–200. https://doi.org/10.1111/jopy.12476
49.
ReiseS. P.HensonJ. M. (2003). A discussion of modern versus traditional psychometrics as applied to personality assessment scales. Journal of Personality Assessment, 81(2), 93–103. https://doi.org/10.1207/S15327752JPA8102_01
50.
SedikidesC.AlickeM. D. (2019). The five pillars of self-enhancement and self-protection. In RyanR. M. (Ed.), Oxford handbook of human motivation (pp.307–319). Guilford Press.
51.
SorkinR. D. (1999). Spreadsheet signal detection. Behavior Research Methods, Instruments, and Computers, 31(1), 46–54. https://doi.org/10.3758/BF03207691
52.
SwamiV.PapanicolaouA.FurnhamA. (2011). Examining mental health literacy and its correlates using the overclaiming technique. British Journal of Psychology, 102(3), 662–675. https://doi.org/10.1111/j.2044-8295.2011.02036.x
53.
SwetsJ. A. (1964). Signal detection and recognition by human observers. Wiley.
54.
VonkovaH.PapajoanuO.StipekJ. (2018). Enhancing the cross-cultural comparability of self-reports using the overclaiming technique: An analysis of accuracy and exaggeration in 64 cultures. Journal of Cross-Cultural Psychology, 49(8), 1247–1268. https://doi.org/10.1177/002202211878704
55.
WangW.-C. (2008). Assessment of differential item functioning. Journal of Applied Measurement, 9(4), 387–408.
56.
WangW.-C.ChenH.-F.JinK.-Y. (2015). Item response theory models for wording effects in mixed-format scales. Educational and Psychological Measurement, 75(1), 157–178. https://doi.org/10.1177/0013164414528209
57.
WangW.-C.WilsonM.ShihC.-L. (2006). Modeling randomness in judging rating scales with a random‐effects rating scale model. Journal of Educational Measurement, 43(4), 335–353. http://doi.org/10.1111/j.1745-3984.2006.00020.x
58.
WilliamsK. M.ZumboB. D. (2003). Item characteristic curve estimation of signal detection theory-based personality data: A two-stage approach to item response modeling. International Journal of Testing, 3(2), 189–213. http://doi.org/10.1207/S15327574IJT0302_7
59.
ZieglerM.KemperC.RammstedtB. (2013). The vocabulary and overclaiming test (VOC-T). Journal of Individual Differences, 34(1), 32–40. https://doi.org/10.1027/1614-0001/a000093
60.
ZieglerM.MacCannC.RobertsR. (Eds.). (2011). New perspectives on faking in personality assessment. Oxford University Press.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
