While rating-scale-based assessments have been shown to be useful for measuring a variety of workplace-relevant constructs, assessment length and response distortion present practical limitations on their use. We describe a new type of measurement method termed rapid response measurement (RRM) in which stimuli are presented on a computer screen one at a time in rapid succession and respondents are asked to quickly provide a dichotomous response. Two personality assessments using RRM were developed and reliability and validity evidence across four independent samples were evaluated. Both RRM assessments showed adequate reliability, even at short test lengths, with acceptable levels of convergent and discriminant validity with traditional survey-based measures. Analyses based on a within-participants design indicated that the RRM was significantly more difficult to fake when instructed than was a survey-based measure of personality. The second RRM was related to several aspects of job performance. While initial results show promise, further research is needed to establish the validity and viability of the RRM for organizational and psychological measurement.
AckermanP. L. (1986). Individual differences in information processing: An investigation of intellectual abilities and task performance during practice. Intelligence, 10(2), 101–139. doi:10.1016/0160-2896(86)90010-3
2.
AckermanP. L.KanferR. (2009). Test length and cognitive fatigue: An empirical examination of effects on performance and test-taker reactions. Journal of Experimental Psychology: Applied, 15, 163–181. doi:10.1037/a0015719
3.
AkramiN.HedlundL.EkehammarB. (2007). Personality scale response latencies as self-schema indicators: The inverted-U effect revisited. Personality and Individual Differences, 43, 611–618.
4.
AlligerG. M.DwightS. A. (2000). A meta-analytic investigation of the susceptibility of integrity tests to faking and coaching. Educational and Psychological Measurement, 60, 59–72. doi:10.1177/00131640021970367
5.
AshtonM. C.LeeK.GoldbergL. R. (2004). A hierarchical analysis of 1,710 English personality-descriptive adjectives. Journal of Personality and Social Psychology, 87, 707–721.
6.
BackM. D.SchmukleS. C.EgloffB. (2009). Predicting actual behavior from the explicit and implicit self-concept of personality. Journal Personality and Social Psychology, 97, 533–548. doi:10.1037/a0016229.
7.
BalzerW. K.KihmJ. A.SmithP. C.IrwinJ. L.BachiochiP. D.RobieC.…ParraL. F. (1997). Users’ manual for the Job Descriptive Index (JDI; 1997 Revision) and the Job in General (JIG) scales. Bowling Green, OH: Bowling Green State University.
8.
BarrickM. R.MountM. K. (1991). The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1–26. doi:10.1111/ j.1744-6570.1991.tb00688.x
9.
BarrickM. R.MountM. K.JudgeT. A. (2001). Personality and performance at the beginning of the new millennium: What do we know and where do we go next?International Journal of Selection and Assessment, 9, 9–30. doi:10.1111/1468-2389.00160
10.
BingM. N.LeBretonJ. M.DavisonH. K.MigetzD. Z.JamesL. R. (2007). Integrating implicit and explicit social cognitions for enhanced personality assessment: A general framework for choosing measurement and statistical methods. Organizational Research Methods, 10(1), 136–179. doi:10.1177/1094428106289396
11.
BinningJ. F.BarrettG. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74(3), 478–494. doi:10.1037/0021-9010.74.3.478
12.
BirkelandS. A.MansonT. M.KisamoreJ. L.BrannickM. T.SmithM. A. (2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14(4), 317–335. doi:10.1111/j.1468-2389.2006.00354.x
BlockJ. (1995). A contrarian view of the five-factor approach to personality description. Psychological Bulletin, 117(2), 187–215. doi:10.1037/0033-2909.117.2.187
15.
BrunettiD. G.SchlottmannR. S.ScottA. B.HollrahJ. L. (1998). Instructed faking and MMPI-2 response latencies: The potential for assessing response validity. Journal of Clinical Psychology, 54(2), 143–153.
16.
CallahanD. (2004). The cheating culture. Orlando, FL: Harcourt.
17.
CampbellD. T.FiskeD. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.
18.
Center for Creative Leadership. (2013). Leading Managers 360 Technical Manual. Greensboro, NC: Center for Creative Leadership.
19.
CostaP. T.JrMcCraeR. R. (1992). Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) manual. Odessa, FL: Psychological Assessment Resources.
20.
CoteJ. A.BuckleyM. R. (1987). Estimating trait, method, and error variance: Generalizing across 70 construct validation studies. Journal of Marketing Research, 24(3), 315–318. doi:10.2307/3151642
DigmanJ. M. (1990). Personality structure: Emergence of the five-factor model. Annual Review of Psychology, 41, 417–440. doi:10.1146/annurev.ps.41.020190.002221
23.
DigmanJ. M. (1997). Higher-order factors of the Big Five. Journal of Personality and Social Psychology, 73(6), 1246–1256. doi:10.1037/0022-3514.73.6.1246
24.
DigmanJ. M.Takemoto-ChockN. (1981). Factors in the natural language of personality: Re-analysis, comparison, and interpretation of six major studies. Multivariate Behavioral Research, 16(2), 149–170. doi:10.1207/s15327906mbr1602_2
25.
DotyD. H.GlickW. H. (1998). Common methods bias: Does common methods variance really bias results?Organizational Research Methods, 1(4), 374–406. doi:10.1177/109442819814002
26.
EllingsonJ. E.SackettP. R.HoughL. M. (1999). Social desirability corrections in personality measurement: Issues of applicant comparison and construct validity. Journal of Applied Psychology, 84(2), 155–166. doi:10.1037/0021-9010.84.2.155
27.
GoldbergL. R. (1990). An alternative “description of personality”: The big-five factor structure. Journal of Personality and Social Psychology, 59(6), 1216–1229. doi:10.1037/0022-3514.59.6.1216
28.
GoldbergL. R. (1992). The development of markers for the big-five factor structure. Psychological Assessment, 4(1), 26–42. doi:10.1037/1040-3590.4.1.26
29.
GoldbergL. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In MervieldeI.DearyI.De FruytF.OstendorfF. (Eds.), Personality psychology in Europe (Vol. 7, pp. 7–28). Tilburg, Netherlands: Tilburg University Press.
30.
GoldbergL. R.JohnsonJ. A.EberH. W.HoganR.AshtonM. C.CloningerC. R.GoughH. C. (2006). The International Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality, 40, 84–96.
31.
GreenwaldA. G.NosekB. A.BanajiM. R. (2003). Understanding and using the implicit association test: An improved scoring algorithm. Journal of Personality and Social Psychology, 85, 197–216.
32.
GreenwaldA. G.PoehlmanT. A.UhlmannE. L.BanajiM. R. (2009). Understanding and using the implicit association test: III. Meta-analysis of predictive validity. Journal of Personality and Social Psychology, 97(1), 17–41. doi:10.1037/a0015575
33.
GrummM.von CollaniG. (2007). Measuring big-five personality dimensions with the implicit association test—Implicit personality traits or self-esteem?Personality and Individual Differences, 43(8), 2205–2217. doi:10.1016/j.paid.2007.06.032
34.
GuilfordJ. P. (1954). Psychometric methods. New York, NY: McGraw-Hill.
35.
GulliksenH. (1950). Theory of mental tests. New York, NY: John Wiley.
36.
HoganJ.BarrettP.HoganR. (2007). Personality measurement, faking, and employment selection. Journal of Applied Psychology, 92(5), 1270–1285. doi:10.1037/0021-9010.92.5.1270
37.
HoldenR. R.FekkenG. G. (1993). Can personality test item response latencies have construct validity? Issues of reliability and convergent and discriminant validity. Personality and Individual Differences, 15, 243–248.
38.
HoldenR. R.FekkenG. C.CottonD. H. G. (1991). Assessing psychopathology using structured test-item response latencies. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 3, 111–118.
39.
HoldenR. R.HibbsN. (1995). Incremental validity of response latencies for detecting fakers on a personality test. Journal of Research in Personality, 29, 362–372. doi:10.1006/jrpe.1995.1021
40.
HoldenR. R.KronerD. G. (1992). Relative efficacy of differential response latencies for detecting faking on a self-report measure of psychopathology. Psychological Assessment, 4, 170–173. doi:10.1037/1040-3590.4.2.170
41.
HoldenR. R.WoodL. L.TomashewskiL. (2001). Do response time limitations counteract the effect of faking on personality inventory validity?Journal of Personality and Social Psychology, 81, 160–169. doi:10.1037/0022-3514.81.1.160
42.
HoltgravesT. (2004). Social desirability and self-reports: Testing models of socially desirable responding. Personality and Social Psychology Bulletin, 30(2), 161–172. doi:10.1177/0146167203259930
43.
HornJ. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179–185.
44.
HoughL. M.EatonN. K.DunnetteM. D.KampJ. D.McCloyR. A. (1990). Criterion-related validities of personality constructs and the effect of response distortion on those validities. Journal of Applied Psychology, 75(5), 581–595. doi:10.1037/0021-9010.75.5.581
45.
HowardP. J.HowardJ. M. (2009). Professional Manual for the Workplace Big Five Profile. Charlotte, NC: Center for Applied Cognitive Studies.
46.
HsuL. M.SantelliJ.HsuJ. R. (1989). Faking detection validity and incremental validity of response latencies to MMPI subtle and obvious items. Journal of Personality Assessment, 53, 278–295. doi:10.1207/s15327752jpa5302_6
47.
International Personality Item Pool. (n.d.). A scientific collaboratory for the development of advanced measures of personality traits and other individual differences. Retrieved fromhttp://ipip.ori.org
48.
JamesL. R. (1998). Measurement of personality via conditional reasoning. Organizational Research Methods, 1(2), 131–163. doi:10.1177/109442819812001
49.
JohnsonJ. A. (2005). Ascertaining the validity of individual protocols from web-based personality inventories. Journal of Research in Personality, 39(1), 103–129. doi:10.1016/j.jrp.2004.09.009
50.
JudgeT. A.MartocchioJ. J.ThoresenC. J. (1997). Five-factor model of personality and employee absence. Journal of Applied Psychology, 82, 745–755.
51.
KanferR.AckermanP. L. (1989). Motivation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition. Journal of Applied Psychology, 74(4), 657–690. doi:10.1037/0021-9010.74.4.657
52.
KarpinskiA.SteinmanR. B. (2006). The single category implicit association test as a measure of implicit social cognition. Journal of Personality and Social Psychology. 91(1), 16–32. doi:10.1037/0022-3514.91.1.16
53.
KnappD. J.HeggestadE. D.YoungM. C. (Eds.). (2004). Understanding and improving the Assessment of Individual Motivation (AIM) in the Army’s GED Plus program(ARI Study Note 2004-03). Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
54.
KomarS.BrownD. J.KomarJ. A.RobieC. (2008). Faking and the validity of conscientiousness: A Monte Carlo investigation. Journal of Applied Psychology, 93, 140–154. doi:10.1037/0021-9010
55.
KomarS.KomarJ. A.RobieC.TaggarS. (2010). Speeding personality measures to reduce faking: A self-regulatory model. Journal of Personnel Psychology, 9(3), 126–137. doi:10.1027/1866-5888/a000016
56.
KuiperN. A. (1981). Convergent evidence for the self as a prototype: The inverted-U RT effect for self and other judgments. Personality and Social Psychology Bulletin, 7, 438–443
57.
MarkusH. (1977). Self-schemata and processing information about the self. Journal of Personality and Social Psychology, 35, 63–78.
58.
McDanielM. A.TimmH. (1990). Lying takes time: Predicting deception in biodata using response latency. In WingH. C. (Ed.), Alternative predictors. Symposium presented at the 98th annual conference of the American Psychological Association, Boston, MA.
59.
McFarlandL. A.RyanA. M. (2000). Variance in faking across non-cognitive measures. Journal of Applied Psychology, 85(5), 812–821. doi:10.1037/0021-9010.85.5.812
60.
MeadeA. W.CraigS. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455. doi:10.1037/a0028085
61.
MorgesonF. P.CampionM. A.DipboyeR. L.HollenbeckJ. R.MurphyK.SchmittN. (2007). Reconsidering the use of personality tests in personnel selection contexts. Personnel Psychology, 60(3), 683–729. doi:10.1111/j.1744-6570.2007.00089.x
62.
MountM. K.BarrickM. R.StraussJ. P. (1994). Validity of observer ratings of the Big Five personality factors. Journal Of Applied Psychology, 79(2), 272–280. doi:10.1037/0021-9010.79.2.272
63.
Mueller-HansonR.HeggestadE. D.ThorntonG. C.III. (2003). Faking and selection: Considering the use of personality from select-in and select-out perspectives. Journal of Applied Psychology, 88, 348–355. doi:10.1037/0021-9010.88.2.348
64.
NormanW. T. (1963). Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings. Journal of Abnormal and Social Psychology, 66(6), 574–583. doi:10.1037/h0040291
65.
NosekB. A.BanajiM. R. (2001). The go/no-go association task. Social Cognition, 19(6), 161–176.
66.
OdellC. W. (1931). Further data concerning the effect of weighting exercises in new-type examinations. Journal of Educational Psychology, 22, 700–704.
67.
OnesD. S.ViswesvaranC.ReissA. D. (1996). Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81(6), 660–679. doi:10.1037/0021-9010.81.6.660
68.
OnesD. S.ViswesvaranC.SchmidtF. L. (1993). Comprehensive meta-analysis of integrity test validities: Findings and implications for personnel selection and theories of job performance. Journal of Applied Psychology, 78(4), 679–703. doi:10.1037/0021-9010.78.4.679
69.
OsgoodC. E.SuciG.TannenbaumP. (1957). The measurement of meaning. Urbana: University of Illinois Press.
70.
OswaldF. L.HoughL. M. (2011). Personality and its assessment in organizations: Theoretical and empirical developments. In ZedeckS. (Ed.), APA handbook of industrial and organizational psychology, Vol. 2: Selecting and developing members for the organization (pp. 153–184). Washington, DC: American Psychological Association. doi:10.1037/12170-005
71.
PaulhusD. L. (1991). Measurement and control of response bias. In RobinsonJ. P.ShaverP. R.WrightsmanL. S. (Eds.), Measures of personality and social psychological attitudes (pp. 17–59). New York, NY: Academic Press.
72.
PodsakoffP. M.MacKenzieS. B.LeeJ. Y.PodsakoffN. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. doi:10.1037/0021-9010.88.5.879
73.
PophamS. M.HoldenR. R. (1990). Assessing MMPI constructs through the measurement of response latencies. Journal of Personality Assessment, 54, 469–478.
74.
RangerJ.OrtnerT. M. (2011). Assessing personality traits through response latencies using item response theory. Educational and Psychological Measurement, 71(2), 389–406. doi:10.1177/0013164410382895
75.
RobieC.BrownD. J.BeatyJ. C. (2007). Do people fake on personality inventories? A verbal protocol analysis. Journal of Business and Psychology, 21, 489–509.
76.
RobieC.KomarS.BrownD. J. (2010). The effects of coaching and speeding on Big Five and impression management scale scores. Human Performance, 23(5), 446–467. doi:10.1080/08959285.2010.515278
77.
RogersT. B. (1981). A model of the self as an aspect of the human information processing system. In CantorN.KihlstromJ. F. (Eds.), Personality, cognition, and social interaction (pp. 193–214). Hillsdale, NJ: Erlbaum.
78.
RosseJ. G.StecherM. D.MillerJ. L.LevinR. A. (1998). The impact of response distortion on preemployment personality testing and hiring decisions. Journal of Applied Psychology, 83(4), 634–644. doi:10.1037/0021-9010.83.4.634
79.
RyanA. M.DelanyT. (2010). Attracting job candidates to organizations. In FarrJ. L.TippinsN. T. (Eds.), Handbook of employee selection (pp. 127–150). New York, NY: Routledge.
80.
RyanA. M.PloyhartR. E. (2000). Applicants’ perceptions of selection procedures and decisions: A critical review and agenda for the future. Journal of Management, 26(3), 565–606. doi:10.1177/014920630002600308
81.
SaucierG.GoldbergL. R. (1996). The language of personality: Lexical perspectives on the five-factor model. In WigginsJ. S. (Ed.), The five-factor model of personality: Theoretical perspectives (pp. 21–50). New York, NY: Guilford.
82.
SchmukleS. C.EgloffB. (2005). A latent state-trait analysis of implicit and explicit personality measures. European Journal of Psychological Assessment, 21(2), 100–107. doi:10.1027/1015-5759.21.2.100
83.
SchnabelK.AsendorpfJ. B.GreenwaldA. G. (2008). Understanding and using the implicit association test: V. Measuring semantic aspects of trait self-concepts. European Journal of Personality, 22(8), 695–706. doi:10.1002/per.697
84.
SchneiderR. J.HoughL. M.DunnetteM. D. (1996). Broadsided by broad traits: How to sink science in five dimensions or less. Journal of Organizational Behavior, 17, 639–655.
85.
SjöbergL. (2015). Correction for faking in self-report personality tests. Scandinavian Journal of Psychology, 56, 582–591.
86.
StarkS.DrasgowF.ChernyshenkoO. S. (2008, 8). Update on the Tailored Adaptive Personality Assessment System (TAPAS): The next generation of personality assessment systems to support personnel selection and classification decisions. Paper presented at the 50th annual conference of the International Military Testing Association, Amsterdam, Netherlands.
87.
SteffensM. C.KönigS. S. (2006). Predicting spontaneous Big Five behavior with implicit association tests. European Journal of Psychological Assessment, 22(1), 13–20. doi:10.1027/1015-5759.22.1.13
88.
StrickerL. J.AldertonD. L. (1999). Using response latency measures for a biographical inventory. Military Psychology, 11(2), 169–188. doi:10.1207/s15327876mp1102_3
89.
TettR. P.ChristiansenN. D. (2007). Personality tests at the crossroads: A response to Morgeson, Campion, Dipboye, Hollenbeck, Murphy, and Schmitt (2007). Personnel Psychology, 60(4), 967–993. doi:10.1111/j.1744-6570.2007.00098.x
90.
TourangeauR.RasinskiK. A. (1988). Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin, 103, 299–314. doi:10.1037/0033-2909.103.3.299
91.
TourangeauR.RipsL. J.RasinskiK. A. (2000). The psychology of survey response. Cambridge, UK: Cambridge University Press.
92.
TupesE. C.ChristalR. E. (1992). Recurrent personality factors based on trait ratings. Journal of Personality, 60(2), 225–251. doi:10.1111/j.1467-6494.1992.tb00973.x
93.
van HooftE. A.BornM. P. (2012). Intentional response distortion on personality tests: Using eye-tracking to understand response processes when faking. Journal of Applied Psychology, 97, 301–306.
94.
VasilopoulosN. L.ReillyR. R.LeamanJ. A. (2000). The influence of job familiarity and impression management on self-report measure scale scores and response latencies. Journal of Applied Psychology, 85(1), 50–64. doi:10.1037/0021-9010.85.1.50
95.
VecchioneM.DentaleF.AlessandriG.BarbaranelliC. (2014). Fakability of implicit and explicit measures of the Big Five: Research findings from organizational settings. International Journal of Selection and Assessment, 22, 211–218. doi:10.1111/ijsa.12070
96.
ViswesvaranC.OnesD. S. (1999). Meta-analyses of fakability estimates: Implications for personality measurement. Educational and Psychological Measurement, 59(2), 197–210. doi:10.1177/00131649921969802
97.
WangM. D.StanleyJ. C. (1970). Differential weighting: A review of methods and empirical studies. Review of Educational Research, 40, 663–705.
98.
WooS. E.ChernyshenkoO. S.LongleyA.ZhangZ. X.ChiuC. Y.StarkS. E. (2014). Openness to experience: Its lower level structure, measurement, and cross-cultural equivalence. Journal of Personality Assessment, 96(1), 29–45.
99.
ZickarM. J.RobieC. (1999). Modeling faking good on personality items: An item-level analysis. Journal of Applied Psychology, 84(4), 551–563. doi:10.1037/0021-9010.84.4.551
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
