Prominent standards/guidelines concerning test validation provide contradictory information about whether content-based evidence should be used as a means of validating personality test inferences for employee selection. This unresolved discrepancy is problematic considering the prevalence of personality testing, the importance of gathering sound validity evidence, and the deference given to these standards/guidelines in contemporary employee selection practice. As a consequence, test users and practitioners are likely to be reticent or uncertain about gathering content-based evidence for personality measures, which, in turn, may cause such evidence to be underutilized when personality testing is of interest. The current investigation critically examines whether (and how) content validity evidence should be used for measures of personality in relation to employee selection. The ensuing discussion, which is especially relevant in highly litigious contexts such as personnel selection in the public sector, sheds new light on test validation practices.
AguinisH.HenleC. A.OstroffC. (2001). Measurement in work and organizational psychology. In AndersonN.OnesD. S.SinangilH. K.ViswesvaranC. (Eds.), Handbook of industrial, work and organizational psychology (Vol. 1, pp. 27–50). SAGE.
2.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing. (Original work published 1999)
3.
ArthurW.Jr.VilladoA. J. (2008). The importance of distinguishing between constructs and methods when comparing predictors in personnel selection research and practice. Journal of Applied Psychology, 93, 435–442. https://doi.org/10.1037/0021-9010.93.2.435
4.
BiddleD. A. (2011). Adverse impact and test validation: A practitioner’s handbook (3rd ed.). Infinity Publishing.
5.
BinningJ. F.BarrettG. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74, 478–494. https://doi.org/10.1037/0021-9010.74.3.478
6.
BinningJ. F.LeBretonJ. M. (2009). Coherent conceptualization is useful for many things, and understanding validity is one of them. Industrial and Organizational Psychology, 2, 486–492. https://doi.org/10.1111/j.1754-9434.2009.01178.x
7.
BrannickM. T.LevineE. L.MorgesonF. P. (2007). Job and work analysis: Methods, research, and applications for human resource management (2nd ed.). SAGE.
8.
EllingsonJ. E.HeggestadE. D.MyersH. (2013). The workplace IPIP: A contextualized measure of personality [Unpublished manuscript].
9.
Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, & Department of Justice. (1978). Uniform guidelines on employee selection procedures. Federal Register, 43, 38290–38315.
10.
FurrR. M.BacharachV. R. (2014). Psychometrics: An introduction (2nd ed.). SAGE.
11.
GoldbergL. R.JohnsonJ. A.EberH. W.HoganR.AshtonM. C.CloningerC. R.GoughH. C. (2006). The international personality item pool and the future of public-domain personality measures. Journal of Research in Personality, 40, 84–96. https://doi.org/10.1016/j.jrp.2005.08.007
12.
GoldsteinI. L.ZedeckS.SchneiderB. (1993). An exploration of the job analysis–content validity process. In SchmittN.BormanW. C. (Eds.), Personnel selection in organizations (pp. 3–34). Jossey-Bass.
GuionR. M. (2004). Validity and reliability. In RogelbergS. G. (Ed.), Handbook of research methods in industrial and organizational psychology (pp. 57–76). Blackwell Publishing.
15.
HaynesS. N.RichardD.KubanyE. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7, 238–247. https://doi.org/10.1037/1040-3590.7.3.238
16.
HsuC. (2004). The testing of America. U.S. News and World Report, 137, 68–69.
17.
HunthausenJ. M.TruxilloD. M.BauerT. N.HammerL. B. (2003). A field study of frame-of-reference effects on personality test validity. Journal of Applied Psychology, 88, 545–551. https://doi.org/10.1037/0021-9010.88.3.545
18.
JeanneretP. R.ZedeckS. (2010). Professional guidelines/standards. In FarrJ. L.TippinsN. T. (Eds.), Handbook of employee selection (pp. 593–625). Routledge.
LievensF.De CorteW.SchollaertE. (2008). A closer look at the frame-of-reference effect in personality scale scores and validity. Journal of Applied Psychology, 93(2), 268–279. https://doi.org/10.1037/0021-9010.93.2.268
23.
McDanielM. A.KepesS.BanksG. C. (2011). The uniform guidelines are a detriment to the field of personnel selection. Industrial and Organizational Psychology, 4, 494–514. https://doi.org/10.1111/j.1754-9434.2011.01382.x
24.
MorgesonF. P.CampionM. A.DipboyeR. L.HollenbeckJ. R.MurphyK.SchmittN. (2007a). Are we getting fooled again? Coming to terms with limitations in the use of personality tests for personnel selection. Personnel Psychology, 60, 1029–1047. https://doi.org/10.1111/j.1744-6570.2007.00100.x
25.
MorgesonF. P.CampionM. A.DipboyeR. L.HollenbeckJ. R.MurphyK.SchmittN. (2007b). Reconsidering the use of personality tests in personnel selection contexts. Personnel Psychology, 60, 683–729. https://doi.org/10.1111/j.1744-6570.2007.00089.x
26.
MurphyK. R.DzieweczynskiJ. L.ZhangY. (2009). Positive manifold limits the relevance of content-matching strategies for validating selection test batteries. Journal of Applied Psychology, 94, 1018–1031. https://doi.org/10.1037/a0014075
27.
O’NeillT. A.GoffinR. D.RothsteinM. (2013). Personality and the need for personality-oriented work analysis. In ChristiansenN. D.TettR. P. (Eds.), Handbook of personality at work (pp. 226–252). Routledge.
28.
O’NeillT. A.GoffinR. D.TettR. P. (2009). Content validation is fundamental for optimizing the criterion validity of personality tests. Industrial and Organizational Psychology, 2, 509–513. https://doi.org/10.1111/j.1754-9434.2009.01184.x
29.
OnesD. S.DilchertS.ViswesvaranC.JudgeT. A. (2007). In support of personality assessment in organizational settings. Personnel Psychology, 60, 995–1027. https://doi.org/10.1111/j.1744-6570.2007.00099.x
30.
OnesD. S.ViswesvaranC. (2001). Integrity tests and other criterion-focused occupational personality scales (COPS) used in personnel selection. International Journal of Selection and Assessment, 9, 31–39. https://doi.org/10.1111/1468-2389.00161
31.
RaymarkP. H.SchmitM. J.GuionR. M. (1997). Identifying potentially useful personality constructs for employee selection. Personnel Psychology, 50, 723–736. https://doi.org/10.1111/j.1744-6570.1997.tb00712.x
32.
RothsteinM. G.GoffinR. D. (2006). The use of personality measures in personnel selection: What does current research support?Human Resource Management Review, 16, 155–180. https://doi.org/10.1016/j.hrmr.2006.03.004
33.
SchmitM. J.RyanA. M. (2013). Legal issues in personality testing. In ChristiansenN. D.TettR. P. (Eds.), Handbook of personality at work (pp. 525–542). Routledge.
34.
SchmitM. J.RyanA. M.StierwaltS. L.PowellA. B. (1995). Frame-of-reference effects on personality scale scores and criterion-related validity. Journal of Applied Psychology, 80, 607–620. https://doi.org/10.1037/0021-9010.80.5.607
35.
SchmittM. (2006). Conceptual, theoretical, and historical foundations of multimethod assessment. In EidM.DienerE. (Eds.), Handbook of multimethod measurement in psychology (pp. 9–25). American Psychological Association.
36.
ShafferJ. A.PostlethwaiteB. W. (2012). A matter of context: A meta-analytic investigation of the relative validity of contextualized and noncontextualized personality measures. Personnel Psychology, 65, 445–494. https://doi.org/10.1111/j.1744-6570.2012.01250.x
Society for Industrial and Organizational Psychology. (2003). Principles for the validation and use of personnel selection procedures (4th ed.).
39.
SpenglerM.GellériP.SchulerH. (2009). The construct behind content validity: New approaches to a better understanding. Industrial and Organizational Psychology, 2, 504–508. https://doi.org/10.1111/j.1754-9434.2009.01183.x
40.
TanJ. A. (2009). Babies, bathwater, and validity: Content validity is useful in the validation process. Industrial and Organizational Psychology, 2, 514–516. https://doi.org/10.1111/j.1754-9434.2009.01185.x
41.
TettR. P.BurnettD. D. (2003). A personality trait-based interactionist model of job performance. Journal of Applied Psychology, 88, 500–517. https://doi.org/10.1037/0021-9010.88.3.500
42.
TettR. P.ChristiansenN. D. (2007). Personality tests at the crossroads: A response to Morgeson, Campion, Dipboye, Hollenbeck, Murphy, and Schmitt (2007). Personnel Psychology, 60, 967–993. https://doi.org/10.1111/j.1744-6570.2007.00098.x
43.
TettR. P.SimonetD. V.WalserB.BrownC. (2013). Trait activation theory: Applications, developments, and implications for person–workplace fit. In ChristiansenN. D.TettR. P. (Eds.), Handbook of personality at work (pp. 71–100). Routledge.
