There is re-emerging interest in adopting forced-choice items to address the issue of response bias in Likert-type items for noncognitive latent traits. Multidimensional pairwise comparison (MPC) items are commonly used forced-choice items. However, few studies have been aimed at developing item response theory models for MPC items owing to the challenges associated with ipsativity. Acknowledging that the absolute scales of latent traits are not identifiable in ipsative tests, this study developed a Rasch ipsative model for MPC items that has desirable measurement properties, yields a single utility value for each statement, and allows for comparing psychological differentiation between and within individuals. The simulation results showed a good parameter recovery for the new model with existing computer programs. This article provides an empirical example of an ipsative test on work style and behaviors.
AdamsR. J.WuM. L.WilsonM. R. (2015). ACER Conquest 4.0 [Computer program]. Melbourne: Australian Council for Educational Research.
2.
AndrichD. (1995). Hyperbolic cosine latent trait models for unfolding direct responses and pairwise preferences. Applied Psychological Measurement, 19, 269-290.
3.
AshmanA.TelferR. (1983). Personality profiles of pilots. Aviation, Space, and Environmental Medicine, 54, 940-943.
4.
BarrettL. F. (2004). Feelings or words? Understanding the content in self-report ratings of experienced emotion. Journal of Personality and Social Psychology, 87, 266-281.
5.
BöckenholtU. (2004). Comparative judgments as an alternative to ratings: Identifying the scale origin. Psychological Methods, 9, 453-465.
6.
BradleyR. A.TerryM. E. (1952). Rank analysis of incomplete block design 1: The method of paired comparisons. Biometrika, 39, 324-345.
7.
BrownA. (2016a). Item response models for forced-choice questionnaires: A common framework. Psychometrika, 81, 135-160.
De BoeckP.WilsonM. (2004). A framework for item response models. In De BoeckP.WilsonM. (Eds.), Explanatory item response models: A generalized linear and nonlinear approach (pp. 3–41). New York, NY: Springer.
12.
GelmanA.MengX.-L.SternH. S. (1996). Posterior predictive assessment of model fitness via realized discrepancies (with discussion). Statistica Sinica, 6, 733-807.
13.
GreenbergerE.ChenC.DmitrievaJ.FarruggiaS. P. (2003). Item-wording and the dimensionality of the Rosenberg Self-Esteem Scale: Do they matter?Personality and Individual Differences, 35, 1241-1254.
14.
GuenoleN.BrownA.CooperA. J. (2016). Forced choice assessment of work related maladaptive personality traits: Preliminary evidence from an application of Thurstonian item response modeling. Assessment. Advance online publication. doi:10.1177/1073191116641181
15.
HarrisJ. A.VernonP. A.JangK. L. (2005). Testing the differentiation of personality by intelligence hypothesis. Personality and Individual Differences, 38, 277-286.
16.
HollandJ. L. (1973). Making vocational choices: A theory of careers. Englewood Cliffs, NJ: Prentice Hall.
17.
HontangasP. M.de la Torre PonsodaJ. V.LeenenI.MorilloD.AbadF. J. (2015). Comparing traditional and IRT scoring of forced-choice tests. Applied Psychological Measurement, 39, 598-612.
18.
HuiC.PakS.ChengH. (2009). Validation studies on a measure of overall managerial readiness for the Chinese. International Journal of Selection and Assessment, 17, 127-141.
19.
JacksonD. N. (1977). Jackson vocational interest survey manual. Port Huron, MI: Research Psychologists Press.
20.
JohnsonE. G. (1992). The design of the national assessment of educational progress. Journal of Educational Measurement, 29, 95-110.
KopelmanR. E.RovenporJ. L.GuanM. W. (2003). The study of values: Construction of the fourth edition. Journal of Vocational Behavior, 62, 203-220.
23.
LuceR. D. (1959). Individual choice behavior. New York, NY: John Wiley.
24.
MatthewsG.OddyK. (1997). Ipsative and normative scales in adjectival measurement of personality: Problems of bias and discrepancy. International Journal of Selection and Assessment, 5, 169-182.
25.
Maydeu-OlivaresA.BrownA. (2010). Item response modeling of paired comparison and ranking data. Multivariate Behavioral Research, 45, 935-974.
26.
McCloyR. A.HeggestadE. D.ReeveC. L. (2005). A silk purse from the sow’s ear: Retrieving normative information from multidimensional forced-choice items. Organizational Research Methods, 8, 222-248.
27.
MeadeA. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of Occupational and Organizational Psychology, 77, 531-552.
28.
MislevyR. J.BeatonA. E.KaplanB.SheehanK. M. (1992). Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement, 29, 133-161.
PlummerM. (2003, March). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. Paper presented at the 3rd International Workshop on Distributed Statistical Computing, Vienna, Austria.
31.
RobertsJ. S.DonoghueJ. R.LaughlinJ. E. (2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24, 3-32.
32.
SpiegelhalterD.ThomasA.BestN. (2003). WinBUGS version 1.4[Computer program]. Cambridge, UK: MRC Biostatistics Unit, Institute of Public Health.
33.
StarkS.ChernyshenkoO. S.DrasgowF. (2005). An IRT approach to constructing and scoring pairwise preference items involving stimuli on different dimensions: The multi-unidimensional pairwise-preference model. Applied Psychological Measurement, 29, 184-203.
34.
StarkS.ChernyshenkoO. S.DrasgowF.WhiteL. A. (2012). Adaptive testing with multidimensional pairwise preference items: Improving the efficiency of personality and other noncognitive assessments. Organizational Research Methods, 15, 463-487.
35.
ThurstoneL. L. (1927). A law of comparative judgment. Psychological Review, 79, 281-299.
36.
WangW.-C.QiuX.-L.ChenC.-W.RoS. (2016). Item response theory models for multidimensional ranking items. In van der ArkL. A.BoltD. M.WangW.-C.DouglasJ. A.WibergM. (Eds.), Quantitative psychology research (pp. 49-65). New York, NY: Springer.
37.
WitkinH. A.GoodenoughD. R.OltmanP. K. (1979). Psychological differentiation: Current status. Journal of Personality and Social Psychology, 37, 1127-1145.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
