The rank two-parameter logistic (Rank-2PL) item response theory models refer to a set of models applying the 2PL model in a sequential ranking process that occurs in forced-choice questionnaires. The multi-unidimensional pairwise preference with 2PL model (MUPP-2PL) is a Rank-2PL model for items with two statements. Focusing on items with three statements, we develop a maximum marginal likelihood estimation with an expectation-maximization algorithm to estimate item parameters and their standard errors. A simulation study is conducted to check parameter recovery, and then the model is applied to a real dataset. Finally, the findings are summarized and discussed, and future research is suggested.
AtkinsonK. E. (1989). An introduction to numerical analysis (2nd ed.). John Wiley.
2.
BirkelandS. A.MansonT. M.KisamoreJ. L.BrannickM. T.SmithM. A. (2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14(4), 317–335. https://doi.org/10.1111/j.1468-2389.2006.00354.x
3.
BirnbaumA. (1968). Some latent trait models and their use in inferring an examinee’s ability. In LordF. M.NovickM. R. (Eds.), Statistical theories of mental test scores (pp. 397–479). Addison-Wesley Pub. Co.
4.
BockR. D.AitkinM. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46(4), 443–459. https://doi.org/10.1007/BF02293801
BrownA.Maydeu-OlivaresA. (2011). Item response modeling of forced-choice questionnaires. Educational and Psychological Measurement, 71(3), 460–502. https://doi.org/10.1177/0013164410375112
7.
BrownA.Maydeu-OlivaresA. (2012). Fitting a Thurstonian IRT model to forced-choice data using Mplus. Behavior Research Methods, 44, 1135–1147. https://doi.org/10.3758/s13428-012-0217-x
8.
BroydenC. G. (1970). The convergence of a class of double-rank minimization algorithms 1. General considerations. IMA Journal of Applied Mathematics, 6(1), 76–90. https://doi.org/10.1093/imamat/6.1.76
9.
BürknerP.-C.SchulteN.HollingH. (2019). On the statistical and practical limitations of Thurstonian IRT models. Educational and Psychological Measurement, 79(5), 827–854. https://doi.org/10.1177/0013164419832063
10.
CaiL. (2010). High-dimensional exploratory item factor analysis by a Metropolis-Hastings Robbins-Monro algorithm. Psychometrika, 75(1), 33–57. https://doi.org/10.1007/s11336-009-9136-x
11.
CaiL. (2023). flexMIRT (Version 3.6): Flexible multilevel multidimensional item analysis and test scoring [Computer software]. Vector Psychometric Group. https://vpgcentral.com/software/flexmirt/
12.
CaiL.HansenM. (2013). Limited-information goodness-of-fit testing of hierarchical item factor models. British Journal of Mathematical and Statistical Psychology, 66(2), 245–276. https://doi.org/10.1111/j.2044-8317.2012.02050.x
13.
ChalmersR., P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. https://doi.org/1.18637/jss.v048.i06
14.
ChalmersR. P. (2018). Numerical approximation of the observed information matrix with Oakes’ identity. British Journal of Mathematical and Statistical Psychology, 71(3), 415–436. https://doi.org/10.1111/bmsp.12127
15.
ChenW. H.ThissenD. (1997). Local dependence indices for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265–289. https://doi.org/10.2307/1165285
16.
ChenY.ZhangS. (2021). Estimation methods for item factor analysis: An overview. In ZhaoY.ChenD. G. (Eds.), Modern statistical methods for health research: Emerging topics in statistics and biostatistics (pp. 329–350). Springer, Cham. https://doi.org/10.1007/978-3-030-72437-5_15
de la TorreJ.PonsodaV.LeenenI.HontangasP. (2012, April). Examining the viability of recent models for forced-choice data [Paper presentation]. American Educational Research Association.
19.
DrasgowF.LevineM. V.WilliamsE. A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38(1), 67–86. https://doi.org/10.1111/j.2044-8317.1985.tb00817.x
20.
ForeroC. G.Maydeu-OlivaresA. (2009). Estimation of IRT graded response models: Limited versus full information methods. Psychological Methods, 14(3), 275–299. https://doi.org/10.1037/a0015825
21.
FuJ. (2019). Maximum marginal likelihood estimation with an expectation-maximization algorithm for multigroup/mixture multidimensional item response theory models. (ETS Research Report No. 19–35). Educational Testing Service. https://doi.org/10.1002/ets2.12272
22.
FuJ.KyllonenP. C.TanX. (2024): From Likert to forced choice: Statement parameter invariance and context effects in personality assessment. Measurement: Interdisciplinary Research and Perspectives. Advance online publication. https://doi.org/10.1080/15366367.2023.2258482
23.
Garnier-VillarrealM.MerkleE. C.MagnusB. E. (2021). Between-item multidimensional IRT: How far can the estimation methods go?Psych, 3, 404–421. https://doi.org/10.3390/psych3030029
24.
GilksW. R.RichardsonS.SpiegelhalterD. J. (1996). Introducing Markov Chain Monte Carlo. In GilksW. R.RichardsonS.SpiegelhalterD. J. (Eds.), Markov Chain Monte Carlo in practice. (pp. 1–20). Chapman and Hall. https://doi.org/10.1201/b14835
25.
HabermanS. J. (2013). A general program for item-response analysis that employs the stabilized Newton-Raphson algorithm (Research Report No. RR-13-32). Educational Testing Service. https://doi.org/10.1002/j.2333-8504.2013.tb02339.x
26.
HontangasP. M.de la TorreJ.PonsodaV.LeenenI.MorilloD.AbadF. J. (2015). Comparing traditional and IRT scoring of forced-choice tests. Applied Psychological Measurement, 39(8), 598–612. https://doi.org/10.1177/0146621615585851
27.
KimS. (2012). A note on the reliability coefficients for item response model-based ability estimates. Psychometrika, 77(4), 153–162. https://doi.org/10.1007/s11336-011-9238-0
28.
LeeP.JooS. H.StarkS.ChernyshenkoO. S. (2019). GGUM-RANK statement and person parameter estimation with multidimensional forced choice triplets. Applied Psychological Measurement, 43(3), 226–240. https://doi.org/10.1177/0146621618768294
29.
LouisT.A. (1982). Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Society. Series B, 44(2), 226–233. http://www.jstor.org/stable/2345828.
30.
LuceR. D. (2005). Individual choice behavior. Dover. (Original work published in 1959.)
31.
Maydeu-OlivaresA. (1999). Thurstonian modeling of ranking data via mean and covariance structure analysis. Psychometrika, 64(3), 325–340. https://doi.org/10.1007/BF02294299
32.
Maydeu-OlivaresA.BrownA. (2010). Item response modeling of paired comparison and ranking data. Multivariate Behavioral Research, 45(6), 935–974. https://doi.org/10.1080/00273171.2010.531231
33.
Maydeu-OlivaresA.JoeH. (2014). Assessing Approximate Fit in Categorical Data Analysis. Multivariate Behavioral Research, 49(4), 305–328. https://doi.org/10.1080/00273171.2014.911075
34.
McDonaldR. P. (1997). Normal-Ogive multidimensional model. In van der LindenW. J.HambletonR. K. (Eds.), Handbook of modern item response theory (pp. 258–270). Springer.
35.
MengX.-L.SchillingS. (1996). Fitting full-information item factor models and an empirical investigation of bridge sampling. Journal of the American Statistical Association, 91(435), 1254–1267. https://doi.org/10.1080/01621459.1996.10476995
36.
MorokoffW. J.CaflischR. E. (1995). Quasi-Monte Carlo integration. Journal of Computational Physics, 122(2), 218–230. https://doi.org/10.1006/jcph.1995.1209
37.
MorilloD.AbadF. J.KreitchmannR. S.LeenenI.HontangasP.PonsodaV. (2019). The journey from Likert to forced-choice questionnaires: Evidence of the invariance of item parameters. Journal of Work and Organizational Psychology, 35(2), 75–83. https://doi.org/10.5093/jwop2019a11
38.
MorilloD.LeenenI.AbadF. J.HontangasP.De la TorreJ.PonsodaV. (2016). A dominance variant under the multi-unidimensional pairwise-preference framework: Model formulation and Markov chain Monte Carlo estimation. Applied Psychological Measurement, 40(7), 500–516. https://doi.org/10.1177/0146621616662226
39.
R Core Team (2021). R: A language and environment for statistical computing (Version 4.1.2). R Foundation for Statistical Computing. https://www.R-project.org/.
40.
RobertsJ. S.DonoghueJ. R.LaughlinJ. E. (2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24(1), 3–32. https://doi.org/10.1177/01466216000241001
41.
SalgadoJ. F.AndersonN.TaurizG. (2015). The validity of ipsative and quasi-ipsative forced-choice personality inventories for different occupational groups: A comprehensive meta-analysis. Journal of Occupational and Organizational Psychology, 88(4), 797–834. https://doi.org/10.1111/joop.12098
42.
SchillingS.BockR. D. (2005). High-dimensional maximum marginal likelihood item factor analysis by adaptive quadrature. Psychometrika, 70(3), 533–555. https://doi.org/10.1007/s11336-003-1141-x
43.
StarkS.ChernyshenkoO. S.DrasgowF. (2005). An IRT approach to constructing and scoring pairwise preference items involving stimuli on different dimensions: The multi-unidimensional pairwise preference model. Applied Psychological Measurement, 29(3), 184–203. https://doi.org/10.1177/0146621604273988
von DavierM. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61(2), 287–307. https://doi.org/10.1348/000711007X193957
46.
WangW.-C.QiuX.-L.ChenC.-W.RoS.JinK.-Y (2017). Item response theory models for ipsative tests with multidimensional pairwise-comparison items. Applied Psychological Measurement, 41(8), 600–613. https://doi.org/10.1177/0146621617703183
47.
YuanK.-H.ChengY.PattonJ. (2014). Information matrices and standard errors for MLEs of item parameters in IRT. Psychometrika, 79(1), 232–254. https://doi.org/10.1007/S11336-013-9334-4
48.
ZhangB.TuN.AngraveL.ZhangS.SunT.TayL.LiJ. (2023). The generalized Thurstonian unfolding model (GTUM): Advancing the modeling of forced-choice data. Organizational Research Methods. Advance online publication. https://doi.org/10.1177/10944281231210481
49.
ZhangS.ChenY.LiuY. (2020). An improved stochastic EM algorithm for large-scale full-information item factor analysis. British Journal of Mathematical and Statistical Psychology, 73(1), 44–71. https://doi.org/10.1111/bmsp.12153
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
