The linear composite direction represents, theoretically, where the unidimensional scale would lie within a multidimensional latent space. Using compensatory multidimensional IRT, the linear composite can be derived from the structure of the items and the latent distribution. The purpose of this study was to evaluate the validity of the linear composite conjecture and examine how well a fitted unidimensional IRT model approximates the linear composite direction in a multidimensional latent space. Simulation experiment results overall show that the fitted unidimensional IRT model sufficiently approximates linear composite direction when correlation between bivariate latent variables is positive. When the correlation between bivariate latent variables is negative, instability occurs when the fitted unidimensional IRT model is used to approximate linear composite direction. A real data experiment was also conducted using 20 items from a multiple-choice mathematics test from American College Testing.
AckermanT. A. (1989). Unidimensional IRT calibration of compensatory and noncompensatory multidimensional items. Applied Psychological Measurement, 13(2), 113–127. https://doi.org/10.1177/014662168901300201
2.
AckermanPL (1992). Predicting individual differences in complex skill acquisition: Dynamics of ability determinants. Journal of Applied Psychology, 77(5), 598–614. https://doi.org/10.1037/0021-9010.77.5.598
3.
AckermanT. (1996). Graphical representation of multidimensional item response theory analyses. Applied Psychological Measurement, 20(4), 311–329. https://doi.org/10.1177/014662169602000402
4.
AckermanT. A.GierlM.WalkerC. M (2003). Using multidimensional item response theory to evaluate educational and psychological tests. Educational Measurement: Issues and Practice, 22(3), 37–51.
5.
AckermanT. A. (1994). Using multidimensional item response theory to understand what items and tests are measuring. Applied Measurement in Education, 7, 255–278.
6.
AckermanT. A.MaY.IpE.H. (In press). Comparison of three unidimensional approaches to represent a two-dimensional latent ability space. In Proceedings of the International Meeting of the Psychometric Society, New York City, July 2018.
7.
BockR. D.AitkinM. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46(4), 443–459. https://doi.org/10.1007/bf02293801
8.
CamilliG. (1992). A conceptual analysis of differential item functioning in terms of a multidimensional item response model. Applied Psychological Measurement, 16(2), 129–147. https://doi.org/10.1177/014662169201600203
9.
CarlsonJ. E (2017). Unidimensional vertical scaling in multidimensional space (ETS RR-17-29). Educational Testing Service.
10.
ChalmersP. R (2012). Mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. https://doi.org/10.18637/jss.v048.i06
11.
Del Rosario BasterraM.TrumbullE.Solano-FloresG (2011). Cultural validity in assessment. Routledge.
12.
DrasgowF.ParsonsC. K. (1983). Application of unidimensional item response theory to multidimensional data. Applied Psychological Measurement, 7(2), 189–199. https://doi.org/10.1177/014662168300700207
13.
EmbretsonS. E.ReiseS. P (2000). Item response theory for psychologists. Erlbaum.
HarrisonD. A. (1986). Robustness of Irt Parameter Estimation to Violations of The Unidimensionality Assumption. Journal of Educational Statistics, 11(2): 91–115.
16.
HanK. T.PaekI. (2014). A review of commercial software packages for multidimensional IRT modeling. Applied Psychological Measurement, 38(6), 486–498. https://doi.org/10.1177/0146621614536770
17.
HumphreysL.G (1982). Systematic heterogeneity of items in tests of meaningful and important psychological attributes: A rejection of unidimensionality. In: Unpublished manuscript. University of Illinois at Urbana-Champaign.
18.
IpE. H. (2010). Empirically indistinguishable multidimensional IRT and locally dependent unidimensional item response models. The British Journal of Mathematical and Statistical Psychology, 63(2), 395–416. https://doi.org/10.1348/000711009X466835
19.
IpE. H.StrachanT.FuY.ChenS.RutkowskiL.LayA.WillseJ.AckermanT. (2019). Bias and bias correction method for non-proportional abilities requirement (NPAR) tests. Journal of Educational Measurement, 56(1), 1–22. https://doi.org/10.1111/jedm.12204
JunkerB. W (1994). Inference, robustness, and essential unidimensionality. Talk given at the 10th annual workshop on item response theory modeling. University of twente.
22.
JunkerB. W.StoutW. F (1994).Robustness of ability estimation when multiple traits are present with one trait dominant. In LaveaultD.ZumboB. D.GessaroliM. E.BossM. W. (Eds), Modern theories of measurement: Problems and issues (pp. 31–61). University of Ottawa.
23.
KahramanN.ThompsonT. (2011). Relating unidimensional IRT parameters to a multidimensional response space: A review of two alternative projection IRT models for subscale scores. Journal of Educational Measurement, 48(2), 146–164. https://doi.org/10.1111/j.1745-3984.2011.00138.x
24.
KimH.R (1994). New techniques for the dimensionality assessment of standardized test data. In: Doctoral thesis. University of Illinois.
25.
KolanM. J.BrennanR. L (2014). Test equating, scaling, and linking: Methods and practices. Springer.
26.
LuechtR. M.MillerT. R. (1992). Unidimensional calibrations and interpretations of composite trait for multidimensional tests. Applied Psychological Measurement, 16(3), 279–293. https://doi.org/10.1177/014662169201600308
27.
McDonaldR. P (1997). Normal-ogive multidimensional model. In HambletonR.K.van der LinderW. (Ed.). Handbook of modern item response theory (pp.257-269). Springer. https://doi.org/10.1007/978-1-4757-2691-6_15
28.
R Core Team (2017). R: A language and environment for statistical computing. In: R foundation for statistical computing. URL. https://www.R-project.org/
29.
ReckaseM. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational Statistics, 4(3), 207–230. https://doi.org/10.3102/10769986004003207
30.
ReckaseM. D. (1985). The difficulty of test items that measure more than one dimension. Applied Psychological Measurement, 9(4), 401–412. https://doi.org/10.1177/014662168500900409
31.
ReckaseM. D (2009). Multidimensionality item response theory. : Springer-Verlag.
32.
ReckaseM.D.CarlsonJ.E.AckermanT.A.SprayJ.A (1986). The interpretation of unidimensional IRT parameters when estimated from multidimensional data. In: Paper presented at the annual meeting of the Psychometric Society, Toronto.
33.
ReiseS.P.CookK.F.MooreT.M (2014). Evaluating the impact of multidimensionality on unidimensional item response theory model parameters. In ReiseS.P.RevickiD.A. (Eds.), Handbook of item response theory modeling: Applications to typical performance assessment. Routledge. (pp. 13–40).
34.
SpearmanC. (1927). The abilities of man, New York: Macmillan.
35.
StoutW. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52(4), 589–617. https://doi.org/10.1007/bf02294821
36.
StrachanT.ChoJ.ChenS-H.AckermanT.KimK.Y.WillseJ.WeeksJ.IpE.H (in press). Using a projection IRT method for vertical scaling when construct shift is present. Journal of Educational Measurement.
37.
van der MaasH. L. J.DolanC. V.GrasmanR. P.WichertsJ. M.HuizengaH. M.RaijmakersM. E. (2006). A dynamical model of general intelligence: the positive man-ifold of intelligence by mutualism. Psychological Review, 13, 842–60. DOI: 10.1037/0033-295X.113.4.842.
38.
WangM.M. (1987). Fitting a unidimensional model to multidimensional item response data (ONR Rep. 042286). Iowa City, IA: University of Iowa.
39.
WatsonDClarkLATellegenA (1988). Development and validation of brief measures of positive and negative affect: the PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063–1070. https://doi.org/10.1037//0022-3514.54.6.1063
40.
ZhangJ.StoutW. F. (1999). The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika, 64, 213–249.
41.
ZhangJ.StoutW. (1999a). The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika, 64(2), 213–249. https://doi.org/10.1007/bf02294536
ZhangJ.WangM. (1998, April). Relating reported scores to latent traits in a multidimensional test. Paper presented at the annual meeting of the American Educational Research Association. San Diego, CA.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
