New measures of test information, termed global information, quantify test information relative to the entire range of the trait being assessed. Estimating global information relative to a non-informative prior distribution results in a measure of how much information could be gained by administering the test to an unspecified examinee. Currently, such measures have been developed only for unidimensional tests. This study introduces measures of multidimensional global test information and validates them in simulated data. Then, the utility of global test information is tested in neuropsychological data collected as part of Rush University’s Memory and Aging Project. These measures allow for direct comparison of complex tests calibrated in different samples, facilitating test development and selection.
AnsleyT. N.ForsythR. A. (1985). An examination of the characteristics of unidimensional IRT parameter estimates derived from two-dimensional data. Applied Psychological Measurement, 9(1), 37–48. https://doi.org/10.1177/014662168500900104
2.
BechgerT. M.MarisG.VerstralenH. H. F. M.BéguinA. A. (2003). Using classical test theory in combination with item response theory. Applied Psychological Measurement, 27(5), 319–334. https://doi.org/10.1177/0146621603257518
3.
BennettD. A.BuchmanA. S.BoyleP. A.BarnesL. L.WilsonR. S.SchneiderJ. A. (2018). Religious orders study and rush memory and aging project. Journal of Alzheimer’s Disease, 64(s1), S161–S189. https://doi.org/10.3233/JAD-179939
4.
BergerJ. O.BernardoJ. M.SunD. (2009). The formal definition of reference priors. The Annals of Statistics, 37(2), 905–938. https://doi.org/10.1214/07-AOS587
BernardoJ. M. (1979b). Reference posterior distributions for Bayesian inference. Journal of the Royal Statistical Society: Series B (Methodological), 41(2), 113–128. https://doi.org/10.1111/j.2517-6161.1979.tb01066.x
BirnbaumA. (1968). Some latent trait models and their use in inferring an examinee’s ability. In: Statistical theories of mental test scores. Addison-Wesley.
9.
BodnarO.ElsterC. (2014). Analytical derivation of the reference prior by sequential maximization of Shannon’s mutual information in the multi-group parameter case. Journal of Statistical Planning and Inference, 147, 106–116. https://doi.org/0.1016/j.jspi.2013.11.003
10.
BonifayW. (2019) Multidimensional item response theory (Vol. 183). SAGE Publications.
11.
CaiL. (2010). High-dimensional exploratory item factor analysis by a metropolis-hastings robbins-monro algorithm. Psychometrika, 75(1), 33–57. https://doi.org/10.1007/s11336-009-9136-x
12.
CamilliG. (1994). Teacher’s corner: origin of the scaling constant d = 1.7 in item response theory. Journal of Educational Statistics, 19(3), 293–295. https://doi.org/10.3102/10769986019003293
13.
ChalmersR. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. https://doi.org/10.18637/jss.v048.i06
14.
ChangH.-H.YingZ. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20(3), 213–229. https://doi.org/10.1177/014662169602000303
15.
ClarkeB. S.BarronA. R. (1990). Information-theoretic asymptotics of Bayes methods. IEEE Transactions on Information Theory, 36(3), 453–471. https://doi.org/10.1109/18.54897
16.
ClarkeB. S.BarronA. R. (1994). Jeffreys’ prior is asymptotically least favorable under entropy risk. Journal of Statistical Planning and Inference, 41(1), 37–60. https://doi.org/10.1016/0378-3758(94)90153-8
17.
FolkV. G.GreenB. F. (1989). Adaptive estimation when the unidimensionality assumption of IRT is violated. Applied Psychological Measurement, 13(4), 373–390. https://doi.org/10.1177/014662168901300404
18.
HulinC. L.LissakR. I.DrasgowF. (1982). Recovery of two- and three-parameter logistic item characteristic curves: A Monte Carlo study. Applied Psychological Measurement, 6(3), 249–260. https://doi.org/10.1177/014662168200600301
19.
KaplanA. (1973). The conduct of inquiry. Transaction Publishers.
LindleyD. V. (1956). On a measure of the information provided by an experiment. The Annals of Mathematical Statistics, 27(4), 986–1005. https://doi.org/10.1214/aoms/1177728069
22.
MarkonK. E. (2013). Information utility: Quantifying the total psychometric information provided by a measure. Psychological Methods, 18(1), 15–35. https://doi.org/10.1037/a0030638
23.
MulderJ.van der LindenW. J. (2009). Multidimensional adaptive testing with optimal design criteria for item selection. Psychometrika, 74(2), 273–296. https://doi.org/10.1007/s11336-008-9097-5
24.
PaulhusD. L. (1981). Control of social desirability in personality inventories: Principal-factor deletion. Journal of Research in Personality, 15(3), 383–388. https://doi.org/10.1016/0092-6566(81)90035-0
25.
R Development Core Team (2010). R: A language and environment for statistical computing. http://www.R-project.org
26.
RajuN. S.PriceL. R.OshimaT. C.NeringM. L. (2007). Standardized conditional SEM: A case for conditional reliability. Applied Psychological Measurement, 31(3), 169–180. https://doi.org/10.1177/0146621606291569
27.
RechaseM. D. (1997). The past and future of multidimensional item response theory. Applied Psychological Measurement, 21(1), 25–36. https://doi.org/10.1177/0146621697211002
28.
SegallD. O. (2001). General ability measurement: An application of multidimensional item response theory. Psychometrika, 66(1), 79–97. https://doi.org/10.1007/BF02295734
29.
VeldkampB. P.van der LindenW. J. (2002). Multidimensional adaptive testing with constraints on test content. Psychometrika, 67(4), 575–588. https://doi.org/10.1007/BF02295132
30.
WangC.ChangH.-H. (2011). Item selection in multidimensional computerized adaptive testing-gaining information from different angles. Psychometrika, 76(3), 363–384. https://doi.org/10.1007/s11336-011-9215-7
31.
WangC.ChangH.-H.BoughtonK. A. (2010). Kullback-Leibler information and its applications in multi-dimensional adaptive testing. Psychometrika, 76(1), 13–39. https://doi.org/10.1007/s11336-010-9186-0
32.
WayW. D.AnsleyT. N.ForsythR. A. (1988). The comparative effects of compensatory and noncompensatory two-dimensional data on unidimensional IRT estimates. Applied Psychological Measurement, 12(3), 239–252. https://doi.org/10.1177/014662168801200303
33.
WirthR. J.EdwardsM. C. (2007). Item factor analysis: current approaches and future directions. Psychological Methods, 12(1), 58–79. https://doi.org/10.1037/1082-989X.12.1.58
