Test-takers, policymakers, teachers, and institutions are increasingly demanding that testing programs provide more detailed feedback regarding test performance. As a result, there has been a growing interest in the reporting of subscores that potentially provide such detailed feedback. Haberman developed a method based on classical test theory for determining whether a subscore has added value over the total score. Sinharay conducted a detailed study using both real and simulated data and concluded that it is not common for subscores to have added value according to Haberman’s criterion. However, Sinharay almost exclusively dealt with data from tests with only dichotomous items. In this article, we show that it is more common for subscores to have added value in tests with polytomous items.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing (5th ed.).
2.
BulutO.DavisonM. L.RodriguezM. C. (2017). Estimating between-person and within-person subscore reliability with profile analysis. Multivariate Behavioral Research, 52(1), 86–104. https://doi.org/10.1080/00273171.2016.1253452
3.
CattellR. B.CattellA. K.CattellH. E. P. (1993). 16PF fifth edition questionnaire. Institute for Personality and Ability Testing.
EdwardsM. C.VeveaJ. L. (2006). An empirical Bayes approach to subscore augmentation: How much strength can we borrow?Journal of Educational and Behavioral Statistics, 31(3), 241–259. https://doi.org/10.3102/10769986031003241
HabermanS. J.SinharayS. (2010). Reporting of subscores using multidimensional item response theory. Psychometrika, 75(2), 209–227. https://doi.org/10.1007/s11336-010-9158-4
8.
HabermanS. J.SinharayS. (2013). Does subgroup membership information lead to better estimation of true subscores?British Journal of Mathematical and Statistical Psychology, 66(3), 452–469. https://doi.org/10.1111/j.2044-8317.2012.02061.x
9.
HaladynaT. M.KramerG. A. (2004). The validity of subscores for a credentialing test. Evaluation & the Health Professions, 27(4), 349–368. https://doi.org/10.1177/0163278704270010
10.
HenleyN. M.MengK.O’BrienD.McCarthyW. J.SockloskieR. J. (1998). Developing a scale to measure the diversity of feminist attitudes. Psychology of Women Quarterly, 22(3), 317–348. https://doi.org/10.1111/j.1471-6402.1998.tb00158.x
11.
HongM.LinL.ChengY. (2021). Asymptotically corrected person fit statistics for multidimensional constructs with simple structure and mixed item types. Psychometrika, 86(2), 464–488. https://doi.org/10.1007/s11336-021-09756-3
12.
LeeK.AshtonM. C. (2004). Psychometric properties of the HEXACO personality inventory. Multivariate Behavioral Research, 39(2), 329–358. https://doi.org/10.1207/s15327906mbr3902_8
13.
MurakiE.CarlsonJ. E. (1995). Full-information factor analysis for polytomous item responses. Applied Psychological Measurement, 19(1), 73–90. https://doi.org/10.1177/014662169501900109
14.
SamejimaF. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika, 34(S1), 1–97. https://doi.org/10.1007/BF03372160
15.
SinharayS. (2010). How often do subscores have added value? Results from operational and simulated data. Journal of Educational Measurement, 47(2), 150–174. https://doi.org/10.1111/j.1745-3984.2010.00106.x
16.
SinharayS. (2019). Added value of subscores and hypothesis testing. Journal of Educational and Behavioral Statistics, 44(1), 25–44. https://doi.org/10.3102/1076998618788862
17.
SinharayS.PuhanG.HabermanS. J. (2010). Reporting diagnostic scores in educational testing: Temptations, pitfalls, and some solutions. Multivariate Behavioral Research, 45(3), 553–573. https://doi.org/10.1080/00273171.2010.483382
18.
WainerH.VeveaJ. L.CamachoF.ReeveB. B.RosaK.NelsonL.SwygertK. A.ThissenD. (2001). Augmented scores—“Borrowing strength” to compute scores based on small numbers of items. In ThissenD.WainerH. (Eds.), Test scoring (pp. 343–387). Lawrence Erlbaum.
