This study presents a latent (item response theory–like) framework of a recently developed classical approach to test scoring, equating, and item analysis, referred to as D-scoring method. Specifically, (a) person and item parameters are estimated under an item response function model on the D-scale (from 0 to 1) using marginal maximum-likelihood estimation and (b) analytic expressions are provided for item information function, test information function, and standard error of estimation for D-scores obtained under the proposed latent treatment of the D-scoring method. The results from a simulation study reveal very good recovery of item and person parameters via the marginal maximum-likelihood estimation method. Discussion and recommendations for practice are provided.
AtanasovD. V.DimitrovD. M. (2018). DELTA: A computer program for test scoring, equating, and item analysis under the D-scoring method. National Center for Assessment (Qiyas).
BechgerT. M.MarisG.VerstralenH. F. M.BeguinA. A. (2003). Using classical test theory in combination with item response theory. Applied Psychological Measurement, 27(5), 319-334. https://doi.org/10.1177/0146621603257518
4.
BirnbaumA. (1968). Some latent trait models and their use in inferring an examinee’s ability. In LordF. M.NovickM. R. (Eds.), Statistical theories of mental test scores (pp. 17-20). Addison Wesley.
5.
CaiL.ThissenD.du ToitS. H. C. (2011). IRTPRO 4.20 for Windows: Item response theory for patient-reported outcomes [Computer software]. Scientific Software International.
6.
DaveyT.HendricksonA. (2010, May). Classical versus IRT statistical test specifications for building test forms[Paper presentation]. Annual Conference of the National Council for Measurement in Education Denver, CO, United States.
7.
DeMarsC. (2008, April). Scoring multiple choice items: A comparison of IRT and classical polytomous and dichotomous methods[Paper presentation]. Annual meeting of the National Council on Measurement in Education, New York, NY, United States. https://www.jmu.edu/assessment/CEDNCMEPaper08.pdf
8.
DimitrovD. M. (2016). An approach to scoring and equating tests with binary items: Piloting with large-scale assessments. Educational and Psychological Measurement, 76(6), 954-975. https://doi.org/10.1177/0013164416631100
9.
DimitrovD. M. (2017). The delta scoring method of tests with binary items: A note on true score estimation and equating. Educational and Psychological Measurement, 78(5), 805-825. https://doi.org/10.1177/0013164417724187
10.
DimitrovD. M. (2019). Modeling of item response functions under the D-scoring method. Educational and Psychological Measurement, 80(1), 126-144. https://doi.org/10.1177/0013164419854176
11.
DimitrovD. M.LuoY. (2018). A note on the D-scoring method adapted for polytomous test items. Educational and Psychological Measurement, 79(3), 545-557. https://doi.org/10.1177/0013164418786014
12.
DomingueB. W.DimitrovD. M. (2015). A comparison of IRT theta estimates and delta scores from the perspective of additive conjoint measurement (Research Report, RR-4-2015). National Center for Assessment.
13.
DrasgowF. (1989). An evaluation of marginal maximum likelihood estimation for the two-parameter logistic model. Applied Psychological Measurement, 13(1), 77-90. https://doi.org/10.1177/014662168901300108
FanX. (1998). Item response theory and classical test theory: An empirical comparison of their item/person statistics. Educational and Psychological Measurement, 58(3), 357-385. https://doi.org/10.1177/0013164498058003001
16.
HambletonR. K.SwaminathanH.RogersH. J. (1991). Fundamentals of item response theory. Sage.
17.
HanK. T.DimitrovD. M.Al-MashariF. (2019). Developing multistage tests using D-scoring method. Educational and Psychological Measurement, 79(5), 988-1008. https://doi.org/10.1177/0013164419841428
18.
HarwellM. R.BakerF. B.ZwartM. (1988). Item parameter estimation via marginal maximum likelihood and an EM algorithm: A didactic. Journal of Educational Statistics, 13(3), 243-271. https://doi.org/10.3102/10769986013003243
19.
KohliN.KoranJ.HennL. (2015). Relationships among classical test theory and item response theory frameworks via factor analytic models. Educational and Psychological Measurement, 75(3), 389-405. https://doi.org/10.1177/0013164414559071
LordF. M. (1980). Application of item response theory to practical testing problems. Lawrence Erlbaum.
22.
MacDonaldP.PaunonenS. (2002). A Monte Carlo comparison of item and person statistics based on item response theory versus classical test theory. Educational and Psychological Measurement, 62(6), 921-943. https://doi.org/10.1177/0013164402238082
OswaldF. L.ShawA.FarmerW. L. (2015). Comparing simple scoring with IRT scoring of personality measures: The Navy Computer Adaptive Personality Scales. Applied Psychological Measurement, 39(2), 144-154. https://doi.org/10.1177/0146621614559517
25.
RaykovT.DimitrovD. M.MarcoulidesG. A.HarrisonM. (2019). On the connections between item response theory and classical test theory: A note on true score evaluation for polytomous items via item response modeling. Educational and Psychological Measurement, 79(6), 1198-1202. https://doi.org/10.1177/0013164417745949
26.
RaykovT.MarcoulidesG. A. (2015). On the relationship between classical test theory and item response theory: From one to the other and back. Educational and Psychological Measurement, 76(2), 325-338. https://doi.org/10.1177/0013164415576958
