A Comparison of the Nominal Response Model and the Three-Parameter Logistic Model in Computerized Adaptive Testing

Abstract

It is intuitive to presume that when an examinee incorrectly answers a test item that he or she may possess partial knowledge of the item's correct answer. However, current use of dichotomous item response theory (IRT) models in IRT-based computerized adaptive testing (CAT) ignore this partial information in their ability estimation. In this study a (polychotomous) nominal response model-based CAT (NR CAT) was simulated using an empirical data set. The ability estimation of NR CAT as well as its overall performance was compared to that of a dichotomous three-parameter logistic model-based CAT (3PL CAT). Results showed that the NR CAT's ability estimates were highly correlated with an external criterion and with those of the 3PL CAT. Both CATs showed only a slight bias in their ability estimates for that region of the ability continuum where their respective model did not provide a great deal of information. Despite the positively skewed information function afforded by the nominal response model the NR CAT was able to provide relatively accurate ability estimates in the upper 6 range, and its convergence rate was superior to that of the 3PL CAT. The implications of a nominal response model-based CAT are discussed.