Assessing Goodness of Fit in Item Response Theory With Nonparametric Models

Abstract

The distance between nonparametric and parametric item characteristic curves has been proposed as an index of goodness of fit in item response theory in the form of a root integrated squared error index. This article proposes to use the posterior distribution of the latent trait as the nonparametric model and compares the performance of an index based on this method with another approach based on the kernel-smoothing model. Error rates and power are evaluated using the two-parameter logistic model and three types of realistic misfitted items. Results show that for fitting items, the distance between parametric and nonparametric item characteristic curves decreased as the sample size increased for both procedures. Kernel-smoothing root integrated squared error also decreased as test length increased. Bootstrap methods are used to obtain a significance test. Both procedures performed adequately in terms of Type I error rates. Regarding power, the posterior probabilities method was superior, especially in small samples, although in short tests both procedures performed in a similar way.