Sample Size and the Recovery of Nominal Response Model Item Parameters

Abstract

In this study of polytomous items, the number of items and categories per item were varied to explore the effects on estimation of item parameters in the nominal response model. De Ayala and Sava-Bolesta's (1999) work suggested that the ratio of the sample size to the total number of item parameters was a key factor. They varied the total number of item parameters by increasing the sample size or changing the number of categories per item while leaving the number of items constant. In this study, the total number of item parameters, the sample size, and the number of categories were manipulated as separate factors. Increasing the number of items had little effect on item parameter recovery, but increasing the number of categories increased the error variance of the parameter estimates. Error variance was also greater for more highly discriminating items and for skewed distributions of ability. Index terms: nominal response, parameter recovery, polytomous IRT, MULTILOG.

Get full access to this article

View all access options for this article.

References

Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.

Childs, R. A., & Chen, W.-H. (1999). Software note: Obtaining comparable item parameter estimates in MULTILOG and PARSCALE for two polytomous IRT models. Applied Psychological Measurement, 21, 89-90.

Choi, S. W., Cook, K. F., & Dodd, B. G. (1997). Parameter recovery for the partial credit model using MULTILOG. Journal of Outcome Measurement, 1, 114-142.

De Ayala, R. J., & Sava-Bolesta, M. (1999). Item parameter recovery for the nominal response model. Applied Psychological Measurement, 23, 3-19.

Fitzpatrick, A. R., Link, V. B., Yen, W. M., Burket, G. R., Ito, K., & Sykes, R. C. (1996). Scaling performance assessments: A comparison of one-parameter and two-parameter partial credit models. Journal of Educational Measurement, 33, 291-314.

Keppel, G. (1982). Design and analysis: A researcher's handbook (2nd ed.). Englewood Cliffs, NJ: Prentice Hall.

Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.

Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159-176.

Reise, S. P., & Yu, J. (1990). Parameter recovery in the graded response model using MULTILOG. Journal of Educational Measurement, 27, 133-144.

10.

Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplements, 17.

11.

Thissen, D. (1991). MULTILOG: Multiple, categorical item analysis and test scoring using item response theory [Computer software and manual]. Chicago: Scientific Software.

12.

Thissen, D. (n.d.). MULTILOG FAQs. Retrieved February 5, 2003, from the Scientific Software International Web site: http://www.ssicentral.com/irt/thissen.htm

13.

Thissen, D., & Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51, 567-577.