A simulation study was conducted to investigate the application of expected a posteriori (EAP) trait estimation in computerized adaptive tests (CAT) based on the partial credit model and compare it with maximum likelihood trait estimation (MLE). The performance of EAP was evaluated under different conditions: the number of quadrature points (10, 20,40, and 80) and the type of prior distribution (normal and uniform). The relative performance of MLE and the EAP estimation methods was assessed under two distributional forms of the latent trait (normal and negatively skewed). Results showed that, regardless of the latent trait distribution, MLE and EAP with a normal prior or a uniform prior using either 20, 40, or 80 quadrature points provided relatively accurate estimation in CAT based on the partial credit model. Also, increasing the number of quadrature points from 20 to 80 did not increase the accuracy of EAP estimation.
Get full access to this article
View all access options for this article.
References
1.
Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573.
2.
Baker, F. B. (1992). Item response theory: Parameter estimation techniques. New York: Marcel Dekker.
3.
Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.
4.
Bock, R. D. , & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.
5.
Chen, S. , Hou, L., Fitzpatrick, S. J., & Dodd, B. G. (1997). The effect of population distribution and methods of theta estimation on CAT using the rating scale model. Educational and Psychological Measurement, 57, 422-439.
6.
De Ayala, R. J. (1989). A comparison of the nominal response model and the three-parameter logistic model in computerized adaptive testing. Educational and Psychological Measurement, 49, 789-805.
7.
De Ayala, R. J. (1992). The nominal response model in computerized adaptive testing. Applied Psychological Measurement, 16, 327-343.
8.
De Ayala, R. J. , Schafer, W. D., & Sava-Bolesta, M. (1994, April). An investigation of the standard errors of expected a posteriori ability estimates. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
9.
Dodd, B. G. (1990). The effect of item selection procedure and stepsize on computerized adaptive attitude measurement using the rating scale model. Applied Psychological Measurement, 14, 355-366.
10.
Dodd, B. G. , & De Ayala, R. J. (1994). Item information as a function of threshold values in the rating scale model. In M. Wilson (Ed.), Objective measurement: Theory into practice (Vol. 2, pp. 301-317). Norwood, NJ: Ablex.
11.
Dodd, B. G. , & Koch, W. R. (1987). Effects variations in item step values on item and test information in the partial credit model. Applied Psychological Measurement, 11, 371-384.
12.
Dodd, B. G. , Koch, W. R., & De Ayala, R. J. (1989). Operational characteristics of adaptive testing procedures using the graded response model. Applied Psychological Measurement, 13, 129-143.
13.
Dodd, B. G. , Koch, W. R., & De Ayala, R. J. (1993). Computerized adaptive testing using the partial credit model: Effects of item pool characteristics and different stopping rules. Educational and Psychological Measurement, 53, 61-77.
14.
Hambleton, R. K. , & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer Nijhoff.
15.
Hildebrand, F. B. (1956). Introduction to numerical analysis. New York: McGraw-Hill.
16.
Hou, L. , Chen, S., Dodd, B. G., & Fitzpatrick, S. J. (1996, April). The effect of methods of theta estimation, prior distribution, and number of quadrature points on CAT using the graded response model. Paper presented at the annual meeting of the American Educational Research Association, New York.
17.
Kim, J. K. , & Nicewander, W. A. (1993). Ability estimation for conventional tests. Psychometrika, 58, 587-599. CHEN ET AL.
18.
Koch, W. R. , & Dodd, B. G. (1985, April). Computerized adaptive attitude measurement. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
19.
Koch, W. R. , & Dodd, B. G. (1989). An investigation of procedures for computerized adaptive testing using partial credit scoring. Applied Measurement in Education, 2, 335-357.
20.
Koch, W. R. , & Dodd, B. G. (1995). An investigation of procedures for computerized adaptive testing using the successive intervals Rasch model. Educational and Psychological Measurement, 55, 976-990.
21.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
22.
Lord, F. M. (1983). Unbiased estimators of ability parameters, of their variance, and of their parallel-form reliability. Psychometrika, 48, 233-245.
23.
Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
24.
Mislevy, R. J. , & Stocking, M. L. (1989). A consumer's guide to LOGIST and BILOG. Applied Psychological Measurement, 13, 57-75.
25.
Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159-176.
26.
Rost, J. (1988). Measuring attitudes with a threshold model drawing on a traditional scaling concept. Applied Psychological Measurement, 12, 397-409.
27.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, No. 17.
28.
Seong, T. (1990). Sensitivity of marginal maximum likelihood estimation of item and ability parameters to the characteristics of the prior ability distributions. Applied Psychological Measurement, 14, 299-311.
29.
Thissen, D. , & Mislevy, R. J. (1990). Testing algorithms. In H. Wainer, N. J. Dorans, R. Flaugher, B. F. Green, R. J. Mislevy, L. Steinburg, & D. Thissen (Eds.), Computerized adaptive testing: A primer (pp. 103-135). Hillsdale, NJ: Lawrence Erlbaum.
30.
Thissen, D. , & Steinberg, L. (1986). Taxonomy of item response models. Psychometrika, 51, 567-577.
31.
Wainer, H. , & Thissen, D. (1987). Estimating ability with the wrong model. Journal of Educational Statistics, 12, 339-368.
32.
Weiss, D. J. (1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6, 473-492.
