Computerized Adaptive Testing With Item Cloning

Abstract

To increase the number of items available for adaptive testing and reduce the cost of item writing, the use of techniques of item cloning has been proposed. An important consequence of item cloning is possible variability between the item parameters. To deal with this variability, a multilevel item response (IRT) model is presented which allows for differences between the distributions of item parameters of families of item clones. A marginal maximum likelihood and a Bayesian procedure for estimating the hyperparameters are presented. In addition, an item-selection procedure for computerized adaptive testing with item cloning is presented which has the following two stages: First, a family of item clones is selected to be optimal at the estimate of the person parameter. Second, an item is randomly selected from the family for administration. Results from simulation studies based on an item pool from the Law School Admission Test (LSAT) illustrate the accuracy of these item pool calibration and adaptive testing procedures. Index terms: computerized adaptive testing, item cloning, multilevel item response theory, marginal maximum likelihood, Bayesian item selection.

Get full access to this article

View all access options for this article.

References

Albers, W., Does, R. J. M. M., Imbos, Tj., & Janssen, M. P. E. (1989). A stochastic growth model applied to repeated test of academic knowledge. Psychometrika, 54, 451-466.

Bejar, I. I. (1993). A generative approach to psychological and educational measurement. In N. Frederiksen, R. J. Mislevy, & I. I. Bejar (Eds.), Test theory for a new generation of tests (pp. 323-357). Hillsdale, NJ: Lawrence Erlbaum.

Bock, R. D., & Zimowski, M. F. (1997). Multiple group IRT. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 433-448). New York: Springer-Verlag.

Bormuth, J. R. (1970). On the theory of achievement test items. Chicago, IL: University of Chicago Press.

Box, G., & Tiao, G. (1973). Bayesian inference in statistical analysis. Reading, MA: Addison-Wesley.

Bradlow, E. T., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64, 153-168.

Efron, B. (1977). Discussion on maximum likelihood from incomplete data via the EM algorithm (by A. P. Dempster, N. M. Laird and D. B. Rubin). Journal of the Royal Statistical Society (Series B), 39, 1-38.

Enright, M. K., Morley, M., & Sheehan, K. M. (2002). Items by design: The impact of systematic feature variation on item statistical characteristics. Applied Measurement in Education, 15, 49-74.

Glas, C. A. W. (2000). Item calibration and parameter drift. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 183-199). Norwell, MA: Kluwer Academic.

10.

Glas, C. A. W., & van der Linden, W. J. (2001). Modeling variability in item parameters in item response models (Research Report 01-11). Enschede: University of Twente.

11.

Hively, W., Patterson, H. L., & Page, S. H. (1968). A “universe-defined” system of arithmetic achievement items. Journal of Educational Measurement, 5, 275-290.

12.

Janssen, R., Tuerlinckx, F., Meulders, M., & de Boeck, P. (2000). A hierarchical IRT model for criterion-referenced measurement. Journal of Educational and Behavioral Statistics, 25, 285-306.

13.

Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

14.

Louis, T. A. (1982). Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Society, Series B, 44, 226-233.

15.

Macready, G. B. (1983). The use of generalizability theory for assessing relations among items within domains in diagnostic testing. Applied Psychological Measurement, 7, 149-157.

16.

Macready, G. B., & Merwin, J. C. (1973). Homogeneity within item forms in domain referenced testing. Educational and Psychological Measurement, 33, 351-360.

17.

Meisner, R., Luecht, R. M., & Reckase, M. D. (1993). The comparability of the statistical characteristics of test items generated by computer algorithms (ACT Research Report Series Nop. 93-9). Iowa City, IA: ACT, Inc.

18.

Millman, J., & Westman, R. S. (1989). Computer-assisted writing of achievement test items: Toward a future technology. Journal of Educational Measurement, 26, 177-190.

19.

Mislevy, R. J. (1986). Bayes modal estimation in item response models. Psychometrika, 51, 177-195.

20.

Osburn, H. G. (1968). Item sampling for achievement testing. Educational and Psychological Measurements, 28, 95-104.

21.

Roid, G., & Haladyna, T. (1982). A technology for test-item writing. New York: Academic Press.

22.

van der Linden, W. J. (1998). Bayesian item-selection criteria for adaptive testing. Psychometrika, 63, 201-216.

23.

van der Linden, W. J., & Pashley, P. J. (2000). Item selection and ability estimation in adaptive testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 1-25). Norwell, MA: Kluwer Academic.

24.

Wainer, H., Bradlow, E. T., & Du, Z. (2000). Testlet response theory: An analog for the 3PL model useful in testlet-based adaptive testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 245-269). Norwell, MA: Kluwer Academic.