A method based on 0-1 linear programming (LP) is presented to stratify an item pool optimally for use in α-stratified adaptive testing. Because the 0-1 LP model belongs to the subclass of models with a network flow structure, efficient solutions are possible. The method is applied to a previous item pool from the computerized adaptive testing (CAT) version of the Graduate Record Exams (GRE) Quantitative Test. The results indicate that the new method performs well in practical situations. It improves item exposure control, reduces the mean squared error in the θ estimates, and increases test reliability. Index terms: computerized adaptive testing, α-stratified adaptive testing, item pool stratification, 0-1 linear programming
Get full access to this article
View all access options for this article.
References
1.
Armstrong, R. D., Jones, D. H., & Wang, Z. (1995). Network optimization in constrained standardized test construction. In K. D. Lawrence (Ed.), Applications of management science: Network optimization applications (Vol. 8, pp. 189-212). Greenwich, CT: JAI.
2.
Ban, J., Wang, T., & Yi, Q. (1999, June). Comparison of the astratified method, the Sympson-Hetter method, and the matched difficulty method in CAT administration. Paper presented at the Annual meeting of the Psychometric Society, Lawrence, KS.
3.
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
4.
Chang, H., Qian, J., & Ying, Z. (2001). A-stratified multistage CAT with b-blocking. Applied Psychological Measurement, 25, 333-341.
5.
Chang, H., & Ying, Z. (1996). Global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213-229.
Davey, T., & Parshall, C. (1995, April). New algorithms for item selection and exposure control with computerized adaptive testing. Paper presented at the Annual meeting of the American Educational Research Association, San Francisco.
8.
ILOG, Inc. (1999). CPLEX 6.5 [Computer program and manual]. Incline Village, NV: Author.
9.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
10.
Lord, F. M., & Wingersky, M. S. (1984). An investigation of methods for reducing sampling error in certain IRT procedures. Applied Psychological Measurement, 8, 347-364.
11.
Lueng, C.-K., Chang, H.-H., & Hau, K.-T. (1999, April). An enhanced stratified computer adaptive testing design. Paper presented at the Annual meeting of the American Educational, Research Association, Montreal, Canada.
12.
McBride, J. R., & Martin, J. T. (1983). Reliability and validity of adaptive ability tests in a military setting. In D. J. Weiss (Ed.), New horizons in testing (pp. 223-226). New York: Academic Press.
13.
Nemhauser, G., & Wolsey, L. (1988). Integer and combinatorial optimization. New York: John Wiley.
14.
Parshall, C., Hogarty, K., & Kromrey, J. (1999, June). Item exposure in adaptive tests: An empirical investigation of control strategies. Paper presented at the Annual meeting of the Psychometric Society, Lawrence, KS.
15.
Stocking, M. L. (1998). A framework for comparing adaptive test designs. Unpublished manuscript.
16.
Stocking, M. L., & Lewis, C. (1998). Controlling item exposure conditional on ability in computerized adaptive testing. Journal of Educational and Behavioral Statistics, 23, 57-75.
17.
Stocking, M. L., & Lewis, C. (2000). Methods of controlling the exposure of items in CAT. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 163-182). Boston: Kluwer.
18.
Sympson, J. B., & Hetter, R. D. (1985, October). Controlling item-exposure rates in computerized adaptive testing. In Proceedings of the 27th Annual Meeting of the Military Testing Association (pp. 973-977). San Diego, CA: Navy Personnel Research and Development Center.
19.
Thomasson, G. L. (1995, June). New item exposure control algorithms for computerized adaptive testing. Paper presented at the Annual meeting of the Psychometric Society, Minneapolis, MN.
20.
van der Linden, W. J. (1998a). Bayesian item selection criteria for adaptive testing. Psychometrika, 63, 201-216.
21.
van der Linden, W. J. (1998b). Optimal test assembly of psychological and educational tests. Applied Psychological Measurement, 22, 195-211.
22.
van der Linden, W. J. (2000). Constrained adaptive testing with shadow tests. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 27-52). Boston: Kluwer.
23.
van der Linden, W. J. (2002). Modifications of the Sympson-Hetter method for item-exposure control in computerized adaptive testing. Manuscript submitted for publication.
24.
van der Linden, W.J., & Chang, H.-H. (2003). Alphastratified adaptive testing with large numbers of content constraints. Applied Psychological Measurement, 27, 107-120.
25.
van der Linden, W. J., Scrams, D. J., & Schnipke, D. L. (1999). Using response-time constraints to control for speededness in computerized adaptive testing. Applied Psychological Measurement, 23, 195-210.
26.
Wainer, H. (1990). Computerized adaptive testing: A primer. Hillsdale, NJ: Lawrence Erlbaum.
27.
Weiss, D.J. (1976). Adaptive testing research in Minnesota: Overview, recent results, and future directions. In C. L. Clark (Ed.), Proceedings of the First conference on computerized adaptive testing (pp. 24-35). Washington, DC: U.S. Civil Service Commission.
