Two mathematical programming approaches are presented for the assembly of ability tests from item pools calibrated under a multidimensional item response theory model. Item selection is based on Fisher information matrix. Several criteria can be used to optimize this matrix. In this article, the A-criterion and the D-criterion are applied. Empirical examples for a two-dimensional mathematics item pool illustrate the methods. Both criteria provided good results. Recommendations are provided about when to apply either approach.
Get full access to this article
View all access options for this article.
References
1.
Ackerman, T. A. (1994). Creating a test information profile for a two-dimensional latent space. Applied Psychological Measurement, 18, 257-275.
2.
Adema, J. J. , Boekkooi-Timminga, E., & van der Linden, W. J. (1991). Achievement test construction using 0-1 linear programming. European Journal of Operational Research, 55, 103-111.
3.
Bazaraa, M. S. , Sherali, H. D., & Shetty, C. M. (1993). Nonlinear programming: Theory and algorithms (2nd ed.). New York: John Wiley.
4.
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 397-479). Reading, MA: Addison-Wesley.
5.
Bock, R. D. , Gibbons, R., & Muraki, E. (1988). Fullinformation item factor analysis. Applied Psychological Measurement, 12, 261-280.
6.
Fraser, C. , & McDonald, R. P. (1988). NOHARM II: A FORTRAN program for fitting unidimensional and multidimensional normal ogive models of latent trait theory. Armidale, Australia: University of New England, Centre for Behavioral Studies.
7.
Lehmann, E. L. (1983). Theory of point estimation. New York: John Wiley.
8.
Lord, F. M. (1980). Application of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
9.
Luecht, R. M. (1996). Multidimensional computerized adaptive testing in a certification or licensure context. Applied Psychological Measurement, 20, 389-404.
10.
McDonald, R. P. (1981). The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology, 34, 100-117.
11.
McKinley, R. L. , & Reckase, M. N. (1983). An extension of the two-parameter logistic model to the multidimensional latent space (Research Rep. ONR 83-2). Iowa City, IA: American College Testing.
12.
Reckase, M. D. (1979). Unifactor latent trait models applied to multi-factor tests: Results and implications. Journal of Educational Statistics, 4, 207-230.
13.
Reckase, M. D. (1985). The difficulty of test items that measure more than one dimension. Applied Psychological Measurement, 9, 401-412.
14.
Segall, D. O. (1996). Multidimensional adaptive testing. Psychometrika, 61, 331-354.
15.
Stout, W. F. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52, 589-617.
16.
Theunissen, T.J.J.M. (1985). Binary programming and test design. Psychometrika, 50, 411-420.
17.
Timminga, E. , van der Linden, W. J., & Schweizer, D. A. (1996). ConTEST 2.0: A decision support system for item banking and optimal test assembly (Computer program and manual). Groningen, The Netherlands: iec ProGAMMA.
18.
van der Linden, W. J. (1994). Optimum design in item response theory: Test assembly and item calibration. In G. H. Fisher & D. LamingContributions to mathematical psychology, psychometrics, and methodology (pp. 305-318). New York: SpringerVerlag.
19.
van der Linden, W. J. (1996). Assembling tests for the measurement of multiple traits. Applied Psychological Measurement, 20, 373-388.
20.
van der Linden, W. J. (1998). Optimal assembly of psychological and educational tests. Applied Psychological Measurement, 22, 195-211.
21.
van der Linden, W. J. (1999). Multidimensional adaptive testing with a minimum error variance criterium. Journal of Educational and Behavioral Statistics, 24, 398-412.
22.
van der Linden, W. J. , & Boekkooi-Timminga, E. (1989). A maximin model for test design with practical constraints. Psychometrika, 54, 237-247.
23.
Wainer, H. , Dorans, N. J., Flaugher, R., Green, B. F., Mislevy, R. J., Steinberg, L., & Tissen, D. (1990). Computerized adaptive testing: A primer. Hillsdale, NJ: Lawrence Erlbaum.
24.
Wilson, D. , Wood, R., & Gibbons, R. (1987). TESTFACT. Mooresville, IN: Scientific Software.
25.
Yen, W. M. (1983). Use of the three-parameter model in the development of standardized achievement test. In R. K. Hambleton (Ed.), Applications of item response theory (pp. 123-141). Vancouver, BC: Educational Research Institute of British Columbia.
