This study examined the effect of dimensionality on the ability estimation of an adaptive test. Two-dimensional data sets were generated which differed from one another in the interdimensional ability association, the correlation among the difficulty parameters, and the presence or absence of confounding of item discriminations with item difficulty. The generated data were used for Bayesian computerized adaptive testing (CAT) simulations (three parameter logistic model), and the CAT ability estimates were compared with the simulees known abilities (θTs). Results show that the dimensionality of the response data shifts the focus for the minimization of measurement errors from OT (with unidimensional data) to the average of the latent abilities (with bidimensional data).
Get full access to this article
View all access options for this article.
References
1.
Ackerman, T. A. (1989). Unidimensional IRT calibration of compensatory and noncompensatory multidimensional items. Applied Psychological Measurement, 13, 113-127.
2.
Ansley, T. N. and Forsyth, R. A. (1985). An examination of the characteristics of unidimensional IRT parameter estimates derived from two-dimensional data. Applied Psychological Measurement, 9, 37-48.
3.
Doody-Bogan, E. and Yen, W. M. (1983, April). Detecting multidimensionality and examining its effects on vertical equating with the three-parameter logistic model. Paper presented at the annual meeting of the American Educational Research Association, Montreal.
4.
Hoffman, P. J. (1959). Generating variables with arbitrary properties. Psychometrika, 24, 265-267.
5.
Hulin, C. L. , Drasgow, F., and Parsons, C. K. (1983). Item response theory: Application to psychological measurement. Homewood, IL: Dow Jones-Irwin.
6.
Jensema, C. J. (1974). The validity of Bayesian tailored testing. Educational and Psychological Measurement, 34, 757-766.
7.
Kingsbury, G. G. and Houser, R. L. (1988, April). A comparison of achievement level estimates from computerized adaptive testing and paper-and-pencil testing. Paper presented at the annual meeting of the American Educational Research Association, New Orleans.
8.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
9.
McBride, J. R. (1977). Some properties of a Bayesian adaptive testing strategy. Applied Psychological Measurement, 1, 121-140.
10.
McBride, J. R. and Martin, J. T. (1983). Reliability and validity of adaptive ability tests in a military setting. In D. J. Weiss (Ed.), New Horizons in Testing (pp. 223-237). New York: Academic.
11.
McKinley, R. and Reckase, M. D. (1983). An extension of the two-parameter logistic model to the multidimensional latent space (Research Report 0NR83-2). Iowa City, IA: American College Testing Program.
12.
Mislevy, R. J. and Bock, R. D. (1982). BILOG, maximum likelihood item analysis and test scoring: Logistic model. Mooresville, IN: Scientific Software.
13.
Patience, W. M. and Reckase, M. D. (1980). Effects of program parameters and item pool characteristics on the bias of a three-parameter tailored testing procedure. Paper presented at the annual meeting of the National Council on Measurement in Education, Boston.
14.
Reckase, M. D. , Ackerman, T. A., and Carlson, J. E. (1988). Building a unidimensional test using multidimensional items. Journal of Educational Measurement, 25, 193-203.
15.
Reckase, M. D. , Carlson, J. E., Ackerman, T. A. and Spray, J. A. (1986, June). The interpretation of unidimensional IRT parameters when estimated from multidimensional data. Paper presented at the annual meeting of Psychometric Society, Toronto.
16.
Reckase, M. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational Statistics, 4, 207-230.
17.
Sympson, J. B. (1978). A model for testing with multidimensional items. In D. J. Weiss (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference (pp. 82-98). Minneapolis: University of Minnesota, Psychometric Methods Program, Department of Psychology.
18.
Urry, V. W. (1977). Tailored testing: A Successful application of latent trait theory. Journal of Educational Measurement, 14. 181-196.
19.
Way, W. D. , Ansley, T. N., and Forsyth, R. A. (1988). The comparative effects of compensatory and noncompensatory two-dimensional data on unidimensional IRT estimates. Applied Psychological Measurement, 12, 239-252.
20.
Weiss, D. J. (1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6, 473-492.
21.
Weiss, D. J. and Kingsbury, G. G. (1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21, 361-375.
22.
Wingersky, M. S. , Barton, M. A., and Lord, F. M. (1982). LOGIST user's guide. Princeton, NJ: Educational Testing Service.
23.
Yen, W. M. (1985). Increasing item complexity: A possible cause of scale shrinkage for unidimensional item response theory. Psychometrika, 50, 399-410.
