The Comparative Effects of Compensatory and Noncompensatory Two-Dimensional Data on Unidimensional IRT Estimates

Abstract

This study compared the effects of using a unidi mensional IRT model with two-dimensional data gener ated by noncompensatory and compensatory multidi mensional IRT models. Within each model, simulated datasets differed according to the degree of correlation between two vectors of θ parameters, ranging from 0 to .95. Results showed that the number-correct distri butions for each group of datasets were generally comparable, although factor analyses of tetrachoric correlations suggested that differences existed in the structure of the data from the two models. For the uni dimensional parameter estimates, it was found that the â values from the noncompensatory model appeared to be averages of the a ₁ and a ₂ values, while the â values from the compensatory model were best considered as an estimate of the sum of the a ₁ and a ₂ values. Con versely, the b values for the noncompensatory data were consistently greater than the b ₁ values, while the b values from the compensatory model were best con sidered as the average of the b ₁ and b ₂ values. For both models the θ estimates were most highly related to the average of the two θ parameters. However, for the noncompensatory model there was a general in crease in the strength of this relationship with in creases in ρ(θ₁,θ ₂). For the compensatory model, the strength of this relationship did not show a great deal of change with differences in ρ(θ₁,θ ₂). Index terms: Compensatory multidimensional IRT models, Item response theory, Multidimensional IRT models, Noncompensatory multidimensional IRT models, Pa rameter estimation, Violations of unidimensionality.

Get full access to this article

View all access options for this article.

References

Ansley, T.N. , & Forsyth, R.A. (1985a). An examination of the characteristics of unidimensional IRT parameter estimates derived from two-dimensional data. Applied Psychological Measurement, 9, 37-48.

Ansley, T.N. , & Forsyth, R.A. (1985b). A feasibility study of compensatory and noncompensatory models in a mathematics problem solving framework. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago.

Doody, E.N. (1985). Examining the effects of multidimensional data on ability and item parameter estimation using the three parameter logistic model. Paper presented at the annual meeting of the American Educational Research Association, Chicago .

Doody-Bogan, E.N. , & Yen, W.M. (1983). Detecting multidimensionality and examining its effects on vertical equating with the three parameter logistic model. Paper presented at the annual meeting of the American Educational Research Association, Montreal.

Drasgow, F. , & Parsons, C.K. (1983). Application of unidimensional item response theory models to multidimensional data. Applied Psychological Measurement , 7, 189-199.

Hambleton, R.K. , & Cook, L.L. ( 1977). Latent trait models and their use in the analysis of educational test data. Journal of Educational Measurement , 14, 75-96.

Hattie, J. (1981). Decision criteria for determining unidimensionality . Unpublished doctoral dissertation, University of Toronto .

Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 139-164.

Lord, F.M. , & Novick, M.R. (1968). Statistical theories of mental test scores. Reading MA: Addison-Wesley.

10.

Loyd, B.H. , & Hoover, H.D. (1980). Vertical equating using the Rasch model. Journal of Educational Measurement, 17, 179-193.

11.

McKinley, R.L. , & Reckase, M.D. (1983). An extension of the two-parameter logistic model to the multidimensional latent space (Research Report ONR832). Iowa City IA: American College Testing Program.

12.

Reckase, M.D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational Statistics , 4, 207-230.

13.

Reckase, M.D. , & McKinley, R.L. (1985). Some latent trait theory in a multidimensional latent space. In D. J. Weiss (Ed.), Proceedings of the 1982 Item Response Theory and Computerized Adaptive Testing Conference. Minneapolis: University of Minnesota, Department of Psychology, Computerized Adaptive Testing Laboratory.

14.

Sympson, J.B. (1978). A model for testing with multidimensional items . In D. J. Weiss (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference . Minneapolis: University of Minnesota, Department of Psychology, Computerized Adaptive Testing Laboratory.

15.

Wang, M. (1987). Estimation of ability parameters from response data to items that are precalibrated with a unidimensional model. Paper presented at the annual meeting of the American Educational Research Association, Washington DC.

16.

Wingersky, M.S. , Barton, M.A. , & Lord, F.M. (1982). LOGIST user's guide. Princeton NJ: Educational Testing Service.

17.

Yen, W.M. (1985). Increasing item complexity: A possible cause of scale shrinkage for unidimensional item response theory. Psychometrika, 50, 399-410.