Sage Journals: Discover world-class research

Abstract

DETECT is a nonparametric methodology to identify the dimensional structure underlying test data. The associated DETECT index, D_max , denotes the degree of multidimensionality in data. Conditional covariances (CCOV) are the building blocks of this index. In specifying population CCOVs, the latent test composite θ_TT is used as the conditional variable. In estimating the CCOVs, the total test score of all items in the test (T) or the rest score of remaining items (S) are generally used as estimates of the latent composite θ_TT . However, estimated CCOVs are biased when using T or S as a proxy for θ_TT. Some type of correction is needed to adjust this bias. This study was an investigation of different ways to estimate the DETECT index based on the conditional scores T and S, and additional bias adjustments, resulting in six different estimates, D ₁ through D ₆. These six indices were investigated in simulated settings, varying the test length, sample size, and the degree of multidimensionality (108 in all). The results showed that indices D ₁, D ₂, and D ₅ are not acceptable as they displayed highly inflated D _max values. No statistically significant differences were found between indices D ₃ and D ₆. Overall comparison of indices D ₃ with D ₄ showed that even though they differed significantly in D _max values, there was no practically meaningful difference between the performance of these two indices. For these reasons, it is recommended that the current index D ₃ be retained even though index D ₄ displayed slightly better results in the current study.

Keywords

DETECT multidimensionality assessment conditional covariances

Get full access to this article

View all access options for this article.

References

Ackerman, T. ( 1996). Graphical representation of multidimensional item response theory. Applied Psychological Measurement, 20, 311-329.

Ackerman, T.A. ( 1994). Using multidimensional item response theory to understand what items and tests are measuring. Applied Measurement in Education, 7, 255-278.

Finch, H. , & Habing, B. ( 2005). Comparison of NOHARM and DETECT in item cluster recovery: Counting dimensions and allocating items. Journal of Educational Measurement, 42, 149-169.

Gierl, M.J. , Leighton, J.P. , & Tan, X. ( 2005, April). Evaluating DETECT classification accuracy and consistency when data display complex structure. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Quebec, Canada.

Holland, P.W. , & Rosenbaum, P.R. (1986). Conditional association and unidimensionality in monotone latent variable models. Annals of Statistics, 14, 1523-1543.

Junker, B. ( 1993). Conditional association, essential independence and monotone unidimensional item response models. Annals of Statistics , 21, 1359-1378.

Kim, H.R. ( 1994). New techniques for the dimensionality assessment of standardized test data (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign.

Nandakumar, R. ( 1991). Traditional dimensionality versus essential dimensionality . Journal of Educational Measurement, 28, 99-117.

Reckase, M.D. ( 1997). A linear logistic multidimensional model for dichotomous item response data. In W. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 271-286). New York, NY: Springer-Verlag.

10.

Reckase, M.D. ( 1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401-412.

11.

Reckase, M.D. , & McKinley, R.L. (1983, April). The definition of difficulty and discrimination for multidimensional item response theory models . Paper presented at the Annual Meeting of American Educational Research Association, Montreal, Quebec, Canada.

12.

Reckase. M. D. and McKinley, R.L. ( 1991). The discriminating power of items that measure more than one dimension. Applied Psychological Measurement, 14, 361-373.

13.

Rosenbaum, P.R. ( 1984). Testing the conditional independence and monotonicity assumptions of item response theory. Psychometrika, 49, 425-435.

14.

Roussos, L.A. , & Ozbek, O. ( 2006). Formulation of the DETECT population parameter and evaluation of DETECT estimator bias. Journal of Educational Measurement , 43, 215-243.

15.

Roussos, L.A. , Stout, W.F. , & Marden, J.I. ( 1993, April). Dimensional and structural analysis of standardized tests using DIMTEST with hierarchical cluster analysis. Paper presented at the annual NCME meeting, Atlanta, GA .

16.

Stout, W. ( 1987). A nonparametric approach for assessing latent trait unidimensionality . Psychometrika, 52, 589-617.

17.

Stout, W. , Habing, B. , Douglas, J. , Kim, H.R. , Roussos, L. , & Zhang, J. ( 1996). Conditional covariance-based nonparametric multidimensionality assessment. Applied Psychological Measurement, 20, 331-354.

18.

Zhang, J. , & Stout, W.F. ( 1999a). Conditional covariance structure of generalized compensatory multidimensional items. Psychometrika, 64, 129-152.

19.

Zhang, J. , & Stout, W.F. ( 1999b). The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika , 64, 213-249.

A Comparison of Bias Correction Adjustments for the DETECT Procedure

Abstract

Keywords

Get full access to this article

References