Educational and psychological tests are often composed of multiple short subtests, each measuring a distinct latent trait. Unfortunately, short subtests suffer from low measurement precision, which makes the bandwidth—fidelity dilemma inevitable. In this study, the authors demonstrate how a multidimensional Rasch analysis can be employed to take into account the information about the correlation between latent traits such that the precision of each subtest measure can be improved and the correlation between latent traits can be accurately estimated. A real data set of the 13-scale Thinking Styles Inventory was analyzed with the traditional unidimensional approach and the multidimensional approach. The results demonstrate that in contrast to the unidimensional approach, the multidimensional approach yields a much higher level of measurement precision and a more appropriate estimate for the correlation between thinking styles. In conclusion, even short subtests can yield highly precise measures such that the bandwidth—fidelity dilemma is resolved.
Adams, R.J., Wilson, M., & Wang, W.-C. (1997). The multidimensional random coefficients multinomial logit model.Applied Psychological Measurement, 21, 1-23.
2.
Adams, R.J., Wilson, M.R., & Wu, M.L. (1997). Multilevel item response models: An approach to errors in variables regression. Journal of Educational and Behavioral Statistics, 22, 47-76.
3.
Andersen, E.B., & Madsen, M. (1977). Estimating the parameters of the latent population distribution. Psychometrika, 42, 357-374.
4.
Andrich, D. (1978). A rating formulation for ordered response categories . Psychometrika, 43, 561-573.
5.
Bock, R.D., & Mislevy, R.J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.
6.
Bock, R.D., Thissen, D., & Zimowski, M.F. (1997). IRT estimation of domain scores. Journal of Educational Measurement, 34, 197-211.
7.
Camilli, G. (1988). Scale shrinkage and the estimation of latent distribution parameters. Journal of Educational Statistics, 13, 227-241.
8.
Cronbach, L.J., & Gleser, G. (1965).Psychological tests and personnel decision (2nd ed.). Urbana: University of Illinois Press.
9.
Holland, P. W., & Wainer, H. (Eds.). (1993). Differential item functioning . Hillsdale, NJ: Lawrence Erlbaum .
10.
Kahraman, N., & Kamata, A. (2004). Increasing the precision of subscale scores by using out-of-scale information. Applied Psychological Measurement , 28, 407-426.
11.
Kelderman, H. (1996). Multidimensional Rasch models for partial-credit scoring. Applied Psychological Measurement, 20, 155-168.
12.
Masters, G.N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
Mislevy, R.J. (1987). Exploiting auxiliary information about examinees in the estimation of item parameters. Applied Psychological Measurement, 11, 81-91.
15.
Organization for Economic Co-operation and Development. (2005). The PISA 2003 technical report. Paris: Author.
16.
Ones, D.S., & Viswesvaran, C. (1996). Bandwidth-fidelity dilemma in personality measurement for personnel selection.Journal of Organizational Behavior, 17, 609-626.
17.
Ormel, J., Oldehinkel, T., Brilman, E., & Van den Brink, W. (1993). Outcome of depression and anxiety in primary care: A three wave, 2 1/2 year study of psychopathology and disability. Archives of General Psychiatry, 50, 759-766.
18.
Pommerich, M., Nicewander, W.A., & Hanson, B. (1999). Estimating average domain scores. Journal of Educational Measurement, 36, 199-216.
19.
Rabe-Hesketh, S., & Skrondal, A. (2005).Multilevel and longitudinal modeling using Stata . College Station, TX: Stata Press .
Sternberg, R.J. (1988). Mental self-government: A theory of intellectual styles and their development. Human Development, 31, 197-224.
24.
Sternberg, R.J., & Wagner, R.K. (1992). The Thinking Styles Inventory. Unpublished manuscript , Yale University, New Haven, CT .
25.
Wainer, H., Sheehan, M., & Wang, X. (2000). Some paths toward making praxis scores more useful . Journal of Educational Measurement, 37, 113-140.
26.
Wang, W.-C. (2004). Effects of anchor item methods on the detection of differential item functioning within the family of Rasch models. Journal of Experimental Education, 72, 221-261.
Wang, W.-C., Chen, P.-H., & Cheng, Y.-Y. (2004). Improving measurement precision of test batteries using multidimensional item response models. Psychological Methods, 9, 116-136.
29.
Wang, W.-C., & Yeh, Y.-L. (2003). Effects of anchor item methods on differential item functioning detection with the likelihood ratio test. Applied Psychological Measurement, 27, 479-498.
30.
Wang, W.-C., Wilson, M., & Adams, R.J. (1997). Rasch models for multidimensionality between and within items. In, M. Wilson, G. Engelhard, & K. Draney (Eds.), Objective measurement IV: Theory into practice (pp. 139-155). Norwood, NJ: Ablex.
31.
Wright, B.D., Linacre, J.M., Gustafson, J.-E., & Martin-Löf, P. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370.
32.
Wu, M.L., Adams, R.J., & Wilson, M.R. (1998). ConQuest [Computer software and manual]. Camber-well, Victoria, Australia: Australian Council for Educational Research.
33.
Yao, L., & Boughton, K.A. (2007). A multidimensional item response modeling approach for improving subscale proficiency estimation and classification. Applied Psychological Measurement, 31, 83-105.
34.
Yao, L., & Schwarz R. (2006). A multidimensional partial credit model with associated item and test statistics: An application to mixed format tests. Applied Psychological Measurement, 30, 469-492.
35.
Zimowski, M.F., Muraki, E., Mislevy, R.J., & Bock, R.D. (1996). BILOG-MG: Multiple-group IRT analysis and test maintenance for binary items [Computer software and manual]. Chicago: Scientific Software international.
