This study investigated the relative model—data fit of an ideal point item response theory (IRT) model (the generalized graded unfolding model [GGUM]) and dominance IRT models (e.g., the two-parameter logistic model [2PLM] and Samejima’s graded response model [GRM]) to simulated dichotomous and polytomous data generated from each of these models. The relative magnitudes of the adjusted χ2/df ratios for item pairs and item triples at the test level were used to evaluate fit. Two simulation studies were conducted, one for dichotomous data and the other for polytomous data. Relative fit of the ideal point and dominance models were compared with respect to different conditions: test length, sample size, and sample type. In many simulated conditions, it was found that comparing relative fits (using test-level doubles and triples adjusted χ2/df ratios) almost always consistently pointed to the correct IRT model. However, GGUM could fit dichotomous two-parameter logistic (2PL) data well when the scale length was short (i.e., 15 items); nevertheless, an examination of estimated GGUM item parameters clearly shows dominance item characteristics. Results of the simulation studies and implications are discussed.
Chernyshenko, O.S., Stark, S., Drasgow, F., & Roberts, B.W. ( 2007). Constructing personality scales under the assumptions of an ideal point response process: Toward increasing the flexibility of personality measures. Psychological Assessment, 19, 88-106.
2.
Coombs, H.C. ( 1964). A theory of data. New York, NY: Wiley.
3.
Coombs, H.C. ( 1975). A note on the relation between the vector model and the unfolding model for preferences. Psychometrika, 40, 115-116.
4.
Coombs, H.C., & Kao, R.C. ( 1960). On a connection between factor analysis and multidimensional unfolding. Psychometrika, 25, 219-231.
5.
Cronbach, L.J. ( 1949). Essentials of psychological testing. New York, NY: Harper.
6.
Davison, M.L. ( 1977). On a metric, unidimensional unfolding model for attitudinal and developmental data. Psychometrika, 42, 523-548.
7.
Diener, E., & Diener, C. ( 1996). Most people are happy. Psychological Science, 7, 181-185.
Fraley, C.R., Waller, N.G., & Brennan, K.A. ( 2000). An item response theory analysis of self-report measures of adult attachment. Journal of Personality and Social Psychology , 2, 350-365.
10.
Guo, Y., Tay, L. & Drasgow, F. ( 2010, April). A comparison of IRT item fit statistics for dichotomous responses. Poster presented at 2010 Society of Industrial and Organizational Psychology Conference, Atlanta, GA.
Koenig, J.A., & Roberts, J.S. ( 2007). Linking parameters estimated with the generalized graded unfolding model: A comparison of the accuracy of characteristic curve methods . Applied Psychological Measurement, 31, 504-524.
13.
Levine, M.V., Drasgow, F., Williams, B., McCusker, C., & Thomasson, G.L. (1992). Measuring the difference between two models. Applied Psychological Measurement, 16, 261-278.
14.
Liao, C., & Mead, A.D. ( 2009, April). Fit of ideal-point and dominance IRT models to simulated data. Paper presented at the 24th annual meeting of the Society for Industrial and Organizational Psychology, New Orleans, LA.
15.
Lord, F.M. ( 1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum .
16.
Penfield, R.D., & Camilli, G. ( 2007). Differential item functioning and item bias. In S. Sinharay & C. R. Rao (Eds.), Handbook of statistics (Vol. 26, pp. 125-167). New York, NY: Elsevier.
17.
Raju, N.S. ( 1988). The area between two item characteristic curves. Psychometrika, 53, 495-502.
18.
Raju, N.S., van der Linden, W.J., & Fleer, P.F. ( 1995). IRT-based internal measures of differential functioning of items and tests. Applied Psychological Measurement, 19, 353-368.
19.
Reise, S.P., & Waller, N.G. ( 1990). Fitting the two-parameter model to personality data. Applied Psychological Measurement, 14, 45-58.
20.
Roberts, J.S. ( 2008). Modified likelihood-based item fit statistics for the generalized graded unfolding model. Applied Psychological Measurement, 32, 407-423.
21.
Roberts, J.S., Donoghue, J.R., & Laughlin, J.E. (2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24, 3-32.
22.
Roberts, J.S., Donoghue, J.R., & Laughlin, J.E. (2002). Characteristics of MML/EAP parameter estimates in the generalized graded unfolding model. Applied Psychological Measurement, 26, 192-207.
23.
Roberts, J.S., Fang, H., Cui, W., & Wang, Y. ( 2004). GGUM2004: A windows based program to estimate parameters in the generalized graded unfolding model. Applied Psychological Measurement, 30, 64-65.
24.
Roberts, J.S., Laughlin, J.E., & Wedell, D.H. ( 1999). Validity issues in the Likert and Thurstone approaches to attitude measurement. Educational and Psychological Measurement , 59, 211-233.
25.
Samejima, F. ( 1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 34, 100-114.
26.
Spector, P.E., Van Katwyk, P.T., Brannick, M.T., & Chen, P.Y. ( 1997). When two factors don’t reflect two constructs: How item characteristics can produce artifactual factors. Journal of Management, 23, 659-677.
27.
Stark, S., Chernyshenko, O.S., Drasgow, F., & Williams, B.A. (2006). Examining assumptions about item responding in personality assessment: Should ideal point models be considered for scale development and scoring? Journal of Applied Psychology , 91, 25-39.
28.
Tay, L., & Drasgow, F. ( 2010). Adjusting the adjusted χ2/df ratio statistic for IRT model-data fit. Manuscript submitted for publication.
29.
Tay, L., Drasgow, F., Rounds, J., & Williams, B. ( 2009). Fitting measurement models to vocational interest data: Are dominance models ideal?Journal of Applied Psychology , 94, 1287-1304.
30.
Thissen, D., Chen, W-H., & Bock, D. ( 2003). MULTILOG (Version 7.03) [Computer Software]. Chicago, IL: Scientific Software.
31.
Thissen, D., Steinberg, L., & Wainer, H. ( 1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67-113). Hillsdale, NJ: Erlbaum.
32.
Thurstone, L.L. ( 1927a). A law of comparative judgment. Psychological Review, 34, 273-286.
33.
Thurstone, L.L. ( 1927b). Psychophysical analysis. American Journal of Psychology, 38, 368-389.
34.
Thurstone, L.L. ( 1928). Attitudes can be measured. American Journal of Psychology, 33, 529-554.
35.
Thurstone, L.L. ( 1945). Multiple factor analysis. Chicago, IL: University of Chicago Press.
36.
Van den Wollenberg, A.L. ( 1982). Two new test statistics for the Rasch model. Psychometrika, 47, 123-140.
37.
van Schuur, W.H., & Kiers, H.A.L. ( 1994). Why factor analysis often is the incorrect model for analyzing bipolar concepts and what model to use instead. Applied Psychological Measurement, 18, 97-110.
38.
Waller, N.G., Tellegen, A., McDonald, R.P., & Lykken, D.T. ( 1996). Exploring nonlinear models in personality assessment: Development and preliminary validation of a negative emotionality scale. Journal of Personality, 3, 545-576.
39.
Waller, N.G., Thompson, J.S., & Wenk, E. ( 2000). Using IRT to separate measurement bias from true group differences on homogeneous and heterogeneous scales: An illustration with the MMPI. Psychological Methods, 5, 125-146.
40.
Williams, B.A., & Levine, M.V. ( 1993). FORSCORE: A computer program for nonparametric item response theory. Unpublished manuscript, University of Illinois at Urbana-Champaign.
