The importance of intermediate items has been overlooked since the emergence of dominance-based Likert-type scales. The current study aims to advance our understanding of the psychometric properties of intermediate items by showing that they can be successfully calibrated by an ideal point model. A student sample and an MTurk sample were selected to answer personality scales with intermediate items included. Results showed that personality scales with intermediate items demonstrated satisfactory model fits to the ideal point model, but not to the dominance model. When analyzed with the ideal point model, intermediate items provided more information than extreme items for respondents with high latent trait levels. Among four proposed domains of intermediate items (Frequency, Average, Condition, and Transition, “FACT”), the Average domain was consistently found to exhibit the best properties. The proportion of intermediate items in the scale also affected the model fit and the validity of the scale.
AllenM. J.YenW. M. (1979). Introduction to measurement theory. Monterey, CA: Sage.
2.
BrownA.Maydeu-OlivaresA. (2010). Issues that should not be overlooked in the dominance versus ideal point controversy. Industrial and Organizational Psychology: Perspectives on Science and Practice, 3, 489–493.
3.
CarterN. T.DalalD. K.BoyceA. S.O’ConnellM. S.KungM.-C.DelgadoK. M. (2014). Uncovering curvilinear relationships between conscientiousness and job performance: How theoretically appropriate measurement makes an empirical difference. Journal of Applied Psychology, 99(4), 564–586.
4.
CarterN. T.ZickarM. J. (2011). A comparison of the LR and DFIT frameworks of differential functioning applied to the generalized graded unfolding model. Applied Psychological Measurement, 35(8), 623–642.
5.
ChernyshenkoO. S.StarkS.ChanK. Y.DrasgowF.WilliamsB. A. (2001). Fitting item response theory models to two personality inventories: Issues and insights. Multivariate Behavioral Research, 36, 523–562.
6.
ChernyshenkoO. S.StarkS.DrasgowF.RobertsB. W. (2007). Constructing personality scales under assumptions of an ideal point response process: Toward increasing the flexibility of personality measures. Psychological Assessment, 19, 88–106.
7.
CoombsC. H. (1964). A theory of data. New York, NY: John Wiley.
8.
DavisonM. L. (1977). On a metric, unidimensional unfolding model for attitudinal and developmental data. Psychometrika, 42, 523–548.
9.
de la TorreJ.StarkS.ChernyshenkoO. S. (2006). Markov chain Monte Carlo estimation of item parameters for the generalized graded unfolding model. Applied Psychological Measurement, 30(3), 216–232.
10.
DrasgowF.ChernyshenkoO. S.StarkS. (2010). 75 years after Likert: Thurstone was right!Industrial & Organizational Psychology: Perspectives on Science and Practice, 3(4), 465–476.
11.
DrasgowF.LevineM. V.TsienS.WilliamsB. A.MeadA. D. (1995). Fitting polytomous item response models to multiple-choice tests. Applied Psychological Measurement, 19, 145–165.
12.
GoldbergL. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In MervieldeI.DearyI.De FruytF.OstendorfF. (Eds.), Personality psychology in Europe (Vol. 7, pp. 7–28). Tilburg, Netherlands: Tilburg University Press.
13.
HinkinT. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1, 104–121.
14.
HuiC. H.TriandisH. C. (1989). Effects of culture and response format on extreme response style. Journal of Cross-Cultural Psychology, 20(3), 296–309.
15.
JohnsonJ. A. (2004). The impact of item characteristics on item and scale validity. Multivariate Behavioral Research, 39(2), 273–302.
16.
LikertR. (1932). A technique for the measurement of attitudes. Archives of Psychology, 140, 5–53.
17.
LordF. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
NyeC. D.NewmanD. A.JosephD. L. (2010). Never say “always”? Extreme item wording effects on scalar invariance and item response curves. Organizational Research Methods, 13(4), 806–830.
20.
O’BrienE.LaHuisD. M. (2011). Do applicants and incumbents respond to personality items similarly? A comparison of dominance and ideal point response models. International Journal of Selection and Assessment, 19(2), 109–118.
21.
ReckaseM. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational and Behavioral Statistics, 4(3), 207–230.
22.
ReiseS. P.HensonJ. M. (2000). Computerization and adaptive administration of the NEO PI-R. Assessment, 7(4), 347–364.
23.
RobertsB. W. (2007). Contextualizing personality psychology. Journal of Personality, 75(6), 1071–1082.
24.
RobertsB. W.ChernyshenkoO. S.StarkS.GoldbergL. R. (2005). The structure of conscientiousness: An empirical investigation based on seven major personality questionnaires. Personnel Psychology, 58(1), 103–139.
25.
RobertsJ. S.DonoghueJ. R.LaughlinJ. E. (2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24, 3–32.
26.
RobertsJ. S.FangH.CuiW.WangY. (2004). GGUM2004: A Windows based program to estimate parameters in the generalized graded unfolding model. Applied Psychological Measurement, 30, 64–65.
SamejimaF. (1969). Estimation of latent ability using a response pattern of graded scores(Psychometrika Monograph Supplement No. 17). Richmond, VA: Psychometric Society.
29.
ScherbaumC. A.SabetJ.KernM. J.AgnelloP. (2013). Examining faking on personality inventories using unfolding item response theory models. Journal of Personality Assessment, 95(2), 207–216.
30.
StarkS. (2001). MODFIT: A computer program for model-data fit (Unpublished manuscript). Champaign: University of Illinois at Urbana-Champaign.
31.
StarkS.ChernyshenkoO. S.DrasgowF.WilliamsB. (2006). Examining assumptions about item responding in personality assessment: Should ideal point methods be considered for scale development and scoring?Journal of Applied Psychology, 91, 25–39.
32.
StarkS.DrasgowF.ChernyshenkoO. S. (2008, October). Update on Tailored Adaptive Personality Assessment System (TAPAS): The next generation of personality assessment systems to support personnel selection and classification decisions. Paper presented at the 50th Annual Conference of the International Military Testing Association, Amsterdam, Netherlands.
33.
TayL.AliU. S.DrasgowF.WilliamsB. (2011). Fitting IRT models to dichotomous and polytomous data: Assessing the relative model-data fit of ideal point and dominance models. Applied Psychological Measurement, 35(4), 280–295.
34.
TayL.DrasgowF. (2012). Theoretical, statistical, and substantive issues in the assessment of construct dimensionality: Accounting for the item response process. Organizational Research Methods, 15(3), 363–384.
35.
TayL.DrasgowF.RoundsJ.WilliamsB. (2009). Fitting measurement models to vocational interest data: Are dominance models ideal?Journal of Applied Psychology, 94, 1287–1304.
36.
ThissenD.ChenW.-H.BockR. D. (2003). MULTILOG 7 for Windows: Multiple-category item analysis and test scoring using item response theory [Computer software]. Skokie, IL: Scientific Software International.
37.
ThurstoneL. L. (1927). A law of comparative judgment. Psychological Review, 34, 273–286.
38.
ThurstoneL. L. (1928). Attitudes can be measured. American Journal of Psychology, 33, 529–554.
39.
ThurstoneL. L. (1929). Theory of attitude measurement. Psychological Review, 36, 222–241.
40.
ThurstoneL. L. (1931). The measurement of change in social attitude. Journal of Social Psychology, 2, 230–245.
41.
VickersR. R.JrConwayT. L.HervigL. K. (1990). Demonstration of replicable dimensions of health behaviors. Preventive Medicine, 19(4), 377–401.
42.
WangW. (2013). A Bayesian Markov-Chain Monte-Carlo approach to the generalized graded unfolding model estimation: The future of non-cognitive measurement(Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign.
43.
WangW.TayL.DrasgowF. (2013). Detecting differential item functioning of polytomous items for an ideal point response process. Applied Psychological Measurement. Advance online publication. doi:10.1177/0146621613476156
44.
WernerP. D.PervinL. A. (1986). The content of personality inventory items. Journal of Personality and Social Psychology, 51(3), 622–628.
45.
WigginsJ. S.GoldbergL. R. (1965). Interrelationships among MMPI item characteristics. Educational and Psychological Measurement, 25(2), 381–397.
