This study compared the functioning of positively and negatively worded personality items using item response theory. In Study 1, word pairs from the Goldberg Adjective Checklist were analyzed using the Graded Response Model. Across subscales, negatively worded items produced comparatively higher difficulty and lower discrimination parameters than positively worded items and yielded almost no information. Model fit was examined for two forms of each scale: parameters freely estimated versus parameters estimated with item pairs constrained to be equal. Greater misfit was found in the latter. In Study 2, positively and negatively worded items from a more commonly formatted personality assessment were compared. Parameters again differed, albeit to a lesser extent, and model fit was improved in four out of five scales by the removal of negatively worded items. These results indicated that positively and negatively worded items were not psychometrically interchangeable and that negatively worded items have limited utility. Implications and future directions are discussed.
BarkerD.EbelR. (1982). A comparison of difficulty and discrimination values of selected true-false item types. Contemporary Educational Psychology, 7(1), 35-40.
2.
BarnetteJ. (2000). Effects of stem and Likert response option reversals on survey internal consistency: If you feel the need, there is a better alternative to using those negatively-worded stems. Educational and Psychological Measurement, 6, 361-370.
3.
BensonJ.HocevarD. (1985). The impact of item phrasing on the validity of attitude scales for elementary school children. Journal of Educational Measurement, 22, 231-240.
4.
CareyL. (2001). Measuring and evaluating school learning (3rd ed.). Boston, MA: Allyn & Bacon.
5.
CronbachL. (1950). Further evidence on response sets and test design. Educational and Psychological Measurement, 10, 3-31.
6.
DeVellisR. F. (2003). Scale development: Theory and applications (2nd ed.). Thousand Oaks, CA: Sage.
7.
DiStefanoC.MotlR. (2006). Further investigating method effects associated with negatively-worded items on self-report surveys. Structural Equation Modeling, 13, 440-464.
8.
EmbretsonS.ReiseS. (2000). Item response theory for psychologists (2nd ed.). Florence, KY: Psychology Press.
9.
FleishmanJ.BensonJ. (1987). Using LISREL to evaluate measurement models and scale reliabilities. Educational and Psychological Measurement, 47, 925-939.
10.
GoldbergL. (1992). The development of markers for the Big-Five factor structure. Psychological Assessment, 4, 26-42.
11.
GoldbergL. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In MervieldeI.DearyI.De FruytF.OstendorfF. (Eds.), Personality psychology in Europe (Vol. 7, pp. 7-28). Tilburg, Netherlands: Tilburg University Press.
12.
HodgeD.GillespieD. (2003). Phrase completions: An alternative to Likert scales. Social Work Research, 27(1), 45-55.
13.
LikertR. (1932). A technique for the measurement of attitudes. Archives of Psychology, 140, 4-55.
14.
MarshH. (1996). Positive and negative global self-esteem: A substantively meaningful distinction or artifactors?Journal of Personality and Social Psychology, 70, 810-819.
15.
MelnickS.GableR. (1990). The use of negative item stems: A cautionary note. Educational Research Quarterly, 14, 31-36.
16.
NardiP. (2003). Doing survey research: A guide to quantitative methods. Boston, MA: Allyn & Bacon.
17.
ReckaseM. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational and Behavioral Statistics, 4, 207-230.
18.
ReiseS.YuJ. (1990). Parameter recovery in the Graded Response Model using MULTILOG. Journal of Educational Measurement, 27, 133-144.
19.
RodebaughT. L.WoodsC. M.HeimbergR. G. (2007). The reverse of social anxiety is not always the opposite: The reverse-scored items of the Social Interaction Anxiety Scale do not belong. Behavior Therapy, 38, 192-206.
20.
RodebaughT. L.WoodsC. M.ThissenD. M.HeimbergR. G.ChamblessD. L.RapeeR. M. (2004). More information from fewer questions: The factor structure and item properties of the original and brief fear of negative evaluation scale. Psychological assessment, 16, 169-181.
21.
RossiP.WrightJ.AndersonA. (1983). Handbook of survey research. New York, NY: Academic Press.
22.
SamejimaF. (1997). Graded response model. In van der LindenW. J.HambletonR. K. (Eds.), Handbook of modern item response theory (pp. 85-100). New York, NY: Springer.
23.
SchmittN.StuitsD. (1985). Factors defined by negatively keyed items: The result of careless respondents?Applied Psychological Measurement, 9, 367-373.
24.
SchriesheimC. A.HillK. (1981). Controlling acquiescence response bias by item reversals: The effect on questionnaire validity. Educational and Psychological Measurement, 41, 1101-1114.
25.
SchweizerK.RauchW. (2008). An investigation of the structure of the social optimism scale in considering the dimensionality problem. Journal of Individual Differences, 29, 223-230.
26.
SpectorP.Van KatwykP.BrannickM.ChenP. (1997). When two factors don’t reflect two constructs: How item characteristics can produce artifactual factors. Journal of Management, 23, 659-677.
27.
StarkS. (2001). MODFIT: A computer program for model-data fit. Champaign: University of Illinois at Urbana-Champaign.
28.
TamirP. (1993). Positive and negative multiple choice items: How different are they?Studies in Educational Evaluation, 19, 311-325.
WeemsG.OnwuegbuzieA. J.CollinsK. M. (2006). The role of reading comprehension in responses to positively and negatively worded items on rating scales. Evaluation and Research in Education, 19(1), 3-20.
31.
WeemsG. H.OnwuegbuzieA. J. (2001). The impact of midpoint responses and reverse coding on survey data. Measurement and Evaluation in Counseling and Development, 34, 166-176.
32.
WoodsC. (2006). Careless responding to reverse-worded items: Implications for confirmatory factor analyses. Journal of Psychopathology and Behavioral Assessment, 28, 189-194.
33.
WuS. M.SchulerT. A.EdwardsM. C.YangH. C.BrothersB. M. (2013). Factor analytic and item response theory evaluation of the Penn State Worry Questionnaire in women with. Quality of Life Research, 22, 1441-1449.
