The Effects of Changing Negatively Worded Items to Positively Worded Items on the Reliability and the Factor Structure of Psychological Scales

Abstract

In survey measurement, acquiescence bias is a response effect that occurs when respondents agree to the item or the question in the scale regardless of its content. It is assumed that negative items force participants not to agree with some items. Using the mixture approach, however, is not without a substantial cost on both the structure and the scale psychometric properties. The effects of including negative items in scales is what this study tried to investigate. Therefore, the aim of the study is to empirically evaluate the effects of changing negative items to their equivalent positively worded items on the reliability and the factor structure of psychological scales. It is hypothesized that this approach improves the scale factors structures and reliability. Seven commonly used psychological scales that have both negatively and positively worded items have been selected. The scales were applied on seven different samples with a total number of 4192 participants from a public university in the United Arab Emirates. The results confirmed that changing negative items to their equivalent positively directed items systematically and significantly increased the reliability values as well as improved the factor structure of psychological scales.

Keywords

psychological scales United Arab Emirates negatively worded positively worded items acquiescence bias

Get full access to this article

View all access options for this article.

References

Barnette

J. J.

(2000). Effects of stem and Likert response option reversals on survey internal consistency: If you feel the need, there is a better alternative to using those negatively worded stems. Educational and Psychological Measurement, 60(3), 361–370. http://doi.org/10.1177/00131640021970592

Chan

J. C.

(1991). Response-order effects in Likert-type scales. Educational and Psychological Measurement, 51(3), 531–540. http://doi.org/10.1177/0013164491513002

Coleman

C. M.

(2013). Effects of negative keying and wording in attitude measures: A mixed-methods study [Doctoral dissertation, James Madison University].

Dodeen

(2015). The effects of positively and negatively worded items on the factor structure of the UCLA loneliness scale. Journal of Psychoeducational Assessment, 33(3), 259–267. http://doi.org/10.1177/0734282914548325

Fowers

B. J.

Olson

D. H.

(1993). ENRICH marital satisfaction scale: A brief research and clinical tool. Journal of Family Psychology, 7(2), 176–185. http://doi.org/10.1037/0893-3200.7.2.176

Heatherton

T. F.

Policy

(1991). Development and validation of a scale for measuring state self-esteem. Journal of Personality and Social Psychology, 60(6), 895–910. http://doi.org/10.1037/0022-3514.60.6.895

Hill

S. J.

Roberts

M. E.

(2021). Acquiescence bias inflates estimates of conspiratorial beliefs and political misperceptions.

Jaensson

Nilsson

(2016). Impact of changing positively worded items to negatively worded items in the Swedish web‐version of the Quality of Recovery (SwQoR) questionnaire. Journal of Evaluation in Clinical Practice, 23(3), 502–507. http://doi.org/10.1111/jep.12639

Józsa

Morgan

G. A.

(2017). Reversed items in Likert scales: Filtering out invalid responders. Journal of Psychological and Educational Research, 25(1), 7–25.

10.

Józsa

Wang

Barrett

K. C.

Morgan

G. A.

(2014). Age and cultural differences in mastery motivation in American, Chinese, and Hungarian school-age children. Child Development Research, Article ID 803061(2014), 1–16. https://doi.org/10.1155/2014/803061 16

11.

Lasgaard

(2007). Reliability and validity of the Danish version of the UCLA loneliness scale. Personality and Individual Differences, 42(7), 1359–1366. http://doi.org/10.1016/j.paid.2006.10.013

12.

Lewis

J. R.

Sauro

(2009). The factor structure of the system usability scale. In Proceedings of international conference (HCII 2009), San Diego, CA, July 19–24, 2009.

13.

Mayerl

Giehl

(2018). A closer look at attitude scales with positive and negative items. Response latency perspectives on measurement quality. Survey Research Methods, 12(3), 193–209.

14.

Motl

Conroy

Horan

(2000). The Social Physique Anxiety Scale: An example of the potential consequence of negatively worded items in factorial validity studies. Journal of Applied Measurement, 1(4), 327–345.

15.

Nickell

(1998). The helping attitudes scale. Paper presented at the 106th Annual Convention of the American Psychological Association.

16.

Paulhus

D. L.

(1991). Measurement and control of response bias. In Robinson

J. P.

Shaver

P. R.

Wrightsman

L. S.

(Eds.), Measures of personality and social psychological attitudes (pp. 17–59). Academic Press.

17.

Qasem

Gul

(2014). Effect of items direction (positive or negative) on the factorial construction and criterion related validity in Likert scale. Asian Journal of Research in Social Sciences and Humanities, 4(4), 114–121.

18.

Rosenberg

(1965). Society and the adolescent self-image. Princeton University Press.

19.

Roszkowski

Soven

(2010) Shifting gears: Consequences of including two negatively worded items in the middle of a positively worded questionnaire. Assessment and Evaluation in Higher Education, 35(1), 113–130. http://doi.org/10.1080/02602930802618344

20.

Russell

D. W.

(1996). UCLA loneliness scale (version 3): Reliability, validity, and factor structure. Journal of Personality Assessment, 66(1), 20–40.

21.

Scheier

M. F.

Carver

C. S.

Bridges

M. W.

(1994). Distinguishing optimism from neuroticism (and trait anxiety, self-mastery, and self-esteem): A re-evaluation of the life orientation test. Journal of Personality and Social Psychology, 67(6), 1063–1078. http://doi.org/10.1037//0022-3514.67.6.1063

22.

Steger

M. F.

Frazier

Oishi

Kaler

(2006). The Meaning in Life Questionnaire: Assessing the presence of and search for meaning in life. Journal of Counseling Psychology, 53(1), 80–93. https://doi.org/10.1037/0022-0167.53.1.80

23.

Weijters

Millet

Cabooter

(2020). Extremity in horizontal and vertical Likert scale format responses. Some evidence on how visual distance between response categories influences extreme responding. International Journal of Research in Marketing, 38(1), 85–103. http://doi.org/10.1016/j.ijresmar.2020.04.002

24.

Woods

C. M.

(2006). Careless responding to reverse-worded items: Implications for confirmatory factor analysis. Journal of Psychopathology and Behavioral Assessment, 28(3), 186. https://doi.org/10.1007/s10862-005-9004-7

25.

Zeng

Wen

Zhang

(2020). How does the valence of wording affect features of a scale? The method effects in the undergraduate learning burnout scale. Frontiers in Psychology, 11, 2638. http://doi.org/10.3389/fpsyg.2020.585179

26.

Zhang

Noor

Savalei

(2016). Examining the effect of reverse worded items on the factor structure of the need for cognition scale. PLoS One 11(6), e0157795. http://doi.org/10.1371/journal.pone.0157795

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.07 MB