In randomized experiments, inventories with reverse-keyed items are compared with inventories in which all the items are either positively or negatively associated with the underlying concept. The results show that with reverse keying, a control of the potential bias was not sufficient; likewise, the factorial structure, reliability, and validity were negatively affected. An eye-tracking study revealed that respondents did not process information more deeply with the reverse-keyed items than with the other forms. Respondents seemed to find it difficult to process reverse keying mentally so that it is not sufficient to use it in heterogeneous samples and with short inventories.
Get full access to this article
View all access options for this article.
References
1.
BalkeD.El-MenouarY.RastetterR.-U.. 2014. Einschätzung von Kosten durch Zuwanderer. Zusammenstellung sozialwissenschaftlicher Items und Skalen. doi: 10.6102/zis156.
2.
BarnetteJ. J.2000. Effects of item and Likert response option reversals on survey internal consistency: If you feel the need, there is a better alternative to using those negatively worded items. Educational and Psychological Measurement60:361–70.
3.
BeierleinC.AsbrockF.KauffM.SchmidtP.. 2014. Die Kurzskala Autoritarismus (KSA-3): Ein ökonomisches Messinstrument zur Erfassung dreier Subdimensionen autoritärer Einstellungen. GESIS Working Paper, 35. Mannheim: GESIS.
BornmannL.DanielH.-D.. 2000. Reliabilität und Konstruktvalidität des Kurzfragebogens für Kontrollüberzeugungen (FKK): Überprüfung der Testgütekriterien im Rahmen einer Mehrthemenbefragung unter Studierenden. Empirische Pädagogik14:391–407.
6.
ColstonH. L.1999. “Not good” is “bad,” but “not bad” is not “good”: An analysis of three accounts of negation asymmetry. Discourse Processes28:237–56.
7.
CronbachL. J.1951. Coefficient alpha and the internal structure of tests. Psychometrika16:297–334.
8.
CurranP. G.2016. Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology66:4–19.
9.
DillmanD. A.SmythJ. D.ChristianL. M.. 2014. Internet, phone, mail, and mixed-mode surveys: The tailored design method. Hoboken, NJ: Wiley.
10.
GuttmanL.1945. A basis for analyzing test–retest reliability. Psychometrika10:255–82.
11.
HornL. R.1989. A natural history of negation. Chicago: University of Chicago Press.
12.
HuangJ. L.CurranP. G.KeeneyJ.PoposkiE. M.DeShonR. P.. 2012. Detecting and deterring insufficient effort responding to surveys. Journal of Business and Psychology27:99–114.
13.
KailR.ParkY.-s.. 1990. Impact of practice on speed of mental rotation. Journal of Experimental Child Psychology49:227–44.
14.
LordF. M.NovickM. R.. 1968. Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
15.
MelnickS. A.GableR. K.. 1990. The use of negative items: A cautionary note. Educational Research Quarterly14:31–36.
16.
MenoldN.2017. Rating-scale labeling in online surveys: An experimental comparison of verbal and numeric rating scales with respect to measurement quality and respondents’ cognitive processes. Sociological Methods & Research. doi: 10.1177/0049124117729694.
17.
MenoldN.KaczmirekL.LenznerT.NeusarA.. 2014. How do respondents attend to verbal labels in rating scales?Field Methods26:21–39.
PaulhusD. L.1991. Measurement and control of response bias. In Measures of social psychological attitudes, eds. RobinsonJ. P.ShaverP. R.WrightsmanS., 17–59. San Diego: Academic Press.
20.
RammstedtB.KemperC. J.KleinM. C.BeierleinC.KovalevaA.. 2014. Eine Kurze Skala zur Messung der Fünf Dimensionen der Persönlichkeit: Big-Five-Inventory-10 (BFI-10). Zusammenstellung Sozialwissenschaftlicher Items und Skalen. doi:10.6102/zis76.
21.
RaykovT.MarcoulidesG. A.. 2011. Introduction to psychometric theory. New York: Taylor & Francis.
22.
RobinsR. W.TracyJ. L.TrzesniewskiK.PotterJ.GoslingS. D.. 2001. Personality correlates of self-esteem. Journal of Research in Personality35:463–82.
23.
RoszkowskiM. J.SovenM.. 2010. Shifting gears: Consequences of including two negatively worded items in the middle of a positively worded questionnaire. Assessment & Evaluation in Higher Education35:113–30.
24.
SchotteC. K. W.MaesM.CluydtsR.CosynsP.. 1996. Effects of affective-semantic mode of item presentation in balanced self-report scales: Biased construct validity of the Zung self-rating depression scale. Psychological Medicine26:1161.
25.
SchriesheimC. A.EisenbachR. J.. 1995. An exploratory and confirmatory factor-analytic investigation of item wording effects on the obtained factor structures of survey questionnaire measures. Journal of Management21:1177–93.
26.
SchriesheimC. A.EisenbachR. J.HillK. D.. 1991. The effect of negation and polar opposite item reversals on questionnaire reliability and validity: An experimental investigation. Educational and Psychological Measurement51:67–78.
27.
SchriesheimC. A.HillK. D.. 1981. Controlling acquiescence response bias by item reversals: The effect on questionnaire validity. Educational and Psychological Measurement41:1101–14.
SwainS. D.WeathersD.NiedrichR. W.. 2008. Assessing three sources of misresponse to reversed Likert items. Journal of Marketing Research45:116–31.
30.
TourangeauRRipsL. J.RasinskiK. A.. 2000. The psychology of survey response. New York: Cambridge University Press.
31.
van SonderenE.SandermanR.CoyneJ. C.. 2013. Ineffectiveness of reverse wording of questionnaire items: Let’s learn from cows in the rain, ed. BaradaranH. R.. PLoS One8:e68967.
32.
WeijtersB.BaumgartnerH.SchillewaertN.. 2013. Reversed item bias: An integrative model. Psychological Methods18:320–34.
33.
WongN.RindfleischA.BurroughsJ. E.. 2003. Do reverse-worded items confound measures in cross-cultural consumer research? The case of the material values scale. Journal of Consumer Research30:72–91.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
