Heterogeneity in response styles can affect the conclusions drawn from rating scale data. In particular, biased estimates can be expected if one ignores a tendency to middle categories or to extreme categories. An adjacent categories model is proposed that simultaneously models the content-related effects and the heterogeneity in response styles. By accounting for response styles, it provides a simple remedy for the bias that occurs if the response style is ignored. The model allows to include explanatory variables that have a content-related effect as well as an effect on the response style. A visualization tool is developed that makes the interpretation of effects easily accessible. The proposed model is embedded into the framework of multivariate generalized linear model, which entails that common estimation and inference tools can be used. Existing software can be used to fit the model, which makes it easy to apply.
AgrestiA. (2009). Analysis of ordinal categorical data (2nd ed.). New York, NY: Wiley.
2.
BachmanJ. G.O’MalleyP. (1984). Yea-saying, nay-saying, and going to extremes: Black-White differences in response styles. Public Opinion Quarterly, 48, 491–509.
3.
BaumgartnerH.SteenkampJ.-B. E. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38, 143–156.
4.
BockR. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29–51.
5.
BöckenholtU. (2012). Modeling multiple response processes in judgment and choice. Psychological Methods, 17, 665–678.
6.
BoltD. M.JohnsonT. R. (2009). Addressing score bias and differential item functioning due to individual differences in response style. Applied Psychological Measurement, 33, 335–352.
7.
BoltD. M.NewtonJ. R. (2011). Multiscale measurement of extreme response style. Educational and Psychological Measurement, 71, 814–833.
8.
ClarkeI. (2000). Extreme response style in cross-cultural research: An empirical investigation. Journal of Social Behavior & Personality, 15, 137–152.
9.
ClarkeI.III (2001). Extreme response style in cross-cultural research. International Marketing Review, 18, 301–324.
10.
De BoeckP.PartchevI. (2012). Irtrees: Tree-based item response models of the glmm family. Journal of Statistical Software, 48, 1–28.
11.
EidM.RauberM. (2000). Detecting measurement invariance in organizational surveys. European Journal of Psychological Assessment, 16, 20.
12.
EilersP. H. C.MarxB. D. (1996). Flexible smoothing with B-splines and Penalties. Statistical Science, 11, 89–121.
13.
FahrmeirL.TutzG. (2001). Multivariate statistical modelling based on generalized linear models. New York, NY: Springer.
JeonM.De BoeckP. (2015). A generalized item response tree model for psychological assessments. Behavior Research Methods, doi:10.3758/s13428-015-0631-y
17.
JohnsonT. R. (2003). On the use of heterogeneous thresholds ordinal regression models to account for individual differences in response style. Psychometrika, 68, 563–583.
18.
KankarašM.MoorsG. (2009). Measurement equivalence in solidarity attitudes in europe insights from a multiple-group latent-class factor approach. International Sociology, 24, 557–579.
19.
KhorramdelL.von DavierM. (2014). Measuring response styles across the big five: A multiscale extension of an approach using multinomial processing trees. Multivariate Behavioral Research, 49, 161–177.
20.
MarinG.GambaR. J.MarinB. V. (1992). Extreme response style and acquiescence among hispanics the role of acculturation and education. Journal of Cross-Cultural Psychology, 23, 498–509.
21.
MastersG. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149–174.
22.
MeisenbergG.WilliamsA. (2008). Are acquiescent and extreme response styles related to low intelligence and education?Personality and Individual Differences, 44, 1539–1550.
23.
MessickS. (1991). Psychology and methodology of response styles. In SnowR. E.WileyD. E. (Eds.), Improving inquiry in social science: A volume in honor of Lee J. Cronbach (pp. 161–200). Hillsdale, NJ: Lawrence Erlbaum.
24.
MoorsG. (2003). Diagnosing response style behavior by means of a latent-class factor approach. Socio-demographic correlates of gender role attitudes and perceptions of ethnic discrimination reexamined. Quality and Quantity, 37, 277–302.
25.
MoorsG. (2004). Facts and artefacts in the comparison of attitudes among ethnic minorities. A multigroup latent class structure model with adjustment for response style behavior. European Sociological Review, 20, 303–320.
26.
MoorsG. (2010). Ranking the ratings: A latent-class regression model to control for overall agreement in opinion research. International Journal of Public Opinion Research, 22, 93–119.
27.
PlieningerH.MeiserT. (2014). Validity of multiprocess IRT models for separating content and response styles. Educational and Psychological Measurement, 74, 875–899. doi:10.1177/0013164413514998
28.
SuhY.BoltD. M. (2010). Nested logit models for multiple-choice item response data. Psychometrika, 75, 454–473.
29.
Thissen-RoeA.ThissenD. (2013). A two-decision model for responses to Likert-type items. Journal of Educational and Behavioral Statistics, 38, 522–547.
TutzG. (2012). Regression for categorical data. Cambridge, MA: Cambridge University Press.
32.
Van HerkH.PoortingaY. H.VerhallenT. M. (2004). Response styles in rating scales evidence of method bias in data from six EU countries. Journal of Cross-Cultural Psychology, 35, 346–360.
33.
Van RosmalenJ.Van HerkH. G. P. (2010). Identifying response styles: A latent-class bilinear multinomial logit model. Journal of Marketing Research, 47, 157–172.
34.
Van VaerenberghY.ThomasT. D. (2013). Response styles in survey research: A literature review of antecedents, consequences, and remedies. International Journal of Public Opinion Research, 25, 195–217.
35.
WeijtersB.CabooterE.SchillewaertN. (2010). The effect of rating scale format on response styles: The number of response categories and response category labels. International Journal of Research in Marketing, 27, 236–247.
36.
YeeT. W. (2010). The VGAM package for categorical data analysis. Journal of Statistical Software, 32, 1–34.
YeeT. W.WildC. J. (1996). Vector generalized additive models. Journal of Royal Statistical Society B, 58, 481–493.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
