In the modeling of ordinal responses in psychological measurement and survey-based research, response styles that represent specific answering patterns of respondents are typically ignored. One consequence is that estimates of item parameters can be poor and considerably biased. The focus here is on the modeling of a tendency to extreme or middle categories. An extension of the partial credit model is proposed that explicitly accounts for this specific response style. In contrast to existing approaches, which are based on finite mixtures, explicit person-specific response style parameters are introduced. The resulting model can be estimated within the framework of generalized mixed linear models. It is shown that estimates can be seriously biased if the response style is ignored. In applications, it is demonstrated that a tendency to extreme or middle categories is not uncommon. A software tool is developed that makes the model easy to apply.
AndersonD. A.AitkinM. (1985). Variance component models with binary response: Interviewer variability. Journal of the Royal Statistical Society, Series B:Statistical Methodology, 47, 203-210.
2.
AustinE. J.DearyI. J.EganV. (2006). Individual differences in response scale use: Mixed rasch modelling of responses to NEO-FFI items. Personality and Individual Differences, 40, 1235-1245.
3.
BaumgartnerH.SteenkampJ.-B. E. M. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38, 143-156.
4.
BockR. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.
5.
BöckenholtU. (2012). Modeling multiple response processes in judgment and choice. Psychological Methods, 17, 665-678.
BöckenholtU.MeiserT. (2017). Response style analysis with threshold and multi-process IRT models: A review and tutorial. British Journal of Mathematical and Statistical Psychology, 70, 159-181.
8.
BoltD. M.NewtonJ. R. (2011). Multiscale measurement of extreme response style. Educational and Psychological Measurement, 71, 814-833.
9.
BoltD. M.JohnsonT. R. (2009). Addressing score bias and differential item functioning due to individual differences in response style. Applied Psychological Measurement, 33, 335-352.
10.
BreslowN. E.ClaytonD. G. (1993). Approximate inference in generalized linear mixed model. Journal of the American Statistical Association, 88(421), 9-25.
11.
ByrdR. H.LuP.NocedalJ.ZhuC. (1995). A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16, 1190-1208.
12.
De BoeckP.PartchevI. (2012). Irtrees: Tree-based item response models of the glmm family. Journal of Statistical Software, 48, 1-28.
13.
EddelbuettelD. (2013). Seamless R and C++ integration with Rcpp. New York, NY: Springer-Verlag.
14.
EidM.RauberM. (2000). Detecting measurement invariance in organizational surveys. European Journal of Psychological Assessment, 16, 20-30.
FahrmeirL.TutzG. (1997). Multivariate statistical modelling based on generalized linear models. New York, NY: Springer-Verlag.
17.
FalkC. F.CaiL. (2016). A flexible full-information approach to the modeling of response styles. Psychological Methods, 21, 328-347.
18.
GollwitzerM.EidM.JürgensenR. (2005). Response styles in the assessment of anger expression. Psychological Assessment, 17, 56-69.
19.
HindeJ. (1982). Compound poisson regression models. In GilchristR. (Ed.), International Conference on Generalized Linear Models (pp. 109-121). New York, NY: Springer-Verlag.
20.
IrtelH. (1995). An extension of the concept of specific objectivity. Psychometrika, 60, 115-118.
21.
JeonM.De BoeckP. (2016). A generalized item response tree model for psychological assessments. Behavior Research Methods, 48, 1070-1085.
22.
JinK.-Y.WangW.-C. (2014). Generalized IRT models for extreme response style. Educational and Psychological Measurement, 74, 116-138.
23.
JohnsonT. R. (2003). On the use of heterogeneous thresholds ordinal regression models to account for individual differences in response style. Psychometrika, 68, 563-583.
24.
JohnsonT. R. (2014). A note on conditional and quasi-conditional maximum likelihood estimation for polytomous item response models: Robust inference in the presence of individual differences in response scale use. Applied Psychological Measurement, 38, 599-613.
25.
JohnsonT. R.BoltD. M. (2010). On the use of factor-analytic multinomial logit item response models to account for individual differences in response style. Journal of Educational and Behavioral Statistics, 35, 92-114.
26.
KankarašM.MoorsG. (2009). Measurement equivalence in solidarity attitudes in Europe insights from a multiple-group latent-class factor approach. International Sociology, 24, 557-579.
KhorramdelL.von DavierM. (2014). Measuring response styles across the big five: A multiscale extension of an approach using multinomial processing trees. Multivariate Behavioral Research, 49, 161-177.
29.
Maij-de MeijA. M.KeldermanH.van der FlierH. (2008). Fitting a mixture item response theory model to personality questionnaire data: Characterizing latent classes and investigating possibilities for improving prediction. Applied Psychological Measurement, 32, 611-631.
30.
MastersG. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
31.
MastersG. N.WrightB. (1984). The essential process in a family of measurement models. Psychometrika, 49, 529-544.
32.
McCullochC.SearleS. (2001). Generalized, linear, and mixed models. New York, NY: John Wiley.
33.
MessickS. (1991). Psychology and methodology of response styles. In SnowR. E.WileyD. E. (Eds.), Improving inquiry in social science: A volume in honor of Lee J. Cronbach (pp. 161-200). Hillsdale, NJ: Lawrence Erlbaum
34.
MoorsG. (2004). Facts and artefacts in the comparison of attitudes among ethnic minorities. A multigroup latent class structure model with adjustment for response style behavior. European Sociological Review, 20, 303-320.
35.
MoorsG. (2010). Ranking the ratings: A latent-class regression model to control for overall agreement in opinion research. International Journal of Public Opinion Research, 22, 93-119.
36.
NaemiB. D.BealD. J.PayneS. C. (2009). Personality predictors of extreme response style. Journal of Personality, 77, 261-286.
37.
PlieningerH. (2016). Mountain or molehill? A simulation study on the impact of response styles. Educational and Psychological Measurement, 77, 32-53. doi:10.1177/0013164416636655
38.
PlieningerH.MeiserT. (2014). Validity of multiprocess IRT models for separating content and response styles. Educational and Psychological Measurement, 74, 875-899.
39.
RaschG. (1966). An item analysis which takes individual differences into account. British Journal of Mathematical and Statistical Psychology, 19, 49-57.
40.
RaschG. (1977). On specific objectivity. an attempt at formalizing the request for generality and validity of scientific statements in symposium on scientific objectivity. Danish Year-Book of Philosophy Kobenhavn, 14, 58-94.
41.
R Core Team. (2016). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
42.
RostJ. (1991). A logistic mixture distribution model for polychotomous item responses. British Journal of Mathematical and Statistical Psychology, 44, 75-92.
43.
SamejimaF. (1997). Graded response model. In Van der LindenW.HambletonR. (Eds.), Handbook of modern item response theory (pp. 85-100). New York, NY: Springer.
44.
SuhY.BoltD. M. (2010). Nested logit models for multiple-choice item response data. Psychometrika, 75, 454-473.
45.
Thissen-RoeA.ThissenD. (2013). A two-decision model for responses to likert-type items. Journal of Educational and Behavioral Statistics, 38, 522-547.
46.
TutzG. (2012). Regression for categorical data. Cambridge, UK: Cambridge University Press.
47.
TutzG.BergerM. (2016). Response styles in rating scales: Simultaneous modeling of content-related effects and the tendency to middle or extreme categories. Journal of Educational and Behavioral Statistics, 41, 239-268.
48.
Van RosmalenJ.Van HerkH.GroenenP. (2010). Identifying response styles: A latentclass bilinear multinomial logit model. Journal of Marketing Research, 47, 157-172.
49.
Van VaerenberghY.ThomasT. D. (2013). Response styles in survey research: A literature review of antecedents, consequences, and remedies. International Journal of Public Opinion Research, 25, 195-217.
50.
von DavierM.RostJ. (1995). Polytomous mixed rasch models. In FischerG.MolenaarI. (Eds.), Rasch models (pp. 371-379). New York, NY: Springer-Verlag.
51.
von DavierM.YamamotoK. (2007). Mixture-distribution and hybrid rasch models. In von DavierM.CarstensenC. (Eds.), Multivariate and mixture distribution Rasch models (pp. 99-115). New York, NY: Springer-Verlag.
52.
WangW.-CWilsonM.ShihC.-L. (2006). Modeling randomness in judging rating scales with a random-effects rating scale model. Journal of Educational Measurement, 43, 335-353.
53.
WetzelE.CarstensenC. H. (2017). Multidimensional modeling of traits and response styles. European Journal of Psychological Assessment, 33, 352-364. doi:10.1027/1015-5759/a000291
ZPID. (2013). PsychData of the Leibniz Institute for Psychology Information ZPID. Trier, Germany: Center for Research Data in Psychology.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
