Sage Journals: Discover world-class research

Abstract

We present an item response model for ordinal public opinion data to understand individual-level variation in attitudes as a function of covariates. The approach allows investigating how individuals (or population subgroups) differ in substantive stances and attitude strength. It is a two-dimensional partial credit model that incorporates covariates linked to attitude direction and strength into the basic model. We exemplify the types of substantive insights into heterogeneity that can be obtained from the approach but not from existing models with two applications: attitudes toward gender equality (European Values Study) and the evaluation of presidential candidates (American National Election Study).

Keywords

public opinion attitudes individual-level variation item response theory ordinal responses

Introduction

Attitudes, like many other important concepts in research on public opinion, are latent constructs that cannot be directly observed or measured with a single survey question. Several contributions have shown that multiple survey items better gauge issue attitudes than an individual item does (e.g., Ansolabehere et al. 2008; Carsey and Layman 2006; Layman and Carsey 2002). While these studies used factor analysis or structural equation modeling, item response theory (IRT) modeling is on the rise in public opinion research to address the challenge of measuring latent constructs (e.g., Bafumi and Herron 2010; Caughey and Warshaw 2015; Jessee 2016; Treier and Hillygus 2009). Using a battery of items tapping the specific domain under study, IRT modeling captures the relationship between the latent quantity and the item survey responses by positioning individuals on the latent attitude or preference scale. We contribute to the field by presenting an explanatory approach for understanding how individuals (or population subgroups) differ in attitudes, which draws on classic ordinal data to measure attitudes and can be applied to standard mass public opinion surveys.

Attitudes represent how people evaluate objects, persons, or issues. The common and well-established instrument to measure attitudes is a series of items that are evaluative statements linked to the latent trait along with ordered response options (e.g., Abelson 1992; Bizer et al. 2003), which indicate the respondent’s level of agreement or disagreement with each statement (e.g., ‘‘strongly disagree,” ‘‘disagree,” ‘‘agree,” ‘‘strongly agree’’), or the level of (dis)approval or (un)favorableness. Such graded scales demand IRT models that account for the underlying ordinal data structure. Even though graded scales, designed to measure attitudes (Likert 1932), are standard in public opinion surveys, IRT models that can handle the ordinal responses and thoroughly explore the available information provided by the item batteries and covariates are not widely used in the social sciences; standard models without covariates have been applied, for example, by Duck-Mayr and Montgomery (2023) or Hill and Tausanovitch (2015).

In psychometrics, multidimensional item response models have been proposed that allow the inclusion of not only attitude parameters but also particular response style effects, which are also relevant to the model proposed in the present paper. Models for the analysis of rating data from multiple scales can be based on Bock’s nominal response model (Bock 1972). The link between the basic nominal model (without covariates) has been investigated, for example, by Huggins-Manley and Algina (2015) and Thissen et al. (2010).

Various models that incorporate response style effects and are derived from Bock’s model have been considered by Bolt and Johnson (2009), Falk and Cai (2016), Suh and Bolt (2010), Bolt and Newton (2011), or Bolt and Johnson (2009). However, the considered models do not include covariate vectors. Response style effects are also addressed by so-called IRTrees models (Böckenholt 2017; Böckenholt and Meiser 2017; De Boeck and Partchev 2012; Jeon and De Boeck 2016) but without the inclusion of covariates. Alternative approaches to modeling response styles have been proposed within the framework of latent class models (see, e.g., Moors 2003; Morren et al. 2011).

The effect of covariates is an explicit topic in the explanatory item response modeling (EIRM) approach (De Boeck and Wilson 2004, 2016). Within this framework, various models, including polytomous response models, have been proposed; see, in particular Tuerlinckx (2004). The approach presented here can be seen as an explanatory item response model with a specific, novel parameterization that enables the separation of attitude direction and strength.

The effect of covariates can also be modeled within the general framework of structural equation modeling (SEM); see, for example, Bartholomew et al. (2011) or Skrondal and Rabe-Hesketh (2004). SEM approaches have their roots in factor analysis and path analysis. They are typically multidimensional, involving several latent variables, and are available in program packages such as Mplus (Muthén and Muthén 2023) or mirt (Chalmers et al. 2025). The objective is to study the causal relationship between these latent variables and responses. The model considered here represents an alternative approach to modeling the impact of covariates, utilizing estimation methods and specific parameterizations that have not been employed in SEM.

The statistical models presented here draw on the contribution by Schauberger and Tutz (2022), who introduced multivariate ordinal regression models within the framework of random effects and response styles in the statistical literature. While that work provides the statistical foundations, developing a taxonomy of ordinal models that include subject-specific random effects and response-style components, our contribution adds a substantive layer by interpreting these models through attitudinal constructs from social psychology.

By embedding the ordinal IRT modeling approach with covariates within a substantive conceptual framework informed by the social psychology literature (e.g., Howe and Krosnick 2017; Petty et al. 2023), we demonstrate how it can yield nuanced insights into how individuals differ in the ways they think or feel about topics concerning society and what population subgroups hold stronger or weaker attitudes than others. Without requiring additional survey items, the model captures and separates variation in two key attitude-related dimensions: direction and strength. Attitude direction refers to the substantive stances different individuals have about topics or persons, while attitude strength reflects the intensity with which that stance is held. Such interpretation enables practitioners to address their social science questions about heterogeneity in attitudes from a substantively conceptual perspective.

In contrast to Schauberger and Tutz (2022), we also engage in a discussion and comparison with alternative approaches commonly used in the social sciences, as well as testing models with differing numbers of latent variables. The model estimation is computationally efficient as it is based on convenient maximum likelihood methods. We provide an in-depth discussion of parameter interpretation tailored to the needs of applied social science researchers and outline a straightforward way to jointly visualize the covariate effects for attitude direction and attitude strength. The approach is quite general and can be applied to many different domains to test theoretical expectations about heterogeneity. We exemplify the types of insights that can be obtained with two applications using standard mass public opinion surveys: attitudes toward gender equality (European Values Study, EVS) and the evaluation of presidential candidates (American National Election Study, ANES).

Section “Ordinal IRT Modeling and Adjacent Categories” sets the methodological ground for the ordinal item response model we propose to study heterogeneity in attitudes. Section “Modeling Variation in Attitudes” outlines the model and its substantive motivation, discusses parameter estimation, and investigates differences with alternative approaches. Section “Applications” presents the applications. Section “Conclusion” concludes.

Ordinal IRT Modeling and Adjacent Categories

IRT models that account for ordinal data are multivariate versions of ordinal regression (see Agresti 2010; Tutz 2022). Ordinal IRT modeling in public opinion research is based on the following information: $P$ persons evaluate $I$ items (statements linked to the latent trait) with the response format of graded scales, which are ordered categories ${0, 1, \dots, k}$ . The graded scales indicate the person’s level of (dis)agreement with respect to the value statement.¹ Various versions of such graded scales have become standard tools to measure attitudes since they were introduced by Likert (1932). The grades are symmetrical, and the number of grades can be odd or even. The most popular ones are five-graded and four-graded scales. Five-graded scales come with a neutral (undecided) middle category and are typically interpreted as ‘‘strongly disagree,” ‘‘disagree,” ‘‘neither disagree nor agree,” ‘‘agree,” ‘‘strongly agree,” whereas four-graded scales force the respondents to state a tendency to agree or disagree.

Several ordinal item response models have been proposed in the literature (for taxonomies of models, see Thissen and Steinberg 1986; Tutz 2020).² Our approach to understanding variation in attitudes builds on the partial credit model (PCM). Since the model was proposed by Masters (1982) and Masters and Wright (1984), it has been applied to a wide range of practical measurement problems in psychometrics that demonstrated its usefulness in various contexts (for an overview, see Masters 2016).

The partial credit model is the most general member of the Rasch family of latent trait models (Rasch 1960, 1961), which can be derived from the assumption of a latent continuous random variable. It belongs to the class of adjacent-categorical models whose defining property is a conditional comparison between two adjacent (immediate preceding or following) response categories. The simplicity of the partial credit model lies in this defining characteristic of using pairs of adjacent categories as it assumes that the preference is in response category $r$ under the condition that the preference is in two adjacent categories ${r - 1, r}, r = 1, \dots, k$ . Thus, based on the simple Rasch model of dichotomies, it comes with all the distinguishing features and merits of the Rasch model family.

An alternative model is Samejima (1995)’s graded response model, which uses cumulative response categories. Whereas under adjacent-categorical models the effects pertain to individual categories, they refer to the entire scale under cumulative models, usually yielding larger effect magnitudes (see Agresti 2010). Previous research showed that including covariates is much easier to obtain for the partial credit model than the graded response model, which involves a considerably more complex parameterization and, therefore, way more sophisticated interpretation (Schauberger and Tutz 2022). The graded response model also tends to suffer from estimation problems (with substantially longer computation) because the modeling of cumulative probabilities comes with restrictions on the range of permissible parameter values, and the restriction gets more severe with the inclusion of covariates. Therefore, we draw on extensions of the partial credit model that account for covariate effects to provide a computationally straightforward approach that can be easily applied and interpreted.

We first present the basic version of the partial credit model to outline its underlying conditional structure and implications for parameter interpretation. Then, we discuss specific model properties that are desirable in empirical applications. These properties remain under the extensions to model individual-level variation in attitudes we present in Section “Modeling Variation in Attitudes”. Since the partial credit model is quite new to the discipline, it is worth discussing these issues in detail before introducing the extensions.

The Basic Partial Credit Model

Let $Y_{p i}$ denote the response of person $p$ on item $i$ and the data be given by $Y_{p i} \in {0, 1, \dots, k}$ , $p = 1, \dots, P$ , $i = 1, \dots, I$ , with ${0, 1, \dots, k}$ representing ordered categories. The probabilities of the basic partial credit model are given by

P (Y_{p i} = r) = \frac{\exp (\sum_{l = 1}^{r} (θ_{p} - δ_{i l}))}{\sum_{s = 0}^{k} \exp (\sum_{l = 1}^{s} (θ_{p} - δ_{i l}))}, r = 1, \dots, k,

(1)

where we define

\sum_{k = 1}^{0} (θ_{p} - δ_{i k}) = 0

for notational convenience. The model comes with two types of parameters that are measured on the same latent attitude scale:

the person location parameter $θ_{p}$ gives the attitude (location) of person $p$ on the scale;

the item parameters $δ_{i r}$ , which are category-specific thresholds, indicate the location of item $i$ on the scale.³

The probabilities for the simplest model with three response categories ( $k = 2$ ) are

\begin{aligned} P (Y_{p i} = 0) = \frac{1}{1 + \exp (θ_{p} - δ_{i 1}) + \exp ((θ_{p} - δ_{i 1}) + (θ_{p} - δ_{i 2}))}, \\ P (Y_{p i} = 1) = \frac{\exp (θ_{p} - δ_{i 1})}{1 + \exp (θ_{p} - δ_{i 1}) + \exp ((θ_{p} - δ_{i 1}) + (θ_{p} - δ_{i 2}))}, \\ P (Y_{p i} = 2) = \frac{\exp ((θ_{p} - δ_{i 1}) + (θ_{p} - δ_{i 2}))}{1 + \exp (θ_{p} - δ_{i 1}) + \exp ((θ_{p} - δ_{i 1}) + (θ_{p} - δ_{i 2}))} . \end{aligned}

An alternative, simpler presentation of the model is based on local odds that compare the adjacent categories ${r - 1, r}$

\log (\frac{P (Y_{p i} = r)}{P (Y_{p i} = r - 1)}) = θ_{p} - δ_{i r}, \frac{P (Y_{p i} = r)}{P (Y_{p i} = r - 1)} = \exp (θ_{p} - δ_{i r}), r = 1, \dots, k .

(2)

For a three-graded scale (

k = 2

), one obtains

\log (\frac{P (Y_{p i} = 1)}{P (Y_{p i} = 0)}) = θ_{p} - δ_{i 1}, \log (\frac{P (Y_{p i} = 2)}{P (Y_{p i} = 1)}) = θ_{p} - δ_{i 2} .

It is immediately seen from Equation (2) that

P (Y_{p i} = r) = P (Y_{p i} = r - 1)

holds if

θ_{p} = δ_{i r}

. However, the presentation obfuscates that the model is a conditional model. A form that clarifies the conditional nature, which is crucial for parameter interpretation, is

\begin{aligned} \log (\frac{P (Y_{p i} = r | Y_{p i} \in {r - 1, r})}{P (Y_{p i} = r - 1 | Y_{p i} \in {r - 1, r})}) & = θ_{p} - δ_{i r}, \\ \frac{P (Y_{p i} = r | Y_{p i} \in {r - 1, r})}{P (Y_{p i} = r - 1 | Y_{p i} \in {r - 1, r})} & = \exp (θ_{p} - δ_{i r}), r = 1, \dots, k . \end{aligned}

Then, it is straightforward to see that conditionally, given

Y_{p i} \in {r - 1, r}

, a Rasch model holds,

P (Y_{p i} = r | Y_{p i} \in {r - 1, r}) = \frac{\exp (θ_{p} - δ_{i r})}{1 + \exp (θ_{p} - δ_{i r})} .

(3)

Therefore, the item parameters

δ_{i r}

can be interpreted as the location parameters of a local (conditional) Rasch model.

Specific Properties

The partial credit model has several specific properties that make it attractive in empirical applications. A desirable property of ordinal item response models is specific objectivity, which separates the person location parameter $θ_{p}$ from the item parameters $δ_{i r}$ , allowing a straightforward comparison of items and individuals. In particular, the property of specific objectivity permits a comparison of items independent from the specific individual using the scale and a comparison of individuals independent from the specific item evaluated. The property is seen when considering the local odds for any two items $i_{1}, i_{2}$

\log (\frac{P (Y_{{p i}_{1}} = r) / P (Y_{{p i}_{1}} = r - 1)}{P (Y_{{p i}_{2}} = r) / P (Y_{{p i}_{2}} = r - 1)}) = - (δ_{i_{1} r} - δ_{i_{2} r}) .

Thus, the comparison of any two items is independent from

θ_{p}

. Likewise, the local odds for any two persons

p_{1}, p_{2}

do not depend on

δ_{i r}

\log (\frac{P (Y_{p_{1} i} = r) / P (Y_{p_{1} i} = r - 1)}{P (Y_{p_{2} i} = r) / P (Y_{p_{2} i} = r - 1)}) = θ_{p_{1}} - θ_{p_{2}} .

Another crucial feature relates to the ordering of item parameters

δ_{i r}

, which has caused some confusion in the literature where it was claimed that

δ_{i 1} \leq \dots \leq δ_{i k}

needs to hold (see Andrich 2013). However, due to its conditional parameterization, the item parameters do not have to follow the ordering of the response categories to obtain a sensible model (see Adams et al. 2012; Tutz 2020). This convenient feature distinguishes the partial credit model from the graded response model. The constraint that item parameters have to be ordered is always present in the graded response model as it considers all categories simultaneously, which restricts the permissible range of parameter values, yielding potential estimation problems.

Even though the basic partial credit model measures attitudes and comes with desirable properties, it does not account for variability in attitudes across individuals or population subgroups. Next, we consider model extensions that allow investigating how attitudes and their strength depend on covariates and embed them into a substantive conceptual interpretation.

Modeling Variation in Attitudes

We aim to include covariates to model individual-level variation in attitudes and their strengths based on a simple parameterization that permits a straightforward substantive interpretation without the need for extra survey questions on the intensity of the feeling.

Attitudes, defined as ‘‘mental representations’’ of what people favor or oppose (Bizer et al. 2003: p. 247), are complex latent concepts with many different properties or attributes that have evolved throughout its longstanding research history in social psychology (for recent reviews, see Howe and Krosnick 2017; Petty et al. 2023). Petty and Krosnick (1995: pp. 5–6) proposed in their influential edited volume four dimensions to categorize the manifold attributes of attitudes: ones that refer to (i) the attitude itself (e.g., extremity), (ii) the cognitive structure (e.g., attitude accessibility or ambivalence), (iii) to the subjective beliefs (e.g., personal involvement or attitude certainty), and (iv) the cognitive processes of attitude formation (e.g., elaboration).⁴

We aim to capture the first, potentially most essential, dimension that refers to ‘‘aspects of the attitude itself’’ (Petty and Krosnick 1995: pp. 5–6) defined as:

“Attitudes are presumed to vary along an evaluative continuum ranging from a strongly positive orientation to a neutral orientation to a strongly negative orientation. This continuum can be decomposed into valence (i.e., positive or negative) and extremity (degree of favorability).”

Drawing on this conceptualization, we present a parameterization that incorporates the two components and their variation across individuals so that it allows disentangling what covariates help explain each of the components and the relations among them.

The first component (positive or negative valence), also referred to as attitude direction (e.g., Krosnick and Abelson 1992: p. 178), is captured by modeling the level of (dis)agreement as a function of covariates. The resulting covariate effects permit understanding how individuals differ in their substantive stances, revealing what is favorable or unfavorable by whom and who is pro or con.

The second component (extremity) is typically operationalized as the deviation from the neutral scale midpoint (Abelson 1995; Krosnick and Abelson 1992). Our parameterization incorporates it by modeling variation in individual tendencies to middle or extreme response categories. We call this component attitude strength⁵ because the parameterization captures the spectrum between weak and strong attitudes as follows: individuals with weak attitudes (which can also be understood as nonattitudes) tend to choose neutral middle categories. By contrast, individuals with strong attitudes tend to choose extreme categories (e.g., ‘‘strongly agree’’ or ‘‘strongly disagree’’). We link these individual tendencies to covariates as well. The resulting covariate effects allow for uncovering how individuals differ in the strength or intensity of their substantive stances and what population subgroups hold stronger or weaker attitudes than others.

It is helpful to introduce the extensions to incorporate attitude direction and attitude strength step by step to the basic partial credit model in Equation (1); this is what we do next. Then, we present the comprehensive model with both components and discuss parameter estimation. The section closes by investigating differences with the hierarchical approach proposed by Zhou (2019).

Variation in Attitude Direction: Location Effects

Let $x_{p}^{T} = (x_{p 1}, \dots, x_{p m})$ denote a vector of covariates that characterize the individuals. The basic partial credit model can be extended to account for variation in attitude direction linked to covariates by adding the term $x_{p}^{T} γ$ to the person location parameter $θ_{p}$ , yielding

\log (\frac{P (Y_{p i} = r | x_{p})}{P (Y_{p i} = r - 1 | x_{p})}) = θ_{p} + x_{p}^{T} γ - δ_{i r}, r = 1, \dots, k,

(4)

where the parameter vector

γ^{T} = (γ_{1}, \dots, γ_{m})

contains the effects. The term

x_{p}^{T} γ

specifies the location effects of covariates; it increases or decreases the basic location

θ_{p}

on the latent scale. It should be noted that the inclusion of the term

x_{p}^{T} γ

modifies the latent trait. Heterogeneity is still primarily captured by the latent trait, but covariates serve to explain part of this heterogeneity. Thus, the parameters

θ_{p}

in the extended model represent residual heterogeneity after accounting for covariates.

The interpretation of the covariate effects is seen by considering the local odds

κ_{p i r} (x_{p}) = \frac{P (Y_{p i} = r | x_{p})}{P (Y_{p i} = r - 1 | x_{p})} = \frac{P (Y_{p i} = r | Y_{p i} \in {r - 1, r}, x_{p})}{P (Y_{p i} = r - 1 | Y_{p i} \in {r - 1, r}, x_{p})},

which are the odds of the corresponding local Rasch model. When covariate

x_{p j}

(

j

-th component of vector

x_{p}

) increases by one unit to

x_{p j} + 1

while all other variables are kept fixed, the local odds change by the factor

e^{γ_{j}}

\frac{κ_{p i r} (x_{p 1}, \dots, x_{p j} + 1, \dots, x_{p m})}{κ_{p i r} (x_{p 1}, \dots, x_{p m})} = e^{γ_{j}} .

(5)

Thus,

e^{γ_{j}}

give the relative odds that compare the local odds for categories

{r - 1, r}

. These multiplicative effects do not depend on a specific local comparison but affect all local odds equally. Let low response categories indicate ‘‘disagreement’’ and high response categories ‘‘agreement.’’ When

e^{γ_{j}} > 1

(

\equiv γ_{j} > 0

), higher (agreement) categories are more likely; when

e^{γ_{j}} < 1

(

\equiv γ_{j} < 0

), lower (disagreement) categories are more likely.

Variation in Attitude Strength: Scaling Effects

The variation in attitude strength is captured by the widening or shrinkage of thresholds referred to as scaling. Weak attitudes are linked to the tendency to middle categories (widening of thresholds). Strong attitudes are linked to the tendency to extreme categories (shrinkage of thresholds). In the following, we first add a term for individual scaling to explain the logic of the parameterization; then, we add covariates. For simplicity, we ignore the location effects term $x_{p}^{T} γ$ for the moment.

Individual Scaling

The model that incorporates individual scaling is given by

\log (\frac{P (Y_{p i} = r)}{P (Y_{p i} = r - 1)}) = θ_{p} + s_{r} {\tilde{θ}}_{p} - δ_{i r}, r = 1, \dots, k,

(6)

where

{\tilde{θ}}_{p}

is the person-specific scaling parameter, and

s_{r} = (k / 2 - r + 0.5)

is a simple factor that causes the probabilities for either middle or extreme categories to increase, depending on the sign of

{\tilde{θ}}_{p}

Table 1 illustrates how the factor $s_{r}$ scales ${\tilde{θ}}_{p}$ by shifting the item parameters $δ_{i r}$ . The left part demonstrates the calculation of the linear predictor, and the right part shows the shifted item parameters for odd and even numbers of grades. In the simplest case with three response categories ( $k = 2$ ), ${0}$ is the single disagreement category, ${1}$ is the middle category meaning ‘‘undecided,’’ and ${2}$ is the agreement category. When $k = 3$ , there is no explicit middle category; ${0, 1}$ are the disagreement and ${2, 3}$ the agreement categories. The last example is a five-graded scale ( $k = 4$ ) with the disagreement categories ${0, 1}$ , the middle category ${2}$ , and the agreement categories ${3, 4}$ .

Table 1.

Individual scaling for ordered response categories ${0, 1, \dots, k}$ .

	$θ_{p} + (k / 2 - r + 0.5) {\tilde{θ}}_{p}$
$r = 1$ $r = 2$	$θ_{p} + 0.5 {\tilde{θ}}_{p}$ $θ_{p} - 0.5 {\tilde{θ}}_{p}$

(b) Four-graded scale ( $k = 3$ )
$θ_{p} + (k / 2 - r + 0.5) {\tilde{θ}}_{p}$
$r = 1$	$θ_{p} + {\tilde{θ}}_{p}$
$r = 2$	$θ_{p}$
$r = 3$	$θ_{p} - {\tilde{θ}}_{p}$
(c) Five-graded scale ( $k = 4$ )
$θ_{p} + (k / 2 - r + 0.5) {\tilde{θ}}_{p}$
$r = 1$	$θ_{p} + 1.5 {\tilde{θ}}_{p}$
$r = 2$	$θ_{p} + 0.5 {\tilde{θ}}_{p}$
$r = 3$	$θ_{p} - 0.5 {\tilde{θ}}_{p}$
$r = 4$	$θ_{p} - 1.5 {\tilde{θ}}_{p}$

Note: Left part illustrates the calculation of the linear predictor; $θ_{p}$ is the person location parameter and ${\tilde{θ}}_{p}$ the person scaling parameter. Right part shows the shifted item parameters $δ_{i r}$ .

It is seen from Table 1 that $s_{r} = (k / 2 - r + 0.5)$ scales ${\tilde{θ}}_{p}$ so that the differences between the item parameters of adjacent categories change in a symmetric way, which does not imply that the items parameters need to be ordered. When ${\tilde{θ}}_{p}$ is positive, the differences increase, yielding a concentration of the probability mass in the middle. When ${\tilde{θ}}_{p}$ is negative, the differences decrease, causing the probability mass is shifted to the extreme categories.

Table 2.

Summary of items and response categories.

(a) Item Battery on Attitudes Toward Gender Equality (EVS)
When a mother works for pay, the children suffer.
A job is alright but what most women really want is a home and children.
All in all, family life suffers when the woman has a full-time job.
A man’s job is to earn money; a woman’s job is to look after the home and family.
On the whole, men make better political leaders than women do.
A university education is more important for a boy than for a girl.
On the whole, men make better business executives than women do.

‘‘Strongly disagree,’’ ‘‘disagree,’’ ‘‘agree,’’ ‘‘strongly agree.’’

(b) Item Battery on Evaluation of Presidential Candidate Traits (ANES)
Candidate provides strong leadership.
Candidate really cares about people like you.
Candidate is knowledgeable.
Candidate is honest.
Candidate speaks mind.
Candidate is even-tempered.

‘‘Not all,’’ ‘‘slightly,’’ ‘‘moderately,’’ ‘‘very,’’ ‘‘extremely.’’

In particular, when ${\tilde{θ}}_{p} > 0$ (positive scaling), the term $θ_{p} + (k / 2 - r + 0.5) {\tilde{θ}}_{p}$ becomes larger for lower (disagreement) categories and smaller for higher (agreement) categories. The result is that the probabilities for middle (undecided) categories increase because the item parameters of the disagreement and the agreement categories move apart. The item parameters of the disagreement categories are shifted to the left and the ones for the agreement categories to the right so that the probability mass in the middle categories increases. Thus, individuals who tend to select middle categories show weak attitudes. The larger the value of ${\tilde{θ}}_{p}$ , the weaker the attitude.

When ${\tilde{θ}}_{p} < 0$ (negative scaling), the probabilities for extreme categories increase because the term $θ_{p} + (k / 2 - r + 0.5) {\tilde{θ}}_{p}$ becomes smaller for lower (disagreement) categories and larger for higher (agreement) categories. The item parameters of the disagreement and the agreement categories move toward each other, shifting the probability mass to the extreme categories $0$ and $k$ . Thus, individuals who tend to choose extreme categories show strong attitudes. The smaller the value of ${\tilde{θ}}_{p}$ , the stronger the attitude.

Figure 1 visualizes how the sign of the person scaling parameter ${\tilde{θ}}_{p}$ determines the preference for categories by plotting the probabilities for three response categories (‘‘disagree,’’ ‘‘undecided,’’ ‘‘agree’’) against the person location parameter $θ_{p}$ . Panel (a) depicts the probability curves for ${\tilde{θ}}_{p} = 2$ . It illustrates positive scaling (weak attitude): the item parameters of the ‘‘disagree’’ and ‘‘agree’’ categories move away from each other, increasing the probability for the ‘‘undecided’’ category. Thus, positive scaling means middle categories have much larger probabilities. Panel (b) uses ${\tilde{θ}}_{p} = - 2$ and illustrates negative scaling (strong attitude). Here, the item parameters move toward each other, increasing the probabilities for ‘‘disagree’’ and ‘‘agree’’ and decreasing the one for ‘‘undecided.’’ Therefore, negative scaling means large probabilities in extreme categories.

Figure 1.

Visualization of positive and negative scaling for three-graded scale. (a) Positive scaling ( ${\tilde{θ}}_{p} > 0$ ): weak attitude and (b) negative scaling ( ${\tilde{θ}}_{p} < 0$ ): strong attitude. Note: Probabilities $P (Y_{p i} = r)$ for three response categories (‘‘disagree,’’ ‘‘undecided,’’ ‘‘agree’’) are plotted against the person location parameter $θ_{p}$ . Dashed vertical lines indicate the shifted item parameters.

It is important to note that the parameterization in Equation (6) yields specific traits such that the second dimension ${\tilde{θ}}_{p}$ refers to a tendency to middle or extreme categories. It means, in particular, that the model is not parametrically equivalent to a standard two-trait PCM model.

Linking Individual Scaling to Covariates

Next, we link individual scaling indicating the attitude strength to covariates by adding the term $x_{p}^{T} α$ to the person scaling parameter ${\tilde{θ}}_{p}$ . The corresponding model is

\log (\frac{P (Y_{p i} = r | x_{p})}{P (Y_{p i} = r - 1 | x_{p})}) = θ_{p} + s_{r} ({\tilde{θ}}_{p} + x_{p}^{T} α) - δ_{i r}, r = 1, \dots, k,

(7)

where the parameter vector

α^{T} = (α_{1}, \dots, α_{m})

contains the effects. The term

x_{p}^{T} α

specifies the scaling effects of covariates; it represents the attitude strength linked to the tendency to middle or extreme categories as a function of covariates. Again, the effects are best seen by considering the impact on the local odds when covariate

x_{p j}

increases by one unit

\frac{κ_{p i r} (x_{p 1}, \dots, x_{p j} + 1, \dots, x_{p m})}{κ_{p i r} (x_{p 1}, \dots, x_{p m})} = e^{s_{r} α_{j}} .

(8)

When

e^{α_{j}} > 1

(

\equiv α_{j} > 0

), middle (undecided) categories are more likely, indicating weaker attitudes. When

e^{α_{j}} < 1

(

\equiv α_{j} < 0

), extreme categories are more likely, therefore, indicating stronger attitudes.

The dimensions we consider and link to covariates are the attitude direction and attitude strength, derived from the concepts of valence and extremity proposed by Petty and Krosnick (1995: pp. 5–6). The second dimension, attitude strength, is strongly related to response styles, which have been modeled in particular within the framework of IRT models (Böckenholt 2017; Böckenholt and Meiser 2017; De Boeck and Partchev 2012; Jeon and De Boeck 2016) but typically without the inclusion of covariates. Explanatory item response models, as proposed in De Boeck and Wilson (2004), include covariates but do not model extremity of responses in the way proposed here.

The Comprehensive Model and Parameter Estimation

The comprehensive model with location and scaling effects depending on covariates is

\log (\frac{P (Y_{p i} = r | x_{p})}{P (Y_{p i} = r - 1 | x_{p})}) = θ_{p} + x_{p}^{T} γ + s_{r} ({\tilde{θ}}_{p} + x_{p}^{T} α) - δ_{i r}, r = 1, \dots, k,

(9)

where the covariates that enter the location and scaling terms can be the same, overlapping, or different. The corresponding local odds are

\frac{κ_{p i r} (x_{p 1}, \dots, x_{p j} + 1, \dots, x_{p m})}{κ_{p i r} (x_{p 1}, \dots, x_{p m})} = e^{γ_{j}} e^{s_{r} α_{j}} .

(10)

Covariate effects ( $γ$ and $α$ ) and item parameters $δ_{i r}$ can be estimated with the marginal maximum likelihood (MML) procedure, which relies on standard distributional assumptions for the person location parameter $θ_{p}$ to achieve identifiability (see, e.g., Muraki 1992). We do the same for the person scaling parameter ${\tilde{θ}}_{p}$ to estimate their (co)variances. Let $f (θ_{p}, {\tilde{θ}}_{p})$ be the density of the assumed distribution. The typical assumption is a normal distribution with mean zero and covariance $Σ$ , indicated by $f_{0, Σ} (.)$ . Let the responses $Y_{p i} \in {0, 1, \dots, k}$ be represented by (0-1) indicator variables, defined by

Y_{p i r} = {\begin{matrix} 1 & if Y_{p i} = r \\ 0 & if Y_{p i} \neq r, \end{matrix}

yielding

k - 1

binary variables that directly correspond to the ordinal response

Y_{p i} = r \Leftrightarrow Y_{p i}^{T} = (Y_{p i 0}, \dots, Y_{p i k}) = (0, \dots, 0, 1, 0, \dots, 0),

with the vector

Y_{p i}

containing only a single 1 entry.

The marginal likelihood is given by

L (γ, α, {δ_{i r}}) = \prod_{p = 1}^{P} \int \prod_{i = 1}^{I} \prod_{r = 1}^{k} P (Y_{p i} = r)^{y_{p i r}} f_{0, Σ} (θ_{p}, {\tilde{θ}}_{p}) d θ_{p} d {\tilde{θ}}_{p},

(11)

where

y_{p i r} = 1

Y_{p i} = r

and

y_{p i r} = 0

otherwise. The corresponding marginal log-likelihood has the form

l (γ, α, {δ_{i r}}) = \sum_{p = 1}^{P} \log (\int \prod_{i = 1}^{I} \prod_{r = 1}^{k} P (Y_{p i} = r)^{y_{p i r}} f_{0, Σ} (θ_{p}, {\tilde{θ}}_{p}) d θ_{p} d {\tilde{θ}}_{p}) .

(12)

Maximization can be obtained by the expectation-maximization (EM) algorithm (e.g., Bock and Aitkin 1981) or numerical integration such as Gauss–Hermite quadrature (e.g., Liu and Pierce 1994); for more details, see Schauberger and Tutz (2022).⁶ The models can be fitted using the R package MultOrdRS (Schauberger 2024). Users can choose between two optimization algorithm: L-BFGS-B and nlminb. By default, the Gauss–Hermite quadrature uses 10 nodes per dimension, and a tuning parameter for internal ridge penalty (set to a very small value) is applied to stabilize the estimates.

Alternative Approach

Alternative models that account for heterogeneity beyond location effects have been considered by Zhou (2019) who uses a hierarchical approach to extend graded response and partial credit models. A discussion will illustrate the differences in parameterization and the benefits of our approach to model variation in attitudes.⁷

Zhou (2019) builds on the basic partial credit model (see Equation (1)) with the linear predictor⁸ $η_{p i r} = θ_{p} - δ_{i r}$ and assumes that the person location parameter $θ_{p}$ follows a normal distribution: $θ_{p} \sim N (μ_{p}, σ_{p}^{2})$ . Drawing on standard heteroscedastic regression⁹ (see Harvey 1976; Verbyla 1993), it is assumed that the mean $μ_{p}$ has a linear form, the variance $σ_{p}^{2}$ has a log-linear form, and both are functions of covariates $x_{p}$ ,

μ_{p} = γ_{0} + x_{p}^{T} γ, \log σ_{p}^{2} = α_{0} + x_{p}^{T} α .

Zhou’s model and our approach are equivalent in how location effects are specified but differ in modeling individual scaling. It is helpful to rewrite Zhou’s model to clarify the difference in parameterization. An alternative representation is

\log (\frac{P (Y_{p i} = r)}{P (Y_{p i} = r - 1)}) = γ_{0} + x_{p}^{T} γ + θ_{p}^{(0)} e^{x_{p}^{T} α / 2} - δ_{i r}, r = 1, \dots, k,

(13)

with

θ_{p}^{(0)} \sim N (0, e^{α_{0}})

. The representation in Equation (13) is obtained by using the reparameterization

θ_{p} = γ_{0} + x_{p}^{T} γ + θ_{p}^{(0)} e^{x_{p}^{T} α / 2}

, which yields

E (θ_{p}) = γ_{0} + x_{p}^{T} γ = μ_{p}

and

v a r (θ_{p}) = e^{α_{0} + x_{p}^{T} α} = σ_{p}^{2}

. It is seen that the partial credit model considered by Zhou is equivalent to our model if only location effects are present (i.e.,

α = 0

However, the model differs from the model propagated here with respect to the additional heterogeneity. Zhou’s approach is a one-dimensional person parameter model; it does not include a separate parameter that accounts for scaling effects. Thus, it does not account for the heterogeneity of individuals concerning the tendency to middle or extreme categories. It can only detect which individuals (or population subgroups) show a high, low, or no variance, a rather unstructured variation with no individual effects. By contrast, our parameterization comes with two person parameters, one for the location ( $θ_{p}$ ) and one for the scaling ( ${\tilde{θ}}_{p}$ ) that specifies the attitude strength linked to the tendency to middle or extreme categories. Thus, it offers more fine-graded insights as it models both variations in direction and strength as individual effects embedded into a substantive conceptual interpretation. It also permits investigating the correlation between attitude direction and strength. For more details on the model properties, see Appendix C in the online supplement.

The comparison of the approach propagated here and Zhou’s model illustrates the various ways covariates can be incorporated into a two-trait PCM model. Covariates can directly impact item responses, covariates can fully determine the latent traits, or both strategies can be combined. The model presented here is of the latter type: latent traits are replaced by latent traits that are combined with the effects of covariates. The heterogeneity in the form of latent traits is kept and effects are included that directly influence responses. Zhou’s approach is of the second type: latent traits, more accurately, the parameters that determine the distribution of latent traits are determined by covariates.

It should be mentioned that approximate estimates might be obtained by using the package LatentGOLD (Vermunt and Magidson 2025); see Appendix E in the online supplement for syntax.

Applications

We demonstrate the substantive insights our approach can obtain with two standard public opinion surveys that contain typical item batteries to measure attitudes. The first application investigates attitudes toward gender equality using data from the European Values Study 2017 (EVS 2022). The second uses data from the American National Election Study 2016 (ANES 2017) and analyzes the evaluation of presidential candidates.

The applications differ in the number of items and response categories (summarized in Table 2, see Appendix A in the online supplement for details). The response format for the seven-item battery on statements that tap gender equality attitudes is a four-graded scale without a neutral middle category. Higher categories indicate opposing gender equality and lower ones favoring gender equality. The six-item battery on candidate traits comes with a five-graded scale, running from ‘‘not at all’’ to ‘‘extremely’’ with the middle category‘‘moderately.’’ Higher (lower) categories correspond with positive (negative) evaluations.

Application: Attitudes Toward Gender Equality

Since the 1970s, a growing research body reports gender differences in public opinion, campaigning, or voting, demonstrating the significance of gender equality attitudes (e.g., Fox and Lawless 2011; Oshri et al. 2023; Pérez and Tavits 2019; Schaffner 2005). We investigate variation in gender equality attitudes in three European countries with considerable differences in the EU Gender Equality Index 2023 (EIGE 2023): Sweden, Germany, and Hungary. Sweden occupies the first rank (82.2 points out of 100). Germany (70.8) is ranked slightly above the EU average score of 70.2, and Hungary (57.3) occupies the second-last place before Romania. As covariates, we consider standard demographics: gender, age, income, and urban–rural location. The models were fitted separately for each country.

Model Comparisons

We test the presented extensions of the partial credit model by estimating four models. The first is the basic version. The second adds location effects to account for variation in attitude direction. The third adds individual scaling to incorporate variation in attitude strength. The fourth is the comprehensive model with covariates for both location and scaling effects. The basic models have 22 degrees of freedom (df): 21 item parameters ${\hat{δ}}_{i r}$ (7 items with $4 - 1$ response categories each) and the estimated variance of the person location parameter $θ_{p}$ , ${\hat{σ}}_{θ_{p}}^{2}$ . The first extension with location effects comes with one additional parameter per covariate (male, age, income, town size). The models adding individual scaling also have the estimated (co)variance matrix $\hat{Σ}$ , yielding two additional estimates.¹⁰ The comprehensive models come with four additional parameters (four covariates). Panel (a) in Table 3 summarizes the parameterizations of the estimated partial credit models.

Table 3.

Comparisons of empirical models.

(a) Summary of Parameterizations
Model Structure:
$P (Y_{p i} = r \| Y_{p i} \in {r - 1, r}) = F (η_{p i r}) \Leftrightarrow \log (\frac{P (Y_{p i} = r \| Y_{p i} \in {r - 1, r})}{P (Y_{p i} = r - 1 \| Y_{p i} \in {r - 1, r})}) = η_{p i r}$
Basic Partial Credit Model	$η_{p i r} = θ_{p} - δ_{i r}$
with Location Effects	$η_{p i r} = θ_{p} + x_{p}^{T} γ - δ_{i r}$
with Location & Individual Scaling	$η_{p i r} = θ_{p} + x_{p}^{T} γ + s_{r} {\tilde{θ}}_{p} - δ_{i r}$
with Location & Scaling Effects	$η_{p i r} = θ_{p} + x_{p}^{T} γ + s_{r} ({\tilde{θ}}_{p} + x_{p}^{T} α) - δ_{i r}$

Note: Person location parameter $θ_{p}$ , item parameters $δ_{i r}$ , location effects $γ$ , person scaling parameter ${\tilde{θ}}_{p}$ , scaling effects $α$ , covariates $x_{p}$ .

(b) Performance of Empirical Models
		Sweden	Germany	Hungary
		( $N = 942$ )	( $N = 1267$ )	( $N = 907$ )
	df	Loglik.	Loglik.	Loglik.
Basic Partial Credit Model	22	$- 4445.41$	$- 8081.18$	$- 7058.72$
With Location Effects	26	$- 4378.12$	$- 7966.59$	$- 7013.38$
With Location & Individual Scaling	28	$- 4190.95$	$- 7560.51$	$- 6634.91$
With Location & Scaling Effects	32	$- 4183.11$	$- 7534.02$	$- 6628.31$
Zhou (2019)’s Graded Response Model	38	$- 4306.55$	$- 7836.02$	$- 6901.52$

(c) Information Criterion
	Sweden	Germany	Hungary
	AIC	AIC	AIC
Basic Partial Credit Model	8934.81	16206.36	14161.44
with Location Effects	8808.24	15985.17	14078.76
with Location & Individual Scaling	8437.89	15177.01	13325.83
with Location & Scaling Effects	8430.22	15132.05	13320.62
Zhou (2019)’s Graded Response Model	8689.10	15748.05	13879.04

(d) Likelihood Ratio Tests
		Sweden		Germany		Hungary
Null Hypotheses	df	$χ^{2}$	p-Val.	$χ^{2}$	p-Val.	$χ^{2}$	p-Val.
$H_{0}$ : $\hat{γ} = 0$	4	134.57	0.00	229.19	0.00	90.68	0.00
$H_{0}$ : $\hat{α} = 0$	4	15.67	0.00	52.97	0.00	13.21	0.01

Note: $H_{0} : \hat{γ} = 0$ : Basic Partial Credit Model vs. with Location Effects; $H_{0} : \hat{α} = 0$ : with Location & Individual Scaling vs. Location & Scaling Effects.

Panel (b) compares model performance based on the loglikelihood, and panel (c) reports the Akaike information criterion (AIC), including a comparison with Zhou (2019)’s graded response models (not nested in the previous models, see Section “Alternative Approache”). Comparing the parameterizations shows that model fit improves with each proposed extension. The performance measures suggest that the two-dimensional comprehensive models with attitude direction and strength provide better fits than the one-dimensional models with attitude direction only.

The comprehensive models also lead to better model fits than the graded response models, which have the same number of item and covariate parameters, two intercept estimates ( ${\hat{γ}}_{0}, {\hat{α}}_{0}$ , see Section “Alternative Approach”), and additionally contain seven discrimination parameters; thus, they are more complex. Even though the latter allow for unequal discrimination across items, the results suggest that our parameterization accounting for the direction and the strength better captures heterogeneity in gender equality attitudes.

Panel (d) presents likelihood ratio tests. The first tests operate within the one-dimensional models and contrast the basic models to ones that add location effects $\hat{γ}$ . The results demonstrate that the null hypothesis (no location effects) can be rejected for all three countries. Thus, the covariates help explain who favors and who opposes gender equality. The second tests operate within the two-dimensional models and evaluate the scaling effects $\hat{α}$ . The results reveal the comprehensive models accounting for location and scaling effects fit significantly better than the ones with location effects and individual scaling only. Again, the null hypothesis (no scaling effects) can be rejected for all three countries. Hence, considerable scaling effects are also present that should not be disregarded, and the covariates help to understand the differences in attitude strength.

One- and two-dimensional latent trait models are not compared by likelihood ratio testing. While likelihood ratio tests are formally appropriate when comparing nested models (i.e., when covariates are added), they require certain regularity conditions for their test statistic to follow a $χ^{2}$ distribution under the null hypothesis. However, these regularity conditions may be violated when the models under comparison differ in the number (or structure) of latent variables. In such cases, the null model may lie on the boundary of the parameter space, or contain parameters that are not identifiable under the null. As a result, the asymptotic distribution of the LR statistic may deviate from the standard $χ^{2}$ form and instead follow a nonstandard or mixed distribution (Chen et al. 2020; Drton 2009; Self and Liang 1987).¹¹

Next, we interpret the estimates of the comprehensive models and present an easy-to-grasp way to jointly visualize the covariate effects for attitude direction (location effects $γ$ ) and attitude strength (scaling effects $α$ ), their relative impacts and statistical significances. We use standardized covariates, which makes the effect sizes comparable and has the advantage that the effect presentation in the visualization tools we use is strongly improved. It should be noted that the model fit is the same for standardized and unstandardized estimates; standardization, achieved by dividing through the standard deviation, uses a different scaling of effects, whereas standardization by centering changes only the parameter estimates $δ_{i r}$ .

Effect Visualization and Interpretation

The covariate effects can be easily grasped by depicting the local odds in Equation (10) with the tuples ( $e^{γ}, e^{α}$ ), equipped with pointwise 95% confidence intervals (see Figure 2). The x-axis displays the location effect $e^{γ}$ for attitude direction: Values larger than 1 ( $\equiv γ > 0$ ) indicate opposing gender equality, values smaller than 1 ( $\equiv γ < 0$ ) favoring gender equality. The y-axis displays the scaling effect $e^{α}$ for attitude strength: Values larger than 1 ( $\equiv α > 0$ ) indicate weaker attitudes, values smaller than 1 ( $\equiv α < 0$ ) stronger attitudes. The dashed lines indicate null effects, $e^{γ} = 1$ and $e^{α} = 1$ . Thus, when the rays cross these lines, the effects are not significantly different from zero at the 5% significance level.

All location effects are significant, except town size in Sweden. The estimates are as one would expect: males and older people are more likely to oppose gender equality; those with higher income levels and residing in places with higher populations are more likely to favor gender equality. The effect sizes decrease in the same order as the rating in the gender equality index (except for age), indicating that the differences in attitude direction are more substantial the higher the country is ranked in the gender equality index. The significant scaling effects reveal that males hold weaker attitudes (or, reversely, females hold stronger attitudes) in Sweden and particularly in Germany, whereas there are no gender differences in attitude strength in Hungary. Older people are more likely to have weak (younger people are more likely to have strong) gender equality attitudes, notably in Sweden. Finally, there are only significant differences in attitude strength between urban and rural areas in Germany, suggesting stronger gender equality attitudes the higher the residence population.

Figure 2.

Attitudes toward gender equality: location and scaling effects. Note: Location effects $e^{γ}$ (attitude direction) on the x-axis and scaling effects $e^{α}$ (attitude strength) on the y-axis with 95% confidence intervals. All covariates are standardized.

The capability of star plots lies in their ability to clearly display effects and their significance. As shown in the images, the stars are approximately the same size. However, if covariates are not standardized, variables with a wider range, such as age and income, will produce very small stars. This occurs because the corresponding effects are diminished when covariates are not standardized.

For simplicity, we treated town size as an interval-scaled variable. We reran the model for Sweden with town size as a categorical variable, reported in Appendix D in the online supplement. It is seen that using a linear effect for town size, which is more parsimonious, yields a smaller AIC value (8430.22) than the one with town size as a categorical variable (8436.614). Additionally, the dummy-coded effects (with reference category 1) are monotone, which also supports that a linear effect specification might be a sufficiently good approximation.

Application: Evaluation of Candidate Traits

This section demonstrates the types of insights the approach can obtain in studying the evaluation of candidate traits: the attitudes of voters toward the candidate’s personality or character, also referred to as candidate image or valence (e.g., Franchino and Zucchini 2015; Mauerer and Tutz 2024; Stone and Simas 2010; Stone et al. 2004). The application uses the ANES’s traditional presidential candidate traits battery (see Table 2). We selected the 2016 election and analyze the Democratic candidate Hillary Clinton and the Republican candidate Donald Trump. Again, we include standard demographics as covariates: gender, age, self-identifications as African American or Latino, and education, and fitted separately for each candidate.

As for the first application, likelihood ratio tests demonstrate the null hypotheses ( $H_{0} : \hat{γ} = 0$ , $H_{0} : \hat{α} = 0$ ) can be rejected for both candidate evaluations: the models with location effects fit significantly better than the basic models, and the models with both location and scaling effects fit better than the ones with location effects and individual scaling only. The performance measures also suggest that the models fit way better for the Republican candidate Trump than the Democratic candidate Clinton, and attitude direction (valence) is more important in the evaluation of Clinton, whereas it is attitude strength in the evaluation of Trump.¹²

Figure 3 visualizes the location and scaling effects of the covariates. Almost all location effects indicating attitude direction are significant. Females, African Americans, Latinos, and higher educated are more likely to have a positive orientation (valence) toward Clinton. The results for the Republican candidate point in the opposite direction: the same subgroups are more likely to have a negative orientation toward Trump. These findings are indicative of markedly polarized attitudes. The largest differences in attitude direction are obtained for African Americans. In addition, older people are more likely to evaluate Trump positively, whereas there are no significant age differences in evaluating Clinton. Furthermore, all location effects are more considerable for Clinton than Trump, suggesting the subgroups that positively (or negatively) evaluate the candidates do so much more toward Clinton than Trump.

Figure 3.

Evaluation of candidate traits: location and scaling effects. Note: Location effects $e^{γ}$ (attitude direction) on the x-axis and scaling effects $e^{α}$ (attitude strength) on the y-axis with 95% confidence intervals. All covariates are standardized.

Most scaling effects indicating attitude strength are significant as well, except gender and age for Clinton. We also see pronounced polarization here and the reversed pattern as for the location effects. All scaling effects are larger for Trump than Clinton. All subgroups with a positive valence toward Clinton tend to hold weaker attitudes, and all subgroups with a negative valence toward Trump tend to hold very strong attitudes. Only older people are more likely to have weak attitudes in their evaluation of Trump. Especially African Americans, who are more likely to have a negative orientation toward Trump than the others, are the ones with the highest attitude strength. By contrast, the same subgroup, more likely to evaluate Clinton positively than the others, holds the weakest attitude strength toward Clinton.

Relations Among Attitude Direction and Attitude Strength

Finally, we present the covariance and correlation between location and scaling effects to investigate the relations among attitude direction and attitude strength. These values also help us understand how homogeneous or heterogeneous individuals are in their attitudes. The estimated (co)variance matrix $\hat{Σ}$ contains the variances of the person location and scaling parameters ( ${\hat{σ}}_{θ_{p}}^{2}$ and ${\hat{σ}}_{{\tilde{θ}}_{p}}^{2}$ ) in the diagonals and their covariance in the off diagonals,

\hat{Σ} = (\begin{matrix} {\hat{σ}}_{θ_{p}}^{2} & \hat{c o v} (θ_{p}, {\tilde{θ}}_{p}) \\ \hat{c o v} (θ_{p}, {\tilde{θ}}_{p}) & {\hat{σ}}_{{\tilde{θ}}_{p}}^{2} \end{matrix}) .

Table 4 reports the estimates and the ones for the correlation for both applications. Regarding gender equality attitudes, both variances increase with the gender equalityindex, indicating that lower-ranked countries are more homogenous than higher-ranked countries in attitude direction and attitude strength. The very weak correlation between location and scaling effects tells attitude direction is almost independent of attitude strength. Thus, both components capture distinct attitude-related features in this application.

The values for the evaluation of candidate traits suggest there is much more heterogeneity in attitude direction when evaluating Clinton than Trump. This result reveals considerable variability in the valence for Clinton, whereas the reversed pattern is observed for attitude strength. Here, the respondents are more heterogeneous in the evaluation of Trump. The moderate correlations between location and scaling effects indicate that those respondents with a very positive orientation toward the candidates tend to have strong attitudes, and those with a very negative orientation tend to hold weak attitudes.

Table 4.

Covariance and correlation of comprehensive models.

	Covariance	corr( $θ_{p}, {\tilde{θ}}_{p}$ ) (s.e.)
Attitudes toward gender equality
Sweden	$\hat{Σ} = (\begin{matrix} 2.62 & - 0.37 \\ - 0.37 & 5.79 \end{matrix})$	$- 0.09$ (0.07)
	s.e.( ${\hat{σ}}_{θ_{p}}^{2}$ ) = 0.34, s.e.( ${\hat{σ}}_{{\tilde{θ}}_{p}}^{2}) = 0.69$
Germany	$\hat{Σ} = (\begin{matrix} 1.62 & - 0.13 \\ - 0.13 & 3.72 \end{matrix})$	$- 0.05$ (0.05)
	s.e.( ${\hat{σ}}_{θ_{p}}^{2}$ ) = 0.15, s.e.( ${\hat{σ}}_{{\tilde{θ}}_{p}}^{2}) = 0.32$
Hungary	$\hat{Σ} = (\begin{matrix} 1.40 & 0.15 \\ 0.15 & 3.46 \end{matrix})$	0.07 (0.05)
	s.e.( ${\hat{σ}}_{θ_{p}}^{2}$ ) = 0.12, s.e.( ${\hat{σ}}_{{\tilde{θ}}_{p}}^{2}) = 0.32$
Evaluation of Candidate Traits
Clinton	$\hat{Σ} = (\begin{matrix} 2.26 & 0.95 \\ 0.95 & 0.88 \end{matrix})$	0.67 (0.05)
	s.e.( ${\hat{σ}}_{θ_{p}}^{2}$ ) = 0.13, s.e.( ${\hat{σ}}_{{\tilde{θ}}_{p}}^{2}) = 0.13$
Trump	$\hat{Σ} = (\begin{matrix} 1.57 & 1.13 \\ 1.13 & 1.82 \end{matrix})$	0.67 (0.03)
	s.e.( ${\hat{σ}}_{θ_{p}}^{2}$ ) = 0.09, s.e.( ${\hat{σ}}_{{\tilde{θ}}_{p}}^{2}) = 0.17$

Note: $\hat{Σ}$ contains the variance of the person location parameter ${\hat{σ}}_{θ_{p}}^{2}$ and the variance of the person scaling parameter ${\hat{σ}}_{{\tilde{θ}}_{p}}^{2}$ in the diagonals, and their covariance in the off diagonals.

Conclusion

Attitudes, such as people’s evaluations of issues or political figures, are integral parts of public opinion research. IRT has become a popular modeling approach to measure attitudes by relying on a series of survey items that tap the specific domain of interest. The present contribution provides an explanatory modeling tool to understand attitudes better. It presents, discusses, and applies an item response model that can handle the classic ordered response options of item batteries and allows for investigating variability in attitudes across individuals as a function of covariates. Drawing on conceptualizations from the social psychology literature, we present a parameterization that captures two essential components that refer to the attitude itself: direction and strength. The approach allows the simultaneous study of both components, the relations among them, and disentangles what covariates explain each component. The applications to attitudes toward gender equality and the evaluation of candidate traits exemplify the types of substantive insights that can be gained by the approach but existing models are unable to reveal.

We present extended versions of the partial credit model that include covariates to account for attitude direction and strength. An alternative model, widely used in latent trait theory and often providing a better fit than the PCM (Maydeu-Olivares 2005), is Samejima’s graded response model (Samejima 1969, 2016). The main reason why we rely on the PCM is that parameterization is simpler. A linear predictor of the form $η_{p i r} = θ_{p} + s_{r} {\tilde{θ}}_{p} - δ_{i r}$ in the graded response model, which has the form $P (Y_{p i} \leq r) = F (η_{p i r})$ , is bound to fail since predictors have to be ordered such that $η_{p i 1} \leq \dots \leq η_{p i k}$ . This ordering raises problems since $s_{r}$ is a function of $r$ that takes positive and negative values. Moreover, the order restraint can be very severe when covariates show strong variation or a large number of covariates are included. Then, the parameter space can be empty, which is known in ordinal regression (cross-sectional data) but carries over to the repeated measurement case.

Supplemental Material

sj-pdf-1-smr-10.1177_00491241251403078 - Supplemental material for An Ordinal Item Response Model for Understanding Attitudes

Supplemental material, sj-pdf-1-smr-10.1177_00491241251403078 for An Ordinal Item Response Model for Understanding Attitudes by Ingrid Mauerer and Gerhard Tutz in Sociological Methods & Research

Footnotes

Data,Code,and Materials Availability Statements

The datasets analyzed and the code used during this study and documentation for the code are available in the Harvard Dataverse repository: Mauerer and Tutz (2025). Replication data for: An ordinal item response model for understanding attitudes. , Harvard Dataverse, Version 1.0.

Declaration of Conflicting Interest

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Ingrid Mauerer gratefully acknowledges financial support from the EMERGIA program (EMC21-00256), Junta de Andalucia, Spain.

ORCID iDs

Ingrid Mauerer

Gerhard Tutz

Preregistration Statement

This study was not preregistered because the presented empirical analyses use publicly available data that already existed before the study was begun and are cited in the reference list.

Supplemental Material

Supplemental material for this article is available online.

Notes

Author Biographies

Ingrid Mauerer is a distinguished researcher at the University of Málaga, Spain. Her research focuses on statistical modeling of political behavior, categorical data analysis, and discrete choice models.

Gerhard Tutz is Professor Emeritus of statistics at LMU Munich, Germany. His research interests include categorical data analysis, latent trait modeling, multivariate statistics, and survival analysis.

References

Abelson

R. P.

1992. “Opportunities in Survey Measurement of Attitudes.” Pp. 173–176 in Questions about Questions: Inquiries into the Cognitive Bases of Surveys edited by J. M. Tanur. Russell Sage Foundation.

Abelson

R. P.

1995. “Attitude Extremity.” Pp. 25–41 in Attitude Strength: Antecedents and Consequences, edited by R. E. Petty and J. A. Krosnick. Lawrence Erlbaum Associates, Inc.

Adams

R. J.

M. L.

Wilson

. 2012. “The Rasch Rating Model and the Disordered Threshold Controversy.” Educational and Psychological Measurement 72(4): 547–573.

Agresti

2010. Analysis of Ordinal Categorical Data, 2nd Edition. New York: Wiley.

Aldrich

J. H.

McKelvey

R. D.

. 1977. “A Method of Scaling With Applications to the 1968 and 1972 Presidential Elections.” The American Political Science Review 71(1): 111–130.

Alvarez

R. M.

Brehm

. 1995. “American Ambivalence Towards Abortion Policy: Development of a Heteroskedastic Probit Model of Competing Values.” American Journal of Political Science 39(4): 1055–1082.

Alvarez

R. M.

Brehm

. 1997. “Are Americans Ambivalent Towards Racial Policies?.” American Journal of Political Science 41(2): 345–374.

Andrich

2013. “An Expanded Derivation of the Threshold Structure of the Polytomous Rasch Model That Dispels Any ‘Threshold Disorder Controversy’.” Educational and Psychological Measurement 73(1): 78–124.

ANES 2017. “American National Election Study 2016.” Inter-University Consortium for Political and Social Research [distributor], 2017-09-19. https://doi.org/10.3886/ICPSR36824.v2.

10.

Ansolabehere

Rodden

Snyder Jr

J. M.

. 2008. “The Strength of Issues: Using Multiple Measures to Gauge Preference Stability, Ideological Constraint, and Issue Voting.” American Political Science Review 102(2): 215–232.

11.

Bafumi

Herron

M. C.

. 2010. “Leapfrog Representation and Extremism: A Study of American Voters and Their Members in Congress.” American Political Science Review 104(3): 519–542.

12.

Bartholomew

D. J.

Knott

Moustaki

. 2011. Latent Variable Models and Factor Analysis: A Unified Approach. Chichester: John Wiley & Sons.

13.

Bizer

G. Y.

Barden

J. C.

Petty

R. E.

. 2003. “Attitudes.” Pp. 247–253 in Encyclopedia of Cognitive Science, Vol. 1, edited by L. Nadel. MacMillan.

14.

Bock

R. D.

1972. “Estimating Item Parameters and Latent Ability When Responses are Scored in Two Or More Nominal Categories.” Psychometrika 37(1): 29–51.

15.

Bock

R. D.

Aitkin

. 1981. “Marginal Maximum Likelihood Estimation of Item Parameters: Application of An EM Algorithm.” Psychometrika 46(4): 443–459.

16.

Böckenholt

2017. “Measuring Response Styles in Likert Items.” Psychological Methods 00(22): 69–83.

17.

Böckenholt

Meiser

. 2017. “Response Style Analysis With Threshold and Multi-process Irt Models: A Review and Tutorial.” British Journal of Mathematical and Statistical Psychology 70(1): 159–181.

18.

Bolt

D. M.

Johnson

T. R.

. 2009. “Addressing Score Bias and Differential Item Functioning Due to Individual Differences in Response Style.” Applied Psychological Measurement 33(5): 335–352.

19.

Bolt

D. M.

Newton

J. R.

. 2011. “Multiscale Measurement of Extreme Response Style.” Educational and Psychological Measurement 71(5): 814–833.

20.

Carsey

T. M.

Layman

G. C.

. 2006. “Changing Sides Or Changing Minds? Party Identification and Policy Preferences in the American Electorate.” American Journal of Political Science 50(2): 464–477.

21.

Caughey

Warshaw

. 2015. “Dynamic Estimation of Latent Opinion Using a Hierarchical Group-Level IRT Model.” Political Analysis 23(2): 197–211.

22.

Chalmers

R. P.

Pritikin

Robitzsch

Zoltak

Kim

Falk

C. F.

Meade

et al. 2025. mirt: Multidimensional Item Response Theory. R package, https://CRAN.R-project.org/package=mirt.

23.

Chen

Moustaki

Zhang

J.-T.

. 2020. “On Likelihood Ratio Tests for Models With Latent Variables.” Psychometrika 85(1): 194–215.

24.

De Boeck

Partchev

. 2012. “IRTrees: Tree-Based Item Response Models of the GLMM Family.” Journal of Statistical Software 48(1): 1–28.

25.

De Boeck

Wilson

M. R.

. 2004. Explanatory Item Response Models: A Generalized Linear and Nonlinear Approach. New York: Springer.

26.

De Boeck

Wilson

M. R.

. 2016. “Explanatory Response Models.” Pp. 593–608 in Handbook of Item Response Theory, Volume One, edited by W. Van der Linden. Chapman and Hall/CRC.

27.

Drton

2009. “Likelihood Ratio Tests and Singularities.” The Annals of Statistics 37(2): 979–1012.

28.

Duck-Mayr

Montgomery

. 2023. “Ends Against the Middle: Measuring Latent Traits When Opposites Respond the Same Way for Antithetical Reasons.” Political Analysis 31(4): 606–625.

29.

EIGE. 2023. “European Institute for Gender Equality: Gender Equality Index 2023.” Publications Office of the European Union, https://doi.org/10.2839/64810.

30.

EVS. 2022. “European Values Study 2017: Integrated dataset (EVS 2017).” GESIS, Cologne. ZA7500 Data file Version 5.0.0, https://doi.org/10.4232/1.13897.

31.

Falk

C. F.

Cai

. 2016. “A Flexible Full-Information Approach to the Modeling of Response Styles.” Psychological Methods 21(3): 328–347.

32.

Fox

R. L.

Lawless

J. L.

. 2011. “Gendered Perceptions and Political Candidacies: A Central Barrier to Women’s Equality in Electoral Politics.” American Journal of Political Science 55(1): 59–73.

33.

Franchino

Zucchini

. 2015. “Voting in a Multi-Dimensional Space: A Conjoint Analysis Employing Valence and Ideology Attributes of Candidates.” Political Science Research and Methods 3(2): 221–241.

34.

Hare

Armstrong

D. A.

Bakker

Carroll

Poole

K. T.

. 2015. “Using Bayesian Aldrich–McKelvey Scaling to Study Citizens’ Ideological Preferences and Perceptions.” American Journal of Political Science 59(3): 759–774.

35.

Harvey

A. C.

1976. “Estimating Regression Models With Multiplicative Heteroscedasticity.” Econometrica 44(3): 461–465.

36.

Hill

S. J.

Tausanovitch

. 2015. “A Disconnect in Representation? Comparison of Trends in Congressional and Public Polarization.” The Journal of Politics 77(4): 1058–1075.

37.

Howe

L. C.

Krosnick

J. A.

. 2017. “Attitude Strength.” Annual Review of Psychology 68(1): 327–351.

38.

Huggins-Manley

A. C.

Algina

. 2015. “The Partial Credit Model and Generalized Partial Credit Model As Constrained Nominal Response Models, With Applications in M Plus.” Structural Equation Modeling: A Multidisciplinary Journal 22(2): 308–318.

39.

Jeon

De Boeck

. 2016. “A Generalized Item Response Tree Model for Psychological Assessments.” Behavior Research Methods 48(3): 1070–1085.

40.

Jessee

S. A.

2016. “(How) Can We Estimate the Ideology of Citizens and Political Elites on the Same Scale?.” American Journal of Political Science 60(4): 1108–1124.

41.

Krosnick

J. A.

Abelson

R. P.

. 1992. “The Case for Measuring Attitude Strength in Surveys.” Pp. 177–203 in Questions about Questions: Inquiries Into the Cogntitive Bases of Surveys, edited by J. M. Tanur. Russell Sage Foundation.

42.

Layman

G. C.

Carsey

T. M.

. 2002. “Party Polarization and “Conflict Extension” in the American Electorate.” American Journal of Political Science 46(4): 786–802.

43.

Likert

1932. “A Technique for the Measurement of Attitudes.” Archives of Psychology 00(140): 44–53.

44.

Liu

Pierce

D. A.

. 1994. “A Note on Gauss–Hermite Quadrature.” Biometrika 81(3): 624–629.

45.

Masters

G. N.

1982. “A Rasch Model for Partial Credit Scoring.” Psychometrika 47: 149–174.

46.

Masters

G. N.

2016. “Partial Credit Model.” Pp. 109–126 in Handbook of Item Response Theory, Volume One, edited by W. J. Van der Linden. London, UK: Chapman and Hall/CRC.

47.

Masters

G. N.

Wright

. 1984. “The Essential Process in a Family of Measurement Models.” Psychometrika 49: 529–544.

48.

Mauerer

Schneider

. 2019. “Perceived Party Placements and Uncertainty on Immigration in the 2017 German Election.” Pp. 117–143 in Jahrbuch für Handlungs- und Entscheidungstheorie 11, edited by M. Debus, M. Tepe, and J. Sauermann (Eds.). Wiesbaden: Springer.

49.

Mauerer

Tutz

. 2024. “Vote Choices and Valence: Intercepts and Alternate Specifications.” Political Analysis 32(3): 361–378.

50.

Mauerer

Tutz

. 2025. “Replication Data for: An Ordinal item Response Model for Understanding Attitudes.” https://doi.org/10.7910/DVN/CZJJAF, Harvard Dataverse, Version 1.0.

51.

Maydeu-Olivares

2005. “Further Empirical Results on Parametric Versus Non-Parametric IRT Modeling of Likert-Type Personality Data.” Multivariate Behavioral Research 40(2): 261–279.

52.

Miller

J. M.

Peterson

D. A.

. 2004. “Theoretical and Empirical Implications of Attitude Strength.” The Journal of Politics 66(3): 847–867.

53.

Moors

2003. “Diagnosing Response Style Behavior by Means of a Latent-Class Factor Approach. Socio-Demographic Correlates of Gender Role Attitudes and Perceptions of Ethnic Discrimination Reexamined.” Quality and Quantity 37: 277–302.

54.

Morren

Gelissen

J. P.

Vermunt

J. K.

. 2011. “Dealing With Extreme Response Style in Cross-Cultural Research: A Restricted Latent Class Factor Analysis Approach.” Sociological Methodology 41(1): 13–47.

55.

Muraki

1992. “A Generalized Partial Credit Model: Application of An EM Algorithm.” Applied Psychological Measurement 16(2): 159–176.

56.

Muthén

L. K.

Muthén

B. O.

. 2023. Mplus User’s Guide. (8th ed.). Los Angeles, CA: Muthén & Muthén. Software Version 8.11.

57.

Oshri

Harsgor

Itzkovitch-Malka

Tuttnauer

. 2023. “Risk Aversion and the Gender Gap in the Vote for Populist Radical Right Parties.” American Journal of Political Science 67(3): 701–717.

58.

Pérez

E. O.

Tavits

. 2019. “Language Influences Public Attitudes Toward Gender Equality.” The Journal of Politics 81(1): 81–93.

59.

Petty

R. E.

Krosnick

J. A.

. 1995. Attitude Strength: Antecedents and Consequences. Ohio State University Series on Attitudes and Persuasion. Hillsdale, NJ: Lawrence Erlbaum Associates.

60.

Petty

R. E.

Siev

J. J.

Briñol

. 2023. “Attitude Strength: What’s New?” The Spanish Journal of Psychology 26: e4.

61.

Poole

K. T.

1998. “Recovering a Basic Space From a Set of Issue Scales.” American Journal of Political Science 42(3): 954–993.

62.

Rasch

1960. Studies in Mathematical Psychology: I. Probabilistic Models for Some Intelligence and Attainment Tests. Copenhagen: Nielsen & Lydiche.

63.

Rasch

1961. “On General Laws and the Meaning of Measurement in Psychology.” Pp. 321–333 in Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 4.

64.

Samejima

1969. “Estimation of Latent Ability Using a Response Pattern of Graded Scores.” Psychometrika Monograph Supplement 34(S1): 1–97.

65.

Samejima

1995. “Acceleration Model in the Heterogeneous Case of the General Graded Response Model.” Psychometrika 60(4): 549–572.

66.

Samejima

2016. “Graded Response Model.” Pp. 95–108 in Handbook of Item Response Theory, edited by W. Van der Linden. Chapman and Hall/CRC.

67.

Saris

W. E.

Sniderman

P. M.

. 2004. Studies in Public Opinion: Attitudes, Nonattitudes, Measurement Error, and Change. Princeton: Princeton University Press.

68.

Schaffner

B. F.

2005. “Priming Gender: Campaigning on Women’s Issues in U.S. Senate Elections.” American Journal of Political Science 49(4): 803–817.

69.

Schauberger

2024. MultOrdRS: Model Multivariate Ordinal Responses Including Response Styles. R package version 0.1-3.

70.

Schauberger

Tutz

. 2022. “Multivariate Ordinal Random Effects Models Including Subject and Group Specific Response Style Effects.” Statistical Modelling 22(5): 409–429.

71.

Self

S. G.

Liang

K.-Y.

. 1987. “Asymptotic Properties of Maximum Likelihood Estimators and Likelihood Ratio Tests Under Nonstandard Conditions.” Journal of the American Statistical Association 82(398): 605–610.

72.

Skrondal

Rabe-Hesketh

. 2004. Generalized Latent Variable Modeling: Multilevel, Longitudinal, and Structural Equation Models. Boca Raton: CRC Press.

73.

Stone

Simas

E. N.

. 2010. “Candidate Valence and Ideological Positions in U.S. House Elections.” American Journal of Political Science 54(2): 371–388.

74.

Stone

W. J.

Maisel

L. S.

Maestas

C. D.

. 2004. “Quality Counts: Extending the Strategic Politician Model of Incumbent Deterrence.” American Journal of Political Science 48(3): 479–495.

75.

Suh

Bolt

D. M.

. 2010. “Nested Logit Models for Multiple-Choice Item Response Data.” Psychometrika 75(3): 454–473.

76.

Thissen

Cai

Bock

R. D.

. 2010. “The Nominal Categories item Response Model.” Pp. 43–75 in Handbook of Polytomous Item Response Theory Models, edited by M. L. Nering and R. Ostini (Eds.). Routledge New York, NY.

77.

Thissen

Steinberg

. 1986. “A Taxonomy of Item Response Models.” Psychometrika 51(4): 567–577.

78.

Treier

Hillygus

D. S.

. 2009. “The Nature of Political Ideology in the Contemporary Electorate.” Public Opinion Quarterly 73(4): 679–703.

79.

Tuerlinckx

Wang

W.-C.

. 2004. “Models for Polytomous Data.” Pp. 75–109 in Explanatory Item Response Models: A Generalized Linear and Nonlinear Approach, edited by P. D. Boeck and M. Wilson (Eds.). Springer.

80.

Tutz

2020. “On the Structure of Ordered Latent Trait Models.” Journal of Mathematical Psychology 96: 102346.

81.

Tutz

2022. “Ordinal Regression: A Review and a Taxonomy of Models.” WIREs Computational Statistics 14(2): e1545.

82.

Verbyla

A. P.

1993. “Modelling Variance Heterogeneity: Residual Maximum Likelihood and Diagnostics.” Journal of the Royal Statistical Society: Series B (Methodological) 55(2): 493–508.

83.

Vermunt

J. K.

Magidson

. 2025. Latent GOLD 6.1 User’s Guide. Belmont, MA: Statistical Innovations Inc. Software for Latent Class and Finite Mixture Modeling.

84.

Zhou

2019. “Hierarchical Item Response Models for Analyzing Public Opinion.” Political Analysis 27(4): 481–502.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.25 MB