Formal Modeling of the Resistance to Peer Influence Questionnaire: A Comparison of Adolescent Boys and Girls With and Without Mild-to-Borderline Intellectual Disability

Abstract

Items of the Resistance to Peer Influence Questionnaire (RPIQ) have a tree-based structure. On each item, individuals first choose whether a less versus more peer-resistant group best describes them; they then indicate whether it is “Really true” versus “Sort of true” that they belong to the chosen group. Using tree-based item response theory, we show that RPIQ items tap three dimensions: A Resistance to Peer Influence (RPI) dimension and two Response Polarization dimensions. We then reveal subgroup differences on these dimensions. That is, adolescents with mild-to-borderline intellectual disability, compared with typically developing adolescents, are less RPI and more polarized in their responses. Also, girls, compared with boys, are more RPI, and, when high RPI, more polarized in their responses. Together, these results indicate that a tree-based modeling approach yields a more sensitive measure of individuals’ RPI as well as their tendency to respond more or less extremely.

Keywords

gender differences IRTrees models item response theory mild-to-borderline intellectual disability Resistance to Peer Influence RPI Response Polarization

Adolescence has been described as “a developmental period rife with change” (Hall, 1904), that captures the phase of gradual transition between childhood and adulthood (Spear, 2000), and that encompasses intense changes in neurological, physical, behavioral, and social functioning (Somerville, Jones, & Casey, 2010). This period of life also is often conceptualized as a period of increased risk taking (Dahl, 2004; Steinberg, 2008). This adolescent peak in risk taking has been attributed to relative immaturity of the “cognitive” frontal cortical neural system as opposed to the “affective” subcortical neural system (Casey, Jones, & Hare, 2008; also see, e.g., Somerville et al., 2010). Specifically, increasing evidence points to the importance of subcortically mediated puberty-related changes in social-affective processing in understanding adolescents’ vulnerability to take risks (Crone & Dahl, 2012; for a review, see Defoe, Dubas, Figner, & van Aken, 2015); susceptibility to peer influence seems to be a crucial factor (e.g., Gardner & Steinberg, 2005; Smith, Chein, & Steinberg, 2014; for a review, see Albert, Chein, & Steinberg, 2013).

A popular measure in the study of RPI is the 10-item Resistance to Peer Influence Questionnaire (RPIQ), developed by Steinberg and Monahan (2007). Figure 1 graphically depicts an item of this questionnaire: an overview of all items is presented in the appendix. On each item of this scale, individuals are first asked to choose the option that best describes the group of people (i.e., more vs. less peer resistant) they belong to; they are then asked to indicate to what degree they feel they belong to this group (i.e., “Really true” vs. “Sort of true”). Conventionally, scores to each item are then aggregated in a 4-point Likert-type scale score, in which the “Really true” and “Sort of true” option of the less peer-resistant statement are coded as 1 and 2, respectively, and the “Sort of true” and “Really true” option of the more peer-resistant statement are coded as 3 and 4, respectively. This RPI Likert-type scale has shown to be a valid and reliable (see, Monahan, Steinberg, & Cauffman, 2009; Monahan, Steinberg, Cauffman, & Mulvey, 2009; Stautz & Cooper, 2014; Steinberg & Monahan, 2007; Sumter, Bokhorst, Steinberg, & Westenberg, 2009) indicator of RPI among groups of different ages (e.g., Sumter et al., 2009) and ethnic backgrounds (Steinberg & Monahan, 2007). More specifically, using this measure, it has been repeatedly shown that over the course of adolescence, levels of RPI increase (Monahan, Steinberg, & Cauffman, 2009; Monahan, Steinberg, Cauffman, & Mulvey, 2009; Steinberg & Monahan, 2007; Sumter et al., 2009), and when entering adulthood, stable levels of RPI are reached (Steinberg & Monahan, 2007). In addition, girls, compared with boys, have been shown to report higher levels of RPI, especially during midadolescence (Steinberg & Monahan, 2007; Sumter et al., 2009).

Figure 1.

Example of an item of the Resistance to Peer Influence Questionnaire (RPIQ).

Confirmatory factor analysis (CFA; Steinberg & Monahan, 2007) and principal component analysis (PCA; Sumter et al., 2009) have been used to reveal that a one-factor structure underlies the RPIQ. This indicates that the Likert-type scale scores to all items load on one and the same underlying construct; that is, there is between-item unidimensionality. However, we argue that the structure of each item of the scale suggests within-item multidimensionality. As is illustrated in Figure 1, each item of the RPIQ in fact consists of three subitems—an initial subitem that asks participants which of two groups they belong to most, and then two subitems that ask participants to what degree they feel they belong to the initially chosen group. Choices to each of these subitems may originate in separate dimensions, which are conceptually distinct and may even be unrelated. More specifically, choices to the initial subitem that asks participants which of two groups they belong to most, may be reflective of the construct of RPI. Instead, choices to the two subitems that ask participants to what degree they feel they belong to the initially chosen group, may not be reflective of RPI, but of some distinct and possibly unrelated construct. As outlined below, we propose that this latter construct can be interpreted as the tendency to respond more or less extremely. For each item of the RPIQ, a Likert-type scale score may thus be reflective of a mix of a participant’s position on either of these dimensions. This would render these Likert-type scale scores to be no pure reflection of a participant’s RPI. The multidimensionality within each item cannot be uncovered with a regular CFA/PCA as the single CFA/PCA dimension is based on between-item information and does not reveal the within-item dimensions of which it is a composite. The first aim of the current study was therefore to test whether items of the RPIQ tap a within-item uni- or multidimensional construct. To pursue this goal, we formally modeled choices to items of the RPIQ by means of tree-based item response theory (IRT; De Boeck et al., 2011; De Boeck & Partchev, 2012; Jeon & De Boeck, 2016).

The internal structure of an item of the RPIQ and its associated uni- versus multidimensional conceptual interpretations are illustrated in Figure 2A and B, respectively. From these figures, it can be inferred that each item of the RPIQ can be conceptualized by a branching model (cf. De Boeck et al., 2011; De Boeck & Partchev, 2012) with three nodes. Node1 refers to the highest branching level: the choice between the more or less peer resistant statement. Node2 and node3 refer to the lowest branching levels: choices between strongly (“Really true”) or weakly (“Sort of true”) agreeing with the statement chosen at node1. On each item, participants respond to the node1 subitem. Depending on their choice to this subitem, participants respond to either the node2 or node3 subitem. Our first aim was to test the hypothesis that choices to the different nodes are not reflective of one underlying RPI dimension (i.e., unidimensional, cf. Figure 2A), but of multiple dimensions (i.e., multidimensional, cf. Figure 2B; Hypothesis 1a). We thereby expected that choices to the highest branching-level node (node1) are indicative of more or less RPI (Hypothesis 1b), whereas choices to the lowest branching-level nodes (node2 and node3) are indicative of a different underlying construct, which is conceptually distinct from and possibly unrelated to the construct of RPI. We interpret this latter construct as the tendency to respond more or less extremely (cf. Figure 2B; Hypothesis 1c); throughout this article we will coin this tendency “Response Polarization.”

Figure 2.

Conceptual visualization of an item of the Resistance to Peer Influence Questionnaire (RPIQ) in terms of a uni- versus multidimensional branching model. In both panels, the highest branching-level node (node1) conceptualizes the choice between either the less (Statement A; left) or the more (Statement B; right) peer-resistant statement. The lowest branching-level nodes (node2 and node3) refer to a further differentiation within either of these statements, conceptualizing the choice between strongly (“Really true”) or weakly (“Sort of true”) agreeing with the chosen statement. (A) Conceptualization of the unidimensional branching model in which choices to all nodes are reflective of the same underlying dimension. (B) Conceptualization of the multidimensional branching model in which choices to all nodes are reflective of distinct dimensions.

Previous studies that used a tree-based IRT framework have shown that taking into account the multidimensionality of items of a measure is of added value in the study of individual differences (e.g., Cho, Gilbert, & Goodwin, 2013; Jeon & De Boeck, 2016; López-Sepulcre, De Bona, Valkonen, Umbers, & Mappes, 2015; Partchev & De Boeck, 2012; Willoughby, Wirth, & Blair, 2011; Zettler, Lang, Hülsheger, & Hilbig, 2016). In case such a tree-based modeling approach supports the notion that items of the RPIQ tap a multidimensional construct, modeling this multidimensionality may provide a more fine-grained analysis of individual differences in two ways. First, it may render a more sensitive measure of RPI, which is not affected by variance in an individual’s tendency for more or less polarized responses (also see, Wetzel, Lüdtke, Zettler, & Böhnke, 2016). That is, different combinations of between-subgroup differences on the RPI and Response Polarization dimensions can either accurately reveal, (partly) mask, or spuriously cause between-subgroup differences in RPIQ Likert-type scale scores. Most notably, consider the situation in which two subgroups would differ on both RPI and Response Polarization, though in different directions. That is, people from one group (i.e., Group A) would score higher on RPI but lower on Response Polarization than would people from the other group (i.e., Group B). Using Likert-type scale scores would then obscure differences between subgroups, since the combination of differences between subgroups would yield these scores to be approximately similar for both subgroups. Taking into account possible between-subgroup differences on the Response Polarization dimensions then may provide a more sensitive measure of RPI, which is not affected by differences in Response Polarization. Second, modeling this multidimensionality may reveal additional information in terms of a formal measure of an individual’s tendency for more or less polarized responses. The second aim of the current study was therefore to test whether taking into account the potential multidimensionality of items of the RPIQ is of added value in comparing four subgroups of adolescents, that is, boys and girls with and without mild-to-borderline intellectual disability (MBID).

Adolescents with MBID constitute a group of adolescents with an IQ between 55 and 85, and who experience significant limitations in adaptive functioning (Bexkens, 2013; De Wit, Moonen, & Douma, 2012; also see American Psychiatric Association [APA], 2000, 2013; Schalock et al., 2010). Existing literature suggests that individual differences in RPI as well as Response Polarization may be affected by the presence or absence of MBID and by gender. First, with respect to RPI, there is some direct evidence to suggest that MBID adolescents, as opposed to their Typically Developing (TD) peers, are less resistant to peer influence (Bexkens, 2013; also see Khemka & Hickson, 2006). In addition, there is some indirect evidence to corroborate this (also see Greenspan, 2008). That is, lower RPI is associated with increased impulsivity (Monahan, Steinberg, & Cauffman, 2009; Monahan, Steinberg, Cauffman, & Mulvey, 2009), and less mature patterns of brain structure and function (Grosbras et al., 2007; Paus et al., 2008). As individuals with low intellectual abilities score high on indices of impulsivity (Koolhof, Loeber, Wei, Pardini, & Collot d’Escury, 2007; for a review, see Bexkens, Ruzzano, Collot d’Escury, van der Molen, & Huizenga, 2014), this can be taken to suggest that MBID adolescents, as opposed to their TD peers, would be less RPI. In addition, and as indicated above, there seem to be gender differences in RPI, with adolescent girls to be more RPI than adolescent boys (Sumter et al., 2009). Based on these findings, we hypothesized that MBID, as opposed to TD adolescents, would obtain lower scores on the proposed RPI dimension of the RPIQ (Hypothesis 2a), and that girls, as opposed to boys, would obtain higher scores on this dimension (Hypothesis 2b).

With respect to Response Polarization, most previous studies reveal lower IQ to be predictive of a more extreme response style on a diverse range of scales (Meisenberg & Williams, 2008; Wilkinson, 1970; for reviews, see Van Vaerenbergh & Thomas, 2013; Wetzel et al., 2016). Therefore, we hypothesized that MBID, as opposed to TD adolescents, would obtain higher scores on the proposed Response Polarization dimensions (Hypothesis 3a). However, evidence with respect to gender differences in response style is inconclusive (for a review, see Van Vaerenbergh & Thomas, 2013), with some studies showing males to respond more extremely than females (e.g., Meisenberg & Williams, 2008), and other studies showing either the reverse (e.g., Weijters, Geuens, & Schillewart, 2010) or no gender differences (e.g., Bachman & O’Malley, 1984; Greenleaf, 1992; for a review, see Wetzel et al., 2016). Therefore, we explored the relation between gender and scores on the proposed Response Polarization dimensions of the RPIQ.

In sum, using a tree-based IRT framework, we aimed to study the uni- versus multidimensionality of items of the RPIQ. Expecting items of the RPIQ to be reflective of a multi- rather than unidimensional construct, we additionally aimed to test whether modeling both RPI and Response Polarization dimensions would be of added value in comparing adolescent boys and girls with and without MBID.

Methods

Participants

RPIQ data were available for 179 TD and 190 MBID adolescents. However, participants with missing data on one of more item(s) of the RPIQ (N_TD = 2; N_MBID = 10) or whose gender was not recorded (N_MBID = 6) were excluded from all analyses. This was done because the dendrify function that was used to transform the data (see “Model Estimation and Selection: Methods” section) cannot handle missing item scores.¹ As a result, in the current study, data of 177 TD (N_{Boys_TD} = 117; 66.10%; SPM IQ: M [SD] = 106.00 [14.45]) and 174 MBID adolescents (N_{Boys_MBID} = 103; 59.20%; SPM IQ: M [SD] = 66.72 [11.32]), aged between 12 and 19 years (M [SD] = 15.21 [1.50]), were included. An analysis of variance (ANOVA), with intellectual ability (two levels: TD, MBID) and gender (two levels: boys, girls) as independent variables and age as dependent variable, indicated that MBID adolescents (M [SD] = 15.47 [0.11]) were significantly older than TD adolescents (M [SD] = 14.87 [0.12]), F(1, 347) = 13.76, p < .001; and that boys (M [SD] = 15.33 [0.10]) were marginally, but significantly older than girls (M [SD] = 15.01 [0.13]), F(1, 347) = 3.96, p = .047; whereas the interaction between intellectual ability and gender was not significant (p > .05). Participants were recruited through regular or special education schools in the Netherlands. Assignment to the TD or MBID group consisted of two phases: an initial phase and a refined phase. Initial assignment was based on school type. TD adolescents were selected from regular secondary education schools that prepare students for community college or higher education. MBID adolescents were selected from two types of special education schools: (1) special vocational education (“Praktijkonderwijs”) and (2) special education for adolescents with an intellectual disability. Special vocational education schools have the following admittance criteria: (1) an IQ between 60 and 85, tested no longer than 2 years prior to admittance and (2) learning delays of 50% or more in at least two of the following areas: mathematics, reading accuracy and fluency, reading comprehension, and spelling; one of these delays must be in mathematics or reading comprehension. Special education schools for adolescents with an intellectual disability have the following admittance criteria: (1) an IQ less than 70, tested no longer than 2 years prior to admittance; (2) a severe learning delay, that has not been sufficiently improved after remediation provided at the previous school; and (3) severely limited (social) adaptive skills. Refined assignment was based on the most recent IQ information from teachers or school files. That is, group assignment to the TD group was checked by asking teachers to indicate students who they thought had lower than average IQ, which was not the case for any of the selected students. Group assignment to the MBID group was refined by checking standardized IQ scores from school files and excluding participants who had a standardized IQ greater than 85. Prior to participation, written informed consent was obtained from adolescents and their caregivers. All procedures were approved by the local Ethical Committee of the University of Amsterdam.

Materials

As part of a larger study, that aimed to investigate risk taking in TD and MBID adolescents, both the RPIQ and an experimental risk-taking task were administrated. Only data on the RPIQ were included in the current study; information on the experimental task can be obtained from the corresponding author.

Resistance to Peer Influence Questionnaire

All participants filled out the RPIQ (Steinberg & Monahan, 2007; Dutch translation, Sumter et al., 2009), a 10-item scale that aims to measure Resistance to Peer Influence (RPI). As outlined in the “Introduction” section, each item on this scale requires two choices to two out of three subitems (see, Figure 1). That is, first, individuals are asked to choose the option that best describes the group of people (i.e., more vs. less peer resistant) they belong to; they are then asked to indicate to what degree they feel they belong to this group (i.e., “Really true” vs. “Sort of true”). Conventionally, responses to these two subitems are combined into an aggregated response to a 4-point Likert-type scale, which (after recoding three items for which the nonresistant and resistant statement are mirrored) ranges from 1 for “Really true” for the nonresistant group descriptor to 4 for “Really true” for the resistant group descriptor. Subsequently, an RPI total score is created by summing scores to all 10 items, with higher total scores being indicative of higher RPI. Psychometric properties of the RPIQ have shown to be good, with evidence for both respectable reliability (i.e., internal consistency; Cronbach’s αs > .70; Steinberg & Monahan, 2007; Sumter et al., 2009) and criterion validity (Monahan, Steinberg, & Cauffman, 2009; Monahan, Steinberg, Cauffman, & Mulvey, 2009; Steinberg & Monahan, 2007). An overview of the items of the RPIQ is presented in the appendix.

Procedure

Participants were individually tested at school in an empty classroom, by one of several trained (under)graduate students. After completion of an experimental task, participants were asked to fill out the RPIQ. Scales with items that have a similar tree-based structure as items of the RPIQ have been successfully administrated in research among adolescents with learning disabilities or behavior disorders (e.g., Harter, Whitesell, & Junkin, 1998). In addition, care was taken to make sure that participants understood (the structure of) the items and instructions. That is, instructions were read to participants aloud. Participants were then asked to read aloud the example item and to verbally indicate how they would answer the item. This procedure was repeated, if necessary, until participants understood (the structure of) the items and understood what was required from them. Participants then completed the questionnaire, whereas the experimenter stayed in another part of the room to answer questions when needed.

Analytic Procedure and Results

The analytic procedure consisted of three steps: (1) model estimation and selection, (2) interpretation of uni- versus multidimensionality, and (3) testing for differences across subgroups. Below we first describe the analytic procedure and results for Step 1, and then proceed in an equivalent manner to Steps 2 and 3. On request, a tutorial, with programming code and practical guidelines how to implement the analytic procedure, can be obtained from the corresponding author.

Model Estimation and Selection: Methods

As indicated in the “Introduction” section and illustrated in Figure 2, we modeled choices to items of the RPIQ with a three-node branching model (De Boeck et al., 2011; De Boeck & Partchev, 2012). Node1 refers to the highest branching level: the choice between the more or less peer-resistant statement. Node2 and node3 refer to the lowest branching levels; choices between strongly (“Really true”) or weakly (“Sort of true”) agreeing with the statement chosen at node1.

In order to explain the model conceptually, it is convenient to first consider the case in which participants are only presented with the node1 subitem between the more or less peer-resistant statement. Responses to such two-choice items can be modeled using IRT (De Boeck & Wilson, 2004, 2015; Embretson & Reise, 2000; Hambleton & Swaminathan, 1985; Lord, 1980; Thissen & Wainer, 2001). In such a model, the probability that a participant opts for the more peer-resistant statement is modeled as a function of two types of parameters. The first type is a set of item parameters, which reflects, for each item, the probability of opting for the peer-resistant statement given that the participant has an average level of RPI. That is, if the item parameter of a certain item is high compared with other item parameters, participants will overall be more likely to choose the peer-resistant option on this item. The second type of parameter is a person parameter, which is indicative of the participant’s level of RPI. Independent of the item at hand, participants with a high person parameter will be more likely to choose the peer-resistant option. An IRT model with these types of parameters is referred to as a Rasch model (see Bond & Fox, 2007). This model, in which item parameters are fixed and person parameters vary random over participants, is a specific case of the more general family of generalized linear mixed models (Cho et al., 2013; De Boeck et al., 2011; De Boeck & Partchev, 2012; De Boeck & Wilson, 2004, 2015; Molenaar, Tuerlincks, & van der Maas, 2015; Partchev & De Boeck, 2012; Willoughby et al., 2011).

However, in case of the RPIQ, participants are presented with two out of three potential subitems—reflected by the three different nodes in Figure 2. The generalized linear mixed model (GLMM) framework should therefore be extended such that, for each item, three (potential) choices are modeled instead of one. Note that for each participant (N = 351), for each item, a subresponse to either the node2 or node3 subitem is missing, depending on the participant’s choice to the node1 subitem. Since this missing response is only dependent on the participant’s response to node1, conditioning on this response generates a situation in which the missing data are missing at random (MAR; Little & Rubin, 1987), and therefore the generalized linear mixed model (GLMM) framework can still be used (cf. De Boeck & Partchev, 2012).

Based on its internal structure, we hypothesized that items of the RPIQ tap a multidimensional instead of unidimensional construct (Hypothesis 1a). In its most extensive version, a multidimensional conceptualization of items of the RPIQ becomes a three-dimensional model, in which the three choices to the node1, node2, and node3 subitems are reflective of three different underlying dimensions (Hypotheses 1b and 1c; Figure 2B). This three-dimensional conceptualization can be estimated by a full model, with for each of the three subitems a separate set of item parameters and a separate person parameter. In a less extensive version, the multidimensional conceptualization of items of the RPIQ becomes a two-dimensional model, in which choices to the highest branching-level node (node1) are reflective of an underlying dimension that is distinct from the dimension that underlies choices to the lowest branching-level nodes (node2 and node3), which are themselves indicative of the same underlying dimension (Figure 2B with the only difference that the dimensions underlying choices to the node2 and node3 subitems are identical). This two-dimensional conceptualization can be arrived at by constraining the sets of item parameters and the person parameters to be equal for choices to the node2 and node3 subitems. One additional item parameter should then be estimated, that reflects the overall (collapsed over items) differential probability of opting for the right-sided versus left-sided branch at the node2 versus node3 subitems.

Alternatively, scoring items of the RPIQ on a 4-point Likert-type scale (cf. Steinberg & Monahan, 2007) suggests that responses to each item are reflective of one underlying dimension, in that the three choices to the node1, node2, and node3 subitems tap the same process, which could be captured by one set of item parameters and one person parameter. This unidimensional conceptualization (Figure 2A) can be arrived at by constraining the sets of item parameters and the person parameters to be equal over all three nodes. Two additional item parameters should then be estimated, that reflect the overall (collapsed over items) differential probability of opting for the right-sided versus left-sided statement at the node1 versus node2 subitems, and at the node1 versus node3 subitems.

By systematically constraining parameters, we fitted a series of three models: a unidimensional model with one set of item parameters and one person parameter (Figure 2A), a two-dimensional model with two sets of item parameters and two person parameters (Figure 2B with the constraint that parameters are constrained to be equal over the node2 and node3 subitems), and a three-dimensional model with three sets of item parameters and three person parameters (Figure 2B).

For all models, parameterization was chosen to be in line with the earlier proposed usage of a 4-point Likert-type scale (cf. Steinberg & Monahan, 2007), in that for all three subitems the probability to opt for the higher Likert-type scale score (i.e., right-sided branch) versus the lower Likert-type scale score (i.e., left-sided branch) was modeled.² Model estimation was performed with the glmer function as implemented in the lme4 package in R (Bates & Maechler, 2009). For this purpose, original data were transformed (see, De Boeck et al., 2011) such that they could be modeled as a two-level branching IRT model by using the R function dendrify (De Boeck & Partchev, 2012). As the estimated models were identically parameterized and only differed in the number of freely estimated sets of item parameters and person parameters, model fit of these models could be compared by measures of comparative model fit. Both Akaike information criterion (AIC; Akaike, 1973) and Bayesian information criterion (BIC; Schwarz, 1978) were inspected. As lower AIC as well as BIC indicate better model fit, the model with the lowest AIC and BIC was selected. In case of conflict, additional analyses were performed.

Model Estimation and Selection: Results

AICs of the estimated uni-, two-, and three-dimensional models with, respectively, one, two, or three set(s) of item parameters and one-, two-, or three-person parameter(s), were 8,657 (unidimensional), 8,610 (two-dimensional), and 8,354 (three-dimensional). BICs of these models were 8,746 (unidimensional), 8,775 (two-dimensional), and 8,601 (three-dimensional). Thus, a three-dimensional model, with three sets of item parameters and three person parameters, had both the lowest AIC and lowest BIC, and was initially selected for further analysis.

Interpretation of Multidimensionality: Methods

For the selected three-dimensional model, two interpretations of its three dimensions exist, which are depicted in Figure 3A and B, respectively. First, all three dimensions might be reflective of more or less RPI. This multidimensional conceptualization is in line with the earlier proposed usage of a 4-point Likert-type scale (cf. Steinberg & Monahan, 2007) and will be referred to as the “3RPI conceptualization.” In this conceptualization, all three dimensions tap a different aspect of RPI, such that these cannot be reduced to one underlying RPI dimension (which would be a unidimensional model). Alternatively, and in line with our hypotheses (Hypotheses 1b and 1c), choices to the highest branching-level node (node1) might be reflective of more or less RPI, whereas choices to the lowest branching-level nodes (node2 and node3) might instead be reflective of Response Polarization. This alternative multidimensional conceptualization is conflicting with usage of a 4-point Likert-type scale (cf., Steinberg & Monahan, 2007) and will be referred to as the “1RPI&2POL conceptualization.” This model encompasses two distinct Response Polarization dimensions, in that the tendency to opt for the “Really true” versus “Sort of true” indicator after an initial choice for the more peer-resistant statement (i.e., node3) taps a construct that is different from the tendency to opt for the “Really true” versus “Sort of true” indicator after an initial choice for the less peer-resistant statement (i.e., node2). These two dimensions thus cannot be reduced to one Response Polarization dimension (which would be a two-dimensional model).

Figure 3.

Conceptual visualization of an item of the Resistance to Peer Influence Questionnaire (RPIQ) in terms of two different interpretations of a three-dimensional branching model. For the interpretation of the nodes at each branching level, please see the legend to Figure 2. (A) 3RPI conceptualization in which choices to all branching levels are reflective of three distinct RPI dimensions. (B) 1RPI&2POL conceptualization in which choices to the highest branching-level node (node1) are reflective of an RPI dimension, whereas choices to the lowest branching-level nodes (node2 and node3) are reflective of two distinct Response Polarization dimensions.

To decide which alternative best matched the data, we inspected the correlations between all three nodes.³ If the 3RPI conceptualization best described the multidimensional structure, we would observe all three nodes to be positively correlated, in that higher RPI at one node would be associated with higher RPI at another node. Instead, if the 1RPI&2POL conceptualization best described the multidimensional structure, we would not necessarily observe the highest branching-level node (node1) to be correlated with the lowest branching-level nodes (node2 and node3). More important, however, we would observe the lowest branching-level nodes (node2 and node3) to be negatively correlated. That is, a tendency to respond more polarized after the initial choice to be less peer resistant (i.e., lower Likert-type scale score/left-sided choice to the node2 subitem) would be associated with a tendency to respond more polarized after the initial choice to be more peer resistant (i.e., higher Likert-type scale score/right-sided choice to the node3 subitem).

Interpretation of Multidimensionality: Results

As can be verified in Table 1, we did not observe a positive correlation between all three nodes, rendering the 3RPI conceptualization, which is in line with usage of a 4-point Likert-type scale, not very plausible. In contrast, we did observe low-to-moderate correlations between node1 on one hand and node2 and node3 on the other, and, in accordance with the 1RPI&2POL conceptualization, a negative correlation between node2 and node3. We thus tentatively concluded that, as hypothesized (Hypotheses 1b and 1c), choices to the highest branching-level node (node1) are reflective of more or less RPI, whereas choices to the lowest branching-level nodes (node2 and node3) are reflective of two distinct, but associated, Response Polarization dimensions.

Table 1.

Correlations Between the Three Nodes, as Estimated Under the Initially Selected Three-Dimensional Model.

	Person effect node1	Person effect node2	Person effect node3
Person effect node1	1.000
Person effect node2	.414	1.000
Person effect node3	.409	−.661	1.000

Refined Model Estimation and Selection: Methods

As outlined in the “Model Estimation and Selection: Methods” section, the initially chosen model parameterization was in line with the earlier proposed usage of a 4-point Likert-type scale (cf. Steinberg & Monahan, 2007; 3RPI conceptualization). This choice was inconsistent with the idea that choices to the lowest branching-level nodes (node2 and node3) are reflective of tendencies to respond more or less polarized, that is Response Polarization (1RPI&2POL conceptualization). Since the correlational structure of the initially selected three-dimensional model supports this later 1RPI&2POL conceptualization, model estimation and selection was repeated after reparameterization of the model, such that it was consistent with this conceptualization. This reparameterization entailed that the modeled probabilities for choices to the node2 subitem were mirrored.

After this reparametrization, we fitted, as before, a series of three models: the uni-, two-, and three-dimensional models. As, after reparameterization, the estimated models were identically parameterized and only differed in the number of freely estimated sets of item parameters and person parameters, model fit of these models could again be compared by measures of comparative model fit; model fit of models after reparameterization was not directly compared with model fit of models before reparameterization. Again, both AIC and BIC were inspected, and the model with the lowest AIC and BIC was selected. In case of conflict, additional analyses were performed.

Refined Model Estimation and Selection: Results

AICs of the estimated uni-, two-, and three-dimensional models were 8,625 (unidimensional), 8,416 (two-dimensional), and 8,354 (three-dimensional). BICs of these models were 8,714 (unidimensional), 8,581 (two-dimensional), and 8,601 (three-dimensional). Thus, whereas AIC favored a three-dimensional model, with three sets of item parameters and three person parameters, BIC favored a two-dimensional model, with two sets of item parameters and two person parameters. To resolve this discrepancy, two additional models were estimated. In these models either (instead of both) the sets of item parameters or the person parameters were constrained to be equal over the node2 and node3 subitems. Following previous studies (e.g., Partchev & De Boeck, 2012), likelihood-ratio tests were used to test whether imposing these constraints was tenable. This proved not to be the case; constraining either the sets of item parameters, χ²(9) = 41.98, p < .001, or person parameters, χ²(3) = 55.58, p < .001, to be equal over the node2 and node3 subitems led to a significant worsening in model fit. Therefore, based on this result and in line with AIC, the reparameterized, three-dimensional model, with three sets of item parameters and three person parameters, was retained and finally selected for further analysis. The correlations between nodes for this reparameterized, three-dimensional model are presented in Table 2.

Table 2.

Correlations Between the Three Nodes, as Estimated Under the Finally Selected Reparameterized, Three-Dimensional Model.

	Person effect node1	Person effect node2	Person effect node3
Person effect node1	1.000
Person effect node2	−.414	1.000
Person effect node3	.409	.661	1.000

Effects of Intellectual Ability and Gender: Methods

For comparative purposes, we first performed a regular ANOVA, with intellectual ability (two levels: TD, MBID) and gender (two levels: boys, girls) as between-subjects factors and traditional RPI total scores (based on summing Likert-type scale scores to all items) as dependent variable.

Subsequently, we tested for intellectual ability and gender differences on the RPI and the two Response Polarization dimensions by including effects of intellectual ability (two levels: TD, MBID), gender (two levels: boys, girls), and their interaction at all three nodes of the finally selected reparameterized, three-dimensional branching model. Significance of these main and interaction effects added at each node was evaluated using Wald tests as implemented in the lme4 package (Bates & Maechler, 2009).

A unidimensional branching model is comparable to the linear model that underlies an ANOVA (cf. De Boeck et al., 2011). Therefore, we calculated and compared effect sizes of the effects of intellectual ability and gender on RPI in the unidimensional branching model on one hand and at the node1 subitem of the three-dimensional branching model on the other. Effect sizes, in terms of Cohen’s d, were defined as the ratio between the parameter estimate of the intellectual ability or gender effect and the standard deviation of the (node1) person effect, which was used as an estimate of the population standard deviation of the effect of interest. Note that these branching model effect sizes are measured on a different scale than ANOVA effect sizes, and therefore these cannot be directly compared with one another.

Effects of Intellectual Ability and Gender: Results

The ANOVA on traditional total scores yielded main effects of intellectual ability, F(1, 347) = 14.44, p < .001, d = 0.41; and gender, F(1, 347) = 31.13, p < .001, d = 0.62; whereas the interaction effect was not significant, F(1, 347) < 1, p = .798. MBID adolescents (M [SD] = 28.64 [5.45]) reported to be significantly less RPI than TD adolescents (M [SD] = 30.38 [3.80]); and boys (M [SD] = 28.53 [4.55]) reported to be significantly less RPI than girls (M [SD] = 31.18 [4.66]).

With respect to the multidimensional branching model analysis, it was found that at none of the three nodes the intellectual ability by gender interaction effect reached significance (ps > .05). Therefore, this effect was omitted from the model. There were significant main effects of intellectual ability and gender. More specifically, at the node1 subitem, MBID, compared with TD adolescents, had a significantly lower probability to opt for the more peer-resistant statement. With respect to the lowest branching-level choices, MBID, compared with TD adolescents, were significantly more likely to respond more polarized, both after an initial choice for the more (node3) and the less (node2) peer-resistant statement. In addition, at the node1 subitem, girls, compared with boys, had a significantly higher probability to opt for the more peer-resistant statement. With respect to the lowest branching-level choices, after an initial choice for the more peer-resistant statement (node3), the probability for a more polarized response was significantly higher for girls than for boys. However, after an initial choice for the less peer-resistant statement (node2), no significant gender differences emerged. A similar pattern of findings was obtained after controlling for individual differences in age, by including age as a continuous covariate at each node.

Effect sizes indexing effects of intellectual ability and gender on RPI were smaller for the unidimensional (intellectual ability: d = 0.13, gender: d = 0.38), compared with three-dimensional, branching model (intellectual ability: d = −0.38, gender: d = 0.41). That is, they increased with 65.43% and 7.34%, respectively. The branching model parameter estimates, associated standard errors (SEs), test statistics, levels of significance, and effect sizes of effects of intellectual ability and gender are presented in Table 3.

Table 3.

Parameter Estimates, Standard Errors (SEs), Test Statistics, Levels of Significance, and Effect Sizes of Fixed Effects of Intellectual Ability and Gender, Included in the Reparameterized, Unidimensional Model and in the Reparameterized, Three-Dimensional Model.

Effect	Estimate	SE of estimate	Wald statistic	p	Cohen’s d
Unidimensional model
Intellectual Ability	0.087	0.045	1.939	.053	0.130
Gender	0.314	0.076	4.128	<.001	0.379
Three-dimensional model, node1
Intellectual Ability	−0.283	0.057	−4.961	<.001	−0.376
Gender	0.308	0.060	5.139	<.001	0.409
Three-dimensional model, node2
Intellectual Ability	0.994	0.114	8.747	<.001	0.930
Gender	−0.085	0.110	−0.775	.438	−0.080
Three-dimensional model, node3
Intellectual Ability	0.332	0.075	4.454	<.001	0.317
Gender	0.312	0.076	4.082	<.001	0.297

Note. SE = standard error; TD = typically developing; MBID = mild-to-borderline intellectual disability. Parameterization was chosen such that TD adolescents and boys served as a reference. In the unidimensional model, the intellectual ability and gender parameter estimates then indicate the additional effect of being, respectively, an MBID adolescent or a girl to the probability of opting for the more peer-resistant statement. In the three-dimensional model, at the node1 subitem, the intellectual ability and gender parameter estimates then indicate the additional effect of being, respectively, an MBID adolescent or a girl to the probability of opting for the more peer-resistant statement. At the node2 and node3 subitems, the intellectual ability and gender parameter estimates indicate the additional effect of being, respectively, an MBID adolescent or a girl to the probability of opting for the more extreme statement.

Together, both the ANOVA and multidimensional branching model analysis revealed that, as hypothesized (Hypothesis 2a), adolescents with MBID report to be less peer resistant than adolescents without MBID. Also, these analyses revealed that, as hypothesized (Hypothesis 2b), girls report to be more peer resistant than boys. Moreover, a comparison of effect sizes between a unidimensional versus multidimensional branching model revealed that differences in RPI between TD versus MBID adolescents and boys versus girls were substantially and somewhat larger, respectively, when the multidimensionality of items of the RPIQ was considered, compared with when it was not. In addition, the multidimensional branching model analysis showed that, as hypothesized (Hypothesis 3a), independent of their initial choice, adolescents with MBID responded more extremely than adolescents without MBID. Finally, this analysis revealed that girls responded more extremely than boys, but only in case they initially indicated to be more peer resistant.

Discussion

The first aim of the current study was to investigate the uni- versus multidimensionality underlying items of the RPIQ. Given that participants are instructed to, at each item, first choose the option that best describes the group of people (i.e., more vs. less peer resistant) they belong to, and subsequently indicate the degree to which they feel they belong to this group (i.e., “Really true” vs. “Sort of true”), we expected to find a multidimensional structure to underlie each item of the RPIQ (Hypothesis 1a). More specifically, we hypothesized that participants’ initial choices originate in a dimension that is indicative of more or less RPI (Hypothesis 1b), whereas their subsequent choices originate in dimensions that are indicative of Response Polarization (Hypothesis 1c). Regarding this first aim, we found clear evidence in support of the hypothesized multidimensional structure of items of the RPIQ. Model selection procedures and correlational analyses showed that a three-dimensional structure, which could be interpreted as an RPI dimension and two Response Polarization dimensions, best described the data. With respect to the latter two dimensions, results indicated that Response Polarization is best modeled by two separate dimensions. That is, the tendency for Response Polarization after an initial choice for the more peer-resistant statement taps a construct that is different from, but associated with, the tendency for Response Polarization after an initial choice for the less peer-resistant statement.

The second aim of this study was to determine whether taking into account the multidimensionality of items of the RPIQ provides a more fine-grained analysis of individual differences compared with a traditional analysis based on unidimensional Likert-type scale scores. More specifically, we tested whether considering the multidimensionality of the items is of added value in comparing four subgroups of adolescents, that is, boys and girls with and without MBID.

Regarding RPI, results of the multidimensional branching model analysis corroborated results of the traditional analysis based on unidimensional Likert-type scale scores, in that MBID adolescents report to be less RPI than their TD peers. Lower RPI in adolescents with MBID has often been suggested (e.g., Greenspan, 2008), but has not been extensively studied. In one experimental study, Bexkens and colleagues (Bexkens et al., 2017; also see Bexkens, 2013) observed an MBID-related increase in susceptibility to peer pressure. The present results augment this finding, as they indicate that adolescents with MBID not only show increased susceptibility to peer pressure in an experimental setting but also report to be less RPI on a questionnaire, suggesting that they are aware of their difficulty in resisting the influence of their peers. Note that the MBID adolescents in the current study were slightly older than the TD adolescents. As both RPI (Steinberg & Monahan, 2007) and self-consciousness are considered to increase with age, the observed effect of reporting to be less RPI in MBID adolescents may have been underestimated. In addition, the effect of reporting to be less RPI in MBID adolescents may have also appeared larger, would this group have consisted of more severely disabled individuals, with IQs in the low (i.e., IQs: 55-70; APA, 2000, 2013; Schalock et al., 2010) instead of low-to-borderline (i.e., IQs: 55-85; Bexkens, 2013; De Wit et al., 2012) range.

With respect to gender differences in RPI, results of both the traditional analysis based on unidimensional Likert-type scale scores and the multidimensional branching model analysis revealed that girls report to be more RPI than boys. These findings support previous findings (Sumter et al., 2009). Note that girls in the current study were slightly younger than boys. Again, as RPI is considered to increase with age (Steinberg & Monahan, 2007), the observed effect of reporting to be more RPI in girls may have been underestimated.

Effect sizes of effects of intellectual ability and gender on RPI were found to be larger when the multidimensionality of items of the RPIQ was considered compared with when it was not. Together, these findings provide support for the proposal that when the multidimensionality of items of the RPIQ is considered, subtle differences between subgroups in RPI, that otherwise may go unnoticed, may be detected. That is, taking into account possible differences between subgroups in the tendency to respond more or less extremely, seems to provide a more sensitive measure of RPI.

Regarding Response Polarization, results of the multidimensional branching model analysis revealed that MBID adolescents, compared with their TD peers, provide more polarized responses. This is consistent with previous studies indicating a negative association between IQ and an extreme response style (Meisenberg & Williams, 2008; Wilkinson, 1970; for a review, see Van Vaerenbergh & Thomas, 2013). To our knowledge, there are no empirical studies on Response Polarization in individuals with MBID. However, the observed effect of intellectual ability on Response Polarization is consistent with observations of health-care professionals working with adolescents with MBID, who report notable levels of black and white thinking (Vonk, 2014). Note that, like the effect of reporting to be less RPI in MBID adolescents, the effect of providing more polarized responses in these adolescents may have also appeared larger, would this group have consisted of more severely disabled individuals, with IQs in the low (i.e., IQs: 55-70; APA, 2000, 2013; Schalock et al., 2010) instead of low-to-borderline (i.e., IQs: 55-85; Bexkens, 2013; De Wit et al., 2012) range.

With respect to gender differences in Response Polarization, our results show that after an initial choice to be more RPI, girls, compared with boys, provided a more polarized response, by strongly indicating that they are more RPI, whereas no gender differences were observed after an initial choice to be less RPI. Although these divergent gender effects on the different Response Polarization tendencies further support the finding that these tendencies are best modeled by two separate dimensions, they were unanticipated and we have no ready explanation for their origin. Several factors, like black and white thinking and social desirability, could have generated the more polarized responses that we observed. Currently, we have no way of knowing what factors may account for these more polarized responses, and whether these factors are the same for more polarized responses in MBID, compared with TD adolescents, and, in case of more RPI, girls, compared with boys. Therefore, future studies are needed to elucidate these differences between subgroups in Response Polarization, by including independent measures of candidate constructs, such as black and white thinking and social desirability.

Considering our findings, we conclude that using a tree-based modeling approach, which considers the multidimensionality of items of the RPIQ, provides a more fine-grained analysis of individual differences in two ways. First, in an analysis based on traditional unidimensional Likert-type scale scores, differences between subgroups in RPI seem affected by additional differences between subgroups in Response Polarization. More specifically, Likert-type scale scores provide a composite measure of one’s level of RPI as well as one’s levels of Response Polarization. As a consequence, summing these scores may either attenuate (i.e., subgroup differences in RPI and Response Polarization are in opposite directions) or exaggerate (i.e., subgroup differences in RPI and Response Polarization are in similar directions) differences between subgroups, which hinders detecting true differences in RPI. The multidimensional branching model analysis renders a more sensitive measure of between-subgroup differences in RPI, as variance in individuals’ tendency for more or less polarized responses is partialed out (also, see Wetzel et al., 2016). This is of importance given that RPI is proposed to be a protective factor against increased risk taking, and adequate measurement of this construct may assist in identifying subgroups of adolescents at risk. Second, modeling the multidimensionality of items of the RPIQ reveals additional information in terms of an index of an individual’s tendency for Response Polarization. This may be of value, especially in the assessment of traits and attitudes in groups of individuals who are characterized by dichotomous thinking (e.g., black and white thinking among adolescents with MBID; Vonk, 2014).

The present findings suggest several additional venues for future research. First, as for any comparison of subgroups of individuals, our findings of between-subgroup differences in RPI and Response Polarization are informative to the extent that Measurement Invariance (Meredith, 1993) holds. That is, the same model should best fit the data of different subgroups of adolescents. Future research should address this issue by testing for Measurement Invariance.

Second, using the current approach, existing data on the RPIQ, as well as novel data, from clinical groups or individuals from a wide age range, can easily be (re)analyzed. Such a (re)analysis can further test the hypothesis that, after taking into account between–subgroup differences in Response Polarization, differences between subgroups in RPI are increased or reduced, compared with between-subgroup differences in RPI as found in a traditional analysis based on unidimensional Likert-type scale scores. In case of novel data, it would be of interest to correlate RPI estimates based on multidimensional branching model scores to node1 versus unidimensional Likert-type scale scores on one hand with behavioral indices of RPI on the other. One behavioral index of interest in this realm might be risk taking on the STOPLIGHT task (Chein, Albert, O’Brien, Uckert, & Steinberg, 2011)—a simulated driving task on which participants either perform alone or while observed by peers. Relatedly, it would be of interest to further study the merits of the current approach by comparing RPI estimates derived from the multidimensional branching model with RPI estimates derived from revised and simplified versions of the RPIQ. Such revised and simplified versions could either consist of items that only contain the initial choice between the more or less peer-resistant statement (i.e., node1 subitem; see Stautz & Cooper, 2014), or consist of rephrased items based on only the left-sided (or, for the record, right-sided) statement, and which require responses to a 4-point Likert-type scale (for a similar revision of Harter’s Self-Perception Profile for Adolescents; see, Wichstraum, 1995; but, for the importance of modeling response styles, including Response Polarization, when using Likert-type scales in general, see, e.g., Huang, 2016; Khorramdel & von Davier, 2014; Plieninger & Meiser, 2014).

Third, given its potential as a formal measure of an individual’s tendency to respond more or less polarized, additional research is needed to validate our interpretation of participants’ scores on the lower branching-level dimensions in terms of Response Polarization. Importantly in this realm is the superior model fit of a three- over a two-dimensional model, which suggests that participants’ scores to the lower branching-level nodes originate in two distinct, but associated, constructs. However, the present findings are limited to statistical evidence, and additional research is needed to establish the extent to which these two dimensions indeed tap separate constructs that are distinct in a meaningful way. In addition, it would be pivotal to establish that the observed higher scores on these lower branching-level dimensions in MBID adolescents, compared with their TD peers, are not the result of lower scale comprehension among MBID adolescents, but are indeed reflective of their tendency to respond more extremely. Together, to establish convergent validity, future studies may include, in addition to the RPIQ, measures known to be indicative of Response Polarization to test whether scores on the lower branching-level dimensions correlate, and correlate distinctively, with these established measures, and whether considering scores on these dimensions is indeed of added value in comparing subgroups (e.g., low-to-borderline IQ vs. average IQ) of adolescents.

Finally, future studies may test for the generalizability of participants’ scores on Response Polarization beyond measures of RPI. A measure of interest in this respect is Harter’s Self Perception Profile (Harter, 2012), of which items have a similar tree-based formulation and that in fact formed the basis of the tree-based structure of items of the RPIQ.

At least three limitations of the current study should be mentioned. First, all participants included were recruited through regular and special education schools in the Netherlands. As such, results may restrictively apply to Dutch adolescents, and may not be generalizable to adolescents from other (non-Western) countries. Second, susceptibility to peer influence has been pointed out as a crucial factor to study (e.g., Gardner & Steinberg, 2005; Smith et al., 2014; for a review, see Albert et al., 2013) to further our understanding of the period of adolescence. However, including participants from this relatively narrow age range did not permit to properly study developmental trends in differences between subgroups in RPI and Response Polarization. This limitation should be addressed in future research. Finally, interpretation of the observed three-dimensionality underlying items of the RPIQ was only based on statistical evidence; no relationships with other well-established measures of RPI or Response Polarization could be studied. As outlined before, this limitation should also be addressed in future research.

Notwithstanding these limitations, the current study shows that items of the RPIQ (Steinberg & Monahan, 2007) tap a multi- rather than unidimensional construct, consisting of both an RPI dimension and two Response Polarization dimensions. By considering this multidimensionality, we observed that MBID, compared with TD adolescents, report to be less RPI; and girls, compared with boys, report to be more RPI. These differences between subgroups in RPI were more pronounced when the multidimensionality of items of the RPIQ was considered, compared with when it was not. Moreover, MBID, compared with TD adolescents, are more polarized in their responses. Also, after an initial choice to be more RPI, girls, compared with boys, are more polarized in their responses. Together, these results suggest that using a tree-based modeling approach—that adheres to the internal structure of items—yields more sensitive and additional measures to detect subtle differences between subgroups. Pursuing such an approach may therefore be of relevance in the study of individual differences in RPI, as well as other psychological constructs that are measured with scales that contain items with a tree-based structure.

Footnotes

Appendix

On each item, individuals are first asked to choose the option that best describes the group of people (i.e., more vs. less peer resistant) they belong to; they are then asked to indicate to what degree they feel they belong to this group (i.e., “Really true” vs. “Sort of true”). Conventionally, the scores to each item are then aggregated in a 4-point Likert-type scale score, in which the “Really true” and “Sort of true” option of the less peer-resistant statement are coded as 1 and 2, respectively, and the “Sort of true” and “Really true” option of the more peer-resistant statement are coded as 3 and 4, respectively. Note that Items 2, 6, and 10 were reverse coded before data analysis.

Acknowledgements

We would like to thank all adolescents and their parents as well as the schools for participation in the study. We further would like to thank the involved (under)graduate students for their assistance during data collection, with special thanks to Sabine Feith and Tijs Arbouw.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: AB is supported by a grant of the University of Amsterdam; AH (406-11-163), and HH and LD (VICI 453-12-005) are supported by grants from the Dutch National Science Foundation, N.W.O.

Notes

References

Akaike

(1973). Information theory and an extension of the maximum likelihood principle. In Petrov

B. N.

Csaki

B. F.

(Eds.), Second International Symposium on Information Theory (pp. 267-281). Budapest, Hungary: Academiai Kiado.

Albert

Chein

Steinberg

(2013). The teenage brain. Peer influences on adolescent decision making. Current Directions in Psychological Science, 22, 114-120. doi:10.1177/0963721412471347

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.

Bachman

J. G.

O’Malley

P. M.

(1984). Yea-saying, nay-saying, and going to extremes: Black-white differences in response styles. Public Opinion Quarterly, 48, 491-509. doi:10.1086/268845

Bates

Maechler

(2009). lme4: Linear mixed-effects models using S4 classes. Retrieved from https://CRAN.R-project.org/package=lme4

Bexkens

(2013). Risk-taking in adolescents with mild-to-borderline intellectual disability and/or behavior disorder. An experimental study of cognitive and affective processes (Unpublished doctoral dissertation). Retrieved from http://hdl.handle.net/11245/ 1.400656

Bexkens

Huizenga

H. M.

Neville

D. A.

Bredman

Collot d’Escury

A. M. L.

van der Molen

M. W.

(2017). Peer influence effects on risk-taking in male adolescents with mild-to-borderline intellectual disabilities and/or behavior disorder. Manuscript in preparation.

Bexkens

Ruzzano

Collot d’Escury

A. M. L.

van der Molen

M. W.

Huizenga

H. M.

(2014). Inhibition deficits in individuals with intellectual disability: A meta-regression analysis. Journal of Intellectual Disability Research, 58, 3-16. doi:10.1111/jir.12068

10.

Bond

T. G.

Fox

C. M.

(2007). Applying the Rasch model. Fundamental measurement in the human sciences. Mahwah, NJ: Lawrence Erlbaum.

11.

Casey

B. J.

Jones

R. M.

Hare

T. A.

(2008). The adolescent brain. Annals of the New York Academy of Sciences, 1124, 111-126. doi:10.1196/annals.1440.010

12.

Chein

Albert

O’Brien

Uckert

Steinberg

(2011). Peers increase adolescent risk taking by enhancing activity in the brain’s reward circuitry. Developmental Science, 14(2), F1-F10. doi:10.1111/j.1467-7687.2010.01035.x

13.

Cho

S. J.

Gilbert

J. K.

Goodwin

A. P.

(2013). Explanatory multidimensional multilevel random item response model: An application to simultaneous investigation of word and person contributions to multidimensional lexical representations. Psychometrika, 78, 830-855. doi:10.1007/s11336-013-9333-5

14.

Crone

E. A.

Dahl

R. E.

(2012). Understanding adolescence as a period of social-affective engagement and goal flexibility. Nature Reviews Neuroscience, 13, 636-650. doi:10.1038/nrn3313

15.

Dahl

R. E.

(2004). Adolescent brain development: A period of vulnerabilities and opportunities [Keynote address]. Annals of the New York Academy of Sciences, 1021, 1-22. doi:10.1196/annals.1308.001

16.

De Boeck

Bakker

Zwitser

Nivard

Hofman

Tuerlinckx

Partchev

. (2011). The estimation of item response models with the LMER function from the lme4 package in R. Journal of Statistical Software, 39(12), 1-28.

17.

De Boeck

Partchev

. (2012). IRTrees: Tree-based item response models of the GLMM family. Journal of Statistical Software, 20(2), 1-28.

18.

De Boeck

Wilson

(2004). Explanatory item response models. A generalized linear and nonlinear approach. New York, NY: Springer.

19.

De Boeck

Wilson

. (2015). Multidimensional explanatory item response modeling. In Reise

S. P.

Revicki

D. A.

(Eds.), Handbook of item response theory modeling. Applications of typical performance assessment (pp. 252-271). New York, NY: Routledge.

20.

De Wit

Moonen

Douma

. (2012). Guideline effective interventions for youngsters with MID. Utrecht, Netherlands: Dutch Knowledge Center on MID/Landelijk Kenniscentrum LVB.

21.

Defoe

I. N.

Dubas

J. S.

Figner

van Aken

M. A. G.

(2015). A meta-analysis on age differences in risky decision making: Adolescents versus children and adults. Psychological Bulletin, 14, 48-84. doi:10.1037/a0038088

22.

Embretson

Reise

S. P.

(2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.

23.

Gardner

Steinberg

(2005). Peer influence on risk taking, risk preference, and risky decision making in adolescence and adulthood: An experimental study. Developmental Psychology, 41, 625-635. doi:10.1037/0012-1649.41.4.625

24.

Greenleaf

E. A.

(1992). Measuring extreme response style. Public Opinion Quarterly, 56, 328-351. doi:10.1086/269326

25.

Greenspan

(2008). The age of turbulence: Adventures in a new world. New York, NY: Penguin.

26.

Grosbras

M.-H.

Jansen

Leonard

McIntosh

Osswald

Poulsen

. . .Paus

(2007). Neural mechanisms of resistance to peer influence in early adolescence. Journal of Neuroscience, 27, 8040-8045. doi:10.1523/JNEUROSCI.1360-07.2007

27.

Hall

G. S.

(1904). Adolescence: In psychology and its relation to physiology, anthropology, sociology, sex, crime, religion, and education (Vols. I and II). Englewood Cliffs, NJ: Prentice-Hall.

28.

Hambleton

R. K.

Swaminathan

(1985). Item response theory. Boston, MA: Kluwer-Nijhoff.

29.

Harter

(2012). Self-perception profiles for adolescents: Manual and questionnaires (Rev. ed.). Denver, CO: University of Denver.

30.

Harter

Whitesell

N. R.

Junkin

L. J.

(1998). Similarities and differences in domain-specific and global self-evaluations of learning-disabled, behaviourally disordered, and normally achieving adolescents. American Educational Research Journal, 35, 653-680. doi:10.3102/00028312035004653

31.

Huang

H. Y.

(2016). Mixture random-effect IRT models for controlling extreme response style on rating scales. Frontiers in Psychology, 7, 1706. doi:10.3389/fpsyg.2016.01706

32.

Jeon

De Boeck

(2016). A generalized item response tree model for psychological assessments. Behavioral Research Methods, 48, 1070-1085. doi:10.3758/s13428-015-0631-y

33.

Khemka

Hickson

(2006). The role of motivation in the decision making of adolescents with mental retardation. In Swizky

H. N.

(Ed.), International review of research in mental retardation. Vol. 31: Mental retardation, personality, and motivational systems (pp. 73-115). London, England: Academic Press.

34.

Khorramdel

von Davier

(2014). Measuring response styles across the big five: A multiscale extension of an approach using multinomial processing trees. Multivariate Behavioral Research, 49, 161-177. doi:10.1080/00273171.2013.866536

35.

Koolhof

Loeber

Wei

E. H.

Pardini

Collot d’Escury

A. M. L.

(2007). Inhibition deficits of serious delinquent boys of low intelligence. Criminal Behaviour and Mental Health, 17, 274-292. doi:10.1002/cbm.661

36.

Little

R. J. A.

Rubin

D. N.

(1987). Statistical analysis of missing data. New York, NY: Wiley.

37.

López-Sepulcre

De Bona

Valkonen

J. K.

Umbers

K. D.

Mappes

(2015). Item response trees: A recommended method for analyzing categorical data in behavioral studies. Behavioral Ecology, 26, 1268-1273. doi:10.1093/beheco/arv091

38.

Lord

F. M.

(1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

39.

Meisenberg

Williams

(2008). Are acquiescent and extreme response styles related to low intelligence and education? Personality and Individual Differences, 44, 1539-1550. doi:10.1016/j.paid.2008.01.010

40.

Meredith

(1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525-543. doi:10.1007/BF02294825.

41.

Molenaar

Tuerlinckx

van der Maas

H. L.

(2015). A bivariate generalized linear item response theory modeling framework to the analysis of responses and response times. Multivariate Behavioral Research, 50, 56-74. doi:10.1080/00273171.2014.962684

42.

Monahan

K. C.

Steinberg

Cauffman

(2009). Affiliation with antisocial peers, susceptibility to peer influence, and antisocial behavior during the transition to adulthood. Developmental Psychology, 45, 1520-1530. doi:10.1037/a0017417

43.

Monahan

K. C.

Steinberg

Cauffman

Mulvey

E. P.

(2009). Trajectories of antisocial behaviour and psychosocial maturity form adolescence to young adulthood. Developmental Psychology, 45, 1654-1668. doi:10.1037/a0015862

44.

Partchev

De Boeck

(2012). Can fast and slow intelligence be differentiated? Intelligence, 40, 23-32. doi:10.1016/j.intell.2011.11.002

45.

Paus

Toro

Leonard

Lerner

J. V.

Lerner

R. M.

Perron

. . .Steinberg

L. W.

(2008). Morphological properties of the action-observation cortical network in adolescents with low and high resistance to peer influence. Social Neuroscience, 3, 303-316. doi:10.1080/17470910701563558

46.

Plieninger

Meiser

(2014). Validity of multiprocess IRT models for separating content and response styles. Educational and Psychological Measurement, 74, 875-899. doi:10.1177/0013164413514998

47.

Schalock

R. L.

Borthwick-Duffy

S. A.

Bradley

V. J.

Buntinx

W. H.

Coulter

D. L.

Craig

E. M.

. . .Yeager

M. H.

(2010). Intellectual disability: Definition, classification, and systems of supports. Washington, DC: American Association on Intellectual and Developmental Disabilities.

48.

Schwarz

(1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.

49.

Smith

A. R.

Chein

Steinberg

(2014). Peers increase adolescent risk taking even when the probabilities of negative outcomes are known. Developmental Psychology, 50, 1564-1568. doi:10.1037/a0035696

50.

Somerville

L. H.

Jones

R. M.

Casey

B. J.

(2010). A time of change: Behavioral and neural correlates of adolescent sensitivity to appetitive and aversive environmental cues. Brain and Cognition, 72, 124-133. doi:10.1016/j.bandc.2009.07.003

51.

Spear

L. P.

(2000). The adolescent brain and age-related behavioral manifestations. Neuroscience & Biobehavioral Reviews, 24, 417-463. doi:10.1016/S0149 7634(00)00014-2

52.

Stautz

Cooper

(2014). Brief report: Personality correlates of susceptibility to peer influence in adolescence. Journal of Adolescence, 37, 401-405. doi:10.1016/j.adolescence.2014.03.006

53.

Steinberg

(2008). A social neuroscience perspective on adolescent risk-taking. Developmental Review, 28, 78-106. doi:10.1016/j.dr.2007.08.002

54.

Steinberg

Monahan

K. C.

(2007). Age differences in resistance to peer influence. Developmental Psychology, 43, 1531-1543. doi:10.1037/0012-1649.43.6.1531

55.

Sumter

S. R.

Bokhorst

C. L.

Steinberg

Westenberg

M. P.

(2009). The developmental pattern of resistance to peer influence in adolescence: Will teenagers ever be able to resist? Journal of Adolescence, 32, 1009-1021. doi:10.1016/j.adolescence.2008.08.010

56.

Thissen

Wainer

(2001). Test scoring. Mahwah, NJ: Lawrence Erlbaum.

57.

Van Vaerenbergh

Thomas

T. D

. (2013). Response styles in survey research: A literature review of antecedents, consequences, and remedies. International Journal of Public Opinion Research, 25, 195-217. doi:10.1093/ijpor/eds021

58.

Vonk

(2014). Emotionele ontwikkeling. Samenhang met sociale en cognitieve ontwikkeling en ontwikkelingsleeftijd [Emotional development. Association to social and cognitive development and developmental age.] In De Bruijn

Buntinx

W. H.

Twint

(Eds.), Verstandelijke beperking: Definitie en context [Intellectual disability: Definition and context, pp. 101-112] Amsterdam, Netherlands: SWP Publishers.

59.

Weijters

Geuens

Schillewaert

(2010). The stability of individual response styles. Psychological Methods, 15, 96-110. doi:10.1037/a0018721

60.

Wetzel

Lüdtke

Zettler

Böhnke

J. R.

(2016). The stability of extreme response style and acquiescence over 8 years. Assessment, 23, 279-291. doi:10.1177/1073191115583714

61.

Wichstraum

(1995). Harter’s Self-Perception Profile for Adolescents: Reliability, validity, and evaluation of the question format. Journal of Personality Assessment, 65, 100-116. doi:10.1207/s15327752jpa6501_8

62.

Wilkinson

A. E.

(1970). Relationship between measures of intellectual functioning and extreme response style. Journal of Social Psychology, 81, 271-272. doi:10.1080/00224545.1970.9922451

63.

Willoughby

M. T.

Wirth

R. J.

Blair

C. B.

(2011). Contributions of modern measurement theory to measuring executive function in early childhood: An empirical demonstration. Journal of Experimental Child Psychology, 108, 414-435. doi:10.1016/j.jecp.2010.04.007

64.

Zettler

Lang

J. W.

Hülsheger

U. R.

Hilbig

B. E.

(2016). Dissociating indifferent, directional, and extreme responding in personality data: Applying the three-process model to self and observer reports. Journal of Personality, 84, 461-472. doi:10.1111/jopy.12172