Relationships Between Response Styles and the Hofstede and GLOBE Dimensions of Culture in a Sample of Adolescents From 33 Countries

Abstract

A number of studies have examined the relationships between response styles and the Hofstede and GLOBE cultural indices; however, studies involving adolescent samples or examining the effects of national wealth on observed relationships are scarce. This study addresses these gaps by applying simple and partial correlation analysis to the data of 15-year-olds in 33 PISA 2006 countries. The study found that the relationships between response styles and cultural indices in the two frameworks are similar to those in past studies of adult populations. After accounting for GDP per capita, the majority of relationships remained unchanged. However, others, such as Hofstede’s power distance and acquiescence and dis-acquiescence, lost significance, and Hofstede’s masculinity and extreme response styles only gained significance when GDP per capita was held constant. The findings highlight the influence of cultural values on students’ questionnaire-response behaviours, which should be recognised in comparative studies.

Keywords

Response Styles Acquiescence Programme for International Student Assessment Culture Hofstede

Introduction

Response styles—the tendency for respondents to systematically respond to questionnaire items on some basis other than what the items intend to measure (Paulhus, 1991)—affect attitudinal survey data in many ways, including contaminating questionnaires and answers (Baumgartner & Steenkamp, 2001), resulting in the inflation or deflation of the values of some attitudinal variables and, therefore, their relationships with other variables in a study (Clarke, 2000; Dolnicar & Grün, 2007; Moss & Vijayendra, 2018). As a result, the detection and correction of response styles in cross-national survey data, as well as the understanding of their driving factors, have attracted a great deal of research interest.

There is a common practise in the field of response styles research of labelling studies and their findings as ‘cultural differences in response styles,’ even if they do not directly address the element of culture (Buckley, 2009; Chen et al., 1995; Davis et al., 2019; Lee et al., 2002; Morren et al., 2011). Thus, these studies appear to replace countries with cultures; however, even if such substitutions were plausible, we are still reminded that a categorical treatment of “culture” does not tell us which aspects of it cause changes in external variables or the magnitude of such changes (Minkov, 2013). As a result of such admonition, some studies exploring culture’s effects on response styles operationalise “culture” by employing Hofstede’s cultural framework, which, despite its limitations, is acknowledged as a useful method of measuring cultural disparities (Harzing, 2006). Other researchers employ House et al. (2004) Global Leadership and Organizational Behaviour Effectiveness (GLOBE) dimensions of culture. Hofstede’s model (Hofstede et al., 2010) identifies six dimensions of national culture, while GLOBE identifies nine. These models, among others, provide a “consistent and parsimonious quantification of cultural differences between countries” (Beugelsdijk et al., 2015, p. 225), and they were derived from surveys of employees and middle managers from corporations in various countries.

Even though response styles have been studied with adolescent (10–19-year-old) samples, particularly with the Programme for International Student Assessment (PISA) datasets, these studies have been limited to determining the mean distributions of various response styles across countries, as well as their corrections and relationships with academic performance (Buckley, 2009; He et al., 2020; Lu & Bolt, 2015). As far as we are aware, however, there has not been any study done that specifically examines the associations between response styles and the cultural indices in the GLOBE and Hofstede’s cultural frameworks in the context of PISA or with populations that are comparable to PISA. This may be explained in part by the fact that the cultural frameworks have their origins in international business and that the majority of research on how they relate to response styles is conducted in the marketing and management fields, with samples drawn from employees and adult populations. As a result, it is unclear whether the relationships between the observed response styles from adolescent samples and cultural indices across nations are the same as those found in numerous previous studies involving older samples.

This requires investigation for a number of reasons. First, interests, values, and attitudes vary with age (Albarracin & Shavitt, 2018; Gouveia et al., 2015), and respondents project their response styles onto their responses in scales measuring these interests, attitudes, traits, and other behaviours (Paulhus, 1991). It suggests that a separate analysis for adolescents and adults would be useful when drawing relationships between culture and the results of attitude and interest surveys. Second, there is a statistically significant correlation between response styles and age. For example, extreme and dis-acquiescent response styles increase with age (He et al., 2014a; Schneider, 2018; Stone et al., 2019), whereas acquiescence decreases with age (Costello & Roodenburg, 2015; Soto et al., 2008). This variation in response styles by age also demonstrates the significance of analysing samples of adolescents and adults when examining the relationships between response styles and culture. This is especially important considering that the scores of countries on the Hofstede, GLOBE, and other cultural indices are frequently derived from surveys of adult populations, whereas our understanding of how they relate to response styles is largely based on adult samples only. Since studies involving adolescent samples are scarce, this study seeks to fill the gap in the literature by examining the relationships between Hofstede’s and GLOBE’s cultural dimensions and the four response styles determined by Buckley (2009) for 57 PISA 2006 countries.

The study focuses on response styles in PISA 2006 partly because of the growing references to Buckley (2009) for evidence of differences in response styles involving PISA samples (Wetzel et al., 2013; Khorramdel & von Davier, 2014; Huang, 2016; Reynolds, 2022). Also, there is a detailed commentary on Buckley’s computation of response styles using PISA 2006 datasets and the development of an alternate method for estimating response styles using the same data as Buckley (Lu, 2012). Thus, it appears that the PISA 2006 dataset, and more specifically the response styles in Buckley based on it, have received significant scholarly attention; however, no study linking these response styles to cultural variables has been conducted to illuminate how culturally dependent they are. Consequently, this study aims to produce results that contribute new insights and knowledge about response styles within this wave of PISA, which has garnered considerable attention.

We also examine the proposition that a country’s material prosperity is a significant factor that eliminates the relationships between Hofstede’s cultural indices and other external variables (Hofstede, 2011). This supposition has rarely been tested in response style studies, despite speculations that national wealth may confound the relationships between response styles and certain cultural indices (Johnson et al., 2005). By controlling for national wealth, the study’s findings will aid in elucidating the true relationships between response styles and cultural values. The findings of the study will also provide a more holistic lens through which to analyse response styles in cross-national and even domestic student surveys.

Response Styles and Culture

According to Van Vaerenbergh and Thomas (2013), eight major response styles are frequently studied: acquiescence response style (ARS), dis-acquiescence response style (DARS), mid-point response style (MRS), extreme response style (ERS), mild response style (MLRS), net acquiescence response style (NARS), and non-contingent response style (NONC). This study examines three of the most commonly studied response styles: the acquiescence response style (ARS), the dis-acquiescence response style (DARS), and the extreme response style (ERS), as well as the less commonly studied non-contingent response style (NONC).

Acquiescence response style or bias (ARS) is demonstrated when respondents agree with nearly all scale items irrespective of their contents (Saris et al., 2010). Dis-acquiescence (DARS) is the tendency to disagree with scale items regardless of their contents (Harzing, 2006). Extreme response styles (ERS) are a hybrid of ARS and DARS, characterised by respondents’ proclivity to tick responses at both ends of a rating scale. A non-contingent response style is the sum of the absolute differences between responses to item pairs with the highest correlation and common means among respondents (Baumgartner & Steenkamp, 2001), and it occurs when respondents carelessly and unintentionally respond to scale items (Van Vaerenbergh & Thomas, 2013).

Certain regions of the world are believed to be characterised by specific response styles. For instance, the majority of studies categorise Asians as middle respondents. Studies comparing Asians and North Americans (Chen et al., 1995; Lee et al., 2002; Tasaki & Shin, 2017; Wang et al., 2008) and Asians and Western countries in general (Dolnicar & Grün, 2007; Harzing, 2006) have found this to be the case. Also, ARS is lower among Asians (Dolnicar & Grün, 2007; Tasaki & Shin, 2017). Differences in response styles within regional blocks are also documented. For instance, Baumgartner and Steenkamp’s (2001) study of eleven EU countries revealed that Greece and Portugal have a higher incidence of ARS than the other countries analysed. Compared to European countries, Latin American countries have a higher ARS; compared to Asian countries, they have a higher ERS and a lower MRS (Moss & Vijayendra, 2018). Even immigrants of diverse national origins residing in the same country exhibit distinct response patterns (Morren et al., 2012).

One theoretical perspective asserts that response styles reflect how individuals communicate and interact with one another within well-defined cultural contexts (Smith, 2004). Thus, in order to understand why certain countries or regional blocs respond to survey questions the way they do, the focus should be on culture rather than the geographic unit, which is merely a label for purposes of identification and incapable of causing anything to happen (Minkov, 2013). The study by Morren et al. (2012), for example, supports the need to consider culture as the fundamental analytical unit. Their mixed-methods research in the Netherlands indicates that the response styles of immigrant respondents are influenced by the cultural expectations of their country of origin, suggesting that using a country as the unit of analysis in a cross-national survey may obscure substantial differences that exist within and between countries.

Though one of the difficult concepts to define, culture “comprises the patterns of ideas, values and beliefs common to a particular group of people, their ‘characteristic’ ways of thinking and feeling” (Inglis, 2005, p. 5). The quest to operationalise culture has yielded, among other frameworks, Hofstede’s cultural dimensions, GLOBE’s cultural values, and Schwartz, (1994) cultural value orientations. However, Hofstede’s and GLOBE’s (House et al., 2004) dimensions of culture have been the frameworks most frequently used to explain why certain response styles are typical of certain countries. For example, using the concept of cultural processes underlying response styles, Johnson et al. (2010) argue that ARS is prevalent in collectivist cultures due to their high regard for deference and politeness and that ERS is commonplace in high power distance and masculine cultures due to their sincerity, competitiveness, independence, assertiveness, and decisiveness. These observations suggest that the values proposed by the cultural frameworks are not abstract notions but rather concrete aspects of the cultural resources available to and experienced by respondents (Ainley & Ainley, 2019), and thus influence how they respond to attitude-related questions.

Previous Findings on the Relationships Between Response Styles and the Hofstede and GLOBE Cultural Dimensions

According to the Hofstede cultural model, extreme response styles (ERS) are prevalent in countries with a high power distance (Johnson et al., 2005; Peterson et al., 2014), but they tend to decline in individualistic cultures (Peterson et al., 2014); GLOBE’s power distance and Hofstede’s power distance are positively associated with ARS (Harzing, 2006; Smith, 2004) but negatively associated with dis-acquiescence (DARS) (Hoffmann et al., 2013). Also, Hofstede’s individualism is negatively related to ARS (Harzing, 2006; Johnson et al., 2005; Rammstedt et al., 2017; Smith & Fischer, 2015) but positively related to DARS (Hoffmann et al., 2013). While some studies report a positive relationship between Hofstede’s uncertainty avoidance and ARS, others report a negative relationship (Harzing, 2006; Johnson et al., 2005, 2010). Further, whereas Johnson et al. (2005) found a negative relationship between Hofstede’s masculinity and ARS, a review of the literature by Johnson et al. (2010) revealed that the vast majority of studies find no relationship between masculinity and ARS.

Lastly, Hoffmann et al. (2013), one of the few studies to examine all of GLOBE’s cultural dimensions, found that in-group collectivism is positively associated with ARS but negatively associated with DARS and ERS; assertiveness is positively associated with DARS and negatively associated with ARS; gender egalitarianism is positively associated with ARS and ERS; humane orientation is positively associated with ARS but negatively associated with DARS; future orientation is negatively associated with ARS and ERS but positively associated with DARS; and uncertainty avoidance is negatively related to ARS and ERS but positively related to DARS.

Hofstede’s Dimensions of Culture

The first four of Hofstede’s cultural dimensions—power distance, uncertainty avoidance, individualism-collectivism, and masculinity-femininity—were developed in the early 1980s using data from an IBM survey of the attitudes and values of employees in fifty countries during the 1960s and 1970s (Hofstede, 2011). Over time, replication studies and “informed estimates” have been used to include additional countries (Hofstede et al., 2010, p. 56). Long and short-term orientation was added in 1988, but from 2007 to 2008, new country estimates were derived exclusively from World Values Survey (WVS) data covering 1994–2004. Using the same WVS data, the final dimension, indulgence and restraint, was added in 2007. Other contributors, particularly Misho Minkov, made substantial contributions to the final two dimensions (Hofstede et al., 2010). Building on the notion of cultural processes underlying the occurrence of certain response styles as described above (Johnson et al., 2010), we make additional propositions about how Hofstede’s cultural dimensions may explain patterns of certain response styles.

Power Distance

Hofstede et al. (2010) define power distance as the extent to which those with less power in social institutions and organisations accept and believe that power is not distributed equally. In line with the literature, we expect a positive relationship between power distance and both acquiescence (ARS) and extreme response styles (ERS). Also, we expect a negative relationship between it and dis-acquiescence (DARS). Concerning the cultural processes underlying these relationships, we argue that these relationships may partly be due to the fact that in high-power distance countries, contradiction or saying “no” (disagreement) is frowned upon (Hofstede et al., 2010). This deposition is expected to have the stated effect on ARS and DARS in the context of survey responses, as it could lead to a greater proportion of respondents agreeing than disagreeing with questionnaire items.

Individualism

This describes societies in which members have loose relationships with one another (Hofstede et al., 2010). In highly individualistic cultures, individuals are free to speak their minds and openly disagree on issues (Hofstede et al., 2010), in part because they are viewed as independent actors who can act based on their own personal interests and without consultation (Silber & Johnson, 2020; Da’as, 2017). In responses to survey questionnaires, this underlying mechanism may produce high dis-acquiescence (DARS). Also in line with literature, this study expects a positive relationship between individualism and DARS and a negative relationship between individualism and ARS.

Masculinity

A society is masculine when there are clear distinctions between the emotional roles of men and women, i.e., men are expected to be assertive, tough, and work for material success, whereas woPlease provide author biography for all authors for this article.mPlease provide author biography for all authors for this article.en are expected to be submissive and tender (Hofstede et al., 2010). Due to the high degree of assertiveness and decisiveness of more masculine cultures, respondents may choose the most extreme options on survey instruments (Johnson et al., 2005); consequently, we expect a positive association between masculinity and ERS.

Uncertainty Avoidance

This is the extent to which members of a cultural group are prepared to deal with unanticipated events (Hofstede et al., 2010). High uncertainty avoidance cultures are defined by a high sense of urgency and stress, which leads to individuals taking highly risky actions and completing tasks more quickly (Hofstede et al., 2010). In the context of completing survey questionnaires, faster task completion may lead to careless responses, which is a defining trait of NONC; thus, this study anticipates a positive relationship between uncertainty avoidance and NONC. Because this dimension is also associated with a greater propensity for individuals to seek consensus (Silber & Johnson, 2020) and intolerance for deviant opinions (Hofstede, 2011), respondents in such cultures may be more likely to agree with questionnaire items than disagree; hence, this study expects a positive relationship between uncertainty avoidance and ARS and a negative relationship between it and DARS.

Long-Term Orientation

According to Hofstede et al. (2010), this dimension describes how societies promote future-oriented virtues, such as perseverance and thrift. In particular, thrift motivates actions such as not wasting things (Ratchford et al., 2021). As far as we are aware, the relationships between response styles and long-term orientation have scarcely been investigated in the literature. However, since individuals in long-term oriented countries are more likely to avoid wasting resources, they may also be more likely to select disagree scale categories, resulting in a positive relationship between LTO and DARS.

Indulgence

According to Hofstede et al. (2010), indulgence is the disposition to freely gratify fundamental human desires associated with life enjoyment and fun. Since high indulgence is associated with greater unrestrained self-expression (Hofstede et al., 2010), we could expect this dimension to have a positive relationship with DARS. However, similarly to LTO, the relationships between response styles and indulgence have scarcely been studied in the literature.

GLOBE’s Dimensions of Culture

The Global Leadership and Organizational Behaviour Effectiveness (GLOBE) project of House et al. (2004) produced two frameworks, one measuring societal culture and the other, organisational culture. They categorise societal culture into nine dimensions of practices and values. In contrast to the value profile, which is concerned with the desired ideals of a given society, the practice profile is concerned with actual cultural practises (Liddell, 2005). In their later exegesis, they averred that “reported cultural practices (but not values) are...predictive of societal phenomena...On the other hand...reported cultural values and not practices are associated with reported attributes of outstanding leadership across GLOBE countries” (Javidan et al., 2006, p. 903). Since this study is about response styles and involves adolescent samples, we therefore consider only GLOBE’s practices dimensions of culture.

It is important to note that GLOBE’s dimensions are derived from Hofstede’s cultural dimensions (Hofstede, 2006). As with Hofstede, GLOBE’s cultural dimensions include power distance and uncertainty avoidance. GLOBE’s future orientation dimension intends to measure the same societal value as Hofstede’s long-term orientation dimension. We therefore anticipate that these three GLOBE indices will have the same relationship with the response styles in this study as their Hofstede counterparts. GLOBE’s institutional and in-group collectivism are variants of Hofstede’s collectivism (Peterson, 2004); consequently, we expect GLOBE’s institutional and in-group collectivism to relate to the response styles in this study in the opposite way that Hofstede’s individualism relates to the same response styles. Also, Hofstede’s masculinity-femininity dimension is split into assertiveness (masculinity) and gender egalitarianism in GLOBE (femininity), and some items from GLOBE’s humane orientation dimension are very similar to Hofstede’s femininity (Hofstede, 2006). Consequently, GLOBE’s assertiveness is expected to correlate with response styles in the same way that Hofstede’s masculinity does, but GLOBE’s gender egalitarianism and humane orientation are expected to correlate with response styles in the opposite way that Hofstede’s masculinity does. Note that compared to Hofstede’s (2006) dimensions, GLOBE includes an additional dimension—performance orientation. In the literature, there are no clear relationships between performance orientation and response styles; for example, there are both positive (He et al., 2014b) and negative (Hoffmann et al., 2013) relationships between performance orientation and ARS.

Although the expected relationships between the GLOBE dimensions of culture and response styles are based on Hofstede’s, we still analyse the two cultural dimensions because, while House et al. (2004) presented their dimensions as related versions of Hofstede’s first five dimensions, there is no consensus in the literature that this is the case (Minkov & Blagoev, 2012). Because of the lack of consensus, it is possible that the proposed relationships between the GLOBE dimensions and response styles in this study will not hold true in all cases.

GDP per Capita

Our decision to control for GDP per capita was based on both theory (Hofstede, 2011; Hofstede et al., 2010) and recommendation (Johnson et al., 2005). First, Hofstede et al. (2010) posit that a major criterion that determines whether a cultural dimension forms a valid and distinct dimension is the extent to which it correlates with national wealth. For this reason, Hofstede (2011) maintained that in correlating the cultural variables with other variables, it is important to control for national wealth, and in doing so, the relationships between the cultural variables and the target variables would typically disappear. In response styles studies, Johnson et al. (2005), recommended that when examining the relationships between response styles and culture, it would be important to control for national wealth, based on their finding of a strong negative relationship between a nation’s wealth and acquiescence. To the best of our knowledge, only He et al. (2014) controlled for national wealth in their analysis, but this only covered GLOBE’s value dimension and one of the response styles in our study, ARS. Since other studies also report a negative relationship between wealth and ARS (Meisenberg & Williams, 2008; Van Dijk et al., 2009), we would examine the relationship between per capita GDP and the response styles from the adolescent sample prior to controlling for per capita GDP in the relationship between response styles and cultural indices. Based on the foregoing, we anticipate that once GDP per capita is taken into account, there would be changes in the observed relationships between given cultural variables and response styles. Despite the fact that we controlled for wealth, we acknowledge that other national-level variables, such as education expenditure, the gender inequality index, and the corruption perception index, can also be considered when examining the relationships between culture and response style.

Methods

Data for Hofstede’s Cultural Dimensions

The study is based on secondary data analysis. Individual country scores for Hofstede’s six cultural dimensions are accessible to the public and were retrieved from Hofstede Insights (n.d). Only PISA 2006 countries with validated dimension scores were considered in this study. Below is a brief explanation of how country scores are calculated for each dimension.

Power Distance

Hofstede et al. (2010) measured this based on the responses of employees in non-managerial positions from various countries to three basic questions about how to deal with the fact that people are not equal. One of the questions, for instance, read, “how frequently, in your experience, does the following problem occur: employees being afraid to express disagreement with their managers?” (Hofstede et al., 2010, p. 56). They calculated the mean score based on the responses and standardised the score from zero to one hundred, with higher scores indicating high-power distance between subordinates and superiors. The highest and lowest scores on this dimension for the countries included in this study are 93 for Russia and 11 for Austria.

Individualism

Hofstede et al. (2010) derived individualism from three correlated items (the importance of personal time, freedom, and personal challenge), emphasising the degree to which employees were independent from their organisations. They also calculated the mean score based on the responses and standardised the score from zero to one hundred, with higher scores representing greater individualism. The highest and lowest scores on this dimension for the countries included in this study are 91 for the USA and 13 for Columbia.

Masculinity

They also measured masculinity based on employees’ responses to the importance of specific work goals, such as earnings, recognition, advancement, and challenge. They calculated and standardised the mean score, which ranged from zero to one hundred; the higher the score, the more masculine the country. The highest and lowest scores on this dimension for the countries included in this study are 79 for Austria and 5 for Sweden.

Uncertainty Avoidance

They measured uncertainty avoidance based on three strongly correlated variables: job stress, rule orientation, and the length of time employees believe they would work for an organisation. They computed and standardised the mean score, which ranged from zero to one hundred, with scores closer to one hundred indicating strong uncertainty avoidance. The highest and lowest scores on this dimension for the countries included in this study are 100 for Greece and 23 for Denmark.

Long-Term Orientation

Country scores on this dimension were derived from World Values Survey questions regarding national pride, the significance of service to others, and thrift as an important character trait in children. The mean scores on this dimension were also standardised and ranged from zero to one hundred; the higher the score, the more long-term oriented a country. The highest and lowest scores on this dimension for the countries included in this study are 93 for Taiwan and 13 for Columbia.

Indulgence

They measured this dimension using some items (happiness, life control, and the importance of leisure) from the World Values Survey. Country scores on this dimension were also standardised and ranged from zero to one hundred, with higher scores indicating greater indulgence. The highest and lowest scores on this dimension for the countries included in this study are 97 for Mexico and 17 for Hong Kong.

Data for GLOBE Cultural Values

Individual country scores on the GLOBE practice dimensions—performance orientation, future orientation, assertiveness, humane orientation, institutional collectivism, in-group collectivism, gender egalitarianism, power distance, and uncertainty avoidance—are available to the public and were extracted from GLOBE (n.d.). The questionnaire data for these dimensions were collected by House et al. (2004) from 17,000 middle managers in three distinct industries (telecommunications, food services, and finance) and sixty-two cultures. They measured each dimension on a scale ranging from 1 to 7.

The GLOBE study covered only 62 countries, and of the PISA 2006 countries, only 33 were part of the GLOBE study and have their national scores available. The Hofstede’s dimensions of culture also had scores for 46 of the countries, but we limited the study to 33 countries that had scores for both the Hofstede and GLOBE dimensions to allow for a fair comparison of the results for the two frameworks. Japan was specifically excluded because the scores for its response styles were considered outliers (Buckley, 2009).

Finally, we observed from the country scores on the dimensions of culture that if, for example, country ‘A’ scores higher than country ‘B’ on Hofstede’s power distance, it does not necessarily mean that country ‘A’ would score higher than country ‘B’ on GLOBE’s power distance. For instance, Poland and Portugal’s scores on Hofstede’s power distance are 68 and 63, respectively, whereas their scores on GLOBE’s power distance are 5.10 and 5.44, respectively. The differences may be partially explained by the fact that Hofstede and GLOBE scores were derived from different types of respondents in each country. The differences may also imply that, while equivalent cultural indices in the two frameworks are expected to relate in the same direction with external variables, the strengths of their relationships may not necessarily be the same.

Response Styles

The mean estimates for the four response styles—acquiescence response style (ARS), dis-acquiescence response style (DARS), extreme response style (ERS), and non-contingent response style (NONC)—were adopted from Buckley (2009). We requested and received Buckley’s approval to extract and use this data for this study. Appendix Table A1 contains a list of the thirty-three countries examined in this study, as well as their estimated mean response styles and how they were calculated.

Statistical Analysis

We first conducted a Pearson correlation analysis to determine the relationships between GDP per capita and the four response styles, as well as the relationships between Hofstede (six) and GLOBE (nine) cultural indices and the four response styles: the acquiescence response style (ARS), the dis-acquiescence response style (DARS), the extreme response style (ERS), and the non-contingent response style (NONC). Similar studies that focused on national-level analysis also performed correlation analysis (Harzing, 2006; Hoffman et al., 2013); however, studies that combine national-level and individual-level measures such as education and gender to explain response styles typically use hierarchical linear modelling techniques (Johnson et al., 2005; Rammstedt et al., 2017). We also conducted a partial correlation analysis, wherein we entered one response style and one cultural dimension at a time while holding GDP per capita constant. Since the present study’s response styles are derived from the PISA 2006 datasets, we used each country’s GDP per capita in 2006, which can be found online in World Bank Data (n.d.). All analyses were performed using IBM SPSS Statistics (Version 28).

Results

The correlation between per capita GDP and the four response styles—ERS, ARS, DARS, and NONC—is presented in Table 1. For ease of comparison, the results for both the simple and partial correlation for the cultural indices and the four response styles have been combined and presented in a single table, Table 2. Appendix Figure 1 shows an example of visualisation of the correlation between Hofstede’s individualism and the four response styles for illustrative purposes.

Table 1.

Correlations Between GDP per Capita and Response Styles—ERS, ARS, DARS and NONC.

	ERS	ARS	DARS	NONC
GDP per capita (n = 33)	−.275	−.377*	.430*	−.031

Level of significance: *p < .05, all two-tailed.

Table 2.

Correlations of ERS, ARS, DARS, and NONC With Hofstede and GLOBE’s Cultural Indices, Before and After Adjusting for GDP per Capita (n = 33).

	ERS		ARS		DARS		NONC
Hofstede dimensions of culture	Simple corr	Partial corr	Simple corr	Partial corr	Simple corr	Partial corr	Simple corr	Partial corr
Power distance	.344	.222	.459^**	.299	−.488^**	−.295	−.042	−.087
Individualism	−.514^**	−.484**	−.594^**	−.509**	.355^*	.055	−.032	−.013
Masculinity	.339	.352*	.304	.326	.147	.165	.314	.314
Uncertainty avoidance	.251	.117	.259	.054	−.033	.293	.418^*	.491**
Long-term orientation	.165	.185	.086	.112	.316	.328	.108	.109
Indulgence	−.124	−.040	−.161	−.046	.147	.023	.122	.135
GLOBE practices
Uncertainty avoidance	−.119	.109	−.215	.076	.395^*	.144	−.096	−.104
Future orientation	−.133	.086	−.216	.071	.347^*	.072	−.066	−.062
Power distance	.591^**	.544**	.635^**	.563**	−.191	−.002	.293	.311
Institutional collectivism	−.313	−.253	−.299	−.214	−.057	−.210	−.461^**	−.472**
Humane orientation	−.118	−.127	−.029	−.037	−.370^*	−.403*	−.544^**	−.544**
Performance orientation	.076	.193	.022	.179	.228	.090	−.182	−.183
In-group collectivism	.593^**	.575**	.668^**	.607**	−.331	−.060	.078	.078
Gender egalitarianism	−.438^*	−.492**	−.352^*	−.430	−.342	−.326	−.159	−.164
Assertiveness	.277	.327	.166	.233	.458^**	.451*	.475^**	.482**

Level of significance: **p < .01, *p < .05, all two-tailed; corr = correlation.

Simple Correlation Analysis

From Table 1, it can be observed that whereas GDP per capita relates negatively to ARS and positively to DARS, it has no relationship with ERS or NONC. From Table 2, it can also be observed that Hofstede’s masculinity, long-term orientation, and indulgence have no relationship with any of the four response styles. However, Hofstede’s power distance has a positive relationship with ARS and a negative relationship with DARS. Also, whereas Hofstede’s individualism has negative relationships with ERS and ARS, it is positively related to DARS. Finally, NONC is positively associated with Hofstede’s uncertainty avoidance.

For GLOBE practices, performance orientation has no relationship with any of the four response styles. However, both GLOBE’s uncertainty avoidance and future orientation practices are positively related to DARS. GLOBE’s practices power distance is also positively related to ERS and ARS. Further, whereas GLOBE’s institutional collectivism is only negatively related to NONC, its in-group collectivism practice is positively related to ERS and ARS. Finally, for the three GLOBE gender-related indices, gender egalitarianism is negatively related to ERS and ARS, assertiveness is positively associated with DARS and NONC, and humane orientation is negatively related to DARS and NONC.

Hofstede’s Dimensions and Response Styles with Constant per Capita GDP

The partial correlation analysis revealed a number of significant changes in the strength and directions of correlations between some response styles and cultural dimensions. On Hofstede’s cultural dimensions, when GDP per capita was taken into account, the positive relationship between power distance and ARS and the negative relationship between power distance and DARS all became statistically insignificant. After adjusting for GDP per capita, the relationship between Hofstede’s individualism and ERS and ARS remained statistically significant, whereas the relationship between individualism and DARS became statistically insignificant. The relationship between masculinity and ERS, which was not statistically significant, turned significant after the adjustment. It was also observed that the significance levels for masculinity and ARS were quite modest (p = .068), just as was the case between masculinity and NONC (p = .080) after the adjustment. Finally, the associations between uncertainty avoidance, long-term orientation, and indulgence and the four response styles remained relatively the same after accounting for per capita GDP.

GLOBE’s Practices and Response Styles with Constant per Capita GDP

The positive relationships between uncertainty avoidance and DARS, future orientation and DARS, and the negative relationship between gender egalitarianism and ARS for the GLOBE practices dimensions all lost statistical significance after adjusting for GDP per capita. Even though there is no relationship between uncertainty avoidance and any of ERS or ARS, before and after controlling for GDP per capita, there is a change in the direction of their relationships, from negative to positive. The relatively strong and positive correlation between GLOBE’s power distance and ERS and ARS decreased marginally but stayed in the same direction. These same patterns of stable relationships were observed for the remainder of GLOBE’s practices indices and the response styles after the control for per capita GDP.

Discussion

In this study, we performed two sets of analyses. The first examined the relationships between Hofstede’s and GLOBE’s cultural dimensions and four response styles (ERS, ARS, DARS, and NONC), as well as the relationships between per capita GDP and the four response styles. On the premise that national wealth can confound the relationships between national-level cultural variables and other national-level variables, we conducted a second analysis that controlled for per capita GDP in the relationships between the four response styles and the GLOBE and Hofstede’s cultural dimensions. Our sample included adolescents—15-year-olds—from 33 countries who took part in PISA 2006. Our overarching objective was to determine whether the relationships between the response styles of this sample of students and the national-level cultural values proposed by the two cultural frameworks are the same or different than those found in a number of studies with older samples.

In general, our findings are consistent with the majority of previous studies based on adult samples. Our study of this adolescent sample corroborates the negative association between wealth and ARS found in previous studies. For Hofstede’s dimensions and culture in general, power distance and individualism are the most researched, and our findings are consistent with what has been established in the literature regarding their relationships with ARS and DARS in particular: power distance relates positively to ARS and negatively to DARS, whereas individualism relates negatively to ARS and positively to DARS. We expected GLOBE’s power distance to exhibit similar relationships to Hofstede’s power distance, but this was statistically supported only for ARS.

Long-term orientation (future orientation for GLOBE) and uncertainty avoidance in both the Hofstede and GLOBE dimensions were expected to correlate with the four response styles in the same way. The expected relationships between Hofstede’s long-term orientation and response styles were not statistically supported. Also, the only statistically supported hypothesis for Hofstede’s uncertainty avoidance was its relationship with NONC. This means that for this sample of adolescents, increased stress and a sense of urgency to complete the PISA 2006 survey questionnaires were associated with more careless or unintentional responses to the questions. On the GLOBE model, both future orientation and uncertainty avoidance had statistically significant relationships with DARS. Thus, GLOBE’s future orientation statistically met one of the expectations set for Hofstede’s long-term orientation: a positive relationship with DARS. In contrast, the positive relationship between GLOBE’s uncertainty avoidance and DARS contradicts the expectations for Hofstede’s uncertainty avoidance and DARS. There are at least two reasons why this contradiction may have occurred or why uncertainty avoidance in the GLOBE and Hofstede dimensions of culture may not relate in the same direction to a particular response style. First, Hofstede’s uncertainty avoidance is not a stable predictor of response styles, especially ARS, as negative and positive relationships between uncertainty avoidance and ARS have been reported in the literature (Johnson et al., 2010). Second, despite the fact that GLOBE’s uncertainty avoidance is conceptualised as an equivalent to Hofstede’s, correlation analyses tend to show that GLOBE’s uncertainty avoidance relates to other variables similarly to Hofstede’s individualism and not as Hofstede’s uncertainty avoidance does (Minkov & Bangloev, 2012). It is possible that the positive relationship between GLOBE uncertainty avoidance and DARS reflects the second argument, especially considering its striking similarity to the relationship between Hofstede’s individualism and DARS.

The expectation that GLOBE’s institutional and in-group collectivism would relate to response styles in the opposite ways that Hofstede’s individualism would relate to the same response styles, because these GLOBE dimensions are argued to be variants of Hofstede’s collectivism (the opposite of individualism) (Peterson, 2004), was not supported in all cases. The results for in-group collectivism supported this expectation for two of the response styles, as it was found to have strong positive relationships with ERS and ARS, whereas individualism had a negative relationship with ERS and ARS. This hypothesis was not supported for institutional collectivism, possibly because institutional collectivism is uncorrelated with in-group collectivism and fails to correlate with the majority of its nomological networks (Minkov & Blagoev, 2012).

We anticipated a positive relationship between masculinity and ERS, but the results indicate there is no statistically significant relationship between the two. GLOBE’s assertiveness, which was expected to relate in the same way to the response styles as Hofstede’s masculinity, was found to be positively related to both DARS and NONC. Since we expected GLOBE’s humane orientation to relate to response styles in the opposite ways that Hofstede’s masculinity and GLOBE’s assertiveness do, the discovery of a strong negative relationship between GLOBE’s humane orientation and both DARS and NONC is consistent with the study’s expectations. Since the expectations for GLOBE’s gender egalitarianism were also identical to those for humane orientation, the finding of strong negative relationships between gender egalitarianism and both ERS and ARS was also consistent with expectations.

Finally, GLOBE’s performance orientation was found to have no statistically significant relationship with any response style. The results of this study may be supporting previous observations that performance orientation, in particular, rarely correlates with its nomological networks (Minkov & Bangloev, 2012). However, Hoffmann et al. (2013) found that performance orientation is negatively associated with ARS, while He et al. (2014b) found that it is positively associated with ARS. Hofstede’s indulgence was another dimension with no statistically significant relationship to any response style. Indulgence, in particular, is a new dimension that has received little research attention (Hofstede et al., 2010). This likely extends to response styles research, as information on how this dimension relates to response styles is scarce in the literature.

The Effects of per Capita GDP on Substantial Relationships are Mixed

After adjusting for GDP per capita, the magnitudes and directions of the relationships between some response styles and the Hofstede and GLOBE cultural indices changed, while they remained stable for the vast majority. Based on Hofstede’s (2011) hypothesis, we anticipated that statistically significant relationships would become insignificant after adjusting for per capita GDP. This expectation was met by the relationships between Hofstede’s power distance, ARS, and DARS. Hofstede’s individualism and DARS also lost statistical significance after controlling for per capita GDP, whereas its relationships with ARS and ERS remained unchanged.

In addition, we found a statistically significant relationship between Hofstede’s masculinity and ERS, with the significance levels for the same dimension and ARS and between it and NONC increasing after adjustment for GDP per capita. This indicates that the dimension is susceptible to variables associated with wealth. This is likely the case, as Hofstede et al. (2010) state that although this dimension is the most contentious of their cultural indices, the influence of masculinity on other variables did not become apparent until wealth variables were controlled for in the majority of the studies they reviewed. Few changes were observed in GLOBE’s practices dimensions: the associations between DARS and uncertainty avoidance and future orientation lost their statistical significance. The other was gender equality and ARS, which lost statistical significance after being adjusted for per capita GDP.

In general, the majority of relationships that lost statistical significance following the control for per capita GDP occurred on the DARS. Given the strong relationship between DARS and per capita GDP, this finding suggests that differences in country-level affluence may be driving differences in national dis-acquiescent behaviours (DARS) more than the cultural indices involved. In addition, the majority of cultural indices examined in this study were unaffected by the adjustment for GDP per capita, indicating that their observed relationships with response styles probably exist regardless of differences in country-level wealth.

Conclusions

Recognising that what is known about adults’ response styles in cross-cultural and psychological research cannot be assumed to be automatically the same for adolescents (Gilman et al., 2008), this study investigated how the response styles of a previously under-examined group, adolescents, relate to national-level cultural variables identified in the GLOBE and Hofstede’s cultural frameworks. In doing so, the study also contributed to the advancement of knowledge in cross-cultural research, particularly in relation to the observation that very few correlational studies exist to support the notion of culture-dependent response styles (Hoffmann et al., 2013). The study also extended the investigation of the possibility of national wealth confounding the relationships between culture-level variables and other target variables (Hofstede et al., 2010) to response styles studies by controlling for per capita GDP. In general, the findings of this study are consistent with those of several adult sample-based studies, lending credence to the notion that the cultural dispositions within which respondents’ lives are embedded influence their responses to survey questions, and that this may apply to all age groups within a population. The findings of this study have implications for cross-cultural research. It draws attention, in particular, to the fact that when attributing cultural indices, particularly power distance and individualism, to acquiescent and dis-acquiescent response behaviours, researchers should consider adjusting for wealth before drawing conclusions about observed relationships.

Even though the results suggest that certain student questionnaire response behaviours may be influenced by cultural factors, there are limitations to the associational conclusions that can be drawn between culture and response styles, particularly at the individual level. This is well echoed in De Mooij and Hofstede (2010). From an ecological fallacy standpoint, De Mooij and Hofstede (2010, p. 101) note that while culture may be useful for explaining observed variations in phenomena at the national level, it cannot be considered a “one king-size personality” that explains individual behaviours. According to them, the properties of diverse groups of individuals at the national level are aggregated and presented as culture-level variables. Because of the diversity embedded in the data from which cultural measures are derived, what holds true for the association between a cultural-level variable and a target national-level variable may not therefore hold true at the individual level. To understand the drivers of individual-level response behaviours in the context of PISA in particular, additional research is therefore required, especially when one considers that there are as many cultural variations within a country as there are between countries (Taras et al., 2016). Such studies could focus on gender, ethnicity, socio-economic background, minority and immigration status, locality, or any other significant social variable that categorises people.

There are some potential methodological limitations to the response styles in this study that should be highlighted. Such methodological issues may have implications not only for the response styles in this study but for the study of response styles in general. First, the response styles in this study were based on the scale anchors, disagree/agree (A/D). This method can yield response styles that differ from those obtained using other methods. For example, in an experimental study, Saris et al. (2010) demonstrated that the item-specific response method reduces the incidence of ARS compared to the A/D scale. With the item-specific method, survey responses often include specific information about the question (Liu et al., 2015). Additionally, the questionnaire items used to ascertain the response styles were positively worded. However, there is some evidence suggesting that positively worded questionnaires increase acquiescence bias (Salazar, 2015). To reduce the likelihood of such bias, it is suggested that surveys include both positively and negatively phrased questions (Baumgartner & Steenkamp, 2001; Graeff, 2005). In addition, in PISA 2006, students filled out the questionnaires using paper and pencil, and there is some evidence that paper and pencil can lead to greater acquiescence (Hofstede et al., 2010). Compared with other methods, ARS and ERS, for example, are higher when respondents answer with paper and pencil as opposed to the online method (Weijters et al., 2009). All of these methodological concerns can have an impact on the mean estimates of response styles and, as a result, the relationships that can be drawn between them and culture-level variables.

Despite the fact that the majority of the findings from this study involving a PISA sample of students were consistent with previous findings primarily derived from organisational studies and adult samples, it is important to note that the study was limited to only 33 countries, not all of which may have been included in the studies with which these findings may be compared with. In addition, because the correlational analysis was conducted with only 33 countries, it is possible that the observed relationships between response styles and cultural variables would be distinct if a larger number of countries were included in the analysis. Related to this is the geographic representation of the countries: the majority of the countries included are wealthy industrialised countries that may share certain attributes. We therefore recommend that future studies try as much as possible to diversify the countries to, for instance, include African countries. This is made even more compelling by the finding that GDP per capita changed, for the most part, the relationships between dis-acquiescent response behaviour and certain cultural indices.

Finally, because this study is a first step in investigating culture-dependent response styles in the context of PISA specifically, it opens the door for the conduct of similar PISA-based studies, particularly as it relates to changes in response styles over time and what may be driving any changes. Many changes have occurred at the individual, national, and cultural levels since 2006, and these changes may have had an impact on response styles. Many countries would have undergone educational transformations as well as economic, political, and social changes, all of which would have had an effect on individuals’ interests and attitudes, and by extension, their response styles. In addition, increased international migration would have had an effect on the demographic compositions of national populations, making any survey sample from which response styles are calculated culturally diverse. All of these factors, as well as many others, may have different effects on the response styles of each PISA wave, causing their relationships with the GLOBE and Hofstede’s dimensions of culture to differ from those observed for the PISA 2006 wave. Thus, while this study highlights the presence of culture-dependent response styles in adolescent samples, additional studies for each wave of PISA, taking into account some of the possible intervening factors mentioned here and many others, would yield additional valuable insights into the relationships between response styles and culture.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Czech Science Foundation under Grant GAČR 21-09064S “Heterogeneity in reporting behavior on surveys among different countries, schools and groups of students”.

Ethical Statement

Appendix

Table A1.

Mean Estimates of Four Response Styles (ARS, DARS, ERS, and NONC) for 33 Countries in PISA 2006 (Buckley, 2009, pp. 35–37).

Country	ERS	ARS	DARS	NONC	Country	ERS	ARS	DARS	NONC
Argentina	.309	.266	.043	1.83	Ireland	.34	0.3	.041	1.519
Australia	.264	.232	.032	1.3	Israel	.211	.167	.044	1.838
Austria	.362	.292	.07	2.111	Italy	.297	.267	.03	1.909
Brazil	.286	.26	.026	1.939	Macao-China	.368	.338	.029	1.293
Canada	.331	.304	.028	1.403	Mexico	.343	.324	.02	1.855
Switzerland	.346	.303	.043	1.985	Netherlands	.215	.165	.05	1.498
Colombia	.362	.343	.019	1.779	New Zealand	.252	.219	.033	1.406
Czech Republic	.247	.215	.032	1.487	Poland	.279	.253	.026	1.488
Germany	.351	.296	.056	2.022	Portugal	.333	.321	.012	1.325
Denmark	.208	.172	.035	1.465	Russian Federation	.263	.248	.015	1.725
Spain	.344	.306	.038	1.579	Slovenia	.302	.272	.03	1.799
Finland	.261	.242	.019	1.47	Sweden	.253	.208	.044	1.489
France	.339	.278	.062	1.869	Chinese Taipei	.439	.395	.044	1.307
United Kingdom	.254	.226	.028	1.589	Thailand	.383	.375	.008	1.47
Greece	.337	.287	.049	1.761	Turkey	.407	.366	.041	1.594
Hong Kong-China	.324	.293	.031	1.396	United States of America	.278	.241	.037	1.518
Indonesia	.267	.244	.022	1.674

Note. ERS = Extreme response style; ARS = acquiescence response style; DARS = Dis-acquiescence response style; and NONC = Non-contingent response style.

Author Biographies

Tamilka Bonjeer is a PhD student at the Faculty of Education at the Charles University in Prague. His research interests are gender, reporting behavior in questionnaires, and STEM.

Hana Vonkova is an associate professor at the Faculty of Education at the Charles University in Prague. She is also a fellow researcher at CESR at the University of Southern California and at Department of Education Reform at the University of Arkansas. Her research interests are statistics, education, reporting behavior in questionnaires, and EFL learning motivation.

References

Ainley

(2019). Non-Cognitive attributes: Measurement and meaning. In Larry

E. S.

Emma

Brian

D. D.

(Eds.), The SAGE handbook of comparative studies in education (pp. 103–125). Sage Publications.

Albarracin

Shavitt

(2018). Attitudes and attitude change. Annual Review of Psychology, 69, 299–327.

Bachman

J. G.

O’Malley

P. M.

(1984). Yea-saying, nay-saying, and going to extremes: Black-white differences in response styles. Public Opinion Quarterly, 48(2), 491–509.

Baumgartner

Steenkamp

J. B. E.

(2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38(2), 143–156.

Beugelsdijk

Maseland

Van Hoorn

(2015). Are scores on Hofstede's dimensions of national culture stable over time? A cohort analysis. Global Strategy Journal, 5(3), 223–240.

Buckley

(2009). Cross-national response styles in international educational assessments: Evidence from PISA 2006. In NCES Conference on the Program for international student Assessment: What we can learn from PISA, Washington, DC, June 2, 2009.

Chen

Lee

S. Y.

Stevenson

H. W.

(1995). Response style and cross-cultural comparisons of rating scales among East Asian and North American students. Psychological Science, 6(3), 170–175.

Clarke

(2000). Extreme response style in cross-cultural research: An empirical investigation. Journal of Social Behavior & Personality, 15(1), 137.

Costello

Roodenburg

(2015). Acquiescence response bias—Yeasaying and higher education. The Educational and Developmental Psychologist, 32(2), 105–119.

10.

Da’as

R. A.

(2017). School principals’ leadership skills: Measurement equivalence across cultures. Compare: A Journal of Comparative and International Education, 47(2), 207–222.

11.

Davis

R. E.

Johnson

T. P.

Lee

Werner

(2019). Why do latino survey respondents acquiesce? Respondent and interviewer characteristics as determinants of cultural patterns of acquiescence among latino survey respondents. Cross-Cultural Research, 53(1), 87–115.

12.

De Mooij

Hofstede

(2010). The Hofstede model: Applications to global branding and advertising strategy and research. International Journal of Advertising, 29(1), 85–110.

13.

Dolnicar

Grün

(2007). Cross‐cultural differences in survey response patterns. International Marketing Review, 24, 127–143.

14.

Gilman

Huebner

E. S.

Tian

Park

O’Byrne

Schiff

Langknecht

(2008). Cross-national adolescent multidimensional life satisfaction reports: Analyses of mean scores and response style differences. Journal of Youth and Adolescence, 37, 142–154.

15.

GLOBE (n.d). GLOBE phase 2 aggregated societal level data for society culture scales: May 17, 2004. Accessed July 10, 2022, from https://globeproject.com/study_2004_2007

16.

Gouveia

V. V.

Vione

K. C.

Milfont

T. L.

Fischer

(2015). Patterns of value change during the life span: Some evidence from a functional approach to values. Personality and Social Psychology Bulletin, 41(9), 1276–1290.

17.

Graeff

T. R.

(2005). Response bias. Encyclopedia of Social Measurement, 3.

18.

Harzing

A. W.

(2006). Response styles in cross-national survey research: A 26-country study. International Journal of Cross Cultural Management, 6(2), 243–266.

19.

Bartram

Inceoglu

Van de Vijver

F. J.

(2014a). Response styles and personality traits: A multilevel analysis. Journal of Cross-Cultural Psychology, 45(7), 1028–1045.

20.

Chung

J. M.

Van De Vijver

F. J.

(2020). An examination of different scale usage correction procedures to enhance cross-cultural data comparability. Journal of Cross-Cultural Psychology, 51(10), 794–813.

21.

Espinosa

A. D. C. D.

Poortinga

Y. H.

Van de Vijver

F. J.

(2014b). Acquiescent and socially desirable response styles in cross-cultural value surveys. Toward Sustainable Development through Nurturing Diversity, 113.

22.

Hoffmann

Mai

Cristescu

(2013). Do culture-dependent response styles distort substantial relationships? International Business Review, 22(5), 814-827.

23.

Hofstede

(2006). What did GLOBE really measure? Researchers’ minds versus respondents’ minds. Journal of International Business Studies, 37, 882–896.

24.

Hofstede

(2011). Dimensionalizing cultures: The Hofstede model in context. Online Readings in Psychology and Culture, 2(1), 2307.

25.

Hofstede

G. J.

Minkov

(2010). Cultures and organizations: Software of the mind: Intercultural cooperation and its importance for survival. McGraw-Hill.

26.

Hofstede Insights (n.d). Country comparison. Accessed July 10, 2022 from https://www.hofstede-insights.com/country-comparison/

27.

House

R. J.

Hanges

P. J.

Javidan

Dorfman

P. W.

Gupta

(Eds.), (2004). Culture, leadership, and Organizations: The GLOBE study of 62 societies. Sage Publications.

28.

Huang

Wilson

Wang

(2016). Exploring plausible causes of differential item functioning in the PISA science assessment: Language, curriculum or culture. Educational Psychology, 36(2), 378–390.

29.

Inglis

(2005). Culture and everyday life. Psychology Press.

30.

Javidan

House

R. J.

Dorfman

P. W.

Hanges

P. J.

Sully de Luque

(2006). Conceptualizing and measuring cultures and their consequences: A comparative review of GLOBE’s and Hofstede’s approaches. Journal of International Business Studies, 37, 897–914.

31.

Johnson

Kulesa

Cho

Y. I.

Shavitt

(2005). The relation between culture and response styles: Evidence from 19 countries. Journal of Cross-Cultural Psychology, 36(2), 264–277.

32.

Johnson

T. P.

Shavitt

Holbrook

A. L.

(2010). Survey response styles across cultures. In Cross-cultural research methods in Psychology (pp. 130–176). Cambridge University Press.

33.

Khorramdel

von Davier

(2014). Measuring response styles across the big five: A multiscale extension of an approach using multinomial processing trees. Multivariate Behavioral Research, 49(2), 161–177.

34.

Lee

J. W.

Jones

P. S.

Mineyama

Zhang

X. E.

(2002). Cultural differences in responses to a Likert scale. Research in Nursing and Health, 25(4), 295–306.

35.

Liddell

W. W.

(2005). Project GLOBE: A large scale cross-cultural study of leadership. Problems and Perspectives in Management, 3(3), 5–9.

36.

Liu

Lee

Conrad

F. G.

(2015). Comparing extreme response styles between agree- disagree and item-specific scales. Public Opinion Quarterly, 79(4), 952–975.

37.

(2012). A multilevel multidimensional item response theory model to address the role of response style on measurement of attitudes in PISA 2006. The University of Wisconsin-Madison.

38.

Bolt

D. M.

(2015). Examining the attitude-achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style. Large-scale Assessments in Education, 3(1), 1–18.

39.

Meisenberg

Williams

(2008). Are acquiescent and extreme response styles related to low intelligence and education? Personality and Individual Differences, 44(7), 1539–1550.

40.

Minkov

(2013). Cross-cultural analysis: The science and art of comparing the world's modern societies and their cultures. Sage publications.

41.

Minkov

Blagoev

(2012). What do Project GLOBE's cultural dimensions reflect?=An empirical perspective. Asia Pacific Business Review, 18(1), 27–43.

42.

Morren

Gelissen

J. P.

Vermunt

J. K.

(2011). Dealing with extreme response style in cross-cultural research: A restricted latent class factor analysis approach. Sociological Methodology, 41(1), 13–47.

43.

Morren

Gelissen

J. P.

Vermunt

J. K.

(2012). Response strategies and response styles in cross-cultural surveys. Cross-Cultural Research, 46(3), 255–279.

44.

Moss

Vijayendra

(2018). When difference doesn’t mean different: Understanding cultural bias in global research studies. The Ipsos Views. Ipsos Knowledge Centre.

45.

OECD (2007). PISA 2006: Science competencies for tomorrow’s world: Executive Summary. Retrieved from. https://www.oecd.org/pisa/pisaproducts/39725224.pdf

46.

OECD (2009). PISA 2006 technical report. Accessed July 25, 2023, from https://www.oecd.org/pisa/data/42025182.pdf

47.

Paulhus

D. L.

(1991). Measurement and control of response bias. In Robinson

J. P.

Shaver

P. R.

Wrightsman

L. S.

(eds), Measures of Personality and Social Psychological Attitudes (pp. 17–59). Academic Press.

48.

Peterson

M. F.

(2004). Culture, leadership and organizations: The GLOBE study of 62 societies. Administrative Science Quarterly, 49 (4), 641–647.

49.

Peterson

R. A.

Rhi-Perez

Albaum

(2014). A cross-national comparison of extreme response style measures. International Journal of Market Research, 56(1), 89–110.

50.

Rammstedt

Danner

Bosnjak

(2017). Acquiescence response styles: A multilevel model explaining individual-level and country-level differences. Personality and Individual Differences, 107, 190–194.

51.

Ratchford

J. L.

Schnitker

S. A.

Reppas

(2021). The virtue of thrift: A person- centered conceptualization and measure development. Journal of Happiness Studies, 22, 385–411.

52.

Reynolds

Khorramdel

von Davier

(2022). Can students’ attitudes towards mathematics and science be compared across countries? Evidence from measurement invariance modeling in TIMSS 2019. Studies In Educational Evaluation, 74, 101169.

53.

Salazar

M. S.

(2015). The dilemma of combining positive and negative items in scales. Psicothema, 27(2), 192–199.

54.

Saris

W. E.

Revilla

Krosnick

J. A.

Shaeffer

E. M.

(2010). Comparing questions with agree/disagree response options to questions with construct-specific response options. Survey Research Methods, 4 (1), 61–79.

55.

Schneider

(2018). Extracting response style bias from measures of positive and negative affect in aging research. The Journals of Gerontology: Series B, 73(1), 64–74.

56.

Schwartz

S. H.

(1994). Beyond individualism/collectivism: New cultural dimensions of values. In Yoon

(Ed.), Individualism and collectivism: Theory, method, and applications (pp. 85–119). Sage Publications, Inc.

57.

Silber

Johnson

T. P.

(2020). Culture and response behavior: An overview of cultural mechanisms explaining survey error. Understanding Survey Methodology: Sociological Theory and Applications, 4, 67–86.

58.

Smith

P. B.

(2004). Acquiescent response bias as an aspect of cultural communication style. Journal of Cross-Cultural Psychology, 35(1), 50–61.

59.

Smith

P. B.

Fischer

(2015). Acquiescence, extreme response bias and culture: A multilevel analysis. In Multilevel analysis of individuals and cultures (pp. 285–314). Psychology Press.

60.

Soto

C. J.

John

O. P.

Gosling

S. D.

Potter

(2008). The developmental psychometrics of big five self-reports: Acquiescence, factor structure, coherence, and differentiation from ages 10 to 20. Journal of personality and social psychology, 94(4), 718.

61.

Stone

A. A.

Schneider

Junghaenel

D. U.

Broderick

J. E.

(2019). Response styles confound the age gradient of four health and well-being outcomes. Social Science Research, 78, 215–225.

62.

Taras

Steel

Kirkman

B. L.

(2016). Does country equate with culture? Beyond geography in the search for cultural boundaries. Management International Review, 56, 455–487.

63.

Tasaki

Shin

(2017). Japanese response bias: Cross-level and cross-national comparisons on response styles. Shinrigaku Kenkyu: Japanese Journal of Psychology, 88(1), 32–42.

64.

Van Dijk

T. K.

Datema

Welten

Van de Vijver

F. J.

(2009). Acquiescence and extremity in cross-national surveys: Domain dependence and country-level correlates. In Aikaterini

Mylonas

(Eds.), Quod Erat demonstrandum: From Herodotus’ ethnographic journeys to cross-cultural research: Proceedings from the 18th international Congress of the international association for cross-cultural Psychology, April 23, 2009, Spetses, Greece.

65.

Van Vaerenbergh

Thomas

T. D.

(2013). Response styles in survey research: A literature review of antecedents, consequences, and remedies. International Journal of Public Opinion Research, 25(2), 195–217.

66.

Wang

Hempton

Dugan

J. P.

Komives

S. R.

(2008). Cultural differences: Why do Asians avoid extreme responses? Survey Practice, 1(3).

67.

Watkins

Cheung

(1995). Culture, gender, and response bias: An analysis of re- sponses to the self-description questionnaire. Journal of Cross-Cultural Psychology, 26(5), 490.

68.

Weijters

Geuens

Schillewaert

(2009). Response styles and how to correct them. NIM Marketing Intelligence Review, 1(2), 44–53.

69.

Wetzel

Carstensen

C. H.

Böhnke

J. R.

(2013). Consistency of extreme response style and non-extreme response style across traits. Journal of Research in Personality, 47(2), 178–189.

70.

World Bank Data (n.d). GDP per capita (current US$). Accessed July 10, 2022, from https://data.worldbank.org/indicator/NY.GDP.PCAP.CD