Sage Journals: Discover world-class research

Abstract

Right-wing authoritarianism (RWA) is notoriously complex, multifaceted, and difficult to measure as a psychological construct. Recently, Duckitt et al. developed the ACT scales, offering theoretical refinement of the RWA construct. Although the validity of the ACT scales had been supported by a considerable body of research, shortcomings in previous analyses cannot rule out the existence of possible method effects. In the present research, we sought to test for the presence of such effects in a representative community sample of adults in Singapore (N = 738). We re-evaluated the factor structure of the ACT scales by assessing four separate models using an item-based approach in our confirmatory factor analyses. Results found significant method effects associated with both the pro-trait and con-trait items in the ACT scales. The implications of these results and possible strategies for controlling method effects in the ACT scales are discussed.

Keywords

ideological attitudes method effect authoritarianism traditionalism conservatism factor analysis

Right-wing authoritarianism (RWA), broadly defined as an individual difference in the commitment toward maintaining group cohesiveness (Duckitt, 1989), has been identified as the psychological root of anti-democratic and intolerant sentiment by many political psychologists (e.g., Napier & Jost, 2008; Sibley & Duckitt, 2008). Individuals high in RWA were more likely to endorse a restriction of citizens’ rights to protest, support government-run economy, restrict foreigners from entering one's country during the COVID pandemic (Manson, 2020), agree to the use of torture as an interrogation technique (Benjamin, 2016), disapprove women's right to abortion (Napier & Jost, 2008), and hold negative attitudes towards females, minority races, and immigrants (Sibley & Duckitt, 2008). With the political and social implications of RWA, research interest in RWA has boomed in recent years, especially with claims of a global rise in authoritarianism (Berberoglu, 2020).

However, the psychological construct of RWA is notoriously complex and difficult to measure (Vilanova et al., 2020). Examples of early measures of RWA include the California F scale (Adorno et al., 1950), the Dogmatism scale (Rokeach, 1956), the Conservatism scale (Wilson & Patterson, 1970), and the Right-Wing Authoritarianism scale (Altemeyer, 1981). Unfortunately, most of these early measures of RWA were prone to method effects—systematic sources of measurement error (for a list of method effects, see Podsakoff et al., 2003)—which compromised their validity (Christie, 1991). For example, the California F scale and the Dogmatism scale comprised only pro-trait items, where endorsement of these items may either indicate high RWA or a strong positive responding bias (Christie, 1991). That is, high scores on these scales may instead reflect participants’ tendency to choose “yes” responses regardless of the item content. While the Balanced F scale by Lee and Warr (1969) and the Conservatism scale by Wilson and Patterson (1970) were created to rebalance the California F scale and the Dogmatism scale with con-trait items, these rebalanced scales often reported poor internal consistencies (Altemeyer, 1981). The RWA scale (Altemeyer, 1981) was balanced in its wording and possessed good internal consistencies but suffered from issues such as double-barreled and triple-barreled items (Mavor et al., 2010). Without adequately controlling for method effects, these early measures failed to provide clear insight into the nature of RWA.

In their new theoretical model of RWA, Duckitt et al. (2010) suggested that individuals high in RWA are motivated to maintain group cohesion by adopting three ideological strategies: authoritarianism (using violence and punitive social treatment against “dangerous” individuals), conservatism (using restrictive policies against “disobedient” individuals), and traditionalism (adopting a disapproving stance towards “immoral” individuals). Concurrently, they developed the Authoritarianism–Conservatism–Traditionalism (ACT) scales to measure the three related but distinct dimensions proposed in their ACT model. The combined ACT scales are now amongst the most widely used measures of RWA (Heller et al., 2020), and they have been validated in several countries (e.g., New Zealand, Israel, United States of America, Romania, and Serbia; Duckitt et al., 2010; Duckitt & Bizumic, 2013). However, the ACT scales have not been widely used in Asia, and there have been no formal investigations of their psychometric properties in Asian populations. For the present study, we aimed to examine the psychometric properties of the ACT scales in a Singaporean population with a focus on identifying potential method effects.

Method effect in measures of authoritarianism

As discussed earlier, the presence of method effects in measures of RWA are well-documented. One commonly used strategy to control for method effects is to use confirmatory factor analytic (CFA) approaches, such as the correlated trait-correlated method (CTCM), to examine the potential existence of method effects and to statistically control for them in subsequent analyses (Podsakoff et al., 2003). The CTCM approach divides observed variance into three latent factors: trait, method, and error effects (see Figure 1). Method effects are then inferred from the latent factors derived from items with the same direction of wording.

Figure 1.
Correlated trait, correlated methods (CTCM) model with traits and method factors. Blue arrows indicate the items that are associated with the “authoritarianism” factor, brown arrows indicate the items that are associated with the “conservatism” factor, and green arrows indicate the items that are associated with the “traditionalism” factor. The grey arrows indicate pro-trait items, and the red arrows indicate con-trait items. The double-headed arrows indicate the covariances between latent variables.

In their scale validation study, Duckitt et al. (2010) used CFA approaches (e.g., CTCM models) to test for method effects in their ACT scales. They found the fit indices of their CTCM models were identical with a three-factor solution; a finding that was used to support their claim that method effects in the ACT scales were trivial and therefore support their proposed three-factor structure of RWA (Duckitt et al., 2010). However, Duckitt et al. (2010) adopted an item-parcelling approach (i.e., randomly grouping a pro-trait item and a con-trait item and treating them as a parcel) in their CFAs and not an item-based approach. By grouping con-trait and pro-trait items into a parcel, their parcel-based CFAs models inevitably cancel out any method effects inherent in the ACT scales by aggregating these systematic errors and indirectly inflating the fit indices of their three-factor model (DiStefano & Motl, 2006; Matsunaga, 2008). That is, item-parcelling may have artificially produced better fit indices by masking the poor model fit of the scale items (Little et al., 2002).

The use of item-parcelling also precluded any examination of method effects, making it impossible to determine if any method effects were associated with the pro-trait items, the con-trait items, or both. As mentioned by Duckitt et al. (2010), using an item-parcelling approach to CFA provides more reliable indicators and requires the estimation of fewer parameters. Despite having the advantage of being less resource-intensive, the parcel-based CFA approach is not appropriate during scale validation when the dimensionality of items is still not clear. Like many researchers had argued (e.g., DiStefano & Motl, 2006; Matsunaga, 2008), an item-based approach to CFA should be conducted to support the factor structures of the ACT scales and examined for the influence of method effects during scale validation. Specifically, an item-based approach (instead of a parcel-based approach) to CTCM models should be used to properly uncover cross-loadings and correlated residuals within and across each of the proposed dimensions of RWA, and examine for the influence of method effects.

Cultural differences

Recent cross-cultural examinations of the factor structure of the ACT scales have failed to replicate the three-factor structure (e.g., Vilanova et al., 2020), which brings into question the validity of the ACT scales in other unexamined cultures (e.g., Singapore). In addition, the construct stability of the three ACT scales were found to be higher in individualistic cultures (i.e., .82 ≤ α ≤ .88 in a New Zealand sample) than in collectivistic cultures (i.e., .70 ≤ α ≤ .73 in a Serbian sample; Duckitt & Bizumic, 2013). The past finding suggested that the three ACT scales may have tapped into the higher-order construct of RWA, but more so in individualistic cultures than in collectivistic cultures (e.g., Duckitt et al., 2010; Duckitt & Bizumic, 2013). For instance, the three ACT scales were found to be moderately and strongly inter-correlated (i.e., .33 ≤ r ≤ .59) in individualistic cultures (i.e., New Zealand) but only weakly inter-correlated (i.e., .17 ≤ r ≤ .25) in collectivistic cultures (i.e., Serbia; Duckitt & Bizumic, 2013).

Interestingly, Duckitt et al. (2010) found that being of Asian descent significantly predicted higher scores on the traditionalism subscale of their ACT scales. Although researchers may be inclined to infer that Asian cultures were associated with higher RWA, it is also possible that the ACT scales (particularly, the traditionalism subscale) were tapping into aspects of Asian cultures (e.g., paternalism, duty-based obedience to higher authority, group solidarity; Kim, 2010) instead of tapping into the higher-order construct of RWA. The construct validity of the ACT scales among unexamined Asian cultures was further called into question, given that the extent of methods effects had been found to vary across cultures (Schmitt & Allik, 2005). Together, these results provide a basis for further investigation of the dimensionality and validity of the ACT scales, particular in an Asian cultural context.

To the best of our knowledge, our study is the first to examine the psychometric properties of the ACT scales in an Asian cultural context (i.e., Singapore). Singapore is unique and appears to be the only country successful in blending cultural and religious diversity, a highly educated and well-informed populace, free and capitalist economy while retaining a strong authoritarian governance structure (Tan, 2011). In addition, the inclinations towards showing respect and obeying people of high status (e.g., leaders, authority, and government officials) are ingrained into the value systems of Singaporeans (Kim, 2010). Unlike the countries previously examined (i.e., New Zealand, Israel, United States, Romania, and Serbia), Singapore's unique social and political characteristics made it an interesting case to examine the ACT scales.

Rationale for current study

The purpose of the present study was twofold. First, we sought to assess the ACT scales more rigorously for the possible influence of method effects by using the CTCM approach in the absence of item parceling. Second, we sought to provide the first test of the dimensionality and factor structure of the ACT scales in Southeast Asia; a context in which, to the best of our knowledge, the ACT scales and their conceptualization of RWA have yet to be investigated. Using a representative sample of the national adult population in Singapore, the study also provides a further test of the cultural validity of the ACT scales.

Method

Participants

In this study, 738 Singapore residents participated, of which 717 (97.15%) were Singapore Citizens/Permanent Residents, while the remaining 21 (2.85%) were living and working in Singapore on long-term employment passes. The participants were 377 females (51.1%), 360 males (48.8%), and one participant identified as “Other.” The ethnic composition of the sample was 478 ethnic Chinese (64.8% in this sample, as compared to 74.3% in the national census), 168 ethnic Malays (22.8% in this sample, as compared to 13.5% in the national census), 49 ethnic Indians (6.6% in this sample, as compared to 9.0% in the national census), and 43 (5.8% in this sample, as compared to 3.2% in the national census) who identified with other ethnic groups (e.g., Arab, Boyanese, Eurasian, Filipino, Javanese). While the Chinese population was slightly underrepresented and the Malay population was slightly overrepresented, the sample generally adhered closely to Singapore's ethnic distribution at the national level.

Singapore is one of the most religiously diverse countries in the world, a fact that was reflected adequately in the sample composition. In terms of religious affiliation, 181 (24.5%) participants indicated Christianity as their religion, 224 (30.4%) participants indicated Buddhism, 29 (3.9%) participants indicated Taoism, 208 (28.2%) indicated Islam, 25 (3.4%) participants indicated Hinduism, 3 (0.4%) participants indicated Sikhism, and 68 (9.2%) participants indicated no religious affiliations (i.e., free thinker, atheist, agnostic). Our sample was also representative of the Singapore population in terms of education, where most of the participants were highly educated: 67 (9.1%) participants had postgraduate qualifications, 270 (36.6%) participants had bachelor's degree, 207 (28%) had post-secondary or diploma qualification, 182 (24.7%) had secondary school qualification, and 12 (1.6%) had primary school qualification. The mean age was 39.65 (SD = 12.18, range = 18–78).

Measures

The ACT scales (Duckitt et al., 2010) comprise of 36 items, with 12 items measuring each of the three RWA dimensions: authoritarianism (e.g., “Being kind to loafers or criminals will only encourage them to take advantage of your weakness, so it’s best to use a firm, tough hand when dealing with them”), conservatism (e.g., “What our country needs most is discipline, with everyone following our leaders in unity”), and traditionalism (e.g., “It is important that we preserve our traditional values and moral standards”). Two items were adapted to be less Christian-centric (e.g., “People should pay less attention to the Bible and the other old-fashioned forms of religious guidance, and instead develop their own personal standards of what is moral and immoral” was changed to “People should pay less attention to old-fashioned forms of religious guidance, and instead develop their own personal standards of what is moral and immoral,” and “God's laws about abortion, pornography, and marriage must be strictly followed before it is too late” was changed to “Religious laws about abortion, pornography, and marriage must be strictly followed before it is too late”). One item was adapted to be more cross-culturally appropriate (e.g., “There is absolutely nothing wrong with nudist camps” was changed to “There is absolutely nothing wrong with women wearing revealing clothes”). Responses are given on a 9-point agreement scale ranging from −4 (very strongly disagree) to +4 (very strongly agree).

Following the recommendations of Meade and Craig (2012), a single-item measure of data screening was also included (i.e., “Lastly, it is vital to our study that we only include responses from people that devoted their full attention to this study. Otherwise, years of effort could be wasted. You will receive credit for this study no matter what, however, in your honest opinion, should we use your data in our analyses in this study?”) at the end of the survey. Responses from participants who answered “no” to this item were excluded from the analysis.

Procedures

Following approval by the institutional research ethics committee, we engaged a commercial research panel provider (Qualtrics) to recruit participants for the online survey study. For this study, the 2020 population census in Singapore (Singapore Department of Statistics, 2020) was used to determine the sampling quotas so that the sample recruited would generally follow the gender and age proportion of the population of Singapore. Other inclusion criteria included being Singapore residents and 18 years of age or above. Ineligible participants were directed to exit the online survey upon providing a response that did not meet inclusion criteria or exceeded preestablished quotas. Data collected were also screened for quality issues (i.e., speeding, inattentiveness, inconsistent answers, duplications, and bot responding) and poor-quality data were subsequently removed. Out of the 895 responses collected, 56 responses were screened out based on the screening item, 101 responses were removed for poor quality, and 738 responses were retained for analyses.

Analysis

A total of four models were specified. One model was specified to examine the three-factor structure of the ACT scales (Duckitt et al., 2010) using an item-based approach (Model 1). Three CTCM models were further specified: two four-factor models and a five-factor model (conservatism, traditionalism, authoritarianism, and negative method factor, or positive method factor, or both method factors; Models 2a, 2b, and 2c, respectively).

The models were evaluated using absolute, comparative, and parsimonious fit indices. Absolute fit indices include the chi-square value, chi-square value/degrees of freedom ratio (χ²/df; Wheaton et al., 1977), the root mean square error of approximation (RMSEA; Steiger, 1990), and the standardized root mean square residual (SRMR). Incremental fit indices include the Comparative Fit Index (CFI; Bentler, 1990) and the Tucker–Lewis Index (TLI; Tucker & Lewis, 1973). Finally, parsimonious fix index includes the Akaike's Information Criterion (AIC; Akaike, 1987). A χ²/df < 5, a RMSEA < .06, SRMR < .08, a CFI > .95, and a TLI > .95 indicate a good fit between the hypothesized model and the data (Hu & Bentler, 1999; Marsh & Hocevar, 1985). In contrast, the AIC does not have a recommended cut-off criterion. Instead, it is compared across models, with smaller values indicative of a better fit (Hu & Bentler, 1995).

Results: Confirmatory factor analyses of the ACT scales

The data were analyzed using SPSS and AMOS Version 21. All data and reported analyses are available at https://osf.io/p7bhm/?view_only=cdcc1c5442ac4917a3180b6de38c50e5. A series of confirmatory factor analyses were conducted to examine the relative fit of the four hypothesized models to the data. The results are summarized in Table 1.

Table 1.
Fit indices for the Right-Wing Authoritarianism scale.

Model χ² df χ²/df RMSEA [90% CI] SRMR CFI TLI AIC

1 6773.240 591 11.461 .119 [.117, .122] .196 .417 .378 6923.240

2a 2061.629 573 3.598 .059 [.057, .062] .122 .860 .846 2247.629

2b 2154.485 573 3.760 .061 [.058, .064] .133 .851 .836 2340.485

2c 1599.462 554 2.887 .051 [.048, .054] .056 .901 .888 1823.462

Note: RMSEA = root mean square error of approximation; CI = confidence interval; SRMR = standardized root mean square residual; CFI = comparative fit index; TLI = Tucker–Lewis Index; AIC = Akaike's information criterion; Model 1 = three-factor model; Model 2a = four-factor model (conservatism, traditionalism, authoritarianism, and negative method factor); Model 2b = four-factor model (conservatism, traditionalism, authoritarianism, and positive method factor); Model 2c = five-factor model (conservatism, traditionalism, authoritarianism, both method factors).

As shown in Table 1, our CFAs findings using an item-based approach were inconsistent with the findings reported by Duckitt et al. (2010). Specifically, the three-factor model (Model 1) poorly fit the data. In addition, the results showed that the CTCM models (i.e., Model 2a, 2b, and 2c) had better fit indices.

The superiority of Model 2c among CTCM models suggested that the method effect was associated with both pro-trait and con-trait items. The standardized factor loadings for Model 2c (see Table 2) further revealed that the factor loadings between the method factors (PMF and NMF) and the items were in general larger than those between the content factors (C, T, and A) and the items, which suggested that a large portion of variances for the items were due to the method effects and less so for the three intended constructs of interest (i.e., C, T, and A).

Table 2.
Standardized factor loadings for model 2c.

Items Model 2c

A C T NMF PMF

(A1) Strong, tough government will harm, not help our country (R) .33 .48

(A2) Being kind to loafers or criminals will only encourage them to take advantage of your weakness, so it's best to use a firm, tough hand when dealing with them .15 ns .49

(A3) Our society does NOT need tougher government and stricter laws (R) .71 .41

(A4) The facts on crime and the recent public disorders show we have to crack down harder on troublemakers if we are going preserve law and order .18 .61

(A5) Our prisons are a shocking disgrace. Criminals are unfortunate people who deserve much better care, instead of so much punishment (R) .25 .60

(A6) The way things are going in this country, it's going to take a lot of “strong medicine” to straighten out the troublemakers, criminals, and perverts .19 .55

(A7) We should smash all the negative elements that are causing trouble in our society .02 ns .60

(A8) The situation in our country is getting so serious, the strongest methods would be justified if they eliminated the troublemakers and got us back to our true path .11 ns .66

(A9) People who say our laws should be enforced more strictly and harshly are wrong. We need greater tolerance and more lenient treatment for lawbreakers (R) .18 .61

(A10) The courts are right in being easy on drug offenders. Punishment would not do any good in cases like these (R) .11 ns .63

(A11) What our country really needs is a tough, harsh dose of law and order .36 .57

(A12) Capital punishment is barbaric and never justified (R) .18 .57

(C1) It's great that many young people today are prepared to defy authority (R) .25 .69

(C2) What our country needs most is discipline, with everyone following our leaders in unity .52 .58

(C3) Students at high schools and at university must be encouraged to challenge, criticize, and confront established authorities (R) .29 .55

(C4) Obedience and respect for authority are the most important virtues children should learn .50 .54

(C5) Our country will be great if we show respect for authority and obey our leaders .56 .53

(C6) People should be ready to protest against and challenge laws they don’t agree with (R) .36 .52

(C7) People should be allowed to make speeches and write books urging the overthrow of the government (R) .24 .69

(C8) The more people there are that are prepared to criticize the authorities, challenge and protest against the government, the better it is for society (R) .34 .61

(C9) People should stop teaching children to obey authority (R) .34 .61

(C10) The real keys to the “good life” are respect for authority and obedience to those who are in charge .47 .55

(C11) The authorities should be obeyed because they are in the best position to know what is good for our country .56 .57

(C12) Our leaders should be obeyed without question .30 .55

(T1) Nobody should stick to the “straight and narrow.” Instead people should break loose and try out lots of different ideas and experiences (R) .16 .36

(T2) The “old-fashioned ways” and “old-fashioned values” still show the best way to live .35 .53

(T3) Religious laws about abortion, pornography, and marriage must be strictly followed before it is too late .55 .52

(T4) There is absolutely nothing wrong with women wearing revealing clothes (R) .62 .30

(T5) This country will flourish if young people stop experimenting with drugs, alcohol, and sex, and pay more attention to family values .32 .45

(T6) There is nothing wrong with premarital sexual intercourse (R) .69 .23

(T7) Traditional values, customs, and morality have a lot wrong with them (R) .26 .60

(T8) Everyone should have their own lifestyle, religious beliefs, and sexual preferences, even if it makes them different from everyone else (R) .36 .21

(T9) The radical and sinful new ways of living and behaving of many young people may one day destroy our society .40 .55

(T10) Trashy magazines and radical literature in our communities are poisoning the minds of our young people .26 .53

(T11) It is important that we preserve our traditional values and moral standards .39 49

(T12) People should pay less attention to old-fashioned forms of religious guidance, and instead develop their own personal standards of what is moral and immoral (R) .40 .47

Note: Model 2c = five-factor model (conservatism, traditionalism, authoritarianism, both method factors); A = Authoritarianism; C = Conservatism; T = Traditionalism; NMF = Negative Method Factor; PMF = Positive Method Factor; ns = nonsignificant; all factor loadings without ns are statistically significant at p < .001.

Discussion

In this study, we sought to (a) more rigorously assess the ACT scales for the possible influence of method effects using the CTCM approach in the absence of item parcelling, and (b) examine the factor structure of the ACT scales in Southeast Asia. Our results found significant method effects in the ACT scales, unlike the previous finding reported by Duckitt et al. (2010). Particularly, the three-factor model that did not consider method effects (i.e., Model 1) produced fit indices that failed to meet the required cut-offs (i.e., χ²/df < 5, a RMSEA < .06, and a CFI > .95; Hu & Bentler, 1999; Marsh & Hocevar, 1985). Instead, the CTCM model that included both method effects (i.e., Model 2c) best fitted the data (Table 1). In addition, the method effects accounted for higher factor loadings among the items than the three content constructs of RWA, suggesting that the participants in Singapore were more likely to respond to the direction of the wording than to the content of the ACT scales (Table 2).

Our results also identified several problematic items in the ACT scales that were weakly associated with the content factors but strongly associated with the two method factors (Table 2), suggesting that these items may need to be removed or reworded to better capture the proposed ACT dimensions. For instance, we found several con-trait items (i.e., A5, A9, A10, A12, C1, C3, C7, T1, T7, and T10) had a standardized factor loadings of less than .30 on their respective content factors, but more than .50 loadings on the negative method factor (Table 2). Similarly, the results found several pro-trait items (i.e., A2, A4, A6, A7, A8, and T10) with a loading of less than .30 on their respective content factors but more than .50 loadings on the positive method factor (Table 2). Furthermore, most of the problematic items were from the authoritarianism subscale. Future research may consider improving the ACT scales by developing better-quality items for each of the three dimensions, particularly the authoritarianism subscale.

Whenever a psychometric instrument is first used with a previously unexamined population, it is essential that researchers first examine whether the data collected correspond with the factorial structure set out in theory (Duckitt & Bizumic, 2016). If the data collected do not produce the desired factorial structure, the logical conclusion will be that the theoretical constructs under investigation are not empirically present within the examined population and that the measurement is invalid. Given that Model 1 failed to produce acceptable fit indices (Table 1), we conclude that the RWA construct, conceptualized and operationalized in the ACT scales, cannot be validly measured among the Singapore adult population. Our study consisted of only a Singapore sample, so our results do not necessarily suggest universal method effects in the ACT scales. However, we recommend that researchers planning to use the ACT scales for hypothesis testing should be cautious of method effects and preferably control for these effects using item-based CTCM approach.

Implications for validity in different populations

To the best of our knowledge, our study is the first to examine the construct validity of the ACT scales in Asia. Specifically, we examined the cross-cultural validity of the three-factor structure of ACT scale in Singapore, a country characterized by a Confucian culture (Leong et al., 2014) and so-called Asian values (Chia et al., 2021), that are also found elsewhere in the Southeast Asian region. Unlike the countries previously examined (e.g., New Zealand, Israel, United States, Romania, Serbia), Singapore's social and political characteristics also made it an interesting context in which to study authoritarianism; one that is both relatively new and somewhat representative of an important broader region.

As a social attitudinal expression for collective security (Duckitt et al., 2010), RWA may manifest differently across cultures and display different factor structures (e.g., Chylikova & Buchtik, 2016; Vilanova et al., 2020). Although our study is the first to examine method effects in the ACT scales using an item-based CTCM approach, our study is not the first to uncover method effects in the ACT scales. Specifically, Vilanova et al. (2020) found a methodological split in one of the ACT scales when used with the Brazilian population. Their analyses found the pro-trait and con-trait items of the conservatism subscale clustered to form two separate factors: submission to authority and contestation to authority, respectively. Vilanova et al. (2020) argued that participants in Brazil perceived different ideological significance in the pro-trait and con-trait items of conservatism. Similarly, we suggest that Singapore participants may have perceived different ideological significance in the pro-trait and con-trait items across all three ACT subscales.

Looking at the content of the pro- and con-trait items across all three ACT subscales, we observed that the pro-trait items are generally expressive of support for paternalism (i.e., items A2/A6/A11/C2/C4/C5/C10/C11/C12/T15/T17/T21/T22), where the items contain phrases like “needs more discipline,” “children should obey and respect authority,” “young people should behave,” “leaders should be obeyed without question because they know what is good for us,” “the country will flourish if young people pay more attention to values and obeyed authorities,” and “tough and harsh punishment are needed to bring our country back to our true path”). On the other hand, the con-trait items expressed a rebellion against conformity (i.e., items A1/A3/C1/C3/C6/C7/C8/T13/T20/T24) where the items contain verbs such as “defy,” “challenge,” “overthrow,” “break loose,” and “setting personal standards”). Our observation suggested that the methodological split between pro-trait and con-trait items may also reflect content variance associated with support of paternalism (pro-trait) and rebellion against paternalism (con-trait) in the ACT scales.

While the construct of RWA was founded on Western psychology and sociology (Chien, 2016), recent progress in Chinese indigenous psychology has identified an “authoritarian orientation” construct that is rooted in the Confucian cultural context (Yang, 1993). The indigenous “authoritarian orientation” construct, which represented the sensitivity towards and dependence on authority for gaining access to social rewards (Cheng Wong & Tsang, 2007), consisted of four components; authority sensitization, authority worship, authority dependence, and authority dread (Chien, 2016). Our findings suggest that the pro-trait and con-trait items of the ACT scales might have tapped into the two components of “authoritarian orientation”: authority dependence and authority dread, respectively. As our study is the first to examine the factor structure of the ACT scales in Asia, further investigation of the ACT scales in Asian contexts are needed to validate our findings.

It is important to point out that the validation studies conducted by Duckitt et al. (2010) and Duckitt and Bizumic (2013) adopted convenience sampling and that most of their participants were undergraduate students. On the other hand, our sample was recruited using quota sampling, and participants were mostly working adults with a mean age of forty. Therefore, method effects might be exacerbated in an age-diverse, more representative sample. Consistent with this idea, Lindwall et al. (2012) found that while younger participants had a higher estimated latent factor score on the con-trait method factor, older participants had a higher estimated latent factor score for the pro-trait method factor. It is, therefore, possible that method factors associated with both pro- and con-trait items were uncovered in our data partly because of the use of an age-diverse sample, which may have been missed in Duckitt and Bizumic's (2010) study, given the recruitment of younger participants.

Strategies for minimizing the impact of method effects

Theoretically, items that are similar in content should load on the same content factor regardless of the direction of wording. Similarly, items that are dissimilar in content should not load on the same latent factor despite similar direction of wording. The finding of two method factors provided clear evidence for a direction-of-wording effect in the ACT scales, where items were artifactually connected due to the direction of wording and not because of the item content. Other than causing spurious factor structure of RWA, the method effects in the ACT scales also introduced systematic measurement error that may cause erroneous interpretation of data (e.g., a significant correlation with a criterion variable may be due to the association with the method effects and not the construct under study; DiStefano & Motl, 2006; Van Sonderen et al., 2013). Therefore, it is essential that researchers adopt strategies to screen and subsequently minimize, if not remove, the impact of method effects in the ACT scales.

As mentioned in earlier sections, one strategy to control for method effects is to use CFA approaches such as CTCM models. Researchers may consider using statistical analyses similar to ours, and explicitly include positive and negative method factors to account for them in their analyses (i.e., CTCM) when specifying the relationships between the three ACT factors (i.e., authoritarianism, conservatism, and traditionalism) and other variables of interest. The downside to this method is that it requires Structural Equation Modelling (SEM), a sample-intensive methodology.

Besides using statistical methods to control for method effects, other strategies to handle direction-of-wording effects have been put forth by several researchers (e.g., Lindwall et al., 2012; Marsh et al., 2010; Podsakoff et al., 2003; Quilty et al., 2006; Van Sonderen et al., 2013). One strategy involves using only positive-worded items (Chyung et al., 2018; Lindwall et al., 2012; Van Sonderen et al., 2013). This strategy is controversial, since many researchers suggest that a balanced mixture of pro- and con-trait items may help minimize responding bias (e.g., acquiescence bias; DeVellis, 1991; Hinkin, 1995). This view is not, however, universal. Several researchers had found that reverse wording of questionnaire items does not always prevent response bias (Rammstedt & Farmer, 2013) and, in some cases, may even increase risk of response fatigue and inattentive responding (Schriesheim et al., 1991; Van Sonderen et al., 2013). Although removing con-trait items may help in controlling for the negative method effect, this strategy does not help in controlling for the positive method effect.

Alternatively, researchers may consider administering the ACT scales such that all pro-trait items are presented first before the presentation of all con-trait items. Separate CFAs can then be conducted on the positively and negatively worded items where the factor solutions can be directly compared to determine the relative impact of method effects on the factor structure of the ACT scales (Chyung et al., 2018; Lindwall et al., 2012). Researchers may also consider using the novel approach adopted by Mavor et al. (2010) where responses from another measure (e.g., the social dominance orientation scale) were used to estimate for acquiescence bias. The estimate for acquiescence bias was then used to derive a bias-corrected score for each of the ACT items for subsequent analyses. For a more detailed discussion, interested readers may want to consider the seminal work of Podsakoff et al. (2003).

Limitations and future directions

The key limitation is that only Singapore residents were recruited in this study. Although Singapore is to some extent representative of countries in Southeast Asia, particularly in terms of the extent to which individuals resist social change (i.e., adopt a more conservative sociopolitical orientation; Lee et al., 2018) and accept that power is distributed unequally (i.e., power distance; Hofstede Insights, 2022), there are still some obvious differences (e.g., per capita income, levels of education, religiosity; Lee et al., 2018). As such, our findings may generalize to some nations and cultures in the Southeast Asian region better than others. Further validation studies are required to better understand the extent of method effects in the ACT scales among other nations in Southeast Asia.

Culture can either be operationalized on a societal level (e.g., Hofstede Insights, 2022; House & Javidan, 2004) or on an individual level (Taras et al., 2010). It is not uncommon for the country of data collection to be treated as a culture (Matsumoto & Yoo, 2006). However, the lack of individual-level scales for cultural dimensions in this study limited the ability to make a claim on the link between method effects and cultural dimensions (e.g., power distance). Future studies may consider including individual measures of culture-related constructs such as the power scale of the Schwartz Value survey and Portrait Value Survey (Schwartz & Sagiv, 1995).

Our results do not suggest universal method effects in the ACT scales. However, we suggest that researchers using the ACT scales for hypothesis testing should practice with caution, as previous validation of the ACT scales had used the item-parcelling approach in their analyses, which may have masked the poor model fit of the scale items. Particularly, we recommend for researchers to assess for method effects and validate the factorial structure of the ACT scales. Researchers who are intending to use the ACT scales for hypothesis testing should consider using statistical analyses, similar to ours, and explicitly include pro-trait and con-trait method factors to account for them in their analyses (i.e., item-based CTCM models).

Several problematic items in the ACT scales were uncovered in our study. Future studies should consider building on the work of Duckitt et al. (2010) by developing more items for the authoritarianism subscale, identifying and removing items with poor loading via exploratory factor analysis, and used item-based CFA approach to validate their three-factor structure of RWA. As our result had suggested, the content variance of the ACT scales may be related to components of the indigenous “authoritarian orientation” construct. As such, researchers interested in the expression of RWA in the Asian contexts might also consider developing new measures of RWA that integrate these fundamental cultural characteristics of Asian culture.

Conclusions

Our study is the first to rigorously assess the ACT scales for the influence of method effects using the CTCM approach in the absence of item parceling and is the first study to examine the factor structure of the ACT scales in Southeast Asia. Our results did not support the three-factor structure of the ACT scales but instead revealed significant positive and negative method effects. As our findings were based on a Singapore sample, they do not suggest universal method effects in the ACT scales. Instead, our findings encourage more research into the construct validity of the ACT scales, especially when used in the Asian contexts.

Model	χ²	df	χ²/df	RMSEA [90% CI]	SRMR	CFI	TLI	AIC
1	6773.240	591	11.461	.119 [.117, .122]	.196	.417	.378	6923.240
2a	2061.629	573	3.598	.059 [.057, .062]	.122	.860	.846	2247.629
2b	2154.485	573	3.760	.061 [.058, .064]	.133	.851	.836	2340.485
2c	1599.462	554	2.887	.051 [.048, .054]	.056	.901	.888	1823.462

Items	Model 2c
(A1) Strong, tough government will harm, not help our country (R)	.33			.48
(A2) Being kind to loafers or criminals will only encourage them to take advantage of your weakness, so it's best to use a firm, tough hand when dealing with them	.15 ns				.49
(A3) Our society does NOT need tougher government and stricter laws (R)	.71			.41
(A4) The facts on crime and the recent public disorders show we have to crack down harder on troublemakers if we are going preserve law and order	.18				.61
(A5) Our prisons are a shocking disgrace. Criminals are unfortunate people who deserve much better care, instead of so much punishment (R)	.25			.60
(A6) The way things are going in this country, it's going to take a lot of “strong medicine” to straighten out the troublemakers, criminals, and perverts	.19				.55
(A7) We should smash all the negative elements that are causing trouble in our society	.02 ns				.60
(A8) The situation in our country is getting so serious, the strongest methods would be justified if they eliminated the troublemakers and got us back to our true path	.11 ns				.66
(A9) People who say our laws should be enforced more strictly and harshly are wrong. We need greater tolerance and more lenient treatment for lawbreakers (R)	.18			.61
(A10) The courts are right in being easy on drug offenders. Punishment would not do any good in cases like these (R)	.11 ns			.63
(A11) What our country really needs is a tough, harsh dose of law and order	.36				.57
(A12) Capital punishment is barbaric and never justified (R)	.18			.57
(C1) It's great that many young people today are prepared to defy authority (R)		.25		.69
(C2) What our country needs most is discipline, with everyone following our leaders in unity		.52			.58
(C3) Students at high schools and at university must be encouraged to challenge, criticize, and confront established authorities (R)		.29		.55
(C4) Obedience and respect for authority are the most important virtues children should learn		.50			.54
(C5) Our country will be great if we show respect for authority and obey our leaders		.56			.53
(C6) People should be ready to protest against and challenge laws they don’t agree with (R)		.36		.52
(C7) People should be allowed to make speeches and write books urging the overthrow of the government (R)		.24		.69
(C8) The more people there are that are prepared to criticize the authorities, challenge and protest against the government, the better it is for society (R)		.34		.61
(C9) People should stop teaching children to obey authority (R)		.34		.61
(C10) The real keys to the “good life” are respect for authority and obedience to those who are in charge		.47			.55
(C11) The authorities should be obeyed because they are in the best position to know what is good for our country		.56			.57
(C12) Our leaders should be obeyed without question		.30			.55
(T1) Nobody should stick to the “straight and narrow.” Instead people should break loose and try out lots of different ideas and experiences (R)			.16	.36
(T2) The “old-fashioned ways” and “old-fashioned values” still show the best way to live			.35		.53
(T3) Religious laws about abortion, pornography, and marriage must be strictly followed before it is too late			.55		.52
(T4) There is absolutely nothing wrong with women wearing revealing clothes (R)			.62	.30
(T5) This country will flourish if young people stop experimenting with drugs, alcohol, and sex, and pay more attention to family values			.32		.45
(T6) There is nothing wrong with premarital sexual intercourse (R)			.69	.23
(T7) Traditional values, customs, and morality have a lot wrong with them (R)			.26	.60
(T8) Everyone should have their own lifestyle, religious beliefs, and sexual preferences, even if it makes them different from everyone else (R)			.36	.21
(T9) The radical and sinful new ways of living and behaving of many young people may one day destroy our society			.40		.55
(T10) Trashy magazines and radical literature in our communities are poisoning the minds of our young people			.26		.53
(T11) It is important that we preserve our traditional values and moral standards			.39		49
(T12) People should pay less attention to old-fashioned forms of religious guidance, and instead develop their own personal standards of what is moral and immoral (R)			.40	.47

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: this work was supported by the James Cook University Singapore Internal Research Fund (grant number IRG20210019).

ORCID iDs

Da Xuan Ng

Jonathan Edward Ramsay

Peter K. H. Chew

Transparency and openness

This study was not preregistered. The raw data and AMOS diagram files are available at the online repository ().

References

Adorno

T. W.

Frenkel-Brunswik

Levinson

D. J.

Sanford

R. N.

(1950). The authoritarian personality. Harper and Row.

Akaike

(1987). Factor analysis and AIC. Psychometrika, 52(3), 317–332. https://doi.org/10.1007/BF02294359

Altemeyer

(1981). Right-wing authoritarianism. University of Manitoba Press.

Benjamin

A. J.

(2016). Right-wing authoritarianism and attitudes toward torture. Social Behavior and Personality, 44(6), 881–887. https://doi.org/10.2224/sbp.2016.44.6.881

Bentler

P. M.

(1990). Comparative fit indices in structural models. Psychological Bulletin, 107(2), 238–246. https://doi.org/10.1037/0033-2909.107.2.238

Berberoglu

(2020). The global rise of authoritarianism in the 21st century: Crisis of neoliberal globalisation and the nationalist response. Routledge.

Cheng Wong

R. L. C.

Tsang

S. K. M.

(2007). Evaluating a measure of “social orientation” of Chinese students in social work: Developing its reliability and validity. Social Work Education, 26(5), 458–480. https://doi.org/10.1080/02615470601118613

Chia

Y. T.

Chew

Tan

(2021). Teacher preparation in Singapore: A concise critical history. Emerald Group Publishing.

Chien

C.-L.

(2016). Beyond authoritarian personality: The culture-inclusive theory of Chinese authoritarian orientation. Frontiers in Psychology, 7, https://doi.org/10.3389/fpsyg.2016.00924

10.

Christie

(1991). Authoritarianism and related constructs. In Robinson

J. P.

Shaver

P. R.

(Eds.), Measures of personality and social psychological attitudes (pp. 501–571). Academic Press. https://doi.org/10.1016/B978-0-12-590241-0.50014-9

11.

Chylikova

Buchtik

(2016). Validity of the construct of Right-wing authoritarianism and its measure in post-socialistic region: A case of the Czech Republic. Journal of Social Research & Policy, 7(1), 5–25. Retrieved from https://www.proquest.com/scholarly-journals/validity-construct-right-wing-authoritarianism/docview/1916641697/se-2

12.

Chyung

S. Y.

Barkin

J. R.

Shamsy

J. A.

(2018). Evidence-based survey design: The use of negatively worded items in surveys. Performance Improvement, 57(3), 16–25. https://doi.org/10.1002/pfi.21749

13.

DeVellis

R. F.

(1991). Scale development: Theory and applications. Sage Publications.

14.

DiStefano

Motl

R. W.

(2006). Further investigating method effects associated with negatively worded items on self-report surveys. Structural Equation Modeling, 13(3), 440–464. https://doi.org/10.1207/s15328007sem1303_6

15.

Duckitt

(1989). Authoritarianism and group identification: A new view of an old construct. Political Psychology, 10(1), 63–84. https://doi.org/10.2307/3791588

16.

Duckitt

Bizumic

(2013). Multidimensionality of right-wing authoritarian attitudes: Authoritarianism–conservatism–traditionalism. Political Psychology, 34(6), 841–862. https://doi.org/10.1111/pops.12022

17.

Duckitt

Bizumic

Krauss

S. W.

Heled

(2010). A tripartite approach to right-wing authoritarianism: The authoritarianism–conservatism–traditionalism model. Political Psychology, 31(5), 685–715. https://doi.org/10.1111/j.1467-9221.2010.00781.x

18.

Heller

Decker

Schmalbach

Beutel

Fegert

J. M.

Brahler

Zenger

(2020). Detecting authoritarianism efficiently: Psychometric properties of the screening instrument Authoritarianism-Ultra Short (A-US) in a German representative sample. Frontiers in Psychology, 11, 533863. https://doi.org/10.3389/fpsyg.2020.533863

19.

Hinkin

T. R.

(1995). A review of scale development practices in the study of organizations. Journal of Management, 21, 967–988. https://doi.org/10.1177/014920639502100509

20.

Hofstede Insights. (2022). Country comparison—Hofstede Insights . Retrieved April 17, 2022, from https://www.hofstede-insights.com/country-comparison/indonesia,malaysia,singapore/

21.

House

R. J.

Javidan

(2004). Overview of GLOBE. In House

R. J.

Hanges

P. J.

Javidan

Dorfman

P. W.

Gupta

(Eds.), Culture, leadership, and organisations: The GLOBE study of 62 societies (pp. 9–28). Sage Publications.

22.

Bentler

P. M.

(1995). Evaluating model fit. In Hoyle

R. H.

(Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 76–99). Sage Publications.

23.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

24.

Kim

S. Y.

(2010). Do Asian values exist? Empirical tests of the four dimensions of Asian values. Journal of East Asian Studies, 10(2), 315–344. https://doi.org/10.1017/S1598240800003477

25.

Lee

Ashton

M. C.

Griep

Edmonds

(2018). Personality, religion, and politics: An investigation in 33 countries. European Journal of Personality, 32(2), 100–115. https://doi.org/10.1002/per.2142

26.

Lee

R. E.

III Warr

P. B.

(1969). The development and standardisation of a Balanced F-scale. The Journal of General Psychology, 81(1), 109–129. https://doi.org/10.1080/00221309.1969.9710776

27.

Leong

F. T. L.

Huang

J. L.

Mak

(2014). Protestant work ethic, Confucian values, and work-related attitudes in Singapore. Journal of Career Assessment, 22(2), 304–316. https://doi.org/10.1177/1069072713493985

28.

Lindwall

Barkoukis

Grano

Lucidi

Raudsepp

Liukkonen

Thøgersen-Ntoumani

(2012). Method effects: The problem with negatively versus positively keyed items. Journal of Personality Assessment, 94(2), 196–204. https://doi.org/10.1080/00223891.2011.645936

29.

Little

T. D.

Cunningham

W. A.

Shahar

Widaman

K. F.

(2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling, 9(2), 151–173. https://doi.org/10.1207/S15328007SEM0902_1

30.

Manson

J. H.

(2020). Right-wing authoritarianism, left-wing authoritarianism, and pandemic-mitigation authoritarianism. Personality and Individual Differences, 167, 110251. https://doi.org/10.1016/j.paid.2020.110251

31.

Marsh

H. W.

Hocevar

(1985). Application of confirmatory factor analysis to the study of self-concept: First- and higher order factor models and their invariance across groups. Psychological Bulletin, 97(3), 562–582. https://doi.org/10.1037/0033-2909.97.3.562

32.

Marsh

H. W.

Scalas

L. F.

Nagengast

(2010). Longitudinal tests of competing factor structures for the Rosenberg self-esteem scale: Traits, ephemeral artifacts, and stable response styles. Psychological Assessment, 22(2), 366–381. https://doi.org/10.1037/a0019225

33.

Matsumoto

Yoo

S. H.

(2006). Toward a new generation of cross-cultural research. Perspectives on Psychological Science, 1(3), 234–250. https://doi.org/10.1111/j.1745-6916.2006.00014.x

34.

Matsunaga

(2008). Item parceling in structural equation modelling: A primer. Communication Methods and Measures, 2(4), 260–293. https://doi.org/10.1080/19312450802458935

35.

Mavor

K. I.

Louis

W. R.

Sibley

C. G.

(2010). A bias-corrected exploratory and confirmatory factor analysis of right-wing authoritarianism: Support for a three-factor structure. Personality and Individual Differences, 48, 28–33. https://doi.org/10.1016/j.paid.2009.08.006

36.

Meade

A. W.

Craig

S. B.

(2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455. https://doi.org/10.1037/a0028085

37.

Napier

J. L.

Jost

J. T.

(2008). The anti-democratic personality revisited: A cross-national investigation of working-class authoritarianism. Journal of Social Issues, 64(3), 595–617. https://doi.org/10.1111/j.1540-4560.2008.00579.x

38.

Podsakoff

P. M.

MacKenzie

S. B.

Lee

J.-Y.

Podsakoff

N. P.

(2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879

39.

Quilty

L. C.

Oakman

J. M.

Risko

(2006). Correlates of the Rosenberg self-esteem scale method effects. Structural Equation Modeling, 13(1), 99–117. https://doi.org/10.1207/s15328007sem1301_5

40.

Rammstedt

Farmer

R. F.

(2013). The impact of acquiescence on the evaluation of personality structure. Psychological Assessment, 25(4), 1137–1145. https://doi.org/10.1037/a0033323

41.

Rokeach

(1956). Political and religious dogmatism: An alternative to the authoritarian personality. Psychological Monographs, 70(8), 1–43. https://doi.org/10.1037/h0093727

42.

Schmitt

D. P.

Allik

(2005). Simultaneous administration of the Rosenberg self-esteem scale in 53 nations: Exploring the universal and culture-specific features of global self-esteem. Journal of Personality and Social Psychology, 89(4), 623–642. https://doi.org/10.1037/0022-3514.89.4.623

43.

Schriesheim

C. A.

Eisenbach

R. J.

Hill

K. D.

(1991). The effect of negation and polar opposite item reversals on questionnaire reliability and validity: An experimental investigation. Educational and Psychological Measurement, 51(1), 67–78. https://doi.org/10.1177/0013164491511005

44.

Schwartz

S. H.

Sagiv

(1995). Identifying culture specifics in the content and structure of values. Journal of Cross-Cultural Psychology, 26, 92–116. https://doi.org/10.1177/0022022195261007

45.

Sibley

C. G.

Duckitt

(2008). Personality and prejudice: A meta-analysis and theoretical review. Personality and Social Psychology Review, 12(3), 248–279. https://doi.org/10.1177/1088868308319226

46.

Singapore Department of Statistics (2020). Census of population 2020: Demographics characteristics, education, language and religion . Retrieved October 31, 2021, from https://www.singstat.gov.sg/-/media/files/publications/cop2020/sr1/cop2020sr1.pdf

47.

Steiger

J. H.

(1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25(2), 173–180. https://doi.org/10.1207/s15327906mbr2502_4

48.

Tan

K. P.

(2011). The ideology of pragmatism: Neo-liberal globalisation and political authoritarianism in Singapore. Journal of Contemporary Asia, 42(1), 67–92. https://doi.org/10.1080/00472336.2012.634644

49.

Taras

Kirkman

B. L.

Steel

(2010). Examining the impact of Culture’s consequences: A three-decade, multilevel, meta-analytic review of Hofstede’s cultural value dimensions. Journal of Applied Psychology, 95(3), 405–439. https://doi.org/10.1037/a0018938

50.

Tucker

Lewis

(1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1–10. https://doi.org/10.1007/BF02291170

51.

Van Sonderen

Sanderman

Coyne

J. C.

(2013). Ineffectiveness of reverse wording of questionnaire items: Let’s learn from cows in the rain. PLoS ONE, 8(7), e68967. https://doi.org/10.1371/journal.pone.0068967

52.

Vilanova

Milfont

T. L.

Cantal

Koller

S. H.

Costa

A. B.

(2020). Evidence for cultural variability in right-wing authoritarianism factor structure in a politically unstable context. Social Psychological and Personality Science, 11(5), 658–666. https://doi.org/10.1177/1948550619882038

53.

Wheaton

Muthen

Alwin

D. F.

Summers

G. F.

(1977). Assessing reliability and stability in panel models. Sociological Methodology, 8(1), 84–136. https://doi.org/10.2307/270754

54.

Wilson

G. D.

Patterson

J. R.

(1970). Manual for the Conservatism Scale. N.F.E.R.

55.

Yang

K.-S.

(1993). Chinese Social orientation: A social interactional approach. In Yang

K.-S.

A.-B.

(Eds.), Chinese psychology and behavior: Methodological and conceptual considerations (pp. 87–142). Guiguan. (in Chinese).

Evidence of method effects in the authoritarianism-conservatism-traditionalism scales

Abstract

Keywords

Method effect in measures of authoritarianism

Cultural differences

Rationale for current study

Method

Participants

Measures

Procedures

Analysis

Results: Confirmatory factor analyses of the ACT scales

Discussion

Implications for validity in different populations

Strategies for minimizing the impact of method effects

Limitations and future directions

Conclusions

Footnotes

Declaration of conflicting interests

Funding

ORCID iDs

Transparency and openness

References