Revisiting the Validity of the Implicit Beliefs Measure: Focusing on Dimensionality and Discriminability of the CNAAQ-2 (Conceptions of the Nature of Athletic Ability Questionnaire-2) in the Context of South Korean Adolescent Football Players

Abstract

The aim of the current study was to verify the internal and external validity of the construct of implicit beliefs concept in the sports context, the Conceptions of the Nature of Athletic Ability Questionnaire-2 (CNAAQ-2). The data of this study were collected from 322 adolescent football players by using CNAAQ-2 and MT (Mental Toughness) Questionnaires. The CFA was employed to examine internal validity (dimensionality), and the ANOVA with the K-means cluster analysis was conducted to examine external validity (discriminability). The results of the CFA supported convergent and discriminant validity of the specific factors (incremental / entity belief) but did not support the significance of the general factor (implicit beliefs). The results of clustering showed that implicit beliefs propensity is divided into three clusters. ANOVA, which set these clusters as an independent variable, showed significant differences for the MT. Based on these results, considerations in the utilization of CNAAQ-2 were discussed.

Plain language summary

The purpose of this study was to verify the internal and external validity of CNAAQ-2, which measures implicit athletic beliefs in the context of sport and exercise. Multi-dimensional characteristics of this psychological construct with dichotomous concepts were reviewed as a psychometric flaw. The structure of the CNAAQ-2 was analyzed through a CFA with bifactor model, and the external validity of that was examined via an ANOVA with clustering. We believe that our study makes a significant contribution to the literature because the findings can be generalized to the integrated procedure of analysis of internal and external validity of psychological construct measures.

Keywords

incremental belief entity belief mental toughness psychometrics validation

Introduction

People vary in their implicit (lay) beliefs (also called mindset; Dweck, 2006) on the malleability of human dispositional attribution such as intelligence, morality, and ability of a particular area (Dweck, 1996, 2000; Dweck et al., 1993; Dweck & Leggett, 1988). The perspective of the Social Cognitive Model of Motivation (SCMM; Dweck & Leggett, 1988, 2000; Molden & Dweck, 2006) proposes that individuals differ in their implicit beliefs regarding the nature of their own ability, and these differences influence the process of human motivation by providing a schema that determines an individual’s interpretation and response to achievement. Specifically, implicit beliefs are divided into the categories of incremental belief, which holds that ability is changeable, and entity belief, which asserts that ability is innate and cannot be changed. In the context of achievement, a person with an incremental belief maintains continuous effort characterized by challenge while anticipating improvement and control over their ability to adequately manage experiences of adversity and failure. Conversely, a person with an entity belief experiences emotions of shame and helplessness in the face of experiences of adversity and failure due to their assumption that innate ability is evaluated in the context of adversarial experiences. As such, implicit beliefs may be important in sports and physical activity contexts where failures and setbacks are numerous and occur often, the development of expertise requires effort and persistence over the course of many years, and the core goals of participation often relate to the development of human abilities (Vella et al., 2016).

But how should we understand and employ the construct with dichotomous concepts? Aforementioned above, the implicit beliefs about the malleability or fixedness of human attributes are divided into logically two different forms (i.e., changeable, or not?). In other words, two beliefs are assumed to be in a uni-dimension, in which they are in opposing positions to each other on a continuous line; if one belief is high, the other is low. It was the way Dweck and Leggett’s (1988) first introduced the term in their seminal article and still widely used in research. In doing so, most conceptual papers and empirical studies have treated the two implicit beliefs as opposite ends of a continuum (Dinger & Dickhäuser, 2013; Dweck et al., 1995b; Marquet et al., 2021; Scherer & Campos, 2022; Tempelaar et al., 2014). However, there were some critical comments about the appropriateness of a single-dimensional approach and the possibility of holding competing cognitions (Bodill & Roberts, 2013; J. A. Chen & Pajares, 2010). These studies are focused on the study of Dweck et al. (1993), which asserts a multi-dimensional approach that contradictory beliefs can be simultaneously held in the same domain. Studies with the tune in that people can hold more than one belief, however, did not lead to major changes in the operationalization of implicit beliefs in their empirical studies (Lüftenegger & Chen, 2017).

Accordingly, challenges in measuring or assessing the construct with ambiguous dimensionality emerged. Early studies have used semantic differential measures (i.e., changeable, or not?) to evaluate the degree of belief in the malleability of ability as a bipolar that exists in a single dimension (e.g., Spinath & Schöne, 2003). In this single-dimensional approach, respondents could be evaluated as a one-position between the entity and incremental belief on a continuum within one statement. However, the existence of two different beliefs can’t be verified from this perspective. Instead, relatively recent discussions regarding the multi-dimensional nature of implicit beliefs have drawn attention (Ingebrigtsen, 2018; Preißinger & Schoen, 2018). These studies have reported a better goodness-of-fit following factor analysis of the measurement models containing two opposing sub-factors (incremental/entity belief factor; Burnette et al., 2013; Dupeyrat & Mariné, 2005). In this multi-dimensional approach, by reverse scoring either of the two beliefs (e.g., Tempelaar et al., 2014) or adopting only one of the beliefs (e.g., Pomerantz & Kempner, 2013), a single score is calculated that can determine the dominance of either belief. Nevertheless, treating each as its own construct without considering the interplay of each other has still a non-consistency between conceptualization and measurement (Lüftenegger & Chen, 2017).

At this point, the problem to decide whether to classify a respondent as an entity holder or an incremental holder needs to be solved. Studies dealing with this issue are rare (Lüftenegger & Chen, 2017; Preißinger & Schoen, 2018; Tempelaar et al., 2014), but the consistency between conceptualization and measurement still matters. Dichotomizing the implicit beliefs construct with the multidimensional measurement model should be empirically examined.

The Conceptions of the Nature of Athletic Ability Questionnaire-2 (CNAAQ-2)

Implicit beliefs assessments across domains typically use a self-report questionnaire adapted from the original measure of implicit theories of intelligence (Dweck, 2000). In the context of sport and exercise, the implicit beliefs about athletic (sport) ability were commonly discussed from the multi-dimensional perspective as proposed by Sarrazin et al. (1996). Sarrazin et al.’s measurement tool, the Concepts of the Nature of Athletic Ability Questionnaire (CNAAQ) was developed with children and youth aged 11 to 17 in England and France by testing the psychometric properties of the scale on which assessing respondents’ representations of sport ability was made. It consists of 21 items with six first-order factor measurement model (learning, improvement, specific, gift, stable, and general). However, subsequent empirical studies using CNAAQ had reported low reliability of these sub-factors (Biddle et al., 1999; Lintunen et al., 1999; Ommundsen, 2001a, 2001b). Other researchers (Biddle et al., 2003; Wang et al., 2005) developed a revised version of CNAAQ (i.e., CNAAQ-2) reflecting a higher-order factors model (incremental; learning and improvement, and entity; gift and stable; and three items for each sub-factor) and reported its cross-cultural validity between UK and Asian samples aged 11 to 18. This scale has been used by many studies in the field of exercise and sport settings (e.g., Evans et al., 2020; Hernández-Andreo et al., 2020; Wang & Liu, 2007; Wang et al., 2009; Warburton & Spray, 2017). The multidimensional model is still applied for the psychometric advantages it offers, although the conceptual debates and re-verification of measurement models to define the dimensionality of implicit beliefs are still underway (Burnette et al., 2013; Lüftenegger & Chen, 2017; Vella et al., 2016).

Exploring the Dimensionality of CNAAQ-2

Using this multi-dimensional scale (CNAAQ-2) presupposes the independence of the two implicit beliefs, and has limitations that fail to fully consider the dimensionality and interrelationship of the specific factors (incremental and entity beliefs) and the general factor (implicit beliefs). In other words, the comprehensive concept of implicit beliefs assumed by the scale to measure each dichotomized belief has an issue of contradiction. Therefore, it is necessary to empirically verify the dimensionality of implicit beliefs by identifying the overall factor structure of each specific factor as well as the general factor assumed in the multidimensional measurement model (i.e., the CNAAQ-2).

In relation to this issue of measurement dimensionality, factoring solution with a bifactor model that includes both general and specific factors can be a possible approach to identifying the relative significance of factors (F. F. Chen et al., 2006, 2012). In this model, the explanation of variance in each specific factors’ correspondence to the individual measurement variables is statistically compared to that of the general factor, which is loaded together with all items; if both explanations are significant, the general factor and specific factors may be simultaneously meaningful. If only the domain-specific factors are significant, the utilization of the total score assumed by the higher-order variable (general factor) is not reasonable; meanwhile, if only the general factor is significant, the individual score approach that assumes the multi-dimensionality (specific factor) of the measurement model may be meaningless. This statistical approach is a procedure that can substantially examine the internal validity of the implicit beliefs that comprise CNAAQ-2. Based on previous research, the following hypothesis has been proposed:

H1: Factor analysis will show that the measurement model with general and specific factors fits the data.

Exploring the Discriminability of CNAAQ-2

The problem of classifying respondents based on their possession of a particular belief according to a multidimensional model of implicit beliefs still remains (Lüftenegger & Chen, 2017). For example, if the score for incremental belief and that for entity belief do not contradict each other, it should not be automatically assumed that they hold a certain belief. A study by Gucciardi, Jackson, et al. (2015) used multidimensional questionnaires with participants from different achievement context (students, employees, and athletes), and showed that nearly half (35.9−51.7%) of the cases did not indicate any one predominant belief (incremental/entity), and an endorsement of the entity belief was not observed. On the other hand, Dweck (2012) argued that about 80% of people are expected to maintain the endorsement of a particular incremental or entity belief. As such, the classification of respondents to implicit beliefs differs depending on the study. Using respondent attributes for classification relates to the discriminability of the measurement tool, which allows for more extended discussions on external validity as well as the scale’s internal structure. Therefore, it should be integrally verified whether the measurement scores from CNAAQ-2 classify a given respondent as holding a particular belief and whether such classification predicts significant differences in the variables associated with implicit beliefs.

In relation to this issue of measurement discriminability, an empirical strategy of cut-off was originally used to classify a participant as an incremental or entity holder (e.g., Dweck et al., 1995a). In this approach, though it is a brief way to classify the respondents with specific criteria (mean or deviation score etc.), clear criteria that differentiate the other groups at a specific point are still missing as well as a clear definition of the mixed group (Blackwell et al., 2007). Therefore, standard methods of correlational analysis to the specific factors or person-centered analysis were proposed as an alternative approach (Lüftenegger & Chen, 2017). Especially, cluster analysis can be employed to verify whether classifying respondents as holding implicit beliefs is consistent with the theoretical model, or whether another form of classification exists.

Also, the clusters based on the measures of CNAAQ-2 should predict other variables which are theoretically associated with it (Petscher et al., 2021). It is critically important to expand the practical implication from the perspective of external validity. In the current study, mental toughness (MT) was considered to be a criterion for the level of CNAAQ-2, because its’ empirical finding indicates that the level of implicit beliefs is significantly related to MT (Jang et al., 2020). MT is commonly defined as “the personal capacity to deliver high performance on a regular basis despite varying degrees of situational demands” (Gucciardi, Hanton, et al., 2015), and MT is theoretically associated with social cognitive motivators, such as implicit beliefs (Harmison, 2011; Smith, 2006). This is supported by the SCMM (Dweck & Leggett, 1988, 2000; Molden & Dweck, 2006), which proposed that incremental beliefs are related to an adaptive response, while entity beliefs are related to a nonadaptive response in continuing learning and task performance. Based on previous research, the following hypothesis has been proposed:

H2: The population mean of mental toughness will show a statistically significant difference depending on the implicit beliefs clusters.

Purpose of Study

As discussed above, a multidimensional measurement model with a dichotomous concept could lead to confusion in the interpretation of results. To resolve these problems, the present study has two purposes. The first is to confirm the multidimensional nature of the measurement tool, CNAAQ-2, by verifying the independence of the incremental and entity belief factors assumed in the implicit beliefs construct. The second purpose is to verify whether the individual differences in implicit beliefs can be distinguished, based on the multidimensional scores generated by CNAAQ-2. In addition to this examination of internal validity (i.e., dimensionality), the examination of discriminability of CNAAQ-2-based cluster to external criterion was conducted for examining the external validity. According to the proposed hypotheses, this study aims to test the research model shown in Figure 1.

Figure 1.

Conceptual framework.

Materials and Methods

Data and Measure

The data used in the present study were derived from 322 surveys that were collected in a study conducted by Jang et al. (2020), with authors’ permission. The whole procedure of Jang et al. (2020) was reviewed and approved by the institutional research ethics committee. The study sample comprised middle/high school football players who participated in a weekend league hosted by the Korean Football Association, South Korea. This study applied convenience sampling among non-probabilistic methods and recruited participants upon promotional materials to the football club to request cooperation delivered to the athletes. If they gave consent for the researchers to approach their team, they were asked to set aside 15 min at the beginning of a training session. At the club’s training site, verbal and written instructions were given and athletes signed an informed consent form. The questionnaires took approximately 10 to 15 min to complete. Respondents included in the present study comprised 322 football players (all male) between 13 and 18 years of age (M = 15.02, SD = 1.64).

CNAAQ-2 (Biddle et al., 2003; Wang et al., 2005) was used to measure the implicit beliefs of respondents (Table 1). This questionnaire comprised of 12 items (two sub-factors comprising incremental and entity belief with six items for each), and respondents were asked to respond on a 5-point Likert ranging from 1 (strongly disagree) to 5 (strongly agree). The higher the score, the stronger a given belief. The questionnaire was translated into Korean language, and the cross-validation of the Korean version of CNAAQ-2 was reported by verifying structural equivalence based on gender, school level, and sports (J. Park, 2010; G. Park & Kim, 2010). Jang et al. (2020) conducted a pilot survey of 50 middle and high school football players and reported the reliability (Cronbach’s α) of each factor (incremental belief = .795, entity belief = .719).

Table 1.

Measurement Items of CNAAQ-2.

Items		Constructs
My beliefs about athletic ability in sport:
I1	You must learn and work hard to excel at sports.	Incremental belief
I2	To reach a high level of performance in sports, you must undergo periods of learning and training
I3	To be successful in sports you must learn techniques and skills, and then practice them regularly.
I4	Your athletic ability will always improve if you work at it.
I5	In sports, if you work hard, you will always improve.
I6	If you put enough effort into it, your athletic ability will always improve.
E1	To excel at sports you must be naturally gifted.	Entity belief
E2	You must have certain “gifts” to excel at sports.
E3	To excel at sports, you must be born with the basic qualities that allow you to succeed.
E4	Even if you try to improve, the level you reach in sport will change very little.
E5	It is difficult to change the extent to which you excel at sports.
E6	You have a certain level of ability in sports and you cannot really do much to change that level.

As mentioned above, the level of MT is theoretically associated with implicit beliefs. In this context, the KF-MTI (Korean Football—Mental Toughness Inventory), which was developed by Jang et al. (2020), was used to measure mental toughness in their study. This questionnaire consists of 15 items assessing three factors (subscales of striving, surviving, and confidence). The responses ranged from 1 (Not at all) to 5 (Very true), and the total score on this questionnaire was averaged to generate the mean score for MT (Cronbach’s α = 0.92; striving = 0.84, surviving = 0.91, confidence = 0.79).

Data analysis

In order to accomplish the purposes of the current study, a two-stage analysis procedure was employed. Data analysis stages 1 and 2 using IBM SPSS (Statistics 25.0 and AMOS 23.0) and R Studio version 1.1.463 programs are as follows:

Stage 1: Exploring the Dimensionality of Implicit Beliefs

To verify the dimensionality of the incremental and entity belief factors assumed in the measurement model, we conducted confirmatory factor analyses (CFAs) with the first-order models (one-factor and two-factor models), and a bifactor model (F. F. Chen et al., 2006, 2012) that can identify the relative influence of a higher latent construct (general factor) to each sub-factor (specific factors).

First, the descriptive statistics were calculated to verify the normality of the data. Second, CFAs using the maximum likelihood estimation were conducted for both first-order models, which verified the convergent and discriminant validity of each specific factors assumed in the measurement model, and a bifactor model, which verified the overall construct validity of the higher-order latent variables (with general factor). The validity of the measurement model was evaluated through the goodness-of-fit index ( $χ^{2}$ , comparative fit index [CFI], Tucker–Lewis index [TLI], and root mean square error of approximation [RMSEA]) of each measurement model, as well as the construct reliability (CR) and average variance extracted (AVE) for each factor (Hair et al., 2010; Hu & Bentler, 1999). The final measurement model for the subsequent analyses was accepted based on the $χ^{2}$ difference test and comparing AIC (Akaike’s Information Criterion) and ECVI (Expected Cross Validation Index) between different models.

Stage 2: Exploring the Discriminability of Implicit Beliefs

In order to extend the validity of the questionnaire, k-means cluster analysis was employed to classify respondents into mutually exclusive groups. Following the cluster analysis, an analysis of variance (ANOVA) was performed to confirm that the classification of clusters based on the level of implicit beliefs represents a difference in the relevant variable, MT.

First, the data used in the cluster analysis were standardized to ensure ease in the interpretation of the results, and the optimal number of clusters k was calculated using the ‘NbClust’ package of statistical software R. The R package “NbClust” provides 30 indices that determine the number of clusters in the data set along with the best construct from clustering result (Charrad et al., 2014). Specifically, among the 30 indices, the most frequent number of optimal clusters was shown at three. In k-means clustering, the geometric mean of the observations was positioned in the center of the cluster. For each of the remaining observations, we identified the nearest cluster of k clusters based on the Euclidean distance, included them in that particular cluster, and repeatedly calculated its internal deviation. We interpreted the characteristics of the clusters using their centroid after the iteration.

Next, ANOVA was performed by setting MT as a dependent variable and cluster as an independent variable. Following this, as a post-hoc test, Tukey’s honestly significant difference (HSD) test was conducted to confirm the pattern of differences between clusters.

Results

The Dimensionality of Implicit Belief

Descriptive Statistics

The descriptive statistics of CNAAQ-2 items are shown in Table 2. Two items of incremental belief, I1 and I3, showed high level of kurtosis (each 6.10 and 5.76), and thus these two items were not included in the further analysis

Table 2.

Descriptive Statistics of CNAAQ-2.

Variables	Items	Mean	SD	Skewness	Kurtosis
Athletic (sport) ability beliefs
Incremental belief	I1	4.67	0.53	−1.82	6.10
	I2	4.78	0.44	−1.66	1.62
	I3	4.68	0.57	−2.02	5.76
	I4	4.26	0.80	−0.91	0.28
	I5	4.35	0.77	−1.07	0.66
	I6	4.33	0.80	−1.08	0.63
	Total	4.43	0.57
Entity belief	E1	3.78	0.93	−0.51	−0.22
	E2	4.08	0.74	−0.73	1.03
	E3	3.74	0.97	−0.46	−0.40
	E4	3.12	1.12	−0.17	−1.00
	E5	2.85	1.18	0.10	−0.95
	E6	2.56	0.97	0.25	−0.53
	Total	3.36	0.60

Note. The total mean and standard deviation were calculated after deleting items (incremental 1 and 3).

The CFAs for the Measurement Models

Comparison of the Goodness-of-Fit of the First-Order and Bifactor Models

To verify the dimensionality of CNAAQ-2, CFAs for the first-order model and the bifactor model were conducted. The CFA results for the first-order one-factor model showed TLI = 0.446, CFI = 0.569, and RMSEA = 0.182, indicating that the model fit is unacceptable, and all factor loadings for entity belief items was shown to be insignificant. Accordingly, based on the theoretical background of the implicit beliefs model, CFA for the two-factor model was conducted. Initial result of the CFA for the two-factor model showed TLI = 0.839, CFI = 0.879, and RMSEA = 0.098, indicating that the model fit was marginal. The revised CFA for two-factor model (Figure 2), excluding four questions with factor loadings lower than .7 (i2; e4, e5, and e6), revealed a good model fit ( $χ^{2}$ = 4.846, df = 8, p = .774, TLI = 1.008, CFI = 1.000, RMSEA < .001). The CR value for incremental belief was .856 and entity belief was .786, with an AVE value of .664 for incremental belief and .553 for entity belief, thus supporting the convergent validity of each factor. Further, the correlation between the factors is .095 (p = .160), which is less than the AVE value, thus supporting discriminant validity.

Figure 2.

Final CFA for the first-order model.

CFA for the bifactor model for identifying the relative significance of the general factor (implicit beliefs) and each domain-specific factors (incremental/entity) was conducted (Figure 3). Due to its nested relationship with the first-order model, CFA was applied for the same six items and all goodness-of-fit indices were revealed as satisfactory ( $χ^{2}$ = 2.170, df = 4, p = .705, TLI = 1.010, CFI = 1.000, RMSEA < .001). The CR value for incremental belief is .801 and the entity belief is .751, with the AVE value of .581 for incremental belief and .550 for entity belief, thus supporting the convergent validity of each factor (general factor; implicit beliefs: CR = .876, AVE = .543). In the bifactor model, the relationship between the sub-factors (domain-specific factors) was assumed to be orthogonal (F. F. Chen et al., 2006, 2012), so the correlation was not calculated between the factors.

Figure 3.

CFA for the bifactor model.

Comparing AIC and ECVI, moreover, the two-factor solutions (first-order and bifactor) fit the data better than the one-factor model (Table 3). In addition, the result of $χ^{2}$ difference test showed that the bifactor model did not show a statistically superior fit compared to the first-order (two-factor) model ( $Δ χ^{2} = 2.676, Δ d f = 4$ ).

Table 3.

Comparison of Goodness-of-Fit.

Model	χ² (df, p)	TLI	CFI	RMSEA*	AIC	ECVI
First-order
One-factor	406.356 (35, <.000)	0.446	0.569	0.182	466.356	1.453
Two-factor	4.846 (8, .774)	1.008	1.000	<0.001	42.846	0.133
Bifactor	2.170 (4, .705)	1.010	1.000	<0.001	48.170	0.150

Note. Final first-order model’s RMSEA 90% CI [0.000–0.045], and bifactor model’s RMSEA 90% CI [0.000–0.063].

Comparison of the Relative Influence of a Higher Latent Construct and Each Sub-Factor

The estimated coefficients of the bifactor model are shown in Figure 2. The standardized loadings for the general factor (implicit beliefs) ranged from .149 to .372, the variance of the factors was .089, and all estimates were not significant. Meanwhile, the standardized loadings for the incremental belief ranged from .709 to .795, and those for the entity belief ranged from .598 to .802; the variances for each specific factors were .322 and .442, and all estimates were significant (p < .001).

Taken together, the evidence supported the more parsimonious first-order model with the two factors than the bifactor model. These findings indicate that H1 was supported. We, therefore, utilized this model for the subsequent analyses.

The Discriminability of Implicit Beliefs and the Relationships with External Variables

Results of Cluster Analysis

The results of the “NbClust” procedure showed that among the various indices, the optimal number of clusters is three. According to this result, we conducted cluster analysis by setting the optimal number of clusters at three.

The results of k-means clustering are shown in Table 4. Each cluster was divided into three clusters and consisted of 116 (36%), 95 (29.5%), and 111 (34.5%) observation points respectively. In Cluster 1, a range of the standardized scores for incremental belief items (I4, I5, and I6) was .492 to .568, and those of entity belief items (E1, E2, and E3) was .654 to .738. For Cluster 2, the incremental belief question revealed higher scores than average, ranging from .397 to .508, while the entity belief question revealed scores lower than average, ranging from −746 to −.889. For Cluster 3, the below-average scores were recorded in the incremental belief question, whereas the entity belief scores were close to average (0). According to these results, we characterized each cluster as follows. Cluster 1 has a high level of both incremental and entity belief, Cluster 2 has a high level of incremental belief, but a low level of entity belief, and Cluster 3 has a low incremental belief and a moderate entity belief. In other words, each belief does not take opposite (dichotomous) but is classified independently. These results suggest that the sub-factors of implicit beliefs, incremental and entity beliefs constitute independent dimensions, not the conflicting dichotomous psychological constructs.

Table 4.

Results of k-Means Cluster Analysis.

Constructs		Clusters
		First	Second	Third
Mean of clusters
Incremental belief	I4	0.567	0.397	−0.933
	I5	0.568	0.441	−0.971
	I6	0.492	0.508	−0.945
Entity belief	E1	0.738	−0.850	−0.043
	E2	0.654	−0.746	−0.044
	E3	0.728	−0.889	−0.0004
N (%)		116 (36%)	95 (29.5%)	111 (34.5%)

Note. The mean of each item for the cluster is zero (using standardized data). First cluster: a high level for both incremental and entity beliefs. Second cluster: a high level for incremental belief and a low level for entity belief. Third cluster: a low level for incremental belief and average level for entity belief.

Predicting MT by Cluster of CNAAQ-2

CFA for the Dependent Variable: KF-MTI

The MT was set as a dependent variable, which was measured in the study by Jang et al. (2020) through KF-MTI. The validity of the first-order construct for KF-MTI was verified in the study conducted by Jang et al. (2020), and CFA for the bifactor model for further examining the dimensionality was conducted in the current study. A satisfactory goodness-of-fit index for CFA was found using the same data from the three factors (striving, surviving, and confidence) and 15 items ( $χ^{2}$ = 150.988, df = 78, p < .001, TLI = 0.962, CFI = 0.971, RMSEA = .054). And the all estimates for general factor (mental toughness) were significant (factor loadings = .303–.727, p < .001, variance = .251, p < .001), but not for subfactors (striving, surviving, confidence). These results indicated that the MT, which set as the dependent variable for the following analysis, can be adopted as a higher-order single-factor (Cronbach’s α = .92).

Results of ANOVA

ANOVA was conducted to examine the difference in population means between each cluster for the MT. The results of ANOVA are shown in Table 5.

Table 5.

The Results Table of ANOVA.

	df	Sum of squares	Mean squares	F-value	p-Value
Cluster	2	9.68	4.83	19.34	<.001
Residuals	319	79.79	0.25

As shown in Table 5, a statistically significant difference between the groups for MT was found (F = 19.34, p < .001). Tukey’s HSD test revealed significant differences between Cluster 1 and Cluster 3 (diff. = .40, p < .001, Cluster 1 has a higher score than Cluster 3). Cluster 2 and Cluster 3 also showed a significant difference in their MT means (diff. = .30, p < .001, Cluster 2 has a higher score than Cluster 3). Meanwhile, the difference between Cluster 1 and 2 was not significant (diff. = .10, p = .31). These findings indicate that H2 was supported.

Discussion

Dweck’s (2012, 2017) concept of implicit beliefs has been demonstrated as a meaningful predictor in achievement and adaptive response in the competitive context such as sports and physical education settings (Vella et al., 2016). The question has been raised of this construct, based on “the malleability of a certain concept,” is its inclusion of contrasting dichotomous sub-factors of “incremental belief” and “entity belief.” Accordingly, CNAAQ-2, which measures the implicit beliefs of athletic ability, may create confusion in the classification and interpretation of the results because the measurement simultaneously includes the two opposing concepts. This has been evaluated as an issue for multidimensional measurement models involving dichotomous concepts. Thus, the current study conducted a series of statistical works such as CFA, cluster analysis, and ANOVA to verify the dimensionality and discriminability of CNAAQ-2. The findings support the internal structure of the two independent sub-factors for implicit beliefs and the construct’s predictive power for the external variable (MT).

The multi-dimensionality of the CNAAQ-2 was verified through the CFA procedure, and the convergent and discriminant validity of the incremental and entity belief factors was supported, thus indicating the independence of the sub-factors comprising the measurement model. While these results support recent studies arguing that the multidimensional measurement model of implicit belief should be divided into incremental and entity belief (Evans et al., 2020; Hernández-Andreo et al., 2020; Wang & Liu, 2007; Wang et al., 2009; Warburton & Spray, 2017), the existence of a global concept of implicit beliefs (i.e., general factor) was not empirically verified yet. This raises a question regarding the general factor score of implicit belief, that is, the level of analysis required for a total score approach, and may necessitate subsequent research on the structure of the construct upon which the concept of beliefs regarding athletic ability is based.

Nevertheless, several studies using this scale have identified implicit beliefs as a single-dimensional psychological construct simply by reverse-coding certain factors (mostly those pertaining to entity belief) without considering the influence of each belief (Cabello & Fernández-Berrocal, 2015; Reffi et al., 2020; Tarbetsky et al., 2016). In other studies (Blackwell et al., 2007; Hughes, 2015), a fixed cut-off method using deviation scores was applied to classify groups with certain attributes of high or low malleability following Dweck et al. (1995a), in which the researchers classified four or more points into incremental belief and three or fewer points into entity belief (on 6-point Likert scale). However, such a classification method seems to have insufficiently considered the multidimensional nature of the measurement model that can be extended to n-dimensions depending on the number of items included. These limitations were discussed in Lüftenegger and Chen (2017), and several alternative approaches were proposed in response (e.g., cluster analysis or latent profile analysis). Thus, in the present study, cluster analysis that includes more systematic procedures has been conducted to extend the discriminability of the measurement of implicit beliefs (CNAAQ-2).

The results of cluster analysis showed a predominant incremental belief in Cluster 2, which comprises 29.5% of all participants, and the remaining 70.5% did not show a clear distinction between the two concepts. These results agree with the findings of Gucciardi, Jackson, et al. (2015), which reported that nearly half of the cases did not reveal a dominant belief (incremental or entity). Meanwhile, Dweck (2012) argued that about 80% of people are expected to maintain the endorsement of a particular incremental or entity belief. The major difference between those findings may lay on whether the sub-factors were dealt with as dichotomous or not. To be specific, the previous studies (like Dweck’s approach) regarded implicit beliefs as a dichotomous concept so they simply classified the respondents into the endorse with the dominant belief by using revers-coding or cut-off points to the results of measurement. The current study, however, considered the sub-factors as an independently related construct based on the result of CFA, so all values of each measurement variable were adopted to cluster the respondents. This result also supported the multi-dimensionality of the measurement model which was examined in the prior stage. Accordingly, even though there were much of respondents who were not classified into some cluster with distinct characteristics, this integrated approach could be more valid to interpret the dimensionality and the discriminability of implicit beliefs measure than Dweck’s approach.

Examining external validity is important in the current study because it is related to the extent to which the relationships between samples, populations, and treatment variables can be generalized (Steckler & McLeroy, 2008; Thomas et al., 2015). Accordingly, ANOVA was performed to examine which clusters show significant mean differences in MT, which is defined as “the ability to carry out personal performance in demanding situations without abandonment based on the self-confidence and the desire to achieve a given goal,” was expected to related to implicit beliefs based on the rationale. The results of the cluster analysis showed that the cluster related to the low level of incremental belief (Cluster 3) generated the lowest MT, while that related to a high level of both beliefs (Cluster 1) revealed the highest MT. According to the SCMM presented by Dweck and Leggett (1988), the implicit beliefs in one’s own attributes (e.g., athletic ability) can predict situational judgment and response, that is, cognitive patterns, in a certain context. In the context of exercise and sport, implicit beliefs affect cognitive, emotional, and behavioral responses because beliefs about one’s abilities affect goal orientation and the intention and motivation for performance that follow (Sarrazin et al., 1996; Vella et al., 2016). This further means that task persistence and commitment in competitive situations, such as sports, can be predicted by implicit beliefs in athletic ability (Gardner et al., 2018; Vella et al., 2014). In the current study, this logical inference was supported by the result that the clusters of implicit beliefs can distinguish individual differences in MT.

Academic Contributions

In this paper, we used a step-by-step approach (i.e., a series of statistical examinations) to study issues in the measurement of the construct of implicit beliefs concept, which has contrasting dichotomous sub-factors of “incremental belief” and “entity belief.” As noted earlier, there are conceptual controversies in the current literature examining whether the implicit belief constructs are indeed multidimensional or unidimensional (Lüftenegger & Chen, 2017). The evidence supported a multi-dimensional conceptualization of athletic-malleability, CNAAQ-2. Furthermore, in terms of discriminability, the indicator of each belief classified the respondents into several groups of individual differences (i.e., clusters of implicit beliefs). In addition, this classification was associated with other concepts in expected ways (i.e., predict to MT). Therefore, we suggest that modeling incremental and entity beliefs separately as their multi-dimensional constructs has better predictive validity, rather than modeling the construct as opposite ends of the same construct (Burnette et al., 2013; Tempelaar et al., 2014).

Practical Implications

Our methodology provides initial insight into how important it is to systematically analyze the psychological construct associated with dichotomous/multidimensional problems (Taherdoost, 2016). In order to argue the whole validity of the construct, the dimensionality and discriminability of the instrument should be examined, simultaneously. Therefore, a series of statistical examinations of the current study may be desirable alternatives for future research seeking valid and reliable ways to assess the construct. We also highlighted the need to assess implicit beliefs in much more sophisticated ways than what scholars or experts have traditionally been able to do. As the measurement validity of implicit beliefs, including athletic ability, assumes the existence of each sub-belief, it is necessary to evaluate respondents by fully utilizing the individual scores of each belief rather than by simple methods such as reverse coding.

Limitations and Future Directions

As with all studies, there were a number of limitations in this study. First, the sample of the current study focused only on South Korean male soccer players in their 13 to 18 years of age. In this context, the questionnaire was translated into Korean, and thus half of the items were lost. Studies validating the psychological constructs may show different results depending on the cultural differences in the sample. This critique was argued through various studies (Hirai & Clum, 2000; Moreno-Murcia et al., 2013; Wang et al., 2005). In order to generalize the current findings, future study needs to replicate our approach to other populations. Second, we verified whether there was a difference in mental toughness according to the implicit belief cluster as a procedure to verify the discriminability of the CNAAQ-2. However, this study is limited to the cross-sectional design. Future studies would benefit from the approach to infer the causality between variables, such as longitudinal design or experimental manipulation.

Conclusion

Based on the theoretical background of implicit beliefs and the results of the current study, the following conclusions regarding the use of CNAAQ-2 can be made. First, the factor structure of the CNAAQ-2 examined in this study does not statistically support the existence of a higher-order latent variable (i.e., implicit beliefs), so the total score derived from the general factor approach should be re-considered. The individual scores of each domain-specific sub-scale, incremental and entity belief may be more meaningful; therefore, the analysis of measurement results shall be conducted at the level of the sub-factors. Second, results of cluster analysis derived from the individual scores of each sub-factor of the CNAAQ-2 showed that all the respondents weren’t classified into a specific endorsement with dominant belief, but generally into the ambiguous group. In the same vein as the results of CFA, therefore, it implies that the two beliefs that make up implicit beliefs are not a dichotomous structure, but are independent dimensions. Third, the clusters of implicit beliefs derived from the individual scores of each sub-factor have significantly associated with the theoretically relevant variable (MT). These results suggest that CNAAQ assumed in the multidimensional model can make predictions for external criteria, while at the same time presenting the guidance on how implicit belief scores could be derived. Our study offers several important implications for the academic fields of psychological measures. Considering the findings of the current study comprehensively, discussions on the interpretation of the specific factors constituting implicit beliefs will seem to be continuing. Thus, in order to systematically analyze the psychological construct related to issues on the dichotomous/multidimensional, our findings should be fully considered. Future research is expected to advance our knowledge of the existing problem by considering generalizations to other populations and supplementation of causal inference. We hope that these findings will stimulate future research to contribute more valid and reliable results of the psychological measures.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethics Statement

All procedures for the current study were approved by the Institutional Review Board of Seoul National University (No. 1706/002-011), Seoul, South Korea.

ORCID iDs

Dojin Jang

Soowoong Hwang

Data Availability Statement

The data that support the findings of this research is available from the corresponding author upon request.

References

Biddle

S. J.

Soos

Chatzisarantis

(1999). Predicting physical activity intentions using a goal perspectives approach: A study of Hungarian youth. Scandinavian Journal of Medicine and Science in Sports, 9, 353–357.

Biddle

S. J.

Wang

C. J.

Chatzisarantis

N. L.

Spray

C. M.

(2003). Motivation for physical activity in young people: Entity and incremental beliefs about athletic ability. Journal of Sports Science, 21(12), 973–989.

Blackwell

L. S.

Trzesniewski

K. H.

Dweck

C. S.

(2007). Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child Development, 78(1), 246–263.

Bodill

Roberts

L. D.

(2013). Implicit theories of intelligence and academic locus of control as predictors of studying behaviour. Learning and Individual Differences, 27, 163–166.

Burnette

J. L.

O’Boyle

E. H.

VanEpps

E. M.

Pollack

J. M.

Finkel

E. J.

(2013). MindSets matter: A meta-analytic review of implicit theories and self-regulation. Psychological Bulletin, 139(3), 655–701.

Cabello

Fernández-Berrocal

(2015). Implicit theories and ability emotional intelligence. Frontiers in Psychology, 6, 700.

Charrad

Ghazzali

Boiteau

Niknafs

(2014). NbClust: An R package for determining the relevant number of clusters in a data set. Journal of Statistical Software, 61, 1–36.

Chen

F. F.

Hayes

Carver

C. S.

Laurenceau

J. P.

Zhang

(2012). Modeling general and specific variance in multifaceted constructs: A comparison of the bifactor model to other approaches. Journal of Personality, 80(1), 219–251.

Chen

F. F.

West

S. G.

Sousa

K. H.

(2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41(2), 189–225.

10.

Chen

J. A.

Pajares

(2010). Implicit theories of ability of Grade 6 science students: Relation to epistemological beliefs and academic motivation and achievement in science. Contemporary Educational Psychology, 35(1), 75–87.

11.

Dinger

F. C.

Dickhäuser

(2013). Does implicit theory of intelligence cause achievement goals? Evidence from an experimental study. International Journal of Educational Research, 61, 38–47.

12.

Dupeyrat

Mariné

(2005). Implicit theories of intelligence, goal orientation, cognitive engagement, and achievement: A test of Dweck’s model with returning to school adults. Contemporary Educational Psychology, 30(1), 43–59.

13.

Dweck

C. S.

(1996). Implicit theories as organizers of goals and behavior. In Gollwitzer

P. M.

Bargh

J. A.

(Eds.), The psychology of action: Linking cognition and motivation to behavior (pp. 69–90). Guilford Press.

14.

Dweck

C. S.

(2000). Self-theories: Their role in motivation, personality, and development. Psychology Press.

15.

Dweck

C. S.

(2006). Mindset: The new psychology of success. Ballantine Books.

16.

Dweck

C. S.

(2012). Implicit theories. In Van Lange

P. A. M.

Kruglanski

A. W.

Higgins

E. T.

(Eds.), Handbook of theories in social psychology (pp. 43–62). Sage.

17.

Dweck

C. S.

(2017). The journey to children’s mindsets—and beyond. Child Development Perspectives, 11(2), 139–144.

18.

Dweck

C. S.

Chiu

C. Y.

Hong

Y. Y.

(1995a). Implicit theories and their role in judgments and reactions: A word from two perspectives. Psychological Inquiry, 6(4), 267–285.

19.

Dweck

C. S.

Chiu

C. Y.

Hong

Y. Y.

(1995b). Implicit theories: Elaboration and extension of the model. Psychological Inquiry, 6(4), 322–333.

20.

Dweck

C. S.

Hong

Y. Y.

Chiu

C. Y.

(1993). Implicit theories individual differences in the likelihood and meaning of dispositional inference. Personality and Social Psychology Bulletin, 19(5), 644–656.

21.

Dweck

C. S.

Leggett

E. L.

(1988). A social-cognitive approach to motivation and personality. Psychological Review, 95(2), 256.

22.

Dweck

C. S.

Leggett

E. L.

(2000). A social-cognitive approach to motivation and personality. In Higgins

E. T.

Kruglanski

A. W.

(Eds.), Key reading in social psychology. Motivational science: Social and personality perspectives (pp. 394–415). Psychology Press.

23.

Evans

M. B.

Vierimaa

Budziszewski

Graupensperger

(2020). Coach expectations and athlete lay beliefs: Interactions when predicting adolescent athletes’ enjoyment and intentions to return. Journal of Applied Sport Psychology, 32(4), 416–428.

24.

Gardner

L. A.

Vella

S. A.

Magee

C. A.

(2018). The role of implicit beliefs and achievement goals as protective factors in youth sport. Journal of Applied Sport Psychology, 30(1), 83–95.

25.

Gucciardi

D. F.

Hanton

Gordon

Mallett

C. J.

Temby

(2015). The concept of mental toughness: Tests of dimensionality, nomological network, and traitness. Journal of Personality, 83(1), 26–44.

26.

Gucciardi

D. F.

Jackson

Hodge

Anthony

D. R.

Brooke

L. E.

(2015). Implicit theories of mental toughness: Relations with cognitive, motivational, and behavioral correlates. Sport, Exercise, and Performance Psychology, 4(2), 100–112.

27.

Hair

J. F.

Black

W. C.

Babin

B. J.

Anderson

R. E.

(2010). Multivariate data analysis: Global edition. Pearson Education Limited.

28.

Harmison

R. J.

(2011). A social-cognitive framework for understanding and developing mental toughness in sport. In Gucciardi

D. F.

Gordon

(Eds.), Mental toughness in sport: Developments in theory and research (pp. 47–68). Routledge.

29.

Hernández-Andreo

Gómez-Mármol

Cifo-Izquierdo

M. I.

(2020). Effects on motivation and implicit beliefs about self-ability using the Sports Education Model and the traditional style in secondary education. Sustainability, 12(9), 3843.

30.

Hirai

Clum

G. A.

(2000). Development, reliability, and validity of the beliefs toward mental illness scale. Journal of Psychopathology and Behavioral Assessment, 22(3), 221–236.

31.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55.

32.

Hughes

J. S.

(2015). Support for the domain specificity of implicit beliefs about persons, intelligence, and morality. Personality and Individual Differences, 86, 195–203.

33.

Ingebrigtsen

(2018). How to measure a growth mindset: A validation study of the implicit theories of intelligence scale and a novel Norwegian measure [Master’s thesis, The Arctic University].

34.

Jang

Ahn

Kwon

(2020). Relationships between implicit beliefs and mental toughness: The role of implicit beliefs of adolescent football players and their coaches. Journal of Physical Education and Sport, 20(1), 156–163.

35.

Lintunen

Valkonen

Leskinen

Biddle

(1999). Predicting physical activity intentions using a goal perspectives approach: A study of Finnish youth. Scandinavian Journal of Medicine and Science in Sports, 9(3), 44–352.

36.

Lüftenegger

Chen

J. A.

(2017). Conceptual issues and assessment of implicit theories. Zeitschrift für Psychologie, 225(2), 99–106.

37.

Marquet

Plaks

J. E.

Balasubramaniam

Brunet

Chasteen

A. L.

(2021). Older adults’ lay theories of abilities predict their willingness to engage in preventive health behaviors. Archives of Gerontology and Geriatrics, 93, 104291.

38.

Molden

D. C.

Dweck

C. S.

(2006). Finding “meaning” in psychology: A lay theories approach to self-regulation, social perception, and social development. American Psychologist, 61(3), 192.

39.

Moreno-Murcia

J. A.

Cervelló-Gimeno

E. M.

Martínez-Galindo

M. C.

Moreno

(2013). Validation of the scale of implicit beliefs of ability (CNAAQ-2) to the Spanish context. Differences according to the physical practice sports. RICYDE: Revista Internacional de Ciencias del Deporte, 9(32), 100–113.

40.

Ommundsen

(2001a). Self-handicapping strategies in physical education classes: The influence of implicit theories of the nature of ability and achievement goal orientations. Psychology of Sport and Exercise, 2, 139–156.

41.

Ommundsen

(2001b). Pupils’ affective responses in physical education classes: The association of implicit theories of the nature of ability and achievement goals. European Physical Education Review, 7, 219–242.

42.

Park

Kim

(2010). Structural relationships between sport ability belief, achievement goal, and behavioral regulations among student-athletes: Perspective of three different theories of motivation. Korean Journal of Sport Psychology, 21(2), 101–129.

43.

Park

(2010). Relationships between sport ability beliefs, 2 × 2 achievement goals, and motivational regulations: Multi-group analysis across gender and grade. Korean Journal of Physical Education, 49(1), 113–126.

44.

Petscher

Al Otaiba

Wanzek

(2021). Study of the factor structure, profiles, and concurrent validity of the mindset assessment profile tool for elementary students. Journal of Psychoeducational Assessment, 39(1), 74–88.

45.

Pomerantz

E. M.

Kempner

S. G.

(2013). Mothers’ daily person and process praise: Implications for children’s theory of intelligence and motivation. Developmental Psychology, 49(11), 2040–2046.

46.

Preißinger

Schoen

(2018). Entity and incremental theory of personality: Revisiting the validity of indicators. Personality and Individual Differences, 130, 21–25.

47.

Reffi

A. N.

Darnell

B. C.

Himmerich

S. J.

White

K. J.

(2020). Implicit beliefs of emotion and anxiety in psychotherapy. Motivation and Emotion, 44, 453–463.

48.

Sarrazin

Biddle

Famose

J. P.

Cury

Fox

Durand

(1996). Goal orientations and conceptions of the nature of sport ability in children: A social cognitive approach. British Journal of Social Psychology, 35(3), 399–414.

49.

Scherer

Campos

D. G.

(2022). Measuring those who have their minds set: An item-level meta-analysis of the implicit theories of intelligence scale in education. Educational Research Review, 37, 100479.

50.

Smith

R. E.

(2006). Understanding sport behavior: A cognitive-affective processing systems approach. Journal of Applied Sport Psychology, 18(1), 1–27.

51.

Spinath

Schöne

(2003). Die Skalen zur Erfassung subjektiver Überzeugungen zu Bedingungen von Erfolg in Lern- und Leistungskontexten (SE-SÜBELLKO). In Stiensmeier-Pelster

Rheinberg

(Eds.), Diagnostik von Motivation und Selbstkonzept [Diagnosis of motivation and self-concept] (pp. 15–27). Hogrefe.

52.

Steckler

McLeroy

K. R.

(2008). The importance of external validity. American Journal of Public Health, 98(1), 9–10.

53.

Taherdoost

(2016). Validity and reliability of the research instrument; How to test the validation of a questionnaire/survey in a research. International Journal of Academic Research in Management, 5(3), 28–36.

54.

Tarbetsky

A. L.

Collie

R. J.

Martin

A. J.

(2016). The role of implicit theories of intelligence and ability in predicting achievement for indigenous (aboriginal) Australian students. Contemporary Educational Psychology, 47, 61–71.

55.

Tempelaar

D. T.

Rienties

Giesbers

Gijselaers

W. H.

(2014). The pivotal role of effort beliefs in mediating implicit theories of intelligence and achievement goals and academic motivations. Social Psychology of Education, 18(1), 101–120.

56.

Thomas

J. R.

Nelson

J. K.

Silverman

S. J.

(2015). Research methods in physical activity. Human kinetics.

57.

Vella

S. A.

Braithewaite

R. E.

Gardner

L. A.

Spray

C. M.

(2016). A systematic review and meta-analysis of implicit theory research in sport, physical activity, and physical education. International Review of Sport and Exercise Psychology, 9(1), 191–214.

58.

Vella

S. A.

Cliff

D. P.

Okely

A. D.

Weintraub

D. L.

Robinson

T. N.

(2014). Instructional strategies to promote incremental beliefs in youth sport. Quest, 66(4), 357–370.

59.

Wang

C. K. J.

Liu

W. C.

(2007). Promoting enjoyment in girls’ physical education: The impact of goals, beliefs, and self-determination. European Physical Education Review, 13(2), 145–164.

60.

Wang

C. K. J.

Liu

W. C.

Biddle

S. J.

Spray

C. M.

(2005). Cross-cultural validation of the Conceptions of the Nature of Athletic Ability Questionnaire Version 2. Personality and Individual Differences, 38(6), 1245–1256.

61.

Wang

C. K. J.

Liu

W. C.

Lochbaum

M. R.

Stevenson

S. J.

(2009). Sport ability beliefs, 2 x 2 achievement goals, and intrinsic motivation: The moderating role of perceived competence in sport and exercise. Research Quarterly for Exercise and Sport, 80(2), 303–312.

62.

Warburton

V. E.

Spray

C. M.

(2017). Implicit theories of ability in physical education: Current issues and future directions. Journal of Teaching in Physical Education, 36(3), 252–261.