Sage Journals: Discover world-class research

Abstract

A recent meta-analysis claimed to provide evidence that academic self-concept and achievement have reciprocal prospective effects on each other (reciprocal effects model). However, prospective effects were estimated while adjusting for a prior measurement of the outcome, and this method is susceptible to spurious findings due to correlations with residuals and regression to the mean. Here we re-analyze the meta-analytic effects and show that different plausible models can support opposing claims: either that self-concept had an increasing or a decreasing effect on achievement, and vice versa. Consequently, claims beyond a positive cross-sectional correlation between academic self-concept and achievement, including the reciprocal effects model, can be questioned. The findings were validated by analyses of simulated data, which indicated that true prospective effects were not necessary for the observed meta-analytic associations. We further propose the extended skill development model (ESDM) as a more parsimonious alternative to the reciprocal effects model.

Plain Language Summary

The so called reciprocal effects model of academic self-concept and achievement might be wrong

In a recent aggregation of findings from several studies, a so called meta-analysis, researchers claimed to have found evidence that academic self-concept (that is, self-perceived academic competence) had an increasing effect on future academic achievement and that academic achievement, likewise, had an increasing effect on future academic self-concept. However, the used method of statistical analysis is known to often deliver defective results. In the present study, we analyzed the same data and found that the data can be claimed to show both an increasing and a decreasing effect of academic self-concept on future academic achievement. Due to these incongruent results, we believe that the claims in the previous meta-analysis lack solid support and can be challenged. There may still exist a correlation between academic self-concept and achievement, but correlations do not prove causal effects. We also carried out simulations, and these showed that results as in the challenged meta-analysis can be observed even without any true increasing effects between academic self-concept and achievement. In the present article, we also present something we call the extended skill development model (ESDM).

Keywords

academic achievement academic self-concept reciprocal effects model re-meta-analysis simulation spurious prospective effects

Introduction

Self-concepts refer to mental representations capturing our views and beliefs about ourselves. Self-concepts usually include multiple dimensions, linked to the roles we play, the relationships we have, and our hopes and fears about who we may become in the future (Kenrick et al., 2007). Self-concepts are related, but not identical, to the extensively studied qualities self-esteem and self-efficacy. While self-concepts are more cognitive in character, pertaining to a descriptive component of oneself, self-esteem is more evaluative, that is, our attitudes toward ourselves (Kenrick et al., 2007). Self-efficacy, on the other hand, refers to individuals’ beliefs that they can act in ways necessary to reach specific goals (Bandura, 1977).

Academic Self-Concept and Achievement

Academic self-concept refers to individuals’ self-perceived competence in specific domains, for example, mathematics or English (Marsh & Martin, 2011; Shavelson et al., 1976). Self-concepts appear to be affected by social comparisons. For example, estimated average school/class ability has a negative association with individual self-concept when adjusting for individual ability, meaning that among equally able students, those in a school/class with less able students have a higher self-concept compared with those with more able school/classmates. This phenomenon has been named the big-fish-little-pond effect (Fang et al., 2018; Koivuhovi et al., 2022; Marsh et al., 2008; Seaton et al., 2009).

Academic self-concept is positively associated with academic achievement (Calsyn & Kenny, 1977; Gorges et al., 2018; C. Huang, 2011; Möller et al., 2020; Wolff & Möller, 2022). According to the self-enhancement model, this positive association is due to an effect of self-concept on achievement, for example via motivation and effort (Sewasew et al., 2018; Shavelson & Bolus, 1982; Valentine et al., 2004). Contrarily, the skill development model claims that achievement has an effect on self-concept (Ganley & Lubienski, 2016; Helmke & van Aken, 1995; Newman, 1984; Skaalvik & Valås, 1999). However, support has also been claimed for the reciprocal effects model which suggests effects in both directions (Guay et al., 2003; Marsh, 2022; Marsh & Craven, 2006; Marsh et al., 2002; Marsh & Martin, 2011; Niepel et al., 2014; Retelsdorf et al., 2014). Recently, after conducting a meta-analysis of longitudinal studies, Wu et al. (2021) concluded that academic self-concept and achievement prospectively predicted each other while adjusting for a prior measurement of the outcome. Wu et al. (2021) claimed that their meta-analysis provided evidence for the reciprocal effects model.

Prospective Effects May Be Spurious

However, it is well known that adjusted prospective effects may be spurious due to correlations with residuals and regression to the mean (Castro-Schilo & Grimm, 2018; Eriksson & Häggström, 2014; Glymour et al., 2005; Sorjonen et al., 2019). As an example, if a sumo wrestler and an ascetic eat an equal amount of food a certain week, we should probably assume that the sumo wrestler has eaten less than he usually does, that is, he has experienced a negative residual, and/or that the ascetic has experienced a positive residual. As residuals tend to regress toward a mean value of zero between measurements, we should expect a more positive change in eating to the subsequent week for the sumo wrestler than for the ascetic. On group level, the dichotomous variable “being a sumo wrestler vs. an ascetic” probably has a positive effect on subsequent eating when adjusting for prior eating even if no group level change has taken place. Furthermore, as regression to the mean is independent of the direction of time, we should expect a positive effect of “being a sumo wrestler vs. an ascetic” on prior eating when adjusting for subsequent eating.

Prospective effects between academic achievement and academic self-concept could be spurious in both directions. However, a cross-sectional effect of achievement on self-concept has high credibility on the face of it (Marsh, 2022; Marsh & Martin, 2011). Just as actual income can be expected to have an effect on self-perceived income, and actual height can be expected to have an effect on self-perceived tallness, achievement can be expected to have an effect on self-perceived competence, that is, self-concept. Conversely, an effect of self-perceived income, tallness, and competence on actual income, height, and achievement appears much less credible. It should be noted that assuming a positive cross-sectional effect of achievement on self-concept does not automatically translate to assuming a positive prospective effect, just like assuming a positive cross-sectional effect of height on self-perceived tallness is not the same as assuming that height has a positive effect on subsequent change in self-perceived tallness.

Research Context

The present study is part of a series where we have reanalyzed meta-analyses using cross-lagged panel analyses (Sorjonen, Ingre, et al., 2023; Sorjonen, Melin, et al., 2023; Sorjonen, Nilsonne, et al., 2023; Sorjonen and Melin, 2023, 2024a, 2024b). Most of the reanalyzed adjusted meta-analytic prospective effects have been found to be spurious, probably due to correlations with residuals and regression to the mean. A mutual message in these studies is that cross-lagged effects while adjusting for a prior measurement of the outcome variable seldom prove anything over and above cross-sectional correlations combined with less than perfect reliability in measurements. It is important for researchers to be aware of these limitations of cross-lagged effects, meta-analytically estimated or not, in order not to overinterpret findings. The continued output of studies with uncritical employment of cross-lagged panel analyses indicates that knowledge of these limitations, although far from new, are lacking in the research community. With our reanalyses we hope to alleviate this lack of knowledge, at least to some degree.

Objectives and Research Questions

The objective of the present study was to evaluate if a prospective effect of academic self-concept on subsequent change in academic achievement when adjusting for initial achievement, and vice versa, may be spurious due to correlations with residuals and regression to the mean. A further objective was to evaluate if prospective effects are compatible with a situation where data are generated by a model/mechanism with a unidirectional cross-sectional effect of trait-like achievement on trait-like self-concept. The research questions of the present study were: (1) Could adjusted prospective effects between academic self-concept and academic achievement in the meta-analysis by Wu et al. (2021) have been spurious due to correlations with residuals and regression to the mean? (2) Are prospective effects between academic self-concept and academic achievement compatible with a situation where data are generated by a model/mechanism with a unidirectional cross-sectional effect of trait-like achievement on trait-like self-concept? To achieve the objectives, and to answer the research questions, we re-analyzed effects included in the meta-analysis by Wu et al. (2021) and conducted a simulation.

Method

Reanalysis of Meta-Analysis

We refer to Wu et al. (2021) for information on selection and coding of studies, descriptive statistics, etc. The present analyses were conducted on the same correlations as in Wu et al. (2021, see their online supplementary material). For each of the studies included in their meta-analysis, Wu et al. used equation (1) (Cohen et al., 2003) to estimate the effect of initial academic self-concept on subsequent achievement while adjusting for initial achievement, and vice versa. We did the same. Here, both a hypothesis of true increasing prospective effects and a hypothesis of spurious associations, due to correlation with residuals and regression to the mean, predicted positive effects.

In addition to the analyses conducted by Wu et al. (2021), we used equation (1) to estimate the effect of initial self-concept on initial achievement while adjusting for subsequent achievement, and vice versa. Here, a hypothesis of true increasing prospective effects predicted negative effects, which would mean that among those with the same subsequent achievement (self-concept) those with higher initial self-concept (achievement) tended to have had lower initial achievement (self-concept) and had, consequently, experienced a more positive change in achievement (self-concept) compared with those with the same subsequent achievement (self-concept) but with lower initial self-concept (achievement). Contrarily, a hypothesis of spurious associations predicted positive effects.

E | β_{X 1 Y 2 . Y 1} | = \frac{r_{X 1 Y 2} - r_{X 1 Y 1} r_{Y 1 Y 2}}{1 - r_{X 1 Y 1}^{2}}

(1)

Moreover, we used equation (2) (Guilford, 1965) to estimate the effect of initial self-concept on subsequent change in achievement, and vice versa. Here, a hypothesis of true increasing effects predicted positive effects. Conversely, a hypothesis of spurious prospective associations predicted either effects close to zero (if the concurrent correlation between self-concept and achievement was approximately equally strong as the cross-lagged correlations) or negative effects (if the concurrent correlation was stronger than the cross-lagged correlations). The predictions of a hypothesis of true increasing effects and of spurious prospective effects due to correlations with residuals and regression toward the mean are summarized in Table 1.

E | β_{X 1, Y 2 - Y 1} | = \frac{r_{X 1 Y 2} - r_{X 1 Y 1}}{\sqrt{2 (1 - r_{Y 1 Y 2})}}

(2)

Table 1.

Predicted Sign of Effects Between Academic Self-Concept and Achievement According to a Hypothesis of True Increasing Effects and a Hypothesis of Spurious Prospective Effects.

Effect	True	Spurious
β(s1,a2.a1)	Positive	Positive
β(a1,s2.s1)	Positive	Positive
β(s1,a1.a2)	Negative	Positive
β(a1,s1.s2)	Negative	Positive
β(s1,a2−a1)	Positive	Zero or negative
β(a1,s2−s1)	Positive	Zero or negative

Note. s = academic self-concept; a = academic achievement; 1 = time 1; 2 = time 2; the variables are given in the order predictor, outcome, and covariate.

Employing R 4.1.3 statistical software (R Core Team, 2022) and the metafor package (Viechtbauer, 2010), we conducted six separate multilevel random effects meta-analyses (one for each of the effects in Table 1). Effects from the same study were aggregated with a multilevel analysis. Then, a random meta-analytic effect, with a 95% confidence interval, was calculated across the independent effect sizes. Analyses were conducted on Fisher’s z-transformed standardized regression effects, but these were inverted back to non-transformed effects for the presentations. Data, a list of studies included in the meta-analyses, forest-plots, and script (also for the simulation) are available at the Open Science Framework at https://osf.io/3u79p/.

Simulation

A simulation was conducted to evaluate if prospective effects between academic self-concept and achievement are compatible with a situation where data are generated without any true prospective effects. The MASS package (Venables & Ripley, 2002) was used for the simulation and calculations were conducted with the lavaan package (Rosseel, 2012). The following steps were employed: (1) The six meta-analytic correlations between initial and subsequent academic self-concept and achievement were calculated; (2) Virtual participants (N = 10,000) were allocated scores on initial and subsequent self-concept and achievement, respectively, with the same correlations as calculated in the first step; (3) A SEM-model was defined, including: (i) a unidirectional effect of a latent achievement factor on a latent self-concept factor; (ii) effects of the latent achievement and self-concept factors on the two (initial and subsequent) observed achievement and self-concept scores, respectively; (iii) effects of two state factors on the two initial and the two subsequent scores, respectively. The factor loadings of the two achievement scores, and the two self-concept scores, were restricted to equality; (4) The SEM-model was fitted to the simulated dataset;

(5) Correlations between initial and subsequent achievement and self-concept scores were predicted by the fitted SEM-model; (6) A dataset (N = 10,000) with initial and subsequent achievement and self-concept scores was created, with the same correlations between variables as predicted in step five; (7) The effect of initial achievement on subsequent self-concept while adjusting for initial self-concept, and vice versa, was calculated in the dataset created in step six. It is important to note that nothing in the data generation implies prospective effects between self-concept and achievement, meaning that such effects would be spurious.

Results

Reanalysis of Meta-Analysis

Meta-analytic associations are presented in Table 2. All calculated associations exhibited low, and statistically non-significant, degree of heterogeneity, as estimated by Cochran’s Q and I², which estimates percentage of variation across effects attributable to heterogeneity rather than random variance. This suggested that all individual effects included in the meta-analyses can be assumed to have come from the same distribution.

Table 2.

Meta-Analytic Correlations and Adjusted Regression Effects Between Academic Self-Concept and Achievement Measured at Two Occasions.

Association	N	k	Estimate [95% CI]	Q (df)	I² [95% CI]
1. r(s1,s2)	61	65	0.538 [0.493, 0.579]	60 (60)	0 [0, 32.2]
2. r(s1,a1)	68	72	0.374 [0.334, 0.414]	65 (67)	0 [0, 28.6]
3. r(s1,a2)	62	66	0.340 [0.301, 0.378]	60 (61)	0 [0, 31.3]
4. r(s2,a1)	61	65	0.337 [0.298, 0.376]	58 (60)	0 [0, 29.9]
5. r(s2,a2)	53	56	0.414 [0.368, 0.457]	52 (52)	0 [0, 33.9]
6. r(a1,a2)	62	66	0.698 [0.665, 0.729]	61 (61)	0 [0, 32.3]
7. β(s1,a2.a1)	62	66	0.100 [0.077, 0.122]	58 (61)	0 [0, 28.6]
8. β(s1,a1.a2)	62	66	0.164 [0.135, 0.192]	58 (61)	0 [0, 27.3]
9. β(s1,a2−a1)	62	66	−0.046 [−0.066, −0.026]	60 (61)	0 [0, 31.3]
10. β(a1,s2.s1)	61	65	0.147 [0.121, 0.173]	58 (60)	0 [0, 29.3]
11. β(a1,s1.s2)	61	65	0.218 [0.190, 0.246]	57 (60)	0 [0, 27.6]
12. β(a1.s2−s1)	61	65	−0.046 [−0.070, −0.022]	58 (60)	0 [0, 30.0]

Note. N = number of studies; k = number of effects; Q = Cochran’s Q; I² = percentage of variation due to heterogeneity rather than randomness; s = academic self-concept; a = academic achievement; 1 = time 1; 2 = time 2; the variables are given in the order predictor, outcome, and covariate.

All correlations were positive and quite substantial (Table 2, rows 1–6). Initial academic self-concept was estimated to have (a) Positive effect on subsequent achievement when adjusting for initial achievement (row 7); (b) Positive effect on initial achievement when adjusting for subsequent achievement (row 8); (c) Weaker, but statistically significant, negative effect on subsequent change in achievement (row 9). These effects are illustrated in Figure 1. It is apparent that high initial academic self-concept predicted subsequent increase in achievement only if the effect was adjusted for initial achievement (panel a) but not if the effect was adjusted for subsequent achievement (panel b) nor if the effect of initial self-concept on subsequent change in achievement was calculated without adjustment for initial achievement (panel c). A similar description would apply to the effect of initial academic achievement on self-concept (Table 2, rows 10–12). The meta-analytic adjusted regressions effects (Table 2, rows 7–12) agreed better with a hypothesis of spurious prospective associations due to correlations with residuals and regression to the mean than with a hypothesis of true increasing prospective effects (see Table 1).

Figure 1.

Predicted standardized academic achievement at time 1 and 2 for individuals with low (Z = −1), average, and high (Z = 1) academic self-concept at time 1, when conditioning on average achievement at time 1 (a), when conditioning on average achievement at time 2 (b), and when not conditioning on achievement at time 1 (c).

Simulation

The model illustrated in Figure 2 exhibited near-perfect fit in a simulated dataset (N = 10,000) mimicking the meta-analytic correlations between initial and subsequent academic self-concept and achievement (Table 2, rows 1–6). In the model, initial and subsequent scores on self-concept and achievement were treated as indicators of latent self-concept and achievement factors, respectively. The factor loadings were satisfactory. Latent academic achievement had a fairly strong unidirectional effect on latent self-concept. Reflecting the somewhat weaker factor loadings, self-concept scores were more strongly affected by state factors compared with the achievement scores. It is important to note that nothing in the model suggested prospective effects between self-concept and achievement.

Figure 2.

A model where latent general academic achievement (gAA) has a unidirectional cross-sectional effect on latent general academic self-concept (gASC) and both latent variables have an effect on observed scores at two occasions.

A new simulated dataset (N = 10,000) was created, with the same correlations between initial and subsequent self-concept and achievement as predicted by the model in Figure 2. In this dataset, we calculated the effect of initial self-concept on subsequent achievement when adjusting for initial achievement, and vice versa. It is apparent that the correlations and prospective effects predicted by the model in Figure 2, without any true prospective effects, matched the meta-analytic associations very closely (compare the filled markers with the open markers beneath them in Figure 3). This suggests that no true prospective effects were necessary for observing the meta-analytic prospective effects, that is, they may be spurious, for example, due to correlations with residuals and regression to the mean.

Figure 3.

Meta-analytic associations (filled markers) and corresponding predicted associations based on the model in Figure 2 (open markers).

Discussion

The present reanalysis of meta-analytic findings indicated that depending on the chosen statistical model, one could claim either that academic self-concept had an increasing or a decreasing effect on subsequent change in academic achievement, and vice versa. This inconsistency suggests that the prospective effects may have been spurious and that claims based on the reality of such effects, for example, the reciprocal effects model (Guay et al., 2003; Marsh, 2022; Marsh & Craven, 2006; Marsh et al., 2002; Marsh & Martin, 2011; Niepel et al., 2014; Retelsdorf et al., 2014), can be called into question.

We propose that a trait-like (i.e., stable) academic achievement factor had a unidirectional cross-sectional effect on a trait-like academic self-concept factor, as illustrated in Figure 2. The trait-like factors had an effect on corresponding scores measured at two occasions. However, as the scores were measured with less than perfect reliability, we should expect correlations with residuals when adjusting for covariates. For example, among those with the same initial achievement score, those with a higher initial self-concept score should be expected to have a higher trait-like achievement compared with those with the same initial achievement score but with a lower initial self-concept score. This would mean that the former has experienced a negative and the latter a positive residual in the measurement of initial achievement. As residuals tend to regress toward a mean value of zero between measurements, those with a higher initial self-concept score should be expected to experience a more positive, but spurious, subsequent change in the achievement score compared with those with the same initial achievement score but with a lower initial self-concept score. A similar explanation, based on correlations with residuals and regression to the mean, could be applied to the positive effect of initial achievement score on subsequent change in self-concept score when adjusting for initial self-concept score. The simulation verified that prospective effects between self-concept and achievement scores while adjusting for a prior measurement of the outcome can be observed (see Figure 3) even if the data generating model/mechanism (see Figure 2) does not include any true prospective effects.

It is important to note that although we propose a positive unidirectional cross-sectional effect of trait-like academic achievement on trait-like academic self-concept, we do not propose an increasing prospective effect of measured achievement score on subsequent change in measured self-concept. Contrarily, this prospective effect was estimated to be negative (see row 12 in Table 2). This negative effect could, probably, be explained by a positive association between initial achievement and initial self-concept in combination with a negative association between initial self-concept and subsequent change in self-concept. Due to regression to the mean, a negative association between an initial value and subsequent change on the same variable is common. Consequently, initial achievement may have a negative effect on subsequent change in self-concept mediated by initial self-concept. In summary, we saw a positive effect of achievement on subsequent change in self-concept when adjusting for initial self-concept and a negative effect when not adjusting for initial self-concept and both of these effects may be spurious due to regression to the mean. Furthermore, as initial achievement had a stronger association with initial compared with subsequent self-concept (compare rows 2 and 4 in Table 2), according to equation (2) we should expect a negative association between initial achievement and subsequent change in self-concept. The difference in correlations could be due to influence by some occasion specific state factor, as modeled in Figure 2. The state factors could be, for example, illness and various home-related factors having a more or less temporary effect on achievement and self-concept.

The Extended Skill Development Model (ESDM)

In order to generalize the discussion above, we propose the “extended skill development model (ESDM).” This model makes the following predictions:

Trait-like characteristics, for example, academic achievement and long-jumping ability, have a cross-sectional effect on corresponding trait-like self-concepts, for example, academic self-concept and long-jumping self-concept.

A central difference between underlying characteristics and the corresponding self-concepts is that the latter are often measured with more subjective measures. For example, while academic achievement may be measured with grades or standardized test scores, academic self-concept might be measured with self-rated ability on a Likert-scale from 1 (very bad) to 7 (very good). Similarly, long-jumping ability could be measured by letting people jump and measure the length of their jump in centimeters while long-jumping self-concept, would it ever be measured in a study, might be measured as self-rated ability on a Likert-scale.

The strength of association between trait-like self-concepts and measured self-concepts, and consequently between measured characteristics and measured self-concepts, is sensitive to the measure used. For example, the association between long-jumping ability and long-jumping self-concept should be stronger if the latter is measured by asking people “How long can you jump, in centimeters?” (especially if they have just jumped and seen their result) compared with if they are asked “Are you good at long-jump?” with the response alternatives “Yes” and “No.”

The strength of the association between characteristics and self-concepts can differ between study populations. For example, the association between IQ and intelligence self-concept should be weaker among narcissists than in the general population. The association between academic achievement and self-concept should be stronger in a population with fresh experience of going to school than in a population with no experience of going to school.

Being more subjective, self-concepts are more affected by social comparisons than the underlying characteristics (Seaton et al., 2009). If a tall youth starts playing basketball and socializing with her equally tall team-mates, it may affect her tallness self-concept but not her actual height.

As already mentioned, although ESDM assumes a cross-sectional effect of underlying characteristics on corresponding self-concepts, it does not necessarily assume a true (non-spurious) prospective effect. Academic achievement, long-jumping ability, and height can have a positive effect on academic, long-jumping, and tallness self-concepts without any increasing effect on subsequent change in these self-concepts.

ESDM precludes causal effects of self-concepts on underlying characteristics. Academic self-concept is assumed to have no causal effect on academic achievement just like long-jumping and tallness self-concepts are assumed to have no causal effect on long-jumping ability and height.

ESDM does not preclude prospective effects between underlying characteristics and self-concepts, in either direction, when adjusting for a prior measurement of the outcome. However, prospective effects of self-concepts on subsequent change in characteristics are assumed to be spurious due to correlations with residuals and regression to the mean.

Both underlying characteristics and corresponding self-concepts may be affected by common state-factors. For example, academic achievement and self-concept may be affected by temporary depression; long-jumping ability and self-concept may be affected by an injured leg. An effect of a common state-factor would result in a stronger correlation between concurrent compared with cross-lagged measures of characteristics and self-concepts and, consequently (see equation (2)), in a negative correlation between measured characteristic and subsequent change in self-concept, and vice versa.

Model Comparison

The assumed effect of social comparisons on self-concepts provides an opportunity to pit the self-enhancement and reciprocal effects models against the skill development model and the ESDM. To recapitulate, the big-fish-little-pond effect states that social comparisons with high ability individuals decreases self-concept while social comparisons with low ability individuals increases self-concept (Fang et al., 2018; Koivuhovi et al., 2022; Marsh et al., 2008; Seaton et al., 2009). Moreover, the self-enhancement and reciprocal effects models claim that self-concepts have a positive effect on performance. Combining these effects leads to the conclusion that social comparisons with high ability individuals should have a negative effect, and social comparisons with low ability individuals a positive effect, on own performance (Seaton et al., 2009).

If the big-fish-little-pond effect and the self-enhancement or reciprocal effects model are correct, having low ability classmates should have a positive effect on an individual’s academic achievement while the effect of having high ability classmates should be negative. Similarly, having low and high ability teammates should have a positive and a negative impact on an individual athlete’s, for example, a swimmer’s, performance, respectively. The skill development model and the ESDM both predict that such effects should not be observed. Contrary to the predictions above, if anything, having high ability classmates appears to be associated with higher and having low ability classmates with lower own academic achievement (Balestra et al., 2021; Burke & Sass, 2013; Carrell et al., 2008; Granger-Serrano & Villarraga-Orjuela, 2021; B. Huang & Zhu, 2020; Lavy et al., 2012; Min et al., 2019; Sacerdote, 2011). This suggests that either the big-fish-little-pond effect or the self-enhancement and reciprocal effects models are false. We believe, in accordance with the ESDM, that the self-enhancement and reciprocal effects models are less well supported.

Implications and Contribution of the Present Study

The present study implies that prospective effects between academic self-concept and achievement, when adjusting for a prior value on the outcome, may be spurious. Consequently, the present study challenges the conclusion by Wu et al. (2021) that their meta-analysis provided evidence for the reciprocal effects model of academic self-concept and achievement. The present study contributes the novel “extended skill development model (ESDM),” which suggests that self-concepts are subjective measures of underlying characteristics. ESDM is better supported by data, indicating paradoxical simultaneous increasing and decreasing effects between academic self-concept and achievement, than the alternative self-enhancement and reciprocal effects models. Moreover, the present study also contributes to a growing body of research and methodological literature pointing out that cross-lagged effects while adjusting for a prior measurement of the outcome variable seldom prove anything over and above cross-sectional correlations combined with less than perfect reliability in measurements. The continued output of studies with uncritical employment of cross-lagged panel analyses indicates that knowledge of these limitations is lacking in the research community. We hope the present study may contribute to alleviate this lack of knowledge, at least to some degree.

Limitations

In the present paper, we have assumed a cross-sectional effect of trait-like academic achievement on trait-like academic self-concept as this effect has strong credibility on the face of it (Marsh, 2022; Marsh & Martin, 2011). However, statistically we cannot rule out a cross-sectional self-enhancement model where trait-like academic self-concept has a unidirectional effect on trait-like academic achievement, rather than the other way round, or a model where both trait-like academic achievement and trait-like academic self-concept are affected, that is, confounded, by some third variable. The confounding variable could be, for example, cognitive ability, which has an association both with academic self-concept and achievement (Deary et al., 2007; Kriegbaum et al., 2018; Roth et al., 2015). Both the cross-sectional self-enhancement model and a “full confounding” model would exhibit close fit if applied to simulated data similarly as the present model with a cross-sectional unidirectional effect of achievement on self-concept (see Figure 2). On the available evidence, we can simply not be certain about the true data generating model/mechanism.

Conclusions

Based on the present reanalysis of meta-analyses and simulation, we claim that prospective effects between academic self-concept and achievement, when adjusting for a prior value on the outcome, may be spurious, probably due to correlations with residuals and regression to the mean. We also propose the “extended skill development model (ESDM)” as an alternative to the self-enhancement and reciprocal effects models. According to the ESDM, self-concepts are subjective measures of underlying characteristics. The ESDM assumes that the underlying characteristics usually have a cross-sectional (but not necessarily a prospective) effect on the corresponding self-concept, although the strength of this effect can vary depending on used instruments, study population, etc. Conversely, the ESDM rules out causal effects of self-concepts on the underlying characteristics.

Footnotes

Acknowledgements

We thank our colleague Love Ahnström for verifying the coding of variables, analytical code, and results.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Kimmo Sorjonen

Data Availability Statement

Data, a list of studies included in the meta-analyses, forest-plots, and script (also for the simulation) are available at the Open Science Framework at

References

Balestra

Sallin

Wolter

S. C.

(2021). High-ability influencers? The heterogeneous effects of gifted classmates. Journal of Human Resources, 58(2), 633–665. https://doi.org/10.3368/jhr.58.4.0920-11170R1

Bandura

(1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. https://doi.org/10.1037/0033-295X.84.2.191

Burke

M. A.

Sass

T. R.

(2013). Classroom peer effects and student achievement. Journal of Labor Economics, 31(1), 51–82. https://doi.org/10.1086/666653

Calsyn

R. J.

Kenny

D. A.

(1977). Self-concept of ability and perceived evaluation of others: Cause or effect of academic achievement? Journal of Educational Psychology, 69(2), 136–145. https://doi.org/10.1037/0022-0663.69.2.136

Carrell

Fullerton

West

(2008). Does your cohort matter? Measuring peer effects in college achievement. Journal of Labor Economics, 27(3), 439–464. https://doi.org/10.3386/w14032

Castro-Schilo

Grimm

K. J.

(2018). Using residualized change versus difference scores for longitudinal research. Journal of Social and Personal Relationships, 35(1), 32–58. https://doi.org/10.1177/0265407517718387

Cohen

West

S. G.

Aiken

L. S.

(2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Lawrence Erlbaum Associates.

Deary

I. J.

Strand

Smith

Fernandes

(2007). Intelligence and educational achievement. Intelligence, 35(1), 13–21. https://doi.org/10.1016/j.intell.2006.02.001

Eriksson

Häggström

(2014). Lord’s paradox in a continuous setting and a regression artifact in numerical cognition research. PLoS One, 9(4), e95949. https://doi.org/10.1371/journal.pone.0095949

10.

Fang

Huang

Zhang

Huang

Yuan

(2018). The big-fish-little-pond effect on academic self-concept: A meta-analysis. Frontiers in Psychology, 9, 1569. https://doi.org/10.3389/fpsyg.2018.01569

11.

Ganley

C. M.

Lubienski

S. T.

(2016). Mathematics confidence, interest, and performance: Examining gender patterns and reciprocal relations. Learning and Individual Differences, 47, 182–193. https://doi.org/10.1016/j.lindif.2016.01.002

12.

Glymour

M. M.

Weuve

Berkman

L. F.

Kawachi

Robins

J. M.

(2005). When is baseline adjustment useful in analyses of change? An example with education and cognitive change. American Journal of Epidemiology, 162(3), 267–278. https://doi.org/10.1093/aje/kwi187

13.

Gorges

Neumann

Wild

Stranghöner

Lütje-Klose

(2018). Reciprocal effects between self-concept of ability and performance: A longitudinal study of children with learning disabilities in inclusive versus exclusive elementary education. Learning and Individual Differences, 61, 11–20. https://doi.org/10.1016/j.lindif.2017.11.005

14.

Granger-Serrano

Villarraga-Orjuela

(2021). Peer effects on first-year university students’ results: The role of classmates’ academic performance and socioeconomic status. Mathematics, 9(23), 3115. https://doi.org/10.3390/math9233115

15.

Guay

Marsh

H. W.

Boivin

(2003). Academic self-concept and academic achievement: Developmental perspectives on their causal ordering. Journal of Educational Psychology, 95(1), 124–136. https://doi.org/10.1037/0022-0663.95.1.124

16.

Guilford

J. P.

(1965). Fundamental statistics in psychology and education (4th ed.). McGraw-Hill.

17.

Helmke

van Aken

M. A. G.

(1995). The causal ordering of academic achievement and self-concept of ability during elementary school: A longitudinal study. Journal of Educational Psychology, 87(4), 624–637. https://doi.org/10.1037/0022-0663.87.4.624

18.

Huang

Zhu

(2020). Peer effects of low-ability students in the classroom: Evidence from China’s middle schools. Journal of Population Economics, 33(4), 1343–1380. https://doi.org/10.1007/s00148-020-00780-8

19.

Huang

(2011). Self-concept and academic achievement: A meta-analysis of longitudinal relations. Journal of School Psychology, 49(5), 505–528. https://doi.org/10.1016/j.jsp.2011.07.001

20.

Kenrick

D. T.

Neuberg

S. L.

Cialdini

R. B.

(2007). Social psychology: Goals in interaction (4th ed.). Pearson.

21.

Koivuhovi

Marsh

H. W.

Dicke

Sahdra

Guo

Parker

P. D.

Vainikainen

M.-P.

(2022). Academic self-concept formation and peer-group contagion: Development of the big-fish-little-pond effect in primary-school classrooms and peer groups. Journal of Educational Psychology, 114(1), 198–213. https://doi.org/10.1037/edu0000554

22.

Kriegbaum

Becker

Spinath

(2018). The relative importance of intelligence and motivation as predictors of school achievement: A meta-analysis. Educational Research Review, 25, 120–148. https://doi.org/10.1016/j.edurev.2018.10.001

23.

Lavy

Silva

Weinhardt

(2012). The good, the bad, and the average: Evidence on ability peer effects in schools. Journal of Labor Economics, 30(2), 367–414. https://doi.org/10.1086/663592

24.

Marsh

H. W.

(2022). Extending the reciprocal effects model of math self-concept and achievement: Long-term implications for end-of-high-school, age-26 outcomes, and long-term expectations. Journal of Educational Psychology, 115(2), 193. https://doi.org/10.1037/edu0000750

25.

Marsh

H. W.

Craven

R. G.

(2006). Reciprocal effects of self-concept and performance from a multidimensional perspective: Beyond seductive pleasure and unidimensional perspectives. Perspectives on Psychological Science, 1(2), 133–163. https://doi.org/10.1111/j.1745-6916.2006.00010.x

26.

Marsh

H. W.

Hau

K.-T.

Kong

C.-K.

(2002). Multilevel causal ordering of academic self-concept and achievement: Influence of language of instruction (English compared with Chinese) for Hong Kong students. American Educational Research Journal, 39(3), 727–763. https://doi.org/10.3102/00028312039003727

27.

Marsh

H. W.

Martin

A. J.

(2011). Academic self-concept and academic achievement: Relations and causal ordering. British Journal of Educational Psychology, 81(1), 59–77. https://doi.org/10.1348/000709910X503501

28.

Marsh

H. W.

Seaton

Trautwein

Lüdtke

Hau

K. T.

O’Mara

A. J.

Craven

R. G.

(2008). The big-fish–little-pond-effect stands up to critical scrutiny: Implications for theory, methodology, and future research. Educational Psychology Review, 20(3), 319–350. https://doi.org/10.1007/s10648-008-9075-6

29.

Min

Yuan

Wang

Hou

(2019). Do peer effects influence the academic performance of rural students at private migrant schools in China? China Economic Review, 54, 418–433. https://doi.org/10.1016/j.chieco.2019.02.004

30.

Möller

Zitzmann

Helm

Machts

Wolff

(2020). A meta-analysis of relations between achievement and self-concept. Review of Educational Research, 90(3), 376–419. https://doi.org/10.3102/0034654320919354

31.

Newman

R. S.

(1984). Children’s achievement and self-evaluations in mathematics: A longitudinal study. Journal of Educational Psychology, 76(5), 857–873. https://doi.org/10.1037/0022-0663.76.5.857

32.

Niepel

Brunner

Preckel

(2014). Achievement goals, academic self-concept, and school grades in mathematics: Longitudinal reciprocal relations in above average ability secondary school students. Contemporary Educational Psychology, 39(4), 301–313. https://doi.org/10.1016/j.cedpsych.2014.07.002

33.

R Core Team. (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/

34.

Retelsdorf

Köller

Möller

(2014). Reading achievement and reading self-concept—Testing the reciprocal effects model. Learning and Instruction, 29, 21–30. https://doi.org/10.1016/j.learninstruc.2013.07.004

35.

Rosseel

(2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://www.jstatsoft.org/v48/i02/

36.

Roth

Becker

Romeyke

Schäfer

Domnick

Spinath

F. M.

(2015). Intelligence and school grades: A meta-analysis. Intelligence, 53, 118–137. https://doi.org/10.1016/j.intell.2015.09.002

37.

Sacerdote

(2011). Peer effects in education: How might they work, how big are they and how much do we know thus far? In Hanushek

E. A.

Machin

Woessmann

(Eds.), Handbook of the economics of education (Vol. 3, pp. 249–277). Elsevier. https://doi.org/10.1016/B978-0-444-53429-3.00004-1

38.

Seaton

Marsh

H. W.

Craven

R. G.

(2009). Earning its place as a pan-human theory: Universality of the big-fish-little-pond effect across 41 culturally and economically diverse countries. Journal of Educational Psychology, 101(2), 403–419. https://doi.org/10.1037/a0013838

39.

Sewasew

Schroeders

Schiefer

I. M.

Weirich

Artelt

(2018). Development of sex differences in math achievement, self-concept, and interest from grade 5 to 7. Contemporary Educational Psychology, 54, 55–65. https://doi.org/10.1016/j.cedpsych.2018.05.003

40.

Shavelson

R. J.

Bolus

(1982). Self concept: The interplay of theory and methods. Journal of Educational Psychology, 74(1), 3–17. https://doi.org/10.1037/0022-0663.74.1.3

41.

Shavelson

R. J.

Hubner

J. J.

Stanton

G. C.

(1976). Self-concept: Validation of construct interpretations. Review of Educational Research, 46(3), 407–441. https://doi.org/10.3102/00346543046003407

42.

Skaalvik

E. M.

Valås

(1999). Relations among achievement, self-concept, and motivation in mathematics and language arts: A longitudinal study. The Journal of Experimental Education, 67(2), 135–149. https://doi.org/10.1080/00220979909598349

43.

Sorjonen

Ingre

Melin

Nilsonne

(2023). Unmasking artifactual links: A reanalysis reveals no direct causal relationship between self-esteem and quality of social relations. Heliyon, 9(10), e20397. https://doi.org/10.1016/j.heliyon.2023.e20397

44.

Sorjonen

Melin

(2023). Prospective associations between social support and posttraumatic stress disorder may be spurious: A re-meta-analysis of longitudinal studies and a simulation. Journal of Affective Disorders, 340, 174–180. https://doi.org/10.1016/j.jad.2023.08.014

45.

Sorjonen

Melin

(2024a). Prospective effects of mindfulness on anxiety and depressive symptoms may be spurious: Simulated reanalysis of a meta-analytic cross-lagged panel analysis. PLOS ONE, 19(5), e0302141. https://doi.org/10.1371/journal.pone.0302141

46.

Sorjonen

Melin

(2024b). Spurious prospective effects between general and domain-specific self-esteem: A reanalysis of a meta-analysis of longitudinal studies. PLOS ONE, 19(2), e0298158. https://doi.org/10.1371/journal.pone.0298158

47.

Sorjonen

Melin

Ingre

(2019). Predicting the effect of a predictor when controlling for baseline. Educational and Psychological Measurement, 79(4), 688–698. https://doi.org/10.1177/0013164418822112

48.

Sorjonen

Melin

(2023). Spurious prospective effects between work experiences and self-esteem: A reanalysis of a meta-analysis of longitudinal studies. European Journal of Personality, 08902070231197876. https://doi.org/10.1177/08902070231197876

49.

Sorjonen

Nilsonne

Melin

(2023). Distorted meta-analytic findings on peer influence: A reanalysis. Heliyon, 9(11), e21458. https://doi.org/10.1016/j.heliyon.2023.e21458

50.

Valentine

J. C.

DuBois

D. L.

Cooper

(2004). The relation between self-beliefs and academic achievement: A meta-analytic review. Educational Psychologist, 39(2), 111–133. https://doi.org/10.1207/s15326985ep3902_3

51.

Venables

W. N.

Ripley

B. D.

(2002). Modern applied statistics with S (4th ed.). Springer.

52.

Viechtbauer

(2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. https://doi.org/10.18637/jss.v036.i03

53.

Wolff

Möller

(2022). An individual participant data meta-analysis of the joint effects of social, dimensional, and temporal comparisons on students’ academic self-concepts. Educational Psychology Review, 34(4), 2569–2608. https://doi.org/10.1007/s10648-022-09686-1

54.

Guo

Yang

Zhao

Guo

(2021). A meta-analysis of the longitudinal relationship between academic self-concept and academic achievement. Educational Psychology Review, 33(4), 1749–1778. https://doi.org/10.1007/s10648-021-09600-1

Questioning the Reciprocal Effects Model of Academic Self-Concept and Achievement: A Reanalysis of a Meta-Analysis of Longitudinal Studies and a Simulation

Abstract

Plain Language Summary

Keywords

Introduction

Academic Self-Concept and Achievement

Prospective Effects May Be Spurious

Research Context

Objectives and Research Questions

Method

Reanalysis of Meta-Analysis

Simulation

Results

Reanalysis of Meta-Analysis

Simulation

Discussion

The Extended Skill Development Model (ESDM)

Model Comparison

Implications and Contribution of the Present Study

Limitations

Conclusions

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

ORCID iD

Data Availability Statement

References