Abstract
Drawing on expectancy-value theory, the present study examined the unique contributions of the four major value beliefs and self-concept on achievement, self-reported effort, and teacher-rated behavioral engagement in mathematics. In particular, we examined the multiplicative effects of self-concept and task values on educational outcomes using the latent moderated structural equation approach. Participants were 1,868 German ninth-grade students. The data analyses relied on a higher-order structure of value beliefs, which is suited to parsing the differential patterns of predictive relations for different value beliefs. The findings revealed that (a) self-concept was more predictive of achievement, whereas value beliefs were more predictive of self-rated effort; (b) self-concept and value beliefs emerged as equally important predictors of teacher-reported engagement; (c) among the four value beliefs, achievement was more associated with low cost, whereas effort was more associated with attainment value; and (d) latent interactions between self-concept and value beliefs predicted the three outcomes synergistically.
In addition to their first-order effects, competence beliefs and value beliefs are assumed to interact with each other in influencing achievement-related behaviors and choices (see Atkinson, 1957; Atkinson & Feather, 1966; Feather, 1982; Vroom, 1964). In other words, the interactive associations suggest that the relation between competence beliefs and outcomes depends on the extent to which an individual values a given domain and vice versa. However, empirical research examining interaction effects of motivational beliefs on achievement-related behaviors in nonexperimental settings is surprisingly sparse (for exceptions, see Guo, Parker, Marsh, & Morin, 2015; Nagengast et al., 2011; Trautwein et al., 2012). One of the reasons for this sparsity has been the error-prone specification of interaction effects in latent variable models that account for measurement error (e.g., Bollen, 1996; Jöreskog & Yang, 1996; Kenny & Judd, 1984). In recent years, less-complicated specifications have been published (Marsh, Wen, & Hau, 2004), and new approaches (e.g., Klein & Moosbrugger, 2000; Kelava & Nagengast, 2012; Kelava, Nagengast, & Brandt, 2014) have become available with standard latent variable modeling software (e.g., Mplus; Muthén & Muthén, 1998-2014).
In this study, we draw on the framework of modern EVT (Eccles, 2009), using a large sample of high school students in Germany, to investigate predictive relationships between math motivational beliefs and three achievement-related outcomes: math achievement, self-reported math effort, and teacher-rated behavioral engagement. Of central importance, the present study captured the multidimensional nature of task values (Eccles et al., 1983; Eccles & Wigfield, 2002) to explore the unique predictive power of the four math value components, along with self-concept, on the educational outcomes. The interactive roles of self-concept and value beliefs were also examined in order to address this gap in the literature. In particular, the use of non-self-rated variables has received scant attention in research on expectancy-by-value interactions. Finally, by juxtaposing the recent literature and the results of the present investigation, we provide a more complete evaluation of the nature of expectancy-by-value interactions in support of EVT.
EVT
The modern EVT (Eccles, 2009; Eccles et al., 1983) posits that achievement-related performance and choices are most directly influenced by an individual’s expectations of academic success and a subjective assessment of the inherent value of academic tasks. Modern EVT (Eccles et al., 1983) defines expectancies of success as task-specific beliefs about the possibility of experiencing future success in that task, which is assumed to be mainly influenced by a person’s beliefs about her or his abilities (i.e., ability self-concepts; Marsh, 1986, 2007). However, Eccles (2009) states, “Empirically, we have found that ability self-concepts are so directly linked to expectations for success that it is quite difficult to distinguish between these two constructs” (p. 82). Similarly, in their review of competence self-perceptions more generally, Schunk and Pajares (2005) also emphasize that expectancy-value theorists have concluded that expectations of success and academic self-concept are not empirically separable. This has led to the routine use of academic self-concept in recent EVT studies (e.g., Musu-Gillette, Wigfield, Harring, & Eccles, 2015; Simpkins, Fredricks, & Eccles, 2012; Wang & Eccles, 2013; Wang, Eccles, & Kenny, 2013) as a measure of expectancies of success, particularly so with those examining expectancy-by-value interaction (e.g., Nagengast et al., 2011; Trautwein et al., 2012; Guo, Parker, et al., 2015). Following this tradition, academic self-concept was used in this research to measure expectancies of success.
Modern EVT distinguishes between multiple components of value (Wigfield & Eccles, 1992; Eccles & Wigfield, 2002): Intrinsic value refers to the extent to which the person gains enjoyment from performing an activity. Attainment value is the degree of importance attached to successful performance of a specific task and has been also linked to relevance of a task to one’s personal and social identities (Eccles, 2009, 2011). Utility value is the degree of usefulness that a specific task has for the individual. Cost includes the degree of potential loss of time; effort demands; the loss of valued alternatives, such as spending time with friends; or additional negative experiences, such as stress. Cost is the least-studied component of task value.
Recently, evidence has emerged that the four value components can be empirically differentiated in the math domain (Conley, 2012; Luttrell et al., 2010; Trautwein et al., 2012). These studies found a similar correlation pattern among the value components, with the highest correlations being between intrinsic and attainment value. It has been well documented that correlations between academic self-concept and the value components are usually moderate to large in size (see Wigfield & Eccles, 2002; Wigfield, Tonks, & Klauda, 2009, for reviews). In particular, self-concept is more highly correlated with intrinsic value than other value components within a specific domain (Wigfield et al., 2009). Thus, it is imperative to differentiate and consider all value components along with self-concept in one regression model, which allows us to further disentangle the interactive relationships between self-concept and value beliefs in predicting achievement-related outcomes (see subsequent discussion).
Association of Self-Concept, Task Value, and Achievement-Related Behaviors
An extensive body of EVT research has demonstrated that self-concept is more closely associated with academic achievement than is task value, whereas task value is generally a stronger predictor of course-taking decisions (e.g., Eccles, Barber, & Jozefowicz, 1999; Perez, Cromley, & Kaplan, 2014; Watt, Eccles, & Durik, 2006), academic engagement and effort (e.g., Cole, Bergin, & Whittaker, 2008; Trautwein & Lüdtke, 2009; Wang & Eccles, 2013), and educational and career aspirations (e.g., Simpkins, Davis-Kean, & Eccles, 2006; Watt et al., 2012). However, most of this research has focused predominantly on a single value construct measured by a small number of items or only on one or two of the expected components of value. Utility value and attainment value have often been combined as importance value (Durik, Vida, & Eccles, 2006; Jacobs, Lanza, Osgood, Eccles, & Wigfield, 2002; Watt et al., 2012). For example, Watt et al. (2012) found that importance value was more predictive of educational aspirations than was intrinsic value, whereas intrinsic value more strongly predicted math participation than did importance value, controlling for self-concept. Of particular relevance, no previous studies have simultaneously considered all four components of value, along with self-concept, in the same regression model, although EVT (Eccles, 2009) emphasizes that different value components should play differential roles in influencing educational outcomes.
However, Wigfield and Eccles (2000), along with many others (e.g., Wigfield & Cambria, 2010), have acknowledged that overlapping elements among task values might exist. Indeed, an apparent problem in previous research has been that the four value components have been so highly correlated that the resulting multicollinearity has made it difficult to identify the separate and unique contribution of each value component. Thus, previous studies of the multiple value components have conducted separate analyses of each value component, rather than considering them simultaneously in a single model (e.g., Trautwein et al., 2012). Recognizing this as a limitation in most previous research, the challenge for us was to resolve this problem so that the four value components could be considered together in the same model. In an apparent resolution of this issue, we applied an innovative higher-order bi-factor model that is specifically designed to capture the multidimensional nature of task value to test the unique contribution of value components to students’ academic achievement, behavior engagement, and effort (see subsequent discussion).
The Multiplicative Relation Between Expectancy and Value
Although Eccles (2009) suggested that “the motivational power of ability self-concepts to influence task choice is, at least partially, determined by the value individuals attach to engaging in the domain” (p. 84), the multiplicative relation between expectancies for success and task values, which was the core assumption of classic EVT (Atkinson, 1957; also see Feather, 1982; Vroom, 1967), has not been widely examined. In modern EVT (Eccles, 2009; Eccles et al., 1983), the effects of self-concept and value are often implicitly assumed to be additive, which would suggest that self-concept and task value predict achievement-related behaviors uniquely and independently. A multiplicative relation, on the other hand, would imply that the effect of self-concept on outcomes depends on the extent to which an individual values a given domain and vice versa.
Typically, an interaction between two independent predictors (i.e., self-concept and task value) has been described as having either a compensatory or a synergistic relation to the outcome. The nature of the interactions in relation to the two taxonomies is considerably different; this has theoretical and substantive implications for motivation researchers. Specifically, a compensatory relation suggests that as long as individuals have high expectancy or high value attached to a given academic task, they will be motivated to engage in it. In other words, high expectancy can compensate for low value and vice versa. In contrast, a synergistic relation would suggest that either high expectancy or high value alone is not sufficient to motivate behaviors. Rather, individuals must have both high self-concept and high value to engage in a given academic task. More specifically, recent studies of expectancy-by-value interactions (Nagengast et al., 2011; Trautwein et al., 2012) have argued that support for EVT implies a synergistic expectancy-by-value interaction, suggesting that compensatory interaction might not support EVT.
The omission of the multiplicative relation in modern EVT may be partly due to the shift from experimental designs focusing on intraindividual differences to real-world settings focusing on interindividual differences (for further discussion, see Nagengast et al., 2011; Trautwein et al., 2012). Methodologically, it is difficult to detect interaction effects in nonexperimental designs (Marsh et al., 2004; also see Appendix A in the supplemental materials). However, recently, researchers have been able to examine interaction effects using structural equation modeling (SEM; Bollen, 1989) techniques, such as the latent moderated structural equation approach (LMS; Klein & Moosbrugger, 2000) and the unconstrained product indicator approach (Marsh et al., 2004), in which the measurement error of the predictor variables is accounted for (for an overview, see Schumacker & Marcoulides, 1998).
On the basis of these recent approaches, there is now some recent empirical support for a synergistic relation between expectancy and task value in predicting educational outcomes. For example, Nagengast et al. (2011) found that science self-concept, intrinsic value, and their interaction significantly positively predicted engagement in science extracurricular activities and intentions to pursue a scientific career. Importantly, the pattern of results was similar across 57 countries in the Programme for International Student Assessment 2006 data (Nagengast et al., 2011). In addition, on the basis of a nationally representative sample of Australian youth, Guo, Parker, et al. (2015) reported that the interactions between high school mathematics self-concept and value significantly predicted mathematics course selection; matriculation results; subsequent science, technology, engineering, and mathematics [STEM] major choices; and entry into university when value components (intrinsic value and utility value) are considered separately (also see Guo, Marsh, Parker, Morin, & Yeung, 2015; Nagengast, Trautwein, Kelava, & Lüdtke, 2013; Trautwein et al., 2012). However, when the model included both value components and their interactions with self-concept, only the interaction between self-concept and intrinsic value was found to predict the outcomes significantly.
Although these empirical studies successfully reintroduced the multiplicative relation between expectancy and value in motivation research, three important limitations need to be addressed. First, as discussed above, the multidimensional nature of task value has not been fully taken into account in previous studies, particularly in those with expectancy-by-value interaction.
Second, little is known about whether self-concept and task value interact in predicting academic effort and behavioral engagement, particularly in a classroom setting; these are important determinants of academic success (Wang & Degol, 2014). Students’ effort in learning tasks is highly correlated with their behavioral engagement in classroom and is usually treated as a part of measures of engagement (e.g., Furrer, Skinner, Marchand, & Kindermann, 2006; Skinner, Kindermann, & Furrer, 2008; Skinner, Zimmer-Gembeck, & Connell, 1998). Students’ behavioral engagement is also determined by their attention, self-direction, and persistence in learning activities (Furrer et al., 2006; Skinner et al., 1998, 2008).
Most empirical studies investigating how motivational beliefs relate to academic effort and engagement have relied heavily on student self-report measures (e.g., Trautwein & Lüdtke, 2009; Wang, 2012; Wang & Eccles, 2013). Monitoring the extent to which students are engaged with and make an effort in learning activities is important for teachers in order to provide constructive feedback in the classroom. However, teacher perceptions of student engagement and effort might differ from those of their students. In previous research, the correlation between self-reported and teacher-rated engagement was found to be moderate (average r = .30–.35; Lee & Reeve, 2012; Skinner et al., 2008). Collecting information from teachers can provide an alternative and important perspective on student engagement and effort. To date, little EVT research has simultaneously considered multiple informants (i.e., student as well as teacher reports) with respect to engagement or effort and has examined associations between motivation beliefs and outcomes. Therefore, in this study, we fill this gap in the literature by exploring the interactive relations between math self-concept and all value components in predicting student self-reported effort and teacher-rated engagement.
Third, insufficient attention has been given to the nature of first-order effects (“main” effects of self-concept and value) and interactions (self-concept by value) in support of EVT predictions. Although positive interaction effects indicate synergistic relations, and negative interaction effects indicate compensatory relations, the interpretation of the results in relation to EVT depends fundamentally on the combination of first-order and interaction effects. In particular, superficial interpretations of interaction effects that do not also take into account the size and nature of the first-order effects can be misleading. Rather, interpretation of interaction effects should always be based on a graph of the results in relation to a priori predictions. In this study, we provide a more complete evaluation of the nature of multiplicative relations in support of EVT, showing that compensatory interactions are not necessarily inconsistent with EVT predictions, whereas synergistic interactions are not necessarily consistent with EVT predictions (see subsequent discussion).
The Present Study
Drawing on EVT, we operationalize math subjective task value as a multidimensional construct to examine self-concept, the four value components, and their interactions in predicting three math-related outcomes: objective achievement, self-reported effort, and teacher-rated behavioral engagement. The present study is unique in that it simultaneously includes the four latent value components in the latent SEM to explore the unique contribution of each value component to the prediction of achievement-related outcomes by integrating a second-order model and a bi-factor model.
This integration allows us to extend past research on the application of modern EVT and leads to the following research hypotheses:
Hypothesis 1: We examined whether student self-concept and the four value components predict the three outcomes differentially. Generally, we expected that self-concept would be a stronger predictor of academic achievement, whereas task value would be more predictive of self-reported effort and teacher-rated engagement (e.g., Eccles & Wigfield, 2002). However, specific hypotheses about which value components play more important roles in promoting student’ academic effort and engagement are lacking in the EVT literature. Theoretically, intrinsic value and, perhaps, cost are the most closely tied to effort and engagement. When students value an activity intrinsically, they often become deeply engaged in it and can persist at it for a long time (Wigfield & Cambria, 2010). Perceived negative aspects of engaging in a specific task (i.e., anticipated effort, time, and energy) might also be directly associated with students’ exertion of effort and engagement (Barron & Hulleman, 2015; Flake, Barron, Hulleman, McCoach, & Welsh, 2015). Thus, we expect intrinsic value and cost would make unique contributions to the prediction of self-reported effort and teacher-rated engagement, after controlling for self-concept and other value components.
Hypothesis 2: Of particular importance to the investigation, we expect a synergistic relation between self-concept and value in predicting the outcomes (e.g., Guo, Parker, et al., 2015; Nagengast et al., 2011). Importantly, we also provide a more complete evaluation of the nature of multiplicative relations in support of EVT by juxtaposing the recent literature and the results of the present investigation.
Method
Participants
The data set used in the present study (see Gaspard et al., 2015) is part of the larger Motivation in Mathematics (MoMa) project. The current study’s sample was drawn from ninth-grade high school students from 82 classes in 25 academic track schools (Gymnasium schools) in the German state of Baden-Württemberg in 2012. A total of 1,978 students who had active parental consent participated in the study (53.5% female; age, M = 14.62). The questionnaires were administered to the students in class by trained research assistants.
Measures
Students’ motivational beliefs were measured through student ratings with a 4-point Likert-type scale, systematically recoded so that higher values represented more favorable responses and, thus, higher levels of motivation. In particular, we assessed math-related value beliefs with an instrument developed to measure the multidimensional nature of task beliefs, based on the modern EVT model (Eccles et al., 1983).
Value components/facets
There is recent empirical support that subjective task value not only is defined by four components but could be further characterized by multiple facets within each major component (Trautwein et al., 2013). This is similar to the Big Five personality factor structure, in which each of the Big Five factors is represented by multiple facets and each facet in turn is represented by multiple items (Goldberg, 1992, 1999). But it is worth noting that these facets are merely a means to get at the Big Five factors (Costa & McCrae, 1995; Goldberg, 1992, 1999). Thus, in this study, 37 items were used to measure a total of 10 facets, which form the four value components (see Table 1 for descriptive statistics, sample items, and reliability of value scales).
Sample Items, ICCs, Reliabilities, and Factor Loadings of Motivational Beliefs and Outcome Scales
Note. ICC = intraclass correlation; Model SO-4V = first-order bi-factor for four value components; Model SO-B-4V = second-order bi-factor for four value components with 11 value facets.
Specifically, intrinsic value was measured by four items and attainment value by 10 items tapping two facets (importance of achievement and personal importance; Eccles, 2009; Wigfield & Eccles, 1992). Utility value consisted of 12 items assessing the utility of different life domains from a short-term (school, daily life, social life; Eccles et al., 1983; Hulleman & Harackiewicz, 2009) as well as from a long-term perspective (job, future life in general; Conley, 2012; Hulleman, Durik, Schweigert, & Harackiewicz, 2008). Cost was measured by 11 items tapping three facets (opportunity cost, effort required, and emotional cost; Perez et al., 2014; Wigfield & Eccles, 2002). For a detailed description of the scales and the total set of items, see Gaspard et al. (2015). All value items were measured with a 4-point Likert scale ranging from completely disagree to completely agree. Scale reliabilities for value facets were acceptable (see Table 1).
Self-concept
Math self-concept was assessed with five items (e.g., “I am good at math”; see Appendix B in the supplemental materials), each with a 4-point response format ranging from completely disagree to completely agree. All items were validated and came from the German adaptation (Schwanzer, Trautwein, Lüdtke, & Sydow, 2005) of the Self-Description Questionnaire III (Marsh et al., 2004) as well as from previous large-scale national studies (e.g., Marsh, Trautwein, Lüdtke, Köller, & Baumert, 2005). The reliability of this scale was acceptable (Cronbach’s α = .92).
Math achievement
A validated and comprehensive test developed by the statewide school quality assurance agency (Landesinstitut für Schulentwicklung) was utilized to measure math achievement. The math test is used to assess the quality development of schools on an empirically established, targeted, and systematic basis. To ensure reliable testing and evaluation, this instrument comprises a balance of closed, partially open, and open test item formats. The official test results reported by the schools were used to operationalize students’ math achievement.
Student self-reported effort
This scale consisted of six items measuring students’ effort in math class as well as on math tasks and homework (Organisation for Economic Co-operation and Development, 2003; e.g., “I work hard in math”; 1 = strongly disagree to 4 = strongly agree; see Appendix B in supplemental materials). Reliability of this scale was good (α = .81).
Teacher-rated engagement
This scale comprised two items measuring students’ classroom engagement (“This student participates in math lessons as well as he/she can”) and effort expenditure on homework (“This student works on all of his/her tasks and homework thoroughly”). We again used a Likert-type scale ranging from 1 = strongly disagree to 4 = strongly agree.
Statistical Analyses
In the present study, all data analyses, confirmatory factor analyses (CFAs), and SEMs were conducted with Mplus 7.11 (Muthén & Muthén, 1998-2014) using the robust maximum likelihood estimator. The LMS approach (Klein & Moosbrugger, 2000) was utilized to model the latent interactions between self-concept and task values in predicting the three outcomes. The advantage of the LMS approach is that it corrects for measurement error of latent constructs and provides unbiased estimates of latent interaction effects. Further, LMS represents non-normal distribution as a mixture of conditionally normal distributions; thus, separate indicators of the product terms (latent interaction) are not required (Kelava et al., 2011).
Four value components: Higher-order structure
As emphasized earlier, the multiple facets of the four value components are merely a means to an end. Although further research into specific outcomes might identify the important predictions specific to each of the 10 value facets, the focus in the present investigation is on the four value components originally posited by modern EVT (Eccles et al., 1983; Wigfield & Eccles, 2000). In this respect, our focus is on second-order factors representing the four value components rather than on the first-order factors representing the 10 value facets. Thus, a second-order model was employed to define the hierarchical representation of each value component from multiple dimensions of value facets tapped by multiple items (see Figure 1).
Second-order confirmatory factor analysis models presenting the four value components. All value facets are labeled by their acronym (see Table 1).
Bi-factor models provide a more flexible alternative, a way of capturing the hierarchical and multidimensional nature of task value (Chen, West, & Sousa, 2006; Reise, 2012). The assumption underlying the bi-factor models is that an f-factor solution exists for a set of n items with one global factor (G-factor) and f-1 domain-specific factor (S-factor); the total covariance is partitioned into a G-factor underlying all indicators and f-1 S-factors that reflect the residual covariance not explained by the G-factor (Gustafsson, & Balke, 1993; Holzinger & Swineford, 1937; Morin, Arens, & Marsh, 2015; Mulaik & Quartetti, 1997). This bi-factor specification is consistent with EVT, in which task values might overlap with each other to a certain degree, even though the four value components have emerged from different theoretical perspectives and can be defined separately (Eccles & Wigfield, 2002). These overlapping elements might reflect an overall sense of values students attach to various tasks. Furthermore, as discussed above, these overlapping elements might lead to high correlations among value components, which would make it difficult to isolate and detect the unique contribution of each value component. One of the key features of the bi-factor model is that the residual S-factors typically are specified as uncorrelated (orthogonal) to one another and with the G-factor (Chen et al., 2006). This makes the bi-factor model particularly useful for researchers to study the unique roles of a subset of S-factors in predicting external variables, over and above the general factors.
In this study, we integrated a second-order model and a bi-factor model. More specifically, we applied an innovative second-order bi-factor model that was uniquely suited not only to capture hierarchical and multidimensional features of task value but also to address the challenge of detecting the unique contribution of value components. As illustrated in Figure 1, in the second-order bi-factor model, the covariance among value items is attributable to three major sources: (a) a global (general) value factor representing the common variation shared by all 37 value items; (b) 10 domain-specific first-order factors based on value facets, which represent the unique variances represented by each facet that are independent of the global value factor; and (c) second-order value factors representing the four value components posited in EVT, which are the main focus of the present investigation. In this model, the relations of global task value to first-order value facets and second-order value components were assumed to be orthogonal; the second-order value components are directly represented as independent factors. Hence, this allows us to test whether each value component make a unique contribution to the prediction of the three outcomes, over and above the global value.
Model fit indices
The comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the Tucker-Lewis index (TLI) were used to determine the fit of CFA models. Values greater than .95 and .90 for CFI and TLI typically provide excellent and acceptable fits, respectively, to the data (Hu & Bentler, 1999). RMSEA values of less than .06 and .08 are considered to reflect good and acceptable statistical fits, respectively (Marsh et al., 2004). Nonetheless, these fit statistics are not available for the SEM models including latent interactions (Klein & Moosbrugger, 2000). Akaike information criteria (AIC) and Bayes information criteria (BIC) were utilized for model comparison in the present study (e.g., Kelava et al., 2011; Pek, Losardo, & Bauer, 2011). These indexes have the advantage that they not only consider how well a model fits the data but also reward more parsimonious models in contrast to more complex models in which many parameters are estimated. Smaller values of AIC and BIC indicate better fits to the data (Kelava et al., 2011).
Hierarchical data structure and missing data
The data set has a nested data structure in which students are nested within schools and classes. To account for this nested structure, we used the TYPE = COMPLEX with the CLUSTER and STRATIFICATION options in Mplus to adjust the standard errors. For the variables considered here, the percentage of missing data was low (2.9% at maximum). Full information maximum likelihood (FIML) estimation was used to cope with the missing data. In FIML, the parameters of a statistical model are estimated in the presence of missing data, and all of the information of the observed data is used to inform the parameters’ values and standard errors (Enders, 2010).
Results
In order to test the hierarchical and multidimensional nature of the value components, we employed two alternative models within the CFA framework: second-order models and bi-factor models. Following Gaspard et al. (2015), we began by evaluating a series of CFAs based on second-order models and examined intercorrelations among value components, self-concept, and outcome variables. Subsequently, we tested an innovative second-order bi-factor model that is uniquely suited to parsing the differential patterns of predictive relations for different value beliefs. Finally, a series of SEMs was conducted to explore the unique predictive power of self-concept, value components, and their interactions on math achievement, effort, and engagement.
Second-Order CFA
For each value (except for intrinsic value), the models differentiating value facets consistently yielded better fits to the data, thus providing good support for the dimensionality of value components (Models IV to CO2; see Table 2). To further assess the separability of value components, we evaluated high-order CFAs. The second-order model (Model SO-4V: CFI = .939, TLI = .934, RMSEA = .044; see Figure 1 and Appendix C in the supplemental materials; also see Gaspard et al., 2015), where the four value components were formed by 10 value facets, fitted the data much better than did the first-order four-factor models (Model FO-4V: CFI = .849, TLI = .838, RMSEA = .069). This finding demonstrates the differentiation of value components into distinct facets (see Gaspard et al., 2015, for further discussion).
Model Fit Statistics for the Hypothesized CFA Models
Note. CFA = confirmatory factor analysis; CFI = comparative fit index; TLI = Tucker-Lewis index; RMSEA = root mean square error of approximation.
The first-order factors representing the facets of attainment value, utility value, and cost were set to load on a second-order factor representing the corresponding value component. Given that there were no value facets for intrinsic value, it was still treated as a first-order factor and included in the second-order factor model (see Gaspard et al., 2015).
Correlations among value beliefs, self-concept, and outcomes
Based on Model FO-4V, latent correlations indicated that the four value components were moderately or highly correlated, ranging from .41 (utility value and low cost) to .77 (intrinsic value and low cost). Math self-concept was moderately correlated with math attainment value (r = .55) and utility value (r = .45) and more highly correlated with intrinsic value (r = .80) and low cost (r = .82; see Table 3).
Intercorrelations Among Self-Concept, Value Components, and Outcomes
Note. For presentational purpose, the low cost scale was reverse coded. The correlation coefficients reported above were based on the second-order confirmatory factor analysis model (Model SO-4V).
Correlations between motivational beliefs and the three outcomes were all statistically significant and positive (see Appendix D in the supplemental materials for correlations involving value facets). Specifically, achievement was more highly correlated with self-concept, intrinsic value, and low cost (r = .53, 46, and 42), and self-reported effort was more highly correlated with attainment value (r = .60). Correlations of teacher-rated engagement to motivational beliefs are somewhat smaller (r = .16 to .32, M = .24). In line with prior studies (Lee & Reeve, 2012; Skinner et al., 2008), the correlation between self-reported effort and teacher-rated engagement was moderate in size (r = .32), while both were significantly correlated with achievement (r = .18 and .36, respectively).
Second-Order Bi-Factor CFA
The second-order bi-factor CFA model (SO-B-4V; Table 2; also see Figure 1) posits one global value, 10 first-order value facet factors, and four second-order value factors. This model provided a better fit to the data (CFI = .955; TLI = .949; RMSEA = .039) than did second-order CFA model (Model SO-4V). 1 In Model SO-B-4V, the global value factor was well defined, with generally moderate loadings (|λ| = .19 to .85, M = .51; see Appendix D in the supplemental materials for more details). Beyond this G-factor, the specific first-order factors were also well defined, with largely moderate to strong item loadings (|λ| = .22 to .94, M = .57). The loadings on the second-order factors were substantial for value facets (|λ| = .39 to .99, M = .73), except for the social utility facet. In summary, Model SO-B-4V showed the four well-defined second-order value components along with a global value factor, providing good support for the hierarchical and multidimensional representation of task value as posited in EVT.
Four value components: Unique contributions to outcomes
What is the unique contribution of the four value components and the global value factor to the prediction of our three outcome variables? We tested the predictive effects of the four second-order value components, self-concept, and the global value as well as self-concept-by-value interactions on achievement, self-reported effort, and teacher-rated engagement. We began with the evaluation of a CFA model (Model B1) in which the second-order bi-factor structure of the value components was incorporated with self-concept and the three outcomes. This model fitted the data well (e.g., CFI = .942; TLI = .935). It should be noted that the bi-factor second-order model was applied only to the set of 37 items assessing all value facets. Thus, in Model B1, self-concept is allowed to correlate with the four value components as well as with the global value, whereas the value components are orthogonal to each other and to the global value. Next, we evaluated a series of SEM models (Models C1 through C4). In each of the models considered here, we included all variables, noting that a model with all variables simply correlated is equivalent (in terms of degrees of freedom and goodness of fit) to a model where some of the correlations are represented as path coefficients. Thus, for example, in a preliminary model (Model C1; see Table 4), relations among self-concept and the three outcomes were represented by paths, whereas all other relations were represented as correlations. In the subsequent model, additional correlations were represented by appropriate paths in the SEM. Using this approach, all the different models incorporated the same variables and resulted in the same model fit. This strategy had important advantages for the comparison of models based on different sets of variables that potentially confound aspects of the measurement and structural models (see Marsh et al., 2015, for further discussion).
Standardized Path Coefficients of Self-Concept, Value Components on Three Outcomes Based on the Second-Order Bi-Factor Model
p < .10. *p < .05. **p < .01. *** p < .001.
As seen in Table 4, self-concept substantially predicted self-rated effort, teacher-rated engagement, and in particular, academic achievement, without controlling for value beliefs (β = .32, .36, and .53, respectively; see Model C1). Model C2, in which the four value components were considered along with the global value, intrinsic value, low cost, and global values consistently predicted the three outcomes (β = .15 to .25, .08 to .26, and .23 to .38, respectively). Attainment value had positive predictive effects on engagement (β = .23) and, in particular, on effort (β = .58) but not on achievement. However, the predictive effects of utility value were nonsignificant for each of the outcomes considered here after controlling for the global value.
The sizes of the path coefficients involving self-concept were not altered when the four value components were also considered as predictors excluding global value (see Model C3). However, the predictive effects of intrinsic value and low cost became substantially smaller and even nonsignificant. Finally, in the extended SEM model (Model C4), we included predictive paths from global value to the three outcomes. The model results in similar patterns for achievement and engagement with Model C3. However, the predictive effect of self-concept on effort became nonsignificant. Instead, global value substantially predicted effort (β = .35) but not achievement and engagement.
In summary, self-concept was more predictive of achievement, whereas value beliefs were more predictive of self-rated effort. However, self-concept and value beliefs had similar predictive effects on teacher-rated engagement. More importantly, after partialing out the global value, the findings showed differential patterns of predictive relations to the three outcomes for the different value components. Math achievement was more associated with low cost, whereas self-rated effort was more associated with attainment value. Intrinsic value, attainment value, and low cost had uniquely predictive power on teacher-reported engagement. However, utility value did not make a unique contribution in predicting the three outcomes.
Predictive Effects of Self-Concept and Value Beliefs
To probe the interactive roles of self-concept and value beliefs, we first added the interaction between self-concept and global value along with all the predictive effects of the four value components (see Model D1 in Table 5). We found that the interaction model provided lower AIC and BIC than that without interaction (Model D1 vs. Model C1; ΔAIC = 211; ΔBIC = 227; Δadjust-BIC = 218). Both models showed similar patterns of path coefficients for the first-order effects. To enhance the presentation, we provide graphical depictions of the interaction effects (3-D response surface displays; Myers, Montgomery, & Anderson-Cook, 2009; see Figure 2) using the RSA package (Schönbrodt, 2015) in R (R Core Team, 2013). As is generally the case with interaction effects, researchers are encouraged to plot the interactions in order to better understand their nature. The type of 3-D plot presented here has the added advantage of showing a scatter plot, which allows researchers to evaluate the range of values under consideration. This is useful because the nature of the interaction might not be relevant for very extreme values outside of the range of values actually observed.
Standardized Path Coefficients of Self-Concept, Value Components, and Their Interactions on Three Outcomes Based on the Second-Order Bi-Factor Model
p < .10. *p < .05. **p < .01. *** p < .001.
Response surface displays for the multiplicative effect of self-concept and task value on the three outcomes.
The results showed that the interaction between self-concept and global value positively predicted achievement (β = .15). The simple slope in Figure 2A shows that the effects of self-concept on achievement are positive for all levels of global value, whereas the sizes of this positive simple effect vary substantially as a function of attainment value. More specifically, two latent observations of each student are represented on the surface display as one point; the circle on the surface contains at most 50% of the data points. The color of the surface indicates the level of achievement (from dark red to dark green, indicating −1 SD to +1 SD achievement), which is useful to identify the gradient of the regression line. For instance, the regression line of self-concept is relatively flat at −1.5 SD global value, increasing in steepness with incremental global value, and very steep at +1.5 SD global value. In other words, the effect of self-concept is moderated by global value: weaker with low value and substantially stronger with high value. Figure 2A also demonstrates that the simple effects of global value varied as a function of self-concept. The higher the self-concept, the more the global value contributes to increasing achievement. This finding supports a synergistic relationship between self-concept and value in predicting achievement. It should be noted that slightly negative slopes for the effect of global value on achievement are evident when self-concept is very low (e.g., −1.5 SD self-concept).
For self-reported effort, the interaction effect between self-concept and global value was statistically significant (β = .10). The positive multiplicative relation between self-concept and global value (Figure 2B) indicates that the simple slope for the effect of self-concept on effort is relatively small when global value is low (–1.5 SD) and becomes more positive when global value is high (+1.5 SD). Figure 2C reveals a similar pattern of interaction between self-concept and global value for teacher-rated engagement, but the pattern is somewhat smaller (β = .05) compared to that for achievement.
Subsequently, we added interactions between self-concept and each value component to predict the three outcomes. In this case, we examined only interaction effects between self-concept and one value component at a time, in addition to self-concept-by-global value interaction (Models D2 through D5). However, only path coefficients from interactions between self-concept and low cost to achievement were statistically significant (β = .10; see Figure 2D). The inclusion of additional interaction between self-concept and specific value components did not alter the pattern of results (see Table 5).
In summary, the interactions between self-concept and global value were consistently found to be significant and positive, thus providing support for synergistic relationships in predicting the three outcomes. However, controlling for interaction between self-concept and global value, interaction between self-concept and specific value components did not have additional predictive power except for self-concept-by-low cost interaction on achievement.
Discussion
The current study is the first to evaluate the unique contributions of self-concept and the four math value components on academic achievement, self-rated effort, and teacher-reported engagement. In line with a priori predictions, math self-concept proved to be a relatively important predictor of math achievement, whereas value components were more strongly associated with self-reported effort. We extended past research on the application of modern EVT by linking motivation beliefs to teacher-reported outcomes, and the findings indicate that self-concept and value beliefs emerged as equally important predictors of academic engagement assessed by teacher. More importantly, as expected, different value components have differential contributions to the prediction of the outcomes, particularly for effort and engagement, over and above the global value factor. Furthermore, we provided empirical evidence supporting synergistic interactions between self-concept and value in predicting the achievement-related outcomes; this is consistent with modern EVT.
Unique Contributions of the Four Value Components and Self-Concept
Controlling for self-concept and the global value factor, only one of the specific value beliefs—low cost—significantly predicted math achievement. For self-reported effort and teacher-rated engagement, the predictive effects of the four value factors differed substantially, thus supporting their discriminant validity. Consistent with our expectations, intrinsic value and low cost made unique contributions in predicting engagement and effort. Interestingly, attainment value plays a more important role in promoting students’ effort, over and above the global value factor. Indeed, modern EVT places great emphasis on the roles of both personal and social identities that underlie attainment value over the last decade (Eccles, 2009, 2011). Attainment value, relating to how well the task helps students manifest their personal needs and both their personal and their social identities, becomes more salient for engagement by older students, who have better-articulated identities (Eccles & Wang, 2012). However, utility value did not have unique predictive power on the three outcomes. One potential explanation is that utility value, referring to how useful a task is for fulfilling students’ various short- and long-term goals, may be more directly related to course work choices and enrollment intentions (Eccles et al., 1999; Eccles, Vida, & Barber, 2004; Guo, Parker, et al., 2015) as well as educational and career aspirations (Durik et al., 2006; Watt et al., 2006, 2012). These distinct patterns of results provide strong support for the conceptual differentiation of task value components.
In contrast to self-rated effort, self-concept makes a significant contribution to the prediction of teacher-reported engagement, controlling for task values. One reason might be that teacher-rated behavioral engagement is inherently confounded by the teachers’ knowledge of students’ achievement, which is in turn highly associated with students’ self-concept. Indeed, previous research has demonstrated that teachers appear to use students’ performance- and ability-based information to inform their inferences of engagement (Givvin, Stipek, Salmon, & MacGyvers, 2001; Lee & Reeve, 2012; Skinner et al., 2008). However, it is important to keep in mind that students’ prior achievement most likely also affects students’ perceptions and, consequently, their behavior. Thus, these possible confounding effects should be further investigated in future research.
The Nature of the Multiplicative Relation
In this section we more carefully evaluate what constitutes support for EVT when there is an expectancy-by-value interaction, clarify some apparent misconceptions in the recent literature, and address these clarifications in relation to the results of the present investigation. To do this, we provide a series of graphs of paradigmatic outcomes based on hypothetical results—purely synergistic or compensatory interactions with no first-order effects, or combinations of positive first-order effects and various forms of expectancy-by-value interactions (see Figure 3). These graphs and their interpretation in relation to EVT express certain complexities apparently not identified in previous research.
Response surface display for synergistic and compensatory relations between academic self-concept and task value, based on the hypothetical results. In regression equations containing an interaction term, simple slopes of simple regression equations, and the status and interpretation of interaction (ordinal vs. disordinal), are unaffected by additive scales transformation. Thus, the hypothetical plots presented in this section were independent of scaling (uncentered vs. centered; see Appendix E in the supplemental materials for more discussion); it is important to note that the interactions in which two predictors (i.e., academic self-concept and task value) and the outcome variable (i.e., achievement-related performance and choices) are considered as continuous latent variables. Only the linear effects of both predictors are included in the hypothetical regression equations. ASC = academic self-concept.
Even with relatively simple models, the interaction effects can be substantially different. Typically, synergistic and compensatory relations predict the interaction between two independent variables. The “pure” synergistic model (i.e., positive interaction effect) with no first-order effects indicates that individuals tend to choose and pursue a task only when both academic self-concept and task value are either high or low (Figure 3A). Conversely, the “pure” compensatory model (i.e., negative interaction effect) with no first-order effects indicates that to gain high achievement-related outcomes, individuals need either high self-concept coupled with low task value or vice versa (Figure 3E). Likewise, synergistic and compensatory models with substantially smaller positive first-order effects are similar to the “pure” models, in that the simple effects of self-concept (and task value) are negative for some levels of task value (and self-concept; Figures 3B and 3F). We argue that these forms of interaction would not be in line with modern EVT. In particular, in contrast to suggestions by Nagengast et al. (2011) and Trautwein et al. (2012), neither a purely synergistic interaction (with no first-order effects) nor a result dominated by a synergistic interaction is consistent with EVT predictions. Nevertheless, it should be borne in mind that in empirical settings, interaction effects are typically small to moderate in size, resulting from the sparsity of cases in extreme conditions (e.g., high self-concept coupled with extremely low task value).
When the positive first-order effects are similar in size to or stronger than the interaction effect, the synergistic model shows that the outcome is especially high if individuals have high self-concept and task value. These findings align with modern EVT (see Figures 3C and 3D). Equivalently, this finding indicates that the simple effect of self-concept is stronger for individuals with higher task value and that the simple effect of self-concept is substantially weak when task value is extremely low and vice versa. In contrast, the corresponding compensatory model indicates that self-concept has a stronger positive simple effect on the outcome when task value is lower; the simple effect of self-concept is substantially weaker when task value is extremely high and vice versa (see Figures 3G and 3H). In other words, this finding suggests that high self-concept can only partially compensate for low task value to achieve the outcome (and vice versa), particularly when the first-order effects are substantially larger than the interaction effect. These forms of compensatory interaction are also consistent with modern EVT. In sum, when the size of the first-order effects is similar to (or substantially stronger than) the interaction effect, both synergistic and compensatory interactions support modern EVT.
One of the central contributions of this study is its examination of the interaction effects of self-concept and task values in relation to the modern EVT model (Eccles, 2009). The results show the synergistic interaction between self-concept and global value, with stronger first-order effects on the three outcomes. These findings provide clear evidence for modern EVT predictions, suggesting that students tend to gain high math achievement, to exert great effort, and to be highly engaged only when both self-concept and task value are relatively high. Interestingly, in addition to self-concept-by-global-value interaction, a synergistic interaction is evident between self-concept and low cost for math achievement. This suggests that students with high math self-concept are unlikely to achieve academically if they ascribe a high level of task cost to math in terms of time, effort, and energy. This finding is in line with more recent empirical work on cost, which suggests that cost is better conceived of as a moderator variable for the relations between expectancy and achievement-related behaviors, compared to other value components (Barron & Hulleman, 2015; Flake et al., 2015).
In sum, the multiplicative relations between self-concept and task value for all three outcomes are consistent with our expectations and with modern EVT predictions, highlighting the importance of taking expectancy-by-value interaction into account in future EVT studies. 2
Limitations, Strengths, and Implications
Some limitations should be considered when interpreting the results. First, we focused only on self-concept and task value in the domain of math in the present study. Further examination of the associations between motivation beliefs and achievement-related outcomes in other domains, across diverse national/international samples, would be useful for clarifying whether the current findings are replicable and reflect a generalizable EVT prediction, particularly for the multiplicative relation between self-concept and task values.
Second, teacher-rated engagement was measured by two items in this study: behavioral engagement in math lessons and homework. However, academic engagement has been assumed to be a multidimensional construct and in prior studies was usually assessed by multiple items (Wang, Willett, & Eccles, 2011). For example, engagement was conceptualized by three features: behavioral, emotional, and cognitive (Skinner et al., 2008; Wang & Eccles, 2013). Hence, further use of multidimensional measures of engagement would provide a more nuanced understanding of associations between motivational beliefs and these outcomes.
Third, as with previous studies (Nagengast et al., 2011; Trautwein et al., 2012), we used the measure of self-concept to address this substantive issue—expectancy-by-value interaction—with theoretical and practical implications. A worthwhile further study would be to tackle this issue on the basis of the measure of expectancies of success.
Fourth, to keep the length of the questionnaire in balance, only two items were used to measure two value facets: utility for school and utility for job. This resulted in low reliability (α = .52 and α = .68). Indeed, using short scales can undermine reliability as well as validity (see further discussion in Gaspard et al., 2015). However, in this study, we mainly focus on the major value components posited in modern EVT. If the focus of subsequent research were on the value facets, then the development of a more extensive instrument with more refined items measuring different value facets would be desirable.
Fifth, to evaluate the temporal ordering of the EVT constructs in relation to the achievement-related outcomes implicit in the present investigation, there is a need for carefully constructed longitudinal panel studies and, perhaps, for experimental interventions to better understand the causal mechanisms. Additionally, because the study was based on responses by Year 9 students in German academic-track schools, future studies evaluating the generalizability of the results to students who are younger, less able, in different school types, and from other countries are warranted. For example, it might be that younger, less able students in untracked systems have less well-defined and less differentiated values in relation to mathematics.
Finally, although the global value factor and the specific value factors (i.e., the four value components) are well defined in the second-order bi-factor model, the factor loadings of some value items on the global value are substantially higher than those on the specific value facets. This indicates that the global, overarching value factor may capture the essence of specific value facets, which would lead to the value components losing predictive power on educational outcomes. Thus, it is important to replicate and extend future research to evaluate factor structure of value beliefs using bi-factor models.
Despite these limitations, this study makes significant contributions to the existing research in a number of ways. First, this study expands our understanding of the interplay between self-concept and value beliefs in predicting academic behaviors and provides a heuristic guide for future research and for intervention design. This finding of a synergistic relation between self-concept and value beliefs implies that isolated interventions that aim at strengthening one component would be less effective at promoting academic performance, effort, and engagement. Rather, interventions targeting the promotion of educational outcomes should seek to enhance both self-concept and value beliefs. Second, examining the unique contribution of each value component advances our understanding of what value components lead to gains in achievement, effort, and engagement. Importantly, the distinctive patterns of value components in relation to academic outcomes provide more specific suggestions for intervention strategies. For example, perceived math attainment was more highly associated with students’ effort, compared to other value components. Our findings also have the potential to contribute to the design of more specifically targeted and nuanced student engagement programs. Furthermore, the inclusion and distinguishing of self-reported and teacher-rated effort enabled us to identify differences in the pattern of predictions for these two outcomes. Different patterns of results for student-rated effort and teacher-reported engagement in our study also suggest the importance of assessing both student and teacher perceptions to better understand actual levels of student academic effort and engagement. In conclusion, we have provided a comprehensive picture illuminating the differential roles of motivational beliefs and their interaction with self-concept in predicting achievement-related behaviors. The findings underscore the importance of assessing the unique contribution of value beliefs and self-concept-by-value interaction, which was much less emphasized in modern EVT.
Footnotes
Acknowledgements
This research was funded in part by German Research Foundation Grant TR 553/7-1, awarded to Ulrich Trautwein, Oliver Lüdtke, and Benjamin Nagengast; by an Australian Research Council Grant, awarded to Herbert W. Marsh, Alexandre J. S. Morin, and Philip D. Parker (DP130102713); and by German Research Foundation Grant KE 1664/1–1, awarded to Augustin Kelava. This article was prepared when the first author was a visiting scholar at the Hector Research Institute of Education Sciences and Psychology, University of Tübingen, Germany. His research stay was funded by a scholarship from the German Academic Exchange Service (DAAD; ID: 50015537) for Jiesi Guo. Hanna Gaspard, Brigitte Brisson, and Isabelle Häfner are members of the Cooperative Research Training Group of the University of Education, Ludwigsburg, and the University of Tübingen. This latter is supported by the Ministry of Science, Research, and the Arts in Baden-Württemberg. Hanna Gaspard and Isabelle Häfner are also doctoral students of the LEAD Graduate School (GSC 1028), funded by the Excellence Initiative of the German federal and state governments.
Notes
Authors
JIESI GUO is a PhD student at the Institute for Positive Psychology and Education at the Australian Catholic University. His areas of interest include educational and developmental psychology with a particular focus on academic self-concept and expectancy-value motivation.
BENJAMIN NAGENGAST is a professor of educational psychology at the Hector Research Institute for Education Sciences and Psychology, University of Tübingen. His research interests include quantitative methods (causal inference, latent variable models, multilevel modeling), educational effectiveness, the evaluation of educational interventions and motivation, and academic self-concept.
HERBERT W. MARSH is a distinguished research professor at the Institute for Positive Psychology and Education at the Australian Catholic University and director of the International SELF Research Centre. He coined the phrase “substantive-methodological synergy,” which integrates his methodological focus on structural equation models, factor analysis, multilevel modeling, contextual/climate models, and new statistical approaches to meta-analysis with his major substantive interests, including self-concept and motivational constructs, evaluations of teaching/educational effectiveness, developmental psychology, sports psychology, special education, the peer review process, gender differences, peer support, and antibullying interventions.
AUGUSTIN KELAVA is a professor of educational psychology at the Hector Research Institute for Education Sciences and Psychology, University of Tübingen. His main interests include quantitative research methods (nonlinear latent variable modeling, semi- and nonparametric methods), competence modeling for engineering students, and psychophysiological coherence signals in emotion regulation.
HANNA GASPARD is a postdoctoral research fellow at the Hector Research Institute of Education Sciences and Psychology, University of Tübingen. Her research focuses on student motivation and motivational interventions in the classroom.
HOLGER BRANDT is a postdoctoral research fellow at the Hector Research Institute of Education Sciences and Psychology, University of Tübingen. His main interests are nonlinear structural equation models, modeling of growth curves with latent variable models, and the development of causal mediator and surrogate models.
JENNA CAMBRIA is a postdoctoral scholar at the Hector Research Institute of Education Sciences and Psychology, University of Tübingen. Her research interests focus on the processes and measurement of motivation and engagement as well as the role of motivation on vocational choices.
BARBARA FLUNGER is a postdoctoral research fellow at the Hector Research Institute of Education Sciences and Psychology at the University of Tübingen. Her research focuses on interindividual differences in students’ motivation and their association with students’ academic outcomes.
ANNA-LENA DICKE is a postdoctoral research fellow at the University of California, Irvine. Her research focuses on understanding the driving factors and benefits of students’ interest, motivation, and engagement.
ISABELLE HÄFNER is a PhD student at the Hector Research Institute of Education Sciences and Psychology at the University of Tübingen and affiliated with the LEAD Graduate School. Her research interests include the influences of family background on students’ academic outcomes as well as interventions to promote students’ academic motivation.
BRIGITTE M. BRISSON is a doctoral student at the Hector Research Institute of Education Sciences and Psychology at the University of Tübingen. She is primarily interested in motivation research and in the implementation and evaluation of classroom-based motivation interventions.
ULRICH TRAUTWEIN is a professor of educational science and director of the Hector Research Institute for Education Sciences and Psychology, University of Tübingen. His research interests include the development of self-concept and motivation in the school context, school effectiveness, and the influence of homework on school achievement.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
