Abstract
Using a structural equation modeling approach, this study investigates the structural relationships between second language (L2) oral proficiency and interest in learning the target language (L2 interest), in the presence of potential mediating variables (i.e. perceived importance of speaking, out-of-class contact with L2 resources). It also examines if interactive and noninteractive out-of-class L2 contact function differently in the structural model. Data were collected from 1,922 college students taking Chinese, French, Russian, and Spanish language classes in the United States. Key findings suggest that (1) L2 interest, as one measure of integrativeness and intrinsic motivation, strongly drives L2 oral development. However, its effect is completely mediated by the perceived importance of speaking; (2) Out-of-class L2 contact, either interactive or noninteractive, significantly correlates with L2 interest but has no significant effect on L2 oral proficiency; (3) however, interactive and noninteractive out-of-class L2 contact still function differently in the structural relationship. Interactive language practice based on digital tools could play a promising role in future L2 learning. Pedagogical implications are presented in the discussion section.
Keywords
I Introduction
Second language (L2) oral proficiency refers to L2 learners’ ability to speak their second language to ensure communicative objectives in real-life settings. L2 oral proficiency is an important indicator of communicative language competence. The rise of communicative approaches to L2 teaching has made oral proficiency central to the L2 acquisition process (Moeller & Theiler, 2014).
The development of L2 oral proficiency involves cognitive and metacognitive mechanisms as well as linguistic and pragmatic knowledge. Important non-cognitive correlates of L2 proficiency include but are not limited to motivation factors. Motivation is the driving force and impetus behind language learning (Dörnyei, 2020). Without a sufficient level of motivation, even learners with the most excellent language aptitude may not complete language learning tasks successfully or realize long-term learning goals (Dörnyei, 2005, 2020). Motivation is a complex construct composed of multiple subsets of factors. Interest in the target language (L2 interest) has been regarded as a key component of L2 motivation (e.g. Gardner, 2009; Wen, 2011). Task value is another essential motivation factor. It refers to the importance, interest, and utility that learners attach to specific learning activities (Eccles & Wigfield, 2002).
Another contributing factor for language learning is out-of-class L2 contact, an important supplement to classroom learning. In this study, out-of-class L2 contact is defined as the frequency of out-of-class exposure to L2 resources for informal learning, including talking with speakers of the target languages, using L2 traditional media and digital tools. L2 learners with higher levels of motivation were more likely to pursue some types of out-of-class L2 contact (Ahn, 2007; Freed, 1990; Hernández, 2010). With a few exceptions, prior research generally supported that out-of-class L2 contact had a positive effect on L2 learning and oral proficiency development in particular (e.g. Lee & Dressman, 2018). Out-of-class L2 contact with traditional media, such as TV, movies, newspaper, and radio, is noninteractive. Before the wide use of the internet, interactive L2 contact was typically limited to face-to-face communication between speakers, which was not easily available out of class. Digital tools have greatly increased the accessibility of interactive language practice. However, very little research has investigated the difference between interactive and noninteractive L2 contact in terms of their relations with motivation factors and L2 oral proficiency achievement.
The effect of motivation on L2 learning outcomes is mediated by multiple factors. Motivation orientations indirectly exert their influences on L2 learning achievement through the mediation of factors such as learning strategies, motivational intensity, learners’ perception of self-confidence, learners’ perception of task importance, and motivated behavior (Gardner, 1985, 2001; Kim & Kim, 2014; Masgoret & Gardner, 2003; Pae, 2008; Walker, Greene, & Mansell, 2006). Without examining mediators between motivation orientations and L2 learning outcomes, we do not know how motivation orientations turn into motivated behavior and finally lead to better learning outcomes.
In summary, it is necessary to examine how L2 interest, directly and indirectly, affects L2 oral proficiency in the presence of potential mediators (i.e. perceived importance of speaking, out-of-class L2 contact). However, due to the limitation of traditional data analysis methods, the full structural relationships among L2 oral proficiency and the other three factors are far from clear. In other words, in a structural system, we know very little about how these three factors relate to each other and how they affect L2 oral proficiency in a direct or indirect way. Whether and how L2 interest finally turns into L2 learning gains through the mediation of out-of-class L2 contact and perceived importance of speaking remains unclear. Moreover, existing studies have investigated overall L2 learning gains, but very little research has focused on L2 oral proficiency as measured by the American Council of Teaching of Foreign Languages Oral Proficiency Interview (ACTFL OPI), a widely-used foreign language oral assessment in the United States. Lastly, some previous studies have some deficiencies in research design, which will be discussed in Section II.
This study aims to fill in these gaps. It utilized a structural equation modeling (SEM) approach, which is a hybrid of factor analysis and path analysis, to model the full structural relationships among four variables: L2 oral proficiency, L2 interest, out-of-class L2 contact, and perceived importance of speaking. The significance of the current study is threefold. First, it examines the direct and indirect influences of two motivation factors (i.e. L2 interest, perceived importance of speaking) on L2 oral proficiency. It identifies the determinant motivation factors for L2 oral proficiency development. It also determines mediating links through which motivation orientations exert their influence on L2 learning outcomes. Second, it detects differences between traditional media and digital tools in terms of their functions in this structural model. This study highlights the advantage of social media and email in promoting interaction among L2 speakers. Third, this study provides important implications for instruction regarding how to convert motivation orientations into motivated behaviors, and how to maximize the pedagogical values of out-of-class informal learning. As compared with cognitive aptitude, motivation factors can be more easily influenced by teachers. The investigation of motivation correlates of L2 proficiency is of great pedagogical value. Such studies would help make learning more efficient, effective, and enjoyable for future L2 learners.
II Literature review
This study has three exogenous/independent variables: interest in the target language (L2 interest), perceived importance of speaking, and out-of-class L2 contact. The first two variables are components of L2 motivation, which are first reviewed. L2 motivation is a composite construct, but this study does not aim to investigate its internal structure. The literature review centers on how motivation components correlate with L2 learning outcomes. The researcher then reviews the effect of out-of-class L2 contact on L2 oral proficiency. Out-of-class learning activities with different degrees of interaction are compared in terms of their relationships with L2 interest and with L2 oral proficiency. This section ends with a discussion of the limitations of existing studies before proposing the research questions for the present study.
1 Motivation and L2 proficiency
As the driving force and impetus behind language learning (Dörnyei, 2020), motivation is a critical contributor to language acquisition and an important predictor of language learning success. L2 motivation research has gone through three phases of evolution: (1) the social psychological period (1959–1990) characterized by Gardner’s work; (2) the cognitive-situated period (the 1990s) characterized by work based on cognitive theory; (3) the process-oriented period (2000 to present) characterized by an interest in motivational change.
Some recent empirical studies still orient their discussions around the social psychological conceptualization. Gardner’s model posits that language learning achievement is impacted by language aptitude, integrative motivation, and other factors (Gardner, 2001). Integrative motivation consists of three main components: integrativeness, attitudes toward the learning situation, and motivation. Each component is further broken down into subcomponents. Integrativeness comprises subcomponents such as integrative orientation, interest in foreign languages, and attitudes toward the L2 community. Gardner’s theory has been widely accepted by L2 researchers. However, they narrowly interpreted it as the sum of integrative and instrumental motivation (Dörnyei, 2005). Instrumental motivation is inspired by more functional purposes, such as getting good grades. Integrative motivation refers to the willingness or desire to integrate into the target language group and target culture.
Social psychological theorists also emphasize ‘what individual students think about their tasks, values, competence, and effort devoted to the task’ (Rueda & Chen, 2005, p. 212). The theory of task value suggests importance, interest, and utility as three indicators of task value (Eccles & Wigfield, 2002). L2 learners’ perceived task value positively correlates with the effort expended and therefore is an important mediating link between motivation orientation and learning outcomes (Dweck & Leggett, 1988; Rueda & Chen, 2005).
Later, motivation research interests shifted to the cognitive aspect. Self-determination theory (Deci & Ryan, 1985, 2002) elaborates various types of extrinsic and intrinsic motivations, and they are incorporated to conceptualize L2 motivation. While intrinsic motivation pertains to internal rewarding consequences, such as the feeling of self-determination and competence, extrinsic motivation involves external rewards, such as good grades or feedback from instructors (Deci, 1975). Extrinsic motivation is usually not as powerful as intrinsic motivation because it depends on external rewards (Brown, 2000).
Dörnyei (2005) recast L2 motivation from a socio-dynamic perspective and proposed an L2 Motivational Self System, which represents a major reformation of previous thinking about motivation. Different from traditional practice that examines causality effects between isolated variables, the socio-dynamic approach allows the systematic operation of multiple factors relevant to a specific situation (Ushioda & Dörnyei, 2012). The L2 Motivational Self System consists of three components: Ideal L2 Self (i.e. an image of what the L2 learner would like to become), Ought-to L2 Self (i.e. external expectations that the L2 learner ought to meet in order to avoid possible negative consequences), and L2 Learning Experience (i.e. the immediate learning environment, such as curriculum, and teachers). The Ideal L2 Self has been viewed as the internalized facet of L2 motivation (Dörnyei, 2009) and the more powerful impetus behind L2 learning (Dörnyei, 2009). Learners with a more vivid Ideal L2 Self and higher levels of motivated behavior were more likely to achieve L2 proficiency (Dörnyei & Chan, 2013; Kim, 2012).
The interest in learning the target language (L2 interest) has been regarded as a key motivation component in all three stages of L2 motivation research. It is a principal component of integrativeness (e.g. Gardner, 2009) and is regarded as a primary intrinsic reason to learn a foreign language (e.g. Wen, 1997, 2011). L2 interest is also highly relevant to Ideal L2 Self. The following review of empirical studies centers on three concepts closely related to L2 interest (i.e. integrativeness, intrinsic motivation, and Ideal L2 Self) and how they correlate, directly and indirectly, with L2 learning outcomes.
It is generally accepted that students with higher language learning motivation often achieve higher L2 proficiency (e.g. Segalowitz & Freed, 2004; Yu, 2019). Particularly, integrative motivation and intrinsic motivation have been identified as significant predictors of L2 learning outcomes. Previous studies (Kang, 2001; Wang, 2008; Wen, 1997), using traditional analysis techniques (e.g. ANOVA, t-test, correlation, and linear regression), had mixed findings regarding the effect of extrinsic motivation. However, they all have provided empirical evidence for the positive relationship between intrinsic motivation and L2 learning achievement.
Motivation is a complex and composite construct. Research efforts have been made to understand the relationship between subsets of motivation factors, and their direct and indirect effects on foreign language engagement and learning achievement via the SEM approach. For instance, Kim and Kim (2014) investigated the structural relationship between Ideal L2 Self and other factors, including learning styles, motivated behavior, and ESL learning achievement. Findings varied across school levels. For elementary school students, Ideal L2 Self led to higher L2 proficiency without the mediation of motivated behavior. However, for students in Junior high schools, neither Ideal L2 Self nor motivated behavior was a significant predictor of L2 proficiency. For high school learners, motivated behavior was the most important L2 proficiency correlate.
The following three SEM studies were the main references for the current study in model building and are reviewed in detail. Csizér and Dörnyei (2005) investigated the interrelationship of multiple motivation components and the contribution of each component to motivated behavior, as measured by (a) the students’ language choice for L2 studies in future and (b) the amount of effort they intended to exert on learning a certain language. Findings suggested that integrativeness was a central factor in the motivation construct. Integrativeness directly and strongly related to language choice and intended effort. Most effects of other motivation factors were subsumed by integrativeness or mediated through the relationship between integrativeness and language choice/intended effort. This study collected data by using a Likert-scale survey, and its observed variables were categorical. However, the researchers used the maximum-likelihood estimation method, which was the default estimation of continuous variables. When the distribution is not normal, treating the categorical data as continuous might lead to underestimation of correlations and a less interesting model (Muthén & Kaplan, 1985).
Rueda and Chen (2005) used an SEM approach to investigate the relationship among motivation factors, including instrumentality, intrinsic motivation, passivity toward requirements, students’ perception of task value, belief about effort, self-efficacy, and effort devoted in the Chinese L2 context. Rueda and Chen’s model suggested that both the intrinsic factor and passivity toward requirements highly and significantly determined perceived task value. However, the effect of instrumentality was not significant. Perceived task value could predict effort devoted to learning, and the regression coefficient was high and significantly different from zero. The effects of three motivation factors (i.e. instrumentality, intrinsic motivation, and passivity toward requirements) on effort devoted were completely mediated by the perceived task value. This finding echoes that of Csizér and Dörnyei (2005) in terms of the important role of mediating effect. However, this study did not provide information about the specific SEM estimation methods used. Although researchers reported the regression coefficients among individual motivation factors, they failed to report overall model fit indices. There was no evidence of whether their proposed model fit the data or not, so the conclusions should be interpreted with caution.
Pae (2008) explored the structural relationship between six types of L2 motivation components (i.e. integrative orientation, instrumental orientation, intrinsic motivation, and three subtypes of extrinsic motivation) and L2 achievement in the presence of some potential mediating or moderating variables (e.g. self-confidence, attitudes) in a context of English as a foreign language (EFL) in South Korea. SEM analysis results indicated that intrinsic motivation was the strongest and only significant determinant of students’ self-confidence and impetus to learn a second language. Intrinsic motivation was indirectly associated with L2 learning achievement through the mediation of self-confidence and the mediation of overall motivation to learn the language. However, in contrast to Csizér and Dörnyei (2005), integrativeness was not a significant indicator. Pae (2008) did not find a direct effect of any of the six types of motivation on L2 achievement.
2 Out-of-class L2 contact
Formal class teaching is insufficient for L2 oral proficiency development (Blake, 2013; Kagan & Dillon, 2004). Out-of-class contact with L2 resources presumably provides more linguistic input and is regarded as an important supplement to classroom learning (Ellis, 2002). It is now commonly accepted in second language acquisition literature that interpersonal interaction plays an essential role in L2 learning (Gass & Mackey, 2007, 2015). Before the advent of social media, out-of-class informal learning used to be primarily limited to in-person interaction and noninteractive receptive practice using traditional media, such as reading books, magazines, and listening to music. Since the mid-2000s, the appearance of social media, including blogs, wikis, Facebook, and Twitter, has greatly compensated for the lack of interaction in out-of-class informal learning. Social media enable authentic communication among learners and active engagement with the content (e.g. by tagging and writing comments) (Zourou, 2012). Previous studies reported that social media could facilitate the development of genuine L2 sociopragmatic usage (e.g. Blattner & Fiori, 2009, 2011; McBride, 2009) and increase the accessibility of situated and simulated practice (e.g. Mills, 2011; Reinhardt & Ryu, 2013). Out-of-class digital learning appeared to be promising in enhancing L2 development (Benson & Reinders, 2011; Nunan & Richards, 2015; Reinders & White, 2016).
Prior to the advent of digital tools, Freed (1990) systematically investigated if out-of-class in-person interaction or media-based noninteractive activities (i.e. watching TV and movies, reading books and newspapers, and listening to the radio) had a significant impact on L2 oral proficiency as measured by the ACTFL OPI in a study abroad context. Findings suggested that neither in-person communication nor noninteractive activities based on traditional media had a significant effect on OPI scores at any proficiency level.
In recent years, studies have reported a positive relationship between digital learning and L2 oral proficiency. Sun et al. (2017) investigated the effect of in-class social media-based activities on EFL speaking. While both the experimental and control group showed similar progress in accuracy and pronunciation after one semester, the experimental group had significantly higher gains in fluency. They attributed the gains to the affordances of the social media-based learning platform: a user-friendly, low-stress, contextualized and encouraging learning environment. Lee and Dressman (2018) reported significant correlations between L2 English oral proficiency and the diversity of informal digital learning experience. Engaging in diverse informal digital learning activities significantly predicted L2 English learners’ oral proficiency.
Some studies investigated both traditional media and digital tools. In a longitudinal study, Zhang, Winke and Clark (2020) investigated the relationship between oral proficiency development and out-of-class L2 contact as measured by the exposure to various L2 resources, including social media, news/music, TV/movies, books/newspapers/magazines, and emails. Some types of contact, such as using social media and emails, are more interactive, while others, like watching TV and reading newspapers, are noninteractive. This study did not find a significant effect of exposure to all types of resources as a whole on L2 oral proficiency development. However, Hernández (2010), who investigated adult L2 Spanish learners in a study abroad context, had opposing findings. The amount of student interaction with the L2 resources outside of class (as measured by the amount of exposure to traditional media, email, and the internet) was a significant factor, which accounted for 48% of the variance of pretest to posttest of oral proficiency.
Three additional studies revealed features of effective informal learning by comparing different types of learning activities. De Wilde, Brysbaert, and Eyckmans (2020) examined the effect of eight types of input: watching English spoken TV without/with English subtitles, watching English spoken TV with L1 subtitles, listening to English music, reading English books, using social media in English, playing English games, and speaking English. Only three types of input (i.e. gaming, speaking English, and using social media) were found to be beneficial to L2 oral proficiency. These favorable types of input were interactive, multimodal, and involved authentic communication as well as language production. Similarly, Ahn (2007), in a study on the effect of L2 contact on pragmatic competence, revealed that while the overall amount of L2 contact had a week and non-significant impact, the productive and more interactive types of L2 contact moderately impacted pragmatic competence. Moreover, Freed, Segalowitz, and Dewey (2004) discovered that the amount of time spent on out-of-class writing tasks (e.g. writing emails, homework, personal notes) positively correlated with gains in L2 oral fluency. But reading and listening tasks had no significant effect. These findings suggested that the degree of interaction and the depth of processing might determine the effectiveness of out-of-class L2 contact.
Previous studies also investigated the relationship between motivation and L2 learners’ likelihood of pursuing contact with L2 resources in/out of class. In an earlier study, Freed (1990) found no relationship between overall motivation and student tendency to pursue interactive out-of-class L2 contact (i.e. talking with native speakers). However, L2 learners with higher levels of motivation were more likely to pursue noninteractive out-of-class L2 contact, including watching movies and reading newspapers. Recent studies distinguished different types of motivation. Ahn (2007) reported that integrativeness significantly correlated with conversational interaction. Instrumental orientation significantly correlated with noninteractive L2 contact, such as watching television and listening to the radio. However, all correlations were low or moderate. Hernández (2010) studied the relationship between motivation and L2 Spanish learners’ exposure to L2 resources out of class in a study abroad context. The regression analysis results suggested that L2 learners with higher integrative motivation tended to interact more with the L2 resources. However, instrumental motivation was not a significant predictor.
3 Summary of findings and gaps
Although they conceptualized L2 motivation from different perspectives (i.e. social psychological, cognitive, and socio-dynamic perspectives), previous empirical studies generally supported the positive effect of motivation, especially intrinsic motivation, integrativeness, and Ideal L2 Self, on L2 learning outcomes. However, some findings were contradictory, and contradictions could be explained by the interaction effect between L2 motivation and specific learning contexts (e.g. different languages, different ethnic groups) (Pae, 2008).
With a few exceptions (e.g. Freed, 1990; Zhang et al., 2020), previous studies have tended to support that out-of-class L2 contact was beneficial to L2 oral proficiency development. Particularly, activities involving deep levels of processing, production, and social interaction, such as writing (Freed et al., 2004), interpersonal speaking (De Wilde et al., 2020), and interactive digital learning (De Wilde et al., 2020; Lee & Dressman, 2018) were effective in promoting L2 oral proficiency. L2 learners with a higher level of integrativeness were more likely to pursue some types of L2 contact; whereas there was no agreement on the effect of instrumental motivation on L2 contact (Ahn, 2007; Hernández, 2010). However, it is still necessary to examine whether interactive and noninteractive types of out-of-class L2 contact differ in terms of their relationships with motivation factors and L2 oral proficiency.
Motivation factors indirectly exert their influences on L2 learning achievement through the mediation of other factors. Studies that ignored the mediating link might suggest a false linear relationship between motivation orientations and L2 learning outcomes (Csizér & Dörnyei, 2005). Moreover, the majority of existing studies only investigated the relationship between L2 oral proficiency and a single factor or a single type of factors (e.g. how motivation factors predict oral proficiency, and how oral proficiency regresses on out-of-class contact). They did not model the relationships among L2 oral proficiency and multiple variables within a single framework. Also, they failed to reveal how multiple variables interact with each other while affecting oral proficiency.
Finally, defects in research design discussed previously may also jeopardize the generalizability of some existing research findings. Besides, the majority of existing studies on L2 motivation focused on only one language. The interaction effect between motivation and language types is one cause of contradictory findings (Pae, 2008). A large-scale study of participants from various language groups is in need.
To address these limitations, the current study utilized an SEM approach to model the full structural relationships between interest in learning the target language/L2 interest and L2 oral proficiency, with the perceived importance of speaking, out-of-class L2 contact, as potential mediators. Data were collected from 1,922 learners of four languages (i.e. Chinese, French, Russian, and Spanish). One overarching research question, consisting of two sub-questions, has guided this study:
What are the structural relationships among L2 oral proficiency and the following factors: interest in learning the target language (L2 interest), perceived importance of speaking, and out-of-class contact with L2 resources?
Research question 1: Does L2 interest have a significant direct effect on L2 oral proficiency in the presence of potential mediating variables (i.e. perceived importance of speaking, and out-of-class contact with L2 resources)?
Research question 2: Do interactive and noninteractive out-of-class L2 contact differ in terms of their relationships with motivation factors (i.e. L2 interest, perceived importance of speaking) and L2 oral proficiency?
III Data and methodology
1 Data and measures
Data in this study are preexisting and available for download from Winke, Gass, and Zhang (2019). The data were collected from 1,922 students enrolled in Chinese (n = 219), French (n = 483), Russian (n = 120), and Spanish (n = 1,100) classes at Michigan State University. The endogenous or outcome variable was L2 oral proficiency measured by the ACTFL OPIc, a widely used internet-delivered test in the United States that measures L2 oral proficiency with a high degree of reliability and validity. The ACTFL OPIc is a one-on-one interview between a test-taker and an avatar that lasts 10 to 40 minutes. It intends to generate a ratable sample of test-takers’ oral proficiency in natural conversations, which accommodate their interests/background and ongoing test responses. Test-takers’ performance is rated using the ACTFL proficiency guidelines: Speaking (ACTFL, 2012), a hierarchy of descriptors that specifies ‘what individuals can do with language in terms of speaking in real-world situations in a spontaneous and non-rehearsed context’ (Swender & Vicars, 2014, p. i). The four main levels of grading are Superior, Advanced, Intermediate, and Novice. Advanced, Intermediate, and Novice levels are further divided into three sublevels: High, Mid, and Low (ACTFL, 2012). In total, there are 10 levels of grading, which were coded as 1–10 (Novice Low = 1, Superior = 10) in this study. Since there is no empirical evidence that distances between levels are equal, the proficiency ratings were treated as categorical data.
Three exogenous variables were L2 interest, perceived importance of speaking, and out-of-class L2 contact. The first two variables were measured by six-point Likert-scale items (e.g. your interest in learning the target language; your perceived importance of speaking skills for learning the target language). Out-of-class L2 contact was measured by the frequency at which participants used the following L2 resources out of class: social media, TV/movies, news/music, books/newspapers/magazines, and emails. The frequency was measured using a Likert scale, with 1–6 representing ‘never’, ‘once a month or less’, ‘a few times a month’, ‘weekly’, ‘a few times a week’, and ‘daily’. Social media and email enable social interaction among users and active engagement with the content (e.g. by writing comments and tagging). It was not necessarily true that participants’ use of social media was always for an interactive purpose. Sometimes, they might use social media only for reading or listening. However, as a composite category, social media-based learning activities could be regarded as more interactive than learning based on traditional media. Therefore, the use of social media and email was defined as interactive out-of-class L2 contact, while the use of remaining resources was defined as noninteractive.
The measurements of all exogenous variables were repeated three times within the range of two years. The three measurements took place in the second, third, and fourth semesters respectively, and thus observed variables were named after the course numbers such as 102, 201, and 202. The primary strength of repeated measurements is to get more reliable data. Repeated measurements were treated as data clustered within individuals to adjust the standard errors for clustering. For details on variables, see Tables 1 and 2.
Variables and measurements.
Note. * interactive out-of-class L2 contact.
Descriptive statistics for observed variables.
Note. * interactive out-of-class L2 contact.
2 Modeling strategies and data analysis
a Models to be tested
Reflecting on the relevant theoretical overview and empirical research findings, a set of three SEM models were proposed to explain the structural relationships between L2 interest and L2 oral proficiency mediated by the perceived importance of speaking and out-of-class L2 contact. In Model 1, the effect of L2 interest was completely mediated by perceived importance of speaking and out-of-class L2 contact (Figure 1). Model 2 had a direct effect between L2 interest and L2 oral proficiency (Figure 2). Model 3 treated interactive and noninteractive out-of-class L2 contact as two distinct variables (see Figure 3 below).
The path diagram of Model 1.
The path diagram of Model 2.
The path diagram of Model 3.
b Data analysis procedures
A series of SEM analyses was performed by using Mplus Version 8.3 (Muthén & Muthén, 1998–2019) to test which models could fit data. First, the measurement models that depicted the correspondence between latent variables and their indicators were examined. If the model fit indices were not within the acceptable ranges, modifications would be made to improve the overall model fit of measurement models. Additionally, regression coefficients between latent factors and their indicators were checked for significance and appropriateness.
Second, to answer research question 1, the overall model fit of the structural part of Model 1 and Model 2 to the dataset were examined. Since Model 1 was nested within Model 2, if both models fit the data, a chi-square different test would be conducted to test if Model 2 was better than Model 1. Also, individual parameter estimates, including the regression coefficients between latent variables, and the significance of direct effects were examined. Then, the indirect effects mediated by the perceived importance of speaking and by out-of-class L2 contact were tested. In order to get the correct sampling distribution and to have an adequate estimation of the indirect effects, it is recommended to test with bootstrap confidential intervals rather than Sobel’s test (Shrout & Bolger, 2002).
Third, to answer research question 2, Model 3, which separated interactive and noninteractive L2 contact, would be created based on the better one between Model 1 and Model 2. Then, Model 3 would be compared with that model to see if interactive and noninteractive out-of-class L2 contact functioned differently in the structural relationship.
c Estimation method
Diagonally weighted least squares (WLSMV), which was specifically designed for ordinal data or categorical data (Muthén, 1993), is the default estimator for categorical variables in Mplus. The WLSMV does not make normal distributional assumptions and is the best option for modeling categorical or ordered data (T. Brown, 2006). Since observed variables in the current study were all ordinal, it was decided to implement WLSMV instead of other methods of estimation.
d Assessing model fit
According to Kline (2005), the following model fit indices were used to evaluate the overall model fit: chi-square test of Model Fit, the p-value of chi-square, Root Mean Square Error of Approximation (RMSEA), and Tucker–Lewis index (TLI). If the p-value of chi-square is not significant, it means that the distance in fit between the variance-covariance structure of the observed data and the model-implied variance-covariance structure is close. However, the model chi-square is very sensitive to sample size. When the sample size is large, the chi-square tends to be high, leading to reject a model falsely (Bollen, 1989), so other fit indices should be used for model evaluation. Cut-offs and criteria of these model fit indices are presented in Table 4, 6, 7, and 10 below.
IV Results
1 The measurement model
The overall model fit of the measurement model was first evaluated for Model 1. Table 3 shows the correspondence between latent variables and their indicators. The model fit indices and cut-offs are reported in Table 4. First, the chi-square value (χ2) is significant (p < 0.001), which puts the model fit in doubt. However, as discussed previously, since the value of the model chi-square is highly sensitive to sample size (Bollen, 1989), it should be interpreted with caution when the sample size is large. So, other fit indices were used for model evaluation.
The measurement model for Model 1: Factor loadings.
Notes. * interactive out-of-class L2 contact.
The measurement model for Model 1: Fit statistics.
Notes. NNFI = Non-Normed Fit Index. RMSEA = Root Mean Square Error of Approximation. TLI = Tucker–Lewis index.
The value of TLI is larger than 0.9, indicating that this model’s fit improves significantly compared with its baseline model, which assumed no covariances among the variables. The value of RMSEA is estimated to be 0.04, and the probability of the RMSEA value lower than 0.05 is 0.99. As shown in Table 3, regression coefficients between latent variables and indicators are between 0.62 and 0.9 and are all significant. Similarly, the measurement model for Model 3 was evaluated, and the results are reported in Table 5 and Table 6. In summary, the measurement models for Model 1 and Model 3 fit the data very well, and no modification was needed.
The measurement model for Model 3: Factor loadings.
Notes. * interactive out-of-class L2 contact.
The measurement model for Model 3: Fit statistics.
Notes. NNFI = Non-Normed Fit Index. RMSEA = Root Mean Square Error of Approximation. TLI = Tucker–Lewis index.
In the context of SEM studies, capitalization on chance occurs if the true correlation does not exist in the population but appears in a sample due to sample variability (Rönkkö, 2014). When an initial model drawing on theory does not fit sample data well, it is a common practice to modify it to improve overall model fit (e.g. the changing of the path and/or the observed variables). However, since the modification process is data-driven, it is vulnerable to capitalization on chance characteristics of the data. As a result, model modifications may not be generalizable to other samples or the population (MacCallum, Roznowski & Necowitz, 1992). The issue of potential capitalization on chance must be of concern when (1) the sample size is small; (2) the initial model is modified in a data-driven way; and (3) the modification cannot be substantively justified (MacCallum et al., 1992). In the present study, the sample was large enough (i.e. n = 1,922) for SEM studies. Moreover, as shown previously, the measurement models for the initial models fit the sample data very well, and no modification was made. Therefore, the results were not likely impacted by sample variability.
2 Research question 1
To answer research question 1, Model 1 and Model 2 were tested to determine if they were adequate at explaining the structural relationships between L2 oral proficiency, motivation factors, and out-of-class L2 contact. As shown in Table 7, the two models both fit the data. Since Model 1 was a nested model of Model 2, a chi-square difference test was conducted to compare if one model fit the data better than the other. As shown in Table 7, the chi-square value for the difference is not significant, indicating that Model 2 is not better than Model 1.
Fit statistics for Model 1 and Model 2.
Notes. NNFI = Non-Normed Fit Index. RMSEA = Root Mean Square Error of Approximation. TLI = Tucker–Lewis index.
Direct effects between latent variables in Model 1 and Model 2 were checked for significance and appropriateness, and the results are shown in Table 8. In Model 2, the regression coefficient between L2 interest and L2 oral proficiency equals 0.13 and is not significantly different from 0 (p = 0.56). It indicates that L2 interest has no significant direct effect on L2 oral proficiency. Therefore, Model 1, the model without a direct relation between L2 interest and L2 oral proficiency, was more appropriate for explaining the structural relationships among variables investigated.
Standardized model results of Model 1 and Model 2.
Notes. * p < 0.05. Importance = Perceived importance of speaking.
As shown in Table 8, in Model 1, L2 interest significantly and highly relates to the perceived importance of speaking (β = 0.93*). Controlling for other exogenous variables, with one standardized division unit increase in L2 interest, perceived importance of speaking will increase by 0.93 standardized division unit. The perceived importance of speaking can significantly predict L2 oral proficiency (β = 0.35*). Controlling for other exogenous variables, with one standardized division unit increase in the perceived importance of speaking, L2 oral proficiency will increase by 0.35 standardized division units. L2 interest is also a significant predictor of out-of-class L2 contact (β = 0.35*). However, out-of-class L2 contact does not significantly relate to L2 oral proficiency (β = −0.01).
Table 9 reports the bootstrap confidential interval estimations of indirect effects between variables in Model 1. The indirect effect of L2 interest on L2 oral proficiency mediated by the perceived importance of speaking is significant (β = 0.33*). Controlling for other exogenous variables and with the perceived importance of speaking as the mediator, with one standardized division unit increase in L2 interest, L2 oral proficiency will increase by 0.33 standardized division unit. However, the indirect effect mediated by out-of-class L2 contact (β = −0.003) is close to zero and not significant. This suggests that with out-of-class L2 contact as a mediator, learners’ L2 interest has no significant effect on L2 oral proficiency achievement. Since there is no significant direct effect between L2 interest and L2 oral proficiency, the effect of L2 interest on L2 oral proficiency is completely mediated by learners’ perceived importance of speaking.
Bootstrap confidential intervals for indirect effects (Model 1).
Note. Importance = Perceived importance of speaking. *p < 0.05.
3 Research question 2
Based on Model 1, the researcher created Model 3 by dividing out-of-class L2 contact activities into two categories: interactive and noninteractive L2 contact. The first category consisted of: the frequency of using social media, and the frequency of writing/reading emails. The remaining types of L2 contact were classified as noninteractive L2 contact. To answer research question 2, Model 1 was compared with Model 3 (Figure 3) in terms of overall model fit and individual parameter significance. For comparison, results for Model 1 are reported again in Table 10, which shows that both Model 1 and Model 3 fit the data. Since Model 1 nested within Model 3, a chi-square difference test was conducted. As presented in Table 10, the chi-square value for the chi-square difference test is significant, indicating that Model 3 is a better solution. It means that interactive and noninteractive out-of-class L2 contact functioned differently in terms of their relationships with other factors and should be modeled as two separate variables.
Fit statistics for Model 1 and Model 3.
Notes. *p < 0.05. NNFI = Non-Normed Fit Index. RMSEA = Root Mean Square Error of Approximation. TLI = Tucker–Lewis index.
Direct effects and residual covariances between variables in Model 3 were checked for significance and appropriateness, and the results are reported in Table 11. Both interactive out-of-class L2 contact (β = 0.33*) and noninteractive out-of-class L2 contact (β = 0.37*) significantly and moderately regress on L2 interest. Neither noninteractive out-of-class L2 contact (β = −0.04) nor interactive out-of-class L2 contact (β = 0.03) is significantly related with L2 oral Proficiency.
Standardized model results of Model 3.
Notes. * p < 0.05. Importance = the perceived importance of speaking.
Table 12 reports the bootstrap confidential interval estimations of indirect effects. Similar to Model 1, the indirect effect mediated by the perceived importance of speaking is significant (β = 0.33*). However, interactive L2 contact (β = 0.01) and interactive L2 contact (β = −0.02) are not significant mediators between L2 interest and L2 oral Proficiency.
Bootstrap confidential intervals for indirect effects (Model 3).
Note. Importance = Perceived importance of speaking. *p < 0.05.
V Discussion
1 Interest as a strong driving force of L2 learning
L2 interest has been regarded as an important component of integrativeness and intrinsic motivation (e.g. Gardner, 2009; Wen, 1997, 2011). The findings of the current study indicate that interest in learning the target language plays an essential role in L2 learning. It highly and significantly relates to the perceived importance of speaking. To L2 learners, the more interested they are in the target languages, the more importance they would attach to speaking. L2 interest also has a significant effect on L2 oral proficiency through the mediation effect of perceived importance of speaking. In other words, the higher the level of interest is, the more likely it is that speaking will be perceived as an important task, and finally, the higher the L2 oral proficiency will be. These findings are consistent with previous studies (e.g. Kang, 2001; Wang, 2008; Yu, 2019), which suggested a positive effect of integrative and intrinsic motivation on L2 learning outcomes.
L2 interest is also a significant predictor of learners’ frequency of using L2 resources out of class in the format of both interaction (e.g. reading and writing emails, using social media) and non-interaction (e.g. listening to music, watching movies). This finding is consistent with that of previous studies, which supported integrativeness as significantly correlated with the likelihood of pursuing L2 contact (Ahn, 2007; Hernández, 2010). However, an early study by Freed (1990) had opposite findings. He found a positive relationship between motivation and student tendency to pursue noninteractive out-of-class L2 contact, but did not support the correlation between motivation and interactive out-of-class L2 contact in the format of in-person conversation. This contradiction could be explained by the engagement and accessibility of social media-based interaction. L2 learners often feel nervous talking with others face-to-face, especially with native speakers. As a result, higher L2 motivation does not necessarily lead to more in-person communication with other speakers. However, interaction based on social media is user-friendly, low-stress, contextualized and encouraging (Sun et al., 2017) and has made the situated and simulated practice more accessible (e.g. Mills, 2011; Reinhardt & Ryu, 2013). This finding suggests L2 learners’ positive attitude towards technology-mediated interaction and its advantages in compensating for the limitations of in-person interaction.
L2 interest is a strong impetus behind L2 oral proficiency development. While cognitive correlates of L2 learning outcomes, such as language aptitude, are hard to be altered, non-cognitive factors like motivation and interest can be manipulated by appropriate instructional intervention. How to initiate, maintain, and strengthen learners’ interest in the target language deserves more research and pedagogical efforts.
2 Mediators between L2 interest and L2 oral proficiency
In this study, the examination of mediation effects reveals how L2 interest finally turned into L2 oral proficiency gains. In the presence of the mediators, L2 interest has no direct effect on L2 oral proficiency. The existence of mediators between motivation orientations and learning outcomes has been widely documented by previous research (e.g. Pae, 2008). Nevertheless, it is still surprising that the effect of L2 interest on L2 oral proficiency is completely mediated by perceived importance of speaking in the present study. Without realizing the importance of speaking, even the highly-motivated learners may not be able to achieve high L2 oral proficiency.
The existence of mediators suggested that we should not take it for granted that a high-level motivation will automatically turn into better L2 learning outcomes. Studies are needed to systematically explore more mediating links between motivation and L2 learning gains. This line of research would provide valuable pedagogical implications regarding through which paths motivation orientations can be converted into language proficiency gains, how to exert the influence of motivation, and how to maximize the benefits of motivation.
Findings of this study echo Csizér and Dörnyei’s (2005, p. 20) argument that ‘studies that look only at the impact of motivation on language proficiency or other L2 achievement measures (such as course grades) ignore, in effect, the mediating link, behavior, and suggest a false linear relationship between motivation and learning outcomes.’ Results of previous studies that did not model mediating links should be interpreted with caution. SEM, an advanced statistical technique for modeling structural relationships with indirect effects, should play a more important role in studying mediation effects between motivation and L2 learning outcomes.
In both Model 1 and Model 3, the indirect effects of L2 interest on L2 oral proficiency mediated by out-of-class L2 contact are not significant. Whereas the first half of the indirect effect (i.e. the relationship between L2 interest and out-of-class L2 contact) is significant. So, out-of-class exposure to L2 resource could be regarded as a motivated learning behavior. The second half of the indirect effect is not significant. The frequency of out-of-class L2 contact, either interactive or noninteractive, cannot predict L2 oral proficiency. This finding contrasts with the common sense and empirical evidence that supported the positive relationship between out-of-class L2 contact and L2 learning achievement (e.g. De Wilde et al., 2020; Lee & Dressman, 2018). One possible explanation is that as the motivated learning behavior, out-of-class informal learning might not be effective for beginning L2 learners. Similarly, in the study reported by Kim and Kim (2014), which focused on young L2 learners, motivated behavior was also not a significant mediator between L2 motivation and L2 proficiency. Participants of the present study were lower-level L2 learners who had little L2 learning experience. These findings indicate that for inexperienced learners, if the motivated learning behavior is informal and unguided, it does not necessarily improve learning outcomes. The next section further reveals features of learning activities that can favor L2 oral proficiency and discusses how to enhance the effectiveness of out-of-class informal learning.
3 Non-significant effect of out-of-class L2 contact
Both teachers and learners expect that more exposure to L2 resources out of class is associated with higher language proficiency, and this hypothesis is widely supported by recent research (e.g. De Wilde et al., 2020; Lee & Dressman, 2018). However, the present study did not find any empirical evidence that out-of-class L2 contact, either interactive or noninteractive, significantly related to L2 oral proficiency. Two previous studies (Freed, 1990; Zhang et al., 2020) also had similar findings. The majority of reviewed research and the present study measured the frequency of out-of-class L2 contact using Likert-scale questionnaire items. Participants’ self-reported data may not accurately reflect their actual frequency of using L2 resources. This flaw in the data collection method may result in contradictory results.
The non-significant relationship between out-of-class L2 contact and L2 oral proficiency, on the one hand, suggested that the effect of out-of-class informal learning with L2 resources may have been traditionally overestimated. Formal classroom instruction led by teachers seems to be the more decisive factor for L2 development. Unstructured and unguided out-of-class L2 contact has been demonstrated to have no or very little effect.
On the other hand, educators and researchers need to think about how to enhance the effectiveness of out-of-class informal learning. According to prior studies, compared with noninteractive L2 contact, activities involving language production, social interaction, authentic communication were more likely to have a positive effect on L2 oral proficiency (e.g. De Wilde et al., 2020; Freed et al., 2004; Lee & Dressman, 2018). In the present study, Model 3 fit data better than Model 1, indicating that interactive and noninteractive out-of-class L2 contact played different roles in the structural relationship. In Model 3, L2 oral proficiency regressed on the interactive L2 contact with a positive coefficient (β = 0.03) but regressed on noninteractive L2 contact with a negative coefficient (β = −0.04). Although neither regression coefficient was statistically significant, the difference in coefficient signs provides additional evidence that out-of-class informal learning activities involving higher levels of interaction may be more efficient in improving L2 oral proficiency.
One important pedagogical implication is that out-of-class informal learning activities may be more effective in fostering L2 oral proficiency development when they require deep levels of processing, interaction, and language production. Foreign language teachers need to consider how to make out-of-class informal learning more interactive and cognitively demanding. For instance, teachers may consider providing guiding questions, or asking students to complete tasks like writing reflections, oral reports, pair work, and group discussion with digital or non-digital tools.
VI Conclusions
This study established a structural model (Figure 3) that was adequate at accounting for the relationships between L2 oral proficiency and its correlates (i.e. interest in learning the target language/L2 interest, perceived importance of speaking, out-of-class L2 contact). This model reveals how these variables interact with each other while affecting L2 oral proficiency.
L2 interest is central to the model. L2 learners with high levels of interest tend to perceive speaking as an important task. With perceived importance of speaking as the mediator, L2 interest has a significant and positive effect on L2 oral proficiency. L2 interest also predicts the frequency of both interactive and noninteractive out-of-class L2 contact. The influence of L2 interest on L2 oral proficiency may be larger than traditionally presumed. To foreign language educators, it is necessary to think about how to initiate and maintain students’ interest in the target language and culture.
In this study, the effect of L2 interest was completely mediated by the perceived importance of speaking. Similarly, four reviewed SEM studies (e.g. Rueda & Chen, 2005) all found that motivation orientations exerted their effects through the full or partial mediation of other variables. A systematic investigation of mediating links between motivation orientations and L2 learning outcomes would provide valuable pedagogical implications regarding how to turn motivation orientations into L2 learning gains and how to maximize motivational benefits.
Out-of-class L2 contact, as a motivated behavior in the present study, has no effect on L2 oral proficiency. When there is no instruction, scaffolding, or deep levels of analysis, motivated behaviors do not necessarily lead to higher L2 oral proficiency. How to increase the engagement, depth of analysis, and effectiveness of out-of-class informal learning deserves more research efforts. Interactive L2 contact with social media/email and noninteractive contact with traditional media function differently in the structural model. Digital tools increase the accessibility of interaction between L2 learners and facilitate contextualized language practice. They have advantages in engaging L2 learners and could play a promising role in future L2 learning.
