Metacognitive Listening Strategies and Critical Listening Attitudes as Predictors of Teacher Candidates’ Academic Listening Skills: A Structural Equation Modeling Analysis

Abstract

The objective of this study is to ascertain the relative influences and predictive relationships between metacognitive listening strategies, critical listening attitudes and academic listening skills of Turkish teacher candidates. In consideration of the ease of accessibility and economic factors, the participants were selected through the utilization of the convenience sampling method, which is appropriate when the aim is to obtain a representative sample from an accessible population. A total of 223 teacher candidates enrolled at A University, B University, C University, and D University during the 2021 to 2022 academic year were included in the study. The participants were 161 females and 62 males. A theoretical model was constructed based on the findings of the literature review, and the relationships between these variables were tested using a structural equation model. In order to evaluate the fit of the data with the model, a series of goodness-of-fit indexes were calculated, including χ², RMSEA, CFI, GFI, and AGFI. The data were collected using the Critical Listening Attitude Scale, the Academic Listening Ability Scale, and the Metacognitive Listening Strategies Instrument. A correlation analysis and path analysis were conducted to examine the data using the SPSS v22 and AMOS v24 software packages, respectively. The findings indicate that metacognitive listening strategies are positively and significantly correlated with critical listening attitudes and academic listening skills. Nevertheless, the present study has demonstrated that metacognitive listening strategies are predictive of academic listening skills, and that this is mediated by critical listening attitudes. The results obtained in this study were discussed in the context of previous research, and accordingly, recommendations were made to practitioners, researchers, and policy makers working in this field.

Plain language summary

Metacognitive listening strategies

The results showed that critical listening attitudes were positively related to questioning and making sense of what is heard, problem-solving, planning, and evaluation, interpreting what is heard, and attention dimensions. Furthermore, we found that critical listening attitudes were negatively correlated with active listening process, effective interaction, listening rules, and listening arrangement. Moreover, relationships were determined between metacognitive listening strategies, academic listening skills, and critical listening attitudes of students. The results obtained in this study were discussed based on the findings of previous research and accordingly, various suggestions were made to practitioners, researchers, and policy makers working in this field.

Keywords

metacognitive listening strategies listening attitudes critical listening structural equation modeling

Introduction

Listening, which plays an essential role in the acquisition of information in daily life, is one of the most frequently utilized skills (Nushi & Orouji, 2020). The manner in which listening is employed varies according to the circumstances and the objective of attaining a specific outcome is the primary determinant of the appropriate approach. In particular, given the considerable quantity of information to which individuals are exposed on a daily basis, the capacity to listen in a qualified and goal-oriented manner has become increasingly crucial.

The utilization of academic listening skills in learning environments can facilitate more effective learning among students. However, the development of effective academic listening skills cannot be achieved through mere self-practice; rather, it necessitates a structured and explicit approach to instruction (Graham, 2011).

People who develop critical listening skills question the information they obtain and use their mental abilities more effectively. Bourdeaud’hui et al. (2021) posit that critical listening skills encompass both discriminative and comprehensive listening abilities, yet extend well beyond this to engage multiple higher-order cognitive processes. On the other hand, metacognitive listening strategies is important since it allows one to plan and organize by being aware of their own listening process. These listening dimensions are related to effective listening. In this regard, academic listening, critical listening, and metacognitive listening strategies stand out as essential concepts associated with each other. The relative influences between these concepts, namely the main elements of listening process, are important for listening education.

The capacity to listen effectively is a crucial determinant of academic achievement and teaching efficacy among prospective teachers. The cultivation of these abilities is a crucial step in enhancing educational outcomes. The findings can inform the development of curricula and the design of teaching strategies in teacher education programs, thus contributing to the enhancement of listening skills among future educators. The existing literature includes studies on metacognitive listening (Al-Khresheh & Alruwaili, 2023; Janusik & Varner, 2020; Zarrabi, 2017), critical listening (Bourdeaud’hui et al., 2021; Zandvakili et al., 2019) and academic listening (Ellialtı & Batur, 2021; Taghizadeh & Saadatjoo, 2020). Nevertheless, there is a paucity of studies that examine the interrelationship between metacognitive strategies, critical listening attitudes and academic listening skills, particularly within the context of teacher education. This study aims to address this gap and contribute to the existing body of knowledge in this field.

Academic Listening Attitudes

Academic listening refers to one-way listening in classrooms or two-way listening between a teacher and a student in a learning environment (Lynch, 2011). Academic listening significantly differs from typical listening, characterized by an intense flow of information. In academic environments, a listener refers to their prior knowledge more frequently. This reliance on existing knowledge allows students to make connections between new information and what they already understand, facilitating deeper comprehension and retention of material. Furthermore, academic listening often requires critical thinking skills, as students must evaluate the information presented, discern its relevance, and integrate it into their own frameworks of understanding.

Despite the importance of academic listening, the restricted linguistic abilities and competencies of numerous students can render this process challenging (Graham, 2011; Taghizadeh & Saadatjoo, 2020). This necessitates the provision of supplementary assistance by instructors. First, academic listening requires some prior knowledge about the subject being heard. Second, since an academic course includes both highly important and less-important knowledge about the subject, a listener should be able to distinguish between information that is directly related to the course and that which is not. Third, during academic listening, listeners are expected to grasp the real meaning in the context. Fourth, since academic listening includes longer speaking durations and listeners could not participate in the conversation directly, listeners should endure this process. Finally, educators use gestures and facial expressions during teaching. In addition to providing additional tips about the course, these gestures and facial expressions can also include cultural background, thus they may be misunderstood by the listeners. Therefore, students should identify the true meaning of non-verbal cues during listening to a teacher (Huang, 2004).

It is of particular importance for students who have recently commenced their studies at the university level, as well as those who have done so in previous years, to develop their academic listening skills. As academic listening skills represent the primary component of communicative competence within the university setting (Alyilmaz, 2010; Taghizadeh & Saadatjoo, 2020), The development of academic listening skills is crucial for students in order to ensure their success in academic pursuits, as effective communication is a fundamental aspect of the learning process (Yıldız, 2018). The utilization of academic listening skills in learning environments enables students to maintain contact with their respective courses.

Metacognitive Listening Strategies

Metacognitive listening strategies refer to intrinsic processes that manage and control cognitive processes in listening (Melanlioğlu, 2012). The use of these strategies refers to one’s learning to listening and managing this process him/herself by being aware of the listening process (Baki, 2020).

The metacognitive approach, which includes strategies to overcome academic listening difficulties and make this process easier, has received much attention from researchers and become one of the most popular research subjects (Rahimirad & Moini, 2015). Studies conducted with effective and less-effective listeners showed that metacognitive strategies play an important role in the success of listening (Tavakoli et al., 2012). The metacognitive approach provides students with the opportunity to gain insight into their own listening deficiencies and to develop effective strategies to address these deficiencies (Chero, 2023). Goh (2008) explained some positive effects of metacognitive strategies on understanding what is heard since these strategies reduce students’ anxiety during listening and increase their self-esteem. According to Nemtchinova (2013), the utilization of metacognitive strategies, or the conscious regulation of one’s cognitive processes, enables the planning, monitoring, evaluation and reflection on the listening process. Goh (2008) argues that teaching metacognitive listening strategies can contribute to listeners in at least three ways: promote the effectiveness of listening, improve listeners’ self-esteem and motivation. Also, these strategies positively affect listening performance. Poor listeners potentially obtain a higher benefit (Maftoon & Fakhri Alamdari, 2020). Such teaching strategies can be expanded to a mentoring level to guide students in mental processes for understanding what they heard (Vandergrift, 2002). Since metacognitive strategies provide a context for interpreting, they may help listeners to identify their goals, monitor their development, and evaluate their learning outcomes. Also, metacognitive strategies can significantly improve and speed up listening comprehension (Rahimirad & Moini, 2015). According to Yang (2009), guiding listeners about the role of metacognition on listening help students handle listening to tasks more effectively and reveals the difference between successful listeners and unsuccessful ones. Teaching metacognitive listening strategies contributes to students’ listening skills at every stage.

It was reported that effective teaching of metacognitive strategies might significantly facilitate and promote listening comprehension, improve self-regulated learning by providing a context for interpreting, as well as help listeners monitor their developments and evaluate their learning outcomes (Rahimirad, 2014). Language learners can use these strategies to develop their listening perspective if they are aware of the benefits of listening strategies (Zhang & Goh, 2006). Thus, they may develop effective listening skills. Using metacognitive listening strategies, students can develop forecasting, planning, monitoring, and evaluation abilities also, they gain the ability to identify and correct their mistakes (Melanlıoğlu, 2016).

Critical Listening Skills

Thompson et al. (2004) defines critical listening as the ability to identifying themes, while acquiring new information, also comparing it with previous knowledge and re-evaluate new information using the past knowledge. Critical listening is evaluating what is heard in terms of accuracy, weaknesses, etc. by establishing a cause-effect relationship (O. Kemiksiz, 2015). Critical listening skills include the ability to evaluate speakers’ perspectives, sources used, and the main points of the conversation, accuracy, as well as determining the main ideas in the text and building connections with these ideas (Trace, 2013).

Critical listening is an ability that listening competency is shaped in the context of metacognitive skills (Trace, 2013). Critical listening certainly includes understanding however, it goes far beyond activating other cognitive abilities that imply questioning information, thus encouraging more discussion (Talalakina, 2012). Those who adopt the aforementioned points during the act of listening are demonstrably developing their abilities as critical listeners.

Critical listening also contributes to the thinking process, thus makes significant effects on one’s life. Moreover, critical listening skills play an important role in students’ learning processes both in-class and out-of-school environments. Students need critical listening skills as they should be able to accurately distinguish facts from ideas, fantasy from reality, and untrue ideas from truth during school education (Ediger, 2015).

Model and Hypotheses

Based on a literature survey, a research model given in Figure 1 was created to illustrate the relationships between metacognitive listening strategies, critical listening attitudes, and academic listening skills.

Figure 1.

Research model.

As illustrated in Figure 1, the model demonstrates the interrelationships between the three elements. The development of critical listening attitudes and the implementation of metacognitive listening strategies have the potential to influence the academic listening attitudes. The present study aims to examine the relative influences and predictive relationships between the variables of metacognitive listening strategies, critical listening attitudes, and academic listening skills among Turkish teacher candidates.

The Relationship Between Metacognitive Listening Strategies and Critical Listening Attitudes

The relationship between metacognitive listening strategies and critical listening attitudes represents a significant area of investigation within the disciplines of educational psychology and language learning. The available evidence indicates that the implementation of effective metacognitive strategies can enhance individuals’ critical thinking abilities (Can, 2021). For instance, if listeners engage in active questioning during listening, develop strategies for comprehension, and challenge the content they hear, this can reinforce their critical listening attitudes. The implementation of metacognitive listening strategies has the potential to positively influence individuals’ critical listening attitudes, thereby enhancing the overall effectiveness of the listening experience. This relationship provides a crucial foundation for the implementation of strategies aimed at enhancing listening skills in an educational setting. Hence, the first hypothesis of the study was formed as follow.

H1: Metacognitive listening strategies is positively related to critical listening attitudes.

The Relationship Between Metacognitive Listening Strategies and Academic Listening Skills

The fact that individuals using metacognitive strategies manage their listening processes more effectively contributes significantly to the development of their academic listening skills. Dep (2020) emphasizes that metacognitive strategies help listeners to focus their attention, increase comprehension and access information more deeply.

It is important for students to develop metacognitive listening skills, which include planning their actions, checking the results of their efforts, overcoming obstacles, and evaluating their progress (Baker, 2002). In addition, students can develop their academic listening skills by identifying the purpose of the lesson and building a network between the topics they listen to. These skills enable students to play a more active role in learning processes. Hence, the second hypothesis of the study was formed as follow.

H2: Metacognitive listening strategies is positively related to academic listening skills.

The Relationship Between Critical Listening Attitudes and Academic Listening Skills

The capacity to engage in critical listening is contingent upon an individual’s ability to subject information to scrutiny through questioning, analysis and evaluation during the listening process. These attitudes are of significant consequence in the cultivation of academic listening abilities. Critical listening entails not only comprehension of the information received but also the analysis and evaluation of that information (Azizoğlu & Okur, 2021). The development of academic listening skills is contingent upon the cultivation of critical listening attitudes. The adoption of these attitudes has been demonstrated to enrich students’ learning processes and increase their academic success. The ability to engage in critical listening allows individuals to not only obtain information but also to comprehend and evaluate it (Kemiksiz, 2023). Hence, the third hypothesis of the study was formed as follow.

H3: Critical listening attitudes is positively related to academic listening skills.

Mediating Role of Critical Listening Attitudes in Predicting Academic Listening Skills

The existing literature includes studies that examine metacognitive listening strategies in an academic context, with a particular focus on the relationship between these strategies and academic listening skills (Al-Khresheh & Alruwaili, 2023; Baki, 2020). Moreover, as evidenced by Ivanovska and Petkovska (2019), there is a positive correlation between critical listening attitudes and metacognitive listening strategies. Furthermore, research has been conducted on the development of critical and academic listening skills (Ediger, 2015). In this context, the fact that pre-service teachers’ critical listening attitudes are related to both metacognitive listening strategies and academic listening skills increases the possibility that critical listening attitudes act as a mediating factor between pre-service teachers’ metacognitive listening strategies and academic listening skills. In this framework, it is reasonable to posit that pre-service teachers’ critical listening attitudes will function as a mediating variable between metacognitive listening strategies and academic listening skills. Hence, the forth hypothesis of the study was formed as follow.

H4: Critical listening attitudes mediate the predictive power of metacognitive listening strategies on academic listening skills.

Theoretical Framework

This section will provide a concise overview of the theoretical framework underpinning our research approach, with particular reference to the existing literature on the subject. Given that our research concerns listening skills, we sought to identify literature on the cognitive aspect of listening. In this context, metacognitive listening, critical listening and academic listening skills can be associated with cognitive load theory. This theory addresses the learning of intricate cognitive tasks that emerge from the quantity and interrelation of information that must be processed concurrently prior to the commencement of learning. It underscores the significance of cognitive processes (Paas & Ayres, 2014). As a consequence of the evolution of cognitive structures, the load on working memory is reduced (Cowan, 2014). The cultivation of listening abilities can also facilitate the reduction of this load.

The objective of cognitive load theory is to develop effective teaching methods that will enable individuals to utilize their limited information processing capacity effectively (Sweller, 2024). It is therefore of great importance to examine the relationship between listening, critical listening and academic listening skills in order to optimize students’ cognitive capacities and improve their learning processes. An understanding of this relationship can facilitate the development of effective teaching strategies and assist pre-service teachers in acquiring more proficient listening skills.

Methodology

Research Procedure and Sample

A correlational survey study was designed to determine the effects of Turkish teacher candidates’ metacognitive listening strategies and critical listening attitudes on academic listening skills, as well as relative influences and predictive relationships between variables. A theoretical model was created based on the literature survey (Figure 1) and the relationships between these variables were tested by a Structural Equation Model (hereinafter referred to as SEM). SEM is used to examine the covariance structure between observed variables and linear structural relationships between all variables of a model (Çelik & Yilmaz, 2013).

Considering easy accessibility and economic aspects, the participants were selected using the convenience sampling method. The study recruited a total of 223 teacher candidates enrolled at four different universities during the 2021 to 2022 academic year: A University, B University, C University, and D University. Participants were 161 (72.2%) females and 62 (27.8%) males. A total of 58 teacher candidates were first-year college students, 58 were second-year, 64 were third-year, and 43 were fourth-year college students.

The requisite ethical permissions were obtained for the study. Following the participants’ informed consent to the study’s purpose, the necessary permissions were obtained from the participants. The data were collected by the researchers using Google Forms over the course of one semester. The participants were recruited on a voluntary basis to take part in the study.

Research Instruments

The data were collected from pre-service teachers through the “Critical Listening Attitude Scale” to measure critical listening attitudes, the “Academic Listening Ability Scale” to measure academic listening skills and the “Metacognitive Listening Strategies Instrument” to measure metacognitive listening strategies.

Critical Listening Attitude Scale

The Critical Listening Attitude Scale developed by Taskin (2017) measures teacher candidates’ critical listening attitudes. The scale uses a 5-point Likert rating, Strongly disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), Strongly agree (5). The results showed that this 20-item scale consists of three sub-scales, namely “making sense of what is heard (comprehension),”“questioning what is heard (analysis),” and “interpreting what is heard (evaluation).” To determine the reliability of the scale, the Cronbach’s Alpha coefficient was examined and it was calculated as .81 for the whole scale.

In the current paper, confirmatory factor analysis (hereinafter referred to as CFA) is performed to examine validity and Cronbach’s Alpha coefficient was calculated for reliability. The CFA results confirmed the 3-factor structure of the scale (χ²: 1.82, RMSEA = 0.06, GFI = 0.88, AGFI = 0.82, NFI = 0.88, CFI = 0.88). Also, the reliability analysis for the whole scale showed that Cronbach’s Alpha coefficient is .90.

Academic Listening Ability Scale

The Academic Listening Ability Scale developed by Yıldız (2018) measures teacher candidates’ academic listening ability levels. The items are assessed by using a five-point Likert scale “never” (1), “rarely” (2), “sometimes” (3), “usually” (4), and “always” (5). This valid and reliable scale consists of four sub-scales, namely “active listening process,”“effective interaction,”“listening arrangement,” and “listening rules” with a total of 37 items. The fit indexes were calculated as χ²: 2.31, RMSEA = 0.05, CFI = 0.96, NFI = 0.93, AGFI = 0.86, GFI = 0.91. For the consistency of the whole scale, the Cronbach’s Alpha coefficient was calculated as .92.

In the current study, CFA is conducted for validity analysis and Cronbach’s Alpha coefficient was calculated for reliability. The CFA results confirmed the 4-factor structure of the scale (χ²: 2.31, RMSEA = 0.07, GFI = 0.83, NFI = 0.85, CFI = 0.86). Furthermore, the Cronbach’s Alpha coefficient for the whole scale is calculated as .90.

Metacognitive Listening Strategies Instrument

The Metacognitive Listening Strategies Instrument (MLSI) was developed by Janusik and Keaton (2011) to measure metacognitive listening strategies in native language. The scale consists of 11 items composed of three sub-scales, namely, attention (4 items), problem-solving (3 items), and planning and evaluation (4 items). The study was conducted with college students (cited in Okur & Azizoğlu, 2016).

In the current study, the Turkish version of the MLSI adapted by Okur and Azizoğlu (2016) was used. First, the linguistic equivalence of the scale was obtained. The scale uses a 5-point Likert rating Strongly disagree (1), Disagree (2), Neither agree or disagree (3), Agree (4), Strongly agree (5). The results yielded three factors, namely attention, problem-solving, and planning and evaluation with a total of 11 items. According to the CFA for the MLSI, goodness-of-fit indexes were found to be χ²: 1.71, RMSEA: 0.70, AGFI: 0.87, GFI: 0.92, CFI: 0.98, NFI: 0.95, NNFI: 0.97. Based on the obtained results, it can be argued that the goodness-of-fit indexes of the model developed for the MLSI were acceptable. The Cronbach’s alpha coefficients were calculated for reliability assessment. Accordingly, Cronbach’s alpha coefficients were found to be .79 for problem-solving, .70 for planning and evaluation, .75 for attention, and .83 for the whole scale. Based on these values the scale proved to be reliable.

In the current study, CFA is performed for validity analysis and Cronbach’s alpha coefficient was examined for reliability. The results of the CFA confirmed the 3-factor structure (χ²: 2.36, RMSEA: 0.07, AGFI: 0.87, GFI: 0.92, CFI: 0.89, NFI: 0.89). Also, the reliability analysis for the whole scale showed that Cronbach’s alpha coefficient is .90.

Analysis of the Data

SPSS v22 and AMOS v24 statistical software packages were employed to analyze the data. Structural equation modeling was used to examine the relationships between metacognitive listening strategies, critical listening attitudes, and academic listening skills. To evaluate the fit of the data with the model, χ², RMSEA, CFI, GFI, and AGFI goodness-of-fit indexes were calculated. CFI, GFI, and AGFI values close to 1 and an RMSEA value less than 0.08 are considered acceptable fit (Karagöz, 2016).

The normality of the data was tested using skewness and kurtosis coefficients. The skewness and kurtosis coefficients of the data were found to fall within the specified ranges (±2) for metacognitive listening strategies (−0.12; 0.54), critical listening attitudes (−0.46; 1.88) and academic listening skills (−0.37; 0.97). Subsequent to these analyses, it was concluded that the data were normally distributed (George & Mallery, 2019).

The presence of multicollinearity was assessed through the evaluation of the Variance Inflation Factor (VIF), tolerance, and the Pearson Product-Moment Correlation Coefficient. In this study, it was determined that the variance inflation factor (VIF) value exhibited a range of 1.844, while the tolerance value demonstrated a range of 0.542. In light of these findings, it can be concluded that multicollinearity is not a concern in this study, as the VIF value is less than 5 and the tolerance value is greater than 0.10 (Kline, 2016).

A regression analysis based on the Bootstrap method was conducted to ascertain whether there is a mediating role in the relationship between Turkish language teacher candidates’ critical listening attitudes, metacognitive listening strategies and academic listening skills. The analyses were conducted using the Process macro program, developed by Hayes (2009). The bootstrap technique was employed in the applied analysis, with 5,000 resampling areas selected. The variables of university, gender and grade level are to be regarded as control variables. In mediation effect analyses employing the bootstrap technique, it is essential that the 95% confidence interval (CI) values derived from the data obtained in the analysis do not contain the value of zero in order to verify the hypotheses of the research and base a conclusion (Gürbüz, 2021).

Structural Equation Modeling (SEM)

As defined by Kline (2016), SEM analysis is a multivariate regression model that elucidates the causal relationships between the variables under investigation. As outlined by Schreiber et al. (2006), SEM represents a methodology through which the relationship between two or more observed variables may be examined with a view to making inferences about the latent variables involved. SEM analysis enables the examination of diverse variables associated with complex models. The model proposal was subjected to an examination through the application of path analysis to the data obtained from the study. The SEM analysis enabled the investigation of whether there was a significant causal relationship between the variables of the proposed model and the study hypotheses. The findings of the analysis were assessed in accordance with the model fit assumptions and a range of good fit indices.

Findings

The relationship between Turkish teacher candidates’ metacognitive listening strategies, critical listening attitudes and academic listening skills. The findings are presented in Table 1.

Table 1.

Correlations Between Variables.

Variables	PS	At	PE	A	C	E	AL	EI	LA	LR
PS	1	0.38**	0.52**	0.52**	0.47**	0.31**	0.55**	0.41**	0.34**	0.18**
At		1	0.43**	0.26**	0.17*	−0.03	0.14*	0.21**	0.35**	0.07
PE			1	0.41**	0.27**	0.11	0.35**	0.30**	0.33**	0.02
A				1	0.61**	0.35**	0.59**	0.42**	0.37**	0.28**
C					1	0.28**	0.64**	0.31**	0.29**	0.38**
E						1	0.26**	0.23**	0.19*	0.24
AL							1	0.53**	0.43**	0.34**
EI								1	0.45**	0.11
LA									1	0.36**
LR										1

Note. PS = problem solving; At = attention; PE = planning and evaluation; A = questioning what is heard (analysis); C = making sense of what is heard (comprehension); E = interpreting what is heard (evaluation); AL = active listening process; EI = effective interaction; LA = listening arrangement; LR = listening rules.

p ;< .05. **p ;< .01.

As seen in Table 1, the majority of the variables exhibit a positive and moderate relationship with one another. However, an analysis of the relationships between the variables revealed that there were positive and moderate relationships between problem solving and planning-evaluation (r ;= 0.52; p ;< .01), attention (r ;= 0.52; p ;< .01) and active listening process (r ;= 0.55; p < .01) dimensions. Furthermore, positive moderate relationships were identified between the dimensions of questioning (Analysis; r ;= 0.59; p ;< .01) and comprehension (r ;= 0.61; p ;< .01). Furthermore, positive moderate relationships were identified between the active listening process dimension and the dimensions of questioning (Analysis; r ;= 0.59; p ;< .01), comprehension (r ;= 0.64; p ;< .01) and active participation (r ;= 0.53; p ;< .01). Correlation values between 0 and .29 are indicative of a low correlation, between .30 and .69 of a moderate correlation, and between .70 and 1.00 of a high correlation (Büyüköztürk, 2006).

Following an examination of the relationships between the predicted and predictor variables, the predictive role of academic listening skills on critical listening attitudes and metacognitive listening strategies was investigated through path analysis. The model under examination is illustrated in Figure 2.

Figure 2.

Path diagram for the prediction of Turkish teacher candidates’ academic listening skills.

The goodness of fit indices obtained from the structural equation modeling to determine the effect of critical listening attitudes and metacognitive listening strategies on academic listening skills were χ²: 3.45, RMSEA: 0.07, CFI: 0.90, GFI: 0.91 and AGFI: 0.88. The standardized regression coefficients obtained for the structural model are given in Table 2.

Table 2.

Mathematical Model for Structural Equations Between Variables.

Path		Path coefficient (β)	Standardized estimate	Standard error (SE)	Critical ratio (CR)	Significance (p)
Metacognitive listening strategies	Critical listening attitudes	0.72	0.46	0.09	4.76	**
Metacognitive listening strategies	Academic listening skills	0.14	0.11	0.09	1.22	**
Critical listening attitudes	Academic listening skills	0.77	0.98	0.25	3.81	**

Significant at p < .01.

As seen in Table 2, critical listening attitudes were positively and significantly predicted by metacognitive listening strategies (β: .72; p ;< .01) and the results were determined to be statistically significant (p ;< .01). Accordingly, hypothesis H1 (Metacognitive listening strategies is positively related to critical listening attitudes.) is accepted. Furthermore, we found that academic listening skills were positively and significantly predicted by metacognitive listening strategies (β: .14, p ;< .01) and the results were determined to be statistically significant. Therefore, hypothesis H2 (Metacognitive listening strategies is positively related to academic listening skills.) is accepted. Moreover, it was found that critical listening skills were positively and significantly predicted by academic listening skills (β: .77, p ;< .01) and the results were determined to be statistically significant (p ;< .01). Based on this finding, hypothesis H3 (Critical listening attitudes is positively related to academic listening skills.) is accepted.

In order to statistically test the significance of the mediating effect of critical listening attitudes, a bootstrapping analysis was conducted. The results of this analysis are presented in Figure 3, which shows that there is a statistically significant mediating effect.

Figure 3.

The analysis results regarding the mediating role of the critical listening between metacognitive listening strategies and academic listening skills (N = 223).

A statistical analysis of the findings presented in Figure 3 reveals a statistically significant mediating role of critical listening attitudes in the relationship between metacognitive listening strategies and academic listening skills. This finding corroborates the fourth hypothesis, namely that critical listening attitudes act as a mediator in the relationship between metacognitive listening strategies and academic listening skills. In other words, the difference between the total and direct predictive power of pre-service teachers’ critical listening attitudes on academic listening skills was statistically significant ([Indirect Effects = 0.72; p ;= 00] and 95% Confidence Interval [0.6176, 1.257]).

Conclusion, Discussion, and Interpretations

This study examined the relationship between the dimensions of listening metacognitive strategies (attention, planning-evaluation and problem solving) identified by Okur and Azizoğlu (2016), the dimensions of critical listening attitudes identified by Taşkın (2017; understanding/comprehending what is listened, questioning/analyzing what is listened, interpreting/evaluating what is listened) and the dimensions of academic listening skills identified by Yıldız (2018; active listening process, active participation, listening arrangement and listening rules). These dimensions are found to be positively and moderately related to one another. This finding is consistent with that reported in previous studies. A substantial body of research has demonstrated that listening plays a pivotal role in the development of a range of fundamental competencies, including questioning, comprehension, problem-solving, planning and evaluation, interpretation, and attention (Beyreli et al., 2017; Boyd, 2005; Güneş, 2021; O. Kemiksiz, 2015; Maftoon & Fakhri Alamdari, 2020; Ungan, 2007).

The listening dimensions, the relationships between which were established in the course of the research, form part of the learning process. In particular, the development of listening metacognitive strategies, critical listening attitudes and academic listening skills is essential for the success of the phases of Self-Regulated Learning Models. This theory is linked to students who become critical and creative thinkers, problem solvers, and adaptive, lifelong learners who are able to work and learn both independently and collaboratively (Perry et al., 2017). The integration of effective listening strategies in educational environments is regarded as a crucial step in enhancing learning outcomes.

The initial hypothesis, which posited that critical listening attitudes predict metacognitive listening strategies significantly and positively, was accepted as a result of the research findings. This finding is consistent with those of previous studies. The development of critical listening is shaped by the utilization of metacognitive strategies. As posited by Bourdeaud’hui et al. (2021), critical listening can be achieved through metacognitive listening, and thus is inextricably linked to the concept of metacognition (Trace, 2013). The use of metacognitive strategies has been shown to have a positive effect on listening skills in numerous studies (Field, 2003; Katrancı & Yangın, 2013; Maftoon & Fakhri Alamdari, 2020; Memiş & Erdem, 2013; Selamat & Sidhu, 2013; Vandergrift & Tafaghodtari, 2010; Zheng, 2018). The utilization of metacognitive strategies enables students to enhance their listening efficacy. Such strategies assist students in developing their listening awareness and in demonstrating critical listening skills with greater facility. In particular, metacognitive awareness in the stages of planning, monitoring and evaluating the listening process enables students to identify the difficulties they encounter during listening and develop appropriate solutions to these difficulties. Consequently, students not only comprehend the information they receive, but also have the opportunity to engage in a more profound learning experience through the processes of analysis and synthesis. This enhances their capacity for lifelong learning and contributes to their academic success.

The second hypothesis, which stated that academic listening skills significantly and positively predict metacognitive listening strategies, was accepted. This finding is consistent with that of previous reports. As Katrancı and Yangın (2013) have demonstrated, metacognitive strategies have a beneficial impact on students’ academic listening abilities. Rahimirad and Moini (2015) highlight that the instruction of metacognitive strategies has the potential to markedly enhance students’ abilities to comprehend academic course content. Nevertheless, a study conducted by Melanlıoğlu (2019) indicates that university students perceive themselves to be lacking in academic listening proficiency. The ability to listen academically enables students to plan their actions, evaluate the outcomes of their efforts, overcome difficulties, and review their progress. This process enables them to demonstrate progress in the utilization of metacognitive strategies. Consequently, the efficacious implementation of metacognitive strategies enhances students’ academic listening abilities and optimizes their learning processes. This further substantiates the necessity for educators to integrate metacognitive strategies into their curricula. The second hypothesis (that academic listening skills significantly and positively predict metacognitive listening strategies) was accepted. This finding is consistent with previous reports. According to Katrancı and Yangın (2013), metacognitive strategies contribute positively to students’ academic listening abilities.

The third hypothesis of the study, which posited that academic listening skills significantly and positively predict critical listening attitudes, was accepted. This finding is consistent with those of previous studies. The development of critical listening skills is an important process that enables individuals to adopt a critical perspective (Çarkıt, 2019). Accordingly, the development of a critical perspective can facilitate the acquisition of academic listening skills (Aytan, 2016). Furthermore, critical listening contributes to the development of academic listening skills, as it encourages the identification of relationships between facts and the establishment of cause-and-effect relationships. It can therefore be posited that students who have acquired critical listening skills will be more successful in the domain of academic listening. In this context, the development of critical listening skills in educational environments has been demonstrated to enhance students’ academic achievement while simultaneously reinforcing their capacity for critical thinking. Accordingly, the cultivation of critical listening attitudes is regarded as a pivotal strategy for enhancing the caliber of educational processes.

The fourth hypothesis of the study, which posited that critical reading attitudes mediate the predictive power of metacognitive listening strategies on academic listening skills, was accepted. In this context, the results of the study contribute to the academic discourse on the role of metacognitive listening strategies and critical listening attitudes among pre-service teachers in supporting academic listening skills (Al-Khresheh & Alruwaili, 2023; Baki, 2020; Ivanovska & Petkovska, 2019; Taghizadeh & Saadatjoo, 2020; Yıldız, 2018). A review of the literature reveals no studies that demonstrate a direct effect of critical listening attitudes on academic listening. Nevertheless, this interaction facilitates the acquisition of essential competencies, including enhanced comprehension, active engagement, and effective communication. It can therefore be concluded that the development of critical listening skills represents a crucial factor in the academic achievement and professional development of pre-service teachers. The present study has demonstrated that critical listening attitudes act as a mediator in the relationship between metacognitive listening strategies and academic listening skills. In the existing literature, researchers have put forth the theoretical proposition that metacognitive listening strategies can exert an influence on academic listening (Baker, 2002; Dep, 2020). In this context, it is understood that pre-service teachers who employ metacognitive listening strategies perceive critical listening attitudes as a prerequisite for academic listening.

These findings underscore the necessity for educators to integrate a greater number of metacognitive and critical listening strategies into their curricula. The cultivation of these competencies in students will enhance their academic performance and facilitate their engagement in lifelong learning. The integration of these strategies represents a crucial step in enhancing the effectiveness of the education system.

In light of the findings of the study, the following recommendations were put forth:

It is recommended that students be provided with comprehensive instructions prior to engaging in academic listening activities within the classroom setting. In particular, students should be made aware of the requisite procedures at each stage of this type of listening.

Prior to the implementation of academic listening activities, it is incumbent upon the teacher and students to ensure that the listening environment is conducive to effective listening. It is also crucial that students play an active role in the organization of the listening environment.

Academic listening activities should be designed with the objective of fostering active student participation. In order to ensure optimal engagement with this type of listening, it is essential to provide students with clear guidance on the expected standards of conduct.

The incorporation of metacognitive strategies by educators on a more regular basis within the classroom setting can facilitate the development of students’ critical and academic listening abilities. The guidance of experienced teachers on the utilization of metacognitive listening strategies by those in the early stages of their teaching careers can facilitate the effective deployment of these strategies.

It is recommended that pre-service teachers be encouraged to gain experience by applying metacognitive listening strategies in real lesson environments. This is important in terms of putting theoretical knowledge into practice.

In order for pre-service teachers to develop metacognitive listening strategies, it is recommended that critical listening attitudes should be given more space in education programs. Organizing workshops and seminars to develop critical listening skills will enable pre-service teachers to actively practise these skills.

Limitations and Future Research

Despite the fact that data were collected from four different universities in Turkey, there was an imbalance in the distribution of male and female participants, which may have introduced a certain degree of bias into the study. Given the demographic structure of teacher education programs in Turkey, it is notable that women constitute a larger proportion of these programs. It is possible that the higher proportion of female participants in the study may be attributed to this factor. The use of convenience sampling may have resulted in a selection bias due to the greater accessibility of female participants. In future research, alternative sampling methods may be employed in order to achieve a more balanced gender representation.

A pilot study was not conducted as part of this research project. The measurement tools employed in this study have been validated and tested for reliability in previous studies. Furthermore, due to time and resource constraints, it was not feasible to conduct a pilot test. However, the sample size of the study was deemed sufficient, and it was assumed that direct data collection would be adequate. This situation is acknowledged as a limitation of the study.

Further research in the literature examining the effect of critical listening attitudes on academic listening would enhance the knowledge in this field. Studies could be conducted on how this interaction is manifested in different learning environments.

Footnotes

ORCID iDs

Faruk Polatcan

Nurşat Biçer

Onur Er

Ethical Approval

All procedures performed in studies involving human participants were conducted in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Approval was granted by the Ethics Committee of Sinop University (Ethical Approval No: 08.11.2021-131).

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

Al-Khresheh

M. H.

Alruwaili

S. F.

(2023). Metacognition in listening comprehension: Analyzing strategies and gender differences among Saudi EFL University students. Cogent Social Sciences, 10(1), 1–20. https://doi.org/10.1080/23311886.2023.2291954

Alyilmaz

(2010). Problems of Turkish teaching. Turkish Studies, 5(3), 728–749. https://doi.org/10.7827/TurkishStudies.1629

Aytan

(2016). Evaluation of the 2006 and 2015 Turkish education program in secondary school curriculum in Turkey in terms of critical thinking (EJ1097401). Journal of Education and Learning, 5(2), 38–46. https://files.eric.ed.gov/fulltext/EJ1097401.pdf

Azizoğlu

N. İ.

Okur

(2021). Examining the scope of critical listening skill. RumeliDE Journal of Language and Literature Studies, 22(24), 327–337. https://doi.org/10.29000/rumelide.898553

Baker

(2002). Metacognition in comprehension instruction. In Block

Pressley

(Eds.), Comprehension instruction: Research-based best practices (pp. 77–95). Guilford Press.

Baki

(2020). An evaluation of listening metacognitive strategies of prospective Turkish teachers: The case of Recep Tayyip Erdogan University. Journal History School, 47(2), 2657–2683. https://doi.org/10.29228/Joh.29344

Beyreli

Çetindağ

Celepoglu

(2017). Written and oral experisson. Pegem Academy.

Bourdeaud’hui

Aesaert

van Braak

(2021). Exploring the relationship between metacognitive awareness, motivation, and L1 students’ critical listening skills. Journal of Education and Research, 114(1), 40–51. https://doi.org/10.1080/00220671.2021.1872474

Boyd

F. A.

(2005). Critical listening. In Rivera

I. G.

Muñoz Gutiérrez

(Eds.), Towards greater professionalization in language teaching (pp. 79–90). ANUPI.

10.

Büyüköztürk

. (2006). Data analysis handbook. Pegem Academy.

11.

Can

(2021). Examining the relationship between Turkish teacher candidates’ metacognitive learning strategies and critical listening attitudes (EJ1297475). Shanlax International Journal of Education, (S1), 145–153. https://files.eric.ed.gov/fulltext/EJ1297475.pdf

12.

Çarkıt

(2019). The importance and place of critical listening/monitoring in Turkish education. RumeliDE Journal of Language and Literatüre Studies, 16(16), 180–193. https://doi.org/10.29000/rumelide.618150

13.

Çelik

H. E.

Yilmaz

(2013). Structural equation modeling with Lisrel 9.1 basic concepts-applications-programming. Anı.

14.

Chero

C. A. C.

(2023). The impact of metacognitive instruction on EFL low-level learners’ listening performance and metacognitive awareness. International Journal of Instruction, 16(2), 291–306. https://doi.org/10.29333/iji.2023.16217a

15.

Cowan

(2014). Working memory underpins cognitive development, learning, and education. Educational Psychology Review, 26(2), 197–223. https://doi.org/10.1007/s10648-013-9246-y

16.

Dep

(2020). An exploration of the relationships between students’ listening skills, self-rated academic listening, and their metacognitive awareness (Publication No. 14405) [Master’s thesis, Memorial University of Newfoundland]. Memorial University Research Repository.

17.

Ediger

(2015). Listening in the language arts (EJ1095695). Reading Improvement, 52(2), 69–71. https://eric.ed.gov/?id=EJ1095695

18.

Ellialtı

Batur

(2021). Academic listening skills of the international university students in Turkey (EJ1319128). International Journal of Education and Literacy Studies, 9(3), 20–28. https://files.eric.ed.gov/fulltext/EJ1319128.pdf

19.

Field

(2003). Promoting perception: Lexical segmentation in L2 listening. ELT Journal, 57(4), 325–334. https://doi.org/10.1093/elt/57.4.325

20.

George

Mallery

(Eds.). (2019). IBM SPSS statistics 26 step by step: A simple guide and reference (16th ed.). Routledge.

21.

Goh

(2008). Metacognitive instruction for second language listening development: Theory, practice and research implications. Regional Language Centre Journal, 39(2), 188–213. https://doi.org/10.1177/0033688208092184

22.

Graham

(2011). Self-efficacy and academic listening. Journal of English for Academic Purposes, 10(2), 113–117. https://doi.org/10.1016/j.jeap.2011.04.001

23.

Güneş

(2021). Turkish teaching approaches and models. Pegem Academy.

24.

Gürbüz

(Ed.). (2021). Sosyal bilimlerde aracı ve düzenleyici etki analizleri ibm spss process makro uygulamalı, örnek veri setleri [Mediator and moderator effect analysis in social sciences IBM SPSS process macro applied, sample data sets] (2nd ed.). Seçkin.

25.

Hayes

A. F.

(2009). Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Communication Monographs, 76(4), 408–420. https://doi.org/10.1080/03637750903310360

26.

Huang

(2004). Voices from Chinese students: professors’ use of english affects academic listening (EJ704954). College Student Journal, 38(2), 212. https://eric.ed.gov/?id=EJ704954

27.

Ivanovska

Petkovska

(2019). The relationship between critical thinking ability and metacognitive listening strategies of EFL learners. The International Journal of Applied Language Studies and Culture, 2(1), 23–30. https://doi.org/10.34301/alsc.v2i1.15

28.

Janausik

Keaton

S. A.

(2011). Listening metacognitions: Anothor key to teaching listening? Listening Education, (2), 33-44. https://www.academia.edu/26042904/Listening_Metacognitions_Another_Key_to_Teaching_Listening

29.

Janusik

L. A.

Varner

(2020). (Re)discovering metacognitive listening strategies in L1 contexts: What strategies are the same in the L1 and L2 context? International Journal of Listening, 36(3), 175–186. https://doi.org/10.1080/10904018.2020.1833724

30.

Karagöz

(2016). SPSS and AMOS 23 applied statistical analysis. Nobel.

31.

Katrancı

Yangın

(2013). Effects of teaching metacognition strategies to listening omprehension skills and attitude toward listening. Adıyaman University Journal of Social Sciences Institute, 11(11), 733–771. https://doi.org/10.14520/adyusbd.429

32.

Kemiksiz. (2023). Attitudes of pre-service teachers towards critical listening. e-International Journal of Educational Research, 14(1), 245–264. https://doi.org/10.19160/e-ijer.1208464

33.

Kemiksiz

(2015). Critical listening in Turkish education. Erzincan University Journal of Social Sciences Institute, 9(2), 299–316. https://dergipark.org.tr/tr/pub/erzisosbil/issue/45172/565562

34.

Kline

R. B.

(Ed.). (2016). Principles and practice of structural equation modeling (4th ed.). Guilford Press.

35.

Lynch

(2011). Academic listening in the 21st century: Reviewing a decade of research. Journal of English for Academic Purposes, 10(2), 79–88. https://doi.org/10.1016/j.jeap.2011.03.001

36.

Maftoon

Fakhri Alamdari

(2020). Exploring the effect of metacognitive strategy ınstruction on metacognitive awareness and listening performance through a process-based approach. International Journal of Listening, 34(1), 1–20. https://doi.org/10.1080/10904018.2016.1250632

37.

Melanlioğlu

(2012). Use of metacognition strategies in measurement and evaluation studies oriented with listening skills. Turkish Studies, 7(1), 1583–1595. https://doi.org/10.7827/TurkishStudies.2924

38.

Melanlıoğlu

(2016). A metacognitive rubric for Turkish learners’ listening skills. Journal of Erzincan University Faculty of Education, 18(2), 1206–1229. https://doi.org/10.17556/jef.20880

39.

Melanlıoğlu

(2019). A study on university students’ academic listening skills: When will the class be over? I am bored! Journal of Language and Literature Studies, 20(4), 311–348. https://doi.org/10.30767/diledeara.635543

40.

Memiş

M. R.

Erdem

M. D.

(2013). Methods/usage features that are used in foreign language teaching and critics. Turkish Studies, 8(9), 297–318. https://doi.org/10.7827/TurkishStudies.5089

41.

Nemtchinova

(2013). Teaching listening. TESOL.

42.

Nushi

Orouji

(2020). Investigating EFL teachers’ views on listening difficulties among their learners: The case of Iranian context. Sage Open, 10(2), 1–16. https://doi.org/10.1177/2158244020917393

43.

Okur

Azizoğlu

N. İ

. (2016). Metacognitive listening strategies instrument: Validity and reliability study. Journal of Mehmet Akif Ersoy University Faculty of Education, 40(40), 113–124. https://doi.org/10.21764/efd.04871

44.

Paas

Ayres

(2014). Cognitive load theory: A broader view on the role of memory in learning and education. Educational Psychology Review, 26(2), 191–195. https://doi.org/10.1007/s10648-014-9263-5

45.

Perry

N. E.

Yee

Mazabel

Lisaingo

Maatta

(2017). Using self-regulated learning for creating inclusive classrooms for ethnically and linguistically diverse in Canada. In Cabrera

N. J.

Leyendecker

(Eds.), Handbook of positive development of minority and youth (pp. 361–385). Springer.

46.

Rahimirad

(2014). The impact of metacognitive strategy ınstruction on the listening performance of university students. Procedia - Social and Behavioral Sciences, 98, 1485–1491. https://doi.org/10.1016/j.sbspro.2014.03.569

47.

Rahimirad

Moini

M. R.

(2015). The challenges of listening to academic lectures for eap learners and the ımpact of metacognition on academic lecture listening comprehension. Sage Open, 5(2), 1–9. https://doi.org/10.1177/2158244015590609

48.

Schreiber

J. B.

Nora

Stage

F. K.

Barlow

E. A.

King

(2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. Journal of Education and Research, 99(6), 323–338. https://doi.org/10.3200/JOER.99.6.323-338

49.

Selamat

Sidhu

G. K.

(2013). Enhancing listening comprehension: The role of metacognitive strategy ınstruction (MetSI). Procedia - Social and Behavioral Sciences, 90(InCULT 2012), 421–430. https://doi.org/10.1016/j.sbspro.2013.07.111

50.

Sweller

(2024). Cognitive load theory and individual differences. Learning and Individual Differences, 110, 1–5. https://doi.org/10.1016/j.lindif.2024.102423

51.

Taghizadeh

Saadatjoo

(2020). Engineering students’ needs for listening scaffolding strategies and their perceptions of instructors’ performance in an academic listening course. Research in Science & Technological Education, 39(3), 368–392. https://doi.org/10.1080/02635143.2020.1764925

52.

Talalakina

E. V.

(2012). Audiobook in advanced ESL classroom: Developing critical listening. https://conference.pixel-online.net/conferences/ICT4LL2012/common/download/Paper_pdf/177-IBT34-FP-Talalakina-ICT2012.pdf

53.

Taşkin

(2017). Critical listening attitude scale’s validity and reliability study for prospective teachers. International Journal of Language Academy, 5(2), 116–128. https://doi.org/10.18033/ijla.3486

54.

Tavakoli

Hashemi Shahraki

Rezazadeh

(2012). The relationship between metacognitive awareness and EFL listening performance: Focusing on IELTS higher and lower scorers. The Journal of Language Teaching and Learning, 2(3), 24–37. https://dergipark.org.tr/tr/download/article-file/209057

55.

Thompson

Leintz

Nevers

Witkowski

(2004). The integrative listening model: An approach to teaching and learning listening. Journal of General Education, 53(3), 225–246. https://doi.org/10.1353/jge.2005.0011

56.

Trace

(2013). Designing a task based critical listening construct for listening assessment. https://www.hawaii.edu/sls/wp-content/uploads/2014/09/Trace-Jon.pdf

57.

Ungan

(2007). Listening training. In Kırkkılıç

Akyol

(Eds.), Turkish teaching in primary education (pp. 149–163). Pegem Academy.

58.

Vandergrift

(2002). It was nice to see that our predictions were right’: Developing metacognition in L2 listening comprehension. The Canadian Modern Language Review, 58(4), 555–575. https://doi.org/10.3138/cmlr.58.4.555

59.

Vandergrift

Tafaghodtari

M. H.

(2010). Teaching L2 students how to listen does make a difference: An empirical study. Language Learning, 60(2), 470–497. https://doi.org/10.1111/j.1467-9922.2009.00559.x

60.

Yang

(2009). A study of metacognitive strategies employed by English listeners in an EFL setting. International Education Studies, 2(4), 134–139. https://doi.org/10.5539/ies.v2n4p134

61.

Yıldız

(2018). Academic listening ability scale: Availability and reliability study. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 18(2), 1210–1230. https://doi.org/10.17240/aibuefd.2018..-400747

62.

Zandvakili

Washington

Gordon

E. W.

Wells

Mangaliso

(2019). Teaching patterns of critical thinking: The 3CA model—concept maps, critical thinking, collaboration, and assessment. Sage Open, 9(4), 1–15. https://doi.org/10.1177/2158244019885142

63.

Zarrabi

(2017). Investigating the relationship between learning style and metacognitive listening awareness. International Journal of Listening, 34(1), 21–33. https://doi.org/10.1080/10904018.2016.1276458

64.

Zhang

Goh

C. C. M.

(2006). Strategy knowledge and perceived strategy use: Singaporean students’ awareness of listening and speaking strategies. Language Awareness, 15(3), 199–219. https://doi.org/10.2167/la342.0

65.

Zheng

(2018). The metacognitive strategy in English listening comprehension. Theory and Practice in Language Studies, 8(2), 226–231. https://doi.org/10.17507/tpls.0802.07