Sage Journals: Discover world-class research

Abstract

This two-study research built and validated the new questionnaire “LO-COMPASS: Learning Orientation-Cognition Metacognition Participation Assessment” aimed at capturing the interplay between cognitive, metacognitive, and affective-motivational learning attributes in middle school students’ learning orientations proved to be significant predictors of middle school students’ achievements. In Study 1 (N = 212) an Exploratory Factorial Analysis (EFA) was carried out to investigate the factorial latent structure of the questionnaire, while in Study 2 (N = 233) this structure was checked by a Confirmatory Factorial Analysis (CFA). The goodness of fit indices resulting from the CFA were sufficiently good overall: CFI = 0.89, TLI = 0.92, RMSEA = 0.04, SRMR = 0.06. Furthermore, in Study 2 the convergent/discriminant validity of the latent factors of the questionnaire and their association with school achievement were verified. LO-COMPASS factors including an interplay of cognitive, metacognitive, and affective-motivational attributes of learning orientations are discussed within the framework of the existing literature and the practical implications are delineated.

Keywords

scale development scale validation learning orientation cognitive aspects metacognitive aspects affective aspects middle school

Introduction

The rate of students’ early school leaving, and low school achievements indicate the scarce functioning of an education system and negatively impact on the sociocultural and economic development of worldwide countries (Ministry of Education, University and Research [MIUR], 2018; Organisation for Economic Co-operation and Development[OECD], 2018). Students’ ability to shape and regulate the learning process, emotions, and motivation are important factors to prevent these undesirable outcomes. Data from international comparative assessment (OECD, 2016) show that on average, about 20% of students in OECD countries fail to reach the baseline level of proficiency in reading and more than 40% in 21 countries sat under the baseline level of proficiency in mathematics. Underachievement is a phenomenon associated with dropout beyond compulsory schooling, which in turn leads to difficulties in finding employment (60% of people with below upper-secondary education vs. over 80% of tertiary-educated adults) (OECD, 2014). The issues of dropping out and the negative consequences of poor performance are particularly important (MIUR, 2017). In Italy, the rate of students’ early school leaving, and low school achievements are considerably serious. Data (OECD, 2018) show that Italian students’ educational attainment and proficiency are below the OECD countries average for reading (476 vs. 487), mathematics (487 vs. 489), and science (468 vs. 489). Moreover, mean performance did not change significantly between 2015 and 2018 in reading and mathematics; and it even declined in science (OECD, 2018). In 2018, about 2% of Italian middle-school students and about 7% of Italian upper-secondary school students were not admitted to the following school year (MIUR, 2018). It is important to ensure students experience a successful school transition given that this significant phase impacts on lifelong learning (Boyle et al., 2018). The construct of learning orientations may have practical consequences for studying and success in school. As in previous studies (e.g., Ben-Eliyahu, 2019), it is central to gain a parsimonious holistic view of the learning process, whereby the interplay between learners’ cognitive, metacognitive, affective-motivational, and regulative aspects sustains self-regulation and successful achievements. In this perspective, instruments able to reliably capture the student’s view of their own learning tasks, their chosen learning strategies, the meaning of study in their eyes, and the nature of their motivations for pursuing academic success are particularly useful. Therefore, we conducted the present research composed by two studies, which fulfilled a two-fold purpose: (1) to develop a multi-componential self-report questionnaire to detect students’ interplay between cognitive, metacognitive, affective-motivational, and regulative aspects involved in learning; (2) to empirically test the adequate indices of statistical validity of the self-report questionnaire’s factors.

Students’ Learning Orientations

Learning orientations are interplay between students’ cognitive, metacognitive, and affective-motivational learning attributes contributing to create different learning pathways (Jeffrey, 2009). Learning orientations provide insights about learners’ ability to self-regulate “their cognition, motivation, and behavior, guided and constrained by their goals and the contextual features in the environment” (Pintrich, 2000, p. 453). In students’ learning orientations, the cognitive attributes include cognitive processing to elaborate and gain new information (e.g., Linnenbrink & Pintrich, 2004). The metacognitive attributes include students’ chosen metacognitive regulation strategies, such as prediction, planning, monitoring, and evaluation (see for a review Efklides, 2006) or the reflection upon one’s own conceptions of learning, teaching, and related phenomena (e.g., Van Rossum & Hamer, 2010; Vermunt & Donche, 2017). As stated in previous research (e.g., Lonka et al., 2021), students’ view of learning informs their personal epistemologies, because conceptions of learning implicitly include beliefs of how knowing and learning happens and what the origin of knowledge is.

Finally, the affective-motivational attributes refer to students’ learning motivations, such as mastery goal, extrinsic orientation, ability orientation (Pintrich, 1999; Ryan & Deci, 2000), learning purposes and goals (Vermunt & Donche, 2017).

One important aspect of the research on students’ learning orientations is that it goes beyond the traditional search for antecedents of school success and failure in basic general cognitive skills, such as students’ intellectual endowment that has been measured by students’ prior grades and scores on cognitive-based selection tests. To look at cognitive ability is informative about whether a student has the basic repertoire of ability to comprehend concepts (e.g., Hunter & Hunter, 1984), but it is not informative about whether and how students adopt regulation strategies during learning and studying activity and adapt those repertoires to the demands of the specific task (e.g., Gettinger & Seibert, 2002).

A further important aspect of the research on students’ learning orientations is that it allows one to consider cognitive, metacognitive, and affective-motivational learning attributes in interaction (e.g., Koriat & Levy-Sadot, 2000). Based on a socio-cognitive model (Bandura, 2001), students’ school achievement can be better explained by referring to the contribution of the interplay between cognitive (e.g., attention, memory), metacognitive (e.g., learning strategies), motivational (e.g., self-efficacy beliefs, outcome expectations) (Bandura, 1978; Robbins et al., 2004), emotions (e.g., feeling of confidence and feeling of satisfaction) (Efklides, 2006, 2009) that influence studying behavior and school achievement (McAlister et al., 2008). For example, the interplay between students’ experience of volition, in association with autonomous motivation, and deep learning strategies positively impacts on school achievement (Vermunt & Donche, 2017). Instead, students’ experience of pressure in learning (controlled motivation) is associated with stepwise processing and lack of regulation. Further, students’ feelings of incompetence and lack of intention in learning (amotivation) associate with lack of regulation (e.g., Deci & Ryan, 2000). Several studies have shown that students’ academic motivation is associated with their feelings of social involvement in the context of higher education (Noyens et al., 2019) enhancing learning activities (Meeuwisse et al., 2010).

Assessing Students’ Learning Orientations Significant for School Achievement

From the assessment point of view, the interplay between cognitive, metacognitive, and affective-motivational attributes of students’ learning orientations is challenging to measure.

In the international community, some instruments have been proposed that capture the interaction between these learning attributes. However, instruments and research results of different cultural contexts cannot easily be transferred to the Italian context, because they are context-specific and sensitive to the specific characteristics of the educational systems, such as teachers’ beliefs (e.g., Päuler-Kuppinger & Jucks, 2017), learning contexts (e.g., Vermetten et al., 1999), attendance of public schools, specific policies and practices, and rate of school achievements. Referring to the well-acknowledged in literature “Motivated Strategies for Learning Questionnaire” (MSLQ) (Pintrich et al., 1991; see also, Hilpert et al., 2013), it was built for university students belonging to an educational context profoundly different from that of Europe, and in particularly from that of Italy, in terms of the criteria for students’ progression over the school years, the aggregation of students in school classes that varies with respect to school subjects and courses, and the evaluation system. These aspects limit the generalizability of instruments from different educational and socio-cultural contexts to the Italian one’s.

In the Italian context, the framework of instruments able to measure the interplay between cognitive, metacognitive, and affective-motivational learning attributes in learning orientations is mostly fragmented. Some instruments, indeed, assess students’ points of view about learning (conceptions of learning), while others focus the assessment on the chosen specific strategies for learning by students. In reviewing instruments assessing students’ conceptions of learning in the Italian context, we came across the “Learning Conceptions Questionnaire” (LCQ; Pérez-Tello et al., 2005). The instrument measures students’ conceptions about the way of knowing (individual or collective), the role of the student (active or passive), the spectrum of academic emotions, motivations, as locus of control (internal or external). From the psychometric point of view, the LCQ (Pérez-Tello et al., 2005) has been administered across school years and educational contexts (e.g., Cantoia et al., 2011). The factorial structure of the instrument has been established among middle-school students using both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) (Vezzani et al., 2018). As demonstrated in previous studies, conceptions of learning predict learning outcomes (Vettori et al., 2021) and school achievement in middle school students. As shown by Pinto et al. (2018), the conceptions of “Learning as a co-constructive and cultural process” and “Volition, self-efficacy, and personal growth” were positively associated with school achievements; meanwhile, the conception of “Learning as a reduction of deficiencies knowledge and passive-receptive role” impacted on school achievement in a negative way. Although the LCQ instrument has the merit of measuring conceptions of learning in a multidimensional way (beliefs, emotions, locus of control), it does not contain items aimed at investigating students’ chosen strategies to control and regulate learning.

Turning to learning strategies, in reviewing measures in the Italian context, we came across the “Metacognitive Questionnaire on the Method of Study” (QMS; Cornoldi & De Beni, 2015). This instrument measures learning strategies, cognitive styles of information processing, metacognitive thinking, students’ disposition toward learning, and studying activities that students declare they translate into practice during learning and studying. Even though the latent factorial structure of QMS was checked on many participants, the psychometric analyses carried out would need to be put forward. In fact, only an EFA was carried out on the entire research sample, and not a cross-validation procedure, that is, an EFA on a first subsample was not carried out nor a CFA on a second subsample. Furthermore, the method of extraction of the latent factors was not declared by the authors, but probably a principal component algorithm was used, which presents a problem of overestimation of the factor loadings (Comrey & Lee, 1992). In the end, the eigenvalue higher than one criterion was used to choose the number of factors to be extracted by correlation matrix. This method involves several psychometric problems, and for this reason it is preferred in the literature to adopt other criteria for the choice of the number of latent factors, such as the visual analysis of the Scree Plot (Comrey & Lee, 1992). In literature the importance of learning strategies for school achievement is recognized (Credé & Phillips, 2011). Students with high metacognitive and self-regulatory abilities show active involvement in their own learning process (Zimmerman & Schunk, 2011), a strategic attitude toward studying (e.g., “I study all subjects with the same method”), functional study behaviors (e.g., “I have a habit of checking if I really understood what I studied, asking questions or doing exercises”), or the cognitive processes engaged in by students while learning and studying (e.g., “As the teacher speaks, I make notes to remember and understand better”). The integration of learning strategies with different learning attributes (e.g., conceptions of learning) is an essential step for assessing learning orientations in middle-school students in a more comprehensive way. This would also allow the psychometrical advantage of a better understanding of different learning attributes that interact in latent constructs.The integration of different learning attributes in a unique instrument is sustained by previous research finding demonstrating a relation between conceptions of learning, learning strategies, and school achievement. As shown by Vettori et al. (2018), middle school students with higher scores in deep learning strategies and studying managing activities (i.e., “Active elaboration of scholastic material”, “Flexibility to study”, “Participation in the classroom”, “Selection of principal aspects”, “Strategies of preparation for a test”) obtained higher school achievement across school subjects if their conception of learning was characterized by “Internal attribution of success and failure.”

Rationale

The overall picture of instruments appears fragmented, especially when considering the necessity to measure the learning attributes that act as predictors of school achievements. It is crucial to integrate different learning attributes from cognitive, metacognitive, and affective-motivational learning attributes of learning orientations in a unique instrument. Therefore, we propose to validate a new instrument able to measure, in a parsimonious and unitary way, the interplay between cognitive, metacognitive, affective-motivational learning attributes in learning orientations proven to be effective predictive factors of school achievement in middle school students. The proposed new instrument titled as “LO-COMPASS: Learning Orientation-Cognition Metacognition Participation Assessment” started from 30 items were selected from the a large set of learning orientations that were found to be predictive of school performance based on previous studies conducted with middle school students (i.e., Pinto et al., 2018; Vettori et al., 2018) to identify those significant configurations of learning orientations that are supportive of school performances in middle school students. The development of this new instrument will extend the existing self-report measures, improving the possibility of comparing results across different educational contexts. Specifically, the proposed LO-COMPASS is destined for the Italian middle-school population experiencing a significant school transition characterized by new adjustments in learning and studying approach, students and teachers’ relationships. It is important to identify middle school students’ learning orientations to sustain students in effectively coping with the new school changes in a preventive perspective for school failure and dropout.

Aims

(1) The first aim of this research was to explore the factorial latent structure of a new questionnaire LO-COMPASS with an Exploratory Factor Analysis (EFA) (Study 1);

(2) The second aim was to check the structural validity of the factorial latent structure of LO-COMPASS using a Confirmatory Factor Analysis (CFA) and to check the convergent and discriminant validity (Study 2).

Study 1

In this Study 1 we explored the latent factor structure of LO-COMPASS. An EFA was carried out on the set of 30 items selected from a large set of learning orientations that were found to be predictive of school performance based on previous studies (see “Selection of the initial pattern of specific items for the self-report questionnaire LO-COMPASS” in the “Method” section).

Method

Selection of the Initial Pattern of Specific Items for the Self-Report Questionnaire LO-COMPASS

The initial specific latent factors and specific items used for the construction of LO-COMPASS were selected for their proved significant direct and indirect predictive effects on school performance, namely “Language and literature,”“Foreign language,” and “Maths” (i.e., Pinto et al., 2018; Vettori et al., 2018) and subjected to psychometric analyses. Specifically, a part of selected factors resulted from the research by Vettori et al., (2018) to have an indirect significant predictive effect on middle school students’ school achievement were the LCQ factor “Internal attribution of own success and failure” (example item: “The last time that I successfully passed a school test, I felt interested in the subject matter”), the QMS factor “Active elaboration of scholastic material” (example item: “As the teacher speaks, I make notes to remember and understand better”), the QMS factor “Flexibility to study” (example item: “There is only one correct way to read and study a chapter”), the QMS factor “Participation in classroom” (example item: “I consider it useless to take notes in class”), the QMS factor “Selection of principal aspects” (example item: “To decide what to highlight, I keep in mind the title, the type of text and the purpose of the reading”), and the QMS factor “Strategies of preparation to a test” (example item: “I’m very careful when teachers ask questions, to understand what they expect”). A further part of selected factors resulted from the research by Pinto et al., (2018) and were the LCQ factors “Learning as co-constructive and cultural process” (example item: “Effective learning occurs through discussions and debates with others”), “Learning as a reduction of deficit through individual effort” (example item: “The primary prerequisites for learning are concentration and commitment”) and “Personal challenge, self-efficacy and personal growth” (example item: “I consider learning to be an opportunity to evaluate my intellectual capacity”) that resulted direct significant predictors of middle school students’ achievement (for at least one of the three subjects considered). For each factor, the items with the highest predictors were selected.

Participants and Procedure

The sample (N = 212) consisted of 106 male (M_age = 12.58 ± 1.02) and 106 female first-grade middle-school students (M_age = 12.76 ± 0.99). The participants of Study 1 were recruited from two schools in the central part of Italy through convenience sampling (McBurney & White, 2009). Middle school years, which in Italy covers 11 to 13 years, are particularly critical because students need to progressively move forward a highly structured school environment presented by previous primary school years toward less structured school environment (e.g., Chung et al., 1998; Kingery & Erdley, 2007) where the assumption of a more agentic role and self-regulated learning (Anderson et al., 2000; Schunk & Zimmerman, 1997), sense of independence and responsibility (e.g., Eccles & Wigfield, 1997) are an essential step for subsequent upper secondary school years (Azevedo, 2009). In an Italian middle school classroom, there are about nine teachers instead of two or three teachers as in the previous primary school grade; furthermore, each one teaches a different subject, each lesson of a subject lasts one or maximum 2 hr and in a day the teachers alternate for about five or six consecutive hours with a short interval of 10 min.

This research was conducted with the permission of the institutional authorities and the consent of the students and their parents. It was also approved by the Ethics Committee of the University of [. . .] in Italy. The study adheres to the rules of the Code of Ethics of the Italian Association of Psychology. All participants have provided the consent and responded to the questionnaire. There were no missing data.

After they had provided consent, the participants completed the initial 30-item version of LO-COMPASS in their classroom. The questionnaire was given in the Italian language. They were required to indicate their level of agreement or disagreement with each selected item for the validation procedure of LO-COMPASS on a five-point Likert scale that ranged from 1 (strongly disagree) to 5 (strongly agree). This study was conducted at the beginning of the school year. The students spent on average less than 30 min responding to all the items, with a maximum time of 40 min for some participants. The instrument administration was carried out in the classrooms during school time. During the administration session both the researchers and teachers were available to answer any questions that students might have about the meaning of the individual items and the required response format. Students were informed that their responses were not evaluated and that there were no right or wrong answers. Consent procedure for research consisted of school approval and parental consent by ensuring confidentiality to participants. In attachment to the informed consent forms, a parental survey was used to gain information about their socioeconomic status (SES) (i.e., parental educational qualification and occupation) by referring to the International Standard Classification of Education (ISCED) and the International Standard Classification of Occupations (ISCO). The instrument was administered to everyone in the class, then in the phase of analysis we assume the following exclusion criteria: (1) foreign students who had been residing in Italy for less than 5 years were not included in the analysis given that their experience of the previous school system was linked to a different and uncontrolled context; (2) those students with certified disabilities given that the scope of this study was to provide a first insight of learning orientations in a typical developing sample of students. Within future research development, we advise the necessity to study learning orientations in students with atypical development by recruiting an adequate sample size.

Data Analysis and Results

The distributions of the interactions between “gender x middle school grade” in Study 1 showed that there was an equal representation of the two genders and the three school grades of middle school in the study sample. This was also supported by the results of a chi-squared test that was conducted to examine group differences (i.e., “gender x middle school grade”) (χ² (2) = 4.15, p = .063, Kramer’s V = 0.099). Descriptive statistics (i.e., means, standard deviations, and skewness and kurtosis coefficients) were computed for the 30 items to test the normality of their probability distributions. It is well documented in the existing international scientific literature that Likert scales can yield normally distributed data. Indeed, if a Likert scale consists of at least 5 response categories and the distributions of such responses are approximately normal or only moderately deviate from normality, no significant problems are likely to arise from univariate and multivariate parametric analyses of the resultant data (Babakus et al., 1987; Breckler, 1990; B. Muthén & Kaplan, 1985). If the data do not approximate to a normal curve, it may still be appropriate to use robust estimation algorithms because they are unaffected by deviations from normality ( L. K.Muthén & Muthén, 2010).

Descriptive statistics (i.e., means, standard deviations, and skewness and kurtosis coefficients) for the items that met the aforementioned criteria are presented in Table 1.

Table 1.

Descriptive Statistics for the Initial Pool of 30 Items (N = 212).

Item	M	SD	Skewness	Kurtosis
I feel that I really learn through discussion and confrontation with others	3.76	1.02	−1.02	1.00
Learning is above all a matter of concentration and commitment	3.75	0.87	−0.63	0.31
You really learn when you listen to explanations given by an teacher	3.15	0.88	−0.08	−0.10
Above all, a good teacher makes his/her students compare and discuss with each other the opinions they have about the things that the school teaches	3.84	0.86	−0.50	0.09
A student always has some idea about the lessons that they will learn at school	3.92	0.80	−0.50	0.12
In order to really learn I need someone to teach me	3.27	1.04	−0.20	−0.36
I consider learning to be:
- an opportunity to evaluate my intellectual capacity	3.65	1.17	−0.52	−0.65
- something based on self-reliance	3.30	1.13	−0.27	−0.81
- a time of personal growth and change	3.73	1.03	−0.46	−0.60
The last time that I successfully passed a school test, I felt:
- interested in the subject matter	3.58	0.84	−0.33	0.43
- happy that I had successfully completed my duty	4.10	0.89	−0.93	0.66
- grateful to my teacher	2.79	0.92	0.21	0.10
As the teacher speaks, I make notes to remember and understand better	3.55	1.26	−0.22	−1.21
I study all subjects with the same method	3.43	1.27	−0.10	−1.28
To decide what to highlight, I keep in mind the title, the type of text and the purpose of the reading	3.30	1.19	0.01	−1.64
When I listen to my teacher teach, I try to predict what he/she will say next	2.77	1.18	0.59	−0.82
I have a habit of checking if I really understood what I studied, asking questions or doing exercises	3.58	1.16	−0.39	−1.36
When I read, I do not think that it is particularly important to identify the main ideas that the author intended to convey	4.00	1.18	−0.74	−0.70
When I read a story, I mistake one character for another	4.17	1.11	−0.98	−0.20
There is only one correct way to read and study a chapter	4.27	1.00	−1.37	0.78
I approach the study of all subjects in the same way	3.20	1.30	0.13	−1.35
I prepare for all tests in the same manner because I do not think that my performance will differ based on whether a test is a written or an oral one	2.13	3.55	−0.21	−1.20
When I study, I find it difficult to understand a concept that is unclear to me	4.00	0.76	−0.71	−0.63
When professors ask me questions, I take care to ensure that I understand their expectations	4.04	1.04	−1.01	−0.13
I can immediately assess if a task is easy or difficult	4.12	0.94	−1.19	0.71
I’m always trying to get a place to sit in class to help me keep up with the explanation	3.44	1.17	−0.21	−1.53
When I have not studied adequately, I am aware of it	3.98	0.71	−0.68	−0.62
When I’m listening to a lesson, I try to get some paper to write down the important things	3.43	1.26	−0.09	−1.68
After an exam, I am unable to understand why I have received the results that I have	3.73	0.86	−0.37	−0.91
I consider it useless to take notes in class	4.08	0.73	−0.95	−0.50

It can be inferred from Table 1 that the skewness and kurtosis coefficients of some items included did not lie between the stipulated range (i.e., from −1 to +1). This indicated that the distribution of responses to several items deviated significantly from a Gaussian curve (Marcoulides & Hershberger, 1997). Therefore, the data were analyzed using robust methods (L. K. Muthén & Muthén, 2010).

In order to investigate the latent factorial structure of the initial set of 30 items, EFA was conducted using Mplus (L. K. Muthén & Muthén, 1998). The EFA carried out on the 30 items selected by the QMS and LCQ, according to the empirical psychometric criteria outlined above (see “Selection of the initial pattern of specific items for the self-report questionnaire LO-COMPASS” Section) identified latent factors more dependent on each other than the latent factors proposed individually by the two questionnaires. Since it is a unitary factorial model for all the 30 items initially selected, indeed, there is the advantage that the estimation of each latent factor is computed by partializing the loadings of each other factor of the EFA model, going to identify the interplay between the latent factors.

Since many items were not normally distributed, EFA was conducted using a robust multilevel modeling (MLM) estimator. The MLM algorithm (Bentler, 1995) estimates adjusted standard errors and mean-adjusted chi-square test that results robust to non-normality. Indeed, while parameter estimates are standard ML estimates, standard errors are computed using a sandwich-type estimator, and so the computation of significance of each parameter of the EFA model results not biased by the non-normality of their probability distributions (L. K. Muthén & Muthén, 1998).

Furthermore, because the MLM algorithm is a full information estimator, it was also possible to estimate the goodness of fit indexes for EFA (not only for the CFA in the Study 2). It is helpful to use EFA with goodness of fit indexes, differently from a traditional EFA, because it allows one to compare several alternative factorial models with a different number of factors. In other words, comparing the fit of the alternative models, it is possible to choose the most appropriate number of factors to extract by the correlation matrix (Brown, 2015; Grant & Fabrigar, 2007).

The factorial structure that EFA yielded is shown in Table 2.

Table 2.

Factor Loadings Resulted by EFA (n = 212).

Item	Factor loadings
Item	Learning as a self-regulated and strategic experience	Learning as a process of affective, motivational and co-constructive activation of Self	Learning as guided practice	Learning as participation in school practices
As the teacher speaks, I make notes to remember and understand better	0.80
When I’m listening to a lesson, I try to get some paper to write down the important things	0.69
I have a habit of checking if I really understood what I studied, asking questions or doing exercises	0.54
I’m very careful when teachers ask questions, to understand what they expect	0.49
I study all subjects with the same method	0.45
I’m always trying to get a place to sit in class to help me keep up with the explanation	0.43
To decide what to highlight, I keep in mind the title, the type of text and the purpose of the reading	0.34
I feel learning is a time of personal growth and change		0.62
I feel learning as something based on self-confidence		0.58
I feel that I really learn through discussion and confrontation with others		0.52
Above all, a good teacher makes his/her students compare and discuss with each other the opinions they have about the things that the school teaches		0.50
The learner while studying can create new ideas himself		0.44
In order to really learn I need someone to teach me			0.50
You really learn when you listen to explanations given by an teacher			0.43
The last time I successfully passed a school test, I felt that it was also the merit of those who had taught me			0.43
Learning is above all a matter of concentration and commitment			0.33
During study I find it easy to tell if I’m prepared or not				0.49
When I read a story, I easily orient myself among characters and in the story plot				0.45
I prepare myself in a different way for oral questioning, for a class assignment or for group work				0.37
After an oral examination, I can figure out how I performed				0.37

The results of EFA showed a 4-factor structure of the instrument.

The first factor that explained the higher variance named “Learning as a self-regulated and strategic experience” (variance explained: 14.84%) shows the stronger contribution of the students’ metacognitive regulation of their learning processes. It describes the extent to which students recur to the use of different learning strategies and assume a self-regulated approach to learning (Item examples, “As the teacher speaks, I make notes to remember and understand better”; “When I’m listening to a lesson, I try to get some paper to write down the important things”).

The second factor named “Learning as a process of affective, motivational and co-constructive activation of Self” (variance explained: 5.16%) captures students’ affective and motivational engagement (e.g., heuristic emotions, such as interest and curiosity). There emerges a holistic vision about students’ Self in learning for which affective and motivational aspects interact (Item examples, “I feel learning is a time of personal growth and change”; “I feel learning as something based on self-confidence”).

The third factor named “Learning as guided practice” (variance explained: 4.13%) calls into account students’ cognitive engagement (e.g., executive functions, attention, comprehension processes) contextualized within the privileged student-teacher relationship (Item examples, “In order to really learn I need someone to teach me”; “You really learn when you listen to explanations given by a teacher”).

Finally, the fourth factor named “Learning as participation in school practices” (variance explained: 3.26%) refers to a student’s active involvement in a learning activity (Christenson et al., 2012) (Item examples, “I prepare myself in a different way for oral questioning, for a class assignment or for group work”; “After an oral examination, I can’t figure out how I performed”). In order to check the internal consistencies of the factors, Cronbach’s Alpha and McDonald’s omega coefficients were computed for each factorial factor. All the factors yielded acceptable reliability omega coefficients (“Learning as a self-regulated and strategic experience”: α = 70 −ω = .74; “Learning as a process of emotive, motivational activation and co-construction of Self”: α = 64 −ω = .67; “Learning as guided practice”: α = 62 −ω = 0.65; “Learning as participation in school practices”: α = 61 −ω = .63). The model fit indices that were yielded by EFA, which was conducted using Mplus (v. 7.0; L. K. Muthén & Muthén, 1998), were satisfactory (Root Mean Square Error of Approximation, RMSEA = 0.02; Root Mean Square Residual, RMSR = 0.04).

The independence of the four factors that were yielded by EFA was further examined by inspecting the correlations among them, which are presented in Table 3.

Table 3.

Correlation Matrix Between Factor Scores Pointed Out by EFA: Spearman’s Rho coefficient.

	Factor scores
	Learning as a self-regulated and strategic experience	Learning as a process of affective, motivational and co-constructive activation of Self	Learning as guided practice	Learning as participation in school practices
Learning as a self-regulated and strategic experience	–
Learning as a process of affective, motivational and co-constructive activation of Self	0.48**	–
Learning as guided practice	0.26*	0.31*	–
Learning as participation in school practices	−0.13	−0.09	0.05	–

p < .01. *p < .05.

The magnitude of the correlations that “Learning as a self-regulated and strategic experience” shared with “Learning as a process of emotive, motivational activation and co-construction of Self” and “Learning as guided practice” was similar. “Learning as a process of emotive, motivational activation and co-construction of Self” was positively and significantly associated with “Learning as guided practice,” whereas “Learning as participation in school practices” was unrelated to any of the other factors (Table 3).

Study 2

In this Study 2 the initial factor solution obtained in the previously described Study 1 was statistically checked on a second subsample by a CFA, coherently with a cross-validation procedure. The factorial structure resulting from CFA was put in association with school achievement measures.

Method

Participants and Procedure

A convenience sample (N = 233) of 126 male (M_age = 12.12 ± 0.91) and 107 female (M_age = 12.31 ± 1.13) students who were recruited from two schools in Prato (Tuscany) participated in this study (McBurney & White, 2009). Permission from the institutional authorities and the consent of the students and their parents were collected. All participants have provided the consent and responded to the questionnaire. There were no missing data.

The characteristics of the participants of Study 2 were the same as Study 1. Similar to the sample that was used in Study 1, there was an equal representation of the two genders and the three school grades of middle school in the sample of Study 2.

In Study 2, the procedure was the same as Study 1. In addition, the school achievements of 82 students of this second sample of participants were detected.

Data Analysis and Results

The results of a chi-squared test that was conducted to examine group differences (“gender × middle school grade”) (χ² (2) = 0.28, p = .435, Kramer’s V = 0.024). The data that were obtained from this sample were subjected to CFA to confirm the factorial structure that was yielded by EFA, which in turn was conducted using data that were obtained from the sample that was used in Study 1.

Study 2 was conducted on a different sample with respect to Study 1. This choice was due to the necessity to undertake a cross-validation procedure and identify the latent factorial structure of the questionnaire. Several international research studies that have examined the psychometric properties of assessments have used cross-validation to check the latent structure of self-report instruments (Finch et al., 2016; Suh et al., 2015). Specifically, it allows researchers to strengthen the validity of their findings by cross validating their results across two independent samples.

Descriptive statistics (i.e., means, standard deviations, and skewness and kurtosis coefficients) for the items of the sample of Study 2 are presented in Table 4.

Table 4.

Descriptive Statistics for Study 2 (N = 233).

Item	M	SD	Skewness	Kurtosis
I feel that I really learn through discussion and confrontation with others	3.78	0.97	−1.13	1.45
Learning is above all a matter of concentration and commitment	4.27	0.90	−1.39	1.98
You really learn when you listen to explanations given by an teacher	3.64	1.08	−0.64	−0.23
Above all, a good teacher makes his/her students compare and discuss with each other the opinions they have about the things that the school teaches	3.94	0.97	−0.86	0.61
A student always has some idea about the lessons that they will learn at school	3.26	1.06	−0.28	−0.33
In order to really learn I need someone to teach me	3.93	1.11	−1.02	0.46
I consider learning to be:
- an opportunity to evaluate my intellectual capacity	3.68	1.12	−0.62	−0.39
- something based on self-reliance	3.18	1.23	−0.20	−0.81
- a time of personal growth and change	3.50	1.20	−0.53	−0.59
The last time that I successfully passed a school test, I felt:
- interested in the subject matter	3.32	1.01	−0.29	−0.09
- happy that I had successfully completed my duty	4.31	1.09	−1.68	2.04
- grateful to my teacher
As the teacher speaks, I make notes to remember and understand better	1.76	0.70	0.37	0.32
I study all subjects with the same method	2.05	0.73	−0.08	−1.11
To decide what to highlight, I keep in mind the title, the type of text and the purpose of the reading	2.12	0.72	−0.18	0.32
When I listen to my teacher teach, I try to predict what he/she will say next	1.69	0.68	0.47	−0.81
I have a habit of checking if I really understood what I studied, asking questions or doing exercises	2.00	0.77	−0.01	−1.30
When I read, I do not think that it is particularly important to identify the main ideas that the author intended to convey	2.45	0.70	−0.89	−0.48
When I read a story, I mistake one character for another	2.55	0.65	−1.12	0.10
There is only one correct way to read and study a chapter	2.36	0.69	−0.62	−0.73
I approach the study of all subjects in the same way	1.93	0.74	0.11	−1.19
I prepare for all tests in the same manner because I do not think that my performance will differ based on whether a test is a written or an oral one	2.12	0.72	−0.18	−1.05
When I study, I find it difficult to understand a concept that is unclear to me	2.45	0.68	−0.85	−0.44
When professors ask me questions, I take care to ensure that I understand their expectations	2.12	0.70	−0.17	−0.96
I can immediately assess if a task is easy or difficult	2.2	0.64	−0.25	−0.67
I’m always trying to get a place to sit in class to help me keep up with the explanation	1.92	0.77	0.14	−1.30
When I have not studied adequately, I am aware of it	2.45	0.66	−0.82	−.43
When I’m listening to a lesson, I try to get some paper to write down the important things	1.77	0.80	0.44	−1.31
After an exam, I am unable to understand why I have received the results that I have	2.22	0.70	−0.29	−0.79
I consider it useless to take notes in class	2.50	0.72	−1.08	−0.23

Furthermore, similarly to EFA, CFA was conducted using a robust MLM estimation algorithm. Classic estimation methods in SEM, such as normal theory maximum likelihood (ML), are based on the assumption that the observed variables are treated on a continuous for EFA and CFA (Rhemtulla et al., 2012).

In this study, we also tested a model with 5 factors. The Cronbach’s Alpha and McDonald’s Omega coefficients, calculated on the results of CFA, indicated that the assessment had unacceptable internal consistency, based on George and Mallery’s (2003) criteria. The model fit indices were not all acceptable, for example, CFI = 0.82 and TLI = 0.79. For this reason, we thought it appropriate to continue our analysis based on the 4-factor model which turned out to be the best among the possible models. The factor loadings that were yielded by CFA are presented in Table 5 and Figure 1.

Table 5.

Factor Loadings Resulted by CFA (n = 233).

Item	Factor loadings
Item	Learning as a self-regulated and strategic experience	Learning as a process of affective, motivational and co-constructive activation of Self	Learning as guided practice	Learning as participation in school practices
As the teacher speaks, I make notes to remember and understand better	0.65
I have a habit of checking if I really understood what I studied, asking questions or doing exercises	0.63
When I’m listening to a lesson, I try to get some paper to write down the important things	0.58
I’m very careful when teachers ask questions, to understand what they expect	0.54
I study all subjects with the same method	0.48
To decide what to highlight, I keep in mind the title, the type of text and the purpose of the reading	0.45
I’m always trying to get a place to sit in class to help me keep up with the explanation	0.40
I feel learning as something based on self-confidence		0.63
I feel learning is a time of personal growth and change		0.61
I feel that I really learn through discussion and confrontation with others		0.49
The learner while studying can create new ideas himself		0.48
Above all, a good teacher makes his/her students compare and discuss with each other the opinions they have about the things that the school teaches		0.44
The last time I successfully passed a school test, I felt that it was also the merit of those who had taught me			0.53
Learning is above all a matter of concentration and commitment			0.51
You really learn when you listen to explanations given by the teacher			0.44
In order to really learn I need someone to teach me			0.37
During study I find it easy to tell if I’m prepared or not				0.57
When I read a story, I easily orient myself among characters and in the story plot				0.48
I prepare myself in a different way for oral questioning, for a class assignment or for group work				0.37
After an oral examination, I can figure out how I performed				0.31

Figure 1.

Graphical representation of CFA structure and factor loading values.

Cronbach’s Alpha and McDonald’s Omega coefficients, calculated on the results of CFA, indicated that the assessment had acceptable internal consistency (“Learning as a self-regulated and strategic experience”: α = 72 −ω = .76; “Learning as a process of affective, motivational and co-constructive activation of Self”: α = 68 −ω = .71; “Learning as guided practice”: α = 62 −ω = 0.64; “Learning as participation in school practices”: α = 61 −ω = .64). The model fit indices were acceptable (Confirmatory Fit Index, CFI = 0.89; TLI = 0.92; RMSEA = 0.04; SRMR = 0.06); however, the CFI values observed were lower than the recommended cut-off scores (Hu & Bentler, 1999).

All the factor loadings were greater than 0.30 (Table 5). The relationships between the factors that were yielded by CFA were examined by computing Spearman’s correlation coefficients (Table 6).

Table 6.

Correlation Matrix Between Factor Scores Pointed Out by CFA: Spearman’s Rho coefficient.

	Factor scores
	Learning as a self-regulated and strategic experience	Learning as a process of affective, motivational and co-constructive activation of Self	Learning as guided practice	Learning as participation in school practices
Learning as a self-regulated and strategic experience	–
Learning as a process of affective, motivational and co-constructive activation of Self	0.51***	–
Learning as guided practice	0.53***	0.52**	–
Learning as participation in school practices	0.05	0.18	−0.06	–

***

p < .001. **p < .01.

The convergent and discriminant validity of the factorial model that was yielded by CFA were examined. For convergent validity, average variance extracted (AVE) values were computed for each factor, and they ranged from 0.20 to 0.29. Although these values are below 0.50, the values for composite reliability (CR) ranged from 0.61 to 0.83 (i.e., >0.60). As Fornell and Larcker (1981) and Huang et al. (2013) have argued, when AVE values are lower than 0.50, but CR coefficients are higher than 0.60, the convergent validity of an identified construct may be considered to be adequate. To test the discriminant validity of the CFA model, we used the criteria that were proposed by Gaski and Nevin (1985), as well as by Fornell and Larcker (1981). Gaski and Nevin (1985) contended that Cronbach’s alpha coefficients for each factor must be greater than the correlations between the factors themselves in order for the discriminant validity of the factors to be considered good. It can be inferred from the comparisons of the alpha and omega internal consistency coefficients (the latter are more preferable than the traditionally used Cronbach’s alpha coefficients) and the factorial correlation coefficients presented in Table 7, that all the alpha and omega values were greater than the correlation coefficients between the factors.

Table 7.

Correlation Matrix Between Factor Scores Pointed Out by CFA and Academic Achievement in Language and Literature, Foreign Language and Math: Spearman’s Rho Coefficient.

	School Subjects (n = 82)
	Language and literature	Foreign language	Math
Learning as a self-regulated and strategic experience	0.30**	0.29**	0.24*
Learning as a process of affective, motivational and co-constructive activation of Self	0.40***	0.34**	0.30**
Learning as guided practice	0.24*	0.17	0.09
Learning as participation in school practices	0.34**	0.42***	0.40***

***

p < .001. **p < .01. *p < .05.

Finally, the relation between the factorial model and school achievement was reported in Table 6, in which the Spearman non-parametric correlations are shown between factorial scores of the CFA model and the school grades in Language and literature, Math, and Foreign language.

The results of the correlational analysis pointed out that school achievements were significantly associated with the latent factors across school subjects “Learning as a self-regulated and strategic experience” (Language and literature: r = .30, p = .007; Foreign language: r = .29, p = .008; Math: r = .24, p = .033), “Learning as a process of affective, motivational and co-constructive activation of Self” (Language and literature: r = .40, p < .001; Foreign language: r = .34, p = .002; Math: r = .30, p = .005) and “Learning as participation in school practices” (Language and literature: r = .34, p = .002; Foreign language: r = .42, p < .001; Math: r = .40, p < .001), whereas the factor “Learning as guided practice” resulted significantly correlated uniquely with “Language and literature” (r = .24, p = .032) (Table 7).

Discussion

The results of these two studies support the reliability and validity of the new development “LO-COMPASS: Learning Orientation-Cognition Metacognition Participation Assessment,” a tool assessing the interplay between cognitive, metacognitive, and affective-motivational learning attributes in middle school students’ learning orientations associated to middle school students’ achievements.

Within the current framework of instruments measuring learning variables, LO-COMPASS provides a significant contribution. First, it originates from the recognition of the importance to consider the interplay between cognitive, metacognitive, and affective-motivational attributes in learning orientations. Second, within this interplay between cognitive, metacognitive, and affective-motivational attributes, LO-COMPASS measures the indicators that are predictive of school performance.

The newly developed LO-COMPASS instrument offers a 4-factors solution (see Tables 3 and 6) showing different patterns of middle school students’ learning orientations with adequate internal consistency. The four LO-COMPASS factors show different learning orientations suggesting that middle school students’ different approaches to learning and pathways can be identified and measured. Cross-validation was undertaken by subjecting the results of EFA to CFA. CFA confirmed the initial factor structure that was yielded by EFA. The model fit indices that resulted from CFA and the internal consistency coefficient of each factor were adequate; only the CFI value resulted lower than the cut-off of 0.90, showing a possible problem in the fit of the model. Otherwise, for the TLI coefficient, although considered acceptable by some (Bentler & Bonett, 1980), the value of just over 0.90 this value is actually questionable. Although reliability indexes are also quite low, the values of Cronbach’s Alpha and of McDonald’s Omega are sufficiently homogeneous, also in consideration of the variety of psychological dimensions (learning orientations) of which they are indicative, a multicomponent nature in which the novelty of the LO-COMPASS instrument lies.

The first LO-COMPASS factor “Self-regulated and strategic experience” suggests a learning orientation characterized by the interaction between cognitive and metacognitive learning attributes. Over the middle-school years, students are required to develop a sense of independence and responsibility (e.g., Eccles & Wigfield, 1997), personal agency and self-regulation in learning (Anderson et al., 2000), thus their cognitive and metacognitive engagement is very helpful. Students may vary in adopting cognitive processing and metacognitive regulation strategies (e.g., planning, monitoring, and evaluating). High scores on this first LO-COMPASS factor might be indicative of the active participation of students from the cognitive and metacognitive point of view. Previous research demonstrates the association between sustained attention, working memory, and deep learning strategies registering a positive impact on school achievement (Vermunt & Donche, 2017).

The second LO-COMPASS factor “Learning as a process of affective, motivational and co-constructive activation of Self” suggests a learning orientation characterized by the interplay between internal motivation and a conception of learning as an opportunity for personal growth and personal change. Moreover, this second LO-COMPASS factor shows the value of cooperation with fellow students. Our results shed light on the interplay between Self and others for middle school students experiencing a significant school transition (Benner et al., 2017), so strongly imbued by emotion regulation and relational changes. The characteristics of this second LO-COMPASS factor are in line with previous research showing students’ academic motivation is associated with their feelings of social involvement (Noyens et al., 2019) enhancing learning activities (Meeuwisse et al., 2010).

The third LO-COMPASS factor “Learning as guided practice” suggests a learning orientation characterized by the interplay between cognitive and relational learning attributes. This third LO-COMPASS factor is primarily concerned with a conception of learning as a co-construction of knowledge and the important supportive role of the teacher. This third LO-COMPASS factor highlights the importance of teacher’s guidance and emotional scaffold that is particularly well suited at describing the relational experience of middle school students involved in a challenging school transition. Middle school students experience more autonomy while continuing to feel safe in the protective relational context with the teacher (see, Hopwood et al., 2016).

The fourth LO-COMPASS factor “Learning as participation in school practices” suggests a learning orientation characterized by students’ engagement in the learning context and school practices. This fourth LO-COMPASS factor show the interplay between students’ effort, participation, persistence, interest, enthusiasm (affective-motivational learning attributes), behavioral engagement, depth processing of school material, and use of self-regulated metacognitive strategies (cognitive and metacognitive learning attributes) which are important learning aspects (Christenson et al., 2012). Students’ engagement may be linked to the recognition of the specific way in which learning occurs, including the substantial role of individual and contextual resources to overcome difficulties by enhancing the conditions under which they learn. Middle-school students’ awareness about learning functioning might influence their learning behavior, for example their seeking help in classrooms resulting in different learning approaches. High scores on this fourth LO-COMPASS factor might indicate students’ awareness that intentional and proactive behaviors are helpful ways to improve their opportunities to learn. For these reasons, this fourth LO-COMPASS factor might also capture the agentic aspect of students’ engagement (Reeve & Tseng, 2011) that has been linked to school achievements. Our findings showed that this fourth LO-COMPASS factor is strongly associated with high-school achievements across school subjects. The lacking correlation of the fourth LO-COMPASS factor with the others may be tentatively explained by the original aspects measured by this factor that captures the process in which students are intrinsically motivated to try intentionally and proactively to make their own learning choices (e.g., communicate their level of preparation, seek ways to resolve problems) and otherwise optimize their learning circumstances (Reeve & Tseng, 2011). From a statistical point of view, correlation coefficients that were computed for the relationships between the factors that were yielded by CFA ranged from .05 to .53. In general, the fact that not all factors are correlated with each other does not constitute a problem to the detriment of the statistical goodness of the factorial model, as the factors are by definition orthogonal to each other (at least in their initially estimated loadings) (Comrey & Lee, 1995). The values of the correlation coefficients suggest that the factors are independent albeit interrelated constructs. Thus, correlations among factors rather than multicollinearity (highest Spearman’s ρ = 0.53) are more likely to have been an explanation for the potential emergence of a unidimensional construct. In particular, the AVE values and CR indices were indicative of adequate convergent validity; the discriminant validity of the factorial dimensions was also found to be satisfactory.

The study presented suffers from numerous limitations. From a psychometric point of view, further tests of the psychometric properties of LO-COMPASS are needed to provide additional evidence of its validity and reliability. From a psychometric point of view, a major limitation of this study is that the convergent and discriminant validity of LO-COMPASS was not examined against other existing measures. Results to be taken with caution considering small sample size. The sample is represented in relation to the specificities of the Italian school system that adheres to national rules (e.g., homogeneous for school programs, etc.). In order to increase the sample representativeness, future research could conduct a study with students coming from different Italian geographical areas and with groups that differ in learning environment (see, Han & Ellis, 2020), and sociocultural background (Alivernini et al., 2019). It would be interesting to examine the different patterns of learning orientation by assuming a person-oriented approach. We also advise the necessity to study learning orientations in students with atypical development by recruiting an adequate sample size. Finally, some review work has been undertaken around methods of assessing metacognition which may be useful. Considering that contextualizing information (e.g., teacher questionnaires) seem to give additional valuable information on metacognitive skills of primary-school children (Desoete, 2008), this study would be enriched by the adoption of a multiple-method design, including teacher questionnaires, in future studies.

Overall, this self-report questionnaire provides to school psychologists, teachers, and school practitioners with a means to efficiently identify learning orientations and contribute to the drive for evidence-based practice, particularly useful with students facing a school transition (Eccles & Wigfield, 1997). Specifically, we conducted the present research on a student population challenged by the specific school transition of middle school (Goldstein et al., 2015). Moreover, the middle school transition interests students’ relationships (Eccles & Roeser, 2011), since the previous primary school class group breaks up and a new class group is formed by some students who already know each other and others who are unknown. The student-teacher relationship becomes much more formal than in the previous primary school years. Middle school students are greatly involved in homework management on their own (Xu & Corno, 2003) and often have to give up sports or other recreational activities they have done previously until they acquire an efficient and autonomous method of study.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Vettori Giulia

References

Alivernini

Manganelli

Cavicchiolo

Chirico

Lucidi

(2019). The use of self-regulated cognitive strategies across students with different immigrant backgrounds and gender. Journal of Psychoeducational Assessment, 37, 652–664.

Anderson

L. W.

Jacobs

Schramm

Splittgerber

(2000). School transitions: Beginning of the end or a new beginning? International Journal of Educational Research, 33, 325–339.

Azevedo

(2009). Theoretical, conceptual, methodological, and instructional issues in research on metacognition and self-regulated learning: A discussion. Metacognition and Learning, 4, 87–95.

Babakus

Ferguson

C. E.

Joreskog

K. G.

(1987). The sensitivity of confirmatory maximum likelihood factor analysis to violations of measurement scale and distributional assumptions. Journal of Marketing Research, 24, 222–228. https://doi.org/10.2307/3151512

Bandura

(1978). The self system in reciprocal determinism. American Psychologist, 33, 344–358.

Bandura

(2001). Social cognitive theory of mass communication. Media Psychology, 3, 265–299.

Ben-Eliyahu

(2019). Academic Emotional Learning: A critical component of self-regulated learning in the emotional learning cycle. Educational Psychologist, 54, 84–105. https://doi.org/10.1080/00461520.2019.1582345

Benner

A. D.

Boyle

A. E.

Bakhtiari

(2017). Understanding students’ transition to high school: Demographic variation and the role of supportive relationships. Journal of Youth and Adolescence, 46, 2129–2142.

Bentler

P. M.

(1995). EQS structural equations program manual. Multivariate Software.

10.

Bentler

P. M.

Bonett

D. G.

(1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606. https://doi.org/10.1037/0033-2909.88.3.588

11.

Boyle

Grieshaber

Petriwskyj

(2018). An integrative review of transitions to school literature. Review of Educational Research, 24, 170–180.

12.

Breckler

S. J.

(1990). Applications of covariance structure modeling in psychology: Cause for concern? Psychological Bulletin, 107, 260–273. https://doi.org/10.1037/0033-2909.107.2.260

13.

Brown

T. A.

(2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Publications.

14.

Cantoia

Giordanelli

Pérez-Tello

Antonietti

(2011). Imitating, reasoning, discussing: Student conceptions of learning around the world. Nova Science.

15.

Christenson

S. L.

Reschly

A. L.

Wylie

(Eds.). (2012). Handbook of research on student engagement. Springer Science & Business Media.

16.

Chung

Elias

Schneider

(1998). Patterns of individual adjustment changes during middle school transition. Journal of School Psychology, 36, 83–101.

17.

Comrey

A. L.

Lee

H. B.

(1992). A first course in factor analysis. Academic Press.

18.

Comrey

A. L.

Lee

H. B.

Caprara

G. V.

Ercolani

A. P.

Barbaranelli

(1995). Introduzione all’analisi fattoriale. LED.

19.

Cornoldi

De Beni

; Research Group MT – Memory and Training. (2015). Metacognitive Questionnaire on the Method of Study (QMS). In Learning to study: strategies, cognitive styles, metacognition and attitudes in study: 10-15 years old, Erickson Editor.

20.

Credé

Phillips

L. A.

(2011). A meta-analytic review of the Motivated Strategies for Learning Questionnaire. Learning and Individual Differences, 21, 337–346.

21.

Deci

E. L.

Ryan

R. M.

(2000). The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11, 227–268.

22.

Desoete

(2008). Multi-method assessment of metacognitive skills in elementary school children: How you test is what you get. Metacognition and Learning, 3, 189–206.

23.

Eccles

J. S.

Roeser

R. W.

(2011). Schools as developmental contexts during adolescence. Journal of Research on Adolescence, 21, 225–241. https://doi.org/10.1111/j.1532-7795.2010.00725.x

24.

Eccles

J. S.

Wigfield

(1997). Young adolescent development. In Irvin

J. L.

(Ed.), What current research says to the middle level practitioner (pp. 15–29). National Middle School Association.

25.

Efklides

(2006). Metacognition and affect: What can metacognitive experiences tell us about the learning process? Review of Educational Research, 1, 3–14.

26.

Efklides

(2009). The role of metacognitive experiences in the learning process. Psicothema, 21, 76–82.

27.

Finch

W. H.

Cassady

J. C.

Jones

J. A.

(2016). Investigation of the latent structure for the learning and study strategies inventory. Journal of Psychoeducational Assessment, 34, 73–84.

28.

Fornell

Larcker

D. F.

(1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39–50. https://doi.org/10.1177/002224378101800104

29.

Gaski

J. F.

Nevin

J. R.

(1985). The differential effects of exercised and unexercised power sources in a marketing channel. Journal of Marketing Research, 22, 130–142. https://doi.org/10.1177/002224378502200203

30.

George

Mallery

(2003). SPSS for Windows step by step: A simple guide and reference. 11.0 update (4th ed.). Allyn & Bacon.

31.

Gettinger

Seibert

J. K.

(2002). Contributions of study skills to academic competence. School Psychology Review, 31, 350–365.

32.

Goldstein

S. E.

Boxer

Rudolph

(2015). Middle school transition stress: Links with academic performance, motivation, and school experiences. Contemporary School Psychology, 19, 21–29.

33.

Grant

Fabrigar

L. R.

(2007). Exploratory factor analysis. In Salkind

N. J.

(Ed.), Encyclopedia of measurement and statistics (pp. 332–335). Sage.

34.

Han

Ellis

R. A.

(2020). Initial development and validation of the perceptions of the blended learning environment questionnaire. Journal of Psychoeducational Assessment, 38, 168–181.

35.

Hilpert

J. C.

Stempien

van der Hoeven Kraft

K. J.

Husman

(2013). Evidence for the latent factor structure of the MSLQ: A new conceptualization of an established questionnaire. SAGE Open, 3, 2158244013510305.

36.

Hopwood

Hay

Dyment

(2016). The transition from primary to secondary school: Teachers’ perspectives. The Australian Educational Researcher, 43, 289–307. https://doi.org/10.1007/s13384-016-0200-0

37.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: A multidisciplinary journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

38.

Huang

C. C.

Wang

Y. M.

T. W.

Wang

P. A.

(2013). An empirical analysis of the antecedents and performance consequences of using the moodle platform. International Journal of Information and Education Technology, 3, 217–221. https://doi.org/10.7763/ijiet.2013.v3.267

39.

Hunter

J. E.

Hunter

R. F.

(1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72–98.

40.

Jeffrey

L. M.

(2009). Learning orientations: Diversity in higher education. Learning and Individual Differences, 19, 195–208. https://doi.org/10.1016/j.lindif.2008.09.004

41.

Kingery

J. N.

Erdley

C. A.

(2007). Peer Experience as predictors of adjustment across the middle school transition. Education and Treatment of Children, 30, 73–88.

42.

Koriat

Levy-Sadot

(2000). Conscious and unconscious metacognition: A rejoinder. Consciousness and Cognition, 9, 193–202.

43.

Linnenbrink

E. A.

Pintrich

P. R.

(2004). Role of affect in cognitive processing in academic contexts. In Motivation, emotion, and cognition: Integrative perspectives on intellectual functioning and development (pp. 57–87). Routledge.

44.

Lonka

Ketonen

Vermunt

J. D.

(2021). University students’ epistemic profiles, conceptions of learning, and academic performance. Higher Education, 81, 775–793. https://doi.org/10.1007/s10734-020-00575-6

45.

Marcoulides

G. A.

Hershberger

S. L.

(1997). Multivariate statistical methods: A first course. Psychology Press.

46.

McAlister

A. L.

Perry

C. L.

Parcel

G. S.

(2008). How individuals, environments, and health behaviors interact. In Glanz

Rimer

B. K.

Viswanath

(Eds.), Health behavior and health education: Theory, research, and practice (pp. 169–188). Jossey-Bass.

47.

McBurney

D. H.

White

T. L.

(2009). Research methods. Cengage Learning.

48.

Meeuwisse

Severiens

S. E.

Born

M. P.

(2010). Learning environment, interaction, sense of belonging and study success in ethnically diverse student groups. Research in Higher Education, 51, 528–545.

49.

Ministry of Education, University and Research. (2017). Early school leaving in the 2015/2016 school year and in the transition to the 2016/2017 school year. https://www.miur.gov.it/documents/20182/2155736/La+dispersione+scolastica+nell%27a.s.2016-17+e+nel+passaggio+all%27a.s.2017-18.pdf/1e374ddd-29ac-11e2-dede-4710d6613062?version=1.0&t=1563371652741

50.

Ministry of Education, University and Research. (2018). A national policy to combat educational failure and educational poverty. Steering committee to combat early school leaving and educational poverty. https://www.miur.gov.it/documents/20182/0/Rapporto+sul+contrasto+del+fallimento+formativo/7575f155-63f9-479a-a77f-1da743492e92?version=1.0

51.

Muthén

Kaplan

(1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38, 171–189. https://doi.org/10.1111/j.2044-8317.1985.tb00832.x

52.

Muthén

L. K.

Muthén

B. O.

(1998-2012). Mplus user’s guide (6th ed.). Muthén & Muthén.

53.

Muthén

L. K.

Muthén

B. O.

(2010). Growth modeling with latent variable using Mplus: Advanced growth models, survival analysis and missing data. Mplus Short Courses.

54.

Noyens

Donche

Coertjens

van Daal

Van Petegem

(2019). The directional links between students’ academic motivation and social integration during the first year of higher education. European Journal of Psychology of Education, 34, 67–86.

55.

Organisation for Economic Co-operation and Development. (2014). Indicator D3: How much are teachers paid? Education at a glance 2014: OECD indicators. OECD Publishing. https://www.oecd-ilibrary.org/education/education-at-a-glance-2014/indicator-d3-how-much-are-teachers-paid_eag-2014-31-en

56.

Organisation for Economic Co-operation and Development. (2016). PISA 2015 results (Volume I): Excellence and equity in education. OECD Publishing. http://dx.doi.org/10.1787/9789264266490-en.

57.

Organisation for Economic Co-operation and Development. (2018). Italian students’ performance in reading, mathematics and science, INVALSI, 2018, OECD Publishing. https://www.oecd.org/pisa/publications/PISA2018_CN_ITA_IT.pdf

58.

Päuler-Kuppinger

Jucks

(2017). Perspectives on teaching: Conceptions of teaching and epistemological beliefs of university academics and students in different domains. Active Learning in Higher Education, 18, 63–76.

59.

Pérez-Tello

Antonietti

Marchetti

Liverta Sempio

(2005). Conceptions of learning and use of cultural media. European Journal of School Psychology, 2, 127–148.

60.

Pinto

Bigozzi

Vettori

Vezzani

(2018). The relationship between conceptions of learning and academic outcomes in middle school students according to gender differences. Learning, Culture and Social Interactions, 16, 45–54. https://doi.org/10.1016/j.lcsi.2017.11.001

61.

Pintrich

P. R.

(1999). The role of motivation in promoting and sustaining self-regulated learning. International Journal of Educational Research, 31, 459–470.

62.

Pintrich

P. R.

(2000). Multiple goals, multiple pathways: The role of goal orientation in learning and achievement. Journal of Education & Psychology, 92, 544–555.

63.

Pintrich

P. R.

Smith

Garcia

McKeachie

(1991). A manual for the use of the motivated strategies for learning Questionnaire (MSLQ). The University of Michigan.

64.

Reeve

Tseng

C. M.

(2011). Agency as a fourth aspect of students’ engagement during learning activities. Contemporary Educational Psychology, 36, 257–267.

65.

Rhemtulla

Brosseau-Liard

P. É.

Savalei

(2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354–373. https://doi.org/10.1037%2Fa0029315

66.

Robbins

S. B.

Lauver

Davis

Langley

Carlstrom

(2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261–288.

67.

Ryan

R. M.

Deci

E. L.

(2000). The darker and brighter sides of human existence: Basic psychological needs as a unifying concept. Psychological Inquiry, 11, 319–338.

68.

Schunk

D. H.

Zimmerman

B. J.

(1997). Social origins of self-regulatory competence. Educational Psychologist, 32, 195–208.

69.

Suh

H. N.

Wang

K. T.

Arterberry

B. J.

(2015). Development and initial validation of the self-directed learning inventory with Korean college students. Journal of Psychoeducational Assessment, 33, 687–697.

70.

Van Rossum

E. J.

Hamer

R. N

. (2010). The meaning of learning and knowing. Sense publishers.

71.

Vermetten

Y. J.

Lodewijks

H. G.

Vermunt

J. D.

(1999). Consistency and variability of learning strategies in different university courses. Higher Education, 37, 1–21.

72.

Vermunt

J. D.

Donche

(2017). A learning patterns perspective on student learning in higher education: state of the art and moving forward. Educational Psychology Review, 29, 269–299.

73.

Vettori

Vezzani

Bigozzi

Pinto

(2018). The mediating role of conceptions of learning in the relationship between metacognition and academic outcomes among middle school students. Frontiers in Psychology, 9, 1985. https://doi.org/10.3389/fpsyg.2018.01985

74.

Vettori

Vezzani

Bigozzi

Pinto

(2021). Patterns of academic performances and conceptions of learning in on-time, delayed, and dropout university participants: A Cohort Retrospective Study. Educational Psychology, 41, 1024–1040. https://doi.org/10.1080/01443410.2021.1918643

75.

Vezzani

Vettori

Pinto

(2018). Assessing students’ beliefs, emotions and causal attribution: Validation of ‘Learning Conception Questionnaire’. South African Journal of Education, 38, 1–18. https://doi.org/10.15700/saje.v38n2a1453

76.

Corno

(2003). Family help and homework management reported by middle school students. The Elementary School Journal, 103, 503–517.

77.

Zimmerman

B. J.

Schunk

D. H.

(2011). Handbook of self-regulation of learning and performance. Routledge/Taylor & Francis Group.

Patterns of Cognitive,Metacognitive,and Affective Dimensions in Learning Orientations: A Preliminary Validation of a Self-Report Questionnaire for Middle School Students

Abstract

Keywords

Introduction

Students’ Learning Orientations

Assessing Students’ Learning Orientations Significant for School Achievement

Rationale

Aims

Study 1

Method

Selection of the Initial Pattern of Specific Items for the Self-Report Questionnaire LO-COMPASS

Participants and Procedure

Data Analysis and Results

Study 2

Method

Participants and Procedure

Data Analysis and Results

Discussion

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iD

References