Sage Journals: Discover world-class research

Abstract

Research and practice have recognized the importance of informal learning—a specific type of active learning—for higher education contexts. University students learn not only in formally organized courses, but also in a self-directed and intentional way from fellow students, through trial and error, and by reflection. However, there has been a lack of valid measures to operationalize students’ informal learning. In this study, we thus develop the Informal Student Learning (ISL) scale for higher education, building on the Octagon Model of Informal Workplace Learning and the model’s associated measure from the vocational learning context. Our scale contains eight components with three items each. Using three samples of university students (N = 545, N = 818, N = 310), we examined the model structure of ISL and the scale’s validity in an international context. The results show that the conceptual structure of informal workplace learning and informal student learning is similar for intent to learn and reflection, but different for experience/action and feedback. Examining validity, we found evidence for concurrent validity in terms of study-related self-efficacy and academic performance, and for convergent and discriminant validity indicators of the eight ISL components. The scale has configural and metric measurement invariance for age, gender, and academic achievement, and additionally scalar invariance for age. Potential applications of the new measure in the context of active learning for research, for lecturers, and for students are discussed.

Keywords

development feedback higher education informal learning measure reflection university students validation

The term active learning refers to “moving away from teacher-centered instruction where the teacher controls what gets taught, when it gets taught and the pace by which it gets taught to a student-centered approach” (Nicol et al., 2018: 254). A specific type of active learning is informal learning, which is self-directed by the learner through intentional engagement in, e.g., trying things out, seeking feedback, and reflection (Decius et al., 2019, 2021). Informal learning plays an important role even in formalized student education, because “it is hard to imagine a formal learning context in which only explicit learning of explicit knowledge takes place. To focus only on the explicit learning of formally presented knowledge is to fail to recognize the complexity of learning even in well-ordered classrooms” (Eraut, 2000: 131). Clark (2016) has put it even more distinctly: “Formal learning is incorrectly assumed to occur only in formal settings, and informal and self-directed learning incorrectly assumed to occur only outside of classrooms or institutions” (p. 561; emphases added).

Informal learning may support formal learning by filling gaps in the formal curriculum and contributing to a better understanding of formal learning content (Gramatakos and Lavau, 2019; Peeters et al., 2014). Scholars expect that informal learning will become an even more important part of students’ education and should be investigated more thoroughly (Barth et al., 2007; Jamieson, 2009; Peeters et al., 2014), particularly in view of increasingly heterogeneous student populations (Mertens et al., 2018). Yang and Lu (2001) found that formal ability tests predict one quarter of the variation in academic performance; they concluded that informal learning might also account for a high proportion of the remaining three quarters. This rationale is consistent with findings of vocational psychology research, according to which 70% to 90% of work-related learning occurs informally (Cerasoli et al., 2018). However, there is currently a lack of quantitative empirical studies to support this assumption for higher education.

One reason is that research is hindered by a lack of measures (cf. Gramatakos and Lavau, 2019)—informal learning scales exist in vocational research only (see Park et al., 2021, for an overview). Instead, scholars focused on the recognition of (informally acquired) knowledge for accreditation in higher education systems (Gallacher and Feutrie, 2003; Stenlund, 2010), on part-time work of students as learning opportunity (Muldoon, 2009), or on informal learning in the transition from higher education to work (Grosemans et al., 2018). Attempts were made to create scales designed for psychology students to capture non-obligatory study activities (Soyyılmaz et al., 2017) and to identify cognitions, metacognitions, and motivation to collaborate in the context of “digital informal learning” (He and Zhu, 2017). However, these approaches were neither validated nor based on conceptually sound models of informal learning.

In this article, we therefore present the validation of a measure for operationalizing informal student learning (ISL) in higher education, which is conceptually based on the Octagon Model of Informal Learning from the vocational learning context (Decius et al., 2019). For this purpose, we transfer and modify an already validated informal workplace learning (IWL) scale (Decius et al., 2019) into higher education, developing the ISL scale. Using two German samples and one U.S. sample of university students, we investigate the structure and validity of the new measure in German and English.

As a theoretical contribution, we examine whether the Octagon Model of IWL is applicable to ISL. We elaborate structural similarities and differences between IWL and ISL and present a new structure for ISL that deviates from the Octagon Model. By providing a reliable, valid, and economic measure, we support psychological research and practice to quantify ISL, for example, to investigate facilitative learning conditions in higher education.

Informal learning

Informal learning in higher education

Higher education literature distinguishes three “curricula”: the formal curriculum, the hidden curriculum, and the informal curriculum (Winter and Cotton, 2012). The formal curriculum defines what students should learn in prescribed courses (e.g. lectures and seminars; Hopkinson et al., 2008; Winter and Cotton, 2012). The hidden curriculum comprises implicit behaviors, beliefs, and attitudes (i.e. social and cultural rules, including disciplined behavior and compliance with deadlines; Eraut, 2000) that students learn unconsciously through lecturers and peers (Peeters et al., 2014; Winter and Cotton, 2012). The informal curriculum refers to the personal and unstructured information transfer from lecturers to learners before or after lessons (e.g. daily conversations and small talk; Peeters et al., 2014). Furthermore, the informal curriculum includes student-led activities such as student projects, group activities, voluntary courses, and other extra-curricular activities (Hopkinson et al., 2008; Winter and Cotton, 2012). These informal curriculum activities are “largely student directed, voluntary, open to all and non-credit bearing” (Hopkinson et al., 2008: 439).

Definitional approaches to informal learning discussed so far in higher education, such as the “informal curriculum,” only cover learning behaviors. Informal learning, however, is a multidimensional construct consisting of behavioral, cognitive, and motivational components (Decius et al., 2021). These include applying one’s own solutions, reflecting on work processes, sharing experiences with others, and seeking feedback (Decius et al., 2019). A measure that claims to operationalize ISL holistically should take this multidimensionality into account.

In accordance with vocational learning research (Cerasoli et al., 2018; Decius et al., 2019), we therefore define informal student learning as all behavioral, cognitive, and intentional processes of students that serve the purpose of study-related knowledge acquisition but occur self-directedly outside of formally prescribed, curriculum-based learning objectives. Informal learning in studies is practice-based and task-oriented learning. We understand task both as a study assignment in courses and as a voluntary activity outside of courses, such as in student-driven projects. According to this interpretation, ISL can take place casually during courses in formal settings—but also outside formal settings that are embedded in a social context and thus characterized by interactions with fellow students or lecturers. Here, informal learning shares features of situated learning (cf. Lave and Wenger, 1991). An ISL example during a course is the adoption of successful strategies from fellow students (model learning). Outside of formal settings, students may learn, for instance, when they attend an instructor’s office hours to seek feedback, or when they reflect at home on study-related topics and their own academic performance.

Research assumes that students appreciate ISL opportunities to get feedback on their ideas, to get suggestions to reflect on their thoughts, and to try out their solutions directly (Ebner et al., 2010). However, previous research focused almost exclusively on behavioral ISL elements, such as (lunchtime) discussions with fellow students and teaching staff, use of social networks and online forums, (audio/video) chatting, voluntary or additional preparation for classes and assignments, reading (non-compulsory) books, volunteering in student groups, and participating in campus events (Barth et al., 2007; Clark, 2016; Gramatakos and Lavau, 2019; Hopkinson et al., 2008; Jamieson, 2009; Kassens-Noor, 2012; Martindale and Dowdy, 2010; Mertens et al., 2018; Soyyılmaz et al., 2017; Toffoli and Sockett, 2015). Consequently, evidence on the cognitive and intentional ISL components is lacking. The development of a multidimensional scale could provide impetus for the holistic investigation of ISL.

Differentiation from self-regulated learning

Active learning includes not only informal learning but also self-regulated learning (Nicol et al., 2018). Self-regulated learning is one of the most researched concepts within educational psychology (Panadero, 2017). It is defined as “an active, constructive process whereby learners set goals for their learning and then attempt to monitor, regulate, and control their cognition, motivation, and behavior, guided and constrained by their goals and the contextual features in the environment” (Pintrich, 2000: 453). Self-regulated learning in students can be fostered through teaching strategies employed by instructors (Russell et al., 2020) and through automated time management tools (Khiat, 2022). Although self-regulated learning and informal learning both occur independently of external structural requirements and the responsibility for the learning process remains with the learner, both learning forms differ substantially.

The purpose of informal learning is not learning itself, but rather the solution of a concrete problem that arises when working on a task (Segers et al., 2018; cf. Ebner et al., 2010). The focus in self-regulated learning, in contrast, is on the learning process—the learner sets a concrete learning goal, often in the context of formal university courses. The goal, however, may coincide with or be influenced by course requirements, for example, in self-study phases. Self-regulated learning is deliberately planned by the learner, whereas informal learning occurs spontaneously in response to the demands of a situation or activity to be mastered (cf. Eraut, 2000). In case of informal learning, learners monitor their study processes and stop learning as soon as the problem is solved; in case of self-regulated learning, learners monitor the achievement of their self-defined learning goals and may continue learning by setting new goals.

The Octagon Model of informal learning

Informal learning is a multidimensional construct (Decius et al., 2021). The ISL measure presented in this study is based on the Octagon Model of IWL (Decius et al., 2019), which is an extension of the Dynamic Model of Informal Learning (Tannenbaum et al., 2010). The Octagon Model contains the four factors of the dynamic model and divides each into two components: experience/action into trying/applying own ideas and model learning, feedback into direct feedback and vicarious feedback, reflection into anticipatory reflection and subsequent reflection, and intent to learn into intrinsic intent to learn and extrinsic intent to learn. The Octagon Model is a multilevel model with one core factor Informal Learning, four factors on the first level, and eight components on the second level (1-4-2 structure).

We consider the factors experience/action, feedback, and reflection as situational and behavioral, and the factor intent to learn as motivational and rather stable. The behavior-related factors correspond to the three dimensions of student-led learning activities according to Gramatakos and Lavau (2019): The cognitive dimension represents reflection, the practical dimension represents experience/action, and the affective dimension—described as “listen and respond in interactions with others” (p. 387)—represents feedback. We assume that all IWL components are relevant for ISL, and that the Octagon Model division of eight components into four higher-order factors is also evident in ISL.

H1: The structural division of IWL factors can be transferred to ISL.

a) Experience/action consists of trying/applying own ideas and model learning.

b) Feedback consists of direct feedback and vicarious feedback.

c) Reflection consists of anticipatory reflection and subsequent reflection.

d) Intent to learn consists of intrinsic intent to learn and extrinsic intent to learn.

In addition to the 1-4-2 structure postulated by the Octagon Model, three alternative structures are conceptually conceivable (cf. Decius et al., 2019). First, the eight components could correlate without any higher-order factor (0-0-8 structure). Second, a core factor informal learning could be superordinate to the eight components, but without an intermediate level (1-0-8 structure). Third, two of the eight components each could belong to a higher-order factor, but without a superordinate core factor (0-4-2 structure). However, we expect ISL to be best modeled using the 1-4-2 structure of the Octagon Model.

H 2: The 1-4-2 structure is superior to alternative ISL model structures.

a) The 1-4-2 structure has a better model fit than the 0-0-8 structure.

b) The 1-4-2 structure has a better model fit than the 1-0-8 structure.

c) The 1-4-2 structure has a better model fit than the 0-4-2 structure.

Validity hypotheses

Conceptually, the eight ISL components belong to a higher-order core construct. We thus expect that the ISL components overlap in variance. To test this assumption, we use the average variance extracted (AVE; Fornell and Larcker, 1981): “The AVE estimate is the average amount of variation that a latent construct is able to explain in the observed variables to which it is theoretically related” (Farrell, 2010, p. 324). If an AVE value is >.50, this indicates convergent validity (Hair et al., 2010).

We also expect the components to differ sufficiently to be distinct ISL facets. We therefore use the shared variance (SV; i.e. the squared correlation) across the components. If the AVE values are greater than the SV value between two components of interest, this provides evidence for discriminant validity (Farrell, 2010).

H3: The eight ISL components show convergent validity.

H4: The eight ISL components show discriminant validity.

For examining concurrent validity, we address self-efficacy—a principal construct of Bandura’s Social Cognitive Theory and one of the most studied constructs in academic learning. “Perceived self-efficacy is a judgment of one’s ability to organize and execute given types of performances” (Bandura, 1997: 21). Self-efficacy plays an important role “in influencing such key indices of academic motivation as choice of activities, level of effort, persistence, and emotional reactions” (Zimmerman, 2000: 86) and affects diverse learning constructs in higher education (see van Dinther et al., 2011, for an overview). While empirical research on ISL is scarce, the relationship between IWL and self-efficacy is well established (e.g. Choi and Jacobs, 2011; Kyndt et al., 2016; Noe et al., 2013; van Daal et al., 2014). Based on the conceptual importance for learning processes, we expect a positive relationship between self-efficacy and ISL.

H5: ISL correlates positively with study-related self-efficacy.

Furthermore, we use academic performance as another variable for testing concurrent validity. Academic performance—usually measured by grades averaged across all courses—depends on many academic, psychosocial, cognitive, and demographic factors (McKenzie and Schweitzer, 2001). Informal learning is also considered a predictor of academic performance (Yang and Lu, 2001). We therefore expect a positive relationship ISL and academic performance.

H6: ISL correlates positively with academic performance.

Measurement invariance hypotheses

It is important that the items of a scale are understood and interpreted equally by individuals from different subgroups. If the items are interpreted identically, the scale demonstrates measurement invariance (Vandenberg and Lance, 2000). This is relevant to the scale’s use for the following reason: If, for instance, some items were understood fundamentally differently by men than by women, this could lead to measurement bias. If researchers then find mean differences between men and women, they cannot say for sure whether these differences exist in reality or are only due to the bias caused by the differently interpreted scale.

For ruling out such biases, a scale should have at least configural and metric, and preferably also scalar measurement invariance (Sass, 2011; Vandenberg and Lance, 2000). To test the measurement invariance of the ISL scale, we identify three relevant subgroup characteristics: The scale should be interpreted identically by younger and older students, by female and male students, and by lower performing and higher performing students. We anticipate equivalence between the measurement invariance models on these criteria, which would argue for the scale’s generalizability.

H7: The ISL scale shows measurement invariance among age subgroups (younger vs. older students).

H8: The ISL scale shows measurement invariance among gender subgroups (male vs. female).

H9: The ISL scale shows measurement invariance among performance subgroups (lower-performing vs. higher-performing students).

Scale development

The basis for the ISL scale development was the IWL scale (Decius et al., 2019). This 24-item measure covers each Octagon Model component with three items. We adopted the scale for higher education, replacing company with university, colleagues with fellow students, foreman or head with lecturer, and during work with during studies. All items were measured on a 6-point Likert-type scale (1 = Not agree at all, 2 = Largely not agree, 3 = Rather not agree, 4 = Rather agree, 5 = Largely agree, 6 = Fully agree).

Following recommendations of DeVellis (2003) and MacKenzie et al. (2011), we developed one back-up item for each ISL component, to be used if one of the three original items failed in the higher education context. This resulted in 32 ISL items in the raw measure. We tested the comprehensibility using thinking aloud interviews (Flaherty, 1975) with five students. For this purpose, we provided the students with the questionnaire including the Likert scale and let them fill it out. The students were asked to indicate why they chose an answer option, which trigger words they based their decision on, and which situations from their everyday study life they thought of when answering each item.

Study 1: ISL structure examination

The first objective of Study 1 is to test whether the structural division of IWL factors of the Octagon Model (Decius et al., 2019) can be applied to ISL (H1). The second objective is to investigate whether the Octagon Model structure is superior to alternative model structures (H2).

Participants and instruments

We recruited a sample of 545 students from a German university. 67.7% of the students were female. Fields of study were Economics (51.3%) and Education (42.1%), 6.6% miscellaneous. The average age was 22.5 years (SD = 3.0). The participants answered the 32-item ISL raw measure along with questions on demographic data and study-related self-efficacy. For this purpose, we replaced occupation with studies in the 6-item scale of Abele et al. (2000). An example item is: “I face difficulties in my studies calmly because I can trust my abilities.”

Procedure and results

Structural division of ISL factors

First, we examined the descriptive characteristics of the 24 modified items, ignoring the back-up items in this step (see Table 1; for correlations see Table C in Electronic Supplemental Material (ESM)). The item means are between 2.77 (DF1) and 4.73 (II1) with SD between 1.04 (II1) and 1.50 (DF2). Two items show a discriminatory power (i.e. item-total correlation corrected, within the corresponding three-item component) outside the optimal range for validity and reliability from 0.30 to 0.80 (Lienert and Raatz, 1998: 255): DF3 (0.05) and EI1 (0.27). Due to these items, the internal consistency of two components is substantially below the Cronbach’s alpha limit of 0.7 (Minium, 1992): direct feedback with α = 0.47, ω = 0.60; and extrinsic intent to learn with α = 0.58, ω = 0.62. This is explainable in terms of item content: The IWL scale involves obtaining feedback from superiors with two items and from colleagues with one item (DF3). In higher education, students seem to obtain feedback mainly from lecturers, not from fellow students (with whom they talk rather about hints for studying). We thus replaced item DF3 (i.e. feedback from fellow students) with the back-up item DFbu that covers feedback from lecturers. The more homogeneous component then showed a better consistency (α = 0.77, ω = 0.76). In case of EI1, the item text aims at the extrinsic goal of advancing one’s own “study career.” Although the term “study career” was not questioned in the pre-tests, this metaphor may not be understood equally well. After replacing EI1 with EIbu, which aims at making one’s own performance visible during the course, we received a higher consistency (α = 0.72, ω = 0.72).

Table 1.

Informal student learning scale items.

Initial factor structure	Item no.	Original items (work context)	Modified items (study context)	Included in final scale	Final factor structure	M	SD	Skew	Kurtosis	Discr. power
Exp/Act	TA1	I try a different method to solve new tasks at work.	If I have difficulties with a task in my studies, I simply use a different method.	Yes	Trial/Error	3.52	1.13	0.11	−0.46	0.34
Exp/Act	TA2	I try out my own ideas for new tasks.	I try out my own ideas for new tasks during my studies.	Yes	Trial/Error	3.93	1.21	−0.43	−0.22	0.64
Exp/Act	TA3	I use my own ideas to improve tasks at work.	I use my own ideas to find solutions to the tasks in my studies.	Yes	Trial/Error	4.17	1.14	−0.48	−0.14	0.58
	TAbu		I try out new things in my study tasks that I don’t know yet.	No
Exp/Act	ML1	I look at how others work in the company to improve my work.	I look at how others work at the university to improve my work during my studies.	Yes	Fellows	4.01	1.49	−0.57	−0.70	0.58
Exp/Act	ML2	I look at how my colleagues work so I do not make the same mistakes as they do.	I look at how my fellow students work during their studies so that I don’t make the same mistakes they make.	Yes	Fellows	4.07	1.39	−0.54	−0.61	0.67
Exp/Act	ML3	I try things out at my work, which I have copied from my colleagues.	I try out methods in my studies that I have copied from my fellow students.	Yes	Fellows	4.02	1.26	−0.50	−0.40	0.72
	MLbu		I learn especially quickly from the mistakes of others in my studies.	No
Feedback	DF1	I ask my foreman or head how well I have worked.	I seek feedback from my lecturers on how well I have worked on a task during my studies.	Yes	Trial/Error	2.77	1.45	0.62	−0.58	0.62
Feedback	DF2	I ask my foreman or head when I am not sure how well I worked.	I ask my lecturers if I am not sure whether my solution to a problem is the right one.	Yes	Trial/Error	3.43	1.53	0.01	−1.05	0.68
Feedback	DF3	I ask my colleagues when I am not sure how well I worked.	I often ask my fellow students if I am not sure whether my solution to a task is the right one.	No		4.63	1.28	−0.81	−0.11	0.05*
	DFbu		My lecturers tell me what I can do better in my study tasks.	Yes	Trial/Error	2.98	1.33	0.26	−0.67	0.56
Feedback	VF1	I ask my colleagues about their experience at work.	I often ask my fellow students about their experiences during their studies.	Yes	Fellows	4.66	1.24	−0.87	0.17	0.73
Feedback	VF2	I ask my colleagues about the methods and tricks they use at work.	I often ask my fellow students which methods and tricks they use in their studies.	Yes	Fellows	4.32	1.29	−0.54	−0.28	0.80
Feedback	VF3	I obtain tips and hints about work from my colleagues.	I often ask my fellow students for tips and hints on studying.	Yes	Fellows	4.37	1.33	−0.70	−0.10	0.81
	VFbu		My fellow students talk to me about how I do in my study tasks.	No
Reflection	AR1	Before starting a new task, I think about how I can do my work best.	Before taking on a new task during my studies, I think about how I can best do my job.	Yes	Reflection	4.64	1.11	−0.77	0.38	0.52
Reflection	AR2	Before work, I think about how I prepare my workplace.	Before a course in my studies, I think about how to prepare myself for it.	Yes	Reflection	3.67	1.31	−0.11	−0.74	0.47
Reflection	AR3	Before starting a new task, I think about the things I need to pay attention to.	Before a new task in my studies, I always think about what things I have to pay attention to in my work.	Yes	Reflection	4.30	1.08	−0.46	−0.01	0.50
	ARbu		Before a new assignment in study, I think about where there might be a problem in the assignment.	No
Reflection	SR1	When I have finished a new task, I think about how well I have worked.	When I have finished a new task in my studies, I think about how well I have worked.	Yes	Reflection	4.32	1.26	−0.62	−0.23	0.62
Reflection	SR2	When I have finished a new task, I think about what I still could do better next time.	When I finish a new task in my studies, I think about what I could do better next time.	Yes	Reflection	4.54	1.18	−0.73	0.07	0.64
Reflection	SR3	When I have finished a new task, I think about the quality of my work.	When I have finished a new task in my studies, I always critically question my own solution.	Yes	Reflection	4.09	1.38	−0.24	−0.86	0.52
	SRbu		When I finish a new task in study, I check the result of my work.	No
L Intent	EI1	I want to learn something new at work for myself because then I can pursue my career at the company.	I want to learn something for myself in my studies outside of courses to really push my “study career” forward.	No		4.36	1.17	−0.46	−0.27	0.27*
L Intent	EI2	I want to learn something new for myself because then I am better at work than my colleagues.	I want to learn something for myself in my studies outside of courses, because then I am better in my studies than my fellow students.	Yes	L Intent	3.21	1.48	0.07	−1.03	0.48
L Intent	EI3	I want to learn something new at work for myself because then my foreman or head is impressed by me.	I want to learn something for myself in my studies outside of courses, because then I can prove myself better to my lecturers.	Yes	L Intent	2.89	1.40	0.27	−0.89	0.58
	EIbu		I want to learn something for myself in my studies outside of courses, because then one can see in the courses what I am able to do.	Yes	L Intent	2.88	1.37	0.32	−0.74	0.54
L Intent	II1	I want to learn something new for myself because then I feel more capable to deal with difficulties at work.	I want to learn something for myself in my studies outside of courses, because then I can deal better with difficulties and challenges in my studies.	Yes	L Intent	4.73	1.04	−0.57	−0.17	0.79
L Intent	II2	I want to learn something new for myself because then I can solve problems at work faster.	I want to learn something for myself in my studies outside of courses, because then I can solve problems in my studies much faster.	Yes	L Intent	4.66	1.08	−0.56	−0.21	0.84
L Intent	II3	I want to learn something new for myself because then I can do a good job even though the tasks or instructions are difficult.	I want to learn something for myself in my studies outside of courses, because then I can work well even with difficult tasks or instructions during my studies.	Yes	L Intent	4.64	1.05	−0.58	−0.11	0.81
	IIbu		I want to learn something for myself in my studies outside of courses, because then I can cope better with changed tasks in my studies.	No

N varies between 542 and 545. SD Skewness = 0.40; SD Kurtosis = 0.38; bu = back-up item; Discr. power = discriminatory power (i.e. item-total correlation corrected) according each final three-item component; *Discriminatory power according the three-item component without back-up item; Exp/Act: Experience/Action; L Intent: Intent to learn; TA: trying/applying own ideas; ML: model learning; DF: direct feedback; VF: vicarious feedback; AR: anticipatory reflection; SR: subsequent reflection; EI: extrinsic intent to learn; II: intrinsic intent to learn. This measure is non-proprietary (free) and may be used without permission. English items were translated from original German items (see Table A in Electronic Supplemental Material (ESM)) using a translation/backtranslation procedure. Original items (work context) are taken from Decius et al. (2019).

With the final 24 items—that is, including the two back-up items—we calculated a confirmatory factor analysis (CFA), using the 1-4-2 structure of the Octagon Model as for IWL. For model fit evaluation, we relied on χ² and the following global model fit criteria recommended by Kline (2016: 269): Comparative Fit Index (CFI), Standardized Root Mean Square (SRMR), Root Mean Square Error of Approximation (RMSEA). The model fit was acceptable: χ²(240) = 499.144, p < 0.001; CFI = 0.948; SRMR = 0.061; RMSEA = 0.045, 90% CI = [0.039, 0.050]. However, the results included negative error variance for experience/action and feedback. Such Heywood cases indicate a misspecification of the model (Kolenikov and Bollen, 2012).

We therefore ran an exploratory factor analysis (EFA) to find a better fitting structure. First, we split the total sample into two halves using a random algorithm to subsequently test the new structure using CFA. We used half 1 (N = 273) for EFA and half 2 (N = 272) for CFA. Following methodological recommendations (Conway and Huffcutt, 2003; Costello and Osborne, 2005), we applied principal axis analysis as a factor extraction method and promax rotation as an oblique rotation technique. The Kaiser-Meyer-Olkin (KMO) index was 0.79, so the sample is suitable for conducting an EFA (Hutcheson and Sofroniou, 1999). To determine the number of factors to extract, we used parallel analysis and Velicer’s minimum average partial (MAP) test, which are superior to other methods such as the rule of thumb “Eigenvalue > 1” (Conway and Huffcutt, 2003). Both methods revealed eight factors to extract.

EFA results (see Table B in ESM) showed that three items each can be assigned to one factor. The assignment of 24 items to eight factors was identical to the structure assumed in the previous CFA (N = 545). The reason for the model’s misspecification was therefore not the assignment of the items to the components but could be the relational structure between the components. A subsequent CFA with half 2—in which the eight components did not belong to a higher structure, but were allowed to covariate (0-0-8 structure)—supported this assumption by showing a good model fit, χ²(224) = 302.884, p < 0.001; CFI = 0.969; SRMR = 0.045; RMSEA = 0.036, 90% CI = [0.025, 0.046].

These results suggest that the same eight components are relevant for IWL and ISL—but the higher-level structure of informal learning differs. We explain this conceptually: The IWL scale was developed for blue-collar workers, for whom (direct and vicarious) feedback and experience/action (trying/applying own ideas and model learning) are two separate domains. Model learning among blue-collar workers often directly relates to the work process, as the workers copy actions from their colleagues and integrate them straight into their own work (Decius et al., 2019). Model learning and personal experimentation are thus highly connected. Students, however, learn more cognitively and abstractly from a model: They analyze their fellow students’ behavior, but rarely integrate it without delay into their actions (cf. Gramatakos and Lavau, 2019). Model learning in higher education resembles vicarious feedback (i.e. exchange of experience on tips and tricks), and has fewer similarities with trying things out. Otherwise, students learn by trial (trying/applying own ideas) and error (direct feedback on their performance) (Marques et al., 2013). We therefore assume that—in contrast to the Octagon Model—in higher education model learning and vicarious feedback form a higher-order factor learning from fellow students, and trying/applying own ideas and direct feedback form a higher-order factor learning by trial & error. However, the assignments of reflection and intent to learn remain the same as in the Octagon Model (Figure 1).

Figure 1.

Structural model of informal student learning.

An examination of this assumed structure (1-4-2 structure with the new four ISL factors) using a CFA with half 2 supported this assumption providing a good model fit, χ²(240) = 313.439, p < 0.001; CFI = 0.971; SRMR = 0.047; RMSEA = 0.034, 90% CI = [0.022, 0.043]. Furthermore, the calculation of the 0-0-8 structure mentioned above indicated significant positive correlations between the two reflection components (0.65; p < 0.001) and intention components (0.33; p < 0.001), but no significant correlations between the experience/action components (0.07, p = 0.315) and feedback components (0.12; p = 0.093). H1 c) and d) are thus supported by the data; H1 a) and H1 b) do not receive support.

Comparison of different model structures

Also using half 2 of the sample, we tested alternative model structures according to hypothesis 2. We additionally used two model fit criteria well suited for model comparisons—the Akaike Information Criterion (AIC) and the Bayes Information Criterion (BIC)—and applied χ² difference tests (Kline, 2016). The 1-4-2 model structure proved superior to the 1-0-8 structure, but we did not find significant differences to the 0-0-8 and 0-4-2 structure (Table 2). This supports H2 b) but not a) and c). However, based on our previous conceptual considerations, we finally keep the 1-4-2 model structure.

Table 2.

Model fit indices of alternative model structures in Study 1.

Model structure	χ²/df	p	CFI	SRMR	RMSEA, 90% CI	AIC	BIC	Comparison to 1-4-2 model
1-4-2	313.439/240 = 1.306	0.001	0.971	0.047	0.034 [0.022, 0.043]	19,167.791	19,470.679
0-0-8	302.884/224 = 1.352	<0.001	0.969	0.045	0.036 [0.025, 0.046]	19,189.236	19,549.816	Δχ² = 10.556, Δdf = 16, p = 0.836
1-0-8	381.897/244 = 1.565	<0.001	0.946	0.071	0.046 [0.037, 0.054]	19,228.249	19,516.713	Δχ² = 68.457, Δdf = 4, p < 0.001
0-4-2	313.027/238 = 1.315	0.001	0.971	0.047	0.034 [0.023, 0.044]	19,171.379	19,481.478	Δχ² = 0.412, Δdf = 2, p = 0.814

N = 272; 1-4-2: one core factor, four higher-order factors, two components under each factor. 0-0-8: eight covarying components without a higher-order structure. 1-0-8: one core factor, eight components below it. 0-4-2: four covarying second-order factors, two components under each factor.

Study 2: Examining validity and measurement invariance

The first objective of Study 2 is to replicate the 1-4-2 model structure in an independent sample, and to investigate the ISL scale’s convergent and discriminant validity (H3, H4). The second objective is to investigate concurrent validity regarding self-efficacy (H5) and academic performance (H6). The third objective is to inspect measurement invariance regarding age (H7), gender (H8), and academic performance (H9).

Participants and instruments

We recruited a sample of 818 students from several German universities, as part of a larger project. We advertised study participation online through student organizations and did not track participants’ university affiliation for anonymity. About 69.7% of the students who indicated their gender were female. The most frequently represented fields of study were Social Sciences/Economics (24.4%) and Humanities/Education (24.4%). The average age was 23.0 years (SD = 3.3). The participants completed the final 24-item ISL scale of study 1 (for item and construct correlations see Table C in ESM) along with questions on demographic data and the same self-efficacy scale as in study 1.

Procedure and results

A CFA on ISL with the 1-4-2 structure yielded a good model fit in sample 2, too, χ²(240) = 556.439, p < 0.001; CFI = 0.969; SRMR = 0.043; RMSEA = 0.040, 90% CI = [0.036, 0.045]. This confirms our model’s structural assumption.

Convergent and discriminant validity

For examining convergent and discriminant validity, we calculated the values for AVE and SV using structural equation modeling (SEM). The AVE values of all eight ISL components are >0.50—ranging between 0.52 and 0.82 (see Table 3). We thus can assume convergent validity (Hair et al., 2010). The SV values between two components each range between 0.00 and 0.42. Therefore, the two AVE values in each case are larger than the corresponding SV value, indicating evidence for discriminant validity (Farrell, 2010). The results support H3 and H4.

Table 3.

Average variance extracted, shared variance, internal consistencies, and correlations of Study 2.

Construct		α	ω	1	2	3	4	5	6	7	8	9
1	TA	0.81	0.83	0.63	0.03	0.09	0.00	0.14	0.10	0.04	0.07	–
2	ML	0.80	0.81	0.17^***	0.58	0.05	0.42	0.06	0.05	0.04	0.03	–
3	DF	0.80	0.80	0.25^***	0.18^***	0.57	0.01	0.14	0.10	0.13	0.04	–
4	VF	0.92	0.92	0.00^†	0.55^***	0.10^**	0.79	0.03	0.03	0.01	0.02	–
5	AR	0.76	0.77	0.34^***	0.20^***	0.29^***	0.14^***	0.52	0.40	0.10	0.18	–
6	SR	0.80	0.80	0.29^***	0.18^***	0.26^***	0.14^***	0.49^***	0.58	0.11	0.22	–
7	EI	0.78	0.79	0.18^***	0.16^***	0.29^***	0.10^**	0.27^***	0.30^***	0.56	0.19	–
8	II	0.93	0.93	0.27^***	0.15^***	0.18^***	0.14^***	0.37^***	0.41^***	0.40^***	0.82	–
9	SE	0.78	0.78	0.30^***	−0.06^†	0.18^***	−0.06^†	0.07^*	0.11^**	0.07^*	0.13^***	–
10	AP	–	–	0.17^***	−0.04^†	0.24^***	−0.05^†	0.08^*	0.08^*	0.16^***	0.07^†	0.40^***

N = 818; The average variance extracted (AVE) of each component is presented on the diagonal in bold. The shared variance (i.e. squared latent factor correlation) is presented above the diagonal in italics. The manifest (i.e. mean-based) correlation is presented below the diagonal. TA: trying/applying own ideas; ML: model learning; DF: direct feedback; VF: vicarious feedback; AR: anticipatory reflection; SR: subsequent reflection; EI: extrinsic intent to learn; II: intrinsic intent to learn; SE: self-efficacy; AP: academic performance (study grades, reverse coded). Correlations only are marked according to significance: ^†non-significant, *p < 0.05, **p < 0.01, ***p < 0.001.

Concurrent validity

For concurrent validity, we examined the relationship between ISL, study-related self-efficacy, and academic performance. We operationalized academic performance with study grades (nine levels), which, unlike self-reported data, can be compared more objectively. Latent modeling of these three constructs using SEM—ISL with the 1-4-2 structure as predictor of self-efficacy and academic performance—yielded an acceptable model fit, χ²(420) = 1218.995, p < 0.001; CFI = 0.933; SRMR = 0.059; RMSEA = 0.048, 90% CI = [0.045, 0.051]. ISL and self-efficacy (β = 0.28) and ISL and academic performance (β = 0.22) correlate positively, supporting H5 and H6.

A cross-check with Study 1 data (N = 545) yielded similar results, χ²(420) = 1070.459, p < 0.001; CFI = 0.901; SRMR = 0.063; RMSEA = 0.053, 90% CI = [0.049, 0.057], ISL and self-efficacy (β = 0.32), and ISL and academic performance (β = 0.23).

Measurement invariance

All models of configurational invariance have a strong model fit, so we can assume configurational invariance for age, gender, and performance (see Table 4). Setting the item loadings equal across the subgroups did not significantly change the model fit, so we can assume metric invariance. Next, we additionally set the intercepts equal between the subgroups. There was no significant change in model fit—signaling scalar invariance—for age, but there was for gender and performance (see Table 4). The data thus show support for H7 and limited support for H8 and H9.

Table 4.

Measurement invariance results for informal student learning.

Characteristic/model	χ²	df	CFI	RMSEA	SRMR	Comparison	Δdf	Δχ²
Age
InvMod 1: Configural invariance	855.683	480	0.960	0.046	0.052
InvMod 2: Metric invariance	878.494	496	0.959	0.045	0.053	InvMod 2 versus InvMod 1	16	22.811^†
InvMod 3: Scalar invariance	893.760	512	0.960	0.045	0.053	InvMod 3 versus InvMod 2	16	15.265^†
Gender
InvMod 1: Configural invariance	837.365	480	0.962	0.045	0.052
InvMod 2: Metric invariance	854.661	496	0.961	0.044	0.053	InvMod 2 versus InvMod 1	16	17.297^†
InvMod 3: Scalar invariance	919.590	512	0.956	0.046	0.055	InvMod 3 versus InvMod 2	16	64.929^***
Academic performance
InvMod 1: Configural invariance	825.631	480	0.963	0.044	0.053
InvMod 2: Metric invariance	839.102	496	0.963	0.043	0.054	InvMod 2 versus InvMod 1	16	13.471^†
InvMod 3: Scalar invariance	873.268	512	0.961	0.043	0.055	InvMod 3 versus InvMod 2	16	34.166^**

InvMod: Invariance Model. N = 3134. CFI: Comparative Fit Index; RMSEA: Root Mean Square Error of Approximation; SRMR: Standardized Root Mean Square; EI: Extrinsic intention to learn.

†

non-significant, ^**p < 0.01, ^***p < 0.001.

Study 3: Examining validity in a U.S. sample

The previous studies were conducted with German students. To validate the scale in an international context, we ran study 3 with U.S. students. As in Study 2, the aim was to replicate the 1-4-2 model structure, to investigate the ISL scale’s convergent and discriminant validity (H3, H4), and to examine concurrent validity regarding self-efficacy (H5) and academic performance (H6).

Participants and instruments

We recruited a sample of 319 US students via the online platform Prolific. Students were paid $0.4 for anonymous participation. Nine individuals had to be excluded due to failing the attention check. Of the 310 students in the final data set, 79.7% were female. The most frequently represented fields of study were Social Sciences/Economics (31.9%) and Medical and Health Sciences (20.3%). The average age was 22.4 years (SD = 4.6). The participants completed the same questionnaire as in Study 1 and Study 2 (for item and construct correlations see Table D in ESM), which we had translated from German to English using a translation/backtranslation procedure (Brislin, 1986). The only difference was the academic performance rating (finely tiered with ± in 9 levels) adapted to the U.S. grading system.

Procedure and results

A CFA on ISL with the 1-4-2 structure yielded a good model fit in the U.S. sample, too, χ²(240) = 396.823, p < 0.001; CFI = 0.954; SRMR = 0.055; RMSEA = 0.046, 90% CI = [0.038, 0.054]. This confirms our model’s structural assumption for ISL in an English-speaking context as well.

Convergent and discriminant validity

For examining convergent and discriminant validity, we calculated the values for AVE and SV using SEM as in study 2. The AVE values of all eight ISL components are ⩾ 0.50—ranging between 0.50 and 0.76 (see Table E in ESM). We thus can assume convergent validity (Hair et al., 2010). The SV values between two components each range between 0.01 and 0.36. Therefore, the two AVE values in each case are larger than the corresponding SV value, indicating evidence for discriminant validity (Farrell, 2010). The findings support H3 and H4.

Concurrent validity

For concurrent validity, we again examined the relationship between ISL, study-related self-efficacy, and academic performance. As in study 2, we used SEM to model ISL with the 1-4-2 structure as predictor of self-efficacy and academic performance. This model shows an acceptable fit, χ²(420) = 706.131, p < 0.001; CFI = 0.929; SRMR = 0.062; RMSEA = 0.047, 90% CI = [0.041, 0.053]. ISL and self-efficacy (β = 0.50) and ISL and academic performance (β = 0.20) correlate positively, supporting H5 and H6.

General discussion

Results summary

Despite the consensus on the importance of informal learning for active learning in higher education, no valid measure of ISL has yet existed. In our study, we developed a 24-item ISL scale. The eight subscales each have three items. We based the development on the conceptual Octagon Model of IWL (Decius et al., 2019). The ISL scale has acceptable reliability in terms of internal consistency. Using three independent samples, we tested the structure and the validity of the measure based on five hypotheses.

The ISL structure concurs with the IWL structure in two factors, but also deviates in two other factors. Hypothesis 1 therefore received only partial support. As described in the Structural Division of ISL Factors section, we explain this conceptually: Informal learning differs between workers and students. While the ISL factors reflection and intent to learn comprise the same two components as in IWL, we had to split the factors experience/action and feedback for ISL (cf. Figure 1). Students learn through trial and error and from fellow students but might integrate experiences learned from a model less directly into their actions compared to workers. This conceptual assumption, however, requires further empirical testing.

Hypothesis 2 also received only partial support. The 1-4-2 structure is superior to the 1-0-8 structure, but not to the other two alternative structures. Nevertheless, the Octagon Model includes a core factor of informal learning, and because the alternative models are not superior to the 1-4-2 structure either, we retain the theoretically justified 1-4-2 structure. We recommend conducting further studies to validate this assumption. The eight ISL components show convergent (support for hypothesis 3) and discriminant (support for hypothesis 4) validity. The ISL scale also demonstrates concurrent validity in terms of study-related self-efficacy and academic performance—hypotheses 5 and 6 thus received support.

We could further report configural and metric invariance for age, gender, and academic achievement, and for age additionally scalar invariance. Thus, for the most part, the scale measures ISL reliably regardless of variance in these characteristics. Hypothesis 7 was supported, hypotheses 8 and 9 at least partially.

Limitations and future research directions

Our study faces some limitations. Study 1 contains data of students from only one university. We only surveyed regular universities, not universities of applied sciences. Most participants in all three studies were female, which may have caused a slight bias given an approximately expected equal gender distribution in the student population. We validated the questionnaire in German and in English; however, future research could address validations in other languages.

We investigated study-related self-efficacy and academic performance regarding the ISL scale’s concurrent validity. However, we cannot draw causal conclusions because of the cross-sectional design. Since we surveyed all constructs within the same questionnaire, we cannot rule out common source/method biases. Future longitudinal studies should consider additional constructs from the nomological network of informal learning (e.g. personality factors, or learning goal orientation; Cerasoli et al., 2018) and investigate predictive validity, for example, regarding academic performance. Further research might also monitor the observable part of ISL (e.g. feedback interactions) to continue validating the scale with more objective data. For the unobservable part of ISL (e.g. reflective cognitions), diary studies could provide a more proximate measurement.

All three studies show similar correlations between ISL and academic performance (β between 0.20 and 0.23). The relations between informal learning and self-efficacy are also comparable in the German samples (Study 1: β = 0.32; Study 2: β = 0.28), but higher in the U.S. sample (β = 0.50). This could be due to cultural differences in study systems and associated influences on students in Germany and the U.S. (e.g. see Brückner et al., 2015, for culturally induced differences on students’ economic knowledge in both countries). However, more precise conjectures are difficult to draw without knowing the direction of the effect. Future studies could therefore apply longitudinal or experimental research designs to investigate whether ISL leads to self-efficacy, or whether self-efficacy leads to ISL, or whether both are the case in a self-reinforcing process. Another limitation is that we adapted the scale measuring self-efficacy for academia; however, similar changes to this scale have been made in other studies in higher education (e.g. Eberle and Hobrecht, 2021).

Practical and theoretical implications

The ISL scale has high psychometric quality and is a reliable, valid, and economical ISL measure for research as well as for universities, lecturers, and students. Scientists can use the scale for operationalizing ISL validly. This allows them to empirically test assumptions from previous research, including whether ISL leads to increases in student employability and empowerment, enhanced self-efficacy and student engagement, and to positive long-term effects for the broader community through active citizenship and human capital generation (Peeters et al., 2014; Soyyılmaz et al., 2017).

Universities may design strategies to promote ISL based on ISL assessments. Even though the ISL definition states that ISL occurs self-directedly outside of formally prescribed, curriculum-based learning objectives, university officials might create additional opportunities that increase the likelihood of ISL, for example, open spaces for feedback, reflection, and self-experimentation in voluntary learning settings. Lecturers may optimize their teaching—if they have previously identified the preferred ISL components of their students—to succeed in the stronger integration of formal and informal learning processes that is often called for (cf. Peeters et al., 2014). Students interested in improving their active learning skills can use self-testing to explore their informal learning behaviors and deliberately seek out situations that are conducive to their preferred learning components. However, more research is needed on the beneficial conditions of ISL—also on whether conditions differ for behavioral, cognitive, and intentional components of ISL.

Our study also makes an important theoretical contribution by showing that informal learning differs structurally between workers and students. Research findings thus cannot be simply extrapolated from one to the other target group. In our study, we have therefore presented not only the ISL measure, but also an adapted Octagon Model for ISL on which future research can build. Efforts to recognize informally acquired skills and knowledge in higher education also benefit from our ISL conceptualization (cf. Stenlund, 2010). Future research should also look at informal learning at the group level, as previous literature has highlighted positive effects of team-based learning for knowledge acquisition in higher education (Swanson et al., 2019).

Another important finding of our study is that the hypothesized ISL structure could be detected in both Germany and USA. To our knowledge, our study is the first to use samples from both countries in a quantitative evaluation of ISL and hopefully sets the stage for further studies in this under-researched area. The ability to operationalize ISL will allow future research to empirically examine antecedents and outcomes of ISL quantitatively and expand the body of research that has been primarily qualitative to date.

Supplemental Material

sj-docx-1-alh-10.1177_14697874221087427 – Supplemental material for The casual within the formal: A model and measure of informal learning in higher education

Supplemental material, sj-docx-1-alh-10.1177_14697874221087427 for The casual within the formal: A model and measure of informal learning in higher education by Julian Decius, Janika Dannowsky and Niclas Schaper in Active Learning in Higher Education

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Julian Decius

Supplemental material

Supplemental material for this article is available online.

Author biographies

Julian Decius is post-doc researcher (with tenure track) at University of Bremen, Germany, where he leads the work group of Organizational Psychology. His research interests focus on informal learning in the workplace and in higher education environments.

Janika Dannowsky is research assistant for language education and early childhood education at the University of Education Salzburg Stefan Zweig, Austria. Her research focuses on media use by parents and children, interaction in parent-child relationships, and language development at the early childhood level.

Niclas Schaper is full professor and holder of the Chair of Work and Organizational Psychology, Paderborn University, Germany. His research focuses on questions of competence acquisition and measurement.

References

Abele

Stief

Andrä

(2000) Zur ökonomischen Erfassung beruflicher Selbstwirksamkeitserwartungen – Neukonstruktion einer BSW-Skala [Towards an economic assessment of occupational self-efficacy expectations – constructing a new BSW scale]. Zeitschrift für Arbeits- und Organisationspsychologie 44(3): 145–51.

Bandura

(1997) Self-Efficacy: The Exercise of Control. New York: W.H. Freeman and Company.

Barth

Godemann

Rieckmann

, et al. (2007) Developing key competencies for sustainable development in higher education. International Journal of Sustainability in Higher Education 8(4): 416–30.

Brislin

(1986) The wording and translation of research instruments. In: Lonner

Berry

(eds) Field Methods in Cross-Cultural Research. Volume 8, Cross-Cultural Research and Methodology Series. Beverly Hills, CA: SAGE, pp.137–64.

Brückner

Förster

Zlatkin-Troitschanskaia

, et al. (2015) Effects of prior economic education, native language, and gender on economic knowledge of first-year students in higher education. A comparative study between Germany and the USA. Studies in Higher Education 40(3): 437–53.

Cerasoli

Alliger

Donsbach

, et al. (2018) Antecedents and outcomes of informal learning behaviors: A meta-analysis. Journal of Business and Psychology 33(2): 203–30.

Choi

Jacobs

(2011) Influences of formal learning, personal learning orientation, and supportive learning environment on informal learning. Human Resource Development Quarterly 22(3): 239–57.

Clark

(2016) Collective action competence: An asset to campus sustainability. International Journal of Sustainability in Higher Education 17(4): 559–78.

Conway

Huffcutt

(2003) A review and evaluation of exploratory factor analysis practices in organizational research. Organizational Research Methods 6(2): 147–68.

10.

Costello

Osborne

(2005) Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment Research & Evaluation, 10(7), 1–9.

11.

Decius

Knappstein

Schaper

, et al. (2021) Investigating the multidimensionality of informal learning: Validation of a short measure for white-collar workers. Human Resource Development Quarterly. Epub ahead of print 9 October 2021. DOI: 10.1002/hrdq.21461

12.

Decius

Schaper

Seifert

(2019) Informal workplace learning: Development and validation of a measure. Human Resource Development Quarterly 30(4): 495–535.

13.

DeVellis

(2003) Scale Development. Theory and Applications, 2nd edn. Thousand Oaks, CA: SAGE.

14.

Eberle

Hobrecht

(2021) The lonely struggle with autonomy: A case study of first-year university students’ experiences during emergency online teaching. Computers in Human Behavior 121: 106804.

15.

Ebner

Lienhardt

Rohs

, et al. (2010) Microblogs in Higher Education – A chance to facilitate informal and process-oriented learning? Computers & Education 55(1): 92–100.

16.

Eraut

(2000) Non-formal learning and tacit knowledge in professional work. British Journal of Educational Psychology 70(1): 113–36.

17.

Farrell

(2010) Insufficient discriminant validity: A comment on Bove, Pervan, Beatty, and Shiu (2009). Journal of Business Research 63(3): 324–7.

18.

Flaherty

(1975) The thinking aloud technique and problem solving ability. Educational Research eJournal 68(6): 223–5.

19.

Fornell

Larcker

(1981) Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research 18(1): 39–50.

20.

Gallacher

Feutrie

(2003) Recognising and accrediting informal and non-formal learning in higher education: An analysis of the issues emerging from a study of France and Scotland. European Journal of Education 38(1): 71–83.

21.

Gramatakos

Lavau

(2019) Informal learning for sustainability in higher education institutions. International Journal of Sustainability in Higher Education 20(2): 378–92.

22.

Grosemans

Coertjens

Kyndt

(2018) Work-related learning in the transition from higher education to work: The role of the development of self-efficacy and achievement goals. British Journal of Educational Psychology 90(1): 19–42.

23.

Hair

Black

Babin

, et al. (2010) Multivariate Data Analysis: A Global Perspective, 3rd edn. Upper Saddle River, NJ: Pearson.

24.

Zhu

(2017) Digital informal learning among Chinese university students: The effects of digital competence and personal factors. International Journal of Educational Technology in Higher Education 14(1): 44–63.

25.

Hopkinson

Hughes

Layer

(2008) Sustainable graduates: Linking formal, informal and campus curricula to embed education for sustainable development in the student learning experience. Environmental Education Research 14(4): 435–54.

26.

Hutcheson

Sofroniou

(1999) The Multivariate Social Scientist: Introductory Statistics Using Generalized Linear Models. Thousand Oaks, CA: SAGE.

27.

Jamieson

(2009) The serious matter of informal learning. Planning for Higher Education 37(2): 18–25.

28.

Kassens-Noor

(2012) Twitter as a teaching practice to enhance active and informal learning in higher education: The case of sustainable tweets. Active Learning in Higher Education, 13(1), 9–21.

29.

Khiat

(2022) Using automated time management enablers to improve self-regulated learning. Active Learning in Higher Education 23: 3–15.

30.

Kline

(2016) Principles and Practice of Structural Equation Modeling, 4th edn. New York, NY; Guilford.

31.

Kolenikov

Bollen

(2012) Testing negative error variances: Is a Heywood case a symptom of misspecification? Sociological Methods & Research 41(1): 124–67.

32.

Kyndt

Vermeire

Cabus

(2016) Informal workplace learning among nurses: Organisational learning conditions and personal characteristics that predict learning outcomes. Journal of Workplace Learning 28(7): 435–50.

33.

Lave

Wenger

(1991) Situated Learning: Legitimate Peripheral Participation. New York and Cambridge: Cambridge University Press.

34.

Lienert

Raatz

(1998) Testaufbau und Testanalyse (6. Aufl.) [Test Construction and Test Analysis (6th edn)]. Weinheim, Germany: Beltz.

35.

MacKenzie

Podsakoff

(2011) Construct measurement and validation procedures in MIS and behavioral research: Integrating new and existing techniques. MIS Quarterly 35(2): 293–334.

36.

Marques

Viegas

Alves

, et al. (2013) Managing informal learning in higher education contexts: The learners’ perspective. In de Salvador Ferreira

Pester

(Eds.), Proceedings of the international conference on interactive computer aided blended learning, ICBL2013, Florianópolis, Brazil, 6–8 November 2013, pp. 233–238. Vienna, Austria: International Association of Online Engineering (IAOE). Available online: http://hdl.handle.net/10366/122587

37.

Martindale

Dowdy

(2010) Personal learning environments. In: Veletsianos

(ed.) Emerging Technologies in Distance Education. Edmonton: Athabasca University Press, pp.177–93.

38.

McKenzie

Schweitzer

(2001) Who succeeds at university? Factors predicting academic performance in first year Australian university students. Higher Education Research & Development 20(1): 21–33.

39.

Mertens

Claes

Becker

(2018) Man kann nicht nicht lernen. Informelles Lernen in der Ausbildung von Lehrkräften [One cannot not learn. Informal learning in teacher education]. In: Heuchemer

Siller

van Treeck

(eds) Hochschuldidaktik forscht zu Vielfalt und Offenheit. Cologne: Open Science, pp.105–19.

40.

Minium

(1992) Statistical Reasoning in Psychology and Education, 3rd edn. New York: Wiley.

41.

Muldoon

(2009) Recognizing the enhancement of graduate attributes and employability through part-time work while at university. Active Learning in Higher Education 10(3): 237–52.

42.

Nicol

Owens

Le Coze

, et al. (2018) Comparison of high-technology active learning and low-technology active learning classrooms. Active Learning in Higher Education, 19(3), 253–265.

43.

Noe

Tews

Marand

(2013) Individual differences and informal learning in the workplace. Journal of Vocational Behavior 83(3): 327–35.

44.

Panadero

(2017) A review of self-regulated learning: Six models and four directions for research. Frontiers in Psychology 8: 422.

45.

Park

Woo

, et al. (2021) The role and related variables of workplace learning in quantitative research. Industrial and Commercial Training 53(1): 29–59.

46.

Peeters

De Backer

Buffel

, et al. (2014) Adult learners’ informal learning experiences in formal education setting. Journal of Adult Development 21(3): 181–92.

47.

Pintrich

(2000) The role of goal orientation in self-regulated learning. In: Boekaerts

Pintrich

Zeidner

(eds) Handbook of Self-Regulated Learning. San Diego, CA: Academic Press, pp.451–502.

48.

Russell

Baik

Ryan

, et al. (2020) Fostering self-regulated learning in higher education: Making self-regulation visible. Active Learning in Higher Education, Epub ahead of print 23 December 2020. DOI: 10.1177/1469787420982378.

49.

Sass

(2011) Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment 29(4): 347–363.

50.

Segers

Messmann

Dochy

(2018) Emergence, theoretical foundation, and conceptualisation of informal learning at work. In: Messmann

Segers

Dochy

(eds) Informal Learning at Work: Triggers, Antecedents, and Consequences. London: Routledge, pp.1–11.

51.

Soyyılmaz

Griffin

Martín

, et al. (2017) Formal and informal learning and first-year psychology students’ development of scientific thinking: A Two-Wave Panel Study. Frontiers in Psychology 8: 133.

52.

Stenlund

(2010) Assessment of prior learning in higher education: A review from a validity perspective. Assessment & Evaluation in Higher Education 35(7): 783–97.

53.

Swanson

McCulley

Osman

, et al. (2019) The effect of team-based learning on content knowledge: A meta-analysis. Active Learning in Higher Education 20(1): 39–50.

54.

Tannenbaum

Beard

McNall

, et al. (2010) Informal learning and development in organizations. In: Kozlowski

SWJ

Salas

(eds) Learning, Training, and Development in Organizations. New York, NY; Routledge, pp.303–32.

55.

Toffoli

Sockett

(2015) University teachers’ perceptions of online informal learning of English (OILE). Computer Assisted Language Learning 28(1): 7–21.

56.

van Daal

Donche

De Maeyer

(2014) The impact of personality, goal orientation and self-efficacy on participation of high school teachers in learning activities in the workplace. Vocations and Learning 7(1): 21–40.

57.

Vandenberg

Lance

(2000) A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods 3(1): 4–70.

58.

van Dinther

Dochy

Segers

(2011) Factors affecting students’ self-efficacy in higher education. Review of Educational Research 6(2): 95–108.

59.

Winter

Cotton

(2012) Making the hidden curriculum visible: Sustainability literacy in higher education. Environmental Education Research 18(6): 783–96.

60.

Yang

(2001) Predicting academic performance in management education: An empirical investigation of MBA success. Journal of Education for Business 77(1): 15–20.

61.

Zimmerman

(2000) Self-efficacy: An essential motive to learn. Contemporary Educational Psychology 25(1): 82–91.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB