Sage Journals: Discover world-class research

Abstract

This paper presents a reliable tool for measuring transformative learning in undergraduate social work education, the Social Work Transformation Survey (SWTS). The SWTS was developed from a qualitative theoretical model and translated into quantitative scales. The study collected data from 248 undergraduate students from eight countries who participated in a transnational project using creative journaling to facilitate transformative learning. Structural equation modelling was used to validate the internal structure of the SWTS. We then confirmed the measures’ reliability, and subsequently the effectiveness of creative journaling practices as a pedagogy for facilitating transformative learning in social work students. This paper highlights the potential of combining qualitative and quantitative research approaches to develop educational evaluation tools for higher education settings and presents one specific measure for transformative learning.

Keywords

transformative learning social work creative journaling quantitative educational research

Introduction

The COVID-19 pandemic created emotional and logistical challenges in the realm of learning and required social work education across the globe to creatively adapt its pedagogical approaches. Social work’s inherently relational nature necessitates innovative methods to foster coping, management, and adaptation skills in this transformative era. Consequently, there is a clear call for transformative pedagogies to be employed. Critical pedagogies can facilitate transformative learning, self-awareness, and reflexivity, and assume a vital role in equipping emerging social work practitioners with the resilience required to confront the challenges of our times while aiding others in navigating uncertainty (Morley et al., 2020).

Social work education has an emphasis on the social, emotional, and relational aspects of teaching and learning that require educators to explore creative, innovative, and transformative methods to facilitate student growth and development. As (Afrouz, 2021) aptly states, we must incorporate the concepts of risk and uncertainty into the social work curriculum and establish a dialogic and reflective process. This prepares our students to engage in reflexive processes, a pivotal strategy in confronting uncertainties.

One valuable approach to nurturing self-awareness, reflexivity, and the ability to navigate dilemmas is through critically reflective practice. This approach equips emerging practitioners with the resilience necessary to face challenges and support others in navigating uncertainty.

This project brought together an international team of educator-researchers, each with backgrounds in social work and social policy, hailing from eight different countries: Canada, India, Israel, Jersey Island, Spain, Sweden, the United Kingdom, and the United States of America. The team wanted to investigate whether the use of reflective journaling practices (Capous-Desyllas & Bromfield, 2020; Collins, 2021; Hubbs & Brand, 2010; Stevens & Cooper, 2009; Sudriman et al., 2021; Vinjamuri et al., 2017), could facilitate transformative learning and professional socialization. To accomplish this, the larger project employed a mixed-method design, specifically the Sequential Explanatory approach (Creswell & Plano Clark, 2017). This design encompassed two phases: the first phase involved an original 60-question learning survey, while the second phase incorporated focus group interviews. The mixed methods approach followed an “iterative, cyclical approach,” as articulated by (Teddlie & Tashakkori, 2010), wherein both deductive and inductive processes informed the research questions and analysis. This approach enabled in-depth data integration by establishing connections and linkages within the findings.

The focus of this paper is the learning survey, named the Social Work Transformation Survey (SWTS), which draws from Mezirow’s theory of perspective transformation and is grounded in a model capturing the transformation process of social work students (Damianakis et al., 2019). The primary objective of this paper is to assess the validity of the SWTS through a series of statistical analyses.

Transformative Learning in Social Work

Transformative learning involves profound shifts in students’ perspectives and their construction of meaning, and it is a crucial aspect of social work education. Critiques of the concept of transformative learning argue that it describes nothing more than positive learning experiences, driven by the argument that it is not possible to measure and that it only resides in the realm of theory (Newman, 2012). For others, transformative learning is multidimensional and interconnected, involving profound changes in the self across dimensions beyond the cognitive (Pang, et al., 2023). Several studies have demonstrated that transformative learning can occur among social work students, leading to significant personal and professional growth (Damianakis et al., 2019). These studies have identified key teaching strategies that facilitate transformative learning, including critical reflection, exploration of assumptions, tailored pedagogy to individual learning needs, peer interactions, faculty relationships, and mentoring (Damianakis et al., 2019; McCusker, 2013).

The relevance of transformative learning theory extends to addressing contemporary challenges in social work education, such as environmental issues as it promotes deep understanding and change (Jones et al., 2015). In addition, social learning in the context of transdisciplinary research, closely related to transformative learning, is enhanced by factors like clarifying values, collaboratively constructing research questions with stakeholders, and achieving a balance in power dynamics (Herrero et al., 2018). However, it is essential to acknowledge the inherent complexities in measuring transformative learning and establishing causality, as highlighted by McCusker (2013).

Incorporating transformative pedagogical methods is crucial for nurturing students’ critical reflection skills and meaning-making capabilities. This approach is increasingly adopted to shape critical thinking and reflective learning experiences for adult learners in higher education (Zhang & Dempsey, 2019). Within social work education, transformational learning is evident in students’ personal and professional development, growth, and socialization experiences (Larrison & Korr, 2013). Transformational learning theory, as articulated by Mezirow (2012), sheds light on the transformational process students experience when their newly acquired knowledge challenges their existing beliefs. This theory comprises four key components: (1) encountering a disorienting dilemma; (2) engaging in critical reflection on assumptions, beliefs, and values; (3) exploring new ideas and roles; and, (4) building confidence and skills in newly identified roles and relationships while integrating new information into their actions (Mezirow, 2012).

Mezirow’s framework (1981) suggests that students grapple with dilemmas when new knowledge challenges their existing ways of thinking. This tension prompts students to reflect on their ingrained assumptions, facilitating the integration of prior and new knowledge into a new framework of meaning (Mezirow, 1981). While numerous scholars have explored transformative education in social work (e.g., Bay & Macfarlane, 2011; Capous-Desyllas & Sinclair, 2014; Jones, 2009), Damianakis et al. (2019) provide valuable empirical insights into Mezirow’s transformative learning theory within graduate social work education, both in the classroom and in field education, within a Canadian context (Archer-Kuhn et al., 2020). Damianakis et al. (2019) delineated the process of student transformation, while their work in 2019(b) identified the critical pedagogical factors essential for supporting student transformation in graduate social work education.

Critical Reflection Pedagogies as Triggering Transformative Learning

Critical reflection serves as a cornerstone in the development of social work students, providing them with the essential ability to scrutinize their own assumptions and consider alternative perspectives. Numerous studies and research papers have explored the teaching of critical reflection to social work students, collectively aiming to cultivate their critical thinking skills and foster reflective practice. Askeland and Fook (2009) emphasize that critical reflection plays a vital role in helping social workers understand the theoretical foundations and inherent value of their profession. This foundational understanding serves as a cornerstone for effective social work practice.

Theobald (2017) discovered that instructing Australian social work students in a model of critical reflection enhanced their capacity for critical thinking during field placements. Critical thinking then informed their actions and decision-making in real-world scenarios. Moreover, Karvinen-Niinikoski (2009) suggests that critical reflection equips social workers to adapt to the ever-evolving landscape of their field. It empowers them to reorient their practice as needed, though it also underscores the pressing need to address political issues and societal challenges. Lay and McGuire (2010) present a structured framework for teaching critical reflection within graduate social work education in the U.S. This framework guides students on a journey from basic reflection to a higher level of reflexivity deeply rooted in critical social theory, aligning with the overarching mission of the social work profession. Oterholm (2009), while reflecting an earlier time, discusses the utilization of online tools to teach critical reflection to geographically dispersed Norwegian social work students during their field placements. These digital resources encourage students to explore power relations and embrace alternative perspectives, irrespective of their physical location.

In Australia, Morley and O’Bree (2021) underscore the indispensable role of critical reflection in enabling social workers to uphold their core values, especially within the constraints of neoliberal organizations that may inadvertently undermine these values. In a broader context, Bay and Macfarlane (2011) introduce social work students in Australia to critical reflection through the lens of poststructural theories like Foucault. This approach seeks to disrupt ingrained assumptions and nurture students into becoming autonomous thinkers capable of critically analyzing society and social work practices.

Lastly, Das and Anand (2014) offer strategies for teaching critical reflection to social work students in both the UK and India within the framework of an international exchange program. In these cross-cultural contexts, critical reflection aids students in developing an understanding of international social work, global citizenship, and cultural competency, all within a critical framework.

Measuring Transformative Learning

Snyder, L. G. and Snyder, M. J. (2008) suggest that students’ self-assessment of independent learning can be nuanced, leading to a call for diverse data approaches to effectively detect transformative learning. In the realm of higher education, efforts to explore transformative learning experiences have predominantly leaned on qualitative methodologies for capturing and analyzing these experiences, as observed by Malka (2022). However, there exists a pressing need for further development and comprehension of transformative learning within higher education, with a specific focus on Social Work.

In response to this challenge, introducing quantitative methods to complement existing qualitative research on social work students’ transformative learning experiences (or not) in social work becomes an interesting proposition. Stuckey and Cranton (2013) suggest that scales offer a means to enhance the validity and reliability of evaluating learning experiences and quantifying the processes underpinning transformation. Within higher education, educators can utilize these scales to gain insights into the evolving processes and enhance their grasp of outcomes. These scales introduce a valuable instrument to the Transformative Learning discourse, addressing the complexity of the concept and the demand for accurate, dependable, and user-friendly measures to assess students’ transformative learning experiences.

While crafting a single comprehensive scale capable of encapsulating every facet of transformative learning proves challenging, there are notable surveys that have withstood rigorous statistical validation. The Critical Reflection Questionnaire by Kember et al. (2000) assesses the extent to which students engage in reflective practices. The Learning Activity Survey developed by King (2009) and the Transformative Learning Survey created by Stuckey, Taylor, and Cranton (2013/22) have all contributed to advancing knowledge and understanding of transformative learning. The challenge for all surveys is comprehensive testing: King and Stuckey et al.’s instruments have not seen widespread adoption in replication studies, except for those conducted by their original authors. It can be argued that an instrument’s validity can be gauged by its continued use in new research endeavours. Furthermore, concerns can be raised regarding the reliability and validity of the surveys due to the disparity between the number of items on the questionnaire and the number of responses (Romano, 2018). Cox (2021) posits that reflection plays a pivotal role in the process of transformation. In addition to highlighting the significance of reflection, Cox also delves into the evaluation of reflection through diverse approaches. Notably, Cox introduces the TROPOS subscales as a means to assess reflection. The research by Cox reveals a positive correlation between the TROPOS subscales and transformative learning (TL). Nevertheless, it underscores the necessity for further research and replicability studies to strengthen the credibility of these findings. Walker (2018), guided by the framework of Mezirow, took a different approach and developed the Transformative Learning Environments Survey (TLES). This survey has demonstrated its robustness as an assessment tool, effectively gauging students’ perceptions of their personal transformation and the degree to which the learning environment facilitates such transformative experiences. Furthermore, transformative learning surveys have not been confined solely to educational contexts. They have been adapted and validated for use in non-educational settings, as exemplified by the development of a quantitative measure for parent empowerment through transformative learning by To et al. (2022).

While these surveys represent valuable tools, their application in new environments requires adaptation and validation. The incorporation of validated surveys and quantitative methods, which move beyond the retrospective qualitative approach, offers promising avenues for advancing our comprehension of this intricate concept (Romano, 2018).

Rationale for This Study

The rationale for this study is twofold: first, existing research specifically focused on transformative learning in social work education needs to be expanded; and second, there is a need to bridge the gap between qualitative and quantitative approaches in social work research. Our study aims to fill this gap by employing a two-step approach to create a survey grounded in theory and experience. This approach allows us to cross over from induction (qualitative) to deduction (quantitative) and provide a higher level of abstraction for analysing the underlying empirical structure.

We recognize that social work education highly values critical thinking and reflective learning. Our hope with this project is to create a survey that captures the meaningful educational experiences of social work students and contribute to the ongoing discussion surrounding transformative learning within social work education utilizing transformative learning theory as a framework. This theory can guide the creation of meaningful educational experiences and inform the measurement of meaningful educational outcomes (Zhang & Dempsey, 2019).

This paper reflects the testing of our survey, in the creation of a measure, SWTS. Our survey was created based on the findings of a 4-year qualitative study with 40 social work graduate students (Damianakis et al., 2019). In the Damianakis study, the authors identified three dominant themes that emerged from the students’ interviews: (1) transformation is a process of feeling displaced, re-evaluating, and surfacing; (2) transformative outcomes are multidimensional and embody personal and professional learning; and, (3) transformative learning is a holistic experience. The survey used in this paper was generated from Damianakis et al. (2019) findings and displays an innovative way to link qualitative and quantitative research.

Methodology and Methods

Sample

Purposive sampling was carried out in two waves. Table 1 shows the frequency distribution according to the country of origin. The population of interest are selected students from the Faculties and Schools of Social Work participating in the project who have developed a common reflective journaling assignment during one semester.

Table 1.

Distribution of Frequencies and Percentages of Participants According to Country of Origin.

Country	Course Name	N	%
Canada	Professional use of self	16	6.5
India	Gender based violence, culture, and human rights	21	8.5
Israel	Mindfulness and social work	16	6.5
Israel	The practice of the helping relationship	16	6.5
Spain	Foundations of social work with groups	104	41.9
	Social work with families
	Practicum
	Mediation as an alternative dispute resolution system
Sweden	Social relationships in local context	7	2.8
United Kingdom, including Jersey	Context of social work	11	4.4
United States of America	Introduction to social work	73	29.4
Total		248	100

A sample of 127 cases was obtained in the first wave and 121 in the second wave. The undergraduate students were between 20 and 35 years old, and 94.5% of the sample is under 30. There is a higher presence of female participants; 86% are female (SD = .35). The feminized character of social work studies is present in all universities. The sample we have obtained is representative of the student population within the social work faculties where this research was conducted. To illustrate this, consider the example of the Social Work Faculty at UCM (Spain) in 2016, where 84% of undergraduate students were women (information sourced from the Equality Unit of the Complutense University of Madrid, https://www.ucm.es/cifras-en-la-ucm). Currently, 86.8% of enrolled students are women (Data provided by the UCM Academic Management and Coordination Service). To rigorously assess potential differences between groups, the calculation of factorial invariance is the appropriate method. Factorial invariance is defined as confirming that the measurement properties of the instruments or their items remain consistent across the groups under evaluation, regardless of the construct being measured by the instrument. To achieve this, it is necessary to estimate the model separately for two groups (one composed of men and the other of women), and it is essential that these groups have comparable sample sizes. However, this requirement becomes next to impossible in populations where the majority of students are women.

The differences in the number of participants in each country are due to the voluntary nature of the activity, the ratio of students per class each year, the number of teachers involved in the project in each country (e.g., in Spain there were four teachers involved, while in all other countries only one), as well as the compulsory or optional nature of the subject in which the journaling was applied (generally compulsory subjects had more participants than optional subjects).

According to the distribution by year of study, 33.4% of undergraduates are in their first year, 41.7% in their second year, 16.7% in their third year, and 8.2% in their fourth year.

Ethics

Each educator-researcher attained ethical approval at their respective institutions via the formal processes and procedures required at each location. It was crucial for the researchers that information regarding the project was transparent and accessible to the students. To explain the research in clear and concise terms for the students, the research team, with the support of social work student research assistants, created a website that served as a survey portal and opened to an external site (access the website here: https://bok.uni.mau.se/). When the students clicked on the survey link, they were presented with an information letter outlining the research aims and seeking their participation. The survey and letter were available in four languages (Spanish, Hebrew, Swedish, and English). Survey answers were password-protected.

Design

A pre-experimental cross-sectional study (Campbell & Stanley, 2005) was conducted without a control group, given the characteristics of the measurement instrument that requires the participants to carry out the transformative learning process through a reflective process.

Students who had used the journaling activity during the semester responded to the questionnaire developed ad hoc from the theoretical model of Damianakis et al. (2019). Two 5-point Likert-type scales were designed (1: strongly agree, 5: strongly disagree)—the TT scale on ‘transformative learning outcomes ‘and the TL scale on ‘transformative learning’.

Recruitment

Students were recruited after the completion of their course. Potential participants were provided information about the study (Letter of Information) including access to the questionnaire via a link provided by the lecturer at the end of the fall and spring semesters. Students who chose to participate in the study on their own time completed informed consent when they clicked on the link. Once consent was provided, the questionnaire opened for participants to complete on Survey Monkey.

Procedure

The data collection was carried out through the application of an online questionnaire completed in SurveyMonkey by a social work research practicum student. The questionnaire was developed based on the theoretical model of transformative learning proposed by Damianakis et al. (2019) and considering the criteria and objectives of the research. Each student utilized the journaling activity (known as the BoK; see Wallengren-Lynch et al., 2023) throughout the semester as a normed experience in their course.

Implementing a specific pedagogical method across various sites with varied institutional norms and teaching approaches is a complex enterprise that requires a great deal of consideration, adaptation, and flexibility by all of the educator-researchers involved. This meant numerous online meetings, discussions, and planning.

Validation by expert assessment was carried out to rate the effectiveness of the TT and TL structural models (Escobar-Pérez & Cuervo-Martínez, 2008). The procedure established two mechanisms for identifying the experts: university teaching in the faculties where the project was carried out and experience in educational evaluation. Based on these criteria, eleven experts from different universities were selected to participate in the whole process. All of them accepted to participate and none of them was a member of the research team. The evaluation was carried out using the individual evaluation method, given the high percentage of agreement in the first consultation. The instruments were sent by email: the scales and their statistics as they appear in this document, the BoK instructions for the teacher and the students, and the instructions for the experts requesting a minimum score that the student must obtain in each item to achieve the objective, competence, or skill that the item describes. Experts and learners used the same scale in the application of the questionnaire (5-level Likert-type).

Variables and Measuring Instrument

The percentage of missing cases in the variables was very low, (<2.5%), so an expectation-maximization algorithm was chosen for imputation. The original English version of the questionnaire was translated into Spanish, Hebrew, and Swedish. The version in these languages was checked to ensure that it was equivalent to the English original with a new translation or back translation to ensure semantic and conceptual equivalence in all versions (Carvajal, et al., 2011). The socio-demographic variables selected for the sample description were gender, age, language, year of study, and country of residence. According to the theoretical model proposed by Damianakis et al. (2019), measurement models are constructed for transformative learning as a holistic process of change (TL) and transformative outcomes: embodied and multidimensional (TT). TT scale is composed of two subscales Deep Personal Transformation (TD), consisting of four items and Becoming a Professional (BP), composed of 6 items. The TL scale included three items for the education subscale (PE) and ten items for the features of transformative learning (FTL) subscale (see Table 2).

Table 2.

Means and Standard Deviations of the Items Tested.

Items		ME	SD
TD1	I have increased my self-awareness and critical self-reflection	1.99	.89
TD2	I have noticed a change in knowledge, values, and skills	2.00	.80
TD3	I have developed critical thinking on social issues	1.84	.74
TD33	I have developed greater empathy for others	1.89	.88
BP1	I am developing a professional identity	1.95	.84
BP2	I am better understanding the relationship between personal and professional change	1.90	.75
BP3	I am taking more risks and developing greater self-confidence	2.35	.96
BP4	My peers provided emotional support, a capacity to engage in critical dialogue and helped me to deconstruct complex issues	2.46	1.06
BP5	I am developing greater humility while understanding the limits of knowledge when helping others	1.96	.73
BP6	I am experiencing social work as a calling	1.97	.96
PE1	The BOK helped me to better understand the intersection of my personal and professional identities	3.63	1.02
PE2	The BOK helped me to understand myself in a deeper way than before	3.56	1.11
PE3	I found the BOK to be a useful tool for my personal and professional growth	3.75	1.03
FTL1	The BOK is a good way to express and develop creativity	3.98	.92
FTL2	I found the BOK to be an effective way to consider and process my emotions	3.60	1.10
FTL22	Going back to previous entries, I discovered new information or insights about myself, the content, or class discussion	3.63	1.05
FTL3	I felt more connected to my classmates and instructor through sharing and/or talking about my BOK entries	3.11	1.09
FTL33	I enjoyed seeing other BOK entries and talking to my classmates about their BOK use	3.20	1,05
FTL4	The BOK enabled me to think through concepts and ideas in a more complex way I would have otherwise	3.74	1.07
FTL5	I felt comfortable disclosing my honest thoughts and feelings in my BOK	3.72	1.08
FTL6	I experienced “aha” moments through my use of the BOK	3.44	1.16
FTL66	My use of the BOK contributed to my understanding of the course material and/or readings	3.64	1.11
FTL666	I found the BOK to be an effective method for my self-care	3.49	1.16
FTL7	The use of reflective journaling in this course enhanced my overall learning experience	3.73	.95

The statistical package SPSS (v. 28) was used for descriptive analysis of the data and ordinal alpha coefficient and the ordinal alpha coefficient, and LISREL (v. 8.8) was used for structural equation analysis. Descriptive analyses of the data provide the sample characteristics and the scales’ items. Structural equation modelling was used to examine the internal structure’s validity and confirm the measurement model. The reliability of the dimension scores was also calculated.

Results

Using the maximum likelihood method, the polychoric covariance matrix, and following the model of Damianakis et al. (2019), a confirmatory factor analysis was performed to evaluate the factor structure of the TT and TL scales.

Factor Analysis Scale Transformative Learning

A second-order factor model corresponding to the structure established by the authors is tested (Figure 1). Given the high correlation of the factors >.95, a one-dimensional model was tested (Figure 2). The likelihood ratio test shows that the second-order model does not significantly improve the one-dimensional model (see Table 3). The one-factor model was chosen for the sake of parsimony and given the high correlation of the factors showing no discriminant validity (Henseler et al., 2015).

Figure 1.

Second-order structural model for the TL construct.

Figure 2.

One-dimensional structural model for the TL construct.

Table 3.

Chi-Square Test and General Fit Measures for Competing Models.

Scale	χ2	Df	p	RMSEA	CFI	IFI	RMR	GFI	χ2 diff	Df	p
M1	63.14	42	<.02	.045	.99	.99	.02	.99
M2	65.19	44	.02	.044	.99	.99	.02	.99	2.05	2	.05

M1 second-order model, M2 one-dimensional model.

Items FTL3, FTL6, and FTL666 were rejected because they provided poor fit values with a weight of less than <.30. The saturations for all items were statistically significant and had optimal fit indices (see Tables 3 and 4). The t-statistic is significant for all items with values >.40 (Pituch & Stevens, 2015).

Table 4.

Factor Saturations and Reliability of Indicators. Scale TL.

Items	TL	R ²
The BOK helped me to better understand the intersection of my personal and professional identities PE1	.85	.72
The BOK helped me to understand myself in a deeper way than before PE2	.90	.81
I found the BOK to be a useful tool for my personal and professional growth PE3	.93	.86
The BOK is a good way to express and develop creativity FTL1	.74	.55
I found the BOK to be an effective way to consider and process my emotions FTL2	.85	.72
Going back to previous entries, I discovered new information or insights about myself, the content, or class discussion FTL22	.81	.66
I enjoyed seeing other BOK entries and talking to my classmates about their BOK use FTL33	.51	.26
The BOK enabled me to think through concepts and ideas in a more complex way I would have otherwise FTL4	.87	.75
I felt comfortable disclosing my honest thoughts and feelings in my BOK FTL5	.66	.44
My use of the BOK contributed to my understanding of the course material and/or readings FTL66	.81	.65
The use of reflective journaling in this course enhanced my overall learning experience FTL7	.89	.79

Factor Analysis Scale Transformative Learning

A second-order factor model corresponding to the structure established by the authors was tested (see Figure 3). The saturations for all items were statistically significant and had optimal fit indices (see Table 5). The t-statistic is significant for all items with values >.30.

Figure 3.

Second-order structural model for the TT construct.

Table 5.

Goodness-of-Fit Indices TT Scale.

Scale	χ2	Df	p	RMSEA	CFI	IFI	RMR	GFI
TT	75.86	33	<.00	.07	.98	.98	.03	.99

Reliability of TT and TL

The values show good reliability on all scales and in all cases (Espinoza & Novoa-Muñoz, 2018).

As Table 6 shows the ordinal alpha reaches acceptable levels for all scales, showing a degree of validity defined as the degree to which a test measures what is designed to measure, in this instance, transformative learning in undergraduate social work students.

Table 6.

Internal Consistency Coefficients of the TT and TL Scales and Factors.

Scale	Ordinal Alpha
TL	.95
TT	.85
TD	.83
BP	.84

The ordinal alpha value is based on the polychoric inter-item correlation matrix (more suitable for ordinal data) rather than the Pearson’s covariance (correlation) matrix used for the calculation of Cronbach’s alpha (more suitable for continuous data). Thus, ordinal alpha is more appropriate for estimating internal consistency when the data are ordinal (Peters, 2014). In the framework of factor analysis, it can be easily calculated using the factor loadings obtained from the tetrachoric/polychoric correlation matrix (Dominguez-Lara, 2018; Zumbo et al., 2007).

Eliciting Experts and Evaluation of the Tool in the Sample

The percentage of similarity in the experts’ scores demonstrated a high degree of agreement, reaching 91% for a score of 4 (on a 5-level Likert-type scale). This score represents the minimum threshold that a student must achieve on each item to demonstrate competence or mastery of the skill described by that particular item. This criterion is applied to evaluate the students who participated in this study. Table 7 displays the agreement percentages for student scores on each item, specifically focussing on scores equal to or greater than 4. Around 80% of the students reported a level of agreement greater than or equal to 4 for most items. These data show the high usefulness of the BoK as experienced by the students. The percentage exceptions are found for items FTL33, FTL4, and BP4 (see Table 7). On average, these scores are lower than the score of 4. On a closer examination, we could see that some student groups showed varied preferences in sharing the BoK, as seen in the lower scores on these three scales. For instance, American students preferred sharing their work with peers (79.5%). They found the most emotional support from their peers in engaging in critical dialogue and deconstructing complex issues (75%). Canadian students (84.3%) used the BoK to reflect on concepts and ideas from their course more so than the other participants.

Table 7.

Percentages, Skewness, and Kurtosis of the Agreement for Each Item ≥ 4.

Items		Percentages	Skewness	Kurtosis
TD1	I have noticed a change in knowledge, values, and skills	81.5	2.62	7.10
TD2	I have developed critical thinking on social issues	87.5	3.44	13.28
TD3	I have increased my self-awareness and critical self-reflection	80.2	2.30	4.69
TD33	I have developed greater empathy for others	81.9	2.69	7.43
BP1	I am developing a professional identity	84.3	2.92	8.50
BP2	I am better understanding the relationship between personal and professional change	84.7	2.77	7.93
BP3	I am taking more risks and developing greater self-confidence	87.5	1.44	1.32
BP4	My peers provided emotional support, a capacity to engage in critical dialogue and helped me to deconstruct complex issues	55.6	1.17	.52
BP5	I am developing greater humility while understanding the limits of knowledge when helping others	82.7	2.81	9.30
BP6	I am experiencing social work as a calling	77.4	2.12	3.90
PE1	The BOK helped me to better understand the intersection of my personal and professional identities	84.7	2.66	6.17
PE2	The BOK helped me to understand myself in a deeper way than before	77.8	1.90	2.33
PE3	I found the BOK to be a useful tool for my personal and professional growth	85.5	2.83	7.46
FTL1	The BOK is a good way to express and develop creativity	81.0	2.64	6.54
FTL2	I found the BOK to be an effective way to consider and process my emotions	80.6	2.20	3.69
FTL22	Going back to previous entries, I discovered new information or insights about myself, the content, or class discussion	83.9	2.60	3.69
FTL3	I felt more connected to my classmates and instructor through sharing and/or talking about my BOK entries	37.1	.61	−.62
FTL33	I enjoyed seeing other BOK entries and talking to my classmates about their BOK use	38.7	.71	−.32
FTL5	I felt comfortable disclosing my honest thoughts and feelings in my BOK	84.3	1.46	1.04
FTL66	My use of the BOK contributed to my understanding of the course material and/or readings	81.9	2.45	5.46
FTL7	The use of reflective journaling in this course enhanced my overall learning experience	83.9	2.65	6.34

Discussion

The primary objective of this paper was to assess the validity of a tool we have denoted as the Social Work Transformation Survey (SWTS). This measure is a unique fusion of Mezirow’s theoretical framework and empirical qualitative insights derived from Damianakis’ et al. work in 2019. The SWTS was subjected to testing within the context of a pedagogical approach of journaling known by Larrison and Korr (2013) as “Book of Ken” (BoK), characterized by its emphasis on critical reflection. The integration of quantitative and qualitative research techniques represents a novel and promising avenue for advancing the field of social work education. New approaches in the studies of TL should encourage dialogue among researchers utilizing qualitative, quantitative, and mixed methods as the trajectory of TL theory development leads to new challenges (Pang et al., 2023). This methodological blend can offer researchers a more comprehensive perspective on intricate teaching of social issues and the corresponding interventions. The SWTS offers researchers who study transformative learning the development and validation of a quantitative survey instrument for assessing transformative learning in higher education, the SWTS (Walker, 2018).

This study demonstrates a process where a theoretical model that is grounded in qualitative research undergoes transformation into a quantitative scale and is subsequently subjected to rigorous statistical analysis. This innovative approach serves as a bridge between the traditionally distinct realms of qualitative and quantitative research, yielding a standardized instrument to evaluate transformative learning and its associated outcomes.

One notable contribution of this research is the introduction of a refined quantitative lens through which to explore the multifaceted dynamics of transformative learning. The resultant research instrument is both new and original, boasting a robust theoretical foundation. Its flexibility and reliability can facilitate swift and accurate assessments of various elements within the context of social work education.

The meticulous evaluation of the scale’s metric properties underscores its reliability and validity, rendering it suitable for application among university-level social work students. This measure can effectively gauge transformative learning and differentiate between aspects such as deep learning, critical thinking, personal growth, and professional identity development. Consequently, SWTS serve as an invaluable tool for appraising pedagogical methodologies within the sphere of social work education and applied across diverse international contexts. While the assessment of pedagogical methods linked to transformative learning is a relatively nascent endeavour, this study offers an exemplary model of how a theoretical framework can be translated into a quantitative scale.

The study’s outcomes underscore that students from diverse countries, cultures, and linguistic backgrounds held a favourable perception of the pedagogical approach (journalling activity) under evaluation. Furthermore, the findings suggest that the scales employed in this study possess the adaptability and versatility to yield similar results in varying social work educational settings. This discovery carries significant weight in light of the overarching objective of social work education—to cultivate critical-minded social workers whose perspectives and mindsets are transformed.

Conclusion

The scales employed in this research have demonstrated acceptable reliability and validity metrics. Enhancing their applicability and generalizability could also be achieved through further validation across diverse social work educational settings and with varied populations. Moreover, the development of novel scales designed to capture distinct facets of transformative learning could facilitate a deeper comprehension of this phenomenon.

Although this research evaluation has revealed that students possess positive perceptions of the journaling activity, the translation of these perceptions into sustained alterations in their cognition and behaviour remains a subject for future investigation. It is paramount to ascertain how the Social Work Transformation Survey (SWTS) performs when tested alongside alternative critical reflection pedagogical tools in on-campus settings.

Subsequent research endeavours might include follow-up assessments aimed at evaluating the enduring impact of transformative learning outcomes over time. It would be exciting to see the survey used in connection to other pedagogies in social work education, such as photovoice and poster work or to any of the examples identified in the previously in this paper (e.g., Das & Anand, 2014; Bay & Macfarlane, 2011). Furthermore, exploring the intricate interplay between transformative learning and its role in driving social change holds significant promise. While transformative learning is commonly perceived as a catalyst for social change, the precise nature of this relationship remains elusive. Future research could delve into how transformative learning experiences translate into tangible actions that promote social justice and equity.

This research has effectively demonstrated the applicability of transformative learning scales in diverse educational contexts. Nevertheless, conducting further comparative analyses could shed light on how distinct pedagogical approaches, institutional cultures, and social contexts influence the transformative learning process.

A limitation of this study pertains to the absence of an intersectional analysis of transformative learning. While transformative learning is typically investigated in relation to individual experiences and identities, it is imperative to acknowledge the complex interplay of various identity markers, such as race, gender, class, and others, and their impact on the transformative learning journey. Future research initiatives could explore how these intersecting dimensions shape transformative learning outcomes.

This study offers a valuable illustration of how the fusion of quantitative and qualitative research methodologies can yield a transformative learning scale and facilitate the evaluation of a pedagogical approach. The authors hope that educators in higher education settings across various disciplines will draw inspiration from this approach. Furthermore, this research underscores the necessity for ongoing testing and refinement of the scales employed to assess transformative learning, especially within the context of social work education.

Footnotes

Acknowledgements

The authors would like to thank the student volunteers who gave their time to this project: Melisa Lopez, Erika Melisa Gutierrez, and Iride Sánchez from Spain and Clare Hodgkin, UK and Paige Saddler the USA. These students helped with survey design and recruitment. A special thanks goes to Kemi Adebayo for her dedication and field practicum work on this project as a research assistant. The authors would also like to thank Esther Mercado Garcia, Marta Blanco Carrasco, Einav Segev, Lisa Chen Henglien, and Nishi Mitra who contributed to the project. Finally, the authors are grateful for the support and sponsorship of our project by the International Association of School of Social Work (IASSW).

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the IASSW.

ORCID iD

Michael Wallengren-Lynch

References

Afrouz

(2021). Approaching uncertainty in social work education, a lesson from COVID-19 pandemic. Qualitative Social Work, 20(1–2), 561–567. https://doi.org/10.1177/1473325020981078

Archer-Kuhn

Ayala

Hewson

Letkemann

(2020). Canadian reflections on the Covid-19 pandemic in social work education: From tsunami to innovation. Social Work Education, 39(8), 1010–1018. https://doi.org/10.1080/02615479.2020.1826922

Askeland

G. A.

Fook

(2009). Critical reflection in social work. European Journal of Social Work, 12(3), 287–292. https://doi.org/10.1080/13691450903100851

Bay

Macfarlane

(2011). Teaching critical reflection: A tool for transformative learning in social work? Social Work Education, 30(7), 745–758. https://doi.org/10.1080/02615479.2010.516429

Campbell

Stanley

(2005). Diseños experimentales y cuasi experimentales en la investigación social. Amorrortu.

Capous-Desyllas

Bromfield

N. F.

(2020). Field note – exploring the use of arts-informed journaling in social work field seminars. Journal of Social Work Education, 56(1), 201–209. https://doi.org/10.1080/10437797.2019.1627259

Capous-Desyllas

Sinclair

(2014). Zine-making as a pedagogical tool for transformative learning in social work education. Social Work Education, 33(3), 296–316. https://doi.org/10.1080/02615479.2013.805194

Carvajal

Centeno

Watson

Martínez

Sanz Rubiales

Á.

(2011). Cómo validar un instrumento de medida de la salud? Anales del Sistema Sanitario de Navarra, 34(1), 63–72. https://doi.org/10.4321/S1137-66272011000100007

Collins

(2021). Social Workers and Self-Care: A Promoted yet Unexamined Concept? Practice, 33(2), 87–102. https://doi.org/10.1080/09503153.2019.1709635

10.

Cox

R. C.

(2021). Grounding transformative learning through assessment: TROPOS (TRansformative outcomes and PrOcesses scale). Journal of Transformative Education, 19(4), 383–399. https://doi.org/10.1177/15413446211045163

11.

Creswell

Plano Clark

(2017). Designing and Conducting Mixed Methods Research (3rd ed.). Sage Publications.

12.

Damianakis

Barrett

Archer-Kuhn

Samson

Matin

Ahern

(2019). Transformative learning in graduate education: Masters of social work students’ experiences of personal and professional learning. Studies in Higher Education, 45(9), 2011. https://doi.org/10.1080/03075079.2019.1650735

13.

Das

Anand

J. C.

(2014). Strategies for critical reflection in international contexts for social work students. International Social Work, 57(2), 109–120. https://doi.org/10.1177/0020872812443693

14.

Dominguez-Lara

(2018). Fiabilidad y alfa ordinal. Actas Urológicas españolas, 42(2), 140–141. https://doi.org/10.1016/j.acuro.2017.07.002

15.

Escobar-Pérez

Cuervo-Martínez

Á.

(2008). Validez de contenido y juicio de expertos: Una aproximación a su utilización. Avances en medición, 6(1), 27–36.

16.

Espinoza

S. C.

Novoa-Muñoz

(2018). Ventajas del alfa ordinal respecto al alfa de Cronbach ilustradas con la encuesta AUDIT-OMS. Revista Panamericana de Salud Públic, 42, e65. https://doi.org/10.26633/RPSP.2018.65

17.

Henseler

Ringle

C. M.

Sarstedt

(2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8

18.

Herrero

Dedeurwaerdere

Osinski

(2018). Design features for social learning in transformative transdisciplinary research. Sustainability Science, 14(2), 751–769. https://doi.org/10.1007/s11625-018-0641-7

19.

Hubbs

Brand

C. F.

(2010). Learning from the inside out: A method for analyzing reflective journals in the college classroom. Journal of Experiential Education, 33(1), 56–71. https://doi.org/10.1177/105382591003300105

20.

Jones

(2009). Transformational learners: Transformational teachers. Australian Journal of Teacher Education, 34(2), 2. https://doi.org/10.14221/ajte.2009v34n2.2

21.

Jones

Zhao

(2015). Transformative learning theory: Addressing new challenges in social work education. Exploring learning and teaching in higher education (pp. 267–286). Springer Berlin Heidelberg.

22.

Karvinen-Niinikoski

(2009). Promises and pressures of critical reflection for social work coping in change. European Journal of Social Work, 12(3), 333–348. https://doi.org/10.1080/13691450903090771

23.

Kember

Leung

P. Y.

Jones

Yuen Loke

McKay

Sinclair

Tse

Webb

Yuet Wong

Wong

Yeung

(2000). Development of a questionnaire to measure the level of reflective thinking. Assessment and Evaluation in Higher Education, 25(4), 381–395. https://doi.org/10.1080/713611442

24.

King

K. P.

(2009). Handbook of the evolving research of transformative learning based on the learning activities survey (10th anniversary ed.). Information Age Publishing.

25.

Larrison

Korr

(2013). Does social work have a signature pedagogy? Journal of Social Work Education, 49(2), 194–206. https://doi.org/10.1080/10437797.2013.768102

26.

Lay

McGuire

(2010). Building a lens for critical reflection and reflexivity in social work education. Social Work Education, 29(5), 539–550. https://doi.org/10.1080/02615470903159125

27.

Malka

(2022). Real-time lived experience of social work students in their field training during the coronavirus crisis: Insights from photovoice-based research. British Journal of Social Work, 52(1), 311–333. https://doi.org/10.1093/bjsw/bcaa240

28.

McCusker

(2013). Harnessing the potential of constructive developmental pedagogy to achieve transformative learning in social work education. Journal of Transformative Education, 11(1), 3–25. https://doi.org/10.1177/1541344613482522

29.

Mezirow

(1981). A critical theory of adult learning and education. Adult Education, 32(1), 3–24. https://doi.org/10.1177/074171368103200101

30.

Mezirow

(2012). Learning to think like an adult: Core concepts of transformation theory. In Taylor

E. W.

Cranton

(Eds.), Handbook of transformative learning: Theory, research, and practice (pp. 3–34). Jossey-Bass.

31.

Morley

Ablett

Noble

Cowden

(Eds.), (2020). The routledge handbook of critical pedagogies for social work. Taylor and Francis.

32.

Morley

O’Bree

(2021). Critical reflection: An imperative skill for social work organisations? Social Sciences, 10(3), 97. https://doi.org/10.3390/socsci10030097

33.

Newman

(2012). Calling transformative learning into question: Some mutinous thoughts. Adult Education Quarterly, 62(1), 36–55. https://doi.org/10.1177/0741713610392768

34.

Oterholm

(2009). Online critical reflection in social work education. European Journal of Social Work, 12(3), 363–375. https://doi.org/10.1080/13691450902930738

35.

Pang

Cox

R. C.

Acheson

(2023). Assessing transformative learning in international education: Critiques and new directions based on a systematic review of literature. New Directions for Adult and Continuing Education, 2023(177), 75–89. https://doi.org/10.1002/ace.20480

36.

Peters

G. J. Y.

(2014). The alpha and the omega of scale reliability and validity: Why and how to abandon Cronbach’s alpha and the route towards more comprehensive assessment of scale quality. European Health Psychologist, 16(2), 56–69. https://doi.org/10.4236/health.211.39101

37.

Pituch

K. A.

Stevens

J. P.

(2015). Applied multivariate statistics for the social sciences: Analyses with SAS and IBM’s SPSS. Routledge.

38.

Romano

(2018). Transformative learning: A review of the assessment tools. Journal of Transformative Learning, 5(1), 53–70.

39.

Snyder

L. G.

Snyder

M. J.

(2008). Teaching critical thinking and problem-solving skills. Delta Pi Epsilon Journal, 1(2), 90–99.

40.

Stevens

D. D.

Cooper

J. E.

(2009). Journal keeping: How to use reflective writing for effective learning, teaching, professional insight, and positive change. Stylus Pub.

41.

Stuckey

H. L.

Cranton

(2013). Developing a survey of transformative learning outcomes and processes based on theoretical principles. Journal of Transformative Education, 11(4), 211–228. https://doi.org/10.1177/1541344614540335

42.

Sudriman

Sherly

Candra

Dharma

Darwin

(2021). Determinants of Teacher Performance: Exploring the Role of Satisfaction and Motivation as Mediation. Journal Pendidikan Dan Pengajaran, 54(1), 68. https://doi.org/10.23887/jpp.v54i1

43.

Teddlie

Tashakkori

(2010). Overview of contemporary issues in mixed methods research. In Tashakkori

Teddlie

(Eds.), SAGE Handbook of Mixed Methods in Social & Behavioral Research (2nd ed., pp. 1–42). Sage Publications.

44.

Theobald

Gardner

Long

(2017). Teaching critical reflection in social work field education. Journal of Social Work Education, 53(2), 300–311. https://doi.org/10.1080/10437797.2016.1266978

45.

S. M.

Yang

Dong

Yan

M. W.

Y. Y.

Chung

M. Y.

(2022). Development and validation of a quantitative measure for parent empowerment via transformative learning. Frontiers in Psychology, 13, 934142. https://doi.org/10.3389/fpsyg.2022.934142

46.

Vinjamuri

Warde

Kolb

(2017). The reflective diary: An experiential tool for enhancing social work students’ research learning. Social Work Education, 36(8), 933–945, https://doi.org/10.1080/02615479.2017.1362379

47.

Walker

S. L.

(2018). Development and validation of an instrument for assessing transformative learning: The transformative learning environments survey (TLES). Journal of Transformative Learning, 5(1), 23–46.

48.

Wallengren

Archer-Kuhn

Earls Larrison

Mercado Garcia

Chen Henglien

Mitra

Segev

Blanco

(2023). “Its not a book; it’s a BoK”: Social work students’ experience of using creative journaling practices during the COVID-19 pandemic. Social Work Education. https://doi.org/10.1080/02615479.2023.2292127

49.

Zhang

Dempsey

(2019). Exploration and confirmation of a reflective-thinking scale to measure transformative learning in online courses. Assessment and Evaluation in Higher Education, 44(3), 463–475. https://doi.org/10.1080/02602938.2018.1520194

50.

Zumbo

Gadermann

Zeisser

(2007). Journal of modern applied statistical methods, 6(1). https://doi.org/10.22237/jmasm/1177992180

Measuring and Validating a Transformation Learning Survey Through Social Work Education Research

Abstract

Keywords

Introduction

Transformative Learning in Social Work

Critical Reflection Pedagogies as Triggering Transformative Learning

Measuring Transformative Learning

Rationale for This Study

Methodology and Methods

Sample

Ethics

Design

Recruitment

Procedure

Variables and Measuring Instrument

Results

Factor Analysis Scale Transformative Learning

Factor Analysis Scale Transformative Learning

Reliability of TT and TL

Eliciting Experts and Evaluation of the Tool in the Sample

Discussion

Conclusion

Footnotes

Acknowledgements

Declaration of conflicting interests

Funding

ORCID iD

References