Abstract
Professional development is essential for professionals to maintain their skills and knowledge as industry expectations, technology, and economies evolve. To ensure effective learning outcomes, a Professional Learning Design Framework (PLDF) was created to align evidence-based goals with suitable educational methods and evaluation tools. This study evaluates the PLDF’s effectiveness in designing and implementing ‘Learning from Accident Cases’ workshops to enhance safety decision-making among engineers. These workshops were explicitly created to improve engineers’ safety decision-making capabilities. The research question focuses on whether the PLDF is effective in practice. The study involved designing and assessing workshops based on the PLDF, with evaluations conducted through post-workshop focus groups and follow-up interviews. Results indicated that participants either improved or reaffirmed their safety decision-making processes, demonstrating that the workshops met their learning objectives. Feedback from participants confirmed that the learning outcomes were both achieved and valued. Overall, the PLDF proved to be a valuable tool for guiding the development, delivery, and evaluation of professional development activities, ensuring their impact and effectiveness in practice.
Keywords
Introduction
Ongoing professional learning (PL) is critical for maintaining up-to-date skills and knowledge, ensuring alignment with evolving professional standards and technological advancements. According to Holdsworth et al. (2022), PL is a dynamic social process where individuals actively engage in learning activities rooted in their professional responsibilities, contexts, and groups, addressing ‘real-world’ problems derived from their professional practice. As such, PL is critical to a professional’s working life and should inform professional development (PD) activities (Billett & Harteis, 2014).
When PD activities are grounded in the principles of PL, they foster the ongoing development and application of professional capabilities within the context of actual practice. Professionals acquire knowledge and skills through formal education and experience. Separating learning from day-to-day practice diminishes the quality of the learning experience and its outcomes (Rooney et al., 2012). Decoupling PD from the work context and focusing solely on content delivery is ineffective for advancing professional learning and capability development (Corrigan et al., 2015; Webster-Wright, 2009). Therefore, integrating learning into professional practice is essential for fostering meaningful and impactful development.
Capabilities are qualities and skills that allow professionals to operate and respond effectively in diverse, complex, and uncertain situations (Holdsworth & Thomas, 2020; Sandri, 2010). Capabilities are developed through combining knowledge, skills, personal attributes, and understanding, applied appropriately and effectively in familiar, specialised settings and in response to new and evolving situations (Stephenson, 1998). Consequently, emerging education research on professional and practice-based learning is an important topic whose findings require broad consideration and dissemination to support individuals’ employability and lengthen their professional lives.
This paper describes the use and evaluation of the Professional Learning Design Framework (PLDF) developed by Holdsworth et al. (2022) to guide the development of PD, Learning from Accident Cases workshops, underpinned by a PL philosophy. It answers the research question: Is the PLDF effective in practice? It does this by assessing the learning outcomes and evaluating the achievement of learning objectives from the workshop. The Learning from Accident Cases workshops aimed to develop engineers working in hazardous industries’ capabilities in safety decision-making, as their daily decisions can significantly impact public safety outcomes. Sound engineering practice rests on knowledge beyond the technical, with professionals benefiting from cultivating non-technical capabilities (Holdsworth et al., 2022).
This paper evaluates these workshops using focus groups conducted with participants immediately after the PD session and post-workshop follow-up interviews with participants to determine if the workshop’s learning objectives and outcomes were achieved and valued. This paper begins with a discussion of the importance of PL underpinning PD and its current gap in practice, an overview of the PLDF and its application in the development of the Learning from Accident Cases workshops, followed by the methodology and findings from the evaluation. Finally, implications for good practice PD programs and the value of the PLDF are explored.
Professional Learning Design Framework (PLDF): Design of Learning from Accident Cases workshops
Professional development by professional learning
PD has been widely valued, yet it often centres on building competencies aligned with industry standards and historical employer needs rather than fostering a dynamic understanding of evolving professional contexts (Stephenson, 1998). This traditional approach typically emphasises ‘knowing that’ – the theoretical knowledge and content required for specific tasks and outcomes – rather than ‘knowing in action’ that emphasises practical application and adaptive skills in real-world settings (Cook & Brown, 1999). Consequently, the impact of PD on practical learning outcomes can be limited, as it frequently remains disconnected from authentic work contexts, resulting in a lower return on investment (Corrigan et al., 2015).
Webster-Wright (2009) has critiqued the traditional PD model as ‘deficient and in need of developing’ to address contemporary professional challenges (p. 712). To meet the complex demands of modern industries, PD should integrate technical skills with theoretical knowledge alongside contextual awareness that includes ethical considerations. This holistic approach to competency development enhances professional judgment by acknowledging factors that traditional PD may overlook or inadequately address (Holdsworth et al., 2022).
A shift towards capability development aligns PD with the principles of PL, which emphasise expertise gained from reflective practice and real-world experience (Hetzner et al., 2015). Unlike mere competency, capability encompasses current proficiency and the ability to anticipate future needs and contribute to advancements. As Stephenson (1998) notes, capable professionals can foresee changes, engage proactively in their development, and effectively respond to evolving demands. Capability combines specialist knowledge with ethical decision-making, enabling individuals to act effectively in familiar and novel situations. It also includes the ability to interpret complex, unprecedented scenarios with adaptability, confidence, and wisdom (Sadler, 2013).
Developing capability in professionals requires a critically reflexive approach to growth that emphasises an ‘epistemology of practice’—fostering ‘knowing in practice’ as opposed to mere replication of procedural knowledge (Markauskaite & Goodyear, 2014, p. 87). This reflective praxis supports professionals in making informed judgments that account for ethical implications, long-term outcomes, and the complexity of systems influencing their decisions and actions (Lambert et al., 2012). However, a review of the literature on PD suggests it is often grounded in an ‘objectivist epistemology and dualist ontology’ (Webster-Wright, 2009, p. 714). This framework tends to objectify knowledge, resulting in a top-down, standardised approach to learning where information is transferred from instructor to learner in formal educational settings, disconnected from the sociocultural contexts in which this knowledge is meant to be applied. Such an approach conflicts with PL, which emerges from intentional, active engagement in relevant activities that address authentic professional challenges involving diverse stakeholders (Webster-Wright, 2009).
Despite the recognised value of PL’s active, learner-centred methods, PD programs often lack this focus. This gap is significant, as PL differs from conventional PD, which typically involves periodic updates delivered in a didactic format, detached from real-world experience (Webster-Wright, 2009). PD should facilitate the exploration of new professional identities and ways of being rather than merely reinforcing competence in specific tasks (Wenger, 1998).
To enhance effectiveness, PD must evolve to embrace models of ‘learning as participation’, ‘learning as construction’, and ‘learning as becoming’ (Rooney et al., 2012, p. 118). Situated learning within the workplace can immerse individuals in active problem-solving alongside colleagues, engaging with the actual challenges of their professional practice. In educational research, effective PD is characterised by learning that is continuous, active, social, and grounded in practice (Garet et al., 2001). However, this holistic approach remains underutilised in practice. Learning should be seen as an outcome of intentional, dynamic interaction with experience and reflection, not confined to a formal/informal binary, which Lave and Wenger (1991) argue limits our understanding of learning processes.
Larrea et al. (2024) suggest that when contextualised as part of a developing social environment, formal training enhances individuals’ participation in complex work practices (p. 5). As Wenger emphasises, education at its best is ‘not merely formative — it is transformative’ (Wenger, 1998, p. 263), fostering a learning process that develops professionals into adaptable, reflective practitioners who continuously grow in response to real-world demands.
Adult learning theory, also known as andragogy, was first formalised by Knowles (1978), who emphasised the importance of lifelong learning and the need to understand adult education as a means of fostering continuous development. To keep learning engaging and meaningful, it must incorporate new concepts, motivations, and methods (Kelly, 2017).
Knowles (1978) argued that adults learn most effectively when they recognise a personal need for knowledge, often shaped by their life experiences, which serve as valuable learning resources. Reflecting on acquired knowledge and critically examining one’s beliefs, assumptions, and perspectives can lead to shifts in thinking and influence future actions. This process, which involves both intentional and incidental learning, can result in transformative learning and the refinement of knowledge (Cranton, 2006; Mezirow, 2000).
Furthermore, learning is most effective when it is self-directed and tailored to individual needs (Lindeman, 1926). In PL contexts, as described in this paper, these principles are applied to create relevant and engaging learning experiences that support continuous growth and development.
The PLDF and the Learning from Accident Cases workshops
In this research, the PLDF has been applied to address the current failings of PD and the resultant lost learning opportunities. The PLDF emphasises designing PL experiences, focusing on learning as the outcome rather than instruction as the approach (Wasson & Kirschner, 2020). Depending on the needs identified through the framework, learning experiences could include formal learning modules or other activities tailored to work practices’ needs and contexts. The framework presents a systematic process for designing PL experiences, starting with establishing the PL needs and then aligning the delivery approach and methods to meet these needs. The decision-making in PLDF-guided learning design is based on deliberate choices about why, what, when, where, and how to structure PL rooted in professional practices and the required capabilities for these contexts, thereby transforming how PD is conceptualised and delivered to support continuous learning.
The PLDF responds to a gap observed in the context of the engineering safety PD, where there has been little engagement with the underpinning educational philosophy and a lack of reflection on the andragogy (i.e. theory of adult learning) that informs professional development and training approaches, methods, and learning outcomes (Holdsowrth et al., 2022). In fact, while it is increasingly recognised that lifelong learning and ongoing PD are important for engineering professionals to ensure that their knowledge is maintained as technology, society, and practices evolve (Aerts et al., 2020; Johari, 2021), lifelong learning has not been a focus in engineering education research (Johari, 2021).
The PLDF is built from the Higher Education Learning Design (HELD) Framework, created by one of the authors of this paper, to promote transformative learning in higher education while addressing the needs of industry and society (Holdsworth & Hegarty, 2016). The HELD framework aimed to capture the ‘lived experience’ and practical insights involved in the complex processes of designing and delivering courses and programs, including developing learning outcomes that contribute directly to the learners’ working life and achieving change towards moral outcomes.
The framework is founded on praxis, emphasising the iterative relationship between educational methods and outcomes and the ongoing interaction among educational theory, personal teaching practices, and real-life experiences. The framework encourages reflection on the assumptions surrounding learning and teaching, education’s role, and desired learning outcomes (Holdsworth & Hegarty, 2016).
Good practice education constructively aligns educational approaches, learning activities, and assessments to meet the learner cohort’s needs and support desired learning outcomes (Biggs & Tang, 2011; Holdsworth & Hegarty, 2016). The PLDF promotes such alignment, offering a systematic process for designing learning experiences. The PLDF informed the development of safety-related non-technical PD for professional gas pipeline engineers in Australia (Hayes et al., 2021). For engineers, non-technical skills are essential for safety decision-making in complex socio-technical systems. Yet, PD for engineers prominently focuses only on developing technical skills (Hayes et al., 2021). Thus, the project was conceived from the Australian pipeline industry’s recognition that engineers’ non-technical skills are crucial for maintaining the sector’s strong safety record and are not developed in existing professional competency training.
Underpinning this project was the identification of capabilities beyond the technical that support safe decisions in uncertain, dynamic situations in engineering practice. Major reviews of engineering education and the engineering profession in several countries over decades have emphasised the extent to which the human side of the profession, where decisions are linked to uncertainty and are fundamentally value-driven, has been underemphasised in favour of the technical (Duderstadt, 2010; Edström, 2018). To address this gap, a previous study interviewed 40 pipeline engineers to develop a framework of capabilities for professional pipeline engineers to perform their professional role while navigating a complex organisational context (Hayes et al., 2021). Acknowledging that these capabilities are typically not included in standard educational curricula or easily integrated into PD programs – due to the constraints of conventional approaches – this research aimed to foster and improve these abilities among working engineers. It incorporated ‘formal’ learning experiences that emphasise experiential and situated learning opportunities to reflect on and enhance these capabilities.
The PLDF guided the PD design and informed it with evidence-based desired learning outcomes drawn from these previously identified safety decision-making capabilities, the appropriate education andragogy, learning and teaching methods, and curriculum. The resultant workshops were delivered in Australia with nearly 100 pipeline engineer participants and drew on discussion cases and role-plays to achieve set learning outcomes. The workshop content was drawn from four accident cases that enabled participants to situate and explore professional decision-making in a context related to public safety. The workshops aimed to support PL and integrate learning with professional practice to build functional knowledge and capability rather than imparting content knowledge. The workshops were designed to meet the following four requirements: • Grounded in the learning needs of professionals. • Linked directly to professional practice. • Capabilities-focused. • Evaluated using a workplace-focused approach.
The application of the PLDF and associated activities undertaken to develop the Learning from Accident Cases PD workshop following the six stages is presented below; further details about the PLDF can be explored in Holdsowrth et al. (2022).
PLDF: Stage 1. Needs identification
Action
Identify PL needs, that is, knowledge, skills, competencies/capabilities, and their relationship to professional goals and organisational objectives drawn from literature, professional practice, and lived experience.
Response
Risk is a key focus in engineering safety. However, current safety management often overlooks the role of human error and the need for professional judgment in uncertain circumstances (Lombardi & Fargnoli, 2018). Safety learning should focus on developing capabilities that empower engineers to act effectively in diverse contexts. This approach values both competence and capabilities. It was identified that while engineers’ non-technical skills are critical in maintaining the sector’s good safety record through upcoming transitions, capabilities are not systematically defined, addressed, or included in existing PD.
PLDF: Stage 2. Learning objectives and outcomes
Action
Develop learning objectives and outcomes to address industry/professional knowledge or skill gaps focused on professional contexts.
Response
To improve safety, professional engineers should focus not only on competence and the application of highly technical principles to well-defined technical problems but also in uncertain circumstances where decisions are fundamentally value-driven with an ability to link one’s actions to the potential consequences. As such, the workshop’s objective was to educate on best-practice decision-making for public safety. The workshop’s learning outcome was to develop/reinforce pipeline engineers’ capabilities aligned with decision-making and public safety.
Stage 3: Professional capabilities
Action
Define capabilities, competencies, and skills reflecting the organisation’s and profession’s needs.
Response
Engineering capabilities for decision-making for public safety.
Stage 4: Educational approach
Action
Consider the educational philosophy underpinning teaching methods, curriculum, and assessment.
Response
The workshop design uses an experiential and inquiry-based learning philosophy to meet specific educational objectives. Experiential learning enables participants to apply their knowledge in real scenarios, fostering observation and reflection through iterative cycles of practice and theory development (Kolb, 1976). Although the PD workshop occurs outside the workplace, it incorporates current engineering challenges to enhance experiential learning, addressing the issue of transferability to other situations and encouraging self-reflection for improvement (Nicholls, 2001).
The learning experiences reflected in these educational philosophies lead to reflective observation, forming abstract concepts, and using these concepts for decision-making and experimental action beyond simply reflecting ‘in action’ (immediate decision-making) but also ‘reflection on action’ (longer-term contemplation) (Schön, 1988). Reflection on action manifests in double-loop learning experiences as dilemmas explored through inquiry-based scenarios expose the learner to conflict, and reflection leads to changes in theories or practices. Investigating and building knowledge through experience allows participants to understand the contexts and significance behind the details and facts of the case studies (Biggs & Tang, 2011), facilitating the deep understanding necessary for outcomes focused on capabilities-based education. Such deep learning is founded on reflection (Schön, 1984), using prior knowledge (Knowles, 1980), and motivation. These methods were selected to develop learner capabilities as they require learners to engage meaningfully with ambiguous and unfamiliar contexts, learning by doing or experiential learning (Fraser & Greenhalgh, 2001). The workshops were designed to be delivered face to face. However, an alternative online delivery mode has been developed. While we have found the framework to be equally useful when applied in an e-learning context, the scope of this paper focuses on the face-to-face workshops.
Stage 5: Activities/content
Action
Develop and deliver content materials and learning activities aligned with the capabilities, learning objectives, and outcomes. In the case of PL, a learner-centred philosophy builds on existing professional practices and experiences within the work context and lends itself to activities inviting learners to express their views, share experiences, and develop new knowledge and understanding.
Response
Workshops structured around disaster cases were selected as the learning and teaching method because they: • Immersed participants in a particular case. • Enabled reflections on the case and identification of the gap in the professional practice of those involved and their professionals.
Two different sets of cases were used during the workshop delivery. Each case was structured around 1.5 hours of face-to-face engagement with a delivery facilitator. The first set of cases included the San Bruno pipeline failure (Hayes & Hopkins, 2014) and the Challenger space shuttle accident (Rogers, 1986), designed for transmission pipeline engineers, and the second set included the Massachusetts pipeline failure (NTSB, 2019) and the amusement ride accident at Dreamworld (Coroners Court of Queensland, 2020), designed for network engineers.
The disaster cases were structured as discussion cases and role-plays. Discussion cases involve presenting case material followed by a facilitated discussion to help the learner appreciate the nature of the problem and solutions through analysis and collaborative discussions (Maslen & Hayes, 2020). Role-plays allow learners to assume roles that personalise interactions within a given content. To develop the identified capabilities, participants first engage with a scenario, apply knowledge and skills in response to the scenario, and reflect on the outcomes of their response and their actions and learning through the inquiry process.
A discussion case was presented in the first part of the workshops, introducing participants to an accident case in a familiar professional context. The second part of the workshop focussed on a case role-play exploring a disaster case outside of the pipeline sector. It supported learning about engineering decision-making in a non-pipeline context, encouraging participants to extrapolate learnings to their profession. The workshops concluded with a facilitated discussion encouraging participants to reflect on the links between the accident cases and their professional practice.
Through past disaster cases, workshops were designed to explore professional capabilities, using narratives to give meaning to experiences and highlight the significance of actions and events (Schank, 1990; Schank & Abelson, 1995). Learning activities designed around the disaster cases promoted personalised experience relative to practice and active engagement in critical reflection and helped develop a ‘safety imagination’, linking everyday incidents to potential severe outcomes from failed risk controls (Pidgeon & O'Leary, 2000). This approach underscores the importance of interpreting current events through the lens of past experiences, making the narratives more than just a compilation of facts but a way to understand the significance of events relating to one another (Polkinghorne, 1988).
Content and activities in each workshop design.
Stage 6: Evaluation strategy
Action
Evaluation of PL is crucial for assessing its impact and alignment with organisational needs. The Kirkpatrick model forms the foundation of the evaluation model, gathering data on participants’ learning experiences and workplace applications.
Response
According to Topno (2012), good PD should start with a gap analysis and end with evaluating the resulting training (Gopal, 2008). Therefore, evaluation is best understood as a systematic process to determine the program’s value against a developed comparison criterion. It is important to evaluate whether participants have acquired new knowledge and skills and can implement their learning in their workplace after completing the training (Boulmetis & Dutwin, 2000).
Evaluating workplace training is challenging due to the subjective nature of learning and change. It can be difficult to set measurable objectives, collect information on the results, and decide the appropriate level at which the evaluation should be conducted (Nagar, 2009). The PLDF advocates using the Kirkpatrick Training Evaluation Model, which suggests four levels of assessment and evaluation. Evaluation determines if the module’s learning outcomes have been achieved (learner-focused), and assessment determines the module’s effectiveness and quality (facilitator- and development-focused). Both assessment and evaluation use methods and measures to judge learning and understanding of the material for grading and reporting. However, assessment tends to focus on formative methods, while evaluation focuses are often considered summative.
Kirkpatrick Model (Reio et al., 2017) proposes four levels of evaluation: • Level 1 – Participants’ reactions: This assesses and evaluates the participants’ reactions and satisfaction with the learning module. • Level 2 – Participants’ learning: This assesses and evaluates the participants’ learning outcomes. Learning is defined as acquiring knowledge, skills, attitude, and confidence against the learning objective. • Level 3 –Participants’ application of learning: This assesses and evaluates participants’ learning outcomes by identifying changes in participants’ workplace behaviour after completing the learning module. • Level 4 – Desired results (organisational impact): This level evaluates the module and determines whether the program’s targeted outcomes or primary goal have been achieved from a whole-organisation perspective.
Research methodology and methods
The PD workshops were evaluated to understand the PLDF’s effectiveness as applied to the workshop design and delivery. This was achieved by identifying the impact of the PD on participants’ subsequent professional behaviour. The paper draws on data from the evaluation methodology developed in line with the PLDF. Four assessment and evaluation methods informed by the Kirkpatrick Training Evaluation Model (Kirkpatrick, 1959a; 1959b, 1960) were adopted to evaluate the workshop’s pedagogical value and the participants’ entire learning experience, including the evaluation scenario, survey, focus group, and post-workshop interviews.
This paper presents the findings from the focus groups and follow-up interviews to assess and evaluate Levels 1, 2, and 3 of Kirkpatrick’s levels of evaluation against the workshop’s learning objective: to educate best-practice decision-making for public safety through developing or reinforcing the learners’ capabilities aligned with decision-making that reflects the paramount importance of public safety. Focus groups were used to evaluate Kirkpatrick’s Level 1, collecting data on participants’ reflections on insights and learning. This method also collects data on Kirkpatrick Level 2 evaluation. Levels 2 and 3 evaluations drew on data from follow-up interviews with workshop participants after three to six months. Questions elicited discussion on workshop recall, learning outcomes, reflection on capabilities, impact on professional practice and public safety, and any other feedback.
Data analysis
Data generated by the focus group and follow-up interviews were analysed using deductive thematic analysis. Thematic analysis was chosen as it allows researchers to capture the richness and complexity of learners’ experiences by identifying patterns and themes across the data. This method enables an in-depth exploration of how participants perceive and make sense of their work environment in their own words, leading to a more nuanced and authentic understanding of their experiences. Thematic analysis assumes that ‘what participants think, feel, and do… reflects… the perspectival reality of a particular participant’ (Terry et al., 2017, p. 19). By examining the social, cultural, and organisational contexts and their effect on engineers’ experiences, researchers can gain insights into decision-making processes and the application of capabilities derived from the workshops. Data were thematically analysed using NVivo 12. Two research team members independently analysed qualitative data, who cross-checked for agreement on the emergent themes, thereby ensuring interrater reliability (Ballinger et al., 2004).
Recruitment and participants
Almost 100 Australian gas pipeline engineers attended PD sessions across ten workshops. Participants were recruited via the researchers’ professional networks by direct approach and promotion through professional social media channels. The RMIT University and University of Canberra Human Research Ethics Committees approved this research.
Workshops delivered and participant numbers.
Completion or participation rates for each evaluation method.
Findings
The following describes the key findings from these evaluation methods aligned with the Kirkpatrick Model of evaluation levels, beginning with Level 1, Reaction.
Lack of space prevents detailed reporting of the results of the evaluation scenario and survey. The results are consistent with the more detailed data from the two methods reported here.
Level 1: Participants’ reactions
Focus group
In the focus groups held directly following each workshop, participants provided positive feedback about their experience and the importance of public safety decision-making in professional practice. Participants reflected on their experiences as engaging, enjoyable, and valuable. The workshop was a well-facilitated, interactive opportunity to critically reflect on decision-making, discuss with and learn from others, and learn relevant and valuable skills for their professional practice.
A key theme identified was that the discussion and role-play teaching methods were novel and engaging. The interactive nature of the workshop learning methods added value to the learning experience. Participant engagement was also a consequence of content – the non-technical aspect of professional practice; one participant commented workshops were: really interesting in the context of lessons learnt … [professional development is] generally very technical … We don't [normally] talk about, where is our decision-making flawed. Or where our decision-making could be improved. So, it's interesting how you can morph this into lessons.
Participants also noted a high level of engagement resulting from learning activities promoting critical thinking: The questions asked after each role-play … brought all that critical thinking back onto the table. That was quite powerful.
Critical reflection and focused active engagement in structured activities enabled peer learning, adding novelty, engagement, and satisfaction to the workshop. One participant commented that these combined elements help participants ‘build connections with colleagues and learn new knowledge’. Another participant commented, ‘It’s good to hear all the people’s perspectives and be able to discuss them at the same time’.
Structuring the workshops around existing disaster cases, highlighting capabilities critical to public safety decision-making, was viewed as beneficial and relevant to participant professional practice. This enabled them to: realise the seriousness of our work’s impacts on public safety, especially our responsibility.
Follow-up interviews
The individual follow-up interviews undertaken approximately three to six months after the workshops supported the above aspects influencing positive learning experiences, including the value of the workshop’s critical and reflective discussion, interactivity, content, and activities grounded in professional practice and personalised learning experiences. Additionally, post-workshop reflections emphasised the value of the role-play experience, contributing to participants’ learning in two ways. Firstly, participants felt an emotional connection to the catastrophic outcomes of the non-industry cases because they recognised the possibility that they, their family, or friends could be caught in a similar situation. One participant commented, ‘The lack of safety first and the disregard for the impact on people’s lives really stuck with me because it could have happened to my family’. This emotional connection deeply engaged learners with the content, facilitating individual and group interrogation of actions and decision-making that was then connected to professional responsibility and subsequent practice, reflected in the following comment: ‘[the role-play] brought home we’re making important decisions about public safety, and that certainly has affected my practice’.
Secondly, role-playing different stakeholders associated with the disaster cases exposes and nurtures within participants an awareness and, in some instances, appreciation of different perspectives, motivations, and understandings of the circumstances that led to the catastrophic failure. When placed in the role of others, participants could better connect their experiences with those in the scenario. One participant commented that the role-play ‘reinforces the lessons personally experienced’, which is essential as it enables them ‘to apply some of the learnings in your [to their] engineering [practice] day-to-day’. Further, the role-play assisted participants in ‘seeing the biases that play out in decisions’. In a learning environment away from professional practice, role-plays enabled participants to ‘step in and see the conversations that are happening’
Level 2: Participants’ learning
A thematic analysis of the focus group discussion identified that the workshop learning objective and outcome were achieved. Participants identified and discussed five capabilities they learned from the session.
I think – in both those cases, nobody understood the consequences of the decisions they were making, and you can't make a good decision without understanding the consequences of that decision, especially when your concern is not just the bottom line; it's people's safety.
Participants referred to We look into the strategies and risks, engage stakeholders, talk with the people, and communicate properly. All those things are there. And making decisions in light of standards, industry best practices, and past experiences.
Additionally, participants commented on the importance of speaking up when observing decisions that fail to protect public safety informed by an understanding of the competing business objectives: I learned from your workshop to convince your management about risk. Most of the time we tell them the risk we don't convince or convey them properly … from today's exercise, I understood … make them understand that there's a risk and something must be done.
Looking at different levels, what people's end goals are, and when you make your decision for your area – if that doesn't get through the first time, you look at what everyone else's goals are, understand it from their view, and then try to restructure it in a way that means something to them, rather than doubling down on just your idea.
The participant’s view was clear: one comprehends the differences and interconnected relationships that shape organisational decisions and actively responds to them during professional practice; unintended consequences cannot be avoided. This underscores the importance of understanding interconnectedness in decision-making and public safety.
The focus group discussion also highlighted participant learning aligned with
Participants also reflected that disaster cases often result from decisions made in the abstract and good decisions need to be
The final capability that the participants discussed as important to future practice was We keep all the data … but we also have the responsibility to translate it to transfer that knowledge to something easier for people to understand and make the best decision.
Participants recognise the importance of their communication skills in communicating risk, especially ‘to be able to convince managers’. One participant commented: One lesson I’ve learned from your workshop is how to convince your management about risk. Most of the time, we tell them the risk, but we don’t convince them properly.
The focus groups indicate that all capabilities are important to practice through actions identified as relevant to public safety decision-making. The workshop met its learning objective and learning outcomes as the data indicate participants reflected on and grew in their understanding and application of the capabilities, validating their importance and encouraging the conscious use and development of new approaches when considering public safety in professional practice.
Level 3: Participants’ application of learning
Level 3 evaluation focuses on assessing and evaluating learner behaviour after completing the learning module, as evidenced by the degree to which participants apply the decision-making for public safety capabilities to their workplace activities. Interviews with participants 3–6 months after workshop participation identified all six capabilities influencing current practice.
Post workshop, participants commented that while their decision-making included consideration of public safety outcomes and drew on the capabilities, the workshop reinforced their approach and gave them confidence in their practice, as illustrated by the following: I'm always very cautious and very mindful about safety. Yeah. So it's more on reinforcement of what you normally do … more like assurance than change [in] my thinking.
Participants identified that the workshops ‘reinforce[d] the professional accountability’ and ‘[acted as a] reminder that if we're not careful, it can happen to us’ and ‘if something does go wrong, there are significant consequences’. Finally, some participants recognised that the PL experience allowed them to reflect on practice and identify areas in which they could develop further.
Capability 1: Use long-term, foresighted reasoning, especially in the face of uncertainty
Participants recalled and applied There have been a few times where I have had to pull myself up and go, ‘What would be the worst case scenario if we did this, followed this train of thought?’ and thinking back to that San Bruno situation … we do have to keep in mind what if all of the layers of protection failed… and every time, it does make me think of the workshop, of ‘this is the worst case that we're trying to prevent here’. So, even though we say, ‘Yeah, these controls will be enough’, it has made me challenge myself and go, ‘Well, actually, what if they're not?’ That's how it's been effective so far.
For those already using the consideration of worst-case outcomes to inform decision-making, the workshop confirmed the action as part of good decision-making for public safety, as highlighted in the following: I've always been wary that any mistake could lead to something dangerous and life-threatening … But these real-life examples … show how only one or two missteps can cause a big disaster. I think these things [PD] are important: always have refreshers.
Participants also identified and applied to choose with the The Massachusetts incident involved pausing a project and having the documentation not carried over. That's something that at our company, there's a risk with, so I've been making sure since [the workshop] my documentation … and all the emails are being traced … if I end up quitting next week … no loose ends that will result in a safety incident.
Finally, participants identified and described how they applied to deal with What's changed … when making decisions, you don't rush … you may not be thinking in a particular direction … You get another opinion; that's where this workshop helped.
Capability 2: Understand norms and values that inform actions
Participants recognised the need to understand the The San Bruno incident showed us that it's … not just about meeting regulatory compliance requirements and creating systems. It's about having the self-confidence that whatever we have done will protect the communities and ensure that the assets are safe in the ground – not just a ticking exercise to satisfy the regulator.
This comment illustrates the value of considering the social context when applying technical standards and associated decisions and actions.
Participants also recognised that We do risk assessments all the time. So, it's easy to put yourself in a position where you normalise risk and risk exposure. I find it valuable when you step away from work and look at a very hands-on workshop … you look at some disasters, what went wrong, and maybe put yourself in the shoes of the people making decisions for that scenario. Then, compare and contrast that with what you do day to day …. I appreciated that and would do it again or in a similar workshop because I found it helpful.
Participants also recalled the importance of The workshop outcomes and participation gave me a perspective on how one decision at the top can impact down the chain … and how everything is connected … If I can influence that in the business as part of my position, I will do so.
As reflected in the above quote, participants identified the importance of awareness of potential business pressures that conflict with technical decisions in the advocacy for ethical choices.
Capability 3: Think systematically and understand interconnectedness
Participants recognised the need to When you are at the helm of affairs, and you want to make a decision, it's not about meeting your targets; it's about thinking big and considering the impact your decisions will have on others … not just financially, but more towards your overall industry, colleagues, and the environment.
However, increasing stakeholder engagement requires management given the competing interests, as identified by one participant: ‘A lasting takeaway I’ve had [from the workshop is the] management of competing interests and different people with different perspectives and agendas’. Given the need to understand content from various perspectives, participants also identified the importance of So, generally, [in the] industry, it's more about following the time, the project timelines, and the budget. We mainly got from the workshop that we need to prioritise public safety; otherwise, negative consequences may occur.
Further, recognising the public safety risk can help refocus the importance of seemingly innocuous smaller decisions that may have culminated in catastrophic consequences. As this participant discussed: The workshop reinforced the need to look at the smaller details, not just the obvious features.
Also, relating to context, participants recognised the need to apply risk concepts to the broader context of decision-making, We went through a risk assessment of looking at the consequences of [stated action] and whether or not it's a tolerable risk or if there are any other actions or mitigation measures we can put in place … that scenario was particularly relevant to some of the lessons that were pointed out in the workshop and [I] applied …. to that risk assessment, looking at what's going to happen if we push it out for a year to be able to do this work [participant continues to list risks explored] ... that was a recent example that applies to the workshop for me.
As reflected in the previous quote, taking time to think systematically and thoroughly to make the right choice was also an important learning from the workshop applied in practice.
Capability 4: Collaborate with and draw on the experience of others
Participants identified the need to When I do things now [post-workshop], I always in the back of my head is like, are you sure about that decision? Is that the easy option, or is that the correct option? I'll just give you a silly example: I'm doing a [stated activity] and for a high-consequence error … [I had a conversation with others and said] ‘Well, here are four options’, and [then] working through what the risk of if we don't do something is, what could happen, and coming to an agreement on it. The people I'm going to do it with have all done your course, which helps because you have those conversations now.
Participants also recognised the importance of What I learnt from that workshop was to try to collaborate with other departments and not be siloed ... You can see the bigger picture.
If that [proposal] does come to me, then I read it; if I don't understand it, I ask questions because I don't want to sign off on something and then … since we're working in gas [have it ] explode like what happened in America [referenced workshop case].
Capability 5: Ground decision in reality
Participants explained the importance and application of I liked the workshop’s emphasis on grounding many of your decisions in reality. Following the workshop, I thought about the practical implications of your decisions and their effects on people on the ground. That’s probably one that resonates strongly with me.
Participants felt that learning from small failures within their workplace or from those of others was beneficial as it provided insight and new knowledge into possible future system failures that can then be considered in future decision-making, as reflected in the following quote: When it comes to the daily decision-making process in the job, that [the workshop] is very valuable. The main key element from the workshop is the thought process when something goes wrong… For example, if any gas incident happens, what are the details we need to collect to avoid the next reoccurrence of the same incidents?
Participants also cited checks on decisions by imagining being in the situation as helpful in informing their decision-making and benefiting public safety by imagining being in the situation, as reflected in the following comment: So, with some of the things that I'm deciding, I used to ask, ‘do I want this to be the next disaster?’ But in my brain, now, it's going. Do I want this to be the Dreamworld failure? Do I want this to be that failure? What could cause that? What could cause that failure?
Capability 6: Advocate for action and take responsibility
Participants recognised in light of different priorities and experiences they need to effectively communicate with senior decision-makers, as reflected in the following comment: In the past I may have considered – ‘Well, at the end of the day the manager is responsible/accountable for their final decision’ – I now may push my point of view more if I believe the manager may be biased in his decision making towards commercial issues rather than safety issues.
Participants also recognised the need to advocate for safety more broadly when they notice something that could potentially adversely impact safety outcomes: The key message from the workshop was that ‘if doing any work, we are responsible for our own design’. If there is external pressure, ‘we have to stand up’.
Discussion and implications
The workplace presents a significant learning challenge, requiring individuals to extract relevant information from previous learning environments and reconceptualise it for new situations (Eraut, 2010). PD allows professionals to develop skills needed for practice; however, it often separates formal learning from day-to-day work practice and has been critiqued as reductionist and facilitating learning for specific skills required by organisations given common didactic philosophical underpinnings. Working to bridge this gap and actively engaging and integrating the components of the PLDF, the authors developed a PD designed to educate professional engineers, underpinned by a PL philosophy, on the importance of understanding and applying the core professional capabilities required for public safety outcomes. The PD learning objective and outcomes reflect the very critique of PD addressing an identified gap in capability, not simply the application of competence aligned with existing professional standards.
As part of developing the PD workshop, the learning design team engaged with literature and industry to identify the capabilities vital to achieving the learning outcomes. The learning outcome was determined by a detailed investigation of the existing safety literature and education theory to ensure that professional practice limitations were the key focus of the learners’ experiences and suitable pedagogy.
Experiential and inquiry-focused learning was selected as the learning design team felt it central to a workshop where participants were focused on relating and applying the identified capabilities to their professional practice. Exploring and constructing knowledge through active learning enables participants to see the contexts and meanings underpinning the case studies’ details and facts (Biggs & Tang, 2011), leading to deep learning required for capabilities-based educational outcomes. The learning experience evaluation indicated that participants engaged in meaningful ways to explore the relevance and skills of the identified capabilities as they described applying their learning to their subsequent work practice. These findings indicate that the workshops had an ongoing impact and achieved the desired learning outcome.
The participants’ post-workshop experiences reflected the workshop objective’s intent, and andragogy, which sought to ensure the learning outcomes evolved beyond surface learning and retention of high-level information to deep learning involving interpretation and contextualisation, was realised. This would indicate a degree of effective learning change, as Biggs and Tang (2011, p. 23) argued, where the participant’s experience resulted in a level of ‘conceptual change, not just the acquisition of information’.
Critical elements in the education approach informed by the PLDF were particularly effective at achieving the learning objective and outcomes. The PD workshop’s learning and teaching activities, relating to its andragogy, were deliberately designed to engage participants in constructing knowledge and skills through immersion in a particular disaster case. Rather than learning about skills and knowledge separate from the workplace, knowledge was deliberately designed to engage professional engineers to construct knowledge as part of the social practice that is innately intertwined with work practices (Boud & Hager, 2012).
The role-play allowed participants to think about themselves as either members of the public (in the Dreamworld case study, thinking about their own past experiences visiting a theme park) or similar experiences, making time and cost-pressured decisions (such as those in the Challenger case study). The industry-based discussion cases highlighted the disconnect between decisions and consequences over time. Underpinned by Kolb’s Experiential Learning Cycle (Kolb, 1976), participants reflected and identified the gap in the professional practices of those involved and the outcomes and implications for their professional practice related to public safety.
Using cases helped participants learn and resonated with them after the workshop to support the application of learning to their professional practice. The cases provided the opportunity to emotionally connect with, empathise with, and relate to the people, decisions, and outcomes involved in the cases. As a result of the intent of the workshops, the cases were explored from a socio-emotional, experiential perspective rather than a technical perspective, which made the experience novel and supported this emotional connection. Personalising public safety through the case studies and highlighting the cumulative impact of decisions demonstrated the importance of seemingly inert decisions resulting in severe consequences.
The reflective practice central to this narrative-based reasoning, facilitated by experiential, inquiry-based learning situated in professional scenarios where past decision-making is critically interrogated to explore and foster alternative practices, places the learner in a double-loop learning experience, where the individual recognises conflicts and identifies fundamental changes to practice (Argyris & Schon, 1974). The use of experiential learning in existing disaster cases enables participants to identify the complex nature of practice and develop capability rather than simply competence. Reflecting the social, material, and situated nature of learning, ‘learning as becoming’ (Rooney et al., 2012, p. 118) was particularly important in the Australian pipeline engineering context where the excellent safety record means that engineers have limited opportunities to (re)construct their mental models, habits, and beliefs through active engagement and reflection on their own experiences with accident cases.
The workshop evaluation determined that participants displayed an increased awareness of the importance of public safety in their decision-making after attending the workshop. As such, the workshop’s design facilitated the successful development of the core learning objective. The findings indicate that the emphasis placed on the experience of working through the role-plays and situations (aligning with Kolb’s experiential learning theory) supported the exploration of safety and decision-making capabilities. As Kolb and Goldman (1973) describe, this allowed for translating experiences into ‘concepts which in turn are used as guides in the choice of new experience’ (Kolb & Goldman, 1973, p. 2). Turning the workshop experiences into concepts applied in professional decision-making highlights how experiential learning has supported learning outcomes and ongoing impact on work practices. Aligning with experiential learning theory, the learning experience extended beyond the training session, extending the boundaries of this ‘stand-alone’ learning experience to integrate successfully with ongoing practice.
More broadly, our findings demonstrate that PL should not be considered and implemented through the binaries of formal and informal learning, but be able to provide a ‘common currency’ and means to ‘navigate the set of orthodoxies that permeate education, training, and development’ (McCormack, 2000, p. 398). This reflects the evolving recognition of workplace learning opportunities, which has challenged how PL is understood and evaluated (Boud & Hager, 2012; Fenwick, 2009; Hager & Hodkinson, 2009).
Critiques of contemporary approaches to PD argue for a reconceptualisation that imbues a PL paradigm and leads to new approaches to PD. While the PLDF advocates for alternative approaches to PD, effective learning outcomes from PD will only be realised if designers actively engage with the framework, make informed decisions that best suit their specific needs and contexts, and adhere to the framework’s key requirements. The framework also includes an evaluation methodology based on Kirkpatrick’s theory to ensure these outcomes. Again, the framework does not prescribe specific evaluation methods, allowing developers to select those most appropriate for their goals. The success of the PLDF depends on its application, drawing on educational theory.
Conclusion
The findings presented in this paper highlight that the PLDF was effective in practice. Participants further developed or reconfirmed their existing decision-making processes as informed by the capabilities associated with the workshop. This is important because traditional approaches to PD have been critiqued as reductionist and limited in their ability to develop capabilities in work practice contexts. The PD workshop was designed to engage professional engineers in active, experiential, and inquiry-focused learning, which allowed them to apply identified capabilities to their professional practice. A PL philosophy informed the workshop’s education approach, focused on achieving deep learning and conceptual change, not just acquiring information. It utilised role-play, industry-based discussion cases, and emotional connection through case studies to support learning application to professional practice. The narrative-based reasoning and reflective practice facilitated by experiential, inquiry-based learning were central to the workshop’s success in fostering alternative practices for professional scenarios.
A key value of the PLDF in this engineering context was its alternative approach to traditional PD methods. However, these conventional methods dominate professional fields beyond engineering, and therefore, the PLDF can be applied in other professions. This paper showed that while the PLDF itself provides an alternative to traditional PD approaches, to achieve successful learning results from PD, designers must actively apply the framework and make informed choices that align with their individual needs and circumstances.
Footnotes
Acknowledgements
This work is funded by the Future Fuels CRC, supported through the Australian Government’s Cooperative Research Centres Program. The cash and in-kind support from the industry participants is gratefully acknowledged.
Author contributions
Sarah Holdsworth: Conceptualisation; methodology; investigation; formal analysis; writing – review and editing. Orana Sandri: Conceptualisation; methodology; investigation; formal analysis; writing – original draft. Jan Hayes: Funding acquisition; project administration; conceptualisation; methodology; investigation; writing – review and editing. Sarah Maslen: Conceptualisation; methodology; investigation; writing – review and editing.
Declaration of conflicting interest
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is funded by the Future Fuels CRC, supported through the Australian Government’s Cooperative Research Centres Program.
Ethical statement
Data Availability Statement
The dataset generated during the study is not publicly available as it contains proprietary information that cannot be released without permission from the funding body. Information on how to obtain it and reproduce the analysis is available from the corresponding author on request.