Supporting pedagogic innovators in professional practice through Applied eLearning

Abstract

This study explores the relationship between conceptions of innovation in eLearning pedagogy, the role of artefact-based learning in demonstrating this innovation, and how this can be investigated through critical incidents analysis of personal and collective learning. The context is an accredited masters programmes and the graduates' experience from 2007 to 2017. Graduates are a blend of academic staff in higher education, private sector trainers, and independent eLearning consultants wanting to develop knowledge and skills in eLearning. Key dimensions of pedagogic innovation explored are the continuum of how programme participants learn to innovate, what enables or prohibits them to innovate in their professional practice, and how they lead such innovations. Sixty-five graduates from the programme were invited to a survey and 10 of these who self-formed a LinkedIn community of practice after graduation engaged in a critical incident analysis on pedagogic innovation in their professional practice. As the participants are drawn from a range of academic subject disciplines, and share a cohort with eLearning professionals from the private and public sector, findings show the prohibitors and enablers within their disciplines and organisations in introducing and sustaining innovations to practice. Also shown is the importance of growing confidence in digital pedagogical practice, the power of collaborative cohorts, deconstructing innovative pedagogy for these contexts, and what supports pedagogic innovation. Findings have curriculum design and support implications for practitioners delivering eLearning professional development for academic staff and industry.

Introduction

This study explores the relationship between conceptions of innovation in eLearning pedagogy, the role of artefact-based learning in demonstrating this innovation, and how this can be investigated through critical incidents analysis of personal and collective learning (Figure 1).

Figure 1.

Scope of the study.

This study is based on an accredited two year part-time professional development programme entitled the Masters in Science (MSc) Applied eLearning in an Irish higher education institution. The focus is on how as a result of engagement on the programme, the graduates have integrated the eLearning artefacts in the form of pedagogic innovation into their professional practice.

Even in today’s fast-moving technologically oriented society, educators who are less familiar and less comfortable with technology than their colleagues and students still exist, and struggle to seamlessly integrate a growing list of technology tools into their regular curriculum and professional practice. Therefore, central to the programme team’s decision-making with regards to which tools to infuse to the programme for exploration and why, is an intention to support these educators and eLearning professionals to in turn make innovative use of technology tools for their own instruction and professional contexts, and subsequently to help their students improve their technology skills. An important question is how can we in the programme team effectively support these pedagogic innovators to grow and spread their interest for pedagogical innovation in Applied eLearning?

The next sections of the paper explain the context and rationale for this study, a review of the literature, and the process within the programme by which pedagogic innovation has been developed and supported.

Context and rationale

The MSc Applied eLearning has been running since 2007, and is a 90 ects ¹ Masters programme, with each credit equalling 20 learning hours. There is some flexibility in how participants might distribute their learning hours and the modules within the programme allow for blended and online learning in addition to face-to-face attendance in class. As much of the programme is practice-focused, participants can potentially view some of their professional practice and reflective practice as part of the learning undertaken. This MSc is part-time and has a modular structure. Participants need to achieve 45 ECTS in Year 1 to progress to Year 2 of the programme. The final 15 ECTS are awarded on successful completion of an ePortfolio at the end of Year 2, but formative assessment of the ePortfolio is undertaken in Year 1 to ensure that this will be feasible. Participants otherwise must complete the Core modules and one Elective module in Year 1, and then complete the artefact in a Project Module in Year 2 to achieve the final 30 ECTS for the award of MSc in Applied eLearning. Since 2007 there have been 65 graduates from the MSc; lecturers from a variety of subject disciplines from the host institution and other further and higher education institutions and private colleges; professionals from eLearning companies and training providers; individuals working in public agencies, charities, and other not-for-profit organisations; and individuals who are self-employed and setting up businesses as eLearning providers. Figure 2 shows graduate numbers from 2012 onwards, and from 2007 to 2012, there were 18 graduates.

Figure 2.

Completion of the MSc in Applied eLearning by year 2012–2017.

The breakdown of the cohort shown in Figure 3 indicates that more than two-thirds of the participants in the programme are external to the host institution. The highest rate of participation from within the host institution comes from the College of Sciences and Health. Although it is beyond the scope of this current study, it would be interesting to investigate in an extended piece of work what is meant by pedagogic innovation in eLearning across different disciplines such as Science, Arts, Business, and Engineering. For example, the Higher Education Academy (2014) in the United Kingdom have offered a number of professional development events to educators in Arts and Humanities to more fully comprehend the diverse and complex ways students develop their own ways of knowing and learning in these disciplines, and what innovations are happening in the field. In the United States, reports such as that by Raman (2016) show the latest trends of technology supported pedagogy in higher education, and argue that new technology provides the means through which new instructional models take hold and flourish.

Figure 3.

Analysis of MSc cohorts of participants.

Higher education institutions continue to invest a considerable amount of time and money in technology-enhanced learning strategies. Preparing academics to use learning tools and media in classrooms continues to be an exciting challenge for the academic development community, of which the programme team in this study are a part. Such instruction involves helping the participants understand the role of learning theory in the design and function of eLearning and in the selection and in the use of technologies for their practice. Theories that are explored on a ‘Learning Theories’ 5 ECTS module on the programme in relation to technology integration into practice are cognitivism, social learning, constructivism, and communities of practice. Findings from a study by Serdyukov (2017) show that the primary focus of educational innovations should be on teaching and learning theory and practice, and technology applications need a solid theoretical foundation based on purposeful, systemic research, and a sound pedagogy.

Research aim and objectives

The aim of the study was to explore the relationship between conceptions of innovation in eLearning pedagogy and the role of artefact-based learning in demonstrating this innovation in professional practice. It can also be useful for continued programme review and revision with a view to ascertaining what participants want and need so that they can continue to plan for eLearning more strategically and be better supported in the years ahead. This ongoing commitment to design and development of this programme can help promote future participant learning journeys, progress, and success in eLearning innovation.

The objectives were to investigate:

Participant perspectives and understanding of the concept and impact of pedagogic innovation within the MSc Applied eLearning in the context of their professional practice: from how they learn to innovate to how they can lead innovations.

Enabling and prohibiting factors of becoming pedagogic innovators for academics and other professionals who teach or support eLearning in higher education or in eLearning organisations.

Literature review

This section of the paper addresses what is meant by pedagogic innovation through the lens of Applied eLearning. In particular, the work on innovation in practice-based studies by Gherardi (2012) and Nicolini et al. (2012) gives insights to this context. Artefact-based learning and digital innovation has been informed by Ciriello et al.’s (2017) research.

Exploring the concept of pedagogic innovation in applied eLearning

The participants in this study are pedagogic innovators through Applied eLearning. As innovation is a broad term, it is important to interrogate it in some depth for the context of this study. The theoretical underpinning of pedagogic innovation in Applied eLearning is important to deconstruct. The Applied nature of eLearning in this study refers to how the programme participants have applied or harnessed the use of eLearning in their professional practice. When considering the aims of this study, it was useful to first explore the concept of innovation. For a number of decades, the essence of ‘[…] innovation seems to have had two subcomponents – it begins with the idea or item which is novel to a particular individual or group and, then there is the change which results from the adoption of the object or idea’ (Evans and Leppmann, 1970: 16). More recently, Brewer and Tierney (2012: 15) suggest that innovation is generally understood as ‘[…] the successful introduction of a new thing or method’. Arguably then, innovation requires three main steps: an idea, its implementation, and the outcome that results from the execution of the idea and produces a change.

O’Dwyer (2014) argues that ‘innovation’ is one of those words that, through casual overuse, has come to signify a wide array of distinct concepts – in some sense, the word is literally losing definition. In the Business Technology world, Dediu (2014) has explored some of the key conceptual distinctions between four categories of innovation, describing a hierarchy or taxonomy that contextualises each one. Dediu’s (2014) ‘Factors of Innovation’ are shown in Figure 4: ‘Novelty: Something new; Creation: Something new and valuable; Invention: Something new, having potential value through utility; Innovation: Something new and uniquely useful’. At its essence, creativity is the ability to generate novel and useful ideas, and innovation is the successful implementation of creative ideas. For this current study, the participants’ work environment (management practices, organisational motivation, and resources) can either promote or undermine innovation, so it was useful to explore the inhibiters and supports for pedagogic innovation in how they applied their learning from the programme to their practice.

Figure 4.

Factors of innovation (Dediu, 2014).

Walder (2014) argues that in its literal meaning, the term ‘innovation’ evokes, in its positive sense, an adjustment, improvement, development, study/pilot project, experiment, or even modernisation, reform or renewal and is often associated with pure science or technology and is frequently generalised to mean technological progress.

In the context of higher education, the term ‘innovative’ is often used when we speak about new pedagogical methods and creative educators who represent a deviation from traditional didactics. Innovation can appear as a new pedagogic theory, methodological approach, teaching technique, instructional tool, learning process, or institutional structure that, when implemented, produces a significant change in teaching and learning, which leads to better student learning. Therefore, pedagogic innovations are intended to raise productivity and efficiency of learning and/or improve learning quality. However, the appearance of new technologies and their use in the field of education cannot automatically be regarded as pedagogical innovation. Timmins (2018) discusses how the concept of innovation is everywhere in higher education, encapsulating research, entrepreneurship, technology, learning, teaching, and business improvement. That being so, she argues that the word innovation can be confusing, misleading, and misunderstood.

How does this masters programme support the innovative and experimental nature of the graduates’ practice in Applied eLearning? With the work of Knight (1997) aside, there has been limited research and few papers discussing the style and nature of Master’s level teaching in higher education (Kneale, 2015). There has been recent research exploring what innovation in pedagogy looks like (OECD, 2018). The work of Paniagua and Istance (2018) in this area argues that innovation at the level of practice must be seen as a normal response to addressing the daily challenges of a constantly changing classroom. Natural learning inclinations such as play, creativity, and inquiry are key, and innovative pedagogies should consciously promote the engagement of learners. While their work provides a helpful but generic view of the classroom, for this current study, the discussion of pedagogic innovation in applying eLearning to practice needs to move beyond this to consider what Ciriello et al. (2017) call a practice-based model of digital innovation. They argue that facilitating digital innovation requires a deep understanding of the actual practices that are carried out by innovating people with the help of artefacts. Although set in the context of software companies, their study on the use of artefacts and their different roles in the underlying innovation practices can provide rich insights into digital innovation from a practice perspective. Like Dediu’s factors of innovation, they present a model of interrelated digital innovative practices of making sense of an idea, aligning mental models, negotiating solution paths, and crafting an idea.

Previously, Salmon in 2016 also discussed pedagogic innovation in relation to eLearning, and argues that achieving innovation is a demanding endeavour that will not be achieved by learning technologies alone. Delving deeper into how innovation is understood, arguably it is a response to constantly changing classroom; in terms of what is driving these changes, Lawton et al. (2013) argue that people, not technology, will drive educational change. Similarly, Educause (2018) cite the demands of a more diverse, tech-savvy population and the need to evaluate tech-based instructional innovations (looking for and assessing tools to gather and analyse evidence of learning effectiveness and use the results to adjust curriculum development) as driving HE’s approach to teaching and learning. While demography, economy, technology, and the world of work can be summarised as the four broad drivers of change in higher education, Charlesworth (2018) suggests that for educators to embrace change, there are other factors that need to go along with the interrelated enablers originally highlighted by Jasinski (2007). To a work culture that embraces and supports innovation, a robust technology infrastructure, technology tools that are appropriate for teaching and learning purposes, a senior champion who drives the process, a willingness to consult and share, and supportive managers, peers, and support professionals, he adds the elements of time allowed, valorisation of effort, support, and recognition from all levels.

In the context of this current work, innovative pedagogy is the study of what is involved in being an innovative educator in applying eLearning to professional practice. Innovation on the MSc programme generally refers to the creation of better or more effective artefacts, processes, technologies, or ideas. Specific assessment criteria require that to be a successful graduate of the programme, you need to demonstrate the required knowledge and skills of an innovative eLearning practitioner in a higher education or industry context. The artefact(s) produced by the graduates need to demonstrate a pedagogically sound justification having been made for its design and use of the selected technologies; this may include appropriate, previously untried or uncharted uses of eLearning technologies to support student learning within a specified tertiary education or industry context. The artefacts produced are assessed as part of the demonstration of participant knowledge and skills on the programme. The next section discusses why the creation of artefacts is used in this way, and generally what types of artefacts are produced.

Artefact-based Learning as a form of pedagogic innovation

This section discusses the relationship between pedagogic innovation and artefacts on the MSc programme. The term artefact in this study refers to any kind of material object that innovators create and/or use in practice. An important dimension of the MSc programme is the concept and practice of an ‘artefact’ to explore pedagogic innovation. Therefore, it is useful to explore what role artefacts play in pedagogical innovation as a result of participant engagement on the programme. Tuomi (2002) states that the appropriate level of analysis to capture the complexity of digital innovation is at the level of practice. Building on this, Carlile et al. (2013) suggest that a fundamental starting point to understand practices is to study the use of artefacts. For the creation and use of new IT artefacts, Ciriello et al. (2017) suggest studying the artefacts that innovators use to form, evolve, and add to a shared innovation agenda within organisations.

Insights will be shared on the evolvement of the artefact from a Professional Development module at the outset of the programme to the design, production, and research of one for their practice in Year 2. We begin with the participants choosing an artefact that has influenced them in their professional development. This could take the form of an article, book, poem, video or hardware or software or other object that has helped them to reflect and/or develop their skills as a professional in their eLearning practice. Table 1 shows a sample of artefacts, how they were contextualised by the participants who developed them, and what practice they materialised.

Table 1.

Sample of eLearning artefacts produced on the MSc programme.

2012–13 cohorts	Digital Repositories artefact and Collaborative Resource Development in a Veterinary Community	Veterinary Science
	Artefact on Learners participating in Asynchronous Online Courses	Technology Education
	Health care workers artefact on online educational drama	Heath Care Sector
	Web ecology artefact	Voluntary Education
	Social media artefact for local cultural heritage discovery	National Library
	Podcasts artefact to support Communication Skills Development for Postgraduate Research Students	Science and Engineering
	Instructional Videocast artefact for students in Bricklaying	Apprentice Education
	Shared Social Video Content artefact in a Business Programme	Multimedia in Business
	eLearning artefact used to support group learning in a problem-based learning module	Engineering
	Induction training artefact in a corporate environment	Multinational company
2013–14 cohort	Pre-lecture resource artefact to reduce in-class cognitive load	Chemistry
	Wiki artefact as a collaborative knowledge creation tool	Instructional Design
	Facebook artefact as a learning space	English
	Online first year computer literacy artefact	IT Training
	Student Architectural Technologist Learning Artefact	School of Architecture
	Mobile Learning artefact to support learning and teaching	Econometrics
	Rapid Authoring Tool Artefact	Voluntary Education
2014–15 cohort	Multimedia resource artefact for Business Law Programmes	School of Law
	mLearning artefact on self-direction and engagement in theory	Apprentice Education
	Bespoke mLearning artefact on Student Motivation in Refrigeration	Apprentice Education
	Cognitive learning artefact on interactive multimedia eBooks	Secondary School
	Online Group Reflection artefact on video collaboration	Built Environment
	Scratch coding artefact on 21st century skills	Primary School Sector
	Digital video-based knowledge artefact: students as producers	Biochemistry
	Digital Storytelling artefact on Technology Integration	Primary School Sector
	Moodle artefact for secondary schools	Secondary Education
	Augmented Reality artefact in digital modelling for the architectural design studio	School of Architecture
	Wiki artefact to support Collaborative Learning	School of Marketing
	Professional development training artefact in technology-enhanced learning	Technology Education Centre
	eLearning artefact on Wood Machining Regulations	Apprentice Education
2015–16 cohort	Virtual Classroom artefact in the workplace	Insurance Company
	Online Journaling artefact for Erasmus student’s study visits abroad	School of Languages
	Accounting artefact on learner cognitive load	School of Accounting
	21st Century artefact on online course design	Technology Education

Participants are asked to identify and explain the personal and/or professional meaning and relevance of their artefact, and consider why they have chosen it as a trigger to discuss their professional development with others. We move then to consider artefacts and their role in research and knowledge creation. The subsequent eLearning artefact produced as a culmination of the work in the two year programme can take one of three forms: an interactive learning system, a generic learning tool, or a learning object. Each of these is contextualised for the professional practice of the participant.

To support pedagogic innovation, a variety of teaching and learning strategies are used in this programme to support the development of artefacts:

Crossover learning: In the weekly workshops, learning on the MSc programme is enriched by participants’ sharing innovative experiences from their everyday practice in group activities; informal learning is then deepened by adding questions and knowledge from the classroom. These connected experiences spark further interest and motivation to learn, and can provide participants with authentic and engaging opportunities to consider what artefacts to develop. Nicolini et al.’s (2012) theoretical framework for studying artefact usage in collaborative practices will be useful for future analysis of the artefact-mediated practices of the participants who are keen to continue innovating their digital practice. Currently, the ePortfolio is a useful mechanism for drawing on these across multiple settings and support participants in recording, linking, recalling, and sharing their diverse learning events.

Learning through argumentation: Participants can advance their understanding of eLearning through argumentation, which can help them explore contrasting ideas, which in turn can deepen their learning; by making their reasoning public like this, all have a chance to learn more about specific topics in which they are interested in developing their artefacts. It also allows them to refine ideas with others in the cohort, so they learn how to work together to establish or refute claims. The Programme tutors can spark meaningful discussion by encouraging participants to ask open-ended questions, re-state remarks in more technical language if required, and develop and use models to construct explanations for their artefacts. By doing this, participants learn how to take turns, listen actively, and respond constructively to others in artefact development.

Peer knowledge construction: This takes place through peer learning and feedback, guest tutors and speakers from other higher education institutions, and eLearning external organisations, as well as inputs from programme graduates where appropriate in specific modules. A community of learners like this is integral to the learning experience of the programme participants. Gherardi (2012) suggests that in the practice of a community, innovation emerges from collective action, and also as a process of comparison of participants’ perspectives resulting from their continuous discussion and negotiation.

Figure 5 shows the different modules on the programme (core modules are in dark blue and elective modules in light blue) and the range of Applied eLearning artefacts that the participants produce across the two years. Four core modules are completed in the first year of the programme, and participants select one from the choice of electives. The second year of the programme is devoted completely to undertaking an individual Applied eLearning project, which needs to meet specific innovative pedagogy criteria.

Figure 5.

Modules and the artefact-based learning approach of the MSc programme.

An ePortfolio is maintained by all participants across the two years to store all the artefacts developed. An initiative was introduced to the programme – an award to recognise the innovation in the participant cohort, and celebrate the effort that they have invested in their ePortfolio design and development. An awards ceremony is held in class and five categories have been developed to allow everyone to showcase their academic and personal accomplishments, creativity, and individuality (Organisation and Navigation; Range of Technologies and Media; Reflective Writing (with scholarship); Best Peer Support). The programme team comprises the panel choosing the winner in each category and there is a chance for the students to also pick their winner in the People’s Choice Award. The decision on the overall winner is based on student performance in each category. Integral to this is the students’ sharing of their ePortfolios among each other to comment on their peer’s ePortfolios and to help them make their own decision about who they would like to nominate for the People’s Choice Award.

In order to cultivate the climate for innovation on the MSc programme, time and effort was invested by the instructional team to develop a support model, which had a pedagogical underpinning, theoretical approaches, and practical strategies, which had a combined output of the artefact-based ePortfolio for all participants, shown in Figure 6.

Participants allowed to try new ideas out and given permission to fail.

Continual reinforcement, refreshment, and opportunities for informal discussion are provided.

Participant opinions respected, voice given to each student, openness and tolerance of peers.

Foster comfort and familiarity in the participant group.

Reflection enables standing away and thinking from a different angle: using smartphones and Voki to record quick descriptive reflections.

Helping participants to be adaptable, innovative, to solve problems and communicate well with peers.

Participants enabled to work on small-scale eLearning projects that motivate them intrinsically.

Participants who are self-regulated learners collaborate with other participants in exchanging ideas (CoP), eliciting assistance when needed, and providing support to their peers.

Participants can see the connection between their efforts and learning success.

Figure 6.

Support model for Pedagogic Innovation on a Masters programme.

Figure 6 is a Support Model for Pedagogic Innovation on a Masters Programme. It is based on an examination of pedagogic innovation as a theoretical approach for the programme.

Research design

Analyses of written narratives of critical incidents have been used to understand how the participants on the MSc programme have developed what they consider to be innovative pedagogic practice via eLearning artefacts. These have been complemented with participant perspectives from an online survey.

Sharples et al. (2016) argue that rather than relying solely on controlled experiments to evaluate new pedagogies, research is now piecing together evidence from multiple sources, rather like pieces of a jigsaw puzzle, to build up a picture of effective methods of teaching, learning, and assessment. For this current study, an interpretivist approach drawing on qualitative methods is appropriate to the stance of the programme team as researchers, but for ethical reasons participants were self-selecting, and data are self-reported. The data reflect the participants’ perceptions of the concept, impact, enabling and prohibiting factors of the MSc on their pedagogic innovative practice.

Participants and processes

All 65 internal and external alumni of the MSc were contacted via email to invite their participation, with 42 of them fully completing the online survey (22 from external organisations and 20 academic staff). A Participant Information Sheet and Informed Consent were administered using the online survey system, prior to their engagement with the survey, and an informed consent was completed by all who participated. It was indicated that their participation in the survey was an opportunity to contribute to a body of research and analysis to advance innovative practices around eLearning pedagogic practice in Ireland. The statement of informed content was approved by the institutional Research Ethics Committee. The survey was piloted with a small number of programme graduates for its ability to gather the information required to answer the research questions.

Data collection

A qualitative methodology was used for this study with two forms of data gathered for measuring pedagogic innovation and impact – online survey data from programme graduates from 2007 to 2017, and rich data gleaned from critical incident analysis – so that the outputs could be used to effectively inform what the programme team can deliver in the future. Figure 7 shows the data collected and outlines ‘Exploring Pedagogic Innovation’ for this study; it is based on definitions, examination of current literature, and interrogation of concepts of pedagogic innovation provided earlier.

Figure 7.

Exploring Pedagogic Innovation for this study.

Most of the academic survey respondents were working in Institutes of Technology, ² and there was distribution of responses across subject disciplines. A higher proportion of the respondents from academia were at an earlier point in their teaching careers, teaching for less than five years. For the respondents working in eLearning industry, length of time delivering instruction was more varied. The focus of the survey was to explore participant conceptions and practice of pedagogic innovation in eLearning; survey questions were formed from the literature and they were designed to allow participants to share how their learning/experiences from the MSc programme helped them form conceptions of pedagogic innovation in the context of their practice. This was also an opportunity to highlight what impact their involvement with the programme has had on their role and practice. It involved sharing their experiences of the programme in relation to the impact on practice. (A sample of survey questions is shown in Table 2.)

Table 2.

Sample survey questions.

Pedagogic innovation

What do you understand under the term ‘pedagogic innovator’ in higher education/eLearning organisations? What do you think are the key characteristics of pedagogic innovators? Do you see yourself as a Pedagogic Innovator as a result of engagement on the MSc? Do others see you in this role? What role do enablers play for being a pedagogic innovator? What are the barriers for pedagogic innovators?

Programme impact on practice

What did you learn that will change how you undertake your role? Has your eLearning practice improved as a result of being on the programme? Give one example of how you have applied what you learned on the programme back in your practice. How has the learning from the programme helped your deliver your eLearning aims and objectives? Have you seen change in your own students’/user’s learning as a result of your changed practice – what evidence do you have for this? Since completion of the Programme, have you been involved in participating or leading in pedagogic change in your department/organisation? Have more educational/professional opportunities been available to you?

A key question on pedagogic innovation centred on how valuable are the following to each participant in making pedagogic innovation happen. A five-point Likert scale from extremely valuable–valuable–not sure–reduced value–not valuable at all was used to explore the following attributes of pedagogic innovation: creativity, playfulness, experience, part of a network, collaboration, institutional support, culture, resilience, courage, cautiousness, proactivity, determination, use of technology, spotting opportunities, obstacles, time, space, people, risk, ideas, experimentation.

A month later, the survey was followed by opportunity to complete a critical incident analysis. These graduates of the MSc, from 2007 to 2017 who had formed a LinkedIn Learning Community were then contacted and invited to undertake a critical incident analysis; 10 participants submitted this. Notwithstanding the small size of this group, each participant came from a separate department or school within his/her HEI or organisation. As participation was on a voluntary basis, it was not possible to guarantee attendance from all HEIs.

Critical incident analysis was used to capture the participants’ experiences of the pedagogic innovation in artefacts that they developed on the programme and their considerations of how this also impacted their subsequent professional practice. Despite a lack of consensus in the literature as to the precise defining characteristics of a critical incident, Tripp’s (1993: 24–25) definition works effectively as a contextualising framework. He defines critical incidents as:

straightforward accounts of very commonplace events that occur in routine professional practice which are critical in the rather different sense that they are indicative of underlying trends, motives and structures. These incidents appear to be ‘typical’ rather than ‘critical’ at first sight, but are rendered critical through analysis.

Critical incidents have been utilised as a learning tool in many professional disciplines including education (Kuit et al., 2001). The learning in this context centres on the reflective analysis facilitated via selection and consideration of a critical incident. Similarly, in the present study, participants were guided to reflect on key learning points encountered during the process of developing their artefacts on the modules. The critical incident technique in this context was chosen for its strength in providing ‘a systematic means for gathering the significances others attach to events, analysing the emerging patterns, and laying out tentative conclusions’ (Kain, 2004: 85).

Past inquiries involving participant self-reporting on programmes have used instruments such as multiple-choice surveys and rating scales almost exclusively, thus restricting the extent to which graduate voices have been heard. This study sought to begin to redress these gaps by also using critical incident analysis. This critical incident approach, which has its origins in Psychology (Flanagan, 1954), is used widely in Health Science education (Mahajan, 2010) and increasingly in teacher education contexts and was chosen for this study because of its value in encouraging participants to think more deeply about the impact of their pedagogic innovations on and as a result of the MSc programme – they were asked to consider an incident which in some way had a significant impact on the participants’ personal and professional approach to their pedagogic innovation of eLearning. How it differs from previous instruments used to explore the topic is that it provides an approach to structuring the process of reflexivity within this practice learning context, and is useful for developing reflexive writing (Green Lister and Crisp, 2007), which can get to the crux of exploring innovation to practice.

Participants were asked to describe a critical incident or event through producing a written reflection – something significant from their programme experience of developing eLearning artefacts – from which they could extract in-depth learning. This written reflective critical incident was gathered after the online survey, and probed for an explanation and consideration of impact of the critical incident. Critical incident analysis lends itself to research that seeks context-rich, first-hand perspectives on experience. Broader patterns and themes can be discerned, which can in turn illuminate ‘shared reality’ (Kain, 2004: 82).

Critical incident narrative

Discuss something significant from your programme experience of developing eLearning artefacts that you used in your practice; this may have been something that you learned on the programme that surprised or challenged you.

As part of the ethical dimension to this study, all participants’ permissions were obtained through statements of informed consent. Specific permission was acquired for using any extracts from writing the critical incident. Data collection regarding critical incidents can be undertaken in many ways: through personal interviews, focus group interviews, and direct or participatory observation. As previously stated for this study, written reflective critical incidents were used along with an open-ended online survey, gathering retrospective data. A framework for analysis by Edvardsson and Roos (2001) was considered which focused on the three main areas of cause, course, and result of the critical incident. Another possibility was by Nygren and Blom (2001) who describe three types of codes which they applied to each incident which students in their sample had written about. These related to what happened, who or what was involved, and what reflection was occurring. However, a decision was taken to use the approach advocated by Angelides (2001) who argued that an analysis of critical incidents can be used by researchers interested in collecting qualitative data quickly as a method for doing a case study. Further, the fact that this approach can be used in a participatory way to contribute to understandings about the impact of pedagogic innovation for the purpose of improvement was why it was chosen.

The use of the critical incident analysis is a distinctive element of the evaluation process. The critical incident accounts were analysed by the MSc programme co-ordinator at the time. The analysis was carried out by first reading through the written reflections several times, and then identifying the content or themes represented by clusters of incidents and conducting ‘retranslation’, during which the incidents were sorted into content dimensions or categories. These steps helped to identify incidents that were judged to represent dimensions of the pedagogic innovative behaviour being considered, and recurring themes emerged from the analysis of the critical incidents.

While a critical incident is generally something which can be interpreted as a problem or a challenge in a particular context, rather than a routine occurrence, in this study it was to be something that stood out for the participant; examples included a successful or unsuccessful eLearning artefact development incident, on which they could reflect. Other examples included conversations with peers in the class, a moment when they felt they were beginning to make progress with their eLearning artefacts, or a motivational strategy they found useful for progressing their innovations.

Translating critical incidents into innovative practice

Two examples of critical incidents translated from participants’ programme learning to their practice are shown in Figures 8 and 9. Identifying incidents of how participants self-reported developing as pedagogic innovators through their work on their eLearning artefacts was insightful. They show how an experience from Participant F and then Participant C on the programme (with a brief quotation) ended up translating into a pedagogic innovation for them. It involved exploring how participants applied what they learned in the programme to their own practice. However, it is acknowledged that there is no direct evidence of whether those disclosed beliefs actually were implemented in different practices and to what effect; this is a potential route for future research in a later study.

Figure 8.

Participant F translating critical incident to innovative practice.

Figure 9.

Participant C translating critical incident to innovative practice.

Figure 10.

Findings of the study on pedagogic innovation in applying eLearning.

Discussion of findings

There were five main findings from the study, shown in Figure 10, which are discussed below.

The survey data showed that all MSc alumni had a high degree of satisfaction with the programme (42), and they reported benefits for their departments and organisations and for their students, with 24 indicating they were continuing with future studying in the field. Some of the distinct professional roles that experienced the programme were Lecturer, Academic/Education developer, Director of Quality, Education/Learning Technologist, eLearning Project Manager, Librarian, Programme Manager, Senior Administrator, Director of T&L Unit.

Growing confidence in digital pedagogical practice

There were a number of commonalities in the participant’s motivation for engaging on the programme, with a desire to be more confident & competent in my job (33) featuring strongly. Also highlighted was a need to update skills & knowledge (25), wanting a teaching qualification in current area of work (15), and interested in pedagogic practice and research (15).

The reflections that participants made through writing the critical incidents allowed them to express their initial uncertainty about their ability. Indeed Gazin (2003: 32) reports that critical incidents may offer a clear ‘mechanism for self-reflection’. Certainly on the programme, the critical incidents triggered the expression of thoughts and emotions for several of the participants. As such, it is important for the programme team to take into account the lack of confidence that some participants initially have about applying innovative eLearning to their practice when entering the programme. Given the wide diversity at the start of the programme both in experience and confidence, support needs to be targeted at individual level (1–1 if needed), and involve scoping their starting points with regards to their digital skills.

Growing confidence in eLearning skills emerged as a finding. Thirty-nine indicated that they observed a positive impact on their practice from engagement with the Programme, and a collation of participant responses reflect the sense of risk-taking and self-belief:

I gained more confidence about the field and am not afraid to try new things; I am more likely to try new delivery methods and resources; I am more confident and structured and now can see things from both student and teacher perspectives; I feel confident to use more eLearning resources; Am much more at ease integrating technology to my practice; I have more confidence, more autonomy, and believe more in what I teach; The programme provided a platform for myself to grow within my organisation along with getting a new set of skills.

Survey responses also show how an increase in confidence has manifested itself in a change to participant delivery practice with deeper knowledge of pedagogy underpinning their confidence:

Now have a better understanding of constructively aligning LO and assessments; Take a scholarly approach to teaching; Have adopted new interactive delivery strategies; Am now better at eLearning; Progressed my practice as I am more reflective and can stand back from teaching and be more student led; Have a deeper knowledge of pedagogy; Changed my assessments; Am far more interactive and discursive now; previously I was inadvertently knowledge-transmissive in my delivery; Am now better at online interaction and giving choice to my students; Have greater teaching knowledge and know how to develop good resources; I use flipped classroom approaches and student engagement strategies; I introduced more current learning technologies to teaching; I now use eLearning tools outside of the classroom; Am more confident about structured integrating of digital activities to my teaching; The programme was a way for me to be progressive in my approach to technologies; As a librarian teaching, I did not previously have the pedagogical theory to design activities.

From the critical incident narratives, in terms of developing digital skills, there are some useful areas to consider: the digital skills and usage of something specific like VLE baseline requirements, how to use a specific tool, the more general digital literacy statements and skills, and pedagogic usage and capability (of often the same technologies). The programme team can continue to explore a number of useful existing resources to support the participants in building their confidence levels with these digital tools. JISC’s (2018a) student digital experience tracker is a short survey to gather students’ expectations and experiences of technology, and their digital capability framework (2018b) and guidance on leadership, pedagogy, and efficiency can support ICT proficiency in digital creation, problem-solving and innovation, digital identity and well-being, digital communication, collaboration and participation, among others. JISC’s (2018c) HE Teacher profile which is mapped to the UK’S Professional Standards Framework can be useful for demonstrating how new areas of practice are emerging, and how individuals might use their digital skills in different areas of their designated roles. In the context of Irish higher education, the All-Aboard initiative (2017) has contextual resources that will suit the nature of the work on the programme.

From the narratives, it was also possible to glean suggestions for main areas of focus based on practice. All HEIs represented on the programme require use of a VLE for all courses, but usage and skills varies widely among academic staff, as it does when using digital resources from the library – this is reflected in studies for the Irish HE sector by the National Forum (2015) and Farrelly et al. (2018). Flipped and blended classrooms are essential to models other than knowledge transfer, as demonstrated by Gerstein (2012). Mobile learning is a tsunami, not just a wave, and needs appropriate attention and support, as Jagannathan (2018) also advocates. Personalised learning is facilitated through mobile learning, but teachers and students need awareness and skill-building regarding best practice; a study by Lin and Kim (2013) outlines how this can happen.

Power of the collaborative cohort: Peers supporting each other’s learning

An important theme to emerge from the critical incidents was the benefit of working as part of a collaborative class cohort and interacting with peers during the programme:

The programme had very interesting content, and good support from staff when it came to developing my eLearning artefact…But what gave me a new lease of life in my eLearning practice and provided me with a channel for thinking and discussing my artefact content was the excellent peer learning atmosphere. Our discussions in each module challenged my knowledge and awareness of new technologies which I feel has enhanced my critical thinking skills, and resulted in huge learning in how I integrate technology to my practice…on one particular occasion I was really struggling with the focus of my artefact and it was the feedback from my cohort that bolstered my confidence on what I was trying to achieve and helped me back on track - I attribute my success on the programme to the great peer contact opportunities across the two years.

Peers in the cohort providing continual feedback on each other’s eLearning artefacts were seen as a strength of the programme delivery approach:

I found the listening to my peers’ experiences with eLearning (both good and bad) very insightful and when they shared lessons learnt when things went wrong for them, this helped me build up my own digital skills, knowledge & confidence. It also ‘forced’ me to engage with the wider literature, as I would hear about what they were reading and I would try to pick up on this too.

The tools and media that peers on the programme were using gave me both wide and specific learning across the modules and hearing from a variety of expert speakers in their own contexts was useful as I felt it really benefited my work and helped me to grow as a lecturer.

While I am not in currently in a teaching role, listening to the different workplace perspectives of my peers better prepared me for working in higher education generally.

Formative feedback is crucial for the early establishment of participant engagement in any programme, and particularly in continuing professional development programmes where participants are busy professionals accommodating their studies within a busy schedule. Like Fransen (2011), we sought to enhance the quality of the peer feedback among the participants and also their experience of receiving feedback.

Deconstructing innovative pedagogy – and how is it evidenced?

Twenty-seven participants indicated that innovative pedagogy for them meant having a range of new ideas for delivering instruction; 22 responses said it meant being up-to-date in current digital educational thinking; 12 said it was about having good contacts with others involved in eLearning; 23 indicated it was about improved digital skills and better use of technology; 34 said it needed an ability to reflect on delivery practice; 32 said it involved having a student perspective on different interactive instructional methods; and 25 highlighted the risk-taking mindset that they consider important: ‘I am now much more experimental, use more methods and technology appropriately; I now have a willingness to explore new technologies that facilitate interaction’.

There have been a number of insightful studies exploring what makes pedagogy innovative. Hannan and Silver (2000) defined and scrutinised the pedagogical innovations existing in higher education and how frequently they were used. They discerned 11 types of pedagogical innovation: computer usage; personal communication and problem-solving skills; team projects, and collaborative and co-operative learning; student oral presentations; interactive lectures and tutorials; work-based learning; problem-based learning; educational resource-based learning; open and distance learning; peer mentoring or assessment; others (student-directed learning, logbooks, portfolios, reflexive practice). Several of these have been highlighted by MSc participants as examples of how their learning from the programme was translated to their subsequent practice. More recently, Ryan and Tilbury (2013) proposed new pedagogical ideas for the future of an increasingly ‘flexible’ higher education provision: learner empowerment, future-facing education, transformative capabilities, crossing boundaries, and social learning. Walder’s (2014) study reveals seven distinctive pedagogical innovation categories, spread across social (support schemes, professionalisation, concept of teaching, interdisciplinarity, and interculturality) and technological innovations (tools, pedagogical approaches). Academic transformation in the form of breakthrough teaching and learning models, innovative partnerships and strategic transformation of an institution’s mission is the key opportunity identified from the ELI survey with more than 900 HE community members for 2018–19 (Educause, 2018). This links in with the 21 participants in this study who indicated an impact on the Strategic Aims of their Institution/Organisation after completing the programme:

There are now cohorts of more informed/digitally skilled lecturers; It has created a community of digital research informed 'teachers' and this can only be good for the institute’s strategic vision; It will influence the strategic aim of our company as we will invest in digital training; I have now an input to our strategy and leading the department to achieve these aims; An eLearning department was created, and I have been working closely with them; There has been further adoption/knowledge of flexible modes of learning and delivery, in particular; Our colleagues are definitely developing even more skills around technology and its uses, and there are plans to look into new digital methods of teaching and training.

Thirty-one participants indicated a direct impact observed on student/user engagement from using these innovative pedagogical approaches:

Quality of my lectures were improved by student engagement and technology; I now have a less formal classroom; My students have active engagement with eLearning content; There is more focus on technology in my classes; I completed the MSc in 2013, and can see the impact in novel methods for students; Increased awareness of my practice and underpinning theory has led me to design more interactive classes; My students like the new technology-focused assessments.

Forty participants responded that a change in their current professional role occurred following their application of innovative pedagogies to their practice (five said no); a sample shows the self-reporting evidence of role change:

I am now working in my chosen field and the programme has opened many new doors for me; I am now in a digital literacy role; The MSc led me to a learning technologist role; It gave me an excellent grounding in pedagogy, exceeded my expectations and has influenced a change in my practice and role; I am now playing leading role in introduction of ePortfolios to my Department; I previously worked freelance as an instructional design consultant, now have a full-time post in digital education, so instead of designing and developing eLearning courses myself, I now teach and support academic staff in enhancing and developing blended learning approaches in their own programmes; I took on more eLearning projects and eventually got a new job as a result of the MSc; I have been promoted to a leadership and management position; Successfully changed jobs and organizations as a result of completing the programme, and have moved to a more senior position; My career is now focused in higher education, as opposed to industry; Have responsibility now for eLearning in literacy and numeracy; The MSc was the most rewarding experience of my academic career and is responsible for me being able to move into and thrive as a learning technologist.

There was some discussion of how their own professional identity has changed:

I see myself more as an educator now, rather than in a supporting role, so this has opened up possible opportunities for me; I feel my role as a leader has been strengthened by the team work used during the MSc.

Some also discussed how others in their organisation perceive them now:

I have credible knowledge & skills in digital education; I have been encouraged to contribute to great educational debates in our School; I have taken leadership roles and have been asked to develop a research stream in the Pedagogical Education space; The programme has helped to raise my profile within the department and externally; There are now a lot more potential opportunities in my new Organisation; I have been promoted to a leadership and management position within the department - one year after graduation; I see a huge change on how I am viewed in my role, as I have published my research which led to conference speaking and change of position.

Twenty-six indicated leading on digital projects/initiatives that have resulted in a change in practice:

Have led the development of a number of new online (Masters) courses since graduation and currently also working with staff to pilot new audio feedback approaches; I led the development of the final year project process, embedded research informed teaching practices at a School level, influenced College Policy (around common teaching software), and championed technology enhanced learning; I led the rolling out of E-portfolios in Transition Year, and how they are assessed; I now support my colleagues in digital programme development and have been able to influence the new org with some of the concepts learned on the MSc; I have fronted the work on a digital international project for private companies; I have managed many internal and nationally funded projects and initiatives – all involving harnessing technology to improve the student learning experience;

Have co-ordinated the development of a Masters in Digital Entrepreneurship; Explored Digilanguages and frenchgrammartour.com and have integrated the outputs of both into department teaching.

One participant was candid in not claiming that the programme was the sole influence for change:

I have led on numerous projects, too many to mention but difficult to link any changes in practice directly back to the programme as other experiences before and after have been influential also. However, it is fair to say that the entire experience of the programme has been influential on my career.

What is needed to support pedagogic innovation?

A strong finding was the instructors in the programme team as modellers of pedagogic innovation: The tutors being role models, leading by example. Pinar’s (2012) work on curriculum is interesting as he argues that teachers should be ‘confirmed not as facilitators of learning but as individuated communicants in a complicated conversation that is informed by academic knowledge, subjectivity and the historical moment’ (25–26). Therefore, teachers and students together are communicants in this ‘complicated conversation’. This study looked at how programme participants individually and collectively innovate with the eLearning artefacts they develop. As Ciriello et al. (2017) posit, in theorising the innovation process, the prevailing view in the existing innovation literature is a discrete, linear, and sequential innovation process with clearly ordered, differentiated, and consecutive phases. However, it is important to support innovating participants in awareness that the appropriateness of the use of artefacts depends on the underlying practice.

It is also important for the programme team to have a renewed focus on supporting staff’s digital skills/literacy and media creation. Having an essential (and desirable) list of key skills knowledge regarding technology use that we want to ensure every participant going through the programme can demonstrate by the end. The MSc programme team want to ensure that no academic staff or consultants leaving the programme should still be in the ‘I don’t know/am not confident how to do that/never done it’ type category anymore; whether they choose to do so afterwards is their own pedagogical and professional choice.

Areas that the findings highlighted as useful for supporting current and past participants in pursuing innovation in their practice are:

Supporting current participants	Provision of new ideas for technology-enhanced learning with more choice for content delivery
	Offering practical workshops on specific digital topics are useful to fill gaps in our own skillsets when it comes to teaching
Supporting graduates’ community of practice	Organise seminars to bring back past students to attend that are relevant to the current programme
	Providing opportunities to collaborate on research initiatives
Supporting institutions	Promote further collaboration with other institutions
	Keep promoting the message of pedagogic innovation across the institution
	Important to champion/promote digital pedagogic innovation nationally

Enabling and prohibiting factors of becoming pedagogic innovators

Senior et al. (2018) put forward the view that educators should drive only innovation that has proven to be effective – it is important to not succumb to the need for what they call ‘conspicuous consumption’. Enablers and prohibitors were explored from the perspectives of academics and other professionals who teach or support learning. Key enabling themes identified for change to practice by participants are:

Staff Professional Development	Building staff capacity through professional development (workshops, seminars, CPD) and recognition of the changing role of teaching
	Encouraging new staff to undertake the MSc programme
	Excellent staff who are constantly working towards change
	Conversations with colleagues, opportunities/incentives to participate in high quality CPD in-house
Institutional Collaboration	A key enabler for me has been working with peers from other schools and HEAs
	Within the institution, a strong community, a vision that is achievable, value placed on excellence in teaching, a research informed approach, promotional opportunities
	National Forum projects and funding
Culture of Pedagogic Innovation	Learning from best practice
	Provision of valid reasons and backup information for making changes
	Thinking outside the box, changing the way things are done, involving more industry with academic staff
	Success stories, innovation ambassadors, strong innovatory leadership
Management Support	Management being willing to value the time and effort that goes into pedagogic innovation and furthering our own professional skills
	Management supporting and funding T&L initiatives
	Support of senior management, clear indication of what actions/activities are desirable and supported
	Having my directorate support staff with time to attend courses
	HR recruitment policies that make CPD and technological disposition a prerequisite for all new staff
	Enabling policies; enhancement funding; strategic policies; stakeholder buy-in
	Defined time for innovation as part of for programme review and development
	Appropriate leadership and long term goals and strategies
	Management driving motivation for change for the better, not necessarily for the sake of change; without this motivation then resources will not be allocated to staff to facilitate best practice innovation deployment

The barriers to the implementation of pedagogic innovation in eLearning have been identified as:

Time and Bureaucracy	Time and space; workload pressure of dealing with range of current projects
	Lack of time to invest in CPD, lack of knowledge of the benefits of innovative CPD
	No time to research teaching, increased student numbers and administration
	Bureaucracy (module/assessment changes take up to a year to be approved)
Is Teaching Valued?	Continuous, heavy teaching/workloads preclude time for reflection or innovation
	Curriculum restrictions. Staff shortages; lack of resources
	Over-stretched academic staff with large class sizes; promotion system rewards research outputs; under-funding of HE system; lack of expertise on the part of academics/module co-ordinators
	Teaching is of least priority, it is all about how much industry engagement (sponsorship) you gained, and published research; what happens in the classroom is entirely up to you as an individual
	Research being a higher priority; status of being an excellent ‘teacher’; promotional opportunities ‘seem’ to favour those with strong research backgrounds
Resistance to Change	Some conservative staff, resistance to change with entrenched views and lack of confidence and experience in the subject they teach
	The ideological opposition and (deliberate) competency deficit of many staff to ‘teaching’ and ‘technology’
	Fear of change and adaptability across an organisational culture; lack of understanding/reluctance to change due to the fear of the unknown
	Lack of awareness and engagement in alternative practices
	Inertia (not being open to change and are generally not terribly invested in teaching)

Possibly as a counter to these challenges, and to balance risk and reward of pedagogic innovation, Galley (2018) argues that in developing digital teaching and learning environments, institutions must maintain focus on the essential task of connecting students with their teachers and each other.

Conclusion

Self-evaluation is key to a Masters programme like this, so this study considered how the participants felt pedagogic innovation can be evidenced by artefact-based learning, what are the challenges to creating pedagogic innovation, if such programmes are having a sustained (positive) impact on their innovation in practice. From the programme team’s perspective, we were interested on knowing more about how can we continue to measure the impact of this work effectively, and critical incident analysis was helpful for that. Our experiences with academics, eLearning consultants, and trainers show that technology infusion takes time, steps, and significant patience. Through ongoing professional development, informed leadership, robust support, and long-term planning, however, we believe that all who continue to participate on this MSc Applied eLearning programme can reach their own goals for pedagogic innovation in their professional practice.