Abstract
Following the highly pervasive and effective use of agile methods at the team level, many software organisations now wish to replicate this success at the organisational level, adopting large-scale agile methods. However, an analysis of extant literature reveals (i) a lack of theoretically grounded studies of large-scale agile transformations; (ii) weak assumptions underpinning those that do exist; and (iii) an overly simplistic and often problematic over-reliance on simple adherence to agile methods. In addition, there is a lack of insights on why large-scale agile transformations fail. To explore this, we apply normalisation process theory (NPT) to examine key challenges in normalising (i.e. the process of embedding and sustaining) large-scale agile methods. We present an in-depth case study of large-scale agile transformation and demonstrate how failing to normalise a large-scale agile method can unravel an organisation’s transformation efforts. There are four key contributions from this research. First, this study shows that the standard focus on initial adoption is one of the main reasons for the widespread failure of digital transformations and in this case a large-scale agile transformation. Second, this is the first study to introduce and empirically apply NPT to information system development (ISD) as an illustrative guide for those across information systems field to follow. Third, we present key recommendations based on NPT to guide researchers and practitioners on the normalisation of large-scale transformations. Fourth, we present an agenda for future research on large-scale agile transformations.
Introduction
Agile 1 methods are used in over 90% of software development activity (VersionOne, 2021). Given that agile yields well known benefits such as the ability to respond to requirement changes, higher developer productivity, well-being, and self-organisation, more recent attention has turned to the challenge of designing and implementing large-scale variants of these methods that allow these benefits to spread across the organisation (Dikert et al., 2016; Edison et al., 2021). Examples of these methods include the Scaled Agile Framework (SAFe) (Leffingwell, 2015), and Large-Scale Scrum (LeSS) (Larman and Vodde, 2010). Reasons underpinning the emergence of these large-scale methods include a need for alignment and cohesion across many teams (Poth et al., 2021), interdependencies between software development and other organisational functions (Berntzen et al., 2019; Moe et al., 2019), as well as the global trend toward large, distributed teams and product delivery at scale (Dikert et al., 2016; Uludag et al., 2020). This movement has been given many different labels such as ‘large-scale development’ (Edison et al., 2021), ‘process transformation’, (Dikert et al., 2016), and ‘organisation-wide transformation’ (Laanti et al., 2011). Many organisations turned to large-scale agile development frameworks such as SAFe and LeSS ((Larman and Vodde, 2015; Dikert et al., 2016; Dingsøyr and Moe, 2014; Kalenda et al., 2018; Lagerberg et al., 2013; Paasivaara et al., 2018)), Spotify (a scaling agile approach introduced by Spotify) (Kniberg and Ivarsson, 2012), Nexus (Schwaber, 2015), and Scrum-at-Scale (Brown and Sutherland, 2014). Each framework incorporates predefined workflow patterns and routines and is supported by an ever-increasing set of tools. However, empirical evidence regarding the adoption of such frameworks, their use, effectiveness, and challenges is still very much in its infancy (Conboy and Carroll, 2019). In some cases, organisations can be successful with framework adoption, but for other organisations, frameworks can fail, and managers often experiment with alternative frameworks or abandoned them completely. Yet, large-scale agile transformation has proven challenging, with very few successful cases, or in what specific factors led to failures (Conboy and Carroll, 2019; Uludag et al., 2020; Kasauli et al., 2021).
The literature indicates that there is no firm agreement on what constitutes large-scale development (Dikert et al., 2016; Edison et al., 2021; Kalenda et al., 2018). However, there have been a few attempts to define large-scale development. For example, Barlow et al. (2011) define the term as characterised by multiple stakeholders and complex projects within large organisations. Rolland et al. (2016) suggest that large-scale involves complex integration with various internal and external information systems, multiple stakeholders with different interests. A large-scale agile transformation is an attempt to scale up how development organisations move from small agile practices (e.g. a single agile team in a large setting) to large teams guided by large-scale agile practices to at least 50 people or 6 teams (Dikert et al., 2016). In addition, large-scale agile methods have become widely researched yet there is a lack of research or theoretical developments which focus on how to manage and sustain their implementation. For example, Edison et al. (2021) identified 191 studies of which only 10 (8%) used theories to explain ‘why’ and ‘how’ some phenomena occurred. These outlined issues relating to coordination mode and mechanism theory, and trust–mediated organisational control. Theories were also adopted to understand the causality between people and technology, for example, socio-technical systems theory, project governance, knowledge management theory, relational coordination theory, routine dynamics theory, and adaptive structuration theory. However, after completing a comprehensive systematic review of large-scale agile methods, Edison et al. (2021: 22) conclude that ‘it reveals a number of theoretical and practical issues in the current literature such as over-emphasis on the practices of commercial large-scale agile frameworks at the expense of their foundational principles’. This also indicates that there is a lack of empirical evidence providing a nuanced account of the foundation principles such as (i) individuals and interactions over processes and tools; (ii) working software over comprehensive documentation; (iii) customer collaboration over contract negotiation; and (iv) responding to change over following a plan.
We argue that there is a need to examine in more detail how organisations embed and sustain the foundation of agile principles since scaling agile methods has become a major challenge for organisations (Conboy and Carroll, 2019; Uludag et al., 2020). While some research efforts have explored theoretical views such as routinisation and assimilation, to successfully scale we need a much more nuanced perspective and examine the subtleties around the transformation and scaling process. Therefore, we need a theoretical lens which can explain why and how large-scale agile frameworks are adopted: • By focusing on why we can explain the sensemaking process around the rationale to undergo a large-scale agile transformation and why other stakeholders ‘buy into’ this strategy. • By focusing on how we can explain what specific processes were introduced and integrated into current practices and how the impact of large-scale agile transformation was evaluated by various stakeholders.
Specifically, we argue that there is a need to adopt a normalisation perspective as it allows us to determine ‘the work that actors do as they engage with some ensemble of activities and by which means it becomes routinely embedded in the matrices of already existing, socially patterned, knowledge and practices’ (May and Finch, 2009: 540). This is important because empirical evidence regarding the adoption of methods for large-scale transformations, their use, effectiveness, and challenges is still very much in its infancy. As recent systematic literature reviews indicate (Uludag et al., 2020; Edison et al., 2021) what does exist tends to focus on the initial adoption of the method, and outlines various benefits and challenges associated with adherence to method variations over short timeframes, for example, the implementation phase. However, we lack any insights on how agile methods in large development projects become embedded and sustained in practice over long periods of time. In addition, we also know that there is little empirical evidence of successful cases for large-scale agile transformations (Cao et al., 2004; Conboy and Carroll, 2019; Dikert et al., 2016). This article explains how NPT provides a novel overarching theoretical framework to describe normalisation and challenge key assumptions around large-scale agile transformations.
Challenges with large-scale agile transformation
The extant literature identifies numerous overarching challenges with large-scale agile transformation. For example, it is reported that added complexity and uncertainty can be introduced when a method tries to impose radical and continuous change across a fragmented set of teams and projects across an organisation (Dikert et al., 2016). Adopting large-scale agile methods can also generate some confusion caused by numerous variants and misinterpretations of methods (Conboy and Carroll, 2019). It is also challenging to have a shared understanding of the problem that can be addressed by large-scale agile transformation and obtain commitment to change that needs to be done by all the units involved (Moe and Mikalsen, 2020). There are also a number of reports about the limitations of both top-down (management-driven) and bottom-up (team-driven) agile transformations (Conboy and Carroll, 2019; Dikert et al., 2016; Uludag et al., 2020; Edison et al., 2021). In some cases, masking such challenges behind analytics often directs more attention towards managing metrics (Ram et al., 2018) rather than managing the transformation process.
Previous research has shown that the scaling up of agile methods to a large-scale development setting presents ‘a thorny set of issues’ (Reifer et al., 2003), such as the weak assumption about how effective large-scale development can be achieved by simply scaling up small-scale agile methods (Rolland et al., 2016; Carroll et al., 2020). Regardless of the chosen approach of a prescribed scaled method or allowing them to evolve at a team level to implement large-scale methods, it is extremely challenging to successfully scale agile methods (Paasivaara, 2017; Conboy and Carroll, 2019; Uludag et al., 2020; Edison et al., 2021). The implementation of these scaled-up methods have struggled to deal with the complexities and ambiguities arising from (i) a large number of teams, roles, and personalities; (ii) an often unknown composition of participant teams and projects at the outset; (iii) abstract, knowledge-intensive, and often ill-structured work processes; (iv) diverse and often competing agendas between teams that sometimes contradict the organisation itself; and (v) abstract, complex, and often unknown final outputs and goals (Edison et al., 2021; Rolland et al., 2016; Conboy and Carroll, 2019; Paasivaara, 2017).
Assumptions of large-scale agile transformation
We have become accustomed to many weak assumptions underlying current research on large-scale agile development (Rolland et al., 2016). We need to also develop accounts as to why large-scale agile transformations fail. This calls for the need to reconceptualise how organisations achieve long-lasting, sustained agility across the entity (i.e. through normalisation) rather than simply achieving initial adherence to a subset of agile practices. To expand on this and explore assumptions around this phenomenon (Kane, 2022), we present four key points to support the rationale of focusing on normalisation of large-scale agile transformations:
Many of these weak assumptions contribute to implementation failures of large-scale methods and few leads to a sustained ongoing use of the method, or at least a sustained effective use of that method (Conboy and Carroll, 2019). The purpose of this study is to uncover what specific factors led to the mismanagement and unsustainability of a large-scale agile method. Thus, this study focuses on both ‘how’ and ‘why’ large-scale agile transformations fail.
Towards new theoretical insights on large-scale agile transformations
Although Dikert et al. (2016) describe transformations as being a ‘scaling up’ journey, the IS field lacks a transformation theoretical framework which explains how the process becomes implemented, embedded, integrated, and evaluated in practice. In addition, Fuchs and Hess (2018: 15) ‘call for research on this topic since it may not only spark intriguing theoretical insights but also pertinent practical implications…’. In an effort to explore alternative perspectives on transformation, we reviewed literature on theories ranging from assimilation of agile practices, routinisation, change management, sensemaking, and institutionalisation, dynamic capabilities, to name a few. How large-scale agile transformation become normalised in practice is a key question that holds the key to unpacking the inner-workings and outcomes of concern of researchers and practitioners (Edison et al., 2021). Traditional IS theories have been proven to be useful to focus on a narrow specific part of large-scale agile transformations, yet we need alternative perspectives to examine an overarching transformation process. For example, Uludag et al. (2020: 30) call for ‘future research endeavours should pay greater attention to building a solid theoretical scaffold for the observed phenomena in large-scale agile development’.
As is often the case with new and emerging phenomena in software development, practice has led research, with the creation, promotion, and dissemination of these large-scale agile methods largely due to the efforts of practitioners and consultants (Paasivaara, 2017). While this practice-driven nature certainly has merits, Rolland et al. (2016) caution that implementations are often built on weak assumptions which typically lack of empirical case studies explaining how large-scale agile transformations fail (Dikert et al., 2016). While there are arguments around the need to develop an ‘agile mindset’ to change an organisation’s culture, realistically this often take considerable time and effort, and organisations can fall back into old habits and fail to reap the full benefits of agility (Koutsikouri et al., 2020). A core motivation for this research is to address which emerged from our literature review, which was to explore why large-scale agile transformation failed in practice and to identify how specific events unfolded and prevented a large-scale agile method from becoming embedded and normalised in practice.
Research focus
Building on Struijk et al.’s (2022) advice this section presents the research focus and outlines how it appeals to IS scholars and practitioners. Specifically, in this study, we address two research questions on both ‘how’ and ‘why’ large-scale agile transformations fail. This study (i) presents a case study which goes beyond the implementation of a large-scale agile method to what we describe as the normalisation of a large-scale agile transformation, and (ii) describes what factors lead to a large-scale agile transformation failure. It is hoped that researchers and practitioners can learn from these insights to identify new approaches to better manage and sustain large-scale agile transformations. Specifically, we argue that normalisation therefore offers a novel and critical overarching lens to understand the process of large-scale agile transformations and the importance of managing the normalisation of new practices throughout an organisation’s transformation journey. We provide a number of contributions to theory and practice of large-scale agile transformations.
The remainder of this paper is structured as follows. The next section presents a discussion on NPT. Following this, we describe the single case study methodology adopted for this research and the relevance of the large-scale agile transformation. The then presents the research findings. The article then presents a discussion on the main contributions (practical and theoretical) of this research. The final section which offers a conclusion and outlines some limitations of NPT and identifies new directions for future research opportunities.
Normalisation process theory
Example of NPT studies.
As outlined in Table 1, there are many benefits of adopting NPT to investigate how does large-scale agile transformation become normalised in practice or what factors inhibited the normalisation process. This presents us with the required overarching theoretical perspective of large-scale agile transformation including the socio-technical insights (such as the team and organisational networks, culture, and technology) to embed and sustain the transformation process or the factors which inhibit this.
Normalisation process theory is a derivative sociological theory on the implementation, embedding, and integration of new technologies and organisational innovations (May and Finch, 2009) which allows us to challenge assumptions around embedding change during transformations. Normalisation process theory also identifies factors that promote and inhibit the routine incorporation of complex interventions into everyday practice (May et al., 2009; Murray et al., 2010). It focuses on providing practical insights on specific phenomena both qualitatively and quantitatively which can provide new insights on managing and sustaining large-scale agile transformation.
Applying core theoretical constructs and components of NPT to large-scale agile transformation (adapted from May and Finch, 2009; May et al., 2009; Carroll et al., 2020).
Methodology
Research design
Adopting a qualitative research approach utilises various research designs to obtain key insights from a range of data sources which normalisation processes typically generate. Given the under-researched nature of normalisation in large-scale agile transformations, we employed a single in-depth case study approach (Yin, 1984). A case study examines a phenomenon in its natural setting, employing multiple methods of data collection to gather information from one or a number of entities (people, groups, or organisations). For the purposes of this study, we adopt a single-case study as a revelatory and unique case (Yin, 1984) which examines organisations’ efforts to introduce a large-scale agile method and the process of managing and sustaining the transformation process.
For the purpose of this case study, we adopt Dikert et al. (2016) definition of large-scale development involving more than 50 developers or at least six teams working on a common product or project in the same organisation. This case study focuses on a global financial service organisation (FinanceCo 2 ) who employ approximately 1250 employees across the US, Ireland, India, and China. In the past, FinanceCo have experimented with various agile methods and attempts to customise agile methods. Over a 2-year period (2017–2018), FinanceCo underwent a large-scale agile transformation strategy and adopted the Spotify model (scaling agile approach at Spotify) (Kniberg and Ivarsson, 2012). This exploratory single case study focuses on the Spotify model as the unit of analysis and uncovers individuals experience in adopting Spotify and their experience in managing and attempts to sustain the large-scale agile transformation process.
Data collection
Within FinanceCo, the Center for Technology (C4T)
3
was a team which initiated the adoption of the Spotify model and was responsible for managing the large-scale transformation process. To guide the data collection process, the researchers captured historical accounts of the key motivations to adopt the Spotify model. These are summarised as follows: (i) To establish a cross-functional team structure across Squads (see Figure 1); (ii) To ensure improved integration of teams across projects; (iii) To empower teams to instigate improvements; (iv) To encourage teams to become autonomous in practice; (v) To increase organisational productivity and performance. Overview of the Spotify model adopted by C4T.
Summary of C4T Squads (located in Ireland (IE), USA (US), India (IN), and China (CN)).
Summary of additional interviewees (FinanceCo and external consultants).
The eight squads within C4T were very open in engaging with the researchers to document their large-scale agile transformation journey. The researchers were provided with access to C4T staff, FinanceCo email accounts, organisational tools, and a range of digital records (e.g. records on meetings and events, real-time data sources, and operational and strategic plans). The researchers were also introduced to their team members (see summary of interviewees Tables 3 and 4); agile project management systems (e.g. JIRA); dashboards and metrics (e.g. software flow metrics), team meetings, and other agile-related training events. Given the high level of access to C4T, the researchers could also obtain several sources of evidence across all eight Squads through documentation, archival records, interviews, performance, and productivity data. In addition, throughout the large-scale agile transformation, the researchers had direct observation access to verify interview accounts over an 18-month period on approximately 33 projects (each project did not involve all eight Squads).
The relevance of the case study was evident as the researchers could document efforts to normalise a large-scale agile transformation. Prior to the adoption of the Spotify model, C4T experienced a number of challenges around managing software development at scale to improve productivity and performance across their global software teams. These high-level challenges were first outlined by management and the US-SVP delivered a presentation to outline how the Spotify model and introduction of software flow metrics would support to alleviate these challenges, which included, for example, (i) the need for improved management of the product model; (ii) the need to remove silos and team inefficiencies; (iii) the need to develop new capabilities and processes; (iv) the need for increased transparency and communication; and (v) the need to improve productivity and performance;
The researchers were also invited to deliver regular workshop, for example, to present on our research developments, outline impediments to software flow, and design thinking for large-scale agile transformations. Guided by NPT, data was collected and analysed with a view to explore efforts to normalise the Spotify model and what factors contributed towards the adoption of the Spotify model.
Data analysis
Guided by the NPT framework, the researchers analysed the data guided by four key constructs (i) coherence; (ii) cognitive participation; (iii) collective action; and (iv) reflective monitoring while remaining open to including new constructs based on the data. The interviews allowed us to get an exchange of views between the researcher and interviewees on the large-scale agile transformation. Over the 18-month period, the researchers had built rapport and trust with the interviewee to examine the participants ‘inner world of meaning and feelings, as well as their experienced social reality’ (Schultze and Avital, 2011). The NPT framework provided a guide for the researchers to question and interpret their experiences and structure the conversation while honouring their freedom of thought and expression. As explained by Schultze and Avital (2011: 5), ‘by structuring the interview interaction, such frameworks help the interviewee as well as the researcher surface detail-rich descriptions as well as their significance and meaning…. As such, they assist in the generation of rich data as the interviewees are guided in accessing multiple layers of experience’.
We reviewed the interview transcripts and categorised key insights into the four NPT constructs and the 16 comprehensive components (Table 2). Analysis was necessary from the first interview because it was used to support and direct the next interview and observations. Normalisation process theory also provided a diagnostic frame to categorise various benefits, challenges, and recommendations when interviewees articulated their experiences with the large-scale agile transformation. As illustrated in Figure 2, we employed open coding to identify the relationships among interview statements and observations with NPT construct categories. We then employed axial coding to examine how NPT component categories were related to their subcategories and their relationships before moving to selective coding to understand efforts to normalise a large-scale agile transformation. Levels of abstraction to theorise on large-scale agile transformation.
Open coding was the first step in the analysis of our interview transcription. We split the data into discrete parts and create specific ‘codes’ by labelling the general nature of the text, for example, ‘rationale to transform’, ‘measuring progress’, or ‘team structures’. This allowed us to be open to new theoretical possibilities as we first examined the interview data. Over the 18-month period, we could go back and forth on the data and labelling to compare events as they evolved during the large-scale agile transformation. This also allowed us to remove preconceived ideas around large-scale agile transformation and probed the researchers to continuously question the data. The second major phase of our research focused on axial coding where we began to categorise the data into certain insights. We created ‘concept buckets’ and grouped insights using NPT constructs which allowed us to also identify connections between codes to better explain the normalisation of a large-scale agile transformation. The researchers also examined how the theory-informed conceptualisations of events from various sources facilitated a multi-layered visual mapping of the insights (Langley, 1999).
In addition, Figure 3 below provides examples illustrating the logic of how the case study sources provided insights on the normalisation of a large-scale agile transformation. Example of code aggregation.
Findings
This section summarises the key findings on FinanceCo large-scale agile transformation using the Spotify model. The findings are presented under the theoretical constructs of NPT (listed in Table 2) to examine what actions enabled or inhibited the normalisation of a large-scale agile transformation.
The Spotify model imposed new team structures for C4T. The new team structure was made up of eight Squads and the creation of new roles such as squad leads, chapters, and production support across the C4T tribe. The overall philosophy and principles for the new team structures were that each Squad are autonomous, self-organising, self-managing, and empowered to select the best way to work on an allocated number of epics (a large body of work that can be broken down into a number of smaller stories), using various scrum sprints, Kanban, or a hybrid of approaches. The Squads (comprising of a Squad Lead, and Scrum Master) represented key strategic elements at FinanceCo and their vision for the large-scale agile transformation. For example, squads represented new organisational initiatives such as moving towards cloud, mobile, and new software flow metrics (a data analytics initiative).
Coherence of the large-scale agile transformation
This section describes the insights on the transformation process which individuals and teams experienced with examining the rationale to adopt a large-scale agile transformation using the Spotify model. Specifically, our findings identify the challenges of developing a shared understanding on the rationale, differences, value, and roles associated with planning to undergo the transformation process, and the approach in managing the Spotify model. In doing so, it identifies how FinanceCo had a relatively unstable foundation and a lack of clarity around the rationale to undergo a large-scale agile transformation. As a consequence, our findings drew connections to normalisation phases and how certain factors began to unravel, making the transformation process unsustainable.
Differentiation between new Spotify model and customised agile model
Differentiation captured a comparative view between an old and new set of practices and drew focus around the rationale for changes imposed by the Spotify model. However, within FinanceCo, the findings indicate that there was a lack of clarity and communication across C4T to distinguish what specifically differentiated the Spotify model from the previous customised variants of agile practices. Specifically, there was a lack of evidence around (i) the rationale to undergo a large-scale transformation; (ii) the benefit or value-add of adopting the Spotify model as part of the large-scale agile transformation strategy; and (iii) the need for new software flow metrics to measure squad performance and productivity. It was clear that the C4T were results-driven but so much so that little focus on how the Spoofy method differentiates the overall work culture and network.
Our focus on differentiation led to two key findings at the very early stages around normalising the transformation process. First, the management team failed to differentiate and communicate the value of adopting the Spotify model in tangible terms. Second, the squads never ‘bought into’ the rationale for a large-scale transformation. Management only considered the high-level differences with the Spotify model and used ‘sweeping claims’ to promote the potential of the method. For example, management proposed that the large-scale agile transformation would address some business challenges such as (i) the need to improve the software development culture, (ii) improve fluidity of squad structures and collaboration, and (iii) improve continuous software development flow throughout the organisation. For example, the C4T Senior Vice President (US-SVP) explained that: ‘The Spotify model is different because it will allow us to improve the speed of our development if we can layer it on top of FinanceCo’s development culture – placing more value on autonomy, agile processes independence, democratic teams, and servant leadership’.
However, from a squad perspective, there was a growing sense of uncertainty around operationalising the Spotify model and dissatisfaction around the new proposed team structure changes. In addition, squad members learned that as part of the large-scale agile transformation process, software flow metrics (Vacanti and Vallet, 2014) would be introduced. As described by a Systems Analyst (IE-S2.3), it was envisaged that ‘software flow metrics would provide new insights on team and individual productivity and performance rates with a view to better manage the “flow” and continuous software development’. In the initial stages of the transformation, the interviewees described how there was a narrow focus on the potential of software flow metrics as a data-driven large-scale agile transformation process. For example, during a retrospective event, a Senior Software Engineer in the Data Chapter (IE-S4.6) reported that: ‘There are many metrics reporting on activities which support the overall large-scale agile transformation process, but do we spend more time on activities that are measured or activities that actually drive value?’
Communal specification of the Spotify model
Communal specification focused on how squads built a shared understanding of the Spotify model and the overall transformation process. Our findings indicate how this raised some of the core tensions between the management perspective and software developers’ understanding of the transformation process across C4T. For example, Squads initially perceived that the Spotify model imposed disruptive changes to C4T’s divisional structure which led to separation of powers. From a management perspective, team restructuring was considered necessary in an effort to build and sustain more efficient work patterns for self-organised teams to enjoy autonomy and drive the transformation process.
The C4T Tribe was established as a vertically integrated silo with specific homogenous skillsets. Team structures were reconfigured across horizontal groupings for specific projects. Yet, the introduction of a Tribe and Squads within C4T altered the dynamics of teams, their working relationships, and the cultural norms at FinanceCo. Interviewees described how this led to resistance towards the Spotify model and its newly imposed work practices. For example, the Squad Lead of S2 (IE-S2.1) viewed the Spotify model as: ‘It’s difficult to make sense of the Spotify model. You want certainty, predictivity, and control on the management side. Yet, you adopt the Spotify model because you have admitted that you don’t want predictability or control for the transformation process. So, there is a disparity in terms of reporting mechanisms and insight, and they bring with them, two different cultural norms on relational work’.
In an effort to understand the Squads experiences with the Spotify model, a Senior Project Manager (IE-SPM) distributed an online survey (entitled ‘Road to Agility’) to assess the impact of the large-scale agile transformation. Their findings relating to communal specification (i.e. a shared understanding) included (i) the lack of a clear shared vision; (ii) a reduction in motivation and joy in working together; (iii) increased technical debt; (iv) bureaucracy and time consumed through increased agile ceremonies (i.e. meetings with defined lengths, frequencies, and goals); (v) need for a singular definition of ‘done’; and (vi) the lack of reward for compliant behaviours. However, our findings indicate that managers did not act on the survey findings to implement changes which further compounded tensions associated with the large-scale agile transformation process.
Individual specification on implications of Spotify model
By focusing on individual specification, we identified how the sensemaking process was largely influenced by software flow metrics. From a managerial perspective, the Spotify model provided a transformation guardrail and roadmap that determined new roles and responsibilities required to improve organisational-wide agility and software team performance and productivity. Software flow metrics presented a novel way to ‘monitor and measure agility’ (SFC1). For example, the Senior Vice President (US-SVP) stated that: ‘We often talk about being agile but within a large-scale context, how agile are we organisation-wide? When we explored the Spotify model, we saw how we could scale agile beyond the team-level to the wider organisation which brings in new principles and practices across roles’.
At a Squad level, team members perceived that the Spotify model introduced new metrics for continuous improvement for productivity and performance. However, as a software developer within the Platform Chapter of S3 (IE-S3.7) explained: ‘not only are we forced to change roles but now we are held to account to reach new performance targets in these roles…’.
Internalisation on the transformation process
By focusing on internalisation, our findings focused on the value, benefits, and importance of the Spotify model to deliver financial, business, cultural, and/or personal benefits. However, the concepts of ‘value’, ‘benefits’, and ‘importance’ were largely considered to be vague or elusive terms from both management and Squad perspectives. Rather, stakeholders were typically guided by software flow metrics to determine the value, benefits, and importance of productivity and performance rates of completing various projects. Although the Spotify model was promoted by management as a means to adopt a new culture around team autonomy and empowerment, a Senior Software Engineer within the Data Chapter of S6 (US-S6.6) explained: ‘…with the Spotify model, you have to remember, wherever you have a tribe, you must also have a chief! If autonomy is monitored and measured, how can you give the illusion of working in accordance with our own beliefs, values and interests in the Squad. You still have to cooperate and be counted’.
Squad members questioned the value and benefits of Spotify with the growing bureaucracy of agile ceremonies and how management imposed new rules about the structure and frequency of meetings (Figure 4). For example, a Squad Lead (IE-S1.1) explained how the ceremonies often: ‘…got in the way of work and almost goes against the grain from how Spotify establishes the notion of self-organisation or empowerment of Squads’. Overview of C4T Squad meeting schedule and agile ceremonies (excluding other operational, training, or ad-hoc meetings).
Internalisation comparisons of ‘being agile versus doing agile’ (IN-S8.14) also brought about increased unplanned work across project. Interviewees explained how this raised increased doubts about the value, benefits, and importance of a large-scale agile transformation across the Tribe. This was best summed-up by a Senior Business Intelligence Developer in the Business Intelligence Chapter (CN-S7.3) as: ‘Our progress, or lack of progress, is probably best reflected in the amount of unplanned work we are faced with which makes it difficult to understand the value of using the Spotify model. If we only relied on metrics to guide progress, we wouldn’t take on any unplanned work, but in reality, we have to. This tells a different story on the overall large-scale transformation progress and metrics telling one story, but we have a job to do?’
Our findings identified how the increased level of unplanned work items led to growing frustrations across C4T at the early stage of the transformation process. For example, the Business Intelligence Developer in the Business Intelligence Chapter (IE-S3.3) questioned the value, benefits, and importance of the transformation: ‘Was this Spotify model meant to fit our needs or as I see it, we are tailoring the organisation to fit around this method? I think we’re putting huge efforts into fitting ourselves around the method and metrics and you’ll see this in our productivity trends so I’m not sure where the real value is. We can see progress but at what cost?’
Cognitive participation throughout the large-scale agile transformation
In this section, cognitive participation examines the relational work that C4T did in order to build and sustain the Spotify model and the large-scale agile transformation. While management promoted the notion of team autonomy and empowerment, the findings indicate that FinanceCo over-relied on adherence to the Spotify model to ensure data-driven progress throughout the large-scale agile transformation rather than focus on or understanding the overall sustainability of the transformation process. Across C4T, the Spotify model introduced new team members, roles, responsibilities, and divisions of labour to support the transformation process. Although Squads had initially adapted to a new way of working, their enthusiasm towards the Spotify model began to dwindle as the transformation matured.
Initiation of a transformation process
Initiation focuses on examining how team members contributed towards the large-scale agile transformation process. Within FinanceCo, C4T were the initiators of the large-scale agile transformation. The Senior Vice President and middle management promoted how Squads should be considered to be autonomous, self-organising, self-managing, and empowered to action change. However, interviewees explained how the introduction of new roles were not clearly defined and lacked clarity on how specific roles contributed to the implementation of and adherence to the Spotify model. For example, a Software Developer (IE-S2.9) explained that: ‘I have been working here for the last 5 years and this new method [Spotify] means I have to change how I work…and where I now sit in the room (in various Squads). Realistically, how does this change how I will develop software?’
In addition, while software flow metrics introduced new measures, it also introduced a new vocabulary about how team members were contributing to the large-scale agile transformation process. For example, there was a focus on performance measurement such as speed to complete projects rather than provide insights on how contributions to the transformation were progressing towards the ‘the success of Spotify or how close we were getting to the final destination’ (Software Developer; US-S6.7). As one Scrum Master (CN-S7.2) described it: ‘Metrics are fine but it’s almost as if we are focusing on the dashboard of a car and discussing how fast we are going, how much gas is in the tank, how many kilometres we have travelled, etc. But nobody is talking to us about the map and the journey we have been on and the destination we are headed towards. Metrics can’t really achieve this for the transformation process – we need the bigger picture from time-to-time’.
Enrolment into the Spotify model
Enrolment describes how team members became involved in the large-scale agile transformation process with the Spotify model. Our research indicates that during the early stages of large-scale agile transformation, senior staff members within C4T experimented with various alternative team structures to improve on delivering improved quality. These efforts were guided by software flow metrics. However, contrary to the notion of creating a culture of autonomy and empowerment (discussed in internalisation and initiation), interviewees explained that team members were not provided with the opportunity to restructure and enrol in alternative Squads, nor afforded the opportunity to evolve into new Squads. For example, a Senior Technical Manager (IE-S4.1) sent an email to C4T members with a visual seating plan and explained: ‘I know that the seating was originally supposed to be free seating, but let’s use this seating plan as a starting point. There is no issue with people moving around within their coloured squad area, but you are required to remain in your Squad…’
Although Squad members had committed to new team structures, it became extremely challenging for them to explain how the Spotify model contributed towards sustaining the large-scale agile transformation. For example, a Project Manager (US-S6.1) described: ‘We regularly hear from senior management that it is vital to be involved and keep on track for the transformation process using metrics. But I don’t really know what the endgame is or what success looks like? I’m frustrated trying to juggle and constantly reprioritise tasks – and we don’t have a roadmap. If I can’t connect the dots in terms of commitment, how will my team members do so (locally and globally)?’
Another Software Engineer (US-S5.6) explained their concerns around the lack of opportunity to have any input on how they should enrol on events for the newly transformed agile work practices: ‘I know that agile promotes the need to be collaborative but not all collaboration is productive especially at a large-scale. I’m frustrated with all the [global] meetings, ceremonies, and checkbox exercises we have to endure which, I can see effects how I now engage with colleagues’.
Legitimation of the Spotify model
Legitimation focuses on whether Squad members believed they could make a valid contribution to the large-scale transformation process with the Spotify model. During the transformation process, sprint planning played an important role in supporting Squads to determine the product backlog, assigned contributions from squads, and completion plans for product backlog items. However, sprint planning was reported to be one of the most ‘drawn out processes’ (IE-S1.8). In addition, our findings indicate that Scrum ceremonies would consume one-to-two hours of discussions on how team members should participate rather than being an opportunity for Squads ‘to air issues and discuss ways to better contribute as a team’ (IE-S3.10). For example, a Senior Software Developer (IN-S8.11) explained that: ‘The large nature of our team means we cannot frequently get everyone together for preparations and technical refinements. This is demonstrated by our relentless ceremonies which reduces our contribution to delivering product and sustaining the transformation but spend time discussing what we could do’.
However, championing the overall transformation process and the Spotify model ceremonies did prove to be a key factor to ensure maximum buy-in across Squads. For example, the Senior Vice President (US-SVP) described how: ‘Having identified the potential for Spotify to scale here; we need to sell it at an executive level to ensure full participation and contribution by demonstrating how it not only meets our needs locally, but how other global sites and business units should follow-suit to transform at a large-scale. A key thing is to maintain this momentum and goodwill which is not that straightforward’.
One approach to maintain and monitor the momentum of ‘valid contributions’ was demonstrated around process and productivity improvements through software flow metrics. The research participants referred to how C4T developed a ‘software flow metrics cookbook’ to prescribe how Squads should contribute and self-assess their contributions across planned priority projects.
Activation of the Spotify model
Activation refers to how the Squads collectively defined the actions and procedures needed to adhere to the Spotify model. It also examines initiatives teams adopted to stay committed to the large-scale agile transformation vision. The findings reveal that there was an initial sense that the large-scale agile transformation would offer some flexibility on what procedures may be adopted. Yet, the Spotify model imposed specific work structures and the reallocation of projects and team members. For example, a Senior System Analyst (IE-S3.4) explained that: ‘Due to restructuring, some projects and parts were moved to other global locations, and we also inherited other projects. Team members had to move to certain divisions too. So, they were here one week and gone the next week to work for and report to a totally different business unit’.
The interviewees explained that FinanceCo needed to balance autonomy with accountability of Squads through the introduction of ‘measurable objectives, consistent metrics for progress, and feedback systems to monitor activities’ (Squad Lead – US-S7.1). Therefore, software flow metrics played a key role in collectively redefining actions and procedures and report whether Squads reached or failed to reach specific goals. As one Senior Project Manager (IE-SPM) put it: ‘The flow metrics were useful to inform us how we were progressing. If progressing well, we’d have a conversation on what worked well and establish best practice. Equally, if we were not reaching targets, we would meet to discuss what wasn’t working well and experiment with alternative options for actions and procedures’.
In addition, a Software Engineer (IN-S7.9) described their experience from the initiation of the Spotify model for the transformation and metrics by explaining that: ‘We are told that as a Squad, we are expected to self-organise, and determine the best way to work. But after investing significantly in new tools we are also expected to tailor the approach around project management tools…and now new metrics for new actions and procedures during the large-scale transformation?’
Management formed Squads based on individual team members expertise (e.g. cloud computing, analytics, or cybersecurity). However, the level of differentiation between junior and senior level of expertise in each Squad proved to be a major obstacle throughout the transformation process. For example, a Senior Systems Analyst explained that (IE-S3.1): ‘For people, such as those at an executive level, who have little experience in transforming how we operate, it may feel like progress. For me, I feel that I have slowed down and am getting nothing done because of the disparity in experience with developers across the global Tribe’.
To fully adhere to the Spotify model, C4T defined the actions, procedures, and performance targets through various software flow metrics. By monitoring actions and adherence levels during the large-scale agile transformation via software flow metrics and interactive dashboards (i.e. Tableau), Squads could determine their levels of efficiency. For example, Squads could examine throughput (see Figure 5) and drill-down on adherence to procedures. Example of software flow metrics reporting on monthly comparison throughput (a measure of total work output by a Squad).
Collective action during the large-scale agile transformation
This section examines the relational work to build and sustain the large-scale agile transformation at FinanceCo. Across C4T, there was considerable focus on exploiting software flow metrics to inform the operationalisation of the Spotify model (e.g. faster team productivity rates). However, there was a lack of clarity on how the metrics linked to the large-scale agile transformation strategy or how success could be measured.
Interactional workability of squads
Interactional workability explores the work that Squad members collaborated on, their use of artefacts, and other practices as prescribed by the Spotify model. Although team autonomy and empowerment were promoted by management, according to one Platform Chapter member (IN-S8.7): ‘…autonomy proved to be a double-edged sword. On the one hand, it spurred on problem-solving and a desire to self-organise. On the other hand, autonomy was micro-managed through various software flow metrics to avoid ambiguity around performance and inefficiencies of productivity’.
Interviewees explained that the challenge for C4T was to strike the right balance between the management process and autonomy of Squads. However, issues relating to coherence generated ‘mixed messages’ as to ‘how’ and ‘why’ teams should collaborate under the Spotify model. For example, Squad members reported that the Spotify model hampered their ability to sustain working relationships or sufficiently complete tasks during the large-scale agile transformation. In addition, problems emerged due to the lack of communication, varied interpretation of metrics, and the growing expectations around the Spotify model.
The Spotify model introduced new techniques (e.g. software flow metrics and Kanban boards), new practices (e.g. new processes and agile ceremonies), and procedures. Squads had to accommodate and adapt to new ways of working. For example, a new metric called ‘touch time’ was adopted to monitor the specific time during the process whereby the product was worked on, and when the value is being added as opposed to the overall production time (in backlog, queuing, etc.). Touch time was introduced to allow squads to monitor individual interaction with a specific story and identify ways to improve speed. We can identify how touch time increased (Figure 6) after the initiation of the large-scale agile transformation (using the Spotify model) in Quarter 1 of 2017. However, we can also identify the growing demand on C4T to sustain the transformation process through increased interactional workability (from approximately 375 days before the large-scale agile transformation to 610 days after the adoption of the Spotify model). Example of touch time (by quarter) over a 3-year period showing increase in interactional workability.
Relational integration for the transformation process
Relational integration examines collaboration and training across C4T and efforts to build accountability to maintain confidence in the large-scale agile transformation. Within FinanceCo, Squad accountability for performance and productivity was captured through software flow metrics. Yet, interviewees explained that confidence around the benefits of adhering to the Spotify model was regularly undermined and diminished throughout the transformation process. For example, Squads raised concerns regarding the integrity of new and existing metrics during regular team meetings. For example, a Senior Software Developer (IE-S2.7) explained that: ‘The metrics are telling us we are getting faster, but is faster better? Many of us on the team feel like we’re just on a hamster wheel with growing expectations to support the large-scale transformation, focusing on how fast we work, and moving products to “Done”’.
In addition, a Software Developer (IE-S1.8) described this as: I feel we are losing some sense of accountability as it became the responsibility of a Tribe for certain deliverables. When delays occur, blame can then quickly shift across the cross-functional teams making it really difficult to identify the effectiveness of specific team members, tools, and decision-making to remove impediments.
Skillset workability of the Spotify model
Skillset workability explores how C4T allocated tasks which led to the division of labour, that is, new roles and responsibilities in adherence to the Spotify model. The majority of team members across Squads expressed concerns about the delegation of new responsibilities for the large-scale agile transformation process. Interviewees explained how the Spotify model does not promote standardisation, but rather the ‘need for cross-pollination of actions to transform practices’ (MS). Yet, within FinanceCo, software flow metrics offered actionable insights for adherence and standardisation of practice. By actionable, C4T could learn where to improve based on the software flow results and identify interventions for the transformation process. The rationale for adopting transparent and actionable flow metrics meant that practices met less resistance when Squads tried to adopt them and later became the default standard. To determine the Squads performance, status mapping throughout the large-scale agile transformation played a key role to identify and expose strengths and weakness. For example, as explained by a Senior Software Engineer (S8.5): ‘Our cycle time is important because it indicates the average time it took us to start and finish our stories. The lower the cycle time, the smoother we view our operations to be throughout the transformation’.
Contextual integration of the Spotify model
Contextual integration examines the resourcing of work, that is, managing the Spotify model through the allocation of resources to execute new protocols, policies, and procedures. Within C4T, the reallocation of resources for the transformation process was challenging to plan for given the ‘unchartered territory a large-scale agile transformation presents’ (IE-DSP). For example, interviewees explained that there was a lack of financial investment for skills development and recruitment. Additional focus was placed on reorganising teams across the Tribe and approaches to prioritise the use of resources. For example, a Business Analyst (IE-S4.3) explained that: ‘Defining a team is challenging as team members move from project to project. We try to focus on our top 15 high accountability and high impact projects to focus our effort and resources, but these priorities change so often that we can’t simply pin down fixed resources to see any one project out to completion’.
One Senior Project Manager (IE-SPM) explained the importance of software flow metrics to guide resource allocation and learning from other teams: ‘Having resources across a team is well and good but looking at the high priority projects, flow metrics assisted us to identify strong teams, who should support other teams increase their work-rate, and how they should tweak things to ensure an improved flow of work during the transformation’.
As part of the transformation strategy, the Senior Vice President (US-SVP) also placed emphasis on transparency of procedures as being a key factor of contextual integration: ‘Transparency alone initiates improvements. Once software flow data became assessable, visible and presented in a manner in which teams consume and learn how they were performing, they instinctively put actions in place to improve the large-scale transformation process’.
Reflexive monitoring of the large-scale agile transformation
Reflexive monitoring examined how Squads appraised the new set of practices in the Spotify model for the large-scale agile transformation. Within C4T, management were embarking on their ‘transformational journey’ but had little training and expectations on how the transformation process would impact the organisation. For example, the Senior Vice President (US-SVP) expressed concern about the lack of executive training to appraise the overall large-scale agile transformation process: ‘There are so many opportunities for training on how to operationalise the Spotify model such as for Scrum Masters, one-day training programs, and other events, but we don’t have any training programmes at an executive level to actually manage and evaluate the large-scale transformation process’.
Systemisation of the Spotify model
Systematisation examined how Squads determined the effectiveness and usefulness of the Spotify model for them and their colleagues during the large-scale agile transformation. However, our findings indicate that appraising the effectiveness and usefulness of the large-scale transformation for C4T proved to be a major challenge especially from different stakeholder perspectives. Prioritising the use of software flow metrics constrained C4T’s view of success to focus on narrow short-term (daily and weekly) performance. For example, software flow metrics focused on cycle time, throughput, hands-on-keyboards, and epics and sprint reports with little insight as to how their efforts contributed towards the overall long-term large-scale agile transformation. In addition, software flow metrics focused on the flow of work as opposed to the business objectives which the transformation was initially proposed to be strategically aligned with. For example, a Senior Systems Analyst (IE-S1.2) summed it up by explaining: ‘Management is preoccupied by going back to the product model, but nobody can tell me what the product model is, or how much money is being spent on product X, how much effort is going into product X, cost centre of business unit’s version of work. More importantly, how does our work contribute to the overall transformation strategy?’
Therefore, systemisation primarily focused on gathering software productivity data on C4T Squad performance levels. C4T also gathered anecdotal sources of information on problems associated with operationalising practices for both individual and collective appraisal. One example included what was described as ‘a calling out process of ineffective practices during Scrum events’ (Squad Lead – US-S8.1). According to one Systems Analyst (IE-S2.3), a key issue with a focus on software flow metrics to adhere to the Spotify model was that: ‘Assigning issues may be some form of bad practice but the metrics are accommodating flow rather than informing best practice? So, we can go fast but we might crash, or should we go slower, be careful, and arrive safely?’
With an emphasis on short-term planning rather than long-term planning for the large-scale agile transformation, our findings indicate that this gave rise to negative sentiments towards the Spotify model at the early transformation stage. For example, a Software Engineer (US-S5.2/US-6.2) explained that: ‘Our metrics and key performance indicators inform us that we are getting “faster” which must be a good story for the transformation process. But I am not happy in my work. Many of my colleagues feel the same and there is a lower morale than that of pre-Spotify’.
Communal appraisal of the Spotify model
Communal appraisal examined how squads collaborated (formally or informally) to evaluate the value of the Spotify model, for example, through team meetings, and informally over lunch, coffee breaks, and water cooler conversations. Discussions typically focused on (i) the value of specific actions, (ii) the rationale to introduce new software flow metrics, and (iii) reflections on how new processes or practices may affect changes to their work practices. However, the interviewees explained that knowledge-sharing on process improvements across the Tribe was problematic and often location-specific. For example, during one of the daily stand-up meetings, a Director of Technology based in the US (US-SSI) suggested that: ‘there is a lot of innovation going on in Ireland and it would be great to share across the Tribe if we are to transform together?’ Some Squad members also began to question the benefit of improving Squad performance rates. For example, a Senior Software Engineer (IE-S4.8) referred to a growing paradoxical sense that: ‘We are self-organised to a point. We can drive our own performance. But by reaching new performance targets or levels through those new Spotify team structures, I do feel we are victims of our own success. Performance is only part of the transformation story – there must be some feedback on the value on what and how or why we operate in this way?’
Individual appraisal of the Spotify model
Individual appraisal explored how Squads appraised the effects of the Spotify model on them during the transformation process. For example, we learned about (i) the steep learning curve team members experienced to operate within and adhere to the Spotify model; (ii) the increased workload due to planned and unplanned items; and (iii) lowering of team morale as a result of growing performance targets during the large-scale agile transformation. This was often described as ‘the disconnect’ between the large-scale agile transformation strategy and how it was operationalised in practice. For example, the Director of Strategy & Planning (IE-DSP) cautioned during a strategy planning event that: ‘Planning and delivery on one side of the organisation goes on but then there is the execution on another side of the organisation but there is a real disjoin between the two making it challenging to evaluate the value of the transformation’.
C4T members also appraised the effects of Spotify on themselves. For example, speed was an important measure across the software flow metrics. Defining and monitoring productivity metrics went through a number of iterations to evaluate, for example, (i) speed of individuals and Squads, (ii) individual contribution, (iii) compliance rates, (iii) cycle time, and (iv) hand-on-keyboards. However, our findings indicate that many of these measures were regularly challenged in a constructive manner by Squad members throughout the transformation process. In addition, a Scrum Master overseeing the Spotify model did not only question the value of the method as proposed by Spotify literature but also questioned their overall impact on individuals’ roles and responsibilities. As the Scrum Master (CN-S7.2/CN-8.2) explained: ‘I am also tasked with team evaluations and examine how individuals typically evaluate themselves and colleagues during the large-scale transformation. This provides me with a point of reference in the hope of getting a more accurate picture of the team and an agile compatible appraisal of team members, but it doesn’t seem to be going in a positive direction. The challenge is: who wants to hear this insight?’
Individual specification was largely influenced by the software flow metrics to gauge progress and improvements rather than a measure of transforming. The interviewees explained how software flow metrics were initially ‘based on Little’s Law – which only works if we have a balanced system (i.e., we start work at the same rate we finish it’ (extract from FinanceCo’s ‘Software Flow Metrics Cookbook’). However, from an individual perspective, software flow metrics were regularly undermined due to poor team performance ratings which failed to accommodate for evolving management performance targets (described as ‘moving goalposts’) and the Squads learning needs.
Reconfiguration of practice under the Spotify model
Reconfiguration examined how Squad members’ appraisals led to attempts to redefine procedures or reorganise practices around the Spotify model. The appraisal process led to many attempts by C4T Squads to change and improve processes which were heavily influenced by the use of actionable software flow metrics. While Squads could experiment to improve the ways they worked, Squad Leads explained how they could also capture the relevant data about software processes to reduce complexity and increase the flow efficiency. This principle became known as ‘process independence’. Process independence described a way to preserve independence of Squads to self-organise and optimise the way they worked to suit the specific problem, technology, customer, and general context that each face. Moving towards actionable metrics allowed C4T to understand how diverse workforces were managed and how they reconfigured some form of process change or revised metrics. For example, managers within FinanceCo regularly revisited their agile practices and the metrics to report performance and explore whether restructuring teams or processes could yield better results. However, it was reported by the Senior Vice President (US-SVP) that ‘the frequency of doing so could undermine the transformation process’. Software flow metrics continued to provide increased transparency as management and Squads examined reconfiguration through key performance-related questions, such as (BI Chapter – IE-S1.3): ‘how are we performing and where can we improve?’ Executive management promoted the notion that actionable flow metrics empowered squads at FinanceCo to ‘own their own performance through democratised decision-making’ (US-SVP).
To accommodate for reconfigurations across a diverse workforce, actionable flow metrics encouraged managers and Squads to openly discuss or debate the benefits or drawbacks associated with various agile practices and analytics. There were concerns amongst manager as to whether software flow metrics provided an accurate reflection of current practice. To illustrate this point, C4T developed a control chart which reported the cycle time for a product, version, or sprint. This was used to support teams identify whether data from the current process can be used to determine future performance (Figure 7). The chart reports on the visibility, efficiency, and predictability of team performance. Visibility examines the outliers and investigates their cause to reduce them in the future. Efficiency focuses on decreasing rolling average (blue line) which indicates process improvements and increased throughput. Predictability provides a focus on a narrow standard deviation (light blue fill) through process improvement to improve the predictability of cycle time. As an example, from 27 May 2018 to 19 August 2018, Figure 7 reports an average (red line) of 2 weeks, 4 days, and 10 hours and 8 issues. However, cycle time increased during the large-scale agile transformation process. Yet, Squads were encouraged to take autonomy and drive change rather than evaluate best practice for the overall transformation process to address issues surrounding increased cycle time. For example, a Director of Technology in the US (US-SSI) expressed the need for team members to correct inefficiencies: ‘If you see a problem, take the autonomy to make change to makes things move fast!’ Examining the predictability of C4T performance and productivity illustrating increased elapsed time (6-month view).
Discussion
This section presents a discussion on the main contributions of this research (practical and theoretical) for the normalisation of a large-scale agile transformation. We explain the importance of our theoretical and practical contribution and are guided by Struijk et al.’s (2022) questions: ‘what would the world miss if your manuscript was never published?’ and ‘what would other IS scholars get wrong if they never read your paper?’ In doing so, we expand discussion to explain the novelty of NPT and complex socio-technical interdependencies associated with large-scale agile transformations.
Summary of FinanceCo’s normalisation efforts for the large-scale agile transformation.
We also note that while scalability impacts on the operations within an organisation, it also impacts on outcomes, team structures, and related factors. Whether organisations are at a very early stage of a large-scale agile transformation, or a more mature stage, it is critical to ensure the continuity of business. Some leaders may embrace the challenge of a large-scale agile transformation while others may be more cautious and risk adverse. Regardless of the leadership style, large-scale agile transformations require careful preparation, and committed leadership which a clear understanding of how the transformation process will impact on people (e.g. team structures, networks, and relationships), processes (e.g. coordinating teams, customer engagement, monitoring performance, and productivity metrics), and practices (e.g. agile methods and organisational culture). Understanding these factors will also influence the normalisation of large-scale agile transformations.
From a practical perspective, the FinanceCo case study demonstrated how they failed to consider or act on key normalisation components while other NPT components were planned for in practice. By mapping out the NPT theoretical components against the practical factors, we reflected on how FinanceCo failed to sustain a large-scale agile transformation using the Spotify model (Table 5). It is evident that the large-scale transformation in this case study was neither top-down driven nor bottom-up driven but rather a loose mix of efforts to drive changes as opposed to successfully undergo a large-scale agile transformation. By failing to normalise the Spotify model, efforts surrounding the large-scale agile transformation process began to unravel at the early stages (i.e. coherence) based on weak assumptions and rationale for the transformation process.
Practical contributions
We revisit each of the four main normalisation phases and reflect on the strengths and weakness of FinanceCo’s normalisation efforts to outline some key recommendations for each phase for transformations.
Recommendations on coherence for large-scale agile transformations
The rationale to undergo a large-scale transformation can create a lot of uncertainty among stakeholders (Berntzen et al., 2019; Dingsøyr et al., 2018; Uludag et al., 2018). For example, at the early stages of the transformation process, the management team in C4T promoted high-level expectations on how the Spotify model would improve software productivity and performance. Promoting abstract transformation expectations hampered the execution and evidence-based approach to differentiate between new large-scale agile practices and previous agile practices. Layering a large-scale agile method with new forms of metrics and with little guidance on how they align or contribute to large-scale agile transformations was also met with resistance across Squads. It also became apparent that there was an overemphasis on developing metrics to measure the large-scale transformation progress as opposed to determining the best way to tailor and operationalise the transformation to meet FinanceCo strategic needs. For example, the Squad Lead for S4 (IE-S4.1) explained that: ‘It’s not about process, it shouldn’t be about anything else but trust! All of these structures and things are being imposed by people that don’t have to go through the process and don’t trust the method or the staff to execute the method’.
Recommendations for coherence in large-scale agile transformations.
Recommendations on cognitive participation in large-scale agile transformations
FinanceCo largely focused on adherence to the Spotify model over its long-terms sustainability for the large-scale agile transformation. This was demonstrated by their use of software flow metrics which primarily focused on short-term software team performance and productivity. The literature suggests that this typically presents many challenges and tensions between management and software teams on how to balance adherence with long-term software team performance and productivity (Conboy and Carroll, 2019; Edison et al., 2021). From our research, this issue is best summed-up when we explore these findings with a Manager of Software Operations at Spotify (MS). He speculated on the legitimation of adopting the Spotify model within FinanceCo and some of the challenges experienced with C4T during the large-scale transformation: ‘I must caution that many organisations look at the Spotify model as being their panacea which will solve all of their problems – and use it “out of the box.” The Spotify model worked for us at Spotify! It was not designed to be adopted by every organisation. You must carefully consider what elements work for you, and which don’t’.
Recommendations on cognitive participation for large-scale agile transformations.
Recommendations on collective action for large-scale agile transformations
Recommendations on collective action for large-scale agile transformations.
Recommendations on reflexive monitoring of large-scale agile transformations
The use of software flow metrics was a novel initiative for the large-scale agile transformation at FinanceCo. However, evaluations of the Spotify model indicated that C4T were too narrowly focused on speed and time events and failed to account for factors which influence transformation such as a change in the culture and mindset. The importance of culture (van Wessel et al., 2021) while adopting the Spotify model was explained by a Manager of Software Operations at Spotify (MS) who described how: ‘Culture is an abstraction, yet the forces that are created in social and organisational situations are powerful. If we don’t understand the operation of these forces, we can become victim to them during large-scale agile transformations’.
Recommendations on reflexive monitoring for large-scale agile transformations.
Theoretical contributions
Our research presents four key theoretical contributions. Firstly, this research is the first to apply NPT in the IS field to explore the normalisation of a large-scale agile transformation. Our literature review indicates that there is an apparent need to advance theoretical diversity on overarching perspectives on the large-scale agile transformation process (Edison et al., 2021; Uludag et al., 2020). To address this, we demonstrate how NPT provides an excellent and novel theoretical lens to achieve this which is a first for the IS community. Secondly, the research demonstrates how NPT supports researchers and practitioners to identify which actions facilitate or prevent the sustainability of large-scale agile transformations using this nuanced lens of normalisation. We explain how FinanceCo failed to successfully lead a large-scale agile transformation. For example, we provide a nuanced view on how C4T placed too much emphasis on software flow metrics for short-term software team productivity and performance as a general measure of ‘progress’ and the consequences of doing so. Thirdly, we present key recommendations based on NPT to guide researchers and practitioners on the normalisation of large-scale transformations (Tables 6-9). From an NPT perspective, we outline how organisations can adopt a stronger evidence-based approach to move forward and ensure the adoption of large-scale agile methods provides a means to achieve the transformation objectives, rather than simply restructuring an organisation around a large-scale agile method. We witnessed incremental changes made to the work practices to implement a large-scale transformation which demonstrates the applied nature of NPT. This case study indicates that a large-scale agile transformation process does not necessarily proceed in a strictly linear fashion but rather encounter difficulties and go through revisions as it evolves within a particular context in response to internal and external factors. However, it was clear that failing to consider normalisation over adherence for a large-scale agile method, such as the Spotify model, can lead to an unsuccessful transformation outcome. Fourthly, we identify directions for future research opportunities on large-scale agile transformations.
Our literature review reports on the lack of theoretically grounded research on large-scale agile development and transformations which presents some opportunities for future research opportunities. As part of a future research agenda, firstly we have identified the need to expand on this research around the normalisation of large-scale agile transformations. Therefore, to further ensure rigour and relevance of our NPT theoretical developments and to build a cumulative tradition on IS theories, there is an opportunity to conduct multiple case studies on large-scale transformations in global software organisations to offer case study comparisons with various contextual factors across different industries. Secondly, there is a need to build models that capture the dynamic nature of the transformation process and weave narratives around stakeholders and sequences of events to explain how transformations evolved over time; why they evolved in the way they did; and how they become implemented, embedded, integrated, and evaluated in practice; and long-term metrics to measure normalisation factors via a transformation maturity model. Thirdly, we identified growing assumptions between adherence and sustainability which calls for more research on uncovering sustainability factors (e.g. team communication and coordination) to take account of a broader perspective of transforming versus adopting an agile method. Fourthly, as there continues to be a growing emphasis placed on data analytics to inform organisations around their progress with a large-scale agile transformation, there is an opportunity to research how organisations balance the demand for autonomy with their reliance on data-driven decision-making. Fifthly, there is an opportunity to map or visualise the evolution of a large-scale agile transformation and changes across organisational and social networks using social network analysis (SNA). This may reveal further insights by investigating social structures through the use of networks and graph theory and examine what characterises networked structures in terms of nodes (individual actors, people, or things within the network) and the ties, edges, or links (relationships or interactions) that connect them.
As noted in the discussion, it is important to explore how scalability impacts on outcomes, team structures, and related factors within a large-scale agile context. Scalability typically focuses on the growth and addition of resources to a system and its capabilities. Within an agile context, ‘scaling out’ and ‘scaling up’ often focuses on the adoption of agile methods for large software projects. As noted by Rigdy et al. (2018) on the topic of ‘agile at scale’ within organisations, ‘they place more value on adapting to change than on sticking to a plan…’. This was also reported by Conboy and Carroll (2019) who describe the lack of empirical evidence on the adoption of large-scale agile methods, their effectiveness, or factors leading to failed large-scale agile transformations. Current literature focuses on organisations experience with adopting large-scale agile methods and techniques to implement and scale practices (Saeeda et al., 2015; Edison et al., 2021). However, Rolland et al. (2016) question the assumption that effective large-scale development can be achieved by simply scaling up small-scale agile methods. This calls for more focus on the ‘complex socio-technical interdependencies’ which are considered to be essential characteristics of large-scale agile projects (Rolland et al., 2016). In addition, Edison et al. (2021) identify that difficulties with large-scale agile transformations typically centre around the complexities and ambiguities, arising from teams, their composition and various personalities, ill-structured work processes, diverse and often competing agendas between teams, and vague outputs and goals. Yet, our research indicates a lack of research on the socio-technical interdependencies which support to embed and sustain the large-scale agile methods, that is, the normalisation of large-scale agile transformations. Our research is the first study which demonstrates how NPT provides an excellent theoretical lens which provides a nuanced view of which factors can enable or inhibit the transformation process. It uncovers specific aspects of the organisation scalability efforts and the factors which lead to poor outcomes, impact of new team structures on productivity and performance, and the misalignment of how various stakeholders perceived progress and success of a large-scale agile transformation. To build on this, additional case studies should apply NPT to compare and contrast findings around success and failure for large-scale agile transformations. Additional research is also required on the leadership styles for large-scale agile transformations and how it may impact on people, processes, and practices for scalability. We also envisage that the contributions of this research will be far reaching and support new research developments around IS transformation, for example, transformations efforts spanning across strategies to normalise digital transformations and IT-enabled change (Carroll et al., 2021).
Conclusion
Following the rise and proliferation of agile methods across almost every corner of contemporary software development, more recent attention has turned to the challenge of designing and implementing large-scale, organisation-wide variants of these methods. Many organisations attempt to scale agile methods in an effort to scale success outcomes by introducing methods such as SAFe or Scrum-at-Scale, or some customised version of related methods. Our research indicated a lack of theoretically grounded studies of large-scale agile transformations and weak assumptions underpinning those that do exist. Our research also indicates that practitioners often over-relied on adherence to agile methods rather than identifying approaches to successfully embed and sustain the transformation process.
Our study applies NPT to examine key challenges in normalising large-scale agile methods. An in-depth case study is presented on a large-scale agile transformation and demonstrates ‘how’ and ‘why’ failing to normalise a large-scale agile method can lead to failure. This study presents four key contributions: (i) it reports on how a focus on the initial adoption of a large-scale agile method has major shortcomings; (ii) we introduce and empirically apply NPT as an illustrative guide for the information systems field to identify and mitigate against transformation failure or abandonment; (iii) we present key recommendations on the normalisation of large-scale transformations; and (iv) we present an agenda for future research on large-scale agile transformations.
Limitations of NPT
Future research agenda for large-scale agile transformation.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported with the financial support of the Science Foundation Ireland grant 13/RC/2094 and co-funded under the European Regional Development Fund through the Southern & Eastern Regional Operational Programme to Lero - the Science Foundation Ireland Research Centre for Software (
).