From transformation to normalisation: An exploratory study of a large-scale agile transformation

Following the highly pervasive and effective use of agile methods at the team level, many software organisations now wish to replicate this success at the organisational level, adopting large-scale agile methods. However, an analysis of extant literature reveals (i) a lack of theoretically grounded studies of large-scale agile transformations; (ii) weak assumptions underpinning those that do exist; and (iii) an overly simplistic and often problematic over-reliance on simple adherence to agile methods. In addition, there is a lack of insights on why large-scale agile transformations fail. To explore this, we apply normalisation process theory (NPT) to examine key challenges in normalising (i.e. the process of embedding and sustaining) large-scale agile methods. We present an in-depth case study of large-scale agile transformation and demonstrate how failing to normalise a large-scale agile method can unravel an organisation’s transformation efforts. There are four key contributions from this research. First, this study shows that the standard focus on initial adoption is one of the main reasons for the widespread failure of digital transformations and in this case a large-scale agile transformation. Second, this is the first study to introduce and empirically apply NPT to information system development (ISD) as an illustrative guide for those across information systems field to follow. Third, we present key recommendations based on NPT to guide researchers and practitioners on the normalisation of large-scale transformations. Fourth, we present an agenda for future research on large-scale agile transformations.


Introduction
Agile 1 methods are used in over 90% of software development activity (VersionOne, 2021). Given that agile yields well known benefits such as the ability to respond to requirement changes, higher developer productivity, wellbeing, and self-organisation, more recent attention has turned to the challenge of designing and implementing large-scale variants of these methods that allow these benefits to spread across the organisation (Dikert et al., 2016;Edison et al., 2021). Examples of these methods include the Scaled Agile Framework (SAFe) (Leffingwell, 2015), and Large-Scale Scrum (LeSS) (Larman and Vodde, 2010). Reasons underpinning the emergence of these largescale methods include a need for alignment and cohesion across many teams (Poth et al., 2021), interdependencies between software development and other organisational functions (Berntzen et al., 2019;Moe et al., 2019), as well as the global trend toward large, distributed teams and product delivery at scale (Dikert et al., 2016;Uludag et al., 2020). This movement has been given many different labels such as 'large-scale development' (Edison et al., 2021), 'process transformation', (Dikert et al., 2016), and 'organisation-wide transformation' (Laanti et al., 2011). Many organisations turned to large-scale agile development frameworks such as SAFe and LeSS ( (Larman and Vodde, 2015;Dikert et al., 2016;Dingsøyr and Moe, 2014;Kalenda et al., 2018;Lagerberg et al., 2013;Paasivaara et al., 2018)), Spotify (a scaling agile approach introduced by Spotify) (Kniberg and Ivarsson, 2012), Nexus (Schwaber, 2015), and Scrum-at-Scale (Brown and Sutherland, 2014). Each framework incorporates predefined workflow patterns and routines and is supported by an ever-increasing set of tools. However, empirical evidence regarding the adoption of such frameworks, their use, effectiveness, and challenges is still very much in its infancy (Conboy and Carroll, 2019). In some cases, organisations can be successful with framework adoption, but for other organisations, frameworks can fail, and managers often experiment with alternative frameworks or abandoned them completely. Yet, large-scale agile transformation has proven challenging, with very few successful cases, or in what specific factors led to failures (Conboy and Carroll, 2019;Uludag et al., 2020;Kasauli et al., 2021).
The literature indicates that there is no firm agreement on what constitutes large-scale development (Dikert et al., 2016;Edison et al., 2021;Kalenda et al., 2018). However, there have been a few attempts to define largescale development. For example, Barlow et al. (2011) define the term as characterised by multiple stakeholders and complex projects within large organisations. Rolland et al. (2016) suggest that large-scale involves complex integration with various internal and external information systems, multiple stakeholders with different interests. A large-scale agile transformation is an attempt to scale up how development organisations move from small agile practices (e.g. a single agile team in a large setting) to large teams guided by large-scale agile practices to at least 50 people or 6 teams (Dikert et al., 2016). In addition, large-scale agile methods have become widely researched yet there is a lack of research or theoretical developments which focus on how to manage and sustain their implementation. For example, Edison et al. (2021) identified 191 studies of which only 10 (8%) used theories to explain 'why' and 'how' some phenomena occurred. These outlined issues relating to coordination mode and mechanism theory, and trust-mediated organisational control. Theories were also adopted to understand the causality between people and technology, for example, socio-technical systems theory, project governance, knowledge management theory, relational coordination theory, routine dynamics theory, and adaptive structuration theory. However, after completing a comprehensive systematic review of large-scale agile methods, Edison et al. (2021: 22) conclude that 'it reveals a number of theoretical and practical issues in the current literature such as over-emphasis on the practices of commercial large-scale agile frameworks at the expense of their foundational principles'. This also indicates that there is a lack of empirical evidence providing a nuanced account of the foundation principles such as (i) individuals and interactions over processes and tools; (ii) working software over comprehensive documentation; (iii) customer collaboration over contract negotiation; and (iv) responding to change over following a plan.
We argue that there is a need to examine in more detail how organisations embed and sustain the foundation of agile principles since scaling agile methods has become a major challenge for organisations (Conboy and Carroll, 2019;Uludag et al., 2020). While some research efforts have explored theoretical views such as routinisation and assimilation, to successfully scale we need a much more nuanced perspective and examine the subtleties around the transformation and scaling process. Therefore, we need a theoretical lens which can explain why and how large-scale agile frameworks are adopted: · By focusing on why we can explain the sensemaking process around the rationale to undergo a large-scale agile transformation and why other stakeholders 'buy into' this strategy. · By focusing on how we can explain what specific processes were introduced and integrated into current practices and how the impact of large-scale agile transformation was evaluated by various stakeholders.
Specifically, we argue that there is a need to adopt a normalisation perspective as it allows us to determine 'the work that actors do as they engage with some ensemble of activities and by which means it becomes routinely embedded in the matrices of already existing, socially patterned, knowledge and practices' (May and Finch, 2009: 540). This is important because empirical evidence regarding the adoption of methods for large-scale transformations, their use, effectiveness, and challenges is still very much in its infancy. As recent systematic literature reviews indicate (Uludag et al., 2020;Edison et al., 2021) what does exist tends to focus on the initial adoption of the method, and outlines various benefits and challenges associated with adherence to method variations over short timeframes, for example, the implementation phase. However, we lack any insights on how agile methods in large development projects become embedded and sustained in practice over long periods of time. In addition, we also know that there is little empirical evidence of successful cases for large-scale agile transformations (Cao et al., 2004;Conboy and Carroll, 2019;Dikert et al., 2016). This article explains how NPT provides a novel overarching theoretical framework to describe normalisation and challenge key assumptions around large-scale agile transformations.

Challenges with large-scale agile transformation
The extant literature identifies numerous overarching challenges with large-scale agile transformation. For example, it is reported that added complexity and uncertainty can be introduced when a method tries to impose radical and continuous change across a fragmented set of teams and projects across an organisation (Dikert et al., 2016). Adopting large-scale agile methods can also generate some confusion caused by numerous variants and misinterpretations of methods (Conboy and Carroll, 2019). It is also challenging to have a shared understanding of the problem that can be addressed by large-scale agile transformation and obtain commitment to change that needs to be done by all the units involved (Moe and Mikalsen, 2020). There are also a number of reports about the limitations of both topdown (management-driven) and bottom-up (team-driven) agile transformations (Conboy and Carroll, 2019;Dikert et al., 2016;Uludag et al., 2020;Edison et al., 2021). In some cases, masking such challenges behind analytics often directs more attention towards managing metrics (Ram et al., 2018) rather than managing the transformation process.
Previous research has shown that the scaling up of agile methods to a large-scale development setting presents 'a thorny set of issues' (Reifer et al., 2003), such as the weak assumption about how effective large-scale development can be achieved by simply scaling up small-scale agile methods (Rolland et al., 2016;Carroll et al., 2020). Regardless of the chosen approach of a prescribed scaled method or allowing them to evolve at a team level to implement large-scale methods, it is extremely challenging to successfully scale agile methods (Paasivaara, 2017;Conboy and Carroll, 2019;Uludag et al., 2020;Edison et al., 2021). The implementation of these scaled-up methods have struggled to deal with the complexities and ambiguities arising from (i) a large number of teams, roles, and personalities; (ii) an often unknown composition of participant teams and projects at the outset; (iii) abstract, knowledgeintensive, and often ill-structured work processes; (iv) diverse and often competing agendas between teams that sometimes contradict the organisation itself; and (v) abstract, complex, and often unknown final outputs and goals (Edison et al., 2021;Rolland et al., 2016;Conboy and Carroll, 2019;Paasivaara, 2017).

Assumptions of large-scale agile transformation
We have become accustomed to many weak assumptions underlying current research on large-scale agile development (Rolland et al., 2016). We need to also develop accounts as to why large-scale agile transformations fail. This calls for the need to reconceptualise how organisations achieve long-lasting, sustained agility across the entity (i.e. through normalisation) rather than simply achieving initial adherence to a subset of agile practices. To expand on this and explore assumptions around this phenomenon (Kane, 2022), we present four key points to support the rationale of focusing on normalisation of largescale agile transformations: 1. Challenges with large-scale agile methods. Agile methods or customised variants were traditionally introduced to support the processes that organisations were already familiar with, which were typically designed to be implemented at a smaller scale (Edison et al., 2021). As the numbers of teams and projects grew, management needed to formalise and control the processes across the organisation. They then selected, tailored, and integrated different existing method fragments relevant to their context. Yet, the literature (Dingsøyr and Moe 2013;Conboy and Carroll, 2019) indicates that traditional agile methods cannot simply be scaled as large-scale development introduces challenges related to inter-team coordination, large project organisation, release planning and architecture, customer collaboration (including contracts), and knowledge-sharing and improvementall of which presents some contradictions to the traditional nature of agile methods. Agile methods originally focused on working in small teams focused on individuals and interactions, working software (rather than documentation), customer collaboration, and responding to change. This now requires alternative ways to view and report on large-scale agile frameworks and transformations. 2. Lack of theoretically driven analysis of largescale agile transformations. It is reported that many research efforts on large-scale agile transformation have been largely built on weak assumptions (Rolland et al., 2016) around practitioner views of embedding and sustaining transformations (Carroll et al., 2020). Normalisation explicitly examines weak underlying assumptions of embedding change such as large-scale agile transformations and can identify actions leading to success or failures. Normalisation process theory (NPT) achieves this through its iterative nature by tracing the work that people do and how the programme of work evolves and aligns with the changing nature of the large-scale agile transformation, for example, with new perspectives on coherence, cognitive participation, collective action, reflexive monitoring and their impact on organising factors, organising structures and social norms, immediate factors, and group process and conventions (see section on 'Normalisation process theory'). This is of particular importance within a large-scale agile transformation context as agile methods emerged largely from practice without a great deal of consideration for sound conceptual foundation for the topic (Abrahamsson et al., 2009;Ågerfalk et al., 2009;Conboy, 2009;Conboy and Carroll, 2019;Schnitter and Mackert, 2010). Normalisation proposes a novel working model of implementation, embedding, and integration in conditions marked by complexity and emergence  which is ideal to better explain the context of large-scale agile transformations.
3. Over-reliance on adherence of large-scale agile methods. Formal large-scale agile methods, such as SAFe, Scrum-at-Scale, or Spotify, creates a tendency to measure transformation by adherence to a specific method, rather than the value it provides to sustaining practice (Conboy and Carroll, 2019;Dikert et al., 2016). Normalisation allows us to focus on the specific task and action (e.g. workflow, processes, or events), and embedding and of sustaining practices through specific agile methods which helps in understanding why some methods become normalised while others do not . Normalisation also invites us to move beyond adherence and address incorrect assumptions by examining the fine-grained approaches to sustaining methods as opposed to one-off method use. It achieves this through iterative processes such as interactional workability, relational integration, and contextual integration (within collective action construct of NPT) by using specific artefacts and resources to build accountability and maintain confidence in a set of practices and the team through the execution of protocols, policies, and procedures.
4. Understanding the dynamic nature of large-scale agile transformation. The nature of agile principles (Beck et al., 2001) was developed in response to guide organisations on how to operate in small team and highly dynamic environments. Yet, research on large-scale agile transformation fails to consider the dynamic nature of large-scale agile methods (Dingsøyr et al., 2018) which comprise of, for example, larger teams, rigid organisational structures, and regulations. Nowadays, software organisations operate in ever-changing dynamic environments (Fitzgerald and Stol, 2017;Leffingwell, 2007) making it challenging to normalise transformationsalmost as if chasing a 'moving target' (Posen and Levinthal, 2012). Normalisation provides new insights on how organising factors, structures and associated social norms evolve with the dynamic nature of transformations which need to be continuously implemented, embedded, integrated, and evaluated within software organisations to realign with that moving target.
Many of these weak assumptions contribute to implementation failures of large-scale methods and few leads to a sustained ongoing use of the method, or at least a sustained effective use of that method (Conboy and Carroll, 2019). The purpose of this study is to uncover what specific factors led to the mismanagement and unsustainability of a large-scale agile method. Thus, this study focuses on both 'how' and 'why' large-scale agile transformations fail.

Towards new theoretical insights on large-scale agile transformations
Although Dikert et al. (2016) describe transformations as being a 'scaling up' journey, the IS field lacks a transformation theoretical framework which explains how the process becomes implemented, embedded, integrated, and evaluated in practice. In addition, Fuchs and Hess (2018: 15) 'call for research on this topic since it may not only spark intriguing theoretical insights but also pertinent practical implications…'. In an effort to explore alternative perspectives on transformation, we reviewed literature on theories ranging from assimilation of agile practices, routinisation, change management, sensemaking, and institutionalisation, dynamic capabilities, to name a few. How large-scale agile transformation become normalised in practice is a key question that holds the key to unpacking the inner-workings and outcomes of concern of researchers and practitioners (Edison et al., 2021). Traditional IS theories have been proven to be useful to focus on a narrow specific part of large-scale agile transformations, yet we need alternative perspectives to examine an overarching transformation process. For example, Uludag et al. (2020: 30) call for 'future research endeavours should pay greater attention to building a solid theoretical scaffold for the observed phenomena in large-scale agile development'.
As is often the case with new and emerging phenomena in software development, practice has led research, with the creation, promotion, and dissemination of these large-scale agile methods largely due to the efforts of practitioners and consultants (Paasivaara, 2017). While this practice-driven nature certainly has merits, Rolland et al. (2016) caution that implementations are often built on weak assumptions which typically lack of empirical case studies explaining how largescale agile transformations fail (Dikert et al., 2016). While there are arguments around the need to develop an 'agile mindset' to change an organisation's culture, realistically this often take considerable time and effort, and organisations can fall back into old habits and fail to reap the full benefits of agility (Koutsikouri et al., 2020). A core motivation for this research is to address which emerged from our literature review, which was to explore why large-scale agile transformation failed in practice and to identify how specific events unfolded and prevented a large-scale agile method from becoming embedded and normalised in practice.

Research focus
Building on Struijk et al.'s (2022) advice this section presents the research focus and outlines how it appeals to IS scholars and practitioners. Specifically, in this study, we address two research questions on both 'how' and 'why' large-scale agile transformations fail. This study (i) presents a case study which goes beyond the implementation of a large-scale agile method to what we describe as the normalisation of a large-scale agile transformation, and (ii) describes what factors lead to a large-scale agile transformation failure. It is hoped that researchers and practitioners can learn from these insights to identify new approaches to better manage and sustain large-scale agile transformations. Specifically, we argue that normalisation therefore offers a novel and critical overarching lens to understand the process of large-scale agile transformations and the importance of managing the normalisation of new practices throughout an organisation's transformation journey. We provide a number of contributions to theory and practice of large-scale agile transformations.
The remainder of this paper is structured as follows. The next section presents a discussion on NPT. Following this, we describe the single case study methodology adopted for this research and the relevance of the large-scale agile transformation. The then presents the research findings. The article then presents a discussion on the main contributions (practical and theoretical) of this research. The final section which offers a conclusion and outlines some limitations of NPT and identifies new directions for future research opportunities.

Normalisation process theory
Normalisation process theory has been successfully employed as an explanatory framework to interpret the key processes at play from the initial phases of deployment to innovations being implemented, embedded, integrated, and evaluated within health science and implementation science (i.e. the study of evidence-based practices into routine practice, e.g., healthcare). For example, Table 1 presents a summary of some diverse studies which outlines how NPT has been applied and the reported value of adopting this theoretical lens.
As outlined in Table 1, there are many benefits of adopting NPT to investigate how does large-scale agile transformation become normalised in practice or what factors inhibited the normalisation process. This presents us with the required overarching theoretical perspective of large-scale agile transformation including the socio-technical insights (such as the team and organisational networks, culture, and technology) to embed and sustain the transformation process or the factors which inhibit this.
Normalisation process theory is a derivative sociological theory on the implementation, embedding, and integration of new technologies and organisational innovations ) which allows us to challenge assumptions around embedding change during transformations. Normalisation process theory also identifies factors that promote and inhibit the routine incorporation of complex interventions into everyday practice Murray et al., 2010). It focuses on providing practical insights on specific phenomena both qualitatively and quantitatively which can provide new insights on managing and sustaining large-scale agile transformation.
Normalisation process theory allows us to view transformations not just at the formal stage of their implementation such as a training period or switch-over to the technology or practice, but beyond this point into the subsequent period of time where an intervention becomes so embedded into routine practice that it 'disappears' from view (i.e. it is normalised) (Murray et al., 2010). It therefore allows us to adopt an alternative view by unpacking the dynamic nature of large-scale agile transformation. Normalisation process theory also allows us to focus on the social organisation of the work (implementation) of making practices routine elements of everyday life (embedding), and of sustaining embedded practices in their social context (integration) (May, 2006). In addition, NPT allows us to examine large-scale agile transformations and build a cumulative tradition on IS theories. Normalisation process theory examines the adoption of a new practice. As outlined in Table 2, NPT provides a rich theoretical lens to develop novel insights on the efforts (or lack of) to embed and sustain a large-scale agile transformation. It allows us to uncover whether practices become routinely embedded in their social contexts as the result of people working, individually and collectively, to enact them . Within each of the core theoretical constructs, we can examine the normalisation of large-scale agile transformations and assess patterns of work and outcomes which were imposed by the large-scale agile method. We argue that NPT supports the IS community to theorise about the transformation process which encapsulate the four theoretical constructs. Within each of the NPT core theoretical constructs, there are four additional components (16 in total) which provide a nuanced lens to explain large-scale agile transformations (outlined in Table 2).

Research design
Adopting a qualitative research approach utilises various research designs to obtain key insights from a range of data sources which normalisation processes typically generate. Given the under-researched nature of normalisation in largescale agile transformations, we employed a single in-depth case study approach (Yin, 1984). A case study examines a phenomenon in its natural setting, employing multiple methods of data collection to gather information from one or a number of entities (people, groups, or organisations). For the purposes of this study, we adopt a single-case study as a revelatory and unique case (Yin, 1984) which examines organisations' efforts to introduce a large-scale agile method and the process of managing and sustaining the transformation process.
For the purpose of this case study, we adopt Dikert et al. (2016) definition of large-scale development involving more than 50 developers or at least six teams working on a common product or project in the same organisation. This case study focuses on a global financial service organisation (FinanceCo 2 ) who employ approximately 1250 employees across the US, Ireland, India, and China. In the past, Fi-nanceCo have experimented with various agile methods and attempts to customise agile methods. Over a 2-year period (2017-2018), FinanceCo underwent a large-scale agile transformation strategy and adopted the Spotify model (scaling agile approach at Spotify) (Kniberg and Ivarsson, 2012). This exploratory single case study focuses on the Spotify model as the unit of analysis and uncovers individuals experience in adopting Spotify and their experience in managing and attempts to sustain the large-scale agile transformation process.

Data collection
Within FinanceCo, the Center for Technology (C4T) 3 was a team which initiated the adoption of the Spotify model and was responsible for managing the large-scale transformation The authors focus on the importance of understanding and assessing the conditions in which complex interventions of big data can be introduced and normalised in society. NPT offered a conceptualising of the complex interventions in practice regarding big data and focused on interactions within and between the processes of practice at the micro, meso, and macro levels. Luig et al. (2018) Healthcare Field notes of intervention sessions, logbooks of the practice facilitation team members, and semi-structured interviews This study examines the complexity, barriers, and recommendations of implementation processes within a multi-stakeholder environment. The authors explain key factors around collective and individual sensemaking. They also identify key factors around the adoption of the innovation, iterative evaluation of the implementation process, context-appropriate intervention impact, and stakeholder engagement. Murphy et al. (2021) Remote consulting during the COVID-19 pandemic Longitudinal observational quantitative analysis (interviews) This study describes the success of normalising new general practices around implementing predominantly remote consulting via telephone, video, or online consultation platforms. Table 2. Applying core theoretical constructs and components of NPT to large-scale agile transformation (adapted from Carroll et al., 2020).
Core construct of NPT Construct components of NPT and come to believe that they should be involved with and can make a valid contribution to a large-scale agile transformation process. 2.4 Activation: the process whereby members collectively explore and define the actions and procedures needed to sustain the new practices and commit to the vision of a large-scale agile transformation. 3. Collective Action: the operational work to enact a set of practices by adhering to a new large-scale agile method. This allows us to examine the specific practices associated with a large-scale agile method, new organising factors, and tools used to enact and sustain practices associated with a large-scale agile transformation.
3.1 Interactional Workability: team members' interactional work with each other, artefacts, and other elements in a set of practices, and efforts to operationalise and exploit a large-scale agile method in everyday settings. 3.2 Relational Integration: the process whereby people generate knowledge to build accountability and maintain confidence in a large-scale agile method. 3.3 Skillset Workability: the allocation of tasks underpinning the division of labour typically established around a new large-scale agile method as they are operationalised and exploited in a realworld context. 3.4 Contextual Integration: the resourcing and management of a set of practices through allocation of different kinds of resources and the execution of protocols, policies, and procedures with a large-scale agile method. (continued) process. To guide the data collection process, the researchers captured historical accounts of the key motivations to adopt the Spotify model. These are summarised as follows: (i) To establish a cross-functional team structure across Squads (see Figure 1); (ii) To ensure improved integration of teams across projects; (iii) To empower teams to instigate improvements; (iv) To encourage teams to become autonomous in practice; (v) To increase organisational productivity and performance.
Within C4T, eight squads were formed from across the US, Ireland, India, and China comprising of a specific squad name (pseudonym squad name presented here: S1-S8) squad lead, scrum master, business intelligence chapter, platform chapter, data chapter, and a product support chapter all of which led efforts on the large-scale agile transformation made up of 80 team members (see Table 3).
The researchers had a unique opportunity to observe and conduct 51 semi-structured interviews with 40 members across the 8 squads and 11 additional stakeholders (Table 4). Eleven interviews included the senior vice president of C4T, seven members of a guild within FinanceCo, and three external participants, two of which were software flow consultants, and one was a manager in the Spotify organisation. There are several strengths in interviewing and for the purposes of this study it permitted the researchers to move back and forth on our experiences and perceptions (Glaser and Strauss, 1967). The flexibility of the technique allowed us to probe, to clarify, and to create new questions based on what has already been heard and probing on key components outlined in NPT. Where possible, we also obtained evidence, for example, via document source images, internal report summaries, or screenshots to support interviewee claims.
The eight squads within C4T were very open in engaging with the researchers to document their large-scale agile transformation journey. The researchers were provided with access to C4T staff, FinanceCo email accounts, organisational tools, and a range of digital records (e.g. records on meetings and events, real-time data sources, and operational and strategic plans). The researchers were also introduced to their team members (see summary of interviewees Tables 3 and 4); agile project management systems (e.g. JIRA); dashboards and metrics (e.g. software flow metrics), team meetings, and other agile-related training events. Given the high level of access to C4T, the researchers could also obtain several sources of evidence across all eight Squads through documentation, archival records, interviews, performance, and productivity data. In addition, throughout the largescale agile transformation, the researchers had direct observation access to verify interview accounts over an 18-month period on approximately 33 projects (each project did not involve all eight Squads).
The relevance of the case study was evident as the researchers could document efforts to normalise a large-scale agile transformation. Prior to the adoption of the Spotify model, C4T experienced a number of challenges around managing software development at scale to improve productivity and performance across their global software teams. These high-level challenges were first outlined by management and the US-SVP delivered a presentation to outline how the Spotify model and introduction of software flow metrics would support to alleviate these challenges, which included, for example, In doing so, we can gain insights on the implications of new structures, team norms, and group processes, i.e., assessing work and outcomes.
4.1 Systematisation: evaluation of how effective and useful team members consider a large-scale agile method is for them and others, and ideas to build on future opportunities. 4.2 Communal appraisal: collaborative appraisal (formal or informal) by team members of the value of a set of practices imposed by a large-scale agile method through experiential and systematised approaches. 4.3 Individual appraisal: appraisal by individual team members working with a new set of practices of its effects on them and how to exploit opportunities for performance improvement through a large-scale agile method. 4.4 Reconfiguration: appraisal by individuals or groups that may lead to attempts to redefine procedures, modify practices during a large-scale agile method, and exploit emerging technology capabilities for future performance improvement.
(i) the need for improved management of the product model; (ii) the need to remove silos and team inefficiencies; (iii) the need to develop new capabilities and processes; (iv) the need for increased transparency and communication; and (v) the need to improve productivity and performance; The researchers were also invited to deliver regular workshop, for example, to present on our research developments, outline impediments to software flow, and design thinking for large-scale agile transformations. Guided by NPT, data was collected and analysed with a view to explore efforts to normalise the Spotify model and what factors contributed towards the adoption of the Spotify model.

Data analysis
Guided by the NPT framework, the researchers analysed the data guided by four key constructs (i) coherence; (ii) cognitive participation; (iii) collective action; and (iv) reflective monitoring while remaining open to including new constructs based on the data. The interviews allowed us to get an exchange of views between the researcher and interviewees on the large-scale agile transformation. Over the 18-month period, the researchers had built rapport and trust with the interviewee to examine the participants 'inner world of meaning and feelings, as well as their experienced social reality' (Schultze and Avital, 2011). The NPT framework provided a guide for the researchers to question and interpret their experiences and structure the conversation while honouring their freedom of thought and expression. As explained by Schultze and Avital (2011: 5), 'by structuring the interview interaction, such frameworks help the interviewee as well as the researcher surface detail-rich descriptions as well as their significance and meaning…. As such, they assist in the generation of rich data as the interviewees are guided in accessing multiple layers of experience'. We reviewed the interview transcripts and categorised key insights into the four NPT constructs and the 16 comprehensive components (Table 2). Analysis was necessary from the first interview because it was used to support and direct the next interview and observations. Normalisation process theory also provided a diagnostic frame to categorise various benefits, challenges, and recommendations when interviewees articulated their experiences with the large-scale agile transformation. As illustrated in Figure 2, we employed open coding to identify the relationships among interview statements and observations with NPT construct categories. We then employed axial coding to examine how NPT component categories were related to their subcategories and their relationships before moving to selective coding to understand efforts to normalise a largescale agile transformation.
Open coding was the first step in the analysis of our interview transcription. We split the data into discrete parts and create specific 'codes' by labelling the general nature of the text, for example, 'rationale to transform', 'measuring progress', or 'team structures'. This allowed us to be open to new theoretical possibilities as we first examined the interview data. Over the 18-month period, we could go back and forth on the data and labelling to compare events as they evolved during the large-scale agile transformation. This also allowed us to remove preconceived ideas around largescale agile transformation and probed the researchers to continuously question the data. The second major phase of our research focused on axial coding where we began to categorise the data into certain insights. We created 'concept buckets' and grouped insights using NPT constructs which allowed us to also identify connections between codes to better explain the normalisation of a large-scale agile transformation. The researchers also examined how the theory-informed conceptualisations of events from various sources facilitated a multi-layered visual mapping of the insights (Langley, 1999).
In addition, Figure 3 below provides examples illustrating the logic of how the case study sources provided insights on the normalisation of a large-scale agile transformation.

Findings
This section summarises the key findings on FinanceCo large-scale agile transformation using the Spotify model. The findings are presented under the theoretical constructs of NPT (listed in Table 2) to examine what actions enabled or inhibited the normalisation of a large-scale agile transformation. The Spotify model imposed new team structures for C4T. The new team structure was made up of eight Squads and the creation of new roles such as squad leads, chapters, and production support across the C4T tribe. The overall philosophy and principles for the new team structures were that each Squad are autonomous, self-organising, selfmanaging, and empowered to select the best way to work on an allocated number of epics (a large body of work that can be broken down into a number of smaller stories), using various scrum sprints, Kanban, or a hybrid of approaches. The Squads (comprising of a Squad Lead, and Scrum Master) represented key strategic elements at Fi-nanceCo and their vision for the large-scale agile transformation. For example, squads represented new organisational initiatives such as moving towards cloud, mobile, and new software flow metrics (a data analytics initiative).

Coherence of the large-scale agile transformation
This section describes the insights on the transformation process which individuals and teams experienced with examining the rationale to adopt a large-scale agile transformation using the Spotify model. Specifically, our findings identify the challenges of developing a shared understanding on the rationale, differences, value, and roles associated with planning to undergo the transformation process, and the approach in managing the Spotify model. In doing so, it identifies how FinanceCo had a relatively unstable foundation and a lack of clarity around the rationale to undergo a large-scale agile transformation. As a consequence, our findings drew connections to normalisation phases and how certain factors began to unravel, making the transformation process unsustainable.
Differentiation between new Spotify model and customised agile model. Differentiation captured a comparative view  between an old and new set of practices and drew focus around the rationale for changes imposed by the Spotify model. However, within FinanceCo, the findings indicate that there was a lack of clarity and communication across C4T to distinguish what specifically differentiated the Spotify model from the previous customised variants of agile practices. Specifically, there was a lack of evidence around (i) the rationale to undergo a large-scale transformation; (ii) the benefit or value-add of adopting the Spotify model as part of the large-scale agile transformation strategy; and (iii) the need for new software flow metrics to measure squad performance and productivity. It was clear that the C4T were results-driven but so much so that little focus on how the Spoofy method differentiates the overall work culture and network.
Our focus on differentiation led to two key findings at the very early stages around normalising the transformation process. First, the management team failed to differentiate and communicate the value of adopting the Spotify model in tangible terms. Second, the squads never 'bought into' the rationale for a large-scale transformation. Management only considered the high-level differences with the Spotify model and used 'sweeping claims' to promote the potential of the method. For example, management proposed that the large-scale agile transformation would address some business challenges such as (i) the need to improve the software development culture, (ii) improve fluidity of squad structures and collaboration, and (iii) improve continuous software development flow throughout the organisation. For example, the C4T Senior Vice President (US-SVP) explained that: 'The Spotify model is different because it will allow us to improve the speed of our development if we can layer it on top of FinanceCo's development cultureplacing more value on autonomy, agile processes independence, democratic teams, and servant leadership'.
However, from a squad perspective, there was a growing sense of uncertainty around operationalising the Spotify model and dissatisfaction around the new proposed team structure changes. In addition, squad members learned that as part of the large-scale agile transformation process, software flow metrics (Vacanti and Vallet, 2014) would be introduced. As described by a Systems Analyst (IE-S2.3), it was envisaged that 'software flow metrics would provide new insights on team and individual productivity and performance rates with a view to better manage the "flow" and continuous software development'. In the initial stages of the transformation, the interviewees described how there was a narrow focus on the potential of software flow metrics as a data-driven large-scale agile transformation process. For example, during a retrospective event, a Senior Software Engineer in the Data Chapter (IE-S4.6) reported that: 'There are many metrics reporting on activities which support the overall large-scale agile transformation process, but do we spend more time on activities that are measured or activities that actually drive value?' Communal specification of the Spotify model. Communal specification focused on how squads built a shared understanding of the Spotify model and the overall transformation process. Our findings indicate how this raised some of the core tensions between the management perspective and software developers' understanding of the transformation process across C4T. For example, Squads initially perceived that the Spotify model imposed disruptive changes to C4T's divisional structure which led to separation of powers. From a management perspective, team restructuring was considered necessary in an effort to build and sustain more efficient work patterns for self-organised teams to enjoy autonomy and drive the transformation process.
The C4T Tribe was established as a vertically integrated silo with specific homogenous skillsets. Team structures were reconfigured across horizontal groupings for specific projects. Yet, the introduction of a Tribe and Squads within C4T altered the dynamics of teams, their working relationships, and the cultural norms at FinanceCo. Interviewees described how this led to resistance towards the Spotify model and its newly imposed work practices. For example, the Squad Lead of S2 (IE-S2.1) viewed the Spotify model as: 'It's difficult to make sense of the Spotify model. You want certainty, predictivity, and control on the management side. Yet, you adopt the Spotify model because you have admitted that you don't want predictability or control for the transformation process. So, there is a disparity in terms of reporting mechanisms and insight, and they bring with them, two different cultural norms on relational work'.
In an effort to understand the Squads experiences with the Spotify model, a Senior Project Manager (IE-SPM) distributed an online survey (entitled 'Road to Agility') to assess the impact of the large-scale agile transformation. Their findings relating to communal specification (i.e. a shared understanding) included (i) the lack of a clear shared vision; (ii) a reduction in motivation and joy in working together; (iii) increased technical debt; (iv) bureaucracy and time consumed through increased agile ceremonies (i.e. meetings with defined lengths, frequencies, and goals); (v) need for a singular definition of 'done'; and (vi) the lack of reward for compliant behaviours. However, our findings indicate that managers did not act on the survey findings to implement changes which further compounded tensions associated with the largescale agile transformation process.
Individual specification on implications of Spotify model. By focusing on individual specification, we identified how the sensemaking process was largely influenced by software flow metrics. From a managerial perspective, the Spotify model provided a transformation guardrail and roadmap that determined new roles and responsibilities required to improve organisational-wide agility and software team performance and productivity. Software flow metrics presented a novel way to 'monitor and measure agility' (SFC1). For example, the Senior Vice President (US-SVP) stated that: 'We often talk about being agile but within a large-scale context, how agile are we organisation-wide? When we explored the Spotify model, we saw how we could scale agile beyond the team-level to the wider organisation which brings in new principles and practices across roles'.
At a Squad level, team members perceived that the Spotify model introduced new metrics for continuous improvement for productivity and performance. However, as a software developer within the Platform Chapter of S3 (IE-S3.7) explained: 'not only are we forced to change roles but now we are held to account to reach new performance targets in these roles…'.
Internalisation on the transformation process. By focusing on internalisation, our findings focused on the value, benefits, and importance of the Spotify model to deliver financial, business, cultural, and/or personal benefits. However, the concepts of 'value', 'benefits', and 'importance' were largely considered to be vague or elusive terms from both management and Squad perspectives. Rather, stakeholders were typically guided by software flow metrics to determine the value, benefits, and importance of productivity and performance rates of completing various projects. Although the Spotify model was promoted by management as a means to adopt a new culture around team autonomy and empowerment, a Senior Software Engineer within the Data Chapter of S6 (US-S6.6) explained: '…with the Spotify model, you have to remember, wherever you have a tribe, you must also have a chief! If autonomy is monitored and measured, how can you give the illusion of working in accordance with our own beliefs, values and interests in the Squad. You still have to cooperate and be counted'.
Squad members questioned the value and benefits of Spotify with the growing bureaucracy of agile ceremonies and how management imposed new rules about the structure and frequency of meetings ( Figure 4). For example, a Squad Lead (IE-S1.1) explained how the ceremonies often: '…got in the way of work and almost goes against the grain from how Spotify establishes the notion of self-organisation or empowerment of Squads'.
Internalisation comparisons of 'being agile versus doing agile' (IN-S8.14) also brought about increased unplanned work across project. Interviewees explained how this raised increased doubts about the value, benefits, and importance of a large-scale agile transformation across the Tribe. This was best summed-up by a Senior Business Intelligence Developer in the Business Intelligence Chapter (CN-S7.3) as: 'Our progress, or lack of progress, is probably best reflected in the amount of unplanned work we are faced with which makes it difficult to understand the value of using the Spotify model. If we only relied on metrics to guide progress, we wouldn't take on any unplanned work, but in reality, we have to. This tells a different story on the overall large-scale transformation progress and metrics telling one story, but we have a job to do?' Our findings identified how the increased level of unplanned work items led to growing frustrations across C4T at the early stage of the transformation process. For example, the Business Intelligence Developer in the Business Intelligence Chapter (IE-S3.3) questioned the value, benefits, and importance of the transformation: 'Was this Spotify model meant to fit our needs or as I see it, we are tailoring the organisation to fit around this method? I think we're putting huge efforts into fitting ourselves around the method and metrics and you'll see this in our productivity trends so I'm not sure where the real value is. We can see progress but at what cost?'

Cognitive participation throughout the large-scale agile transformation
In this section, cognitive participation examines the relational work that C4T did in order to build and sustain the Spotify model and the large-scale agile transformation. While management promoted the notion of team autonomy and empowerment, the findings indicate that FinanceCo over-relied on adherence to the Spotify model to ensure data-driven progress throughout the large-scale agile transformation rather than focus on or understanding the overall sustainability of the transformation process. Across C4T, the Spotify model introduced new team members, roles, responsibilities, and divisions of labour to support the transformation process. Although Squads had initially adapted to a new way of working, their enthusiasm towards the Spotify model began to dwindle as the transformation matured.
Initiation of a transformation process. Initiation focuses on examining how team members contributed towards the large-scale agile transformation process. Within FinanceCo, C4T were the initiators of the large-scale agile transformation. The Senior Vice President and middle management promoted how Squads should be considered to be autonomous, self-organising, self-managing, and empowered to action change. However, interviewees explained how the introduction of new roles were not clearly defined and lacked clarity on how specific roles contributed to the implementation of and adherence to the Spotify model. For example, a Software Developer (IE-S2.9) explained that: 'I have been working here for the last 5 years and this new method [Spotify] means I have to change how I work…and where I now sit in the room (in various Squads). Realistically, how does this change how I will develop software?' In addition, while software flow metrics introduced new measures, it also introduced a new vocabulary about how team members were contributing to the large-scale agile transformation process. For example, there was a focus on performance measurement such as speed to complete projects rather than provide insights on how contributions to the transformation were progressing towards the 'the success of Spotify or how close we were getting to the final destination' (Software Developer; US-S6.7). As one Scrum Master (CN-S7.2) described it: 'Metrics are fine but it's almost as if we are focusing on the dashboard of a car and discussing how fast we are going, how much gas is in the tank, how many kilometres we have travelled, etc. But nobody is talking to us about the map and the journey we have been on and the destination we are headed towards. Metrics can't really achieve this for the transformation processwe need the bigger picture from time-to-time'.
Enrolment into the Spotify model. Enrolment describes how team members became involved in the large-scale agile transformation process with the Spotify model. Our research indicates that during the early stages of large-scale agile transformation, senior staff members within C4T experimented with various alternative team structures to improve on delivering improved quality. These efforts were guided by software flow metrics. However, contrary to the notion of creating a culture of autonomy and empowerment (discussed in internalisation and initiation), interviewees explained that team members were not provided with the opportunity to restructure and enrol in alternative Squads, nor afforded the opportunity to evolve into new Squads. For example, a Senior Technical Manager (IE-S4.1) sent an email to C4T members with a visual seating plan and explained: 'I know that the seating was originally supposed to be free seating, but let's use this seating plan as a starting point. There is no issue with people moving around within their coloured squad area, but you are required to remain in your Squad…' Although Squad members had committed to new team structures, it became extremely challenging for them to explain how the Spotify model contributed towards sustaining the large-scale agile transformation. For example, a Project Manager (US-S6.1) described: 'We regularly hear from senior management that it is vital to be involved and keep on track for the transformation process using metrics. But I don't really know what the endgame is or what success looks like? I'm frustrated trying to juggle and constantly reprioritise tasksand we don't have a roadmap. If I can't connect the dots in terms of commitment, how will my team members do so (locally and globally)?' Another Software Engineer (US-S5.6) explained their concerns around the lack of opportunity to have any input on how they should enrol on events for the newly transformed agile work practices: 'I know that agile promotes the need to be collaborative but not all collaboration is productive especially at a large-scale. I'm frustrated with all the [global] meetings, ceremonies, and checkbox exercises we have to endure which, I can see effects how I now engage with colleagues'.
Legitimation of the Spotify model. Legitimation focuses on whether Squad members believed they could make a valid contribution to the large-scale transformation process with the Spotify model. During the transformation process, sprint planning played an important role in supporting Squads to determine the product backlog, assigned contributions from squads, and completion plans for product backlog items. However, sprint planning was reported to be one of the most 'drawn out processes' (IE-S1.8). In addition, our findings indicate that Scrum ceremonies would consume one-to-two hours of discussions on how team members should participate rather than being an opportunity for Squads 'to air issues and discuss ways to better contribute as a team' (IE-S3.10). For example, a Senior Software Developer (IN-S8.11) explained that: 'The large nature of our team means we cannot frequently get everyone together for preparations and technical refinements. This is demonstrated by our relentless ceremonies which reduces our contribution to delivering product and sustaining the transformation but spend time discussing what we could do'.
However, championing the overall transformation process and the Spotify model ceremonies did prove to be a key factor to ensure maximum buy-in across Squads. For example, the Senior Vice President (US-SVP) described how: 'Having identified the potential for Spotify to scale here; we need to sell it at an executive level to ensure full participation and contribution by demonstrating how it not only meets our needs locally, but how other global sites and business units should follow-suit to transform at a large-scale. A key thing is to maintain this momentum and goodwill which is not that straightforward'. One approach to maintain and monitor the momentum of 'valid contributions' was demonstrated around process and productivity improvements through software flow metrics. The research participants referred to how C4T developed a 'software flow metrics cookbook' to prescribe how Squads should contribute and self-assess their contributions across planned priority projects.
Activation of the Spotify model. Activation refers to how the Squads collectively defined the actions and procedures needed to adhere to the Spotify model. It also examines initiatives teams adopted to stay committed to the largescale agile transformation vision. The findings reveal that there was an initial sense that the large-scale agile transformation would offer some flexibility on what procedures may be adopted. Yet, the Spotify model imposed specific work structures and the reallocation of projects and team members. For example, a Senior System Analyst (IE-S3.4) explained that: 'Due to restructuring, some projects and parts were moved to other global locations, and we also inherited other projects. Team members had to move to certain divisions too. So, they were here one week and gone the next week to work for and report to a totally different business unit'.
The interviewees explained that FinanceCo needed to balance autonomy with accountability of Squads through the introduction of 'measurable objectives, consistent metrics for progress, and feedback systems to monitor activities' (Squad Lead -US-S7.1). Therefore, software flow metrics played a key role in collectively redefining actions and procedures and report whether Squads reached or failed to reach specific goals. As one Senior Project Manager (IE-SPM) put it: 'The flow metrics were useful to inform us how we were progressing. If progressing well, we'd have a conversation on what worked well and establish best practice. Equally, if we were not reaching targets, we would meet to discuss what wasn't working well and experiment with alternative options for actions and procedures'.
In addition, a Software Engineer (IN-S7.9) described their experience from the initiation of the Spotify model for the transformation and metrics by explaining that: 'We are told that as a Squad, we are expected to self-organise, and determine the best way to work. But after investing significantly in new tools we are also expected to tailor the approach around project management tools…and now new metrics for new actions and procedures during the large-scale transformation?' Management formed Squads based on individual team members expertise (e.g. cloud computing, analytics, or cybersecurity). However, the level of differentiation between junior and senior level of expertise in each Squad proved to be a major obstacle throughout the transformation process. For example, a Senior Systems Analyst explained that (IE-S3.1): 'For people, such as those at an executive level, who have little experience in transforming how we operate, it may feel like progress. For me, I feel that I have slowed down and am getting nothing done because of the disparity in experience with developers across the global Tribe'.
To fully adhere to the Spotify model, C4T defined the actions, procedures, and performance targets through various software flow metrics. By monitoring actions and adherence levels during the large-scale agile transformation via software flow metrics and interactive dashboards (i.e. Tableau), Squads could determine their levels of efficiency. For example, Squads could examine throughput (see Figure 5) and drill-down on adherence to procedures.

Collective action during the large-scale agile transformation
This section examines the relational work to build and sustain the large-scale agile transformation at FinanceCo. Across C4T, there was considerable focus on exploiting software flow metrics to inform the operationalisation of the Spotify model (e.g. faster team productivity rates). However, there was a lack of clarity on how the metrics linked to the large-scale agile transformation strategy or how success could be measured.
Interactional workability of squads. Interactional workability explores the work that Squad members collaborated on, their use of artefacts, and other practices as prescribed by the Spotify model. Although team autonomy and empowerment were promoted by management, according to one Platform Chapter member (IN-S8.7): '…autonomy proved to be a double-edged sword. On the one hand, it spurred on problem-solving and a desire to selforganise. On the other hand, autonomy was micro-managed through various software flow metrics to avoid ambiguity around performance and inefficiencies of productivity'.
Interviewees explained that the challenge for C4T was to strike the right balance between the management process and autonomy of Squads. However, issues relating to coherence generated 'mixed messages' as to 'how' and 'why' teams should collaborate under the Spotify model.
For example, Squad members reported that the Spotify model hampered their ability to sustain working relationships or sufficiently complete tasks during the largescale agile transformation. In addition, problems emerged due to the lack of communication, varied interpretation of metrics, and the growing expectations around the Spotify model.
The Spotify model introduced new techniques (e.g. software flow metrics and Kanban boards), new practices (e.g. new processes and agile ceremonies), and procedures. Squads had to accommodate and adapt to new ways of working. For example, a new metric called 'touch time' was adopted to monitor the specific time during the process whereby the product was worked on, and when the value is being added as opposed to the overall production time (in backlog, queuing, etc.). Touch time was introduced to allow squads to monitor individual interaction with a specific story and identify ways to improve speed. We can identify how touch time increased ( Figure 6) after the initiation of the large-scale agile transformation (using the Spotify model) in Quarter 1 of 2017. However, we can also identify the growing demand on C4T to sustain the transformation process through increased interactional workability (from approximately 375 days before the large-scale agile transformation to 610 days after the adoption of the Spotify model).
Relational integration for the transformation process. Relational integration examines collaboration and training across C4T and efforts to build accountability to maintain confidence in the large-scale agile transformation. Within FinanceCo, Squad accountability for performance and productivity was captured through software flow metrics. Yet, interviewees explained that confidence around the benefits of adhering to the Spotify model was regularly undermined and diminished throughout the transformation process. For example, Squads raised concerns regarding the integrity of new and existing metrics during regular team meetings. For example, a Senior Software Developer (IE-S2.7) explained that: 'The metrics are telling us we are getting faster, but is faster better? Many of us on the team feel like we're just on a hamster wheel with growing expectations to support the large-scale transformation, focusing on how fast we work, and moving products to "Done"'.
In addition, a Software Developer (IE-S1.8) described this as: I feel we are losing some sense of accountability as it became the responsibility of a Tribe for certain deliverables. When delays occur, blame can then quickly shift across the crossfunctional teams making it really difficult to identify the effectiveness of specific team members, tools, and decisionmaking to remove impediments.
Skillset workability of the Spotify model. Skillset workability explores how C4T allocated tasks which led to the division of labour, that is, new roles and responsibilities in adherence to the Spotify model. The majority of team members across Squads expressed concerns about the delegation of new responsibilities for the large-scale agile transformation process. Interviewees explained how the Spotify model does not promote standardisation, but rather the 'need for cross-pollination of actions to transform practices' (MS). Yet, within FinanceCo, software flow metrics offered actionable insights for adherence and standardisation of practice. By actionable, C4T could learn where to improve based on the software flow results and identify interventions for the transformation process. The rationale for adopting transparent and actionable flow metrics meant that practices met less resistance when Squads tried to adopt them and later became the default standard. To determine the Squads performance, status mapping throughout the large-scale agile transformation played a key role to identify and expose strengths and weakness. For example, as explained by a Senior Software Engineer (S8.5): 'Our cycle time is important because it indicates the average time it took us to start and finish our stories. The lower the cycle time, the smoother we view our operations to be throughout the transformation'.
Contextual integration of the Spotify model. Contextual integration examines the resourcing of work, that is, managing the Spotify model through the allocation of resources to execute new protocols, policies, and procedures. Within C4T, the reallocation of resources for the transformation process was challenging to plan for given the 'unchartered territory a large-scale agile transformation presents' (IE-DSP). For example, interviewees explained that there was a lack of financial investment for skills development and recruitment. Additional focus was placed on reorganising teams across the Tribe and approaches to prioritise the use of resources. For example, a Business Analyst (IE-S4.3) explained that: 'Defining a team is challenging as team members move from project to project. We try to focus on our top 15 high accountability and high impact projects to focus our effort and resources, but these priorities change so often that we can't simply pin down fixed resources to see any one project out to completion'.
One Senior Project Manager (IE-SPM) explained the importance of software flow metrics to guide resource allocation and learning from other teams: 'Having resources across a team is well and good but looking at the high priority projects, flow metrics assisted us to identify strong teams, who should support other teams increase their work-rate, and how they should tweak things to ensure an improved flow of work during the transformation'.
As part of the transformation strategy, the Senior Vice President (US-SVP) also placed emphasis on transparency of procedures as being a key factor of contextual integration: 'Transparency alone initiates improvements. Once software flow data became assessable, visible and presented in a manner in which teams consume and learn how they were performing, they instinctively put actions in place to improve the large-scale transformation process'.

Reflexive monitoring of the large-scale agile transformation
Reflexive monitoring examined how Squads appraised the new set of practices in the Spotify model for the large-scale agile transformation. Within C4T, management were embarking on their 'transformational journey' but had little training and expectations on how the transformation process would impact the organisation. For example, the Senior Vice President (US-SVP) expressed concern about the lack of executive training to appraise the overall large-scale agile transformation process: 'There are so many opportunities for training on how to operationalise the Spotify model such as for Scrum Masters, oneday training programs, and other events, but we don't have any training programmes at an executive level to actually manage and evaluate the large-scale transformation process'.
Systemisation of the Spotify model. Systematisation examined how Squads determined the effectiveness and usefulness of the Spotify model for them and their colleagues during the large-scale agile transformation. However, our findings indicate that appraising the effectiveness and usefulness of the large-scale transformation for C4T proved to be a major challenge especially from different stakeholder perspectives. Prioritising the use of software flow metrics constrained C4T's view of success to focus on narrow shortterm (daily and weekly) performance. For example, software flow metrics focused on cycle time, throughput, hands-on-keyboards, and epics and sprint reports with little insight as to how their efforts contributed towards the overall long-term large-scale agile transformation. In addition, software flow metrics focused on the flow of work as opposed to the business objectives which the transformation was initially proposed to be strategically aligned with. For example, a Senior Systems Analyst (IE-S1.2) summed it up by explaining: 'Management is preoccupied by going back to the product model, but nobody can tell me what the product model is, or how much money is being spent on product X, how much effort is going into product X, cost centre of business unit's version of work. More importantly, how does our work contribute to the overall transformation strategy?' Therefore, systemisation primarily focused on gathering software productivity data on C4T Squad performance levels. C4T also gathered anecdotal sources of information on problems associated with operationalising practices for both individual and collective appraisal. One example included what was described as 'a calling out process of ineffective practices during Scrum events' (Squad Lead -US-S8.1). According to one Systems Analyst (IE-S2.3), a key issue with a focus on software flow metrics to adhere to the Spotify model was that: 'Assigning issues may be some form of bad practice but the metrics are accommodating flow rather than informing best practice? So, we can go fast but we might crash, or should we go slower, be careful, and arrive safely?' With an emphasis on short-term planning rather than long-term planning for the large-scale agile transformation, our findings indicate that this gave rise to negative sentiments towards the Spotify model at the early transformation stage. For example, a Software Engineer (US-S5.2/US-6.2) explained that: 'Our metrics and key performance indicators inform us that we are getting "faster" which must be a good story for the transformation process. But I am not happy in my work. Many of my colleagues feel the same and there is a lower morale than that of pre-Spotify'.
Communal appraisal of the Spotify model. Communal appraisal examined how squads collaborated (formally or informally) to evaluate the value of the Spotify model, for example, through team meetings, and informally over lunch, coffee breaks, and water cooler conversations. Discussions typically focused on (i) the value of specific actions, (ii) the rationale to introduce new software flow metrics, and (iii) reflections on how new processes or practices may affect changes to their work practices. However, the interviewees explained that knowledgesharing on process improvements across the Tribe was problematic and often location-specific. For example, during one of the daily stand-up meetings, a Director of Technology based in the US (US-SSI) suggested that: 'there is a lot of innovation going on in Ireland and it would be great to share across the Tribe if we are to transform together?' Some Squad members also began to question the benefit of improving Squad performance rates. For example, a Senior Software Engineer (IE-S4.8) referred to a growing paradoxical sense that: 'We are self-organised to a point. We can drive our own performance. But by reaching new performance targets or levels through those new Spotify team structures, I do feel we are victims of our own success. Performance is only part of the transformation storythere must be some feedback on the value on what and how or why we operate in this way?' Individual appraisal of the Spotify model. Individual appraisal explored how Squads appraised the effects of the Spotify model on them during the transformation process. For example, we learned about (i) the steep learning curve team members experienced to operate within and adhere to the Spotify model; (ii) the increased workload due to planned and unplanned items; and (iii) lowering of team morale as a result of growing performance targets during the large-scale agile transformation. This was often described as 'the disconnect' between the large-scale agile transformation strategy and how it was operationalised in practice. For example, the Director of Strategy & Planning (IE-DSP) cautioned during a strategy planning event that: 'Planning and delivery on one side of the organisation goes on but then there is the execution on another side of the organisation but there is a real disjoin between the two making it challenging to evaluate the value of the transformation'.
C4T members also appraised the effects of Spotify on themselves. For example, speed was an important measure across the software flow metrics. Defining and monitoring productivity metrics went through a number of iterations to evaluate, for example, (i) speed of individuals and Squads, (ii) individual contribution, (iii) compliance rates, (iii) cycle time, and (iv) hand-on-keyboards. However, our findings indicate that many of these measures were regularly challenged in a constructive manner by Squad members throughout the transformation process. In addition, a Scrum Master overseeing the Spotify model did not only question the value of the method as proposed by Spotify literature but also questioned their overall impact on individuals' roles and responsibilities. As the Scrum Master (CN-S7.2/CN-8.2) explained: 'I am also tasked with team evaluations and examine how individuals typically evaluate themselves and colleagues during the large-scale transformation. This provides me with a point of reference in the hope of getting a more accurate picture of the team and an agile compatible appraisal of team members, but it doesn't seem to be going in a positive direction. The challenge is: who wants to hear this insight?' Individual specification was largely influenced by the software flow metrics to gauge progress and improvements rather than a measure of transforming. The interviewees explained how software flow metrics were initially 'based on Little's Lawwhich only works if we have a balanced system (i.e., we start work at the same rate we finish it' (extract from FinanceCo's 'Software Flow Metrics Cookbook'). However, from an individual perspective, software flow metrics were regularly undermined due to poor team performance ratings which failed to accommodate for evolving management performance targets (described as 'moving goalposts') and the Squads learning needs.
Reconfiguration of practice under the Spotify model. Reconfiguration examined how Squad members' appraisals led to attempts to redefine procedures or reorganise practices around the Spotify model. The appraisal process led to many attempts by C4T Squads to change and improve processes which were heavily influenced by the use of actionable software flow metrics. While Squads could experiment to improve the ways they worked, Squad Leads explained how they could also capture the relevant data about software processes to reduce complexity and increase the flow efficiency. This principle became known as 'process independence'. Process independence described a way to preserve independence of Squads to self-organise and optimise the way they worked to suit the specific problem, technology, customer, and general context that each face. Moving towards actionable metrics allowed C4T to understand how diverse workforces were managed and how they reconfigured some form of process change or revised metrics. For example, managers within FinanceCo regularly revisited their agile practices and the metrics to report performance and explore whether restructuring teams or processes could yield better results. However, it was reported by the Senior Vice President (US-SVP) that 'the frequency of doing so could undermine the transformation process'. Software flow metrics continued to provide increased transparency as management and Squads examined reconfiguration through key performance-related questions, such as (BI Chapter -IE-S1.3): 'how are we performing and where can we improve?' Executive management promoted the notion that actionable flow metrics empowered squads at FinanceCo to 'own their own performance through democratised decision-making' (US-SVP).
To accommodate for reconfigurations across a diverse workforce, actionable flow metrics encouraged managers and Squads to openly discuss or debate the benefits or drawbacks associated with various agile practices and analytics. There were concerns amongst manager as to whether software flow metrics provided an accurate reflection of current practice. To illustrate this point, C4T developed a control chart which reported the cycle time for a product, version, or sprint. This was used to support teams identify whether data from the current process can be used to determine future performance (Figure 7). The chart reports on the visibility, efficiency, and predictability of team performance. Visibility examines the outliers and investigates their cause to reduce them in the future. Efficiency focuses on decreasing rolling average (blue line) which indicates process improvements and increased throughput. Predictability provides a focus on a narrow standard deviation (light blue fill) through process improvement to improve the predictability of cycle time. As an example, from 27 May 2018 to 19 August 2018, Figure 7 reports an average (red line) of 2 weeks, 4 days, and 10 hours and 8 issues. However, cycle time increased during the largescale agile transformation process. Yet, Squads were encouraged to take autonomy and drive change rather than evaluate best practice for the overall transformation process to address issues surrounding increased cycle time. For example, a Director of Technology in the US (US-SSI) expressed the need for team members to correct inefficiencies: 'If you see a problem, take the autonomy to make change to makes things move fast!'

Discussion
This section presents a discussion on the main contributions of this research (practical and theoretical) for the normalisation of a large-scale agile transformation. We explain the importance of our theoretical and practical contribution and are guided by Struijk et al.'s (2022) questions: 'what would the world miss if your manuscript was never published?' and 'what would other IS scholars get wrong if they never read your paper?' In doing so, we expand discussion to explain the novelty of NPT and complex socio-technical interdependencies associated with large-scale agile transformations.
From a theoretical perspective, NPT guided this research to determine efforts by C4T to normalise the Spotify model and software flow metrics as part of their large-scale agile transformation over the 18-month period. However, our findings identify how FinanceCo failed to normalise the Spotify model and how the large-scale agile transformation began to unravel and fail within an 18-month period. Specifically, our analysis indicates that large-scale agile transformation methods placed considerable emphasis on (a) assessing adherence over sustainability to the Spotify model and (b) guiding a short-term view of the transformation process by monitoring software team productivity and performance through the use of software flow metrics. As a reflection through the findings, Table 5 summarises the key normalisation components and maps out whether they were considered by management or Squads during the large-scale agile transformation process. What this reveals is a lack of focus and responsibility taken by management to place efforts on normalising the large-scale agile transformation. This is important since it specifically draws focus on what was described in the section on Normalisation Process Theory as 'the work that actors do as they engage with some ensemble of activities and by which means it becomes routinely embedded…'' (May and Finch, 2009: 540). This is also important because we can identify a pattern of inconsistencies preventing the Spotify model from becoming embedded and sustained in practice.
We also note that while scalability impacts on the operations within an organisation, it also impacts on outcomes, team structures, and related factors. Whether organisations are at a very early stage of a large-scale agile transformation, or a more mature stage, it is critical to ensure the continuity of business. Some leaders may embrace the challenge of a large-scale agile transformation while others may be more cautious and risk adverse. Regardless of the leadership style, large-scale agile transformations require careful preparation, and committed leadership which a clear understanding of how the transformation process will impact on people (e.g. team structures, networks, and relationships), processes (e.g. coordinating teams, customer engagement, monitoring performance, and productivity metrics), and practices (e.g. agile methods and organisational culture). Understanding these factors will also influence the normalisation of large-scale agile transformations.
From a practical perspective, the FinanceCo case study demonstrated how they failed to consider or act on key normalisation components while other NPT components were planned for in practice. By mapping out the NPT theoretical components against the practical factors, we reflected on how FinanceCo failed to sustain a large-scale agile transformation using the Spotify model (Table 5). It is evident that the large-scale transformation in this case study was neither top-down driven nor bottom-up driven but rather a loose mix of efforts to drive changes as opposed to successfully undergo a large-scale agile transformation. By failing to normalise the Spotify model, efforts surrounding the large-scale agile transformation process began to unravel at the early stages (i.e. coherence) based on weak assumptions and rationale for the transformation process.

Practical contributions
We revisit each of the four main normalisation phases and reflect on the strengths and weakness of FinanceCo's normalisation efforts to outline some key recommendations for each phase for transformations.
Recommendations on coherence for large-scale agile transformations. The rationale to undergo a large-scale transformation can create a lot of uncertainty among stakeholders (Berntzen et al., 2019;Dingsøyr et al., 2018;Uludag et al., 2018). For example, at the early stages of the transformation process, the management team in C4T promoted high-level expectations on how the Spotify model would improve software productivity and performance. Promoting abstract transformation expectations hampered the execution and evidence-based approach to differentiate between new large-scale agile practices and previous agile practices. Layering a large-scale agile method with new forms of metrics and with little guidance on how they align or contribute to large-scale agile transformations was also met with resistance across Squads. It also became apparent that there was an overemphasis on developing metrics to measure the large-scale transformation progress as opposed to determining the best way to tailor and operationalise the transformation to meet FinanceCo strategic needs. For example, the Squad Lead for S4 (IE-S4.1) explained that: 'It's not about process, it shouldn't be about anything else but trust! All of these structures and things are being imposed by people that don't have to go through the process and don't trust the method or the staff to execute the method'.  Table 6 presents a summary of key recommendations on coherence for organisations to normalise a large-scale agile transformation based on our case study findings, which are backed up by relevant literature wherever possible.
Recommendations on cognitive participation in large-scale agile transformations. FinanceCo largely focused on adherence to the Spotify model over its long-terms sustainability for the large-scale agile transformation. This was demonstrated by their use of software flow metrics which primarily focused on short-term software team performance and productivity. The literature suggests that this typically presents many challenges and tensions between management and software teams on how to balance adherence with long-term software team performance and productivity (Conboy and Carroll, 2019;Edison et al., 2021). From our research, this issue is best summed-up when we explore these findings with a Manager of Software Operations at Spotify (MS). He speculated on the legitimation of adopting the Spotify model within FinanceCo and some of the challenges experienced with C4T during the large-scale transformation: 'I must caution that many organisations look at the Spotify model as being their panacea which will solve all of their problemsand use it "out of the box." The Spotify model worked for us at Spotify! It was not designed to be adopted by every organisation. You must carefully consider what elements work for you, and which don't'.
There was a tendency to measure agile transformations by adherence to the Spotify model, rather than the value it provides. For example, progress was often described in terms of the number of Tribes established, or the number of teams participating in various agile ceremonies (Smite et al., 2019). In this case study, progress was primarily assessed through (i) the ability to immediately and tangibly measure the impact of actions or events on the transformation process and (ii) difficulties in defining exactly how much a metric change was due to the method versus other internal factors. However, it is also evident in the findings that managers have little guidance on how to find the optimal degree of transformation (Conboy and Carroll, 2019), placing emphasis on productivity and performance data as a means to gauge 'progress'. Table 7 presents a set of key recommendations around cognitive participation for organisations to normalise a large-scale agile transformation based on our case study findings, which are backed up by relevant literature wherever possible.
Recommendations on collective action for large-scale agile transformations. The notion of self-organised teams and team autonomy can be promoted for large-scale agile transformations. However, when self-organised teams become managed through a singular view using metrics, it can provide a 'Big Brother effect' 4 by monitoring productivity targets. For example, the FinanceCo case study introduced 'touch time' to monitor individual engagement and contribution to specific projects. It is important to address fundamental questions such as: 'What is the rationale for increased speed?'; 'Is faster, better?'; and 'How do you account for time to learn or training on new initiatives and see this as a contribution to transformations?' Table 8 presents a set of key recommendations around collective action for organisations to normalise a large-scale agile transformation based on our case study findings, which are backed up by relevant literature wherever possible.
Recommendations on reflexive monitoring of large-scale agile transformations. The use of software flow metrics was a novel initiative for the large-scale agile transformation at FinanceCo. However, evaluations of the Spotify model indicated that C4T were too narrowly focused on speed and time events and failed to account for factors which influence transformation such as a change in the culture and mindset. The importance of culture (van Wessel et al., 2021) while adopting the Spotify model was explained by a Manager of Software Operations at Spotify (MS) who described how: 'Culture is an abstraction, yet the forces that are created in social and organisational situations are powerful. If we don't understand the operation of these forces, we can become victim to them during large-scale agile transformations'.
In summary, the adoption of metrics demonstrated some short-term promise in terms of building evidence around the use of a large-scale agile method, but efforts must also focus on the transformation process rather than simply focusing on adherence to a method (Conboy and Carroll, 2019). Table 9 presents a set of key recommendations around reflexive monitoring for organisations to normalise a largescale agile transformation based on our case study findings, which are backed up by relevant literature wherever possible.

Theoretical contributions
Our research presents four key theoretical contributions. Firstly, this research is the first to apply NPT in the IS field to explore the normalisation of a large-scale agile transformation. Our literature review indicates that there is an apparent need to advance theoretical diversity on overarching perspectives on the large-scale agile transformation process (Edison et al., 2021;Uludag et al., 2020). To address this, we demonstrate how NPT provides an excellent and novel theoretical lens to achieve this which is a first for the IS community. Secondly, the research demonstrates how NPT supports researchers and practitioners to identify which actions facilitate or prevent the This is supported by Paasivaara (2017) and Birkinshaw (2018) that contend that not informing the stakeholders the reason of adopting a largescale agile method can lead to change resistance. By informing and engaging people, the change process gains legitimacy and supports across an organisation (Paasivaara and Lassenius, 2011). The recommendation points out a more concrete way to implement 'sharing a common vision', one of the success factors for large-scale agile transformation (Edison et al., 2021 No existing studies, as far as we are aware, offer a similar recommendation for large-scale agile transformation.

Communal specification of a large-scale agile transformation
Invest in training needed for sustaining rather than just adopting the large-scale agile method.
The importance of training and coaching is recognised in relevant literature, but the focus is largely on adoption of agile methods (e.g. Laanti et al., 2015, Ebert and Paasivaara, 2017, Laanti, 2017, Lai and Clear, 2018. This recommendation emphasises the need for regular training to sustain their use and build a shared understanding around the vision, aims, objectives, and benefits of the large-scale agile method. Formally assess whether all team members have a clear understanding of how their specific role, responsibilities, and the ceremonies imposed by the adopted large-scale agile method contributes to sustaining a large-scale agile transformation. This is supported by the literature that recognises the challenges imposed by the adopted large-scale agile method, including the fluidity of agile roles and transition from old job roles to new ones (Hui, 2013, Jovanović et al., 2017, Paasivaara et al., 2018 and an increased number of agile ceremonies (Moore and Spens, 2008, Gustavsson, 2019a. However, the literature does not indicate any effective solutions so far. This study provides a specific and direct way to address these challenges.  sustainability of large-scale agile transformations using this nuanced lens of normalisation. We explain how Fi-nanceCo failed to successfully lead a large-scale agile transformation. For example, we provide a nuanced view on how C4T placed too much emphasis on software flow metrics for short-term software team productivity and performance as a general measure of 'progress' and the consequences of doing so. Thirdly, we present key recommendations based on NPT to guide researchers and practitioners on the normalisation of large-scale transformations (Tables 6-9). From an NPT perspective, we outline how organisations can adopt a stronger evidencebased approach to move forward and ensure the adoption of large-scale agile methods provides a means to achieve the transformation objectives, rather than simply restructuring an organisation around a large-scale agile method. We witnessed incremental changes made to the work practices to implement a large-scale transformation which demonstrates the applied nature of NPT. This case study indicates that a large-scale agile transformation process does not necessarily proceed in a strictly linear fashion but rather encounter difficulties and go through revisions as it evolves within a particular context in response to internal and external factors. However, it was clear that failing to consider normalisation over adherence for a large-scale agile method, such as the Spotify model, can lead to an unsuccessful transformation outcome. Fourthly, we identify directions for future research opportunities on large-scale agile transformations. Our literature review reports on the lack of theoretically grounded research on large-scale agile development and transformations which presents some opportunities for future research opportunities. As part of a future research agenda, firstly we have identified the need to expand on this research around the normalisation of large-scale agile transformations. Therefore, to further ensure rigour and relevance of our NPT theoretical developments and to build a cumulative tradition on IS theories, there is an opportunity to conduct multiple case studies on large-scale transformations in global software organisations to offer case study comparisons with various contextual factors across different industries. Secondly, there is a need to build models that capture the dynamic nature of the transformation process and weave narratives around stakeholders and sequences of events to explain how transformations evolved over time; why they evolved in the way they did; and how they become implemented, embedded, integrated, and evaluated in practice; and longterm metrics to measure normalisation factors via a transformation maturity model. Thirdly, we identified growing assumptions between adherence and sustainability which calls for more research on uncovering sustainability factors (e.g. team communication and coordination) to take account of a broader perspective of transforming versus adopting an agile method. Fourthly, as there continues to be a growing emphasis placed on data analytics to inform organisations around their progress with a large-scale agile transformation, there is an opportunity to research how organisations balance the demand for autonomy with their reliance on data-driven decision-making. Fifthly, there is an opportunity to map or visualise the evolution of a large-scale agile transformation and changes across organisational and social networks using social network analysis (SNA). This may No existing studies, as far as we are aware, offer a similar recommendation for large-scale agile transformation.
Host regular training events and develop training resources such as charts, guidelines, and policies to ensure that all organisational stakeholders are aware of the core values, benefits, and importance of the adopted large-scale agile method to sustain competitiveness for the organisation.
This is in line with the literature that recognises the importance of training (e.g. Ebert and Paasivaara, 2017, Laanti, 2017, Paasivaara, 2017, Lai and Clear, 2018 as one success factor of adopting large-scale methods. The case study findings draws focus on the consequences of teams failing to clearly explore and understand the proposed value, benefits, and importance of new practices associated with a new large-scale agile method. We explain that management needs to understand the team members internalisation process in order to sustain large-scale agile transformations. Initiation of a large-scale agile transformation Plan for the optimal degree of transformation with the large-scale agile method as per the organisational goals and objectives to remain competitive The optimal degree of transformation with the large-scale agile methods was also identified by Conboy and Carroll (2019). However, the literature does not indicate any effective solutions so far. This study is the first to provide ways to address this from a teams and software flow metrics perspective Determine whether management prioritises adherence to the method or value of the large-scale agile method to the organisation This is in line with Laanti (2017) that contends that there is a general assumption that organisations adopt large-scale agile methods to achieve business success. It can help to address the challenge that organisations face to find meaningful metrics for understanding the performance of their agile transformation initiatives (Brown, 2011, Laanti, 2017 Enrolment within a largescale agile transformation Communicate plans on the large-scale agile transformation as early as possible to ensure all team members are fully aware of strategic initiatives and rationale to adopt new roles and responsibilities This recommendation extends Brown (2011) that demonstrates a wellstructured adoption plan is a success factor of large-scale agile by emphasising the importance of communicating such a plan early on Clearly communicate the benefits of (re)organising team structures and why reallocating team members based on expertise contributes to a large-scale agile transformation strategy and the vision for success This recommendation is in line with the team-related success factors for large-scale agile transformation reported in literature, including fluid team structure (Birkinshaw, 2018) and a dedicated team to lead inter-team coordination and continuous improvement (Ebert andPaasivaara, 2017, Paasivaara, 2017). The recommendation extends these factors by highlighting the need to communicate the rationale behind them Legitimation of a large-scale agile transformation Introduce mechanisms and metrics to learn how team members contribute to the large-scale agile transformation No existing studies, as far as we are aware, offer a similar recommendation for large-scale agile transformation Provide team members with transparency to assess progress during the transformation journey, e.g. a long-term view such as a maturity model approach and/or suitable analytics No existing studies, as far as we are aware, offer a similar recommendation for large-scale agile transformation Promote team autonomy and an open learning culture which ensures teams can make a valid contribution to a transformation process and identify impediments This is in line with the challenges reported in literature that some teams may feel a lack of autonomy when using large-scale agile methods and certain decisions are made at a higher organisation level (Eklund and Berger, 2017, Paasivaara et al., 2018, Gustavsson, 2019b Invest in peer support and team resources to build new skillsets to alleviate fears of inadequate contributions Gustavsson (2019b) reports that team members sometimes are afraid of getting criticism and being humiliated in joint activities required by largescale agile methods. The recommendation provides concrete ways to alleviate the challenge, which is not evident in extant literature (continued) Activation of a large-scale agile transformation Identify which factors will influence adherence over value (and vice-versa) of an agile method (e.g. standards compliance, speed, cost, technology, and customer requirements) This is in line with Denning (2011), Nikitina et al. (2012, and Laanti et al. (2015) that argue that agile mindset and principles should be focused on rather than mechanics such as practices or tools. The recommendation goes beyond this argument and suggests finding the factors behind the mislaid focus Ensure the adoption of large-scale agile transformation metrics are not used in a competitive sense between teams. This can lead to conflicting tensions, reduced morale, and gaming of team performance reporting mechanisms No existing studies, as far as we are aware, offer a similar recommendation for large-scale agile transformation Encourage team members to collaborate and collectively (re)define the actions and procedures required to sustain best practice under the new large-scale agile method This is in line with the suggestion in literature that an effective mechanism for continuous process improvement brings positive results to transformation initiatives (Nikitina et al., 2012, Ebert and Paasivaara, 2017, Paasivaara, 2017, and it provides more tangible, prescriptive actions on how to achieve it. For example, facilitate a consultation process around altering team structures; explain how new team structures contribute towards the vision of the large-scale agile transformation; and explore how software flow metrics can build team efforts to sustain the overall transformation process Introduce incentives for teams to stay committed to sustaining the vision of the transformation No existing studies, as far as we are aware, offer a similar recommendation for large-scale agile transformation reveal further insights by investigating social structures through the use of networks and graph theory and examine what characterises networked structures in terms of nodes (individual actors, people, or things within the network) and the ties, edges, or links (relationships or interactions) that connect them. As noted in the discussion, it is important to explore how scalability impacts on outcomes, team structures, and related factors within a large-scale agile context. Scalability typically focuses on the growth and addition of resources to a system and its capabilities. Within an agile context, 'scaling out' and 'scaling up' often focuses on the adoption of agile methods for large software projects. As noted by Rigdy et al. (2018) on the topic of 'agile at scale' within organisations, 'they place more value on adapting to change than on sticking to a plan…'. This was also reported by Conboy and Carroll (2019) who describe the lack of empirical evidence on the adoption of large-scale agile methods, their effectiveness, or factors leading to failed large-scale agile transformations. Current literature focuses on organisations experience with adopting large-scale agile methods and techniques to implement and scale practices (Saeeda et al., 2015;Edison et al., 2021). However, Rolland et al. (2016) question the assumption that effective largescale development can be achieved by simply scaling up small-scale agile methods. This calls for more focus on the 'complex socio-technical interdependencies' which are considered to be essential characteristics of large-scale agile projects (Rolland et al., 2016). In addition, Edison et al. (2021) identify that difficulties with large-scale agile transformations typically centre around the complexities and ambiguities, arising from teams, their composition and various personalities, ill-structured work processes, diverse and often competing agendas between teams, and vague outputs and goals. Yet, our research indicates a lack of research on the socio-technical interdependencies which  (2007), Roman et al. (2015), Lindlof and Furuhjelm (2018), and Birkinshaw (2018) that suggest such a balance affects the sense of commitment of teams involved Clearly define the allocation of tasks that underpins the division of labour imposed by operationalising a new method in practice This is supported by literature that reports challenges such as the fluidity of agile roles and transition from old to new job roles (Hui, 2013, Jovanović, 2017, Paasivaara et al., 2018 and a lack of ownership of the work (Cho, 2007 This recommendation is supported by Brown (2011) and Laanti (2017) that highlight the importance for organisations to define meaningful metrics for understanding their agile transformation process. This recommendation emphasises the use of metrics that assess sustaining the transformation process, which is lacking in extant literature  (Brown, 2011).
Communal appraisal of a largescale agile transformation Assess team efforts to sustain the transformation at an organisational level rather than relying on team productivity and performance metrics to inform best practice and influence strategic initiatives. The literature indicates that organisations struggle to define meaningful metrics for understanding the performance of their agile transformation initiatives (Brown, 2011, Laanti, 2017. This recommendation provides tangible, prescriptive advice to tackle this challenge, which is lacking in the current literature.
Build evidence (qualitative and quantitative) to inform and communicate progress of the large-scale agile transformation and build momentum. This is another concrete recommendation to tackle the challenge of defining meaningful metrics to evaluate agile transformation initiatives reported in literature (Brown 2011, Laanti 2017. Introduce knowledge-sharing initiatives to improve innovation under the large-scale agile method across the organisation. This recommendation is in line with several studies (Paasivaara andLassenius, 2014, 2019)   To explore teams understanding of the vision, aims, objectives, and expected benefits of a set of practices associated with a new large-scale agile method.
• What are team members perceptions of the vision, aims, objectives, and expected benefits of a set of practices associated with a new large-scale agile method? • How do management identify and promote the vision, aims, objectives, and expected benefits of a set of practices associated with a new large-scale agile method? Individual specification of a large-scale agile transformation To identify individual team members sensemaking process around their specific tasks and responsibilities related to new practices associated with a new large-scale agile method.
• How do team members perceive their specific tasks and responsibilities related to new practices associated with a new large-scale agile method?
Internalisation of a large-scale agile transformation To examine sensemaking process which involves various organisational stakeholders to clearly explore and understand the proposed value, benefits, and importance of new practices associated with a new large-scale agile method.
• What is the sensemaking process across various organisational stakeholders to explore and understand the proposed value, benefits, and importance of new practices associated with a new large-scale agile method? • What range of training events are introduced to create awareness around the core values, benefits, and importance of a large-scale agile method? • What guides management expectations and targets around the value, benefits, and importance of a large-scale agile transformation? Initiation of a large-scale agile transformation To explore team members' expected contributions are to a large-scale agile method through the establishment of new or modified agile practices.
• How do management identify and align specific goals and objectives of a large-scale agile method with organisational goals and objectives? • What are practitioners' perceptions regarding the optimal degree of transformation with the large-scale agile method as per the organisational goals and objectives? • How do managers determine whether the organisation's agile transformation prioritises adherence to the method or value to the organisation? Enrolment within a large-scale agile transformation To identify the process whereby team members organise or reorganise themselves and others to collectively contribute to work involved in new practices imposed by a new large-scale agile method.
• What process is implemented to coordinate teams to collectively contribute to work involved in new practices imposed by a new large-scale agile method? • What is the rationale to adopt specific team structures, roles, and responsibilities for largescale agile transformations?
(continued) To examine the process whereby team members explore and believe that they can make a valid contribution to a large-scale agile transformation process.
• How do team members explore and believe that they can make a valid contribution to a large-scale agile transformation process? • What mechanisms do organisations introduce to assess how team members contribute to the large-scale agile transformation? • How do organisations create a learning culture around valid contributions to sustain the largescale agile transformation? Activation of a large-scale agile transformation To examine the process whereby members collectively explore and define the actions and procedures needed to sustain the new practices and commit to the vision of a large-scale agile transformation.
• To examine how team members' work with each other, artefacts, and other elements in a set of practices, and efforts to operationalise and exploit a large-scale agile method.
• How do team members' work with each other using artefacts and other elements in a set of practices to operationalise and exploit a largescale agile method? • How do managers empower teams to adopt using artefacts and other elements to improve new team performance and productivity with a new large-scale agile method? • How do managers allocate new tasks imposed to operationalise and exploit a large-scale agile method? Relational integration within a large-scale agile transformation To examine the process whereby people generate knowledge to build accountability and maintain confidence in a large-scale agile method.
• How do management build accountability and maintain confidence in a large-scale agile method across teams? • How satisfied are team members with autonomy to maintain accountability and maintain confidence under a large-scale agile method? Skillset workability of a largescale agile transformation To examine the allocation of tasks underpinning the division of labour typically established around a new large-scale agile method.
• How do managers allocate specific tasks underpinning the division of labour established around a new large-scale agile method? • How do organisations define key actions and metrics associated with a new large-scale agile transformation process?
(continued) support to embed and sustain the large-scale agile methods, that is, the normalisation of large-scale agile transformations. Our research is the first study which demonstrates how NPT provides an excellent theoretical lens which provides a nuanced view of which factors can enable or inhibit the transformation process. It uncovers specific aspects of the organisation scalability efforts and the factors which lead to poor outcomes, impact of new team structures on productivity and performance, and the misalignment of how various stakeholders perceived progress and success of a large-scale agile transformation. To build on this, additional case studies should apply NPT to compare and To examine how resourcing and management of a set of practices through allocation of different kinds of resources and the execution of protocols, policies, and procedures with a largescale agile method.
• How do management allocate resources and execute protocols, policies, and procedures with a large-scale agile method? • How do teams carry out audits to guide resource allocation to sustain a large-scale agile method? • What techniques are used to ensure a large-scale agile transformation resource allocation is transparent and relevant within and across teams? Systematisation of a largescale agile transformation To evaluate how effective and useful team members consider a large-scale agile method is for them and others and plans to build on opportunities.
• How do team members evaluate the effectiveness and usefulness of a large-scale agile method? • How do managers examine the maturity of a large-scale agile transformation process and plan for future opportunities? Communal appraisal of a large-scale agile transformation To examine how team members appraisal on the value of a specific set of practices imposed by a large-scale agile method.
• How do team members appraise the value of a specific set of practices imposed by a large-scale agile method? • How do management assess the benefits of a set of practices imposed by a large-scale agile method to sustain a transformation process? • What metrics are adopted to appraise the value of a specific set of practices imposed by a largescale agile method and inform best practice for a transformation? • What knowledge-sharing initiatives are introduced to communicate the value of a specific set of practices imposed by a large-scale agile method? • How do organisations build evidence to communicate the value of a specific set of practices imposed by a large-scale agile method? Individual appraisal of a largescale agile transformation To explore individual team members perceptions of working with a large-scale agile method, a new set of practices, and its direct effects on them.
• How do individual team members perceive working with a large-scale agile method, a new set of practices, and its direct effects on them? • How do organisations support team members to work together (formal or informal collaborations) to evaluate the value of a largescale agile methods? Reconfiguration of a largescale agile transformation To examine how team members efforts to redefine procedures, modify practices during a large-scale agile transformation, and exploit emerging technology capabilities for future performance improvement.
• How do team members redefine procedures, modify practices to sustain a large-scale agile transformation? • What techniques do teams adopt to sustain and build momentum for a large-scale agile transformation process? • How do team members explore and exploit emerging technology capabilities for future performance improvement for large-scale agile transformations?
contrast findings around success and failure for large-scale agile transformations. Additional research is also required on the leadership styles for large-scale agile transformations and how it may impact on people, processes, and practices for scalability. We also envisage that the contributions of this research will be far reaching and support new research developments around IS transformation, for example, transformations efforts spanning across strategies to normalise digital transformations and IT-enabled change (Carroll et al., 2021).

Conclusion
Following the rise and proliferation of agile methods across almost every corner of contemporary software development, more recent attention has turned to the challenge of designing and implementing large-scale, organisation-wide variants of these methods. Many organisations attempt to scale agile methods in an effort to scale success outcomes by introducing methods such as SAFe or Scrum-at-Scale, or some customised version of related methods. Our research indicated a lack of theoretically grounded studies of largescale agile transformations and weak assumptions underpinning those that do exist. Our research also indicates that practitioners often over-relied on adherence to agile methods rather than identifying approaches to successfully embed and sustain the transformation process.
Our study applies NPT to examine key challenges in normalising large-scale agile methods. An in-depth case study is presented on a large-scale agile transformation and demonstrates 'how' and 'why' failing to normalise a large-scale agile method can lead to failure. This study presents four key contributions: (i) it reports on how a focus on the initial adoption of a large-scale agile method has major shortcomings; (ii) we introduce and empirically apply NPT as an illustrative guide for the information systems field to identify and mitigate against transformation failure or abandonment; (iii) we present key recommendations on the normalisation of large-scale transformations; and (iv) we present an agenda for future research on large-scale agile transformations.

Limitations of NPT
This research demonstrates the application of NPT to ISrelated research. However, there are three broad limitations with NPT. Firstly, NPT is less sensitive to specific research contextual factors beyond the research site or the business context, for example, whether new regulations may impact on a large-scale transformation. Normalisation process theory focused on how practices become implemented, embedded, integrated, and evaluated regardless of the organisational setting, policies, or potential regulations constraining change. Secondly, it can be challenging to trace resource allocation and financial constraints on organisations to scale transformations due to the sensitive nature of disclosing such information in case study research. Normalisation process theory focuses on actions around the transformations with less consideration for the financial power of organisations to enable transformations and their financial resources to learn from loss-generating practices, although it could be factored in for future research cases that adopt NPT. Thirdly, NPT places emphasis on actors undertaking specific roles to sustain a transformation process and the effect of reorganising staff in roles and locations. However, NPT does not indicate what roles staff ought to take on and sustain the transformation process to ensure team resilience during a turbulent long-term transformation process, or how roles and their relationships should evolve during the transformation process. These limitations have also informed future research and transformation case studies in an effort to build a culminate tradition of NPT in IS. Table 10 presents an agenda for future research on largescale agile transformations and outlines (i) the normalisation focus; (ii) rationale for research; and (iii) examples of research questions for future research.
2. FinanceCo is pseudonym which is adopted to protect the organisations' anonymity 3. Center for Technology (C4T) is pseudonym which is adopted to protect the organisations' division name 4. In this case study, Squads reported that software flow metrics monitored their actions and behaviours and therefore could not enjoy autonomy or the freedom to self-organise. Xiaofeng Wang is an associate professor with the Computer Science Faculty of UNIBZ, Italy. Her main research interests include software startups, agile and lean software development and innovation, and human factors in software engineering. She is actively publishing in SE venues, including IEEE Software, JSS, Empirical Software Engineering, etc. She is also active in serving various SE conferences and workshops and has collaborated with large companies and software startups on national and European projects.