A framework for cut-over management

Introduction

Most IT-related projects, in particular implementation and software development projects, change a productive system landscape when taken live. On the one hand these projects face the challenge of delivering a change (within an ITIL context) in a timely and cost-effective manner. On the other hand, IT organizations need to assure the integrity of the system landscape and are required to minimize potential interference to the ongoing business, as Service Level Agreements (SLAs) need to be fulfilled.

The Central Computing and Telecommunications Agency (CCTA), a government agency in Great Britain, already developed the Information Technology Infrastructure Library (ITIL) at the end of the 1980s. The latest update of ITIL V3 was published in 2011 which is frequently referenced as ITIL 2011. Is it still relevant? Yes, because it helps bring order to IT chaos. Proehl et al. (2013) analyzed 255 articles in the field of IT Service Management. They also confirmed the work by Shahsavarani & Ji (2011). Both studies found a growing number of published papers dealing with the development of concepts, constructs, models, methods and implementations for theory development. Performance issues in IT Service Management, justifications, and IT Infrastructure Library topics are among the most popular topics of research. Only a few articles have used or developed theories. Marrone et al. (2014) found organizations adopting ITIL implemented more operational level processes than the tactical/strategic level processes. The study utilizes 623 survey responses from the UK, USA, DACH (German-speaking countries) and Australia.

ITIL is rarely seen in isolation and current research focuses on the integration with IT governance standards, such as ISO/IEC 38500 and other methodologies, such as COBIT, PRINCE2 and the ISO2700x-family (Tanovic & Orucevic , 2012; Shivashankarappa et al., 2012; Jäntti et al., 2014; Larsson, Rusu & Aasi, 2015).

ITIL’s release and deployment management processes are demanding. Jokela & Jäntti (2012) as well as Lahtela & Jaentti (2011) identified within their case studies the following common challenges:

• no existing process for product portfolio release and deployment management

• lack of communication

• unclear release and deployment management and/or product portfolio management process roles

• lack of resources and time for product portfolio integration, testing and reviewing

• existing models and practices are not suitable for agile software development

• uncertainty about of the contents of release packages

• high release distribution frequency

• different and tailored release packages

• the change management is not up to date and

• no existing service catalog.

The project-specific cut-over, as defined in more detail below, requires detailed planning, many meetings and several agreements with all stakeholders involved. Which questions need to be addressed? Here are just some of them:

• How should the project outcomes be transferred to operation?

• Which scenario provides the best compromise between time, cost and risk?

• What happens if the cut-over fails?

• How can the previous system condition or version be reinstalled in case cut-over fails?

• How can we ensure that projects start early enough, with all the necessary preparation work?

• How can cut-over activities be aligned within an IT service organization?

• How do we measure the success of the cut-over–and how can we maximise it?

Unfortunately, answers and activities associated with these cut-over related topics are frequently addressed too late, probably because the cut-over is one of the final steps in projects and can therefore be hidden for a long time. If a cut-over and its associated deliverables are not integrated within the overall Project Plan then projects are at risk of being delivered too late. For this reason, there is an urgent business need to govern projects in such a manner that they do not forget to cover cut-over related items. So, which items should projects cover?

This question and many others are addressed by a ‘framework for deployment management’ as referred to below in this document. The purpose of this paper is to provide a methodology which assures a robust and timely go-live for projects.

An itemised objective overview:

• to list the key deliverables projects need to provide in order to assure a timely and safe go-live

• to illustrate how these deliverables can be integrated within the ITIL release and deployment process

• to look into different deployment types, which this paper refers to as (a) project-specific cut-over, (b) application-specific cut-over and (c) release-specific rollout along with the timeframes and timepoints involved, and lastly

• to present a framework which enables an IT-service organization to transition a project to a productive environment.

The scope of the deployment framework introduced is required

• to be applicable within an Information Technology Infrastructure Library (ITIL)-orientated service organization, and

• to extend commonly facilitated project-management methodologies.

As with most management practices, there are always more ways to achieve similar results with other approaches. However, in order to examine and establish good management practice, these methodologies need to be published to facilitate comparison. The framework published here is just one of many potential options. It illustrates how deployment activities can be harmonised with commonly facilitated IT projects as well as quality-management practices, and integrated in an ITIL environment.

Even though it is possible to take advantage of the deployment framework presented here without ITIL, many companies have adopted ITIL recommendations and best practices and are organized in a service-orientated manner. IT projects are therefore unlikely to exist in isolation and are more likely to be embedded in a service-orientated manner. ITIL is known to be generic and by itself is not complete. Its purpose is not to provide advice on how to implement IT Service Management processes. Instead it requires integration with other disciplines such as project management and quality management.

The framework presented here in the form of an illustrative case study arose during my consultancy work and has been implemented for some applications within the Department of Testing and Release Management of Otto (GmbH & Co KG) which is a member of the otto group. It is not a hypothetical model. Projects which wish to cut-over in form of a release-specific rollout are requested to comply with this framework. Within every release between 15 to 25 projects are participating. Six release-specific rollouts are conducted every year. Even though consulting methodologies such as ASAP (Accelerated SAP) recommend scattered project-specific deliverables useful for Cut-over Planning, this publication offers an integrated approach on how to systematically prepare for a project-specific cut-over with all required deliverables. The framework provided extends the ITIL release and deployment process by means of IT projects and allows IT projects to interface easily with the ITIL change-management process.

The activities of the otto group, with its headquarters located in Hamburg, Germany, are grouped into three main business areas: (i) Multichannel Retail, (ii) Financial Services and (iii) Service. This structure is consistently reflected in the group’s activities along the retail value chain, in Logistics for instance. In the year 2014/15, the Group generated consolidated revenue of more than 12 billion euros and employed about 54,000 staff worldwide.

Project-Specific Cut-over Versus Release-Specific Rollout

ITIL is often facilitated as a checklist. With regard to the release and deployment process, it consists of the following steps (Rance, 2011):

• plan and prepare a release

• build and test a release

• plan and prepare deployment

• conduct deployment

• conduct review, and

• provide early-life support.

Due to its generic nature and the need to interface with other disciplines, IT-service organizations implement these steps quite differently. The variability of implementation options may be caused by different interpretations of the term ‘release’, which is also related to the term ‘change’. How does ITIL define these terms?

The term ‘release’ is defined as: “One or more changes to an IT service that are built, tested and deployed together. A single release may include changes to hardware, software, documentation, processes and other components.” (AXELOS, 2011).

The term ‘change’ is defined as: “The addition, modification or removal of anything that could have an effect on IT services. The scope should include changes to all architectures, processes, tools, metrics and documentation, as well as changes to IT services and other configuration items.” (AXELOS, 2011).

In order to safeguard the related system landscape as well as associated services, conventions need to be outlined on the practical arrangement and organization of release-related changes. Here, the term ‘release-model’ is facilitated for a set of rules and conventions used to bundle and organize a release.

Figure 1 illustrates a semi-agile release model, which is used within this publication as an example to address some dependencies (Nageldinger, 2015). This release model is called ‘semi-agile’ here because it consists of an agile section and a release phase, which follows a classical sequential pattern (cascade model). During the agile period, phases between projects are non-clocked. Project phases which fall within the agile period relate to the Project Planning, design and realization section, and can be conducted in sprints. Once these projects participate in an integration test, then their phases need to be clocked. Independently of which release model is facilitated, the release phase will most likely consist of (i) an entry gate, (ii) technical and business-related integration tests with their acceptance procedures, and (iii) the release-specific rollout (see Fig. 2).

Let us now look at the deployment types. In case the bundling of projects is foreseen by the release model, such as in the one presented here, then we encounter deployment related to the release as well as deployment related to participating projects. ITIL does not distinguish between these. Here, the term ‘deployment’ (AXELOS, 2011) is defined as “an activity responsible for movement of new or changed hardware, software, documentation, process etc. to the live environment. Deployment is part of the release and deployment management process.” It can probably be argued that such a distinction is unnecessary, since all bundled projects are to be deployed in the same time slot. However, the project-specific deployment and its associated preparation work are owned by the Project Manager and, in the case of larger projects, by a designated Cut-over Manager. According to ITIL the Release and Deployment Manager owns the release and its associated deployment.

ITIL’s definition of ‘deployment’ lacks conceptual clarity. This is because if we look only at the movement of software to the live environment then we also need to distinguish between the actual movement and its use when the software is ‘switched on’. This is necessary if legal changes come into effect at a specific point in time, for example, but the software is required to be moved to the live environment beforehand. For the purposes of this paper the term ‘deployment’ from now on is solely used to refer to the movement of software to the live environment, which is commonly conducted within a restricted timeframe. The term ‘point-of-use’ is facilitated to account for the point in time in which the software is ‘switched on’.

In order to safeguard our system landscape and associated services we need to monitor both the deployment (timeframe) and point(s)-of-use (point in time). In the context of a release, one deployment and potentially more points-of-use are scheduled. This aspect is elaborated further below in the discussion of the release-change.

Besides the participating projects, a release additionally includes service packs (a phrase used for minor upgrades), bug fixes and smaller system-related features. These non-project elements are usually governed by separate changes. All ingredients of a release should provide the option of independent transitioning to the productive environment. If this option is not provided, then projects or non-project elements cannot easily be excluded from the release if they fail the integration test. Smaller projects are likely to be bundled in form of a release, and then transitioned to a productive environment in form of a release-specific rollout. Major IT implementation projects are preferably transitioned to a productive environment independently, in order to reduce complexity.

The term ‘project-specific cut-over’ defines the transition of a project to a productive environment. It relates to the project as a whole and includes the software deployment and its point of use. The actual timeframe associated with the project-specific cut-over is called the ‘cut-over window’. The ‘release-specific rollout’ defines the collective transition of bundled projects to a productive environment. In the same way, the associated timeframe is called the ‘Rollout Window’ (Nageldinger, 2014).

Frequently, projects communicate what is frequently called a ‘go-live date’ on their overall project chart. In view of the reasons given above, this could mean (i) end of the software deployment, (ii) end of cut-over window, which relates to the transitioning of the whole project, or (iii) point-of-use. It is therefore recommended to question what exactly this date refers to.

Beside the project-specific cut-over and the release-specific rollout we are most likely to face a third transitioning aspect, here called the ‘application-specific cut-over’: this relates to a specific system or application and mainly covers approaches related to maintenance or upgrades. It differs from the project-specific cut-over in that it consists of canonical (i.e., reoccurring with every rollout) activities and tasks. The deployment of service packs can be organized canonically, for example.

Within an SAP context the term ‘rollout’ is frequently associated with a so-called ‘template approach’. In this, the core configuration is embedded within a reusable template and the country-specific characteristics are added separately in the form of a local configuration (SAP, 2009). SAP’s usage of the term ‘rollout’ is quite similar to the definition provided here and its associated release context. This is because the release-specific rollout also consists of reoccurring activities, here referred to as “application-specific cut-over”, which is similar to SAP’s template approach. The project-specific cut-over activities, which are non-reoccurring, may be considered in the same way as SAP’s country-specific characteristics.

The Cut-over Schedule is a list of tasks and activities required in order to conduct the cut-over. It differs from the ordinary overall project schedule as it only focuses on tasks and activities within the cut-over window, of short duration and lasting only for a couple of hours or a weekend. The Cut-over Schedule is created by duration-based scheduling techniques. Additionally, it contains a list of prerequisites, such as the delivery of training courses and the installation of printers etc. These prerequisites need to be completed before the actual cut-over can be conducted and are part of the overall project schedule. The Cut-over Schedule does not relate to the post go-live phase. For this objective projects need to have a separate plan, which for example addresses aspects of support, data-conservation and accounting closure procedures.

Besides the Cut-over Schedule the Cut-over Plan contains a variety of other deliverables, in the same way as a project-management plan, which are further elaborated below. Key elements of a Cut-over Schedule are for example termination points, frequently called ‘points-of-return’ (POR). Theses PORs aim to trigger activities used (i) to restore the initial condition or (ii) incrementally fall back to a previous POR. The last POR is referred to as the ‘point-of-no-return’ (PONR). The PONR does not always fall within the cut-over window. The Fallback Concept is the name of a document used to describe activities which are potentially necessary at these PORs, in case the system needs to be rolled back to a previous state or even its initial state. If the cut-over activities pass the PONR then we enter the disaster scenario. For such events, a disaster-recovery concept defining appropriate measures should be in place. The disaster-recovery concept is mentioned here because it is unfortunately frequently neglected. It is regularly associated with earthquakes, hurricanes or terrorist attacks. However, unforeseen events during a cut-over after the PONR can also trigger the disaster scenario (BSI, 2008). The cut-over window is illustrated in Fig. 3.

The sections above clarify why certain release-models, which foresee the bundling of projects for collective rollout, require at least two perspectives—a project-specific and a release-specific one. An important outcome for the preparation of a project-specific cut-over and a release-specific rollout is the go-live scenario. A go-live scenario defines the overall approach on how a cut-over or rollout is to be conducted. The go-live scenarios differ in most cases, which is illustrated by an example in Fig. 4. The go-live scenario of project A utilizes the Rollout Window of release 1 and 2, whereas projects B and C merely take advantage of one Rollout Window. The release-specific rollout also facilitates go-live scenarios. Here, the rollout of release 1 uses a sequential rollout strategy, while the rollout of release 2 uses a parallel rollout strategy.

Recommended Deliverables and Outcomes for the Preparation of a Project-Specific Cut-over

This chapter covers recommended deliverables and outcomes for projects in order to prepare for a project specific cut-over. Projects are assumed to be arranged in a cascading manner with classic phases, in order to keep this publication as simple as possible. However, as stated above, activities may be undertaken in an agile manner before entering the release phase.

The release model presented in Fig. 1 illustrates the following generic phases, which are quite similar in the case of IT-related projects:

• Project-planning phase

• Design phase

• Realization phase

• Test phase

• Cut-over phase

• Support phase.

Quality Management is facilitated in order to request these cut-over relevant project deliverables, which are then reviewed by Rollout Management. Table 1 illustrates potential deliverables (Nageldinger, 2015). Some of them are mandatory, others are optional. These deliverables are further elaborated here below and presented according to their project phases.

Realization phase

Decision related to the go-live and Fallback Strategy

The decision related to the go-live and Fallback Strategy is a key outcome of the realization phase. It needs to be conducted by the Steering Committee since it relates to the triple constraints, such as scope, timeframe, budget and risk. The decision is required to be prepared in form of a decision-model presentation and commonly extensively discussed, prior to a Steering Committee meeting. It is then up to the members of the Steering Committee to consult or involve further senior management if the go-live risks associated require this. This decision may present several political challenges and if it is conducted too late then the timely delivery of the project is at risk.

One-pager

The One-Pager summarizes the go-live and Fallback Strategy decided. It is later integrated within the release-specific Rollout Handbook. The objective of the one-pager is to inform all other projects and participating parties about the rollout and to facilitate the preparation of the release-specific rollout.

Draft Cut-over Plan

The Cut-over Plan and the Cut-over Schedule are not the same and are explained further below, within the context of the Rollout Handbook. The Cut-over Schedule is an important part of the Cut-over Plan. As a minimum requirement a draft Cut-over Schedule should be available at the end of the realization phase, sufficient to support the planning activities for the release-specific rollout. Since the term ‘draft’ can be interpreted in several ways, it is suggested to define the outcome in advance in order to avoid disappointment at the end of the realization phase. This draft schedule should contain all activity blocks, with their subordinated activities as well as their durations, dependencies within these activity blocks, with an attached working breakdown structure (WBS) and a Glossary.

Risk register

The cut-over related Risk Register does not differ much from an ordinary project Risk Register. The administrative work of this Risk Register should be kept to a minimum, since the risks solely relate to the cut-over. However, it is still necessary that such a record is properly administrated. Since key elements of such a Risk Register are quite frequently forgotten, such as the Risk Owner or Risk Indicator, a practical set of parameters are suggested here, such as:

1. Risk ID: the ID number assigned to the risk

2. Risk Category: a sorting criterion used to categorise risks. Commonly used categories are, for example, applications or phases during a rollout.

3. Risk Description: this can follow a simple scheme by addressing three questions: (i) What is the actual risk? (ii) How is the risk caused? (iii) What is the impact if the risk does occur?

4. Alerter: can be anyone who identifies and reports a risk.

5. Alerting Date: the date on which a risk has been recorded.

6. Magnitude: calculated on the basis of the probability and potential impact of a risk occurrence.

7. Risk Indicator: criteria used to recognize that a risk has manifested itself; for instance, error messages after a database update etc.

8. Mitigation Measure: a counter-action that can be taken if a risk manifests itself.

9. Risk Owner: a person with the assigned authority and competency to mitigate a risk. The Risk Owner identifies mitigation strategies, defines Risk Indicators and nominates a Technical Contact for the respective rollout. This role also includes refining the Risk Description, since the risk as such is frequently confused with the causes of a risk.

10. Technical Contact: a technical expert or a team which shares the required technical expertise and is in any case present during the rollout, usually fulfils this role. This person is responsible during the rollout (i) to verify whether a risk has manifested itself, and (ii) to coordinate Mitigation Measures defined within the Risk Register.

11. Processing Status: describes how far the processing of the risk has progressed.

    • New: a new risk has been recorded, described and assigned to an initial Risk Owner;

    • Under Way: the Risk owner has accepted his role for the particular risk raised, and communicates a delivery date on which the Mitigation Measures and Risk Indicator are defined;

    • Completed: as soon as the Mitigation Measures and Risk Indicator are described and a Technical Contact for the particular risk has been nominated, Risk Processing has been completed.

12. Completion Date: the date on which processing status is set to ‘Completed’.

Dependency matrix

A collective participation by projects in a release-specific rollout causes a need to manage dependencies. These dependencies can occur between Project Teams as well as jointly used personnel resources, here called ‘Core Teams’. The Dependency Matrix is a tool used to evaluate the potential impact and interrelation between Project Teams and Core Teams. Project Leads are requested to fill out a template and to identify which key personnel are required to be present during the cut-over.

The Dependency Matrix is hard to define as a tool because the terms (i) ‘Core Team’ as well as (ii) ‘dependency’ are themselves difficult to define. Here, a Core Team is seen as a heterogeneous group of key personnel required to be present during a cut-over/rollout. It consists, for example, of administrative personnel related to key applications and databases as well as Service Owners (within an ITIL context). A dependency between a Project Team and a Core Team is present if communication between the Project Team and the Core Team or its related technology is necessary during cut-over/rollout. Since Project Teams usually identify similar Core Teams, a template can be created. This is illustrated in Fig. 5. The bold points visualize dependencies between Project Teams and Core Teams participating in a release.

Even though the term ‘Core Team’ is treated less strictly, these dependencies can be facilitated to quantify the complexity of a release. Complex projects usually require more key personnel than less complex projects. A complex release usually consists of several projects with many dependencies. Therefore, one possible measurement of the complexity of a release is the total amount of dependencies. This idea is further elaborated below. Additionally, such a Dependency Matrix is quite a sensible instrument to have in order to identify projects which have probably influenced a particular business process negatively after rollout.

Dialogue with Rollout Management

Projects are requested to pass a gate prior entering the release phase, which consists of a variety of project deliverables. Most of the cut-over related deliverables have already presented here, such as (i) the decision related to the Go-live Strategy, (ii) the Cut-over Plan (draft), (iii) a Risk Register, (iv) Dependency Matrix and (v) an obligatory dialogue with Rollout Management. The cut-over related deliverables are discussed during the dialogue along with a couple of questions which the Rollout Manager is to address.

Recommended Deliverables and Outcomes for the Preparation of a Release-Specific Rollout

This chapter highlights a couple of important deliverables and outcomes of the release-specific Rollout Management. Large-scale projects which are likely to cut-over independently may consider the outcomes described here as well, even though these are assigned to the release phase.

Communication during release planning

Preparation work related to the release-specific rollout mainly covers coordination meetings with Project Teams and associated Core Teams participating in the release. Additionally, a large number of staff is required to be informed about the status related to the rollout preparation within the company. A rollout newsletter is a fit-for-purpose communication tool which can be easily customized to cover the coordination aspect and meet the corporate information need.

The release model illustrated here is used in such a manner that 6 rollouts are conducted on an annual basis, which results in an approximately 8-week preparation cycle for every rollout. A typical schedule related to the planning activities and publication dates of such a release cycle is illustrated in Fig. 6. The newsletter is published about every other week. It comprises topics related to the rollout preparation work, as well as subjects related to general IT questions such as new documents and procedures published by the security personnel, changes conducted by the Change Management etc. The newsletter is 2–6 pages long and announces deadlines and important dates, such as the date for the approval of the Rollout Handbook, which is done during a meeting with all key personnel. Documents related to the newsletter, such as the Rollout Handbook, are stored on MS SharePoint^®. This provides the desirable function of enabling distribution of the newsletter to a wider community and limiting access to more confidential documentation.

The 4th edition is published a couple of days prior to rollout. By this stage the Rollout Handbook is in a final stage and everybody involved has a last chance to incorporate necessary changes. The newsletter publishes the major steps involved during the rollout, key contact phone numbers and an overview illustration. All interested parties thus know the overall picture of the rollout, whereas access to the Rollout Handbook is restricted. The 4th edition publishes all major planning steps and dates for the preparation work of the following rollout as well.

The 1st edition focuses on the incidents after the rollout. Incidents are observed up to 2 weeks after the rollout. The observation period closes with a Lessons-learned Workshop, the submission of the final report and the closure of the Rollout Change (see further below for the treatment of the Rollout Change).

The 2nd and 3rd editions provide temporary planning results and draft versions of the Rollout Handbook. Planning is conducted in three phases: during the 1st phase, interviews are conducted between Rollout Management and Project Teams, see Table 1 for the deliverable ‘Dialogue with Rollout Management’. During the 2nd phase, Rollout Management interviews the Core Teams, and throughout the 3rd phase fine tuning of the Rollout Handbook is carried out.

Lessons-learned workshop

Above it has already been explained that a Lessons-learned Workshop should be conducted after completion of the rollout. Lessons-learned Workshops provide many generic benefits, such as:

• the reuse of knowledge throughout an organization

• the reduction of project cost

• the improvement of methods and practices, and

• the enhancement of project delivery and quality management

just to mention a few of them. However, the Lessons-learned Workshop mentioned here relates solely to the rollout conducted and is not of generic nature. Let us step back a little to add some higher-level perspective.

Any IT organization has a need to improve its processes, ensure agreed service levels and justify its projects or initiatives. For this reason, Continuous Service Improvement (CSI) represents one of ITIL’s five process groups. The CSI Register, which lists all improvement options and initiatives, is one of an IT organization’s key deliverables (Cabinet Office, 2011). Key Performance Indicators (KPIs) are used to measure the process. How can we measure the success of a rollout? What do we mean by success? ITIL publications already suggest a variety of KPIs, such as the number of incidents caused after the release-specific rollout. These simple KPIs can, when implemented in practice, be quite tricky. Even though this particular KPI is suggested for the service validation and test process within ITIL’s service transition process group (Rance, 2011), incidents are always a good indication of disturbances and are here also used for the evaluation of the rollout’s success. Figure 7 illustrates the total number of incidents recorded within one IT service organization. The dotted lines indicate the week during which a rollout occurred. The sharp decline of incidents during CW 30 is because a service-request process was introduced. The dips between 200 and 500 incidents/week are caused by the Christmas season, where many people are on holiday. Otherwise, it is quite difficult to acknowledge any impact of the rollout on the total number of incidents recorded. The picture changes once we take a closer look at a particular system. Figure 8 illustrates second-level incidents for a main order-handling system, which participated in all rollouts. These incidents account for about 5% of the total number of incidents recorded. After some rollouts a rise of incidents can be observed. How should we measure this?

Figure 9 shows the variability of incidents four weeks before and after rollout. The connecting lines are to aid visualisation of the form of the curves and are not intended to suggest data points at all. The data provided are normalized. Here, the mean value of incidents during a period of four weeks before rollout is set to 100%, with a standard deviation of ±18%. The figure suggests a significant increase of incidents 1–2 weeks after rollout. This increase is summarized in Fig. 10, where the average increase in incidents two weeks after rollout is plotted. Most rollouts show a significant increase. The negative value occurred during the summer holiday period time, when less people recorded and reported incidents.

The values provided have been incorporated for illustrative purposes. It is hardly surprising that if the number of projects participating in a rollout increases, then the number of incidents increases as well. Figure 11 shows a correlation coefficient of 0.4 between the average increase of incidents two weeks after rollout and the number of participating projects per release. If we consider the complexity of a rollout as introduced above, see also Fig. 5, which is the total number of interactions between Project Teams and Core Teams participating in a rollout, then we obtain a similar correlation coefficient of 0.36, see Fig. 12. The high level of incidents provides a good primary source of information on how well the rollout and release performed. However, these data should only be used indicatively and need to be interpreted carefully, as:

• processes or the ways processes are managed change over time, such as the above-mentioned introduction of a service-request process, which reduced the number of incidents

• some projects separate their cut-over and point-of-use. Incidents can, for example, pop up two or three weeks after rollout once the software goes live

• the number of incidents recorded is impacted by seasonal variability, such as holiday periods and Christmas

• changes are conducted after and prior to rollout, and can significantly influence the statistics.

In order to learn what actually caused these incidents they need to be examined in a more detailed manner, which is the purpose of the Lessons-learned Workshop recommended here. The Lessons-learned Workshop should be attended by all key personnel involved, such as the Project Manager/Cut-over Manager of the participating projects, Test Manager, and staff involved in the resolution of 2nd and 3rd level incidents—just to mention some of them. In order to run such a workshop it is essential to identify the incidents/service requests most frequently caused by the rollout and to classify them. These incident categories are used solely for rollout-evaluation purposes and additionally to the ordinary classification scheme, which is part of the incident-management process.

Table 3 provides an example of typical incident categories used after rollout evaluation. Between 60 and 80% of the incidents evaluated have usually not been detected by the test. Incidents caused due to bad Rollout Schedule or descriptions are rare. This suggests that the evaluation of incidents as such is more a measure of the test quality than of the Rollout Planning and execution performance.

Table 3:

Incident categories used solely for rollout evaluation.

This classification scheme isused in addition to the schema already facilitated by the incident management process, which is not further described here.

Incident category	Description
Error within release/ not detected through testing	Incident was caused by a software error which was not detected in testing. This category is also used for service requests, which fix release-specific errors, e.g., in the form of patches.
Human failure	This category is used for incidents caused by human failure, such as faulty execution of patches or noncompliance with agreements.
Incident caused by project	Incident is related to a project; this category makes particular sense if a significant number of incidents are related to a particular project.
Rollout Schedule requires change	Activity required to be conducted was not properly described. If it is a reoccurring activity then the description needs to change.
Expected behaviour	Irregularities recorded are considered to be an accepted and expected behaviour of the system or application.
Sequential error within the Rollout Schedule (application-specific)	Incident is caused by a sequential error within the Rollout Schedule; activity was requested to be conducted by an application which regularly participates during the rollout.
Sequential error within the Rollout Schedule (project-specific)	Incident is caused by a sequential error within the Rollout Schedule; the activity was requested to be conducted by a Project Team.
Bugfix	This category relates to incident-response measures which occur prior to the rollout and which aim to eliminate software error. It is merely a convention to relate them to the Rollout Change, which also fixes these errors.
Missing contact person	The contact person who logged this change could not be contacted. The number of incidents assigned to this category should be kept small.
Incident is not reproducible	The incident cannot be reproduced.
Wrongly connected	Incident was not related to the Rollout Change and was wrongly related.
Not classifiable	Incident cannot be classified with available classification; the classification is of course extended with additional categories where required.

DOI: 10.7717/peerj-cs.29/table-3

One of the challenges these workshops face is related to the proper assignment of incidents to the Rollout Change. The application-specific increase of 2nd level incidents after rollout provides a good indication of the expected number of incidents required for analysis.

The second KPI introduced is specifically designed to measure the planning and execution performance of a rollout. Here, the scheduled time is compared with the actual execution time. The measurement can easily be conducted by the inclusion of communication items within the Rollout Schedule, such as e-mails. These e-mails are sent by the rollout personnel shortly prior to execution of critical items, here called ‘measurement points’. The time stamps of these e-mails provide a measure of the actual execution time. Figure 13 illustrates the discrepancy between the scheduled time (open circles) and actual execution time (closed circles). If the closed circles are below the open ones then the actual rollout was executed quicker than foreseen. If vice versa, a delay occurred. Figure 13 illustrates that the activity related to measurement point 2 finished sooner than foreseen. The activity related to measurement point 3, however, was delayed. Such a diagram provides a good basis for a structured discussion on why certain rollout activities started later or finished earlier.

Conclusions

Within this publication a deployment framework is presented which illustrates how good ITIL management practices can be aligned with a project-management model in order to assure timely delivery of what this paper refers to as ‘project-specific cut-over’. This alignment is shown by the use of a semi-agile release model. Due to the release model selected, which is particularly characterized by its bundling of projects for what is here referred to as ‘release-specific rollout’, a new definition and interpretation of ITIL’s generic view of deployment is required. The facilitated release model additionally required two types of go-live scenarios: one for each participating project and one for each release.

In order to finalize a project-specific cut-over punctually, a variety of cut-over related deliverables are presented and chronologically assigned to a typical IT project managed in a cascading manner. Projects already need to take into account cut-over related deliverables during the initial planning phase. It is also shown how project-specific outcomes interrelate with release-specific planning activities and are finally aggregated within the Rollout Handbook and proposed as Rollout Change for approval by the CAB. Emphasis is placed on the Lessons-learned Workshop recommended to be conducted a couple of weeks after the rollout, once the level of incidents has normalised. This Workshop should be guided by the rollout/cut-over related incidents, and a comparison of the scheduled time and actual execution time of the rollout should be included in the Workshop.

Even though consulting methodologies such as ASAP (Accelerated SAP) recommend scattered, project-specific deliverables useful for Cut-over Planning, this publication offers an integrated approach on how to systematically prepare for a project-specific cut-over with all required deliverables. The framework provided extends the ITIL release and deployment process by means of IT projects and allows IT projects to interface easily with the ITIL change-management process.

The framework has proven to be successful since it forces projects to think already during the planning phase about cut-over related topics. After implementation of this framework we see projects better prepared prior entering the release. However, we also do have projects which wish to cut-over by the use of their own methodology. Probably this formal approach is not everybody’s cup of tea.

Supplemental Information