From Monolithic Systems to Microservices: An Assessment Framework

Context. Re-architecting monolithic systems with Microservices-based architecture is a common trend. Various companies are migrating to Microservices for different reasons. However, making such an important decision like re-architecting an entire system must be based on real facts and not only on gut feelings. Objective. The goal of this work is to propose an evidence-based decision support framework for companies that need to migrate to Microservices, based on the analysis of a set of characteristics and metrics they should collect before re-architecting their monolithic system. Method. We designed this study with a mixed-methods approach combining a Systematic Mapping Study with a survey done in the form of interviews with professionals to derive the assessment framework based on Grounded Theory. Results. We identified a set consisting of information and metrics that companies can use to decide whether to migrate to Microservices or not. The proposed assessment framework, based on the aforementioned metrics, could be useful for companies if they need to migrate to Microservices and do not want to run the risk of failing to consider some important information.


Introduction
Microservices are becoming more and more popular.Big players such as Amazon 1 , Netflix 2 , Spotify 3 , as well as small and medium-sized enterprises are developing Microservices-based systems [1].
Microservices are relatively small and autonomous services deployed independently, with a single and clearly defined purpose [2].Microservices propose vertically decomposing applications into a subset of business-driven independent services.Each service can be developed, deployed, and tested independently by different development teams and using different technology stacks.Microservices have a variety of different advantages.They can be developed in different programming languages, can scale independently from other services, and can be deployed on the hardware that best suits their needs.Moreover, because of their size, they are easier to maintain and more fault-tolerant since the failure of one service will not disrupt the whole system, which could happen in a monolithic system.However, the migration to Microservices is not an easy task [1] [3].Companies commonly start the migration without any experience with Microservices, only rarely hiring a consultant to support them during the migration [1] [3].
Various companies are adopting Microservices since they believe that it will facilitate their software maintenance.In addition, companies hope to improve the delegation of responsibilities among teams.Furthermore, there are still some companies that refactor their applications with a Microservicesbased architecture just to follow the current trend [1] [3].
The economic impact of such a change is not negligible, and taking such an important decision to re-architect an existing system should always be based on solid information, so as to ensure that the migration will allow achieving the expected benefits.
In this work, we propose an evidence-based decision support framework to allow companies, and especially software architects, to make their decision on migrating monolithic systems to Microservices based on the evaluation of a set of objective measures regarding their systems.The framework supports companies in discussing and analyzing potential benefits and drawbacks of the migration and re-architecting process.
We designed this study with a mixed-methods empirical research de-1 https://gigaom.com/2011/10/12/419-the-biggest-thing-amazon-got-right-theplatform/ 2 http://nginx.com/blog/Microservices-at-netflix-architectural-best-practices/ 3www.infoq.com/presentations/linkedin-Microservices-urnsign.We first performed a systematic mapping study of the literature to classify the characteristics and metrics adopted in empirical studies that compared monolithic and Microservices-based systems.Then we ran a set of interviews with experienced practitioners to understand which characteristics and metrics they had considered during the migration and which they should have considered, comparing the usefulness of the collection of these characteristics.Finally, based on the application of Grounded Theory on the interviews, we developed our decision support framework.Paper structure.Section 2 presents the background and related work.In Section 3, we describe the mixed-methods research approach we applied.In Section 4, we describe the Systematic Mapping Study, focusing on the protocol and the results, while Section 5 presents the design and the results of the survey.In Section 6, we present the defined framework.In Section 7, we discuss the results we obtained and the defined framework.In Section 8, we identify threats to the validity of this work.Finally, we draw conclusions in Section 9 and highlight future work.

Background and Related Work
In this section, we will first introduce Microservices and then analyze the characteristics and measures adopted by previous studies.

Microservices
The Microservice architecture pattern emerged from Service-Oriented Architecture (SOA).Although services in SOA have dedicated responsibilities, too, they are not independent.The services in such an architecture cannot be turned on or off independently.This is because the individual services are neither full-stack (e.g., the same database is shared among multiple services) nor fully autonomous (e.g., service A depends on service B).As a result, services in SOA cannot be deployed independently.
In contrast, Microservices are independent, deployable, and have a lot of advantages in terms of continuous delivery compared to SOA services.They can be developed in different programming languages, can scale independently from other services, and can be deployed on the hardware that best suits their needs because of their autonomous characteristics.Moreover, their typically small size facilitates maintainability and improves the fault tolerance of the services.One consequence of this architecture is that the failure of one service will not disrupt the whole system, which could happen in a monolithic system [2].Nevertheless, the overall system architecture changes dramatically (see Figure 1).One monolithic service is broken down into several Microservices.Thus, not only the service's internal architecture changes, but also the requirements on the environment.Each Microservice can be considered as a full-stack that requires a full environment (e.g., its own database, its own service interface).Hence, coordination among the services is needed.Despite the novelty of the field of Microservices, many studies concerning specific characteristics of them have already been published.However, there are still some challenges in understanding how to develop such kinds of architectures [4] [5] [6].A few studies in the field of Microservices (i.e., [3], [7], [8], [9], [10], and [11]) have synthesized the research in this field and provide an overview of the state of the art and further research directions.
Di Francesco et al. [7] studied a large corpus of 71 studies in order to identify the current state of the art on Microservices architecture.They found that the number of publications about Microservices sharply increased in 2015.In addition, they observed that most publications are spread across many publication venues and concluded that the field is rooted in practice.In their follow-up work, Di Francesco et al. [8], provided an improved version, considering 103 papers.
Pahl et al. [11] covered 21 studies.They discovered, among other things, that most papers are about technological reviews, test environments, and use case architectures.Furthermore, they found no large-scale empirical evaluation of Microservices.These observations made them conclude that the field is still immature.Furthermore, they stated a lack of deployment of Microservice examples beyond large corporations like Netflix.
Soldani et al. [3] identified and provided a taxonomic classification comparing the existing gray literature on the pains and gains of Microservices, from design to development.They considered 51 industrial studies.Based on the results, they prepared a catalog of migration and re-architecting patterns in order to facilitate re-architecting non-cloud-native architectures during migration to a cloud-native Microservices-based architecture.
All studies agree that it is not clear when companies should migrate to Microservices and which characteristics the companies or the software should have in order to benefit from the advantages of Microservices.
Thus, our work is an attempt to close this gap by providing a set of characteristics and measures together with an assessment framework, as planned in our previous proposal [12].

The Approach
In this section, we will describe the two-step mixed-methods approach applied in this work.The approach is shown in Figure 2.
The goal of this work is to understand which metrics are considered important by practitioners before and after the migration to Microservices.Therefore, we decided to conduct a survey based on semi-structured interviews.
In order to avoid bias due to open-answer questions, we first performed a Systematic Mapping Study to identify a list of characteristics and measures considered in previous works for the identification of potential benefits and issues of the migration to Microservices.
Then we conducted the survey among professionals to identify in practice which metrics they considered important before and after the migration, asking them to first report the metrics they considered useful as open questions, and then asking whether they considered the metrics used in the previous studies useful.
In the next section (Section 4), we will report on the mapping study process, and in Section 5, we will describe the survey design and the results obtained.

Characteristics and measures investigated in empirical studies on Microservices
In this section, we aim to identify the characteristics and measures that companies should collect before re-architecting their monolithic system into Microservices in order to enablethem to make a rational decision based on evidence instead of gut feeling.Therefore, the goal of this work is twofold: First, we aim to characterize which characteristics have been adopted in empirical studies to evaluate the migration from monolithic systems to Microservices.Second, we aim to map the measures adopted to measure the aforementioned characteristics.
The contribution of this section can be summarized as follows: We identify and classify the different characteristics and measures that have been studied in empirical studies comparing monolithic systems with Microservices architectures.These measures will be used in the survey presented in Section 5.

Methodology
Here, we will describe the protocol followed in this Systematic Mapping Study.We will define the goal and the research questions (Section 4.1.1)and report the search strategy approach (Section 4.1.2)based on the guidelines defined by Petersen et al. [13,14] and the "snowballing" procedure defined by Wohlin [15].We will also outline the data extraction and the analysis (Section 4.1.3) of the corresponding data.The adopted protocol is depicted in Figure 3.

Goal and Research Questions
The goal of this Systematic Mapping Study is to analyze the characteristics and measures considered in empirical studies that evaluated the migration from monolithic systems to Microservices or that evaluated Microservices.For this purpose, we addressed the following research questions (RQs): RQ1.Which characteristics have been investigated during the analysis of the migration from monolithic systems to Microservices architectures?With this RQ, we aim to classify the characteristics reported by the empirical studies that analyzed the migration from monolithic systems to Microservices.
RQ2.What measures have been adopted to empirically evaluate the characteristics identified in RQ1?For each characteristic, we identified the measures adopted for the evaluation of the migration to Microservices.

RQ3. What effects have been measured after the migration to Microservices?
With this RQ, we aim to analyze the results reported in the measures identified in RQ2.For example, we aim to understand whether the selected studies agree about the decreased maintenance effort of Microservices expected by numerous practitioners [1].

Search Strategy
We adopted the protocol defined by Petersen et al. [13,14] for a Systematic Mapping Study and integrated it with the systematic inclusion of references -a method also referred to as "snowballing"-defined by Wohlin [15].
The protocol involves the outline of the search strategy including bibliographic source selection, identification of inclusion and exclusion criteria, definition of keywords, and the selection process that is relevant for the inclusion decision.The search and selection process is depicted in Figure 3.  Bibliographic Sources.We selected the list of relevant bibliographic sources following the suggestions of Kitchenham and Charters [16], since these sources are recognized as the most representative in the software engineering domain and are used in many reviews.The list includes: ACM Digital Library, IEEEXplore Digital Library, Science Direct, Scopus, Google Scholar, Citeseer Library, Inspec, Springer Link.
Inclusion and Exclusion Criteria.We defined inclusion and exclusion criteria based on the papers' title and abstract in order to identify the most relevant papers.We obtained the final criteria by means of refinements from an initial set of inclusion and exclusion criteria.
Inclusion criteria.The selected papers fulfilled all of the following criteria: • Relation to Microservices migration can be deduced.
• Study has to provide empirical evidence (measures) between the previous monolithic system and the refactored Microservices-based system.Examples of measures may include maintenance effort, costs, infrastructure costs, response time, and others.
Exclusion criteria.Selected papers not fulfilling any of the following criteria were left out: • Not written in English.
• Duplicated paper (only the most recent version was considered).
• Not published in a peer-reviewed journal or conference proceedings before the end of July 20174 .
• Short paper, workshop paper, and work plan (i.e., paper that does not report results).
Definition of Search Keywords.We defined search keywords based on the PICO [16] structure5 as reported in Table 1.The symbol * allowed us to capture possible variations in the search terms such as plurals and verb conjugations.
Search and Selection.The application of the search keywords returned 142 unique papers.Next, we applied the inclusion and exclusion criteria to the retrieved papers.As suggested by Kitchenham and Brereton [17], we first tested the applicability of the inclusion and exclusion criteria: 1.A set of 15 papers was selected randomly from the 142 papers.
2. Three authors applied the inclusion and exclusion criteria to these 15 papers, with every paper being evaluated by two authors.
3. On three of the 15 selected papers, two authors disagreed and a third author joined the discussion to clear up the disagreements.
The refined inclusion and exclusion criteria were applied to the remaining 127 papers.Out of 142 initial papers, we excluded 70 by title and abstract and another 61 after a full reading process.We settled on 11 papers as potentially relevant contributions.In order to retrieve all relevant papers, we additionally integrated the procedure of also taking into account forward and backward systematic snowballing [15] on the 11 remaining papers.As for backward snowballing, we considered all the references in the papers retrieved, while for the forward snowballing we evaluated all the papers referencing the retrieved ones, which resulted in one additional relevant paper.Table 2 summarizes the search and selection results obtained.This process resulted in us retaining 12 papers for the review.The list of these 12 papers is reported in Table 3. 4.1.3.Data Extraction Data addressing our RQs was extracted from each of the 12 papers that were ultimately included in the review.For this purpose, two of the authors extracted the data independently and then compared the results.If the results differed, a third author verified the correctness of the extraction.
Our goal was to collect data that would allow us to characterize the measures that can be used to evaluate the migration to Microservices.Two groups of data were extracted from each primary study: • Context data.Data showing the context of each selected study in terms of: the goal of the study, the source of the data studied, the number of Microservices developed, the application area (i.e., insurance system, banking system, room reservation, ...), and the programming language of the studied system(s).
• Empirically Evaluated Characteristics.Data related to the characteristics under study (e.g., maintenance, cost, performance, ...) and the measures adopted in the studies (e.g., number of requests per minute, cyclomatic complexity, ...).

Data Synthesis
The data extracted from each paper was aggregated and summarized by two of the authors in order to better answer our RQs.First, we identified and classified the set of characteristics to answer RQ2.Before identifying the technique to be used in the classification, we first screened the characteristics mentioned in the papers.Since the papers clearly reported the characteristics under study, using the same terminology (e.g., performance was always referred to as performance, maintenance as maintenance, and so on), we simply took all the categories as they were.
As for the measures adopted for measuring each characteristic, we followed the same process.In this case, the papers adopted different terms for similar measures, or in some cases only different units of measurement (e.g., number of requests per minute instead of number of requests per hour).In order to classify similar measures, three authors proposed their own classification independently.Then they discussed the final classification in a workshop so as to resolve any incongruences.

Study Replicability
In order to allow replication and extension of our work, we prepared a replication package 6 for this Systematic Mapping Study, including the complete results obtained.

Results
In this section, we will answer our research questions based on the data extracted from each selected paper.
In order to get a general overview of the selected papers, we extracted information on publication year, type, and venue for each publication.We are aware that the limited number of selected studies is not enough to draw statistical conclusions.However, the results of this RQ help to understand the growing trend of works in this domain.
Publication per year.The term "Microservice" was introduced in 2012.Therefore, we did not consider any work before 2012.The scientific interest in Microservices and in particular in empirical studies evaluating the migration from monolithic systems has increased in recent years.The twelve selected papers were all published between 2015 and 2017.No relevant papers were found before 2015. 6The raw data is temporarily stored on Google Drive: https://drive.google.com/open?id=1GNnDMmNckRgrxk6yEw8wsiHOT2iFDWEj.The data will be moved to a permanent repository in case of acceptance.Publication type and venue.The selected papers appeared in eleven different publication venues, including ten international conferences and one national conference.Specifically, the papers were published in: Symposium on the Foundations of Software Engineering (FSE 2015)

Studied Characteristics (RQ1)
In order to better answer RQ1, we briefly present an overview of the papers in terms of the research strategy, the adopted evaluation approach, and the main purpose of each study.
The selected studies focus mainly on analyzing the migration benefits and challenges ([s1], [s3], [s11], [s12]).The other subjects on which they focus are distributed system architectures ([s2], [s12]) and evaluation models and frameworks to validate the performance ( Addressed characteristics.In order to better classify the results, we distinguish between product and process characteristics.Moreover, we also consider cost as an organizational characteristic.The selected studies mainly focus on product characteristics ( Regarding the product characteristics, we identified four sub-characteristics: performance, scalability, availability, and maintenance.We also divided cost comparison into personnel and infrastructure costs. The most frequently addressed characteristic is performance (see Table 5).In detail, the papers [s11] have a focus on performance.This is followed by scalability, which is discussed by the papers [s10], and [s11].Other characteristics like availability ([s4], [s9]) or maintenance ([s1],[s5], [s7], [s12]) are considered only in a few papers.
Overall, we identified the following characteristics as reported in Tables 4, 5, and 6:

Measures Adopted to Evaluate Characteristics (RQ2)
Two authors analyzed each paper and identified 18 measures for the three main characteristics considered in RQ1, as depicted in Figure 4 and reported in Tables 4, 5, and 6.
Product-related measures.We identified 13 measures (Table 4) for the four identified sub-characteristics (performance, scalability, availability, and maintenance).
From the obtained results, we can see that the highest number of measures is related to performance and scalability, where we identified a total of nine studies referring to them.Among them, response time, number of requests per minute or second, and waiting time are the most commonly addressed measures.For availability, we derived only three measures and for maintainability only two.
Process-related measures.Seven studies investigated the migration process using three factors: development independence between teams, usage of continuous delivery, and reusability (Table 5).These three factors can be considered as "Boolean measures" and can be used by companies to understand whether their process can be easily adapted to the development of Microservices-based systems.
Existing independent teams could easily migrate and benefit from the independence freedom provided by Microservices.Continuous delivery is a must in Microservices-based systems.The lack of a continuous delivery pipeline eliminates most of the benefits of Microservices.Reusability is amplified in Microservices.Therefore, systems that need to reuse the same business processes can benefit more from Microservices, while monolithic systems in which there is no need to reuse the same processes will not experience the same benefits.
Besides the analyzed characteristics, the papers also discuss several processrelated benefits of the migration.Technological heterogeneity, scalability, continuous delivery support, and simplified maintenance are the most frequently mentioned benefits.Furthermore, the need for recruiting highly skilled developers and software architects is considered as a main motivation for migrating to Microservices.
Cost-comparison-related measures.As for this characteristic, three studies include it in their analysis and consider three measures for the comparison (Table 6).Path length: The number of CPU instructions to process a client request.
[s2] reports that the length of the code path of a Microservice application developed using Java with a hardware configuration of one core, using a bare process, docker host, and docker bridge, is nearly twice as high as in a monolithic system.8% Usage of containers: The usage of containers can influence performance, since they need additional computational time compared to monolithic applications deployed in a single container.
[s7] reports that the impact of containers on performance might not always be negligible.Reusability: Microservices are designed to be independent of their environment and other services [s11].This facilitates their reusability.

Microservices Migration Effects (RQ3)
The analysis of the characteristics and measures adopted in the empirical studies considered in this review allowed us to classify a set of measures that are sensitive to variations when migrating to Microservices.The detailed mapping between the benefits and issues of each measure is reported in Table 4.
Product Characteristics.Regarding product characteristics, performance is slightly reduced in Microservices.
When considering the different measures adopted to measure performance, the usage of containers turned out to decrease performance.This is also confirmed by the higher number of CPU instructions needed to process a client request (path length), which is at least double that of monolithic systems and therefore results in high CPU utilization.However, the impact  of the usage of different programming languages in different services is negligible.Even if different protocols have different interpreters for different languages, the computational time is comparable.When considering scalability, Microservices-based systems outperform monolithic systems.Compared to monolithic systems, response time is lower in Microservices.However, when the number of requests grows, Microservices are easier to scale, mainly because of their relatively small size, and can keep on serving clients with the same response time, whereas monolithic systems commonly decrease their response time when the number of requests peaks.
Taking into account availability, [s4] and [s9] report that Microservices can be affected by higher availability problems.This is due to the higher number of connected components, which, in the event of a failure, could disrupt the whole system.Although several practitioners claim the opposite -that Microservices are more robust, and that in the event of the failure of one Microservice, the remaining part of the system will still be available [3][1] -the systems analyzed by [s4] and [s9] seemed to suffer from lower availability compared to the previous monolithic systems.
Maintenance is considered more expensive in the selected studies.The selected studies agree that the maintenance of a single Microservice is easier than maintaining the same feature in Microservices.However, testing is much more complex in Microservices [s7], and the usage of different programming languages, the need for orchestration, and the overall system architecture increase the overall maintenance effort.
Cost-related measures The development of Microservices-based systems is reported to be more expensive than the development of monolithic systems [s5].Moreover, infrastructure costs are usually also higher for Microservices than for monolithic systems [s1][s3].

The Survey
In this section, we will present the survey we performed and its results.We will describe the research questions, the study design, the execution, and the data analysis, as well as the results of the survey.

Goal and Research Questions
We conducted a case study among developers and professionals in order to identify in practice which metrics they considered important before and after migration based on the results obtained in the Systematic Mapping Study.
Based on our goal, we derived the following research questions (RQs): RQ1.Why did companies migrate to Microservices?With this RQ, we aim to understand the main reasons why companies migrated to Microservices, i.e., to understand whether they considered only metrics related to these reasons or other aspects as well.For example, we expect that companies that migrate to increase velocity considered velocity as a metric, but we also expect them to consider other information not related to velocity, such as maintenance effort or deployment time.RQ2.Which information/metrics was/were useful before, during, and after the migration?
With this RQ, we want to understand the information/metrics that companies considered as decision factors for migrating to Microservices.However, we are also interested in understanding whether they also collected this information/these metrics during and after the development of Microservicesbased systems.RQ3.Which information/metrics was/were considered useful by the practitioners?With this RQ, we want to understand which information/metrics practitioners collected and considered useful to collect during the migration process, and which they did not collect but now believe they should have collected.

Study Design
The information was collected by means of a questionnaire composed of three sections, as described in the following: 1. Demographic information: In order to define the respondents' profile, we collected demographic background information.This information considered predominant roles and relative experience.We also collected company information such as application domain, organizations size via number of employees, and number of employees in the respondents' own team.

Project information:
We collected the following information on the project migrated to Microservices: creation and migration dates of the project, dimension of the application in terms of number of Microservices, and number of releases.
3. Migration information/metrics: This section was composed of three main questions: • Which information/metrics were considered before the migration, to decide if migrate or not?
• Which information/metrics you did not consider, but you think you should have considered before the migration, to decide if migrate or not?
• Which information/metrics were considered as useful after the migration?
• Ranking of the usefulness of the metrics identified in Section 4.1, by means of a 6-point Likert scale, where 1 means absolutely not useful and 6 extremely useful.
• Report any information/measure not easy to collect 4. Perceived usefulness of the collected information/metrics: In this section, we collected information on the usefulness of an assessment framework based on the metrics identified and ranked in the previous section.The goal was to understand whether the set of metrics could be useful for deciding whether to migrate a system or not in the future.
This section was based on three questions: • Ranking of the usefulness of an assessment framework based on the previous information, before the migration to Microservices.This question was answered with the same 6-point Likert scale adopted for the previous questions.
• Do you think the factors or measures support a reasoned choice of migrating or not?(if not, please motivate) • Would you use this set of factors and measures in the future, in case of migration of other systems to Microservices?If not, please motivate.
The questionnaire adopted in the interviews is reported in Appendix 2.

Study Execution
The survey was conducted over the course of five days, during the 19 th International Conference on Agile Processes in Software Engineering, and Extreme Programming (XP 2018).We interviewed a total of 52 practitioners.We selected only experienced participants and did not consider any profiles coming from academia, such as researchers or students.

Data Analysis
Two authors manually produced a transcript of the answers of each interview and then provided a hierarchical set of codes from all the transcribed answers, applying the open coding methodology [18].The authors discussed and resolved coding discrepancies and then applied the axial coding methodology [18].
Nominal data was analyzed by determining the proportion of responses in each category.Ordinal data, such as 5-point Likert scales, was not converted into numerical equivalents since using a conversion from ordinal to numerical data entails the risk that any subsequent analysis will yield misleading results if the equidistance between the values cannot be guaranteed.Moreover, analyzing each value of the scale allowed us to better identify the potential distribution of the answers.Open questions were analyzed via open and selective coding [18].The answers were interpreted by extracting concrete sets of similar answers and grouping them based on their perceived similarity.

Replication
In order to allow replication and extension of our work, we prepared a replication package including the questionnaire with the complete results obtained7 .

Results
In this section, we will report the obtained results, including the demographic information regarding the respondents, information about the projects migrated to Microservices, and the answers to our research questions.
Demographic information.The respondents were mainly working as developers (31 out of 52) and project managers (11 out of 52), as shown in Table 7.The majority (23 out of 52) of them had between 2 and 5 years of experience in this role (Table 8).Regarding company information, out of the 52 respondents, 10 worked in IT consultant companies, 6 in software houses, 8 in e-commerce, and 6 in banks.The remaining 9 respondents who provided an answer worked in different domains (Table 9).The majority of the companies (15 out of 52 respondents) were small and medium-sized enterprises (SMEs) with a number of employees between 100 and 200, while 9 companies had less than 50 employees.We also interviewed people from 3 large companies with more than 300 employees (Table 11).Regarding the team size, the vast majority of the teams had less than 50 members (33 out of 52 respondents).14 teams had less than 10 members, 12 teams had between 10 and 20 members, and 7 teams had between 20 and 50 members.Only one team was composed of more than 50 members (Table 10).Project information As for the project's age (Table 12), about 69% of the respondents (36 out of 52) started the development less than 10 years ago, while 9 interviewees created the project between 10 and 15 years ago.Another 8 interviewees referred to projects with an age between 15 and 20 years, while 5 respondents started the development more than 20 years ago.As for the migration to Microservices, 23 respondents reported that the process had started 2 years ago or less, while for 20 interviewees the process had started between 2 and 4 years ago.

Migration Motivations (RQ1)
In answers to the question about the interviewees' motivation to migrate from their existing architecture to Microservices, a total of 97 reasons were mentioned.The open coding of the answers classified the 97 reasons into 22 motivations.In Figure 5, all motivations that were mentioned three or more times are presented.The three main motivations are maintainability, deployability, and team organization.The most commonly mentioned motivation was to improve the maintainability of the system (19 out of 97).They reported, among other thing, that the maintenance of the existing system had become too expensive due to increased complexity, legacy technology, or size of the code base.
Deployability was another important motivation for many interviewees (12 out of 97).They expected improved deployability of their system after the migration.The improvement they hoped to achieve with the migration was a reduction of the delivery times of the software itself as well as of updates.Moreover, some interviewees saw the migration as an important enabler for automated deployment (continuous deployment).
The third most frequently mentioned motivation was not related to expected technical effects of the migration but was organizational in nature, namely team organization (11 out of 97).With the migration to Microservices, the interviewees expected to improve the autonomy of teams, delegate the responsibility placed on teams, and reduce the need for synchronization between teams.
The remaining motivations like cost, modularity, willingness, or complexity seem to be motivations that are part of the three main motivations discussed above, or at least influence one of them.For example, complexity was often mentioned in combination with maintenance, or scalability together with team organization.Thus, it appears that these three motivations are the main overall motivations for the migration from monoliths to Microservices.5.6.2.Information/metrics considered before, during, and after the migration (RQ2) We collected 46 different pieces of information/metrics, which were considered a total of 107 times by the interviewees before or during the migration to Microservices.The most commonly mentioned ones were the number of bugs, complexity, and maintenance effort (see Table 14).Considering the information/metrics that is/are of interest for the interviewees after migration to Microservices, 26 clearly distinguishable types were identified that were mentioned a total of 66 times by the participants.Again, the number of bugs, complexity, and maintenance effort were the most frequently mentioned ones.(see Table 15) As expected, the vast majority of the considered information/metrics was aimed at measuring characteristics related to the migration motivations.As maintainability was the most important reason to migrate to Microservices, maintainability-related metrics turned out to be the most important metrics considered before the migration.It is interesting to note that in some cases, companies collected this information before the migration but stopped collecting it during and after the migration (e.g., 4 interviewees out of 16 who had collected the number of bugs in their monolithic system did not collect the same information in the Microservices-based system).
The results suggest that the most important information needs remain the same from the start of the migration until its completion.Thus, there may be a set of migration information/metrics that is fundamentally impor-tant for the process of migration and that should be collected and measured throughout the migration.

Information/metrics considered useful (RQ3)
In this section, we will report the results on the perceived usefulness of an assessment framework based on the metrics reported above.
Asking the interviewees how easy they think it is to collect the factors and measures, 41 answered that they considered it easy, while 10 did not consider it easy (one interviewee did not provide an answer to this question).Regarding the follow-up question about which metrics are not easy to collect, 20 different metrics were mentioned.The answer given most often (6 times) was complexity.Other metrics that were stated by the interviewees were testability, response time, benchmark data, and availability (see Table 16).The usefulness of the discussion of the set of information/metrics before migration was confirmed by the majority of the interviewees (see Table 17).Almost all interviewees categorized the usefulness of the metrics as "very/a lot" (24 out of 52) or "absolutely" (25 out of 52).Furthermore, all but three interviewees confirmed that they believed that the metrics support a rational choice on whether to migrate or not.Finally, 65% (34 out of 52) of the interviewees stated that they would use the set of information/metrics in the future,

The Assessment Framework
In this section, we propose an assessment framework based on the characteristics that should be considered before migration.
The goal of the framework is to support companies in reasoning about the usefulness of migration and make decisions based on real facts and actual issues regarding their existing monolithic systems.
Based on the results obtained in our survey (Section 5), we grouped the different pieces of information and metrics into homogeneous categories, based on the classification proposed by the ISO/IEC 25010 standard [19].However, we also considered two extra categories not included in ISO/IEC 25010, which focus on product characteristics, namely cost and processes.
The framework is applied in four steps: Step 1 Motivation reasons identification Step 2 Metrics identification Step 3 Migration decisions Step 4 Migration In the next sub-sections, we will describe each of the four steps in detail.

Motivations reasons identification
Before migrating to Microservices, companies should clarify why they are migrating and discuss their motivation.As highlighted by previous studies [1] [3], companies migrate to Microservice for various reasons and often migrate to solve some issues that need to be solved differently.Moreover, sometimes the migration can have negative impacts, for instance when companies do not have enough expertise or only have a small team that cannot work on different independent projects.The quality characteristics listed in Table 18 could be used as a checklist to determine whether there is some common problem in the system that the company intends to solve with the migration.
Based on the motivation, companies should reason -optimally including the whole team in the process -on whether the migration could be the solution to their problems or whether it could create more issues than benefits.If, for any reason, it is not possible to include the whole team in this discussion, we recommend including at least the project manager and a software architect, ideally with knowledge about Microservices.
In case the team still wants to migrate to Microservices after this initial discussion, it could start discussing how to collect the metrics (Step 2).

STEP 2 -Metrics Identification
In order to finalize the decision on whether or not to migrate to Microservices, teams should first analyze their existing monolithic system.The system should be analyzed by considering the metrics reported in Table 18.
We recommend starting by considering the information and metrics related to the motivation for the migration.However, for the sake of completeness, we recommend discussing the whole set of metrics.For example, if a team needs to migrate to Microservices because of maintenance issues, they should not only consider the block "maintenance" but should also consider the remaining metrics, since other related information such as the independence between teams (process-related) could still be very relevant for maintenance purposes.
The list of metrics reported in Table 18 is not meant to be complete for each characteristic, but is rather to be used as a reference guide for companies to help them consider all possible aspects that could affect the migration.For example, a company's monolithic system might suffer from performance issues (characteristic "Functional Suitability").The analysis of the sub-characteristics will help them to reason about "Overall performance", but they could also consider whether it is a problem related to "Time behavior" by analyzing the metric "Response time" and also considering the other sub-characteristics listed.However, if the motivation of the performance issue is different, the company will also be able to reason about it.

Migration Decisions
After a thorough discussion of the collected metrics, the team can decide whether to migrate or not based on the results of the discussion performed in the previous step.
For example, there will be cases where a company may decide not to migrate after all.If the company realizes that the reason for the low performance is due to the inefficient implementation of an algorithm, they might decide to implement it better.If the main issue is cost of maintenance and the company wants to migrate mainly to reduce this cost, they might think of better team allocation or reason about the root causes of the high costs, instead of migrating with the hope that the investment will enable them to save money.

Migration
The team can then start the migration to Microservices.During this phase, we recommend that companies automate measurement of the rele-vant metrics and set up measurement tools to continuously collect relevant information as identified in Step 2.

Discussion
In this section, we will discuss the implications of the Systematic Mapping Study and the survey we performed as well as the assessment framework we developed.
The Systematic Mapping Study we performed enables us to group the main characteristics for migrating to Microservices into three main groups: product, process, and cost.Considering the number of papers that cover the respective characteristics, we can observe a focus on product-related characteristics, i.e., availability, scalability, performance, and maintenance.Although these characteristics are important for migration (for instance, in order to make informed decisions), other characteristics such as reliability, robustness, or understandability are missing.One reason for the focus on the covered characteristics may be the environment in which Microservices were "invented".This is an environment with comprehensive services that have an ever growing number of features, increasing complexity, and the need for scalability to handle multiple requests with high performance, probably in the cloud.
The identified characteristics are in line with the results of our survey.The vast majority of the interviewees migrated to Microservices in order to improve maintainability.However, deployability, team organization (such as the independence between teams), and cost are also important characteristics mentioned frequently in the interviews.Modularity, complexity, fault tolerance, scalability, and reusability were mentioned several times as well.
The proposed framework therefore covers characteristics and sub-characteristics that take the results of the Systematic Mapping Study and the survey into account and are aligned with the established ISO/IEC 25010 standard.The top-level characteristics are functional suitability, reliability, maintainability, cost, and process.The characteristics cover all the relevant subcharacteristics and metrics identified in the Systematic Mapping Study and the survey.For instance, modularity is a sub-characteristic of maintainability and scalability is a metric for performance efficiency.
Finally, the migration assessment framework suggests concrete metrics for measuring the characteristics.Our Systematic Mapping Study identified a total of 18 metrics related to characteristics that are relevant for the migration to Microservices.Given that all discussed characteristics are covered by metrics identified in the papers, the metrics can be used as an initial tool set to measure the main influencing factors for migrating a monolithic system to Microservices.Some characteristics are not easy to quantify, however.For instance, testability has effectiveness and efficiency aspects that can only be approximated by different metrics [20], like the degree of coverage or the number of defects covered.The survey was used to confirm the metrics found and to identify additional ones.The metrics most commonly mentioned in the survey are the number of bugs, complexity, and maintenance effort.It turns out that for the characteristics that are most relevant for migration, these metrics are also mentioned more often than for other characteristics.Maintainability is mentioned as the most important reason for migration, and maintainability-related metrics are also highlighted as the most important metrics.
In our study, we discovered that practitioners often do not properly measure their product, process, and cost before migrating to Microservices and realize only later (during or after migration) that relevant information is missing.Our proposed assessment framework should not only help to identify the most relevant characteristics and metrics for migration, but also make professionals aware of the importance of measurement before, during, and after migration to Microservices.In addition, there has not been a clear understanding what to measure before migrating to Microservices.Our proposed assessment framework intends to fill this gap.However, evaluation and refinement of the framework in industrial case studies is required as part of future work.

Threats to Validity
We applied the structure suggested by Yin [21] to report threats to the validity of this study and measures for mitigating them.We report internal validity, external validity, construct validity, and reliability.As we performed a mixed-methods approach comprising a Systematic Mapping Study and a survey, we will identify in this section different threats to validity regarding both parts of our study.

Threats to Validity regarding the Systematic Mapping Study
Internal Validity.We designed the review procedure (Section 4.1) based on the guidelines proposed by [13] and [14] and followed it rigorously.This protocol is the one used most frequently by researchers in the software engineering field.This confirms that we avoided any possible bias for the methodological design of the review process.We also reduced the bias related to the classification of metrics by having three authors propose their own classification independent of each other.The results of the final classification were then discussed in a workshop to resolve incongruences.However, we are aware that different authors might have come up with a different classification.
External Validity.Regarding the representation of the state of the art on Microservices, we avoided possible issues in the search and selection strategy by adopting a combination of automatic search in the bibliographic sources and backward-forward snowballing on the selected study references.We strictly excluded papers that were not peer-reviewed from our selected papers in order to ensure high quality of the results.
Construct Validity.Based on the protocol proposed by Kitchenham and Charters [16], we used the relevant bibliographic sources suggested by them.These sources are considered the most representative ones for the software engineering domain and are used in many reviews.We iteratively refined the inclusion and exclusion criteria by selecting a set of initial papers to test their performance with our goal.We tightened the strictness of the protocol methodology by applying a backward-forward snowballing process [15].Furthermore, We ensured inter-researcher agreement in the search and selection process.
Reliability.We were able to answer the defined research questions by strictly following the defined protocol.The fact that we designed our Systematic Mapping Study according to the most frequently used and strict guidelines ( [13], [14], and [15]) and that we published the raw data8 will allow other researchers to easily replicate our study.

Threats to Validity regarding the Survey
Internal Validity.One limitation that is always a part of survey research is that surveys can only reveal the perceptions of the respondents which might not fully represent reality.However, our analysis was performed by means of semi-structured interviews, which gave the interviewers the possibility to request additional information regarding unclear or imprecise statements by the respondents.The responses were analyzed and quality-checked by a team of four researchers.
External Validity.Overall, a total of 52 practitioners were interviewed at the 19 th International Conference on Agile Processes in Software Engineering, and Extreme Programming (XP 2018).We considered only expe-rienced respondents and did not accept any interviewees with an academic background.XP 2018 covers a broad range of participants from different domains who are interested in Microservices and the migration to Microservices.We therefore think that threats to external validity are reasonable.However, additional responses should be collected in the future.
Construct Validity.The interview guidelines were developed on the basis of the previously performed Systematic Mapping Study on Microservices migration.Therefore, the questions are aligned with standard terminology and cover the most relevant characteristics and metrics.In addition, the survey was conducted in interviews, which allowed both the interviewees and the interviewer to ask questions if something was unclear.
Reliability.The survey design, its execution, and the analysis followed a strict protocol, which allows replication of the survey.However, the open questions were analyzed qualitatively, which is always subjective to some extent, but the resulting codes were documented.

Conclusion
In this paper, we proposed an assessment framework to support companies in reasoning on the usefulness of the migration to Microservices.
We identified a set of characteristics and metrics that companies should discuss when they consider migrating to Microservices.The identification of these characteristics was performed by means of an industrial survey, where we interviewed 52 practitioners with experience in developing Microservices.The interviews were based on a questionnaire in which we asked the respondents to identify which metrics and characteristics had been adopted when they migrated to Microservices, which of these were useful, and which had not been adopted but should have been.The metrics were collected by means of open questions so as to avoid any bias of the results due to a set of predefined answers.After the open questions, we also asked the practitioners to check whether they had also collected some of the metrics proposed in the literature (which we had identified by means of a Systematic Mapping Study), and whether they believed it would have been useful to collect them.
The result of this work is an assessment framework that can support companies in discussing whether it is necessary for them to migrate or not.The framework will help them avoid migration if it is not necessary, especially when they might get better results by refactoring their monolithic system or re-structuring their internal organization.
Future work include the validation of the framework in industrial settings, and the identification of a set of automatically applicable measure, that could easily provide a set of meaningful information, reducing the subjectivity of the decisions.

Figure 5 :
Figure 5: Migration motivations mentioned by more than three participants

Table 1 :
Definition of Search Keywords

Table 2 :
Search and Selection Results

Table 3 :
Selected Papers

Table 4 :
Product-related measures The percentage of time the CPU is not idle.Used to measure performance.[s9]reportstherelationshipbetweenthenumber of VMs and the overall VMs utilization.In addition,[s11]analyzes the impact of the decision between VMs and containers on CPU utilization.16%Impact of programming language: Communication between Microservices is network-based.Most of the time is spent on network input and output operations rather than on processing of the request.Programming languages can influence communication performance due to the different ways that they implement the communication protocols.[s7]reports that the impact of the programming language on performance is negligible [s8].8%

Table 5 :
Process-related factors

Table 6 :
Cost-related measures 8%Development costs: [s5] argues that Microservices reduce the development costs given that complex monolithic applications are broken down into a set of services that only provide a single functionality.Furthermore, most changes affect only one service instead of the whole system.Infrastructure Cost 16% Cost per hour: Is a measure used to determine the infrastructure costs [s1].According to the experiment done in [s3], the Microservices architecture had lower infrastructure costs compared to monolithic designs.8% Cost per million requests: In comparison to cost per hour, this measure is based on the number of requests / usage of the infrastructure.[s3] uses the infrastructure costs of a million requests to compare different deployment scenarios.

Table 9 :
Organization Domain

Table 10 :
Team Size

Table 11 :
Organization Size

Table 12 :
Application Age

Table 13 :
Migration Time

Table 14 :
Information/metrics considered before or during migration mentioned at least three times.

Table 15 :
Information/metrics considered after migration mentioned at least three times.

Table 16 :
Information/metrics not easy to collect and mentioned by more than one interviewee.

Table 17 :
How useful did the interviewees consider discussion of the set of information/metrics before migration.