REPRESENTING VARIABILITY IN SOFTWARE ARCHITECTURE: A SYSTEMATIC LITERATURE REVIEW

Variability in software-intensive systems is the ability of a software artefact (e.g., a system, subsystem, or component) to be extended, customised or configured for deployment in a specific context. Software Architecture is a high-level description of a software-intensive system that abstracts the system implementation details allowing the architect to view the system as a whole. Although variability in software architecture is recognised as a challenge in multiple domains, there has been no formal consensus on how variability should be captured or represented. The objective of this research was to provide a snapshot of the state-of-the-art on representing variability in software architecture while assessing the nature of the different approaches. To achieve this objective, a Systematic Literature Review (SLR) was conducted covering literature produced from January 1991 until June 2016. Then, grounded theory was used to conduct the analysis and draw conclusions from data, minimising threats to validity. In this paper, we report on the findings from the study.


INTRODUCTION
Over the past two decades, the field of Software Architecture has come a very long way, with increased interest in its application in various application domains.Systems are becoming more complex and larger in scale reinforcing the need for robust system architectures.The software architecture of a system is usually designed at the early stages of the system development lifecycle once initial requirements are understood.However, in some cases, as in the case of some legacy systems, the software architecture is defined implicitly and emerges during system development, rather than being developed explicitly.
The increase in variability in hardware, software, and operating environments, along with the strong business case for delaying design decisions as long as it is economically feasible, has required architectures to capture and cater for more complex variability in order to produce highly configurable and adaptable systems.It is common today to have systems encompassing thousands of interdependent variability points, making the process of modelling and maintaining these variability points a cumbersome process.
Variability covers several areas, from the software design and engineering process itself, to the output of such activities, including requirements, architecture, source code, test cases, binary code, configuration files, etc. (Svahnberg, van Gurp, & Bosch, 2005).
In this review, we focus solely on representing variability in software architecture design.Galster and Avgeriou (Galster & Avgeriou, 2011a) discussed that variability in software architecture can be a difficult activity to manage and comprehend.Thus, to address this challenge, one would need to understand the variety of challenges faced by practicing architects when dealing with variability management in practice (Galster & Avgeriou, 2011a).
Over the last 15 years, a lot of work has been reported that addresses the representation of variability in software architecture in different domains.Some approaches have defined variability in software architecture as a way of representing and reasoning about alternative system implementations (Bachmann & Bass, 2001;Galster & Avgeriou, 2011b).Similarly, a number of different mechanisms have been used to represent variability at the architecture level (e.g.Software Product Lines (SPL), Service-oriented architecture (SOA), etc.).Although it is generally agreed that variability representation is a key step of the development process, which can affect the success or failure of a system or a product line (Bashroush, 2010), there seems to be little consensus on how the representation is best conducted.
Very few Systematic Literature Reviews (SLR) considered variability beyond Software Product Lines.Galster et al. (Galster, Weyns, Tofan, Michalik, & Avgeriou, 2014) presented an SLR on variability in software systems in which they investigated variability handling in the various software engineering development phases (from requirements and design, to testing and maintenance).They found that most of the studies dealing with variability focused at the architecture and design phases; However, they did not analyse the modelling approaches or techniques covered within the studies.
In this paper, we present a systematically conducted literature review that capture and summarizes the state-of-the-art in representing variability in software architecture.The presentation of the work makes it accessible to practitioners working in the area who are looking to choose the best variability approach that fits their design needs, as well as researchers trying to identify areas that require further investigation The rest of the paper is organized as follows.Section 2 describes the research methodology and lists the study research questions.Section 3 presents the data and meta-analysis of the results.Section 4 discusses and answers the study research questions.And, section 5 reports the study limitations and threats to validity.Finally, section 6 concludes the paper with final remarks.

RESEARCH METHODOLOGY
This section describes the research methodology adopted, namely a Systematic Literature Review (SLR), referred to as systematic review or review hereafter."A systematic review is a well-defined and methodical way to identify, evaluate, and synthesize the available evidence concerning a particular technology to understand the current direction and status of research or to provide background in order to identify research challenges" (Kitchenham & Charters, 2007).This method was chosen because of the requirement to have a credible, repeatable and fair evaluation of the available studies on representing variability in software architectures.
The review starts by defining the research questions, followed by a definition of the search strategy process to be followed (sections 2.1 and 2.2).Then, inclusion and exclusion criteria are developed to provide a systematic way of selecting among identified primary studies (section 2.3).Finally, the data has been extracted from the primary studies to help answer the research questions (section 2.4).Once the data is extracted, grounded theory is used to help analyse and draw conclusions to minimize threats to validity (discussed in section 5).

Research Questions
This research addresses concerns that relate to practitioners as well as researchers.As such, our review covers the following research questions: RQ1: What approaches have been proposed to represent variability in software architecture?
RQ2: What is the context and areas of research of the studies employing variability in software architecture?
RQ3: What are the limitations of the existing approaches to represent variability in software architecture?RQ1 is motivated by the need to describe the state-of -the art of how existing approaches represent variability.RQ2 helps better understand the applicability of each of the identified approaches, and to analyse any recurring patterns in different domain, while helping practitioners navigate through the reviewed approaches.We pose RQ3 to provide an overview of existing challenges in order to provide the directions for further research.

Search Strategy
The search string we have adopted to identify primary studies was designed following the criteria below:  Extract terms from the topic being researched as well as research questions;  Then, alternative terms (and synonyms) are included.This also covers terms that are spelt in different ways (e.g.British English vs American English);  Based on the papers known to the researchers, related keywords are used to perform initial search in relevant repositories;  Include other relevant terms where there is a possibility of identifying further material related to the topic. The identified words and keywords are then combined together in one string using instructs such as "OR" and "AND";  Finally, run the search strings and check relevance of returned results, then amend accordingly.
Following this strategy, the search string below was adopted: << (Variability OR Variabilities) AND (reference architecture OR software architecture OR architectural) >> Seven digital repositories (1.IEEExplore; 2. ACM Digital library; 3. Citeseer; 4. SpringerLink; 5. Google Scholar; 6. ScienceDirect and 7. SCOPUS) were queried for primary studies.Validity was checked by looking for known papers the authors were already aware of.All papers checked were found in the identified primary studies.Papers that we were not able to access online were acquired by contacting the relevant authors via email.As an additional measure to ensure the comprehensiveness of the review, a manual check was conducted of the proceedings of the major conferences (such as ICSE, WICSA, ECSA and SPLC) and workshops (such as QoSA and VaMoS) that the researchers were aware of that published relevant papers.
The publication lists of known researchers publishing in the area were also checked manually.Finally, for the primary studies identified, forward and backward reference checking was conducted.For backward reference checking, we examined the reference list of the papers searching for any potential primary studies that had been missed.Similarly, for forward reference checking, we used search engines to identify citations to the primary studies that could be relevant to the review.This process helped to identify a number of additional potential primary studies.In terms of timeline, we searched for primary studies published between January 1991 and June 2016.The start date was set to be as early as possible (the earliest relevant primary studies identified were published in 2002).The search stage of this SLR was concluded in June 2016 (hence the end date), after that, the data extraction stage commenced.

Study Selection
The outcome from the different initial searches on digital libraries, manual searches, and known author searches, produced 1053 primary studies.After initial screening by the authors of this SLR based on title, abstract and keywords and excluding papers that were irrelevant or duplicates, 139 primary studies were selected.These remaining primary studies were subject to a more detailed review (of the full papers) where each paper was checked by three researchers.This process resulted in 30 papers being excluded.Of the remaining 109 primary studies, forward references (papers citing the primary study) and backward references (papers cited in the primary study) were followed which helped to identify a further 13 studies.The resulting 122 papers were then reviewed by applying the following inclusion and exclusion criteria: -Inclusion criteria:  IC1: The primary study proposes or uses an approach to represent variability in software architecture;  IC2: In cases where the same content was published multiple times by the authors, the most recent and complete version was included as the primary study.
-Exclusion criteria:  EC1: The primary study addresses variability but not in software architecture domain. EC2: The primary study is in the domain of software architecture, but does not consider variability.A paper that does not address variability along with software architecture has no value in answering our research questions. EC3: Lack of sufficient details about representing variability in software architecture to make any useful contribution towards addressing research questions. EC4: The primary study is a short (less than 3000 words) or symposium paper, opinion, abstract, tutorial summary, keynote, panel discussion, presentation slides, technical report, proceedings overview (for instance, from a conference, workshop or special issue) or a book chapter.Only books and book chapters that were published as proceedings of peer-reviewed conferences were included (e.g.Springer LNCS or LNBIP series).These also had to be available through the corresponding digital libraries.
This led to the exclusion of 62 papers leaving us with 60 primary studies.

Data Extraction and Synthesis
On completion of the search and selection steps, data extraction was then conducted on the selected 60 primary studies to help answer the research questions defined in Section 2.1.
During data extraction, we captured information related to the paper synopsis, variability approach and the limitations.We made every effort to capture as much information as possible, but at the same time, kept the data as succinct as possible in order to avoid any potential influence of a taxonomic or classification framework on our results.
GoogleDocs was used to collect the extracted data from the different researchers and the aggregated results were made available in Excel spreadsheets for analysis.Finally, two researchers independently performed sanity checks on the results and the differences were reconciled collaboratively.

DATA AND ANALYSIS
Once the data extraction phase has been completed, data synthesis and analysis was conducted on the collected information.In this section, we provide an analysis of the primary studies in relation to their publication type, venues and trends.
Although we set our search period to start from January 1991 but unfortunately no studies were found in the 90s decade, the earliest primary studies identified were published in 2002.This could be due to the timing of the first major paper on the topic of Software Architecture by Shaw et al. (Shaw et al., 1995) in mid 90's.It is also worth mentioning here that no primary studies were identified in 2016.This is because 2016 is partially covered, when the search and selection process of this study was completed, and within that duration no such major conferences/workshops were conducted.
Excluding VaMoS 2016 and WICSA 2016, which has been manually scanned for this review with no relevant studies.
Fig. 1 shows the number of primary studies identified, along with the breakdown of numbers of papers published via each publication outlet type (Conference, Journal or Workshop).The data presented shows papers bundled in 5 year brackets to smooth the effect of conference frequency (e.g.some conferences happen every 18 months, while others every 12 months) and public funding call trends (e.g.EU funded research projects addressing a specific challenge tend to start and end during the same time frame leading to increased paper publications in the area around the end of the funding period).Looking at the chart, it can be seen that there is an uptrend in research publications relating to variability in software architecture.
Additionally, it has been observed that the majority of the primary studies were published in the proceedings of conferences (72%, 43 papers), followed by Workshops (15%, 9 papers), and then Journals (13%, 8 papers).

RQ1: What approaches have been proposed to represent variability in
software architecture?Two major approaches for representing variability in software architecture were identified in the primary studies: (1) defining variability using Unified Modelling Language (UML) or one of its extension in the form of another method, domainontology etc., and (2) using an ADL with explicit variability representation mechanisms.A detailed classification can be found in Table 1.
-ADLARS: (Bashroush et al., 2005) presents the ADL "ADLARS", a 3-view description of software architecture.This is an ADL with first class support for embedded systems product lines.It captures the relationship with explicit support for variability between the system's feature model and the architectural structures (using keywords like "supported", "unsupported" and "otherwise" in the description).
-ACME: (Satyananda, Danhyung, Sungwon, & Hashmi, 2007) describes two modelling notations, Forfamel for feature models and ACME (Garlan, Monroe, & Wile, 1997) for the architecture model.They are evaluated using the Formal Concept Analysis (FCA) technique, using a tool that generates a concept lattice graph that defines a mapping relationship between feature and architecture components.
-ALI: (Bashroush et al., 2008) presents an ADL called "ALI" (a descendent of "ADLARS" (Bashroush et al., 2005)) that aims to support product line engineering (and therefore also variability) as well as non-variant and individual system architectures.
-Darwin: (Yu et al., 2008) presents a framework with the Darwin ADL (with elements borrowed from one of its extensions, Koala (Ommering, van der Linden, Kramer, & Magee, 2000)).The paper proposes "a decision-making process to generate a generic software design that can accommodate the full space of design alternatives from a goal model with high variability in configurations." -MontiArc: an ADL designed to model architectures for asynchronously communicating logically distributed systems.Two studies present extension to MontiArc: (1) delta-modelling to represent variability -∆-MontiArc in (Haber, Rendel, Rumpe, Schaefer, & van der Linden, 2011) and (Haber, Kutz, Rendel, Rumpe, & Schaefer, 2011), and (2) using hierarchical variability modelling -MontiArc HV in (Haber et al., 2011).The given examples were difficult to extend if one is not using MontiArc, but the proposed variability modelling techniques were not new.
-PL-AspectualACME: (Barbosa et al., 2011) presents PL-AspectualACME (an extension to AspectualACME (Garcia et al., 2006)) with a graphical representation of the architectural model.The associated tool interprets the annotations, adding or removing the correct variant elements in the specification.(Coelho & Batista, 2011) presents the ADL PL-Aspectual ACME specifying the architecture for software product lines.The description is related to a goal model described in a formal visual notation PL-AOV Graph.
-CBabel: (Carvalho et al., 2012) presents the CBabel language, with features to support software architecture and contract description with a meta-model defined for architectural contracts.
-LightPL-ACME: (Silva et al., 2013) presents an ADL (an extension to ACME (Garlan et al., 1997)) with the aim of having "a simple, lightweight language for SPL architecture description.

It enables the association between the architectural specification and the artefacts involved in the SPL development process, including the relationship with the feature model by categorically defining the variability and the representation of both domain and application engineering elements."
Most of the work reported on the use of UML and ADLs for capturing variability at the architectural level was conducted by their original authors.A small proportion of these papers (e.g.(Sánchez et al., 2009) (Carvalho et al., 2012) (Albassam & Gomaa, 2013)) reported on work conducted in an industrial setting, but the rest used prototype implementations based in academia.We discuss the context of the research in more detail under RQ2 analysis.OVM (Orthogonal Variability Model) and XML (EXtensible Markup Language) approaches represent variability in 7% (4 papers) and 3% (2 papers) of the selected primary studies respectively.Other ways that were identified to capture variability in the software architecture are: CVL (Common Variability Language) in (Pascual et al., 2013); LISA (Language for Integrated Software Architecture) in (Groher & Weinreich, 2013); formal modelling languages/framework (e.g.(Sinnema et al., 2006) (Satyananda, Danhyung, & Sungwon, 2007) (Hwi et al., 2013)) and modelling tools (e.g.(Dhungana et al., 2008) (Mann & Rock, 2009) (Lytra et al., 2014) (Smiley et al., 2014)), and; formal/informal textual and visual descriptions such as spreadsheets and process diagrams (e.g.(Thiel & Hein, 2002a) (Andersson & Bosch, 2005) (Kakarontzas et al., 2008;Zhu et al., 2011) (Galster et al., 2013) (Groher & Weinreich, 2013)).
It is important to state that the number of studies cross-cut multiple variability approaches, and accordingly, appear under more than one category in Table 1 (hence the total of 69 rather than 60).For instance, (Razavian & Khosravi, 2008) and (Helleboogh et al., 2009) (Myllärniemi et al., 2012) covers UML and xml and variability mechanisms simultaneously.Also, (Helleboogh et al., 2009), (de Moraes et al., 2010) and (Duran-Limon et al., 2015) represent variability in both UML class and component diagrams.
Overall, UML and ADLs seemed to be the most commonly used approaches for capturing variability at an architectural level, making up 72% (43 papers) of the selected primary studies.UML was used in almost half of the studies, where it was extended through various mechanisms to support variability.While ADLs were mostly used in the product line domain.

RQ2: What is the context and areas of research of the studies employing
variability in software architecture?

Research Context (Academia vs. Industry)
The research context of each primary study was classified as either: Academia (if the research was conducted in academia and by academics with no reference to industrial usage); Industry (if the research was conducted by industry based researchers or had direct industrial relevance); or both (when the research was a joint undertaking with both academic and industrial relevance).From the selected primary studies, we identified that only a small proportion of research (18%, 11 papers) was conducted in industry.73% (44 papers) of the research surveyed was academic while 9% (5 papers) was classified as joint context (both industry and academia).
Figure 2: Research context Fig. 2 shows that the majority of studies belong to the academia sector (73%), with 18% in industry and 9% joint.However, it was noticeable that the industry initiated papers doubled between 2011-2015 compared with 2006-2010, while academic papers only gone up by 9%.Yet, joint papers between industry and academia is going down with only 1 primary study published between 2011-2015.

Research Context (Theoretical vs. Practical)
Another way the research context of the primary studies was analysed was by checking whether the reported research had a practical or theoretical focus, or both.The results are reported in Fig. 3 which shows the majority of the work conducted is theoretical work with no direct application to practical problems.
Overall, 67% (40 papers) of the primary studies were focused purely on theoretical work with only 13% (8 papers) addressing practical issues and another 20% (12 papers) that can be classified as both.

Research Areas
During the analysis, it became clear that the primary studies can be categorised under four main research areas: The breakdown of primary studies per research area is shown in Table 2. Fig. 4 shows a graphical distribution of the primary studies over the different areas identified.Noticeably, the work on variability in software architecture is dominated by work in the area of Software Product Lines.Understanding the limitations of a particular piece of research is an important step towards understanding its applicability and utility.Unfortunately, in the literature reviewed for this study, 75% of the papers surveyed (45 of 60) did not make any attempt to report limitations of the research performed and 2% (1 study, (Eklund et al., 2005)) did not report the limitations of the research explicitly.
This left 14 studies (23%) that fully or partially identified limitations of their work, so helping to understand its maturity and the areas of its likely applicability.The limitations reported can be categorised under the following headers: -Technical limitations with the research methodology adopted: For example, some papers only used one case study ((Zhu et al., 2011), (Diaz et al., 2013)), while others used small unrepresentative study groups ((Andersson & Bosch, 2005)).

THREATS TO VALIDITY AND STUDY LIMITATIONS
This section discusses the limitations and threats to validity of our study.As with most research methods, there are some inherent limitations to the SLR methodology.The first limitation is the possibility that the search and selection process may not have identified all of the relevant primary studies.This can be due to various reasons such as the use of different terminology in primary studies to the one we adopted in the search term (particularly given that the work covered by this SLR cuts across multiple domains and research communities).To address this limitation, we have extended the search protocol and introduced a number of mitigating measures.First, we ran our automated searches on web sites of prominent publishers (e.g.IEEEXplore) as well as against general indexing search engines (e.g.Google Scholar) which helps to ensure comprehensiveness of results as different search engines use different ranking algorithms.Then, we conducted manual searches on proceedings of known publication outlets and publication lists of known authors in the domain and cross-examined the findings with the results produced from the automated search.Finally, we conducted forward and backward reference checks on the identified primary studies to further ensure that all of the relevant literature was identified.
Another limitation of SLRs is the exclusion of grey literature, such as thesis documents, white papers and technical reports.This could be a problem in some areas such as those where the work is led by industry, as practitioners tend to publish less in peer-reviewed outlets.However, looking at the analysis of RQ2, and to some extent at the initial results we had from the automated searches (conducted on general indexing websites such as Google Scholar), we notice that this study area is largely dominated by academic researchers with minimal potential for grey literature.Last but not least, there is the limitation of the language barrier where only primary studies published in English were searched and analysed.This could potentially mean that relevant primary studies published in other languages might have been missed.We do not have a strong mitigation to this threat other than noting that the majority of research in these areas appears to be published in English and so we do not believe that there is a high likelihood of significant research in this field remaining unpublished in English for long.
Given the end date of the search process of June 2016, few papers have been published since then.However, conducting a basic search just before publication, we managed to only identify one new primary study that could have been included in the SLR (Ali & Hong, 2017).Yet, this would not have impacted the findings and conclusions.
Beyond the inherent SLR methodology limitations, threats to validity can be classified under four main headers: construct, internal, external and conclusion (Matt & Cook, 1994).
Some of the threats to construct and internal validity have already been discussed above.These threats arise from weaknesses in the execution of the research method adopted.A popular construct validity problem in SLRs is author bias and we have addressed this by having multiple authors review each primary study and had the overall process reviewed by an independent researcher (who was not one of the authors).This was discussed in the section on research methodology (Section 2).
On the other hand, the threat to external validity relates to the applicability of the results of the study beyond the context where it was conducted.Given that this study was not limited to one area, but studied multiple areas where variability in software architecture is used, inductive generalization is considerably strengthened.Moreover, we have made all of the raw data used for the study available for readers to better help them understand the reasoning and analysis conducted.
Finally, conclusion validity threats relate to the robustness of conclusions made based on the data available.A typical threat is when researchers gear conclusions to agree with their initial hypotheses.In our case, the research did not set any initial hypotheses but rather addressed the research questions with an open view.Additionally, we based all our conclusions on grounded theory (Martin & Turner, 1986) and other analysis methods where multiple researchers were involved and independently agreed on the conclusions made.

CONCLUSIONS
This work aimed at capturing and cataloguing the state-of-the-art in representing variability in software architecture, making it more accessible to practitioners and researchers alike.Overall, it can be said that this research area is witnessing an uptrend, especially since 2006 (see Fig. 1), and that work in this domain is starting to mature.In summary, we found that: -UML (including various extensions) and Architecture Description Languages (ADL) were the most commonly used notations to represent variability in software architecture.-The work on variability representation at the software architecture level can be largely mapped to three main research areas: Software Product Lines (SPL); Reference Architecture; and Service Oriented Architecture (SOA).-Most of the work surveyed focused on proposing some form of new or improved design process or traceability technique relating to the development of systems that include variability.-The majority of the work conducted (73%) was academically led, much of it with a fairly theoretical focus (67%).-Overall, the research in this domain was found to have clear rationale and objectives, but generally lacking proper validation.
Future work should consider the creation of benchmarks to enable the comparison of the various techniques and their applicability to certain domains or problems.

Figure 3 :
Figure 3: Research relevance That said, Fig. 3 also shows that the trend is changing with higher percentage of papers with practical relevance appearing in the past 5 years compared to 2006-2010.
Product lines (including Product Line Architectures -PLA and Dynamic SPL -DSPL) 4. Other (general Software Architecture)

Figure 2 :
Figure 2: Breakdown of primary studies over research areas software architecture: a systematic literature review

Table 2 :
Breakdown of primary studies over research areas § A number of studies cross-cut multiple research areas, and accordingly, appear under more than one research area (hence the total of 66 rather than 60)